Multidimensional analysis, the basis for OLAP, is not new. In fact, it goes back to 1962, with the publication of Ken Iverson’s book, A Programming Language. The first computer implementation of the APL language was in the late 1960s, by IBM. APL is a mathematically defined language with multidimensional variables and elegant, if rather abstract, processing operators. It was originally intended more as a way of defining multidimensional transformations than as a practical programming language, so it did not pay attention to mundane concepts like files and printers. In the interests of a succinct notation, the operators were Greek symbols. In fact, the resulting programs were so succinct that few could predict what an APL program would do. It became known as a ‘Write Only Language’ (WOL), because it was easier to rewrite a program that needed maintenance than to fix it.
Unfortunately, this was all long before the days of high resolution GUI screens and laser printers, so APL’s Greek symbols needed special screens, keyboards and printers. Later, English words were sometimes used as substitutes for the Greek operators, but purists took a dim view of this attempted popularization of their elite language. APL also devoured machine resources, partly because early APL systems were interpreted rather than being compiled. This was in the days of very costly, under-powered mainframes, so applications that did APL justice were slow to process and very expensive to run. APL also had a, perhaps undeserved, reputation for being particularly hungry for memory, as arrays were processed in RAM.
However, in spite of these inauspicious beginnings, APL did not go away. It was used in many 1970s and 1980s business applications that had similar functions to today’s OLAP systems. Indeed, IBM developed an entire mainframe operating system for APL, called VSPC, and some people regarded it as the personal productivity environment of choice long before the spreadsheet made an appearance. Even today, there are OLAP products that use APL internally.
However, APL was simply too elitist to catch on with a larger audience, even if the hardware problems were eventually to be solved or become irrelevant. It did make an appearance on PCs in the 1980s (and is still used, sometimes in a revamped form called “J”) but it ceased to have any market significance after about 1980. Although it was possible to program multidimensional applications using arrays in other languages, it was too hard for any but professional programmers to do so, and even technical end-users had to wait for a new generation of multidimensional products.
By 1970, a more application-oriented multidimensional product, with academic origins, had made its first appearance: Express. This, in a completely rewritten form and with a modern code-base, has become a widely used contemporary OLAP offering, but the original 1970’s concepts still lie just below the surface. Even after 30 years, Express remains one of the major OLAP technologies, although Oracle has struggled and failed to keep it up-to-date with the many newer client/server products. Oracle announced in late 2000 that it will build OLAP server capabilities into Oracle9i starting in mid 2001. The Oracle9i OLAP Services includes both a version of the Express engine, called the analytic workspace, and a new ROLAP engine. This will mean that the Express engine lives on well into its fourth decade, and perhaps even its fifth.
More multidimensional products appeared in the 1980s. Early in the decade, Stratagem appeared, and in its eventual guise of Acumate (now owned by Lucent), this too was still marketed to a limited extent till the mid 1990s. However, although it is a distant cousin of Express, it has never had Express’ market share, and is now little used. Along the way, Stratagem was owned by CA, which was later to acquire two ROLAPs, the former Prodea Beacon and the former Information Advantage DecisionSuite, both of which soon died.
Comshare’s System W was a different style of multidimensional product. Introduced in 1981, it was the first to have a hypercube approach and was much more oriented to end-user development of financial applications. It brought in many concepts that are still not widely adopted, like full non-procedural rules, full screen multidimensional viewing and data editing, automatic recalculation and (batch) integration with relational data. However, it too was heavy on hardware and was less programmable than the other products of its day, and so was less popular with IT professionals. It is also still used, but is no longer sold and no enhancements are likely. Although it is now available on Unix, it is not a client/server product and was never promoted by the vendor as an OLAP offering. In the late 1980s, Comshare’s DOS One-Up and later, Windows-based Commander Prism (now called Comshare Planning) products used similar concepts to the host-based System W. Hyperion Solution’s Essbase product, though not a direct descendant of System W, was also clearly influenced by its financially oriented, fully pre-calculated hypercube approach. Ironically, Comshare subsequently licensed Essbase (rather than using any of its own engines) for the engine in some of its modern OLAP products.
Another creative product of the early 1980s was Metaphor. This was aimed at marketing professionals in consumer goods companies. This too introduced many new concepts that only became popular in the 1990s, like client/server computing, multidimensional processing on relational data, workgroup processing and object oriented development. Unfortunately, the standard PC hardware of the day was not capable of delivering the response and human factors that Metaphor required, so the vendor was forced to create totally proprietary PCs and network technology. Subsequently, Metaphor struggled to get the product to work successfully on non-proprietary hardware and right to the end it never used a standard GUI.
Eventually, Metaphor formed a marketing alliance with IBM, which went to acquire the company. By mid 1994, IBM had decided to integrate Metaphor’s unique technology (renamed DIS) with future IBM technology and to disband the subsidiary, although customer protests led to the continuing support for the product. The product continues to be supported for its remaining loyal customers, and IBM relaunched it under the IDS name but hardly promoted it. However, Metaphor’s creative concepts have not gone and the former Information Advantage, Brio, Sagent, MicroStrategy and Gentia are examples of vendors covered in The OLAP Report that have obviously been influenced by it.
One other surviving Metaphor tradition is the unprofitability of independent ROLAP vendors: no ROLAP vendor has ever made a cumulative profit, as demonstrated by Metaphor, MicroStrategy, MineShare, WhiteLight, STG, IA and Prodea. The natural market for ROLAPs seems to be just too small, and the deployments too labor intensive, for there to be a sustainable business model.
By the mid 1980s, the term EIS (Executive Information System) had been born. The first explicit EIS product was Pilot’s Command Center though there had been EIS applications implemented by IRI and Comshare earlier in the decade. This was a cooperative processing product, an architecture that would now be called client/server. Because of the limited power of mid 1980s PCs, it was very server-centric, but that approach came back into fashion again with products like EUREKA:Strategy and Holos and the Web. Command Center is no longer on sale, but it introduced many concepts that are recognizable in today’s OLAP products, including automatic time series handling, multidimensional client/server processing and simplified human factors (suitable for touch screen or mouse use). Some of these concepts were re-implemented in Pilot’s Analysis Server product, which is now also at the end of its life, as is Pilot, which changed hands in August 2000, almost certainly for the last time, when it was bought by Accrue Software.
By the late 1980s, the spreadsheet was already becoming dominant in end-user analysis, so the first multidimensional spreadsheet appeared in the form of Compete. This was originally marketed as a very expensive specialist tool, but the vendor could not generate the volumes to stay in business, and Computer Associates acquired it, along with a number of other spreadsheet products including SuperCalc and 20/20. The main effect of CA’s acquisition of Compete was that the price was slashed, the copy protection removed and the product was heavily promoted. However, it was still not a success, a trend that was to be repeated with CA’s other OLAP acquisitions. For a few years, the old Compete was still occasionally found, bundled into a heavily discounted bargain pack. Later, Compete formed the basis for CA’s version 5 of SuperCalc, but the multidimensionality aspect of it was not promoted.
Lotus was the next to attempt to enter the multidimensional spreadsheet market with Improv. Bravely, this was launched on the NeXT machine. This at least guaranteed that it could not take sales away from 1-2-3, but when it was eventually ported to Windows, Excel was already too big a threat to 1-2-3 for Improv’s sales to make any difference. Lotus, like CA with Compete, moved Improv down market, but this was still not enough for market success, and new development was soon discontinued. It seems that personal computer users liked their spreadsheets to be supersets of the original 1-2-3, and were not interested in new multidimensional replacements if these were not also fully compatible with their old, macro driven worksheets. Also, the concept of a small multidimensional spreadsheet, sold as a personal productivity application, clearly does not fit in with the real business world. Microsoft went this way, by adding PivotTables to Excel. Although only a small minority of Excel users take advantage of the feature, this is probably the single most widely used multidimensional analysis capability in the world, simply because there are so many users of Excel. Excel 2000 includes a more sophisticated version of PivotTables, capable of acting as both a desktop OLAP, and as a client to Microsoft Analysis Services. However, the OLAP features in Excel 2000 are inferior to those in OLAP add-ins, so there is a good opportunity for third-party options as well.
By the late 1980s, Sinper had entered the multidimensional spreadsheet world, originally with a proprietary DOS spreadsheet, and then by linking to DOS 1-2-3. It entered the Windows era by turning its (then named) TM/1 product into a multidimensional back-end server for standard Excel and 1-2-3. Slightly later, Arbor did the same thing, although its new Essbase product could then only work in client/server mode, whereas Sinper’s could also work on a stand-alone PC. This approach to bringing multidimensionality to spreadsheet users has been far more popular with users. So much so, in fact, that traditional vendors of proprietary front-ends have been forced to follow suit, and products like Express, Holos, Gentia, MineShare, PowerPlay, MetaCube and WhiteLight now proudly offer highly integrated spreadsheet access to their application servers. Ironically, for its first six months, Microsoft OLAP Services was one of the few OLAP servers not to have a vendor-developed spreadsheet client, as Microsoft’s (very basic) offering only appeared in June 1999 in Excel 2000. However, the (then) OLAP@Work Excel add-in filled the gap, and still (under its new snappy name, BusinessQuery MD for Excel) provides much better exploitation of the server than does Microsoft’s own Excel interface.
A few users demanded multidimensional applications that were much too large to be handled in multidimensional databases, and the relational OLAP tools evolved to meet this need. These presented the usual multidimensional view to users, sometimes even including a spreadsheet front-end, even though all the data was stored in an RDBMS. These have a much higher cost per user, and lower performance than specialized multidimensional tools, but they are a way of providing this popular form of analysis even to data not stored in a multidimensional structure.
Other vendors expanded into what is now called desktop OLAP (even though, in Web implementations, the cubes are usually resident on the server): small cubes, generated from large databases, but downloaded to PCs for processing. These have proved very successful indeed, and the one vendor that sells both a relational query tool and a multidimensional analysis tool (Cognos, with Impromptu and PowerPlay) reports that the latter is much more popular with end-users than is the former.
Now, even the relational database vendors have embraced multidimensional analysis, with Oracle, IBM, Microsoft, Informix, CA and Sybase all developing or marketing products in this area. Ironically, having largely ignored multidimensionality for so many years, it now looks like Oracle, Microsoft and IBM might be the new ‘OLAP triad’, with large OLAP market shares, based on selling multidimensional products they did not invent.
So, what lessons can we draw from this 35-year history?
You can contact Nigel Pendse, the author of this paper, by e-mail at NigelP@OLAPReport.com if you have any comments or observations. Nigel Pendse is principal of OLAP Solutions and has had over twenty-five years experience as a user, vendor and independent consultant in the areas now known as OLAP and Business Intelligence. He is a frequent speaker at conferences, and as a consultant, advises both users and vendors on OLAP product issues. He has degrees in mechanical and nuclear engineering, and was international marketing director of a well-known software firm until 1994. He is based in London, England.Nigel Pendse provided permission on behalf of OLAPReport.com to archive this article and feature it at DSSResources.COM on Saturday, 20 July 2002. This article, "The origins of todays OLAP products", was posted at DSSResources.COM on October 6, 2002. All information copyright © 2002, Business Intelligence Ltd, all rights reserved. This version was last updated on July 20, 2002. The current version should be online at http://olapreport.com/origins.htm.