Competitive Intelligence Software Applications

by Arik Johnson, Founder and Managing Director of the Recon Competitive Intelligence (CI) outsourcing and support bureau, Aurora WDC

Competitive intelligence (CI) is fast becoming a strategic imperative to move ahead, stay ahead or simply compete in the e-business marketplace. The need to compete when the Internet lowers barriers to entry, product development cycles continue to contract to near zero as mass customization takes over, customer expectations require increased speed and responsiveness, proliferation of alternative business models and the fear of being displaced by unforeseen market shifts are all contributing to the need for CI. GartnerGroup defines E-Business as "the integration of business strategies, processes and technologies that focus on breaking down the barriers among business divisions and enterprises". Today, the need for systematized CI systems is greater than ever. And, while most organizations gather competitive information, few enterprises have adequate CI in place, requiring a system for analyzing and integrating it into their formal business strategy.

For several weeks in late 1999, I interviewed representatives of a number of specialized competitive intelligence (CI) software vendors, as well as more traditional Knowledge Management systems companies providing CI applications within their broader "portal" frameworks. Let me preface what follows by saying, I didn't get a chance to talk to "everybody" active in this sector, so this analysis remains incomplete, but those I did chat with had a great deal to say about their "philosophies" of CI in particular and KM in general. Their approaches differ widely, but all of them, thankfully, have at least some degree of focus on the processes required to overcome the business problem - in this case, to better understand business rivals that potentially threaten a firm's market position, customer relationships and value chain for stakeholders.

Many CI practitioners in companies have built intranet systems of their own to collect, process, help analyze and disseminate mission-vital CI to their constituents. The variety of these systems, because they're "home-grown" tend to be as great as the number who've built them. But, the one approach I approve of with do-it-yourself CI systems is that, it's most often based on what's needed to accomplish the business objective. It follows that, it tends to be much more costly to deploy a system from scratch than to buy one and have it installed - but, the advantages of focus and specificity are strong. That said, as a traditional opponent of shrink-wrap software "solutions" for CI, I've had a number of revelations about the advantages of using enterprise-wide, functionally specific and task-specific CI apps as adjuncts or full-scale replacements of the DIY approach.

When I spoke with Mark Demers, VP of Marketing at Vienna, VA-based Excalibur Technologies Corporation about the CI philosophy behind its flagship product RetrievalWare version 6.7, he noted that, "KM is not something you buy in a box - it's about people, processes and technology. On the tech side of things, one of the most critical enablers is 'knowledge retrieval'. Intelligent, search-based applications as part of intranets or online e-commerce apps, that give a single point of access to any data, anywhere. Knowledge doesn't exist in structured data repositories - it's in mostly unstructured repositories - such as images and video -- as well as text documents and people's skill sets. Excalibur's main goal is to provide unparalleled accurate search across all of that data."

To index the very complex search criteria necessary for image and video files, rather than using meta-data tags to search keyword densities, Excalibur uses video deconstruction for indexing by its elements - like an image storyboard for scenes in the video and voice-text plus closed-captioning for content - which is then searchable. Finally, with regard to subject-matter experts, there are two methods of support - one called the "knowledge contributor", a desktop applet to contribute documents and extrapolate fields-of-expertise for particular users; and an experts directory, which combines personnel profiles with knowledge contributor results inferred from the user's desktop.

CI theorists have long believed that, as much as 80 percent of what a firm needs to know about its competition is already present within the organization - that was the original connection with KM a few years ago. Assuming the other 20 percent is external, Excalibur includes search indexing of external information sources as well. "Multimedia spidering and external indexing of the Internet can be setup as a service to monitor things outside of the organization," Mark said. Finally, with regard to creating intelligent agents and context sensitive federated search for a context-specific thesaurus, the application includes tools for building a lexical approach to federated search and context. Intelligent agents, which Excalibur calls "profiling", can be saved as a category against the whole document base - providing persistent updating of a search or the saving of queries in a profile that can be added to as new information becomes available. Categorization of information makes it possible to exclude a great deal of manual intervention - because of the semantic network and thesaurus.

In conversations with other CI professionals, I have discovered that, RetrievalWare is used fairly widely in CI applications - both intra-firm and with commercial services that deliver services for CI. monitors content that appears on the Internet - public discussion in USENET, message boards on commercial online services, popular finance sites, obscure Web sites (such as rogue or activist sites for disgruntled employees) as well as competitor Web sites, along with 1200 online Web publications (global/national newspapers and magazines, broadcast Web sites, etc.). James Alexander, VP Sales/Marketing at eWatch said that, "in the CI area, Excalibur is used as the profiling mechanism, scanning the content as it comes in through the areas they target - specific competitor Web sites with specific words or search criteria, key executives names, product names, and so on. The delivered product is a digest of the monitored services. If you can identify problems where products are being discussed you can exploit the problem or introduce a solution." How to make the content relevant is the problem that Excalibur solves for eWatch - it flags pieces of information that matches the Excalibur profiles and hands that off to an Oracle back-end database for storage.

"KM divides into four major categories of business problems to be solved - improved productivity, enhanced skills transfer, enhanced competitiveness of the organization and better, faster innovation - those are how the CEO would define it." Peter McKenney, Managing Director of Arnold, MD-based Cipher Systems finds CI lacking in most companies, even those positioning for a new millennium of e-business. "Andy Grove said that, five years from now, all companies will be Internet companies, or they won't be companies at all. Beyond the high-tech hype there is much truth in this statement. He is talking about E-Business. Mere E-Commerce pales by comparison to E-Business by every analyst who studies the trends and every digerati moneyman who invests in it. Why? Because E-Business has a very good chance of fundamentally changing the way companies operate; going way beyond simple electronic selling and deep into the processes and culture of the enterprise. E-Business will dramatically lower costs across supply and demand chains, allow customer service onto a higher plane, create new revenue opportunities and redefine business relationships." Peter adds, "This rings true not so much because firms will adopt E-Business for the opportunities they present --- as the other disruptive technologies of the 20th century did before them, such as electricity, the telephone and the automobile. E-Business will be adopted by all firms because they will have to. Competitors will force them to do so. Competition will drive E-Business more than anything else."

"Knowledge.Works", Cipher's application, takes two forms - around the two dominant messaging platform, Notes and Exchange - and enhances the process of gathering and sharing intelligence in an organization. "It's used for key decision support, tactical responses and knowledge sharing of marketing awareness, for the purpose of enhancing of the general marketing activities of the organization." The least return on investment is market awareness building - where the majority of the portal companies in the market are focused - creating a common, single site that unifies competitive information that matters to them. That can be filtered for the desktop environment - in the form of enterprise portals (general information), workgroup portals (a set theme that's defined around an issue) and personal portals (customized with the view individuals are most interested in).

The upper level is key decision support - when the CI group is pulled in from the beginning with the planning and execution of a business initiative of some kind. Peter gave us an example. "A large energy utility used the software to affect favorable deregulation legislation - a 35-person team within the CI component focused on understanding the positions of all of the actors in the legislation and drive that internal effort making sure the resources lined up with where competitors, customers, municipalities and special interest groups were putting theirs. This was a process driven from the chairman's office directly. The first time around, they killed the bill, but Wall Street knew a bill was coming regardless so the stock took a beating. The company found they needed to get a bill passed -- but it had to be the right bill. They used the capital's legislative translation service to transcribe committee meeting notes and then the software was setup to pull it right from the firm providing the service rather than waiting for the public version of those notes to come out a week later. This helped route and interpret the points of interest in the notes. The next day, they'd go into the committee room, they'd provide the technical and marketing points that were developed as responses to influence congress-peoples' decision making."

Like most of the other applications I studied, the software itself is customized to the particular needs of the user environment. "One of the most important features is the linkage to key decision support - intelligence is created not found (people + workflow). This moves people away from thinking about CI in a data warehousing sense and towards analysis. The technology is merely the enabler of the mission. Specific questions are tough to answer when the answer is inside someone's head and you need the power offered by a platform like Notes or Exchange to do that."

Don Smith, President of Chicago-based Wincite Systems is perhaps the oldest and best-established of all CI software vendors, having originally deployed a DOS-based client-server CI application in the late 80's, known as Incite, which later became Wincite in the Windows environment and eventually gave way to Web-based technologies built around Active Server Pages using the browser as the client. Today, Wincite is optimized to be adapted for CI or used broadly as a marketing information system to support marketing/sales organizations, organizes information from a number of different sources in line with a specific business operation - whether it's free-standing or part of a larger corporate structure.

According to Don, "Competitive intelligence in the past has always been centralized in a corporate or centralized environment - now, systems allow distribution to the extended enterprise. Central repositories of information are important, but they must be contributed to from a number of different sources. If, for example, the core information is provided by the CI/CA staff at a central location, then feedback will not come back from sales, unless the pump is primed by information given to those sales personnel - the CI system must provide an incentive and a willingness to participate."

He added, "A CI system also has to have collaborative capabilities - traditionally, this has been centralized through the 'gatekeeper'. Still, in my opinion, the collaboration side of KM has been vastly overstated in the past -- intelligence gathered by the front-line personnel is often never given feedback from higher-up the corporate ladder in terms of its value - the lack of a solid frame of reference is what we call the 'Black Hole Effect'."

"Now that Y2K has been conquered, the emphasis will shift from transaction systems to more of the analytical side of the value-chain of what is being sold by companies. CRM is part of that, ERP is part of that - we're making our people 'smarter' and doing it for the customers' benefit. Now that Y2K is behind us, there's not a lot of managers feeling good about the return they got on their ERP upgrades - so, they're looking for better payback to the customer relationship. In terms of CI, it's moving out of the librarian area and into an enabling system that networks throughout the whole organization - pull on the part of the end user rather than push on the part of the CI staff."

The Department of Defense uses Wincite to develop strategy on large government contracts - providing context in terms of the history of the relationship with the defense buyer and political influences, with a SWOT analysis of how comparisons take place with other competitors on particular products. All this happens before an RFP is ever issued - then, they do a post-award audit of the sales process to isolate best practices in both winning and losing deals.

The biggest challenge for any CI application is to create a system that is both scalable to very sophisticated features, while, promoting the analytical side of the process. For example, SWOT analysis is specifically one of the tools Wincite uses as well as Porter's Five Forces for competitor analysis. Don adds, "Very visible interfaces force discipline in using the raw information to create meaning - which is what CI is really all about. There's also strong support of benchmark reporting - products and competitors on one axis and then features and strengths on another, you can begin to see where the gaps show up."

Don said the most important features of the system are its ability to automate the creation of visual presentations for management, "it's the primary means of communicating from a strategic perspective the importance of the knowledge available - give 'em one page of conclusions and recommendations, and retain the sources of information behind it - the presentation really spells out the decision making in terms executives can understand."

"Strategy! is not a CI system it's a competitive information management system, and the word 'intelligence' is left out intentionally - intelligence is something created by people, not software - it's only built on information," said Matt Kelly, Business Development Manager at Kirkland, WA-based Strategy Software, Inc.

"Our software gives people a framework for tracking and outsmarting competitors - for sales and marketing personnel as well as strategic planning in the macro view. Research for its own sake is of little value, it should be directed towards a specific decision. What this allows one to do is to preserve previously completed research. It's based loosely on Porter's Five Forces Model - extended a bit for keeping track of products from the company and competitors - to compare product positioning between companies." One new feature allows a company to create a benchmark template very quickly to create a graphical representation of numerical information.

"We'd like to see CI demystified as a normal part of doing business rather than a complex, specialized function done in the short-term - it's really longer-term and can reflect perspectives as they change over time. Because it's a database app, users can disseminate competitive info for quick victories for CI managers and directors." This is often a critical component, especially for new CI units that have a very short period of time in which to prove their worth. "People expect software to allow ordinary people to do extraordinary things. When that's possible, the whole KM effort is more successful."

The solution is very complimentary to other applications. "KM brings together keyword hits, CI gives meaning to the results - so, we have here a database that allows us to pull out specific nuggets of information that supports the analysis primarily, but ties it back to the source materials."

For example, a manufacturer of lift-trucks (forklifts) deployed a system where their salespeople can pull up the interface with detailed information about a competitors' offering or a strengths-weaknesses relative rating of product comparisons - where do they stand feature for feature to generate a sales strategy for that particular prospective customer. It provides sales personnel a "Zero-to-Five" rating scale - which lets sales defend value based on needs of customers -- giving them the ability to stick to a price versus needing to discount in order to get the sale because they were up against a very strong product.

When I asked Matt about the most important features of the software, he mentioned, "Ease-of-use and quick implementation time gives administrators the ability to put up a CI system in an hour."

Matt added, "The pace at which businesses are able to enter new markets are very quick and barriers are low - CI plays an important role in making sure that you're first to a given market niche, but it'll also determine whether or not your product is superior in that niche, and when the time comes, CI will give you a strategic advantage to determine what to do when that product niche becomes mature - as it inevitably will." He added, "Companies ignoring CI are like ships in the fog operating without radar. Companies should be doing CI even if they're doing it badly - it's less important that people are doing it in a world-class fashion than that they get started doing it. Work on getting better later."

About the Author

If you have any comments or observations, you can contact Arik Johnson by e-mail at Arik R. Johnson is Managing Director of the Competitive Intelligence (CI) outsourcing and support bureau at Aurora WDC. Arik has degrees in International Business, History, Political Science and International Relations from the University of Wisconsin-Madison and is adjunct faculty or visiting lecturer on business competitiveness topics at a number of educational institutions around the United States. Prior to founding the firm, Arik was a business analyst with a major international management consulting firm. For more information about Arik Johnson check


Johnson, A., "Competitive Intelligence Software Applications", DSSResources.COM, 12/06/2002.

Arik Johnson provided permission to archive this article and feature it at DSSResources.COM on Sunday, November 10, 2002. This article was posted at DSSResources.COM on December 6, 2002.