*********************************************************** DSS News D. J. Power, Editor October 22, 2006 -- Vol. 7, No. 22 A Free Bi-Weekly Publication of DSSResources.COM approximately 1,800 Subscribers ************************************************************ Check the DSS Thought Leader interview with James Taylor "Automating Decision Making" at DSSResources.com ************************************************************ Featured: * Ask Dan: What staffing is needed for DW/BI/DSS? * DSS Conferences * DSS News Releases ************************************************************ "Decision Support for Global Enterprises" Conference Jan. 2-4, 2007, Calcutta, India Check ICDSS2007.org ************************************************************ What staffing is needed for DW/BI/DSS? by Dan Power Editor, DSSResources.com Some Ask Dan! questions are more difficult for me to answer than others. On July 6, 2006, Aileen MacKay sent me some especially hard questions and she followed up with some additional questions on September 29, 2006. Over the past 4 months, I've done some searching and thinking trying to figure out how to answer her questions. In this Ask Dan!, I'll focus only on her staffing questions. Aileen writes "I would like to find out more information on staffing and skill levels for adequately maintaining a Data Warehouse and a DSS. What staffing level are most companies maintaining to effectively manage their Data Warehouse and DSS? Are they flexing using consultants? or do they have a core DW Team? What is the ideal staffing level? What is the "best practice" organizational structure that is used today? Someone has likely conducted a survey to answer these questions quantitatively. Once that data was collected, it was probably grouped by the size of the responding organization and possibly by how long the organization had had a data warehouse and perhaps by some measure of scope like the number of data marts or the amount of data stored in the warehouse. All of these situational factors impact staffing. In searching the Web and trade publications however, I haven't found the results of that survey. What did I find? Info-Tech Research Group (ITRG) recently surveyed 1,600 IT decision-makers about their IT budget and staffing patterns. ITRG prepared reports (June 2006) for nine industries. One 2006 report costs $990, "each additional industry report may be purchased for just $500." These reports are probably too general to answer Aileen's questions and the cost is a barrier. Foote Partners has a Quarterly Salary and Skills Pay Report for Data Warehousing/Business Intelligence for 20 selected U.S. cities. The job titles included in the report are: Vice President/Director – Data Warehousing/BI; Data Warehouse/BI Project Manager – Processing; Data Warehouse/BI Project Manager – Decision Support Services; Data Warehouse/BI Information Security Manager; Data Warehouse/BI Architect; Data Warehouse/BI Engineer; Data Warehouse/BI Developer; Data Warehouse/BI Auditor; Sr. Decision Support System(DSS) Analyst; Sr. BI Analyst; Sr. DSS/BI Engineer; Data Warehouse/BI Administrator; and Data Warehouse/BI Management Specialist. Again this report won't answer Aileen's questions and the $800 cost is a barrier. It is interesting to examine the job titles included in the report. So based on case studies, anecdotes and my past readings, I'll give my opinions. If you are familiar with a survey related to these questions, please send me an email. My DSS book (Power, 2002) notes "A complex DSS built using either an SDLC or a prototyping approach requires a team development approach. Once the system is developed a group may also be needed to maintain the system. Some large-scale DSS are built with teams of 2-3 people or with a larger group of 10 or more. ... The composition of the DSS team will change over the development cycle so the project manager needs to provide direction and motivation for the DSS team." The data warehousing project that I am most familiar with was at Ertl (Power and Roth, 2003). In 1996 "the Ertl IS organization consisted of 23 employees, including six full time developers, three contract programmers, three managers of systems development, one database administrator, one business analyst, one manager of end user computing, two help desk specialists, and one network specialist. At the peak of data warehouse development efforts 21 contract programmers had been employed." Ertl is best classified as a medium-sized manufacturing organization. I'm familiar with data warehouse projects that involved a single internal staff person and projects that have been completely outsourced. At an insurance company, a data warehouse consultant was hired to develop a feasibility prototype for the senior actuary. He designed a star schema, established dimensions and an ETL interface. He used Cognos Enterprise PowerPlay to provide the BI mechanism for the Actuary. At a publishing and direct mail company an internal staff person built a financial data warehouse for corporate management in about 9 months. Many small-scale decision support projects can be completed by a Senior Decision Support System Analyst or Senior BI Analyst. Let's look at some examples from cases at DSSResources.com. Peter Barton (2003) reported the George Washington University data warehouse project team "consists of 1 full-time project manager and 5 project team members." The Iowa Department of Revenue data warehouse was initially built beginning in 1999 with extensive help from Teradata Professional Services staff. In March 2003, both the Web-based Audit Support Component and the Business Objects interface were completed. In 2004 after expanding its technical services staff, "one Teradata Professional Services staff member works full time alongside the revenue department staff, writing queries and ensuring the logical data model is fully operational" (Teradata Staff, 2004). Anissa Stevens, AVANCO International, worked with a small consulting team to build a data mart for Redland Genstar (cf., Stevens, 2004). Recently, Computer Sciences Corp. received a two year, $21.5 million contract to develop the Blue Health Intelligence data warehouse for the Blue Cross and Blue Shield Association. "CSC is designing, developing, deploying and hosting the warehouse. It will store clinical records for 20 Blue Plan operations and 79 million people, and is expandable to house records for 100 million people. The data warehouse is being tested and implemented this year, and plans to be fully operational in 2007." Check the August 4, 2006 release at DSSResources.com. What staffing level are most companies maintaining to effectively manage their Data Warehouse and data-driven DSS? Once a data-driven DSS has been built, at a minimum an organization needs a Data Warehouse/BI Management Specialist. It is possible a data base administrator with responsibility for a transaction system, can maintain the actual data warehouse, but someone needs to be concerned with extracting and loading additional data. This "bare bones" approach probably won't lead to any new capabilities. My best guess is that a staff of 3-5 people is a realistic minimum for maintaining any significant data mart/data warehouse/BI/data-driven DSS program. Are they flexing using consultants? or do they have a core DW Team? In general, once a data-driven DSS capability has been built, I think it is best to rely on a core team of permanent staff. Finding the right people may be difficult, so training and internal reassignment may be the best way to create a more permanent DW core team. The Iowa Department of Revenue case shows that external professional services staff from vendors or consulting groups may be involved for many years. If one checks the resumes of major data warehouse consultants, the tasks they perform varies tremendously from project to project. For example, Sid Adelman is a well know data warehouse writer and consultant. He worked with a large bank "in the development and implementation of a large marketing data mart." Sid helped in architecting, planning and staffing. He assisted in developing a scope agreement that documented the function, schedule and responsibilities to be delivered in each phase of implementation. He helped develop a metadata strategy. He assisted in the evaluation and selection of a RDBMS and of query tools. Also, Sid worked with the bank to create a capacity plan for a multi-terabyte data warehouse." Someone in an organization must have the knowledge to perform these tasks to build a sophisticated data warehouse. The knowledge needed to maintain the warehouse is much less and quite different. What is the "best practice" organizational structure that is used today? Lou Agosta in a Forrester/GIGA report in 2003 concluded "The common organizational structures for data warehousing staffing are centralized, decentralized, project-based and cross-functional. Giga knows of examples of successful data warehousing development and operations with each one of these types." I think that once large-scale, enterprise-wide data warehouses have been built, the technology system should be maintained and enhanced by a centralized unit in the IT organization with a Vice President/Director of Data Warehousing/BI/DSS. The organization structure for decision support is an important issue that must be addressed. A centralized structure for the decision support group can lead to problems in a large, decentralized organization. A project-based structure works well for development, but not as well for ongoing operation and enhancement of the decision support capabilities. Finally, a cross-functional structure may reduce some of the political problems, but I think that the dual reporting relationships and "turf" issues create an inherent long-run instability for this structure. In a recent column (DSS News, August 13, 2006), I discussed organizing for BI in a University setting. In that column, I argued that "how a DSS task group is organized depends upon the maturity of current data-driven DSS applications and upon how much maintenance they require and the aspirations for developing novel DSS applications in performance monitoring, budgeting or enrollment forecasting." In general, staffing for data-driven DSS should depend upon the magnitude of the business problem you are trying to solve. Structure of a decision support unit should reflect its functions and the other structures in the organization. As always, your comments and feedback are appreciated. References Agosta, L., "Organizational Design for Data Warehousing and Data Mart Support," Jan. 7, 2003, http://www.forrester.com . Barton, P., "The George Washington University Data-Driven Decision Support Project", 2003, posted at DSSResources.COM August 15, 2003. Computer Sciences Corporation developing IT data warehouse for Blue Cross and Blue Shield Association, 08/04/2006, DSSResources.com, URL http://www.dssresources.com/news/1680.php . Power, D.J., Decision Support Systems: Concepts and Resources for Managers, Westport, CT: Quorum/Greenwood, 2002. Power, D.J., "What is the process for designing decision support software?" DSS News, Vol. 7, No. 17, August 13, 2006. Stevens, A., "Implementing the Redland Genstar Data Mart", posted at DSSResources.COM July 2, 2004 Teradata Staff, "Closing the Tax Gap in Iowa", posted at DSSResources.COM February 21, 2004. ************************************************************ Purchase Dan Power's DSS FAQ book 83 frequently asked questions about computerized DSS http://dssresources.com/dssbookstore/power2005.html ************************************************************ DSS Conferences 1. Pre-ICIS 2006: SIG DSS Research workshop, Sunday, December 10, 2006, Milwaukee, WI. Check http://dssresources.com/news/1494.php . 2. ICDSS 2007, 9th International Conference on DSS, Jan. 2-4, 2007, Calcutta, India. Theme: Decision Support for Global Enterprises. Check http://www.ICDSS2007.org . ************************************************************ Visit DSSResources.com; Support our advertisers Advertise here! ************************************************************ DSS News Releases - October 9 to October 20, 2006 Read them at DSSResources.COM and search the DSS News Archive 10/20/2006 DecisionEdge acknowledged as a 'First to Market' partner for Microsoft Office Project 2007. 10/19/2006 Polycom updates Certified Video Conferencing Engineer Exam to help customers address evolving technology. 10/19/2006 EMC enhances Captiva InputAccel software for improved enterprise input management. 10/19/2006 Spreadsheet compliance solution from Mobius featured at Institute of Internal Auditors (IIA) All Star Conference. 10/18/2006 Microsoft releases Windows Internet Explorer 7 for Windows XP. 10/18/2006 Project management software company launches consultant partnership program. 10/18/2006 ClearForest Modeler: Putting the power of advanced text analytics in the hands of the business user. 10/18/2006 Stellent positioned in "Leaders" quadrant of enterprise content magic quadrant report from top industry analyst firm. 10/18/2006 Confio Software announces new Ignite™ for RAC performance management extends wait-time analysis specifically for Oracle RAC. 10/17/2006 79 percent of top performing enterprises will implement or enhance mobile sales solutions in next 12 months. 10/17/2006 Actuate and Webalo make spreadsheets mobile. 10/17/2006 One in three employees compromise corporate security through lax password practices, Nucleus Research study finds. 10/17/2006 SAP ecosystem continues to show strong global momentum. 10/16/2006 TIBCO helps businesses seize full benefits of SOA with master data management software. 10/16/2006 Call for papers: 4th International Conference on Informatics in Control, Automation and Robotics (ICINCO-2007), due January 10, 2007. 10/16/2006 LAPD deploys COPLINK(R) to support crime reduction, gang interdiction and counter-terrorism initiatives. 10/16/2006 Halogen Software's employee performance management solutions selected for American Hospital Association endorsement. 10/16/2006 Cognos continues retail industry leadership, launches new performance solution to help optimize promotions. 10/16/2006 Dubai Municipality selects Cognos 8 Business Intelligence. 10/16/2006 SAIC produces Future Force Company Commander (F2C2): The Future Combat Systems Videogame for the PC. 10/16/2006 Fair Isaac’s Falcon Fraud Manager 5.2 to drive payment card fraud detection at SHAZAM Network. 10/13/2006 Fair Isaac's model builder 3.5 delivers more analytic innovation for smarter decisions. 10/12/2006 New Crisis Management Center launched by Food and Agriculture Organization of the United Nations. 10/12/2006 Northwest Naturals selects Adonix X3 process software. 10/12/2006 Newly integrated Oracle Academy Program benefits students and academic institutions worldwide. 10/12/2006 Industry leader in foodservice redistribution selects ORTEC for transport planning and optimization. 10/11/2006 Workbrain clients showcase workforce management successes at Velocity 2006. 10/11/2006 Oracle announces keynote speakers for Oracle(R) OpenWorld San Francisco 2006. 10/11/2006 Data Warehouse changes the direction of lead generation technology with new services for the mortgage industry. 10/10/2006 Northwestern University standardizes on Cognos 8 business intelligence. 10/10/2006 Netflix reports global response to its $1 Million prize to promote progress in recommendation systems. 10/10/2006 Overall supply chain progress stalled according to Computer Sciences Corporation survey. 10/10/2006 Orbitz asks via online chat: Can I help you?. 10/10/2006 Call for papers: ISCRAM 2007, deadline January 15, 2007. 10/10/2006 On-the-go employees and customers can now access expert corporate advice via handheld mobile devices. 10/09/2006 Meridian Systems launches Prolog 2007 with improved construction project management features. 10/09/2006 CA's new Clarity Risk and Controls Manager draws praise from customers, partners and experts. 10/09/2006 Calyon selects Asset Control to centralize their enterprise-wide data management for the financial and commodities markets. ************************************************************ Please tell your DSS friends about DSSResources.COM ************************************************************ DSS News is copyrighted (c) 2006 by D. J. Power. Please send your questions to daniel.power@dssresources.com |