What is data-driven decision making?
by Daniel J. Power
A recent Capgemini (2012) study found that 9 out of 10 business leaders “believe data is now the fourth factor of production, as fundamental to business as land, labor and capital.” That report concludes “Big Data represents a fundamental shift in business decision making.” Data-driven decision making refers to the collection and analysis of data to guide or make decisions. Data has been used to support structured decisions since people began collecting and recording data. High velocity data can potentially improve routine, reoccurring decisions, but the impact on unstructured strategic decisions is less certain. Some vendors broadly assert that big data and statistical analysis will be helpful for "all" decisions if we are creative in using data. Decisions are choices among options, but some choices are more complex and difficult than others and some options are ambiguous and poorly defined.
In a Deloitte Review article, Guszcza and Richardson (2014/28/7) state "Today few doubt that, properly planned and executed, data analytic methods enable organizations to make more effective decisions. Anecdotal evidence abounds." They are more skeptical about the necessity of using big data, noting it is false that big data is necessary for analytics to provide big value.
Provost and Fawcett (2013) define data-driven decision making (DDD) as “the practice of basing decisions on the analysis of data rather than purely on intuition." They state “The benefits of data-driven decision making have been demonstrated conclusively.” They note in the next sentence “Economist Erik Brynjolfsson and his colleagues from MIT and Penn’s Wharton School recently conducted a study of how DDD affects firm performance.”
Frick (2014) also cites McAfee and Brynjolfsson (2012) to justify the importance of "big data". He notes they "describe the opportunity and report that 'companies in the top third of their industry in the use of data-driven decision making were, on average, 5% more productive and 6% more profitable than their competitors' even after accounting for several confounding factors."
Former Chairman of the Board of Governors of the IBM Academy of Technology, Irving Wladawsky-Berger (2013/27/9) noted in a Guest column in the Wall Street Journal that “Decision making has long been a subject of study and given the explosive growth of Big Data over the past decade, it’s not surprising that data-driven decision making is one of the most promising applications in the emerging discipline of data science.” He explores the use of big data in decision-making and concludes “the use of Big Data and data science to help with strategic decisions is in its early stages and requires quite a bit more research to understand how to use them under different contexts.”
Ma (2010) is a proponent of data-driven decision making, but he explains that “In the real world the number of variables are often too numerous and the sample size too small to create perfect models. But that doesn’t mean you should ignore the data. It just means that you always strive to find the best data—the most relevant data with which to make the best decisions. I call this data-driven intuition, and it is a process that many successful people implement when faced with a situation where using a purely data-driven statistical model is impossible.” For a discussion of this issue see Harris (2014).
Power (2008) argued it is important for managers and Information Technology professionals to understand data-driven decision support systems and how such systems can provide business intelligence and performance monitoring. Data-driven DSS is one of five major types of computerized decision support systems.
In the Banking Industry, Crosman (2014) reports Bank CEOs fear data-driven decisions. She notes "A recent study found that analytics are underused at banks and that senior executives are cold to the technology: a scant 20% said that if it were up to them their organization would be highly data driven." She thinks "the concept of data-driven decisions makes some executives nervous." Bob Meara, senior analyst at Celent and author of the report Crosman cites is quoted "By definition, the interesting discoveries are intuitively very uncomfortable — if it doesn't seem right and you can't trust your gut, then you've got to trust the numbers. That's hard for people."
Attunity (http://www.attunity.com/) notes in a blog post (2013) "In the past, business decision-making processes were largely based on intuition and gut feeling, suggesting that executives who had the most experience in the workplace could often resolve issues more quickly than new managers. ... As companies move away from emotional decision-making processes and recognize the benefits behind fact-driven procedures, the need to analyze information will rise and give way to the growing use of enterprise analytics. ... The bottom line is that the rapidly evolving digital landscape is driving monumental changes in the corporate environment, including the realization that data-driven decision-making is one of the most effective ways to gain a strong competitive advantage over rival firms that still rely on outdated procedures. By planning for a cultural and technological change, executives can embrace new business intelligence solutions without experiencing too much negative feedback."
Let's examine three research studies focused on important strategic decisions:
Creation of a new product or service decisions: Brynjolfsson, Hitt, and Kim’s (2011) research is frequently cited as a justification for making increased investments in using new data streams and analytics. They did report that effective use of data and analytics correlated with a 5 to 6 percent improvement in productivity, as well as higher profitability and market value. The study used survey data on the business practices and information technology investments of 179 large publicly traded firms collected in 2008. The survey was conducted in conjunction with McKinsey and Company. Their key independent variable, data-driven decision-making (DDD), combined responses from three questions in the survey: 1) the usage of data for the creation of a new product or service, 2) the usage of data for business decision=making in the entire company, and 3) the existence of data for decision-making in the entire company. The survey questions used a scale from 1-5 with 5 indicating high on data-driven. The specific questions used were:
1. Typical basis for the decision on the creation of a new product or service 1 (Experience/expertise) to 5 (Data)
2. We depend on data to support our decision-making (the work practices and environment of the entire company) 1 (Strongly Disagree) to 5 (Strongly agree)
3. We have the data we need to make decisions 1 (Strongly Disagree) to 5 (Strongly agree)
Survey data was only collected at one point in time, but the researchers assumed the 3 item data-driven decision-making variable was quasi-fixed for the years from 2005 to 2009. The internal consistency of the measure was low (Chronbach's Alpha=0.58). The item means indicate the managers used data. The mean for Item 1 of 2.97 (SD 1.13) is at the indifference or balance point on the scale. The respondents use data and experience/expertise equally for deciding on the creation of a new product or service. For item 2 the mean was 3.85 with a standard deviation of 0.85 and for item 3 the mean was 3.43 (SD 0.87). One can conclude both data and experience are used in many organizations surveyed, but there is a high standard deviation. Respondents depend on data and think they have the data they need. A single 2008 survey is hardly conclusive evidence about the benefits of data-driven decision making in all decision making situations.
McAfee and Brynjolfsson (2012) tempered the conclusion stating "The more companies characterized themselves as data-driven, the better they performed on objective measures of financial and operational results."
Crisis response decisions: In 2010, Drnevich, Brush and Chaturvedi examined the differential impact of rational data-driven decision making versus political decision making on the effectiveness of crisis response decisions. They used a computational experimentation methodology. They surveyed decision makers to determine the process used to choose responses and they evaluated the effectiveness using an agent-based simulation to generate results from the choices. They failed to "find support for rational decision making processes, the effects of political approaches are significantly more negative on effectiveness in our current context (p. 12)."
Resource allocation decisions: Lachat and Smith (2005) evaluated data-driven decision making processes in five urban, low performing high schools. Before the initiative, the high schools faced problems related to the data they collected. In general, the schools’ data were inaccurate, not timely, and limited in scope. They helped the schools to collaborate to make a district-level plan to improve the data collection process. The researchers helped the schools to develop a leadership team to share and coach each other through the data gathering and analysis process. Providing data helped the staff to feel more comfortable reviewing and using the results. The staff supposedly learned to use data to allocate resources towards programs that were more beneficial for their school.
In these three studies of three different strategic decision situations, there was support for using data-driven decision making. The data-driven support was however limited and did not use "big data". In all three studies, decision processes may explain more of the variance in strategic decision outcomes than the amount and use of data.
Data did not "drive" strategic decision making in these situations. The U.S. Department of Education prefers the term data-informed decision-making asserting that decisions should not be based solely on quantitative data. Data-driven, data-informed or fact-based decision making means managers use and evaluate data to make decisions. Data is useful, providing more data is not necessarily the way to improve decision making effectiveness. Implementing data-driven decision-making with big data for all decisions is unlikely to improve outcomes.
Managers gain advantage by making decisions better and faster. The U.S. Military defines decision superiority as the "ability of the commander, based upon information superiority and situational understanding, to make effective decisions more rapidly than the adversary, thereby allowing one to dramatically increase the pace, coherence, and effectiveness of operations."
Peter Drucker (1954) observed that ".. management is always a decision-making process." He notes that in regard to the then new decision tools from Operations Research, managers "must understand the basic method involved in making decisions. Without such understanding he will either be unable to use the new tools at all, or he will overemphasize their contribution and see in them the key to problem-solving which can only result in the substitution of gadgets for thinking, and of mechanics for judgment. Instead of being helped by the new tools, the manager who does not understand decision-making as a process in which he has to define, to analyze, to judge, to take risks, and to lead to effective action, will, like the Sorcerer's Apprentice, become the victim of his own bag of tricks (p. 368)." Managers have this same risk today with "big data" and analytics.
The goal of senior managers should be to selectively use data and analytics and to selectively implement new data-driven systems to achieve decision superiority over competitors and adversaries. Thinking and decision processes remain the key to successful decision-making.
Attunity Blog Post, "Data-driven decision-making will push analytic needs forward," Attunity, December 20, 2013 at URL http://www.attunity.com/learning/articles/data-driven-decision-making-will-push-analytic-needs-forward .
Bruner, J. "Five Steps For Making Data-Driven Decisions," Forbes, 4/20/2012 at URL http://www.forbes.com/sites/jonbruner/2012/04/20/five-steps-for-making-data-driven-decisions/
Brynjolfsson, E., L. M. Hitt, and H. H. Kim, “Strength in numbers: How does data-driven decisionmaking affect firm performance?,” Working paper, Social Science Research Network (SSRN), April 2011 at URL http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1819486.
Capgemini, Inc., “The Deciding Factor: Big Data & Decision making June 4, 2012 at URL http://www.capgemini.com/resources/the-deciding-factor-big-data-decision-making .
Crosman, P., "Bank CEOs Fear Data-Driven Decisions," Information Management, Novemebr 20, 2014 at URL http://www.information-management.com/news/Banks-Big-Data-Analytics-Decisions-Financial-Services-10026271-1.html .
Drnevich, P.L., T.H. Brush and A. Chaturvedi, "Examining the Implications of Process and Choice for Strategic Decision Making Effectiveness," International Journal of Decision Support System Technology, Vol. 2, No. 3, July-September 2010, pp. 1-15.
Drucker, P. The Practice of Management. New York: Harper and Brothers, 1954.
Frick, W., "An Introduction to Data-Driven Decisions for Managers Who Don’t Like Math," Harvard Business Review, May 19, 2014 at URL https://hbr.org/2014/05/an-introduction-to-data-driven-decisions-for-managers-who-dont-like-math/ .
Guszcza, J. and B. Richardson, “Two dogmas of big data: Understanding the power of analytics for predicting human behavior,” Deloitte Review, issue 15, 2014, 161-175 at URL http://dupress.com/articles/behavioral-data-driven-decision-making/ .
Harris, J., "Time for Data-Driven Intuition," Information Management, November 18, 2014 at URL http://www.information-management.com/blogs/Data-Driven-Intuition-10026261-1.html .
Lachat, .A., & Smith, S. (2005). Practices that support data use in urban high schools. Journal of Education for Students Placed at Risk, 10(3), 333-349.
Ma, J., The House Advantage: Playing the Odds to Win Big In Business, Palgrave Macmillan Trade, July 6, 2010.
McAfee, A. and E. Brynjolfsson, "Big Data: The Management Revolution," Harvard Business Review, October 2012 at https://hbr.org/2012/10/big-data-the-management-revolution/ar .
Power, D. “Understanding Data-driven Decision Support Systems,” Information Systems Management, Volume 25 Issue 2, March 2008, 149-154.
Provost, F. and T. Fawcett, “Data Science and its relationship to Big Data and Data-driven Decision Making,” Big Data, Vol. 1, No. 1, March 2013 at URL http://online.liebertpub.com/doi/pdf/10.1089/big.2013.1508 .
U.S. Department of Education Office of Planning, Evaluation and Policy Development (2009). Implementing data-informed decision making in schools: Teacher access, supports and use. United States Department of Education (ERIC Document Reproduction Service No. ED504191)
Wladawsky-Berger, I. “Data-Driven Decision Making: Promises and Limits,” Wall Street Journal, September 27, 2013 at URL http://blogs.wsj.com/cio/2013/09/27/data-driven-decision-making-promises-and-limits/ .
keyphrases: big data, use case, strategic decision, decision making, strategic decision making, decision support, computerized decision support, decision maker
Last update: 2014-11-27 09:03
Author: Daniel Power
You cannot comment on this entry