What skills do data-driven decision makers need?
by Daniel J. Power
Editor, DSSResources.COM
Decision making habits are often learned by trial and error. Decision making skills are learned through deliberate, systematic effort. Organizational culture also influences decision making behaviors. A culture can promote use of data to make decisions. More data, including "big data", and more and easier to use analytical tools provide an opportunity for improving operational decision making, but many managers must learn new behaviors and skills to actually use data and analyses effectively. Generally, managers must expand their skill sets to become effective data-driven decision makers.
Data-driven decision making refers to the collection and analysis of data to guide or make decisions. Provost and Fawcett (2013) define data-driven decision making as “the practice of basing decisions on the analysis of data rather than purely on intuition." Effective data-driven decision making requires a specialized skill set in addition to other decision making skills.
Organizations that embrace measurement have a data-driven culture. The culture encourages and rewards managers for making decisions based on meaningful data, rather than solely based on intuition, cf., Kanter (2011). Providing data analyses and encouraging its use does not however guarantee better decisions. Managers must understand the analyses and they must want to incorporate the results into their thinking about a situation. Data can be used to bolster previously made decisions and analyses can be requested to support rationalizing. Skilled data-driven decision makers must learn to reserve judgement and postpone a final choice until the facts are presented.
Shea, Santos and Byrnes (2012) differentiate between data-based and data-supported decisions. Both use quantitative and/or qualitative data to inform decisions, while with data-based decisions there is a presumption that the underlying ethics have been properly considered. Data-supported decisions "use the same data but they also take into account people, issues, ethics, and broader system effects." They caution that "an excessive 'data driven' emphasis can contribute to ethical blind spots, altering logic, ethics, and systems outcomes. It is important to note that not all aspects of a system can be measured and captured by data, and that the situational context must be considered when using data to inform decisions."
Enderle (2013) provides examples of poor use of data and analyses at both IBM and Microsoft. He reports Microsoft's internal market research organization was giving "executives results that made decisions they had already made look smarter—apparently so they were better protected when the decision didn't work out." He provides other examples from his personal experience and observes "a surprisingly small number of the companies that sell analytics tools actually rely on those tools for major decisions."
Blogger Kalie Moore (2014) at Business Intelligence software vendor datapine.com reflects that the "insights we provide are completely useless if, at the end of the day, these reports are ignored by the actual decision makers. This conundrum prompted us to take a deep look: Why are business leaders not utilizing data driven decision making? And what should you be aware of to make sure your decisions are based on numbers, not feelings?" Her answer was overreliance on past experience, going with their gut and cooking the data, and cognitive biases. Moore notes "The good news is that there are ways to overcome biased behavior."
Data-driven decision making is a major topic in the educational learning assessment and education management literature, cf., http://www.clrn.org/. According to Mandinach, Gummer and Muller (N.D.), "Data-driven decision making has become a national education priority". They note a "lack of clarity in terminology around data-driven decision making whereby multiple definitions create the potential for confusion." Their symposium group also concluded that data-driven skills "form a developmental continuum rather than a set body of knowledge." According to the Using Data to Make Decisions course (http://fcit.usf.edu/data/) "the underlying premise is that if instructional change is based on data-driven decision making, student learning will increase to a higher level."
Attendees at the 2013 MIT Sloan CIO Symposium (May 24, 2013), were told that companies will need to develop data-driven decision making skills to take full advantage of new big data analytics technologies." It was not clear what skills should be developed.
WestEd Data for Decisions Initiative identifies eight data literacy skills: "inquiry processes, habits of mind, general data use skills, understanding of data quality, understanding of data properties, data use procedural skills, the ability to transform data to information, the ability to transform data to implementation, and collaboration."
So what data analytics skills are needed by managers and decision makers? Although no agreement currently exists, the primary data-driven decision making skills seem to broadly encompass: 1) collecting and identifying relevant data, 2) using software to perform statistical analysis including charting of data, 3) interpreting data and analyses in the context of an actual decision situation, and 4) using analyses of data, including sensitivity analyses, to inform decisions.
Collecting and identifying relevant data
Organizations collect large amounts of data, including "big" data, and external data can also be purchased. Often times new data can also be captured from websites, customer surveys or passive data collection. Data-driven decision makers need to understand data resources and data capture and how to work with stored data, to use metadata, to identify what data is available or what new data should be captured.
Using software to perform statistical analysis including charting of data
There are many software tools that decision makers can learn. Often desktop tools like Excel and Tableau are adequate. Managers can request help to prepare more sophisticated analyses using software like SAS or R. Learning a statistical analysis package will help decision makers interpret analytical results and understand the limitations of statistical analysis.
Interpreting data and analyses in the context of an actual decision situation
Interpreting data analyses is in many situation the most difficult task of all. Decision makers need to match data and analyses to questions of interest. Decision makers should ask: What do I need to know about this situation? Is there data that will help me understand my choices in the situation? What does the data mean in my specific situation? Do I have preconceived solutions or biases? Cognitive biases exist. People are predisposed to make choices by the way information is presented and the way analyses are conducted, cf., Power (2005). Many decisions can be framed either in terms of gains or losses. How a decision is framed can also impact choices of a decision maker, cf., Whyte (1991).
Using analyses of data, including sensitivity analyses, to inform decisions
Data can inform decisions, but data does not always provide conclusive evidence. In some situation data analysis shows a strong correlation, but the causal evidence is much more circumstantial. Correlation is not causation, but in many cases correlation is the strongest conclusion that can be drawn about a relationship. Frick (2014) briefly examined when to act on correlation and concluded "The more frequent the correlation, and the lower the risk of being wrong, the more it makes sense to act based on that correlation." It seems best to cautiously use available data to inform decisions rather than ignoring data.
Tom Davenport, MIT Center for Digital Business, argues "Quantitative analytical thinking is key to successful individual and organizational decision making. Yet many managers lack the necessary orientation to quantitative matters." Learning to think using quantitative analyses is an important skill.
Overall, data analysis skills have been taught to business and social science researchers more than to managers. In Colleges of Business, courses in basic statistical inference are more common than applied courses in finding and analyzing data, and in interpreting correlations. There is a skills gap and managers and decision makers need to be skilled and sophisticated data users and data interpreters. Managers need to be skilled data-supported decision makers.
Software and Information Technology company managers think organization decision makers will want to use "big" data in their decision making (cf., Lohr, 2011). If that belief is realized, then decision makers must develop the skills needed to use data effectively. Part of the problem that hinders using data effectively is timing. Enderle (2013) reminds us "If you're going to do research, it needs to come before the decision is made ..." Analyse data and share the results before a decision is made.
References
Enderle, R., "Data Analytics Will Fail If Executives Ignore the Numbers," CIO, January 4, 2013 at URL http://www.cio.com/article/2389409/data-management/data-analytics-will-fail-if-executives-ignore-the-numbers.html
Frick, W., "An Introduction to Data-Driven Decisions for Managers Who Don’t Like Math," Harvard Business Reviwe, May 19, 2014 at URL https://hbr.org/2014/05/an-introduction-to-data-driven-decisions-for-managers-who-dont-like-math/
Kanter, B., "How Can Nonprofits Switch to a Data-Informed Culture?" 7/19/2011 at URL http://www.bethkanter.org/switch-data-driven/
Lohr, S., "When There’s No Such Thing as Too Much Information," New York Times, April 23, 2011 at URL http://www.nytimes.com/2011/04/24/business/24unboxed.html?_r=0
Mandinach, E. B., E. S. Gummer, and R. D. Muller, "The Complexities of Integrating Data-Driven Decision Making into Professional Preparation in Schools of Education: It’s Harder Than You Think," at http://educationnorthwest.org/sites/default/files/gummer-mandinach-report-summary.pdf
Moore, K., "Data Driven Decision Making? Why We Ignore Data and Go With Our Guts," August 6, 2014 at URL http://www.datapine.com/blog/data-driven-decision-making/ .
Power, D. J. "What is data-driven decision making?" Decision Support News, Vol. 15, No. 23, November 9, 2014 at URL http://dssresources.com/newsletters/383.php
Power, D. J. "Can computerized decision support systems impact, eliminate, exploit, or reduce cognitive biases in decision making?" DSS News, Vol. 6, No. 20, September 11, 2005; updated September 13, 2014 for Decision Support News Vol. 15, No. 19.
Provost, F. and T. Fawcett, “Data Science and its relationship to Big Data and Data-driven Decision Making,” Big Data, Vol. 1, No. 1, March 2013 at URL http://online.liebertpub.com/doi/pdf/10.1089/big.2013.1508 .
Shea, J., J. Jones Santos, and P. Byrnes, "Community Needs Assessment and Data-Supported Decision Making: Keys to Building Responsive and Effective Health Centers," National Association of Community Health Centers, November 2012 at URL http://www.nachc.com/client/documents/2012%20Data%20Supported%20Decision%20Making.pdf
WestEd Data for Decisions Initiative, "Frequently Asked Questions related to data for decision making," at URL http://datafordecisions.wested.org/data-use-basics/frequently-asked-questions/ .
Whyte, G., "Decision failures: why the occur and how to prevent them," Academy of Management Executive, Vol. 5, No. 3, Aug., 1991.
keyphrases: decision making, strategic decision making, decision support, computerized decision support, data-driven decision makers, data-driven decision making skills
Last update: 2014-12-22 07:24
Author: Daniel Power
Print this record
Show this as PDF file
You cannot comment on this entry
