from DSSResources.com


Can decision support developers reduce the probability of solving the ‘wrong’ problem?

by Daniel J. Power
Editor, DSSResources.COM

Data analytics and decision support systems are intended to help people, especially managers, make better decisions. Decisions are not made in a vacuum. Rather like the testing of scientific ideas, decisions "are always nurtured against a background of prior, partially conflicting ideas", cf. Mitroff and Featheringham (1976). Decision support and analytics can assist in making more systematic decisions and hence reduce the incidence of bad decisions. Decision support developers and data scientists may however neglect to examine the possibility that the algorithms and systems they are designing and building encourage or actually do solve the wrong problem. This error has been called the error of the third kind (abbreviated EIII or Em).

Kimball (1957) defines EIII as "the error committed by giving the right answer to the wrong problem (p. 134)". Similarly, Mitroff and Featheringham (1974) define EIII "as the probability of having solved the wrong problem when one should have solved the right problem (p. 383)." EIII is an error of conceptualization. This simple definition of the conceptualization error assumes we can know with some certainty the right decision problem to solve. We can not have certainty about our framing and conceptualization of a problem, but thoughtful problem formulation can help reduce this error. Systematic problem formulation is central to developing effective decision support and analytics.

Krippendorff (1986) defines problem formulation as "an activity aimed at identifying a problem by specifying (a) the undesirable and problematic state currently occupied, (b) the resources currently available to move away from that problematic state, particularly the available courses of actions, the combinatorial constraints on using them, etc., and (c) the criteria that need to be satisfied to say that a problem no longer exists or is solved. This activity defines the cognitive gap between what is and what is desirable and delineates the resources for closing it. Problem formulation is the creative and probably the more important step towards overcoming a problematic state than problem-solving. A good definition of what the problem is, is believed to be more than half of the way towards its eventual elimination."

Rahim (2001) notes "In contemporary organizations, problem formulation in the problem recognition phase is often distorted. As a result, old policies, procedures, and practices continue to be followed although they may have been rendered ineffective because of changes in the external environment. This typically results in a type III error ... Effective conflict-management strategies should be able to minimize type III .. errors (p. 371)."

What can be done during problem formulation to increase the probability that a proposed analytical decision support capability will help solve the ‘right’ problem?

Kimball (1957) argued that in the realm of statistical consulting Masters degree students should learn about the EIII problem and be exposed to examples. He also suggested that better communication between the decision maker and the expert about all dimensions of a problem could help reduce the occurrence of a type three error (EIII). Some have suggested that using a Devil's Advocate approach to question the problem formulation can help, cf. Schwenk, 1984. The appointed Devil's Advocate critiques and questions the problem formulation. Finally, asking questions is an important skill for decision support developer to learn. Framing and re-framing a problem based upon answers to situation relevant questions helps improve the final problem definition.

In conclusion, ideally providing decision support and analytics to decision makers should help them answer the right question and resolve the "real" problem in a situation. Sadly, this doesn't always happen. Helping people answer a related question or a narrower one or even a broader question because that is what someone thinks is appropriate, wastes resources and creates an illusion of helping a decision maker. Decision support developers and data scientists must work hard to develop tools that are relevant and that are accurate in resolving the "real" problem, issue, or decision task. "Good enough" is not good enough.

References

Kimball, A. W., "Errors of the Third Kind in Statistical Consulting," Journal of the American Statistical Association, Vol. 52, No. 278, June 1957, pp 133-142.

Krippendorff, K., "A Dictionary of Cybernetics," 1986 at URL http://pespmc1.vub.ac.be/ASC/Kripp.html .

Mitroff, I.I. and T. R. Featheringham, "On systemic problem solving and the error of the third kind," Systems Research and Behavioral Science, Volume 19, Issue 6 November 1974, pp 383–393 at URL http://onlinelibrary.wiley.com/doi/10.1002/bs.3830190605/abstract.

Mitroff, I.I. and T. R. Featheringham, "Towards a behavioral theory of systemic hypothesis-testing and the error of the third kind," Theory and Decision July 1976, Volume 7, Issue 3, pp 205–220.

Rahim, M. A., "Managing Organizational Conflict: Challenges for Organization Development and Change," in Golembiewski, R. T. Handbook of Organizational Behavior, Second Edition, Revised and Expanded, Marcel Dekker, Inc., New York, NY, 2001, pp 365-388 at https://books.google.com/books?id=g5pD9ciNvA0C .

Schwenk, C. R., "Devil's Advocacy in Managerial Decision-Making," Journal of Management Studies, Volume 21, Issue 2, April 1984, pp 153–168.

Last update: 2017-06-25 10:05
Author: Daniel Power

Print this record Print this record
Show this as PDF file Show this as PDF file

Please rate this entry:

Average rating: 4 from 5 (1 Votes )

completely useless 1 2 3 4 5 most valuable

You cannot comment on this entry





DSS Home |  About Us |  Contact Us |  Site Index |  Subscribe | What's New
Please Tell Your Friends about DSSResources.COMCopyright © 1995-2015 by D. J. Power (see his home page).
DSSResources.COMsm is maintained by Daniel J. Power. Please contact him at djpower1950@gmail.com with questions. See disclaimer and privacy statement.


Google
 
Web DSSResources.com

powered by phpMyFAQ 1.5.3