Professional and Collaborative Decision-Support: Many Ways to Improve Decision-Making by Christopher Lang, Philosophy Ph.D. student at University of Wisconsin-Madison Introduction Since we are a social species, the most obviously far-reaching way to help our companies, nations and world is to improve the decision-making of our leaders. Accountants, statisticians, strategists, futurists and many other kinds of knowledge professionals each work on some part of a decision-support system. Stepping back to analyze the system as a whole helps us understand the role each part plays, the limits to what it can accomplish, and how information technology can best be applied. This essay identifies four potential bottlenecks for all decision-making processes, and lists the appropriate strategies for improving decision-making as dependent on where the limiting bottleneck lies. It then derives parameters of future decision-support technologies and techniques as required for optimal performance against decision-support problems. Because of the multiplicity of potential bottlenecks, the optimal system is found to include a combination of three cutting edge technologies: formal systems processing, collaborative filtering and self-filtering communities. Potential Bottlenecks in Decision-Making Processes The quality of most, if not all, common decision-making processes depends on the quality of four aspects: imaginative, experiential, inferential and evaluative. Figure 1 identifies these four aspects in two contrasting decision-making processes as examples. The imaginative aspect of decision-making produces a list of considered actions or strategies and outcomes. If the best action is never imagined, then the quality of our decision-making is limited. Also, if the outcomes that would actually result are never considered, then again the quality of the decision is limited. When we imitate or use the work of architects, designers, inventors, artists, historians or others who conceive or record potential possibilities, we are outsourcing at least part of the imaginative aspect of decision-making. The experiential aspect of decision-making produces measurements of the world. If one knew absolutely nothing about the real world, then they could not distinguish the possible from the impossible, so their list of considered actions and outcomes would have no correlation to real possibility, and would be useless. When we consult others who have been down our path before or use the work of pollsters, accountants, experimenters and other researchers who specialize in measurement, we are outsourcing at least part of the experiential aspect of decision-making. The inferential aspect of decision-making connects considered actions to outcomes. The future is what ultimately matters in decision-making, and knowledge about what we should expect requires inference, so decision-making quality can be limited by unsound or incomplete inference about expectations. When we use the work of statisticians, engineers, logicians and other theoreticians, we are outsourcing at least part of this aspect of decision-making. The evaluative aspect of decision-making assigns values to potential outcomes. This aspect is often viewed as central to decision-making because it cannot be outsourced without also delegating power in the process (e.g. when a general delegates tactical decisions to field commanders, when a decision is put to a vote, or when a group elects a representative). A decision-maker who lacked moral sensitivity or who was outright evil might make the wrong decision, despite having complete understanding of all possible actions and the outcomes that would follow. As shown in Figure 2, "decision-support" services are distinguished from "decision-making" services in that the former do not include evaluation. It is improper for decision-support providers to influence outcomes by misrepresenting or withholding possibilities, evidence or inference because decision-support includes reserving all evaluative aspects for the support recipient. Advances in quality and usage of decision-support services should allow leaders to retain control and responsibility for decisions, yet diminish the need for them to provide anything other than values and agency. Choosing the Best Strategy for the Situation The best strategy for improving decision-making, like the best strategy for fixing a car, depends upon what's going wrong: Symptoms Treatment When to use it Lack of innovation (i.e. the best actions and strategies are rarely even considered) Choose a more imaginative leader When good decision-making requires only average imagination Get thousands of people to brainstorm ideas, as in the Scenario Planning Database and Open Consulting When good decision-making requires greater-than-average imagination and communication networks like the Possible Futures Database become practical (e.g. setting public policy, corporate strategy, developing a marketing plan, developing new products, etc.) Train decision-maker’s imagination or hire an expert to independently come up with ideas When good decision-making requires greater-than-average imagination and communication networks like the Possible Futures Database are not practical (e.g. development of secret weapons, planning a crime, etc.). Research and develop imagination tools and techniques Use as a long-term supplement to shorter-term treatments when good decision-making requires greater-than-average imagination Relevant facts are unknown Choose a more experienced leader When good decision-making requires only average experience Get thousands of people to brainstorm relevant facts, as in Open Consulting When good decision-making requires greater-than-average experience and communication networks as in Open Consulting are practical (e.g. deciding whether to implement a proposed marketing plan, choosing from a set of options) Broaden decision-maker’s experience or hire an expert to gather data for you When good decision-making requires greater-than-average experience and communication networks as in Open Consulting are not practical (e.g. field soldier, oil speculation, etc.). Research and develop measurement tools and techniques Use as a long-term supplement to shorter-term treatments when good decision-making requires greater-than-average imagination Despite access to relevant facts, one is still unsure what consequences to expect from a considered action Choose a leader with greater inferential skills When good decision-making requires only average inferential skills Get thousands of people to attack inferential problems as with RIGOR software and Open Consulting When good decision-making requires better-than-average inferential skills and communication networks like RIGOR become practical (e.g. anticipating environmental consequences, optimizing economic policy, etc.) Train decision-maker’s inferential skills or hire an expert to independently conduct inference When good decision-making requires better-than-average inferential skills and communication networks like RIGOR are not practical (e.g. playing chess, cracking secret codes, etc.). Research and develop inferential tools and techniques Use as a long-term supplement to shorter-term treatments when good decision-making requires greater-than-average imagination Bad decisions are made despite excellent understanding of the possible actions and consequences Choose a leader with greater moral sensitivity. Always. If the decision-maker truly lacks moral sensitivity (or is evil), then training and decision-support will be ineffective (one might doubt, however, that such people exist outside of fiction). The Future of Decision-Support Optimal decision-support must be collaborative, since the most exhaustive collections of imagination, experience and inferential power are those of the largest groups. The technologies that facilitate collaboration will have to solve the following problems: Sorting through vast amounts of information Motivating collaborators to contribute Establishing trustworthiness of contributions Facilitating appreciation of insights Providing security Providing confidentiality/personal protection Optimal decision-support should optimize decision-making even of recipients that bring only values and agency. Values include standards of trustworthiness, so optimal decision-support may need to include more than just an accurate list of options and their outcomes-it must include whatever is required to establish its trustworthiness. Since this is typically dealt with in different ways for imagination, inference and experience, the optimal system must have different parts for the collection of each: Gathering Imagination: The barrier here is appreciation rather than establishment of trustworthiness. Think science fiction--no one need prove that it is trustworthy, but the author has to convey a great deal of detail before the reader will appreciate what is being suggested. The optimal decision-support system must contain a scenario planning database of imagined futures conveyed as stories, images, movies, 3D-models and other simulations that convey great detail without taxing the audience. Users will be able to type things like "law offices 2007", and discover a wide range of scenarios. Gathering Inferential Insights: The trustworthiness of inferential insights, such as proofs and solutions, is established in relation to formal systems, such as systems of logic, mathematics, music theory, grammar or rules to games (etc.). Calculators, spell-checkers and logic-checkers are early examples of technologies used to overcome the tedium and complication of formal systems. Since there is nothing subjective about the trustworthiness of inferential work, these technologies can sometimes proactively suggest ways to correct or finish inferential work. User interfaces may vary from things that look like writing on paper to jigsaw puzzles or virtual reality simulations--the underlying inferential problem is equivalent. The obvious next step is "RIGOR" software, a generalized checker that can increase functionality as new formal systems and user interfaces are invented. Gathering experience: To establish the trustworthiness of experiential claims, we publish them where credentialed "experts" with related experience can check them. Credentials motivate cooperation, and provide "stamp of trustworthiness". Where work is independent, the queue of claims yet to be confirmed (often sitting on the World Wide Web) becomes large, redundant and cryptic. The optimal decision-support system must facilitate teamwork among claim-makers, so they can create more streamlined inputs for credentialed checkers. Since these checkers will be inclined to pay more attention to work customized to meet their needs, challenges they post will serve as catalysts for team formation in Open Consulting forums, much as SourceForge does for software development. Open systems: The technologies described above can be implemented on corporate Intranets or through password protected portals, but advantages come from making them more open and distributed. First, greater openness provides access to wider range of imagination, inference and experience. Second, a distributed system is less vulnerable to attack. Third, increased openness makes anonymity a more meaningful option for contributors. Fourth, with completely open systems, all participants can be legally protected through open source licenses such as the open publication license. Fifth, as in academia and open source software, openness generates goodwill that motivates contributions. Collaborative filtering: All three of the technologies described above can be enhanced with collaborative filtering. Machine learning algorithms crunch on the history of all users to predict which insights will be most helpful to a given user at a given moment. Users might type in search terms, but search results would be customized. Alternately, recommendations can be "pushed" at users proactively, like at Amazon.com. These emerging technologies address the six problems mentioned above. Collaborative filtering and self-filtering communities address the information sorting problem. Credentials and formal systems address the problem of establishing trustworthiness. Multimedia formats, automated checking and self-filtering communities address problems with appreciation. Distributed architectures address security problems, and open access addresses the issue of motivation and personal protection. Together, they make collaborative decision-support feasible.For extensions of this article, including a description of how one might try to assure optimality of a decision-support system using "information aggregating markets", see http://www.predictionscience.org/ml/dsupport.php. Acknowledgments Several people deserve credit for shaping my thinking on this subject. Credit is due mostly to people who synthesized and refined information before sharing it with me. Bob Cordery, Brian Romansky, Rick Heiden and others in the Pitney Bowes secure systems group allowed me to take an invasive look at their decision-making processes and taught me about open source and security issues. Anand Chhatpar inspired my original thinking about open consulting and the possible futures database, and the Collaborative Leadership Club at the University of Wisconsin-Madison has facilitated ongoing experiments with these concepts. Jason Strutz and David Page worked with me to develop thoughts about RIGOR. Herbert Simon and the neo-classical economists are credited with developing previous formalisms for analysis of decision-making. I thank Ray Aldag and Dan Power for their comments on earlier drafts. About the Author If you have any comments or observations, you can contact Chris Lang by e-mail at cclang@wisc.edu. Chris Lang is a second year MBA student and 4th year Ph.D. student in philosophy at the UW-Madison, specializing in decision-support techniques. He has served as marketing research analyst for over a dozen Fortune 500 clients. He earned his BS degree in engineering physics from Cornell University in 1994. Citation Lang, C., "Professional and Collaborative Decision-Support: Many Ways to Improve Decision-Making", DSSResources.COM, 07/11/2003. Chris Lang provided permission to archive this article and feature it at DSSResources.COM on May 3, 2003. This article was posted at DSSResources.COM on July 11, 2003.