What does it mean to keep a person in the decision loop?
by Daniel J. Power
In this Age of Automation, machines with appropriate software can make well-understood decisions without involving a person. We must ask what role, if any, a person should have in an automated decision-making loop. Should there be a role in every phase/step of 1) initiation, 2) analysis, 3) choice, 4) action-taking, and 5) feedback and monitoring? Many managers have a generalized concern that automated decisions are still poorly understood and that they need to be more transparent. Also, there is concern about responsibility, stakeholders need to know who is responsible for a decision error.
Does "being in the decision loop" mean that a human should make the final decision? Perhaps it means that a person should regularly monitor the decisions that are made? Is it enough that a person approves the use of the decision algorithm or its use in a particular situation? The speed of the decision loop can be very fast with decision automation and that can limit human participation. The easiest role for a person in-the-loop is associated with feedback and monitoring the decision process. Historically, feedback in the decision-making process has been delayed and largely ignored because of changing conditions. That would need to change.
The OODA Loop, which stands for Observe, Orient, Decide and Act, is John Boyd’s way of explaining how we go through the process of reacting to stimulus. A person is involved in all four OODA activities, cf., Wikipedia.
Ismail (2016) argued "The future of AI-powered decision-making needs to be transparent with humans being kept in the loop ... Ultimately the benefit of automated decision making is about scalability and consistency in decision-making." He also noted, "Trust in AI is a big problem, and the only way to create trust is to make the technology transparent in everything it is doing – and that includes how it is used to make automated decisions."
Perhaps for reasons of caution and care it is necessary to keep a person-in-the-loop for automated decision systems for the remainder of this 2020 decade. Keeping someone in the loop means a person is furnished with sufficient relevant information to assess a situation and that a person has a significant role in the decision-making process. Keeping a person in the loop will incur costs. There are obvious advantages of not requiring the most costly component of all in decision-making, the human decision-makers. Also, we do not know the level of expertise that is needed to recognize erroneous machine decisions and we don't know how to help an expert monitor the automated process. Keeping a person in-the-loop may be an impossible quest for automated decision processes.
So being in the loop indicates that a person is aware of a decision situation. Asking to be kept in the loop means you are asking to be kept informed about what is happening in a timely manner so you could intervene. Commanding the decision loop means that person is the final decision maker and manages the process. What involvement do we want? Is/Are the person (s) who is/are in the loop responsible for errors and mistakes?
In general, the role a person should have in a specific decision loop should vary by the complexity and importance of the decision and the decision situation. We, the collective "we", remain responsible for decisions made by machines.
References
Hightower, T.A., "Boyd's O.O.D.A. Loop and How We Use It," at URL https://www.tacticalresponse.com/blogs/library/18649427-boyd-s-o-o-d-a-loop-and-how-we-use-it
Ismail, N., "Artificial intelligence: increasing transparency while keeping humans in the loop," InformationAge, December 12, 2016 at URL
https://www.information-age.com/keeping-humans-loop-ai-world-123463556/
Wikipedia, "OODA loop," https://en.wikipedia.org/wiki/OODA_loop
Last update: 2020-08-29 01:27
Author: Daniel Power
Print this record
Show this as PDF file
You cannot comment on this entry