What is the role of people in machine learning?
by Daniel J. Power
Editor, DSSResources.com
Many tools exist for modeling data using machine learning (ML). The tools have been automated and the desire for "easy to use" software is now a reality for complex statistical analysis. Machine learning (ML) is still evolving, but in general computer programs access data and with or without human supervision a model is developed and with additional data the "learning" and model refinement continues. The key idea behind ML is that it is possible to create algorithms that learn from and make predictions using data (Furbish, 2018). Machine learning algorithms use historical data as input to predict new output values.
Multiple steps in the ML process have been automated, but people continue to have important roles including: 1) problem finding and problem framing, 2) determining the question(s) to answer, 3) identifying relevant data potentially associated with answering the question(s), 4) explaining the domain, 5) preparing data; 6) model testing; 7) providing feedback, and 8) interpreting and understanding the results of machine learning analysis. Let's examine what tasks people can and should do to control and improve ML:
1) Problem finding means discovering a problem that needs to be resolved. Problem framing is the process of describing and interpreting a problem to arrive at a problem statement. Set realistic expectations for the ML project.
2) Identifying relevant questions to answer -- If you don't ask the right questions, your data won't give you relevant answers.
3) Data Identification -- Data is the fuel that drives high-scale innovation with AI. Ironically, many organizations struggle to use their data effectively because of the overwhelming number of data sources that are available, and no clear way to identify the most trusted sources of data.
4) Answering Domain Related Questions -- Help Technical specialists understand the context and the mean of data entities.
5) Data Preparation -- Every machine learning algorithm works differently and has different data requirements. Get the right data, take your time. For example, some algorithms require numeric features and some do not.
6) Interactive Model Testing -- Some ML software enables managers and other "everyday users to interactively explore the model space and drive the system toward an intended behavior, reducing the need for supervision by practitioners. (Amershi et al., 2014, p. 106)"
7) Providing Feedback -- During training or at various planned points in model development, people often provide positive and negative feedback about outomes and performance.
8) Understanding results -- In recent few years, some automated and interactive machine learning tools have significantly increased predictive power, but model complexity has increased. A predictive model must demonstrate both accuracy and fast results, but people must also trust the answers and results from the model. And in a regulated industry, managers must justify the model to a regulator. ML software must be understood and explainable by domain experts. People provide explanations of the why of the results. Managers should expect transparency not black boxes from Machine Learning algorithms.
Data scientists may perform some of these tasks, data engineers and software engineers may perform others, cf., Kotecki (2020). Domain experts or subject-matter experts also often are important participants. Domain experts have special knowledge or skills in a particular area and they understand the practical implications of the data.
People must take responsibility for ML use and results. People should look for bias, monitor the fairness of the results, and work to determine the validity of the model. A human analyst selects data for a learning system and then observes and interprets the outputs of the system to make choices for any subsequent iterations. People should make sure the human consumers of ML are respected both during the development process and after as the results are used. People should remain in the decision loop (Power, July 5, 2020) and participate in the algorithm development process.
References
Amershi, S., Maya Cakmak, W. Bradley Knox, Todd Kulesza, "Power to the People:
The Role of Humans in Interactive Machine Learning," AI Magazine, Winter 2014, pp. 105-120 at URL https://aaai.org/ojs/index.php/aimagazine/article/view/2513/2456 DOI: https://doi.org/10.1609/aimag.v35i4.2513
Furbush, J., "Machine learning: A quick and simple definition," O'Reilly, May 3, 2018 at URL https://www.oreilly.com/content/machine-learning-a-quick-and-simple-definition/
Kotecki, J., "Roles on a Machine Learning Project," Medium, January 16, 2020 at URL https://medium.com/machine-learning-in-practice/roles-on-a-machine-learning-project-216903a6dc12
Power, D., "What does it mean to keep a person in the decision loop?" Decision Support News, Vol. 21, No. 14, July 5, 2020 at URL http://dssresources.com/newsletters/526.php
https://www.datarobot.com/platform/what-is-automated-machine-learning/
Last update: 2021-01-13 10:27
Author: Daniel Power
Print this record
Show this as PDF file
You cannot comment on this entry
