What are ethical issues associated with building and using DSS?
by Daniel J. Power
Editor, DSSResources.com
Decision support builders and users face ethical challenges and issues. Some people think that building and using a computerized decision support capability is ethically neutral. That view is wrong. People are faced with ethical choices when dealing with computerized decision support that we are only beginning to consider and evaluate. Decision support project proposals can raise ethical and value issues for systems builders. Choosing to use or not use a computerized decision support system may itself be an ethical issue. We need to identify plausible critical ethical incidents that may be faced by DSS builders, data scientists, and other users.
So what situations might occur? The following ten incidents are a starting point:
Scenario 1: A builder of an Executive Information/Business Intelligence System chooses not to include a key metric because the data is hard to capture and display. Eventually that metric, for example the weight of a prototype airplane, becomes a critical flaw that leads to major cost overruns.
Scenario 2: A sponsor proposes combining sales and credit card data and a DSS builder becomes concerned the privacy rights of past customers will be in jeopardy. The sponsor is a powerful figure in the company who doesn't like dissent.
Scenario 3: A builder realizes the quality of data for a proposed data-driven DSS is flawed and inaccurate and still proceeds to build the system. The system is never really used because of complaints of poor data quality.
Scenario 4: A builder fails to validate a forecast model and the system users report large inventory problems. The company takes a major write-down on obsolete inventory.
Scenario 5: A manager/user of a data-driven DSS notices a sales problem in a store and drills down into the underlying data and sees a large transaction by his wife. The manager confronts his wife with the information he found using the information system.
Scenario 6: A manager fails to use an investment management and control system in a timely manner and a subordinate makes a large, unauthorized trade. The trade is ill-advised and significant losses result.
Scenario 7: The knowledge base for a knowledge-driven DSS is obviously out of date and no one acts to fix the problem. The recommendations become increasingly error prone and managers start ignoring the results from the DSS.
Scenario 8: Conversations that occur during use of a communications-driven DSS are recorded automatically and users are unaware that is happening. Some negative personal comments are made and a senior manager requests the recording.
Scenario 9: A manager extracts data from a decision support data base and transfers the data in an email to a home computer. The home computer is subsequently stolen.
Scenario 10: Government intelligence analysts are encouraged to use analytical tools to look at the content of email messages based on their own discretion.
What would you do in each situation? Why would you take that action?
In many of these situations we encounter an ethics slippery slope. Initially the situation seems clear cut, but sometimes we go from choosing an action that is questionable to choosing one that is unethical. We slip down the slope.
From a decision support builder perspective. Principles and values play an important role in making many significant organizational decisions. When DSS are constructed, developers make assumptions that can have ethical impacts on user choices. Also, some decisions are considered so value-laden that many people would be uncomfortable with developing a Decision Support System to assist a decision-maker. One cannot specify all of the ethical issues that might be relevant to a specific decision support proposal, but once a proposal reaches the feasibility stage, the project sponsor needs to specifically address the ethical issues associated with the project. Also, during development builders need to be sensitive to how the representations like charts and tables designed to present information impact the decision maker.
Privacy concerns are also easy to ignore during the evaluation of a Decision Support = proposal. In many societies, people expect that certain personal and behavioral information about them will be kept private. This information belongs to the person and doesn’t belong to a company, the public, or the government. Managers need to insure that data used in Decision Support Systems doesn’t infringe on the privacy rights of individuals. The exact extent of privacy rights for employees, customers, and other data providers is not always clearly defined. In general, unless there is a clearly compelling reason to risk violating an individual’s privacy, the "fence" to protect privacy of data should be higher and larger than any minimum requirements.
From a user perspective. Managers who have access to an appropriate DSS may have a fiduciary duty to use a DSS, especially if the system is demonstrated to improve decision quality. A fiduciary is expected to provide the "highest standard of care" to the person to whom they owe the duty. For example, what if an investment advisor or a bank trust officer is not using a DSS? Have they shirked their fiduciary duty? Investment fiduciaries include investment advisors, trustees, and investment committee members. When does using a DSS become a professional requirement to do a job well? Is using a DSS a "prudent practice" in some situations"? Are there situations where not using a DSS/IS should be considered malpractice?
Misuse of decision support data is difficult to monitor. For example, a Police Officer may use a law enforcement database to find out information about a neighbor or friend. Some DSS queries need to be logged so abuse can be monitored. Also, data quality is a problem in some organizations for both users and builders. When is poor data quality so serious that a data-driven DSS should not be built or used? Also, users may make unauthorized file transfers of DSS data that is subsequently misused. An ungoing issue for users and builders is what information will be accessible and by whom and when? Written policies must outline access and system use permissions and restrictions to reduce ethical lapses and prevent poor judgment from leading to additional problems.
So the following potential analytics/decision support ethical issues require more thought: 1) data quality assurance, 2) hidden data capture, 3) propagating data errors, 4) ongoing use of an obsolete decision support, 5) data mashups and data linking, 6) combining internal and external data sources, 7) inappropriate use of customer profiles/data, 8) legal liability issues from failing to use or from misuse of a decision support capability, 9) data/key metrics exclusion, 10) analytics/decision support model validation, 11)unauthorized data transfers, 12) lack of policies or poor policy enforcement, and 13) invasion of personal privacy. Company policies should guide the behavior of managers and builders on these topics.
Incorporating Artificial Intelligence (AI) technologies like case-based reasoning and Machine Learning (ML) can incorporate the biases and limits of its human developers. Diversity of developers, review boards and application of a strong AI ethics framework will promote and help ensure ethical use of AI in decision support.
We should want to build a high fence to ensure ethical use and construction of analytical decision support capabilities and to do so we need to discuss the subtleties of a wide variety of ethical situations that builders and users may encounter. When in doubt about an ethical use for a DSS, the need to use decision support, or the consequences of poor design decisions on the behavior of decision makers, do not ignore the issue, rather ask, consult, and discuss with others. Ignoring ethical issues associated with building and using computerized decision support is not an option.
References
Lynch, M., "Ethical Issues in Electronic Information Systems, 1994, URL http://www.colorado.edu/geography/gcraft/notes/ethics/ethics_f.html
Power, D. J., Decision Support Systems: Concepts and Resources for Managers, Westport, CT: Greenwood/Quorum Books, 2002.
Please cite as:
Power, D.J., "What are ethical issues associated with building and using DSS?" DSS News, Vol. 9, No. 5, March 9, 2008; last updated April 17, 2016.
Last update: 2019-05-10 01:24
Author: Daniel Power
Print this record
Show this as PDF file
You cannot comment on this entry
