What is the telephone test?
by Dan Power
Some semi-structured decisions can be supported using Artificial Intelligence technologies delivered as part of a knowledge-driven DSS. If a business decision problem cannot be solved and supported using quantitative methods, it may be appropriate to try an expert system solution. Prior to proposing a DSS project involving expert system technologies, the decision domain should first be evaluated for appropriateness. How do we evaluate the appropriateness of a knowledge-driven DSS project? One approach is the telephone test which helps us quickly evaluate the appropriateness and feasibility of building such a DSS.
What is the "telephone test"? According to Waterman (1986) and others, to apply the test, we ask "Can a domain expert solve the problem and support decision-making using a telephone exchange with a user decision-maker?" Sometimes it is even helpful to ask the domain expert to interact with a potential user of a new DSS over the telephone and record the interaction that occurs. The domain expert should be told to ask structured rather than open-ended questions.
Structured, close-ended or fixed response questions offer the respondent a fixed set of answers. The question answers may be Yes, No, Unsure or Not Relevant, True/False, Strongly Agree to Strongly Disagree responses, or a multiple choice menu of answers. The structure and sequence of questions and the final recommendation are all evaluated to determine the appropriateness of building a knowledge-driven DSS. The expert is demonstrating that relevant knowledge can be structured for a potential user of the DSS.
If after the test, we answer YES the telephone exchange works, then a knowledge-driven DSS can be developed to support a decision-maker. On the other hand, if the decision-maker is unable to describe the problem verbally, or if the expert is unable to consistently recommend a reasonable solution, then development of a knowledge-driven DSS will probably be unsatisfactory (cf., Waterman, 1986; Power, 2002).
Using a restricted channel like a telephone exchange limits what information a human expert receives and insures the expert is able to adequately describe his/her problem solving process with structured questions.
Kernighan and Plauger (1978) describe a similar telephone test for program code readability. Their rule is "If someone can understand your code when read aloud over the telephone, it's clear enough. If not, then it needs rewriting." The domain expert can develop questions and responses for the decision problem in advance and then read them in an appropriate sequence. We are trying to determine if the targeted user understands the questions and if the users responses lead to "good" recommendations.
A telephone test is "quick and dirty" prototyping without writing any code. It is important to try the test with multiple interrelated decision problems and analyze the transcribed exchange between the expert and the user/decision maker who will seek advice. "Quick and dirty" prototyping like this is a simple way to stimulate communication about a novel DSS without committing extensive resources and then perhaps having to admit a knowledge-driven DSS was inappropriate.
Jones, D.D. and J.R. Barrett (1989) Building expert systems. In J.R. Barrett and D.D. Jones (eds). Knowledge Engineering in Agriculture. ASAE Monograph No. 8, ASAE, St. Joseph, MI.
Kernighan, B. W. and P. J. Plauger (1978) The Elements of Programming Style (2nd Ed.), McGraw Hill, New York.
Power, D. J. (2002) Decision support systems: Concepts and resources for managers. Westport, CT: Greenwood/Quorum Books.
Waterman, D.A. (1986) A guide to expert systems. Addison-Wesley Publishing Co., Inc., Reading, MA.
Last update: 2011-08-21 02:21
Author: Daniel Power
You cannot comment on this entry