How smart is IBM Watson?
by Daniel J. Power
Editor, DSSResources.COM
IBM is heavily promoting a cognitive system named Watson in television ads. A major selling point is that "IBM Watson thinks with us to help outthink competitors." This claim positions IBM Watson as a business decision support system. Has Watson advanced to the point where this claim of "help outthink competitors" is true?
My initial reaction is no, probably not.
For many years my research has focused on building, testing and evaluating decision-making and planning process decision support systems. The overriding research goal has been to develop computer software to help people make better decisions and make better plans. There have been some small successes and lessons have been learned, but the task has been challenging. Software that helps managers "outthinking competitors" would be a major advance.
What is Watson?
According to the IBM Watson website, "IBM Watson is a technology platform that uses natural language processing and machine learning to reveal insights from large amounts of unstructured data."
So I am skeptical of IBM's claims and ads, but I want to know for sure where cognitive systems are in terms of thinking capabilities. Are the current ads scripted dialog or unscripted human-computer interactions? Is Watson 2015 smart enough to carry on unscripted conversations with a person? Can Watson help managers compete? Can Watson pass the Turing test? Is IBM Watson as smart as fictional Hal 9000 in 2001: A Space Odyssey?
What is the Turing test? Alan Turing's 1950 paper "Computing Machinery and Intelligence" proposed a test of machine intelligence (www.turing.org.uk/scrapbook/test.html). The Turing test examines a "machine's ability to exhibit intelligent behavior equivalent to, or indistinguishable from, that of a human." Turing proposed that if a human evaluator could not distinguish accurately whether a natural language conversation was with another person or a machine designed to generate human-like responses, then the machine had demonstrated some level of human intelligence.
In early 2011, IBM Watson was introduced on Jeopardy as the "world’s most advanced 'question answering' machine, able to understand a question posed in everyday human elocution — 'natural language,' as computer scientists call it — and respond with a precise, factual answer (NY Times, 6/16/2010)."
In science and TV fiction there have been a number of smart computers. For example, I.R.A.C. nicknamed IRA on the Wonder Woman TV show in the mid-1970s. Wonder Woman develops a special relationship with the artificially intelligent government supercomputer named IRA. On the original Star Trek TV series, Kirk and Spock frequently spoke with the Enterprise's computer, an intelligent assistant. Michael Knight talks to his car, K.I.T.T., on Knight Rider."
Carmel notes question answering (QA) is a specific natural language understanding (NLU) "sub-task in which questions posed in natural language are automatically answered. QA has been extensively studied over the years and many QA systems were developed that can answer a wide range of question types including: facts, lists, definitions, and others." IBM Watson is a question answering system.
Eliza (http://www.manifestation.com/neurotoys/eliza.php3) was an early questioning answering system that imitated a Rogerian psychotherapist. The software used tricks like like string substitution and canned responses based on keywords to interact with a user. Is Watson an enhanced Eliza?
Opusresearch Decision-Makers’ Guide to Enterprise Intelligent Assistants (July 2015) explains that "Enterprise Intelligent Assistants (EIAs) offer human-like, automated services as a natural way for individuals to carry out commerce
through companies’ voice response systems, Web sites and mobile apps." In the report, Opus Research Lead Analyst and Founder Dan Miller defines “Eight Characteristics of Highly Successful EIAs” and identifies 13 firms whose platforms best embody those traits. More than 1,000 companies around the world are using the systems (Miller, 2015).
Tools like Siri and Amazon Echo with Alexa in the consumer market place show the possibility. You can use the cloud-based voice service Alexa to answer questions, play music, read the news, etc. Siri will locate a nearby restaurant or provide turn-by-turn driving directions.
There may even be cognitive software smarter that IBM Watson. Flamm (2014) reported a smart computer named Amelia "can digest an oil-well centrifugal-pump manual in 31 seconds—and give instructions for repairs—and do the job of a call-center operator, a mortgage or insurance agent, even a medical assistant, with virtually no human help. Fluent in 21 languages, she understands implied, not just stated, meanings, and improves her performance by hearing humans deal with questions she can't yet answer. ... the company has been improving Amelia's ability to handle ambiguity—and she asks questions when something isn't clear."
Patrick Henry Winston argues "Watson lacks the ability to connect life experiences to form cohesive thoughts, which is what gives humans their cognitive ability, cf., Mearian, 2011." Hofstadter (Brundage and Bryson, 2014; Herkewitz, 2014) claims Watson is “just a text search algorithm connected to a database, just like Google search. It doesn’t understand what it’s reading.”
So perhaps we are at or near the tipping point to create smart machines and intelligent assistants. What do we want in cognitive computing? Can IBM Watson meet the expanded criteria set of Winston, Hofstadter and others?
Summary of Commercials
IBM Watson TV Commercial, 'Bob Dylan & IBM Watson on Language'
Singer-songwriter Bob Dylan and IBM Watson have a heart-to-heart conversation where Watson shares he read all of Dylan's song lyrics in order to improve on language skills. Watson learned that the main themes Dylan has expressed over the years have been the passing of time as well as fading love. In fact, Watson has never known love, to which Bob suggests the two of them write a song together. Now, listen as IBM Watson attempts at singing, starting with "Doo bee bop" and going from there.
http://www.ispot.tv/ad/AYls/ibm-watson-bob-dylan-and-ibm-watson-on-language
IBM Watson TV Commercial, 'Ken Jennings & IBM Watson on Competition'
Record-holding "Jeopardy!" contestant, Ken Jennings sits down in a library to have a conversation with IBM Watson. He learns that Watson can now recognize people, analyze images, help doctors identify cancer treatments and more in order to help outthink competitors. Although Ken isn't able to one-up Watson by talking about his speaking tour or books he wrote, he does realize the one thing he can help with: sarcasm.
http://www.ispot.tv/ad/AYlC/ibm-watson-ken-jennings-and-ibm-watson-on-competition
IBM Watson TV Commercial, 'Annabelle & IBM Watson on Life Experience'
IBM Watson holds a conversation on life experience with a little girl, Annabelle. She asked for a princess, pony and a cake with pink frosting this year for her seventh birthday. She was too sick to have a cake last year. But, Watson notes the data her doctor shared this year shows that she's healthy. IBM Watson isn't a doctor but he is helping doctors outthink cancer one patient at a time.
http://www.ispot.tv/ad/AYMt/ibm-watson-annabelle-and-ibm-watson-on-life-experience
IBM Watson TV Commercial, 'Cognitive Computing in Healthcare'
Watson, technology that doctors use to keep people healthy, is given a narrative voice in this commercial. He describes all the ways that he collects and interrupts data for Ted, a man who loves to run and eat the occasional ice cream cone. Watson encourages Ted to continue to be the running beast that he is.
http://www.ispot.tv/ad/AVWj/ibm-watson-cognitive-computing-in-healthcare
References
http://www.ibm.com/cognitive/outthink/
"Introducing a new feature of IBM’s Watson: The Debater," May 5, 2014 at URL http://www.kurzweilai.net/introducing-a-new-feature-of-ibms-watson-the-debater
Best, J., "IBM Watson: The inside story of how the Jeopardy-winning supercomputer was born, and what it wants to do next," TechRepublic at URL http://www.techrepublic.com/article/ibm-watson-the-inside-story-of-how-the-jeopardy-winning-supercomputer-was-born-and-what-it-wants-to-do-next/
Brundage, M. and J. Bryson, "Why Watson Is Real Artificial Intelligence," Slate, February 14, 2015 at http://www.slate.com/blogs/future_tense/2014/02/14/watson_is_real_artificial_intelligence_despite_claims_to_the_contrary.html.
Carmel, D., "Did IBM Watson Pass the Turing Test?" February 21, 2014 at URL http://davidcarmel.org/2014/02/21/did-ibm-watson-pass-the-turing-test/
Flamm, M., "Meet the computer that can outsmart IBM's Watson," Crain's, Sept. 29, 2014 at URL http://www.crainsnewyork.com/article/20140929/TECHNOLOGY/140929888/meet-the-computer-that-can-outsmart-ibms-watson .
Herkewitz, W., "Why Watson and Siri Are Not Real AI," Popular Mechanics, February 10, 2014 at URL http://www.popularmechanics.com/science/a3278/why-watson-and-siri-are-not-real-ai-16477207/ .
IBM Watson at URL http://www.ibm.com/smarterplanet/us/en/ibmwatson/what-is-watson.html .
MacIver, M., "IBM Computer Fails the Turing Test But Just Might Pass the Jeopardy Test," Discover, June 28, 2010, at URL http://blogs.discovermagazine.com/sciencenotfiction/2010/06/28/watson-fails-the-turing-test-but-just-might-pass-the-jeopardy-test/#.Viv9CLerSUk
Mearian, L., "IBM's Watson not as smart as you think," Computerworld, April 12, 2011 at URL http://www.computerworld.com/article/2507369/emerging-technology/ibm-s-watson-not-as-smart-as-you-think.html.
Miller, D., "Enterprise Intelligent Assistants are Everywhere!" opusresearch, August 10, 2015 at URL http://opusresearch.net/wordpress/2015/08/10/enterprise-intelligent-asssitants-are-everywhere/
Rhodin, M., "IBM Watson: The Future of Computers with Conversational Skills," May 19, 2014 at http://asmarterplanet.com/blog/2014/05/ibm-watson-future-computers-conversational-skills.html.
Thompson, C., "What Is I.B.M.’s Watson" June 16, 2010 at URL http://www.nytimes.com/2010/06/20/magazine/20Computer-t.html?_r=0
Last update: 2015-12-16 12:20
Author: Daniel Power
Print this record
Show this as PDF file
You cannot comment on this entry
