Last week I wrote about how Watson, IBM's Jeopardy computer, is "smarter" than Google. But there are plenty of things that throw the computer for a loss. In researching my book, Final Jeopardy, which amounts to a "biography" of Watson, I got a feel for how it "thinks." Some examples:
Category: Diplomatic Relations
Clue: Of the four countries in the world that the U.S. does not have diplomatic relations with, the one that's farthest north.
Correct Response: What is North Korea?
Once Watson understands this clue, which is not easy, one of its many algorithms launches two separate hunts. First, it looks for the countries on the outs with the United States (Cuba, Bhutan, North Korea and Iran). Then it checks the geographic coordinates to find which one is the farthest north. On each query, this and scores of other algorithms bring back hundreds of possible responses. To build its confidence in North Korea, Watson must recheck that it is indeed a country, and that it has no diplomatic relations with the United States. (This isn't so easy, because it might come across news that former President Jimmy Carter was on a diplomatic mission there.) On many of these questions, Watson gets the right answer -- but cannot muster enough confidence to bet on it.
Category: Signature Songs
Clue: Fever
Correct Response: Who is Peggy Lee?
This is a hard one for Watson, because the category and the clue combined give it only three words. There is a lot of implicit context that humans bring to this. They know that "signature" in this sense has nothing to do with signing, or autographs. Older players might recall an image of the aging Peggy Lee on the Tonight Show, and Doc Severinsen's band striking up "Fever." These are sense-based memories that Watson lacks. Even if Watson could search an iTunes database for singers of the song, Fever, it would find Michael Buble and Elvis Presley ahead of Peggy Lee.
Category: Anagramed Shakespeare Characters
Clue: Venice disguiser: Air Top
Correct Response: Who is Portia?
This one's a cinch for Watson. Shakespeare categories limits the field of possible answers to several hundred. Also, Watson's great at anagrams. They play to a computer's strengths. So it simply has to find the Shakespeare character whose name includes those six letters. If only all of them could be this easy.
Category: Decorating
Clue: If you're standing, it's the direction you should look to check out the wainscoting.
Correct Response:: What is down?
Almost impossible for Watson. The clue hinges upon the experience of someone with a body standing in a room. Unless Watson comes across lots of references in its data to the physical experience of perceiving wainscoting -- it's likely a miss.
Category: People & Places
Clue: This Mediterranean island shares its name with President Garfield's nickname for his wife
Correct Response: What is Crete?
Easy. First, the syntax of the clue is clear and easy for Watson to understand. The machine has access to documents about First Ladies. And Lucretia Garfield is often referred to as "Crete." After that, it just takes a quick hunt on Watson's part to determine that Crete is indeed a Mediterranean island. (Solving this clue in a Final Jeopardy, incidentally, gave Brad Rutter a big victory in a 2005 Ultimate Tournament of Champions.)
Those are some examples, but I have to add a caveat. Just when you think you have Watson figured out, it'll surprise you with a response -- either stunningly on target or laughably wrong -- and you'll wonder: How did it come up with that?