What IBM's Jeopardy Machine Can Teach Us: Humility

01/20/2011 04:49 pm ET | Updated May 25, 2011
  • Stephen Baker Author of The Numerati and Final Jeopardy, former BusinessWeek tech writer

Ken Jennings and Brad Rutter, Jeopardy's two greatest champions, have brains packed with facts. In one Final Jeopardy, Rutter actually recalled that President James Garfield's wife was named Lucretia. And he deduced from this that the Mediterranean island that shared a nickname with a 19th century First Lady was "Crete."

You might think that IBM's Jeopardy computer, which is taking on Rutter and Jennings next month, would "know" billions of facts. But in truth, Watson is sure of nothing. It treats each statement it comes across -- pillows are soft, water is a liquid, etc. -- as an assertion. In the two- to three-second process of responding to a Jeopardy clue, it builds varying levels of confidence in these apparent facts. Through the course of this painstaking work the Jeopardy computer builds up statistically based beliefs. But it's never 100% sure of anything. It doesn't "know." There can always be exceptions.

An example: David Ferrucci, the chief scientist on the Jeopardy project, was telling me as I researched my book that his team was tempted to teach Watson some basic facts, including the months of the year. Months didn't change. Why shouldn't Watson simply know them? But then they came across a Jeopardy clue about the Muslim "holy month." If Watson had been convinced that there were only 12 possible answers, from January to December, it would have missed "What is Ramadan?"

One reason humans are faster than Watson at answering certain questions is that our minds are full of "facts" that appear to require no further research. It makes thinking easy, so easy in fact, that we're tempted to expand our universe of facts. We build beliefs. They can be about country music, literature, politics or religion. And in our minds, these beliefs often become facts as well: truths that are beyond debate. The accretion of beliefs-behaving-as-facts makes us extremely efficient, but less flexible thinkers. (And this hinders human relationships, democracy and diplomacy, making it difficult to reach agreements with others. But that's for another article.)

Watson, in a sense, is more like the 16th century French philosopher Michel de Montaigne, whose motto was "Que sais-je?" or "What do I know?" Montaigne, who spent his life studying, probably knew as much as anyone in Europe at the time. (They didn't have Jeopardy tournaments back then; so we can't be sure.) But he was ready to accept that someone might change his thinking. "There's a plague on man," he wrote, "the opinion that he knows something."

Watson's cognitive processes are oddly similar to Montaigne's. Of course the two do have their differences. Montaigne wrestled with deep questions, such as the nature of cruelty, sadness, lying and pain. Watson is engineered for a statistical analysis of millions of documents with the goal of delivering correct responses, within two or three seconds, to Jeopardy clues. The supercomputer and the French philosopher are hardly soul mates. But they're both ready to avoid easy mental short-cuts. They both entertain the possibility that they may be wrong, and are inclined (or programmed) to carry out further research. Intellectually, they're humble. In that respect, Watson and Montaigne both have something to teach us.