Watson's Final Jeopardy Blunder In Day 2 Of IBM Challenge (VIDEO)

It was all Watson in Day 2 of the Jeopardy IBM Challenge, until Final Jeopardy anyway. The category was "U.S. Cities" and the clue was: "Its largest airport is named for a World War II hero; its second largest for a World War II battle."

The human competitors Ken Jennings and Brad Rutter both answered correctly with "Chicago" but IBM's supercomputer Watson said "Toronto."

Watson was mocked on Twitter for the mistake, as of course Toronto is a Canadian city. Wrote one Tweeter, @writersblanc, "Oh Watson... 'What is Toronto?' Why not ask 'Who is Arcade Fire?'" Said another, @erickohn, "Watson guessed 'Toronto.' Toronto! The machines don't know all. Yet."

Fortunately, the super-machine didn't wager much and he still won big, finishing with $35,734 to Rutter's $10,400 and Jennings' $4,800.

UPDATE: IBM posted this explanation for the mistake on its "Smarter Planet" blog:

David Ferrucci, the manager of the Watson project at IBM Research, explained during a viewing of the show on Monday morning that several of things probably confused Watson. First, the category names on Jeopardy! are tricky. The answers often do not exactly fit the category. Watson, in his training phase, learned that categories only weakly suggest the kind of answer that is expected, and, therefore, the machine downgrades their significance. The way the language was parsed provided an advantage for the humans and a disadvantage for Watson, as well. "What US city" wasn't in the question. If it had been, Watson would have given US cities much more weight as it searched for the answer. Adding to the confusion for Watson, there are cities named Toronto in the United States and the Toronto in Canada has an American League baseball team. It probably picked up those facts from the written material it has digested. Also, the machine didn't find much evidence to connect either city's airport to World War II. (Chicago was a very close second on Watson's list of possible answers.) So this is just one of those situations that's a snap for a reasonably knowledgeable human but a true brain teaser for the machine.

Scroll down for video of Watson's Final Jeopardy blunder. And catch up on Day 1 of the competition here.

WATCH:

Comments

[InofTouch]

10:45 AM on 02/25/2011

It official, Toronto, New York :D

[En]

04:33 PM on 02/17/2011

Begining of your news quote: The human competitors Ken Jennings and Brad Rutter both answered correctly with "Chicago" but IBM's supercomputer Watson said "Toronto."

Watson was mocked on Twitter for the mistake, as of course Toronto is a Canadian city. end quote

What makes you think that "Toronto" is NOT a city in the United States. Have you ever heard of "Toronto, California", or "Toronto, Kansas" or "Toronto, Ohio" ????? These United States cities have been in the United States for many years. You don't know that???? "Watson" knew it but you HUMANS think that "Toronto" is a city that is ONLY in Canada?????

[Joseph Watters]

11:58 PM on 02/16/2011

Possible statement from the future: "Here's a computer that can do everything a human can do. Well, except for use creativity, judgment, wisdom, compassion, valor, humor, empathy, intuition, or conscience. And well, yes, it can't make a nice cup of tea, or comfort you in times of woe."

At a certain ultimate level, all of the atrocities in human history came about because information was detached from the aforementioned qualities. Watson or its successors will never have those qualities. Thus, they will simply become the next form of atrocity.

No doubt the capabilities and potential of Watson are appealing to technocratic organizations that already see humans as replaceable, commoditized, machine-like parts. Fortunately these organizations are the minority numerically. Unfortunately, they control a disproportionate amount of the wealth and influence in the world. Rejecting and dismantling the technocratic organizations will guarantee that Watson and its ilk are unable to facilitate atrocities. Resistance in that way is not futile.

[Manny Zamudio]

07:32 AM on 02/17/2011

I'm sorry, but not only did you state the obvious about Watson being a computer, but your entire statement made me believe that you're a Luddite. Granted, humans have had emotion influence our judgments and our actions, but I think the real atrocity was humanity without emotion, without judgment, without any feeling whatsoever.

[Jon Mendoza]

10:08 PM on 02/16/2011

Next, Watson will be on "Are you smarter than a 5th grader?" I want to see kids cry and stick gum on it.

[aero56]

04:34 PM on 02/16/2011

I thought maybe Sarah Palin snuck in that answer.

[Justin the believer]

01:43 PM on 02/16/2011

It is not wise to mock the super computers... That is how Skynet became fully operational... I'll be back!

[Jon Mendoza]

10:05 PM on 02/16/2011

We're already in the Matrix, the machines just put Watson there to mock us.

[PedroLeon]

12:59 PM on 02/16/2011

Hal effed up.

[TedEjr]

12:35 PM on 02/16/2011

Well, this does confirm that a computer is only as good as the instructions that it is programmed with.

But, there is more here than just a goof in final jeopardy. This "machine" is, what, ten racks, I believe (each roughly eight feet tall) full of equipment, in a super cooled room. Eniac, 1946, was also a room full of machines, and it did not have the computing power of today's smart phones.

What we are witnessing here, along with a cute promotional episode, is the dawn of a new era in technology. In 60 years, the technology advance will probably be amazing.

[Chas37]

11:52 AM on 02/16/2011

You could see both guys trying to hit their buttons on many of the questions so it may have come down to the fact the electric pulse of Watson was just faster than the thumb.and not necessarily smarter.

[mcreel]

11:15 AM on 02/16/2011

Well Watson, V is correct because it is the roman numeral for 5...
Now let's see how much you wagered

[PedroLeon]

01:00 PM on 02/16/2011

SucK it Trebek!

[mcreel]

01:06 PM on 02/16/2011

Thanks,
I was wondering if anyone would ever get it. F/F

[mcreel]

01:21 PM on 02/16/2011

Glad at least one person got it.

[austinfuller]

11:05 AM on 02/16/2011

While the entertainment value is questionable, Watson is an achievement in the world of computer science IMO. Before these specials aired I saw the PBS Nova on the 3-year development and testing of this thing. The idea is designing a computer system that can not only better understand the relationship between words in language, but also be able to find similarities in language from a huge amount of data.

Jeopardy! was a good testing ground because of the category-clue language relationship and the challenge of writing a program that could interpret such linguistically complex clues and find a probable answer. The task was sort-of teaching a computer not only how to "think" but how to "learn" similar to human intelligence.

But yeah, would rather just watch Jennings against humans.

[editor]

10:46 AM on 02/16/2011

Bad programming. "I think as usual the problem can only be attributed to human error" - HAL 9000

[Liberal2009]

10:21 AM on 02/16/2011

Manifest Destiny? Terminator? Anyone?

[cdiasmd]

10:24 AM on 02/16/2011

I'm going w/ HAL 9000

[cdiasmd]

09:58 AM on 02/16/2011

"What is a Quince" - Gloria Clemente

I'm waiting for Watson to announce the presence of a black monolith and try to kill Jennings and Rutter.

[WDavid]

09:45 AM on 02/16/2011

I agree the show was a little boring with the computer being so fast to click. What I did find interesting was the three possible answers the computer had come up with. It also makes you marvel at the human computer (brain). I mean 'Watson' was a modern computer the size of a large room and its only task was to play Jeopardy. Artificial Intelligence is quite a ways off.

I didn't mind so much that it was an IBM commercial, at least it was an American company with the goods.