`Jeopardy!': Computer CRUSHES Competition

NEW YORK — The computer brained its human competition in Game 1 of the Man vs. Machine competition on "Jeopardy!"

On the 30-question game board, veteran "Jeopardy!" champs Ken Jennings and Brad Rutter managed only five correct responses between them during the Double Jeopardy round that aired Tuesday. They ended the first game of the two-game face-off with paltry earnings of $4,800 and $10,400 respectively.

Watson, their IBM supercomputer nemesis, emerged from the Final Jeopardy round with $35,734.

Tuesday's competition began with Jennings (who has the longest "Jeopardy!" winning streak at 74 games) making the first choice. But Watson jumped in with the correct response: What is leprosy?

He followed that with bang-on responses Franz Liszt, dengue fever, violin, Rachmaninoff and albinism, then landed on a Daily Double in the "Cambridge" category.

"I'll wager $6,435," Watson (named for IBM founder Thomas J. Watson) said in his pleasant electronic voice.

"I won't ask," said host Alex Trebek, wondering with everybody else where that figure came from.

But Watson knew what he was doing. Sir Christopher Wren was the correct response, and Watson's total vaulted to $21,035 as the humans stood by helplessly.

Watson blew his next response. But so did both his opponents. He guessed Picasso. Jennings guessed Cubism. Rutter guessed Impressionism. (Correct question: What is modern art?)

Back to Watson, who soon hit the game's second Daily Double. But even when he was only 32 percent sure (you could see his precise level of certainty displayed on the screen), Watson correctly guessed Baghdad as the city from whose national museum the ancient Lion of Nimrud ivory relief went missing (along with "a lot of other stuff") in 2003. Watson added $1,246 to his stash.

He even correctly identified the Church Lady character from "Saturday Night Live."

One answer stumped everyone: "A Titian portrait of this Spanish king was stolen at gunpoint from an Argentine museum in 1987." (Correct response: Philip.) Jennings shook his head. Rutter wrenched his face. Watson, as usual, seemed unfazed.

Even when he bungled Final Jeopardy, Watson (with his 10 offstage racks of computer servers) remained poised.

The answer: "Its largest airport is named for a World War II hero; its second largest, for a World War II battle."

Both Jennings and Rutter knew the right response was Chicago.

Watson guessed doubtfully, "What is Toronto?????" It didn't matter. He had shrewdly wagered only $947.

The trio will return on Wednesday, when their second game is aired. The overall winner will collect $1 million.

The bouts were taped at the IBM research center in Yorktown Heights, N.Y., last month.

___

Online:

http://jeopardy.com

(This version CORRECTS final totals of Jennings and Rutter to $4,800 and $10,400.)

Comments

[Barbara Kelly]

08:57 AM on 02/17/2011

Given the number of people whose background input fed Watson's data base, there is no way to compare the 'winning' by the machine over the individuals who competed.

[RedDogBear]

12:55 PM on 02/17/2011

The team was fairly small. The input was acquired by taking existing sources such as Wkipedia, Encyclopedia Britanica, IMDB, and feeding them into Watson.

[aristippe]

01:11 AM on 02/17/2011

I would just like to know if Watson used voice recognition as an input?

[kapalabhati]

06:40 AM on 02/17/2011

Yes.

[RedDogBear]

11:56 AM on 02/17/2011

No, I don't think it does. It gets the questions as a text message. It doesn't know about other contestant's wrong answers, which is why it made one dumb mistake once giving the same wrong answer as the first human who tried to answer. It does process correct answers that it got wrong but I'm almost certain those are also fed to it as text.

[aristippe]

01:05 AM on 02/17/2011

But will it blend?

[castleb]

09:10 PM on 02/16/2011

Knowledge is not Wisdom. That's the difference between computer potential and human potential.

[rotorhead1871]

10:43 PM on 02/16/2011

but knowledge is POWER!!!!

[glentrs]

04:52 AM on 02/17/2011

nope in this case power is knowledge, (its plugged into a wall)

[castleb]

09:05 PM on 02/16/2011

Maybe Watson should run for President.

[GEM-592]

08:50 PM on 02/16/2011

Can we replace Alex with him? Now that would be a triumph.

[GEM-592]

08:49 PM on 02/16/2011

... and he beats Alex in personality hands down.

[Regats]

10:02 PM on 02/16/2011

Geoff Peterson beats beats Alex in personalit­y hands down.

[laoshi]

08:37 PM on 02/16/2011

Snore.....

Do we want to go to the ball park and watch a pitching machine strikeout every batter? Do we want to listen to a computer play Liszt "better" than a human?

Could this computer aid a doctor in providing a diagnosis? Probably. That would be a better use of the bits.

Computers are good at trivia games. Let's move on to something interesting.

[glentrs]

04:54 AM on 02/17/2011

question 2, its potential in medicine, hmmmmm with the the perfection of nanobots it could well do better than a human in discovery.

[RedDogBear]

12:03 PM on 02/17/2011

This wasn't about being "good at trivia games". It was about making a computer that can process natural language and what AI researchers call common sense reasoning, understanding all the incredibly diverse facts that form the background for human knowledge, the kind of thing that most children know but until Watson no computer could deal with.

Solving that problem (not that Watson has completely solved it but this is a very significant step forward) makes all kinds of new real world applications possible for information retrieval, robotics, new types of human-computer interfaces, etc. Imagine if you could just ask your computer questions like "who was the 38th president of the US" and get the answer rather than doing a google search for "US presidents 38" and then wading through the articles for the answer.

The focus on Jeopardy was just a solid research technique: don't define some artificial task but pick something from the real world that gives you real world constraints. Its a way that good researchers make sure they don't tune the task to the solution. Also, I'm sure it helped the lead scientist convince the marketing suits to give him more money for the project.

[RationalVoice]

08:25 PM on 02/16/2011

Before people gripe about Watson or dismiss this as an IBM PR stunt, please know something about how Watson works. He doesn't search the internet for answers. He buzzes in with the same buzzer as Ken and Brad. Yes he has stored terabytes of data, but so have Ken and Brad--and you for that matter. He may have a faster reaction time than a human, but he has one huge disadvantage: He doesn't naturally understand language. The human brain evolved over millions of years to have the capacity for language. We take for granted that when we read a Jeopardy clue, we instantly understand its meaning and can guess at its answer. We can parse through the 30 or so words and find the most important ones. That is not an easy computation. This is bigger than playing chess. You can find the best move at any point in a chess game, if you have enough computer power. To understand language, you need to invent knew algorithms and programs. Watson isn't perfect, but he's an impressive step towards computers understanding language.

[glentrs]

04:59 AM on 02/17/2011

one other attribute you seem to have missed, the emotional quality of voice, typed words as in a chat can be misunderstood due to lack of punctuation or smileys, its pure binary input for a comp, so far anyway, havnt tried that dragon program yet tho.

[Philm35]

01:02 PM on 02/17/2011

I believe that Watson did have one significant advantage over the humans. Watson was fed the questions via a text file, which a computer of that power could read in nanoseconds. The humans had to have the questions read to them by Alex, at a normal speaking pace. Did Watson win big because it could buzz in faster? And could Watson buzz in faster because it was able to input the questions thousands of times faster than the humans could?

I think the contest would have been fairer, if Watson was forced to use speech recognition to input the questions, just as the humans did. The technology is certainly available. It exists on my iPhone.

[brt 177]

08:20 PM on 02/17/2011

I definitely agree. The process of the human hand and arm actually holding and pressing the buzzer puts a significant delay on the humans as well; and yes, thousandths of a second are significant for those who would throw that back at me. It is definitely a PR stunt by IBM. The only real thing IBM has going for it right now is scientific computing advancements; super computers, taking the first real time picture of an atom, funding quantum computer research, etc.

[cynicalmatt]

08:06 PM on 02/16/2011

Of course the computer is winning. Number one, it's a computer. Two, it searches the Internet for answers and three it has no physical appendage to cause a delay in clicking the buzzer. The only cool achievement is that it's able to interpret the answers into the correct questions.

[whatwasthat]

08:27 PM on 02/16/2011

one, no one is allowed to go on the internet and google stuff. even watson. They use complex algorithms etc etc - if you had watched the first episode you would have known.

[glentrs]

05:02 AM on 02/17/2011

really? one has to wonder how all that info got into it, so much miniscule stuff and history, must have taken thousands of people programming it. i wonder, its amazing really.

[Regats]

05:52 PM on 02/16/2011

The computer winning? That was a foregone conclusion.
No human on earth can store the vast amounts of information and
recall it as quickly and accurately as a well programmed computer.

Here's what I'd like to see -
Just two contestants:
China's best computer vs. America's best computer.

Place your bets.

[SuperRyan]

11:59 PM on 02/16/2011

It's not as simple as you would think.
Storing the information is easy. Knowing where the answer is is the hard part.
Computers don't think like humans. They execute commands, and have their own language that doesn't really translate to english.
The answers on Jeopardy would need to be translated into a computer command, then searched and communicated back in english.

[RedDogBear]

12:08 PM on 02/17/2011

As SperRyan said it wasn't at all a foregone conclusion. Understanding things like puns and other play on words, common cultural facts, conventions (e.g., one way it messed up earlier was that it thought a question that referenced the year "72" could be 1872) If you watch the Nova special the initial version of Watson was terrible. It got about 10% of the questions correct. They had to work on it and tune it for quite some time to get it to the expert Jeopardy level.

[RedLeg2]

05:15 PM on 02/16/2011

Please watch last weeks Nova first before posting something foolish. This is quite an accomplishment for IBM and they're R&D team deserve some accolades. Computers are the "dumbest" invention ever created by man. They are only capable of either "yes" or "no". To work within those parameters to get a computer to win at a game based on trivia and the english language is remarkable. Think of the real world applications for Watson.

[LaplacesDemon]

07:12 PM on 02/16/2011

Red I take it you aren't a computer programer. Computers are very capable of answering more than yes or no. What you are erroneously refering to is "binary language". A bit (there are 8 bits to a byte) is either on or off. But add up those bits and they make letters and numbers. Add those together and you can create anything. For all we know the entire universe is nothing but a computer simulation. And actually the dumbest things known to man are called "Republicans".

[SuperRyan]

12:03 AM on 02/17/2011

I'm just trying to figure out what human inventions are capabe of even understanding ones and zeroes (things with a computer device doesn't count).

[RedLeg2]

02:54 AM on 02/17/2011

Actually, I was a failed "comp sci" major. That was a direct quote from one of my professors. Yes and no was referring to binary. yes or no, on or off, 1 or 0. That is all a computer does. Yes, when compounded, computers can do amazing things. But I stand by my statement.

[glentrs]

05:05 AM on 02/17/2011

oh I am, i wish I knew more, nova eh? god i wish OMNI was still published.

[Marmashi]

04:24 PM on 02/16/2011

This is just a big ad for IBM. It is not fair that Watson gets the info digitally while the two REAL men have to wait until the reading is done orally to hit the button. If they try to hit their buzzers before the answer is read they get a penalty of some few seconds before their buzzer will work again. Meanwhile the computer has all the possibilities weighted and probabilities analyzed before the two guys can react.
I watched the first show and now I am on strike. This stinks as a game show. It is making 2 geniuses look like dumbos for NO GOOD REASON. IBM's ego is not a good reason.

[RedDogBear]

04:30 PM on 02/16/2011

Its your impression that this makes them "look like dumbos". I certainly don't think that. The fact that a computer can do as well (or almost as well) as a human on a certain task doesn't IMO change the value of the human at all. I don't even understand why anyone would think that. When Deep Thought beat the world chess champion (think it was Kasparov) that didn't mean Kasparov was suddenly less of a chess player.

[RedDogBear]

04:34 PM on 02/16/2011

I've done research at universities and corporations in artificial intelligence and I can tell you this is a lot more than just an ad for IBM. Its a major accomplishment in AI research. Although of course it also IS an ad for IBM. IBM invested a lot of money in this and from my experience I can tell you one of the justifications the scientists used to sell it was that it would be great press for the company. Everyone has to pay the bills one way or another.

[rlia81]

04:01 PM on 02/16/2011

There were definite flaws in this competition. (I am speaking in the past tense since the shows were taped prior to airing.) First, Watson was able to click in at a machine-only lightning speed that even Ken Jennnings could not match most of the time. Second, and most important, Watson was given a choice of three answers on many questions while his two comptetitors had to search their brains for the answers. The men should have been given the same advantage of the three option answers. What this "competition" is boiling down to is a huge commercial for IBM, a celebration of the computer's amazing capabilities, and very little in the way of a game. While I remain a long time Jeopardy fan, I have found this competition boring.

[Toonguy]

04:20 PM on 02/16/2011

The only flaws were your interpretation of the facts.

1 - Watson was designed to not have any advantage over the other contestants when "ringing in."
2 - What you interpreted as the "choice of answers" were actually a window into Watson's thinking process. We were being shown what Watson considered the top three answers and by what percentage Watson determined the answer was the likely one. No one was feeding those answers to Watson.

This was all explained on a recent episode of Nova and reiterated on the first night of the competition.

[RedDogBear]

04:27 PM on 02/16/2011

In case anyone is interested here is a link to the Nova episode: http://www.pbs.org/wgbh/nova/tech/smartest-machine-on-earth.html

[MCJanes]

04:20 PM on 02/16/2011

No, he wasn't given three answers to choose from...the three answers you saw on the screen were the three answers he came up with, and his "confidence" in each answer.

I agree with you on the button clicking, there's no way they could've beaten Watson in the speed department. To be fair though, Ken and Brad were able to read the answers in addition to hearing them, whereas Watson could only do the latter.

[Regats]

10:24 PM on 02/16/2011

Any competent Jeopardy contestant quickly reads the clue before waiting for Alex to complete the question verbally - this gives the contestant a little extra time to come up with an answer. Watson was given the questions in text form at the same time (I'm guessing) A brief lag to equal human reading speed would have been fair.
Watson was supposed to be programmed to have about the same click response time as a human, but I don't know how they could come up with this.
(Maybe take the average of twenty clicks from Brad and Ken?)

[RepugsHateFreedom]

02:36 PM on 02/16/2011

Now we have a computer smarter than any re-pube-lican or teabagger.

[J0E1]

02:58 PM on 02/16/2011

But not democrat right?  You are sad.

[RepugsHateFreedom]

12:38 PM on 02/17/2011

Not sad, just realistic. The record of failure, lies, death, anger, misery, and ignorance by all re-pube-licans is unmatched in human history. Probably unmatched in the history of any sentient beings in the universe.