What's Next For Watson, IBM's Jeopardy Champ?

SAN FRANCISCO — Fresh off its shellacking of two human champions of the "Jeopardy!" television show, a computer program developed by IBM Corp. will soon get a workout in two hospitals that have signed up to test the technology.

The agreements with the Columbia University Medical Center and the University of Maryland School of Medicine will be the program's first real-world tests outside of the trivia game show and IBM's laboratories.

Watson, as IBM has dubbed the program, represents a breakthrough in the ability of computers to understand human language and scour massive databases to supply the most likely answer to questions. It's not always right; some of its errors in its "Jeopardy!" debut this week were amusingly off-base.

But it holds promise for doctors and hedge fund managers and other industries that need to sift through large amounts of data to answer questions.

Eliot Siegel, a professor at the Maryland university's medical school, said other artificial intelligence programs for hospitals have been slower and more limited in their responses than Watson promises to be. They have also been largely limited by a physician's knowledge of a particular symptom or disease.

"In a busy medical practice, if you want help from the computer, you really don't have time to manually input all that information," he said.

Siegel says Watson could prove valuable one day in helping diagnose patients by scouring journals and other medical literature that physicians often don't have time to keep up with.

Yet the skills Watson showed in easily winning the three-day televised "Jeopardy!" tournament Wednesday also suggests shortcomings that have long perplexed artificial intelligence researchers and which IBM's researchers will have to fix before the software can be used on patients.

"What you want is a system that understands you're not playing a quiz game in medicine and there's not one answer you're looking for," Siegel said.

"In playing 'Jeopardy!', there is one correct answer. The challenge we have in medicine is we have multiple diagnoses and the information is sometimes true and sometimes not true and sometimes conflicting. The Watson team is going to need to make the transition to an environment in which it comes up with multiple hypotheses – it will be a really interesting challenge for the team to be able to do that."

Siegel said it would likely be at least two years before Watson will be used on patients at his hospital. It will take that much time to train the program to understand electronic medical records, feed it information from medical literature, and test whether what it's learned leads to accurate analyses of patient symptoms.

He said he wasn't bothered by Watson's on-screen blunders; even highly trained medical professionals make dumb mistakes.

"I will take an assistant that is that fast and that powerful and that tireless any time," he said. "This is going to be something that 10 years from now will be a completely accepted way that we wind up practicing."

Watson could be a boon for IBM, the world's biggest computer services company, if it works as promised in the real world. IBM makes a mint on "analytics" software that helps companies mine their data and predict future trends, such as shopping patterns at a retailer, for instance.

Watson currently runs on 10 racks of IBM servers, but computing power generally doubles every two years so the amount of hardware needed to run the same program will soon be significantly less. And the program can be tweaked to run slower, or scan less information, to make the program easier to deploy in a business setting.

IBM hasn't disclosed prices for the commercial sale of Watson, nor details of the financial arrangements with the hospitals.

Comments

[iCareToo]

08:56 PM on 03/12/2011

"I can be as good as this smarter person", "Want to be smart" is today's problem. First not everyone should be "smarter", second the smarter usually very fool in area they are no good at. My 2 cents.

[Robert SF]

03:11 PM on 03/08/2011

I don't know exactly what's next for Watson, but you don't need to be Jeopardy-smart to answer questions over a phone about a narrow topic, so I expect Watson to replace call centers within five years.

[tj3usa]

07:10 AM on 03/08/2011

WATSON NATURAL LANGUAGE COMPUTER WITH NUANCE

As a health care insurance adjuster: "Oxygen? Oxygen! You don't need no stinking oxygen". Natural language lots of nuance.

For me, it all depends on who owns Watson types of computers. As Ester Dyson said "on the other end of all the wires it is just people."

One of the guys on the Watson team was asked if it could be abused to game the stock market. He answered that anything can be abused, a chair can be abused. Now this is a non-genuine response that is worrisome. A chair cannot inflict the damage that the worlds most advanced technology can. And for a team member to compare it to a chair is a warning to us all as to what kind of people are on the other end of all the wires. No? Yes?

[Raunak Roy]

03:59 PM on 02/28/2011

You did a really good job of showing how readily applicable Watson's technology is to Healthcare, something of more substance than answering Jeopardy questions. While IBM obviously has a goal of licensing this technology to other industries, could it not also license some of this technology for personal use?.

You mention how companies already use IBM analytics software to track shopping trends for example. But what about the shoppers themselves, could it not also be useful for people to be able to track and notice trends in their life? I feel that personal data tracking and analysis is something that will become an important part of life in the coming future. As more and more of our data is available for computers, as more of our actions are documented through computers the more power we should have over this data -to be able to make predictions and actively reflect on ourselves and change bad behavior or promote good habits. Say for example you notice that you eat healthier on days that you wake up earlier and have time to make yourself a meal rather than going to a fast-food restaurant. When you can track that pattern over time it becomes much easier to change your habits when that data is readily viewable rather than anecdotally trying to make sense of your sleep and eating habits...

http://bit.ly/hWcpXe

[Phil Waste]

10:18 AM on 02/21/2011

I would like to ask Watson a simple question. Is 'GOD' a space alien?

Surely, with all the information in it's data base it could answer the question. I mean that if they think they can make a complicated medical diagnoses using it, it could answer my simple question.

[DRaymond]

11:01 AM on 02/20/2011

What is Leg?

[eric14]

05:01 PM on 02/19/2011

Having a machine aimed at individual diagnosis is wrong approach. With universal socialized medicine and every citizen's data on-line and kept to to date, mass diagnostics based on epidemiological heuristics could knock out appointments for people to have tests to check for early onset of everything. But that would be commie nonsense. Better to have another ten years spent working on a machine for individuals. After all individualism is the correct philosophy. Look at all these guys who became billionaires living alone on desert islands.

[RedDogBear]

05:27 PM on 02/19/2011

"Having a machine aimed at individual diagnosis is wrong approach."

That's not what the medical version of Watson would do. It would be an aid to doctors as the doctors developed a diagnosis but the doctors still do the diagnosis. They ask Watson various questions along the way (e.g. how likely is it for a 20 year old male to have MS?)

The approach of having a computer system actually do the diagnosis has actually been done and fairly successfully although not used much, partly (at least this is what a lot of AI people think and I agree) because experts like doctors resist turning over decision making to computers. They would rather have the computers help them while they make the diagnosis.

There are a lot of fields where we've seen this. One field where I worked a lot was in software development. Initial R&D was in "automated programming" give the computer an english description of what you want it to do and have the computer write the code. It was a total flop. On the other hand Computer Aided Software Engineerng (CASE) where you provide tools to help people collaborate, document, design, debug, etc. has been very succesful.

[MTGradwell]

04:33 PM on 02/19/2011

"The Watson team is going to need to make the transition to an environment in which it comes up with multiple hypotheses".

Watson already does that. It may only speak one answer out loud, but it displays the three answers which it considers most likely, together with bars displaying the degree of confidence it has in those answers. Sometimes the right answer isn't in the top 3, but when that happens it's usually indicated by the low degree of confidence assigned to the wrong answers.

Furthermore, when the right answer isn't in the top 3, it's usually because Watson knows nothing whatsoever about the subject of the question. This is bound to happen occasionally, partly because general knowledge/trivia is such a broad field, and partly because there's still a lot of stuff that's generally known but which is not committed to any online encyclopedias or transcripts. Medical knowledge is a much narrower field, so it ought in principle be relatively easy to fit it into a Watson-sized database.

All the researchers have to do is replace Watson's trivia database with a medical one. If Watson builds its database autonomously by scouring encyclopedias and such, as I believe it does, then they just have to feed it medical journals instead. The fact that much information will be in the form of tables and diagrams might be a problem, but Watson should still be very useful even if it skips the diagrams and just processes the text.

[tploomis]

03:45 PM on 02/19/2011

Watson should have another Jeopardy match with Washington politicians on the subject of basic economics. Watson would win 258,000 to minus 50,000.

[RedDogBear]

12:00 PM on 02/19/2011

AI technology has been used in the medical field since the 1970's. One of the first, I think it actually was THE first expert systems used in the real world was for doing medical diagnosis. Watson is a very different kind of technology. An expert system tries to capture the knowledge of the best diagnosticians in the field. Watson is more like a very fast executive assistant really good at searching the web.

An expert system works by being given (and asking for) lots of information about the problem and then giving a full blown diagnosis. A Watson type system would be much more interactive answering various questions for the doctor to help her both diagnose and treat the disease. With an expert system the system is more in control, it defines what it needs to know. With a Watson system the human is more in control, it waits for the doctor to ask it the question. In general AI systems where the user retains more control are more successful in the real world.

[Martel LXIV]

11:20 AM on 02/19/2011

In 15 years, the processing power of IBM's Watson will cost about the same as a current desktop system and the software will be significantly better. Also, at that time, mobile all-purpose robots with that hardware and software capability will be available for about the cost of a mid-sized auto. They'll take your orders, drive your car to the market, return to put the purchases away, cook your meals, clean your house, do the gardening and pay the bills. They could do your job and earn your salary and all you'd have to do is recharge them.

On the other hand, when they start writing comments on blog posts, sites like this will become very boring.

[RedDogBear]

11:54 AM on 02/19/2011

"In 15 years, the processing power of IBM's Watson will cost about the same as a current desktop system"

That is certainly a reasonable thing to think based on Moore's law and the way that the computer industry has kept to it over the past few decades. However, Moore's law doesn't go on forever. At a certain point (which isn't that far away) you run into fundamental physical constraints and you can't make chips any smaller. According to Wikipedia Moore's law may stop giving us our regular chip improvements as early as 2015 http://en.wikipedia.org/wiki/Moore%27s_law

I have no idea how accurate that is. It may be many years further but unless we create some fundamental new computing techniques we may be hitting a wall fairly soon.

Of course that doesn't mean improvements stop all together. As you said the software will improve and other technologies for storing and accessing huge amounts of data will improve but the regular CPU improvements we've taken for granted for so long may end soon.

[Qirfiraz]

08:08 AM on 02/19/2011

Power of human brain lies in its ability to forget, selectively. Human brain remembers somethings and forgets other things, thru a selection algorithm which depicts the person's whole life. Computers will NEVER be able to do it.

Another complex aspect of human brain is its ability to dream, meaningfully. It should be possible in future, to induce some fuzzy thoughts in computer memory, but to have meaningful dreams will always be limited to conscious human brains. More spiritually conscious a person is, more meaningful dreams, the person will have. Yes, computer will be able to day-dream, but computer will NEVER be able to have meaningful dreams.

[RedDogBear]

10:46 AM on 02/19/2011

"Human brain remembers somethings and forgets other things, thru a selection algorithm which depicts the person's whole life. "

You should publish your findings. The last time I checked psychologists and neurologists still considered human memory to be very poorly understood and I've never heard of this selection algorithm. You've probably got a nobel prize coming.

[RedDogBear]

10:49 AM on 02/19/2011

"but computer will NEVER be able to have meaningful dreams. "

We don't yet have a good understanding of why humans dream yet. There are various hypotheses but nothing that is widely accepted or backed up by hard solid evidence. So it seems to me that we can't really say for sure that computers will never dream since we don't understand why humans do.

[Spike5]

10:50 AM on 02/18/2011

This really does have the potential to revolutionize medical training and practice.

Right now one of the biggest jobs in med school is to memorize so many facts at the expense of more 'human' issues like how to deal with aging patients or depressed people or patients who are not compliant with their post-hospitalization discharge plans. Sheer memorization will be able to take on its appropriate role since the physician won't have only his own memory to rely on in treating patients.

Pharmacy salesmen will have less influence on doctors' prescription habits because there will be databases that provide more factual data about the relative value of various drugs.

And doctors will feel more confident that they are considering all reasonable options as they come up with their preliminary diagnoses and order tests.

This really is the kind of thing that we've been hoping for since computers moved beyond simple calculating machines.

[RedDogBear]

12:53 PM on 02/19/2011

Here is an idea I think would have a lot of potential: create an "Ask Watson" web site with a small subscription fee or just with advertising. It could be a much more focused engine than Google and would be very valuable for consumers and professionals. (Have to make sure gets can't get access during exams though)

[Prousa]

12:48 PM on 02/17/2011

Watson in a hospital? Really?

Hope they turn off the "All responses must be in the form of a question" option.

What is that on your back?
You think you deserve to be here?
What, are you crazy?
You wanna come back when that thing clears up?