Today I watched something that completely and totally captured my attention: IBM’s supercomputer, dubbed Watson, took on two Jeopardy champions (Ken Jennings and Brad Rutter). Watson is packing some serious hardware:
“The 360 Power7 chips that make up Watson’s brain represent IBM’s best and brightest processor technology. Each Power7 is capable of over 500 GB/second of aggregate bandwidth, making it particularly adept at manipulating data at high speeds. FLOPS-wise, a 3.55 GHz Power7 delivers 218 Linpack gigaflops. For comparison, the POWER2 SC processor, which was the chip that powered cyber-chessmaster Deep Blue, managed a paltry 0.48 gigaflops, with the whole machine delivering a mere 11.4 Linpack gigaflops.” – via HPCwire
So how did it go? In short, Watson got off to a commanding lead and was crushing it early on…until the humans adapted. You see, Watson is extremely accurate when it comes to coming up with the right question to fit the answer to the clue. There were a couple of flubs, but for the most part I’d say Watson was as good or better than the human champions. Where Watson truly shone though was reaction time. You could just tell watching the humans madly pressing the buzzer that Watson had them beat when it came to triggering the buzzer when it knew the answer. I noticed something though about halfway through: I think that the humans started buzzing in without knowing the answer.
This is pure speculation, but to me it looked like the two human champs switched to triggering the buzzer before the clue was even partly finished – I think they both realized that if they waited until they heard the full clue, Watson would continue to beat them. By the end of the game Rutter was at $5000, Watson was at $5000, and Jennings was at $2000. This is a far cry from the first 5 minutes where Watson was snagging all the questions.
I don’t know the mechanics of Jeopardy very well, but it looks like the contestant can buzz in at any time after the clue has started to be read aloud. It’s a risk buzzing in on something that you might not know the answer to, but it’s a call that a human can make on the fly – and one that a machine would be hard-pressed to make.
One of the things they didn’t explain fully was how Watson go the clue: they said it was “delivered as a text file”, but what does that mean exactly? Is there someone pressing a SUBMIT key on a keyboard as Alex Trebek finishes the clue? No, there would be too much latency with that approach. Does Watson get permission to read a text file as Trebek finished the question? That seems equally problematic – it would have to be machine driven somehow. Voice recognition would seem to be the only fair way to tackle this problem, but if Watson mis-recognized a word, it would throw the whole game off. This is something I hope they explain more about later on.
This was some fascinating TV! I’m looking forward to watching the next two episodes.
UPDATE: This article says that Watson is “fed the answer in text form at the same time the answer panel appears to the two human players.” That explains Watson’s edge in buzzing in – it can parse the text file faster than a human being can read the clue. The only way for the humans to win then is to buzz in before they read the clue. Gutsy!