February 16th, 2011

530nm330Hz

My two cents on Watson

First of all, this is a demonstration, not an experiment. And of course it's not fair; "Jeopardy!" contestants at this level know almost all of the answers, and it's really all about who buzzes in first. A computer will almost always buzz in first. Game over. And I'm surprised by how many people seem to think that the contestants don't get to read the question onscreen as soon as it's revealed.

So to everyone who's whining about this being neither an honest experiment nor a level playing field, my advice is to sit back and relax; enjoy the show.

But it's been interesting to look at Watson's mistakes. Collapse )

Overall, IBM got their money's worth. They've clearly demonstrated that a bank of System 7s running their very sophisticated software can effectively data mine a large data set that is not very structured, and can quickly assign confidence values to its results.

What I'd love to see is a similar demonstration -- roughly based on the "Jeopardy!" format, except that all three contestants respond to every clue, and perhaps with each clue being scored with a daily-double-like wager -- with Watson versus equivalent systems from Google and Microsoft.