When your company has already defeated the best chess player in the world, you might need a few years to find a new challenge. The Watson project began as a brainstorming session at IBM, with teams searching for new problems they could solve in the public eye. Tonight, the results of four years of work will culminate in Watson taking the stage against reigning champions on the famous TV quiz show “Jeopardy.”
Rod Smith, vice president of emerging technology at IBM, said that Watson is actually a major step forward for analytics, not just information retrieval. He said that Watson had to meet a number of challenges just to compete, and that good analytics were a must to keep the system competitive.
“There were two sticking points,” said Smith. “One, there is really no type of analytics that covers everything you want to do. [The team] had to look at techniques and blend different techniques, some that stay at surface level and scan lots of data, then others are more machine learning and deep analytics on smaller sets of data.
“There was no Holy Grail here; they had to look at the different techniques. I think that was a huge breakthrough: realizing that there’s not just the one way of doing it, and figuring out how do we blend these different techniques that have different success rates in a parallel fashion, then aggregate that and come up with an answer.”
Smith said that the project was not, by any means, a cakewalk. “It was an iterative process. They would use what they had to play ‘Jeopardy,’ and some things looked promising because of the analytics. Then they’d hit some stumbling blocks,” he said.
“As research goes, it was very practical oriented. If results didn’t come back, you didn’t rest on your laurels. Emotionally, it was extremely tough. You think you’re getting close answering X number of questions right, but it turns out next time you get some more right but it gets the previous ones wrong.
“It was about figuring out how the algorithms work. You had to be a fairly ego-less team and look at the results and iterate them. Plus, you have to answer a question in less than three seconds, so it’s a unique hardware configuration as well.”
Watson is taking the stage against returning “Jeopardy” champions Ken Jennings and Brad Rutter. It will be left unattended when on the show, and the episodes featuring it begin airing tonight, though they were filmed last week. Even the IBMers don’t know how Watson did.
