Follow the data
The turning point toward Big Data for Dunning came in 1984 when he joined New Mexico State University’s Computing Research Laboratory to work on large-scale projects for DARPA. He experimented with projects on statistical symbol and genomic analysis, machine translation, and forays into computer vision and robotics.
The lab started as one of five centers of excellence funded by the state, but “Within a few years, we were one of the few human language technology (HLT) contractors for DARPA,” said Dunning. “That’s where a lot of the techniques came from that I’ve been able to apply in many different situations.”
In the mid-1990s, startup culture lured Dunning to California. He left New Mexico State in 1996 to work at Aptex, a startup spun off from HNC Software. There Dunning helped build the first behaviorally targeted advertising system primarily for the company’s biggest customer, the InfoSeek search engine, using what he called context vector technology to transform raw user data into ad insights.
“The ability to target ads based on what people did and what they clicked on was a very interesting opportunity,” said Dunning. “That work was based quite literally on research I’d done on sequences in symbols. I’d previously thought of the sequences as language, either human or genomic, but it could be applied to sequences representing things you typed into a query engine; places you visit and the content of websites.”
When HNC Software bought back Aptex in late 1999, Dunning continued his symbol sequencing work at Musicmatch. He applied the same data principles to build some of the first commercially viable music recommendation engines around early Internet radio, integrating the recommendations into streaming protocols. Dunning’s name can be found on several of the first patents around the technology.
When Yahoo bought Musicmatch in 2004, Simon Ferrett, a systems administrator at Slacker Radio who worked with Dunning at Musicmatch, followed as Dunning cofounded Veoh Networks, a user-generated video content platform that served as a precursor of sorts to YouTube. Veoh built video recommendation engines and generated behavioral analytics around the modern notion of multi-modal recommendation—looking at multiple kinds of behavior integrated into a coherent view of what causes people to act.
“If you can retain the full nuance of [users’] actions, whether it’s scrolling on a website, looking at reviews or playing a video, you can make much better-informed recommendations because they are talking to you, telling you what they like and don’t like,” said Dunning. “We also used that same behavioral knowledge to predict what was going to be popular, allowing us to populate a peer-to-peer network that acted almost as a self-organized content-delivery network to substantially decrease streaming costs.”
Ferrett spoke about how Dunning brought the same data-informed problem-solving perspective to Veoh, and also how the way he approached code then and now makes Dunning a good fit for his current role in the Apache Incubator.
“Whenever I had some issues with the code I was writing and wasn’t sure if I was attacking it the right way, Ted had a great way of looking at it,” said Ferrett. “Some of the code Ted writes hews a bit closer to the way a professor would write it—assuming an infinitely perfect computer with an infinite drive, etc.—but the concepts were sound. For him to be reviewing these sorts of startup projects seems like the best combination of applying that theoretical and analytical perspective to other folks’ code, mentoring to make sure it’s done in the correct manner.”
Dunning left Veoh in 2007, but between then and the beginning of his work with the ASF, he founded one more startup: ID Analytics. The company offered consumer risk-management software with real-time behavioral insights to identify credit and financial identity fraud. LifeLock, an identity theft protection company, bought ID Analytics in 2012.