I stopped in on the MongoDB Developer Days in San Francisco on Friday. After all the discussion of NoSQLs we’ve heard over the past few years, it was great to see how enterprises are actually using MongoDB.
What came out of the event were a number of stacks and processes that companies had chained to MongoDB. Yuri Finkelstein, lead architect of platform services at eBay, explained how MongoDB is used to solve the auction service’s photo-hosting problems.
eBay hosts millions of images every day, and Finkelstein said that eliminating duplicate images from the data stores is actually chartable to a dollar amount of savings in processing, storage, bandwidth and other common IT costs. As such, the team at eBay had to find a way to check on page load whether images were duplicates. This required a separate data store to hold all the image MD5 checksums—and some additional meta information—in case of a rare but not impossible chance for MD5 collisions. And for this task, they chose MongoDB.
“We are running MongoDB in a very demanding environment. It’s business-critical at the moment. Is it reliable? Yes,” said Finkelstein. He also advised that running MongoDB requires intimate knowledge of just how the database works.
“MongoDB has lots of features,” he said. “Really, there are too many to pick from, and it’s tempting to use them all. The problem is unless, you understand how every feature works, you’re going to find yourself in trouble. If you want to be successful, you need to deeply understand every aspect of how this database works in order to be successful.”
Elsewhere at MongoDB Days, Peter Bakkum, member of the technical staff at Groupon, described a complex system at Groupon, the star of which was actually Twitter’s open-source project, the Storm Framework.