RAM is the hip place to be. For modern applications built to scale out to thousands or even millions of users, scaling the data behind those applications has long been a difficult task. But a host of in-memory data grids from companies like McObject, Oracle and Terracotta are solving the scalable data-store problem, and they all expect 2013 to be a banner year for such software.
Massimo Pezzini, vice president and fellow at Gartner Research, said he expects explosive growth in the in-memory data store market in 2013. While the research he cited is not yet published, he did have figures on the market and its potential for growth.
“We think in 2011 the market for in-memory database was approximately US$250 million in terms of license and maintenance revenue… We’ve spoken with vendors, and some are projecting high double-digit, if not triple-digit growth in 2013. Terracotta is expecting to triple its revenues this year. We think this is going to be a $1 billion market by 2016. In software, $1 billion is a big market,” he said.
There are many reasons for this growth, but Pezzini said a few use cases are most common. “I would say the most obvious use case is really caching: caching a database, caching a session in a website, etc.,” he said. “Quite a lot of customers have started using an in-memory data grid in that way: their own layer to speed up the performance of Web applications.” But new use cases are cropping up thanks to the proliferation of clouds and the need to operate at scale.
“Lately, we have seen examples of customers using an in-memory data grid as a data-management platform, as a platform to host the database of record,” said Pezzini. “In business practice, that is not relational, because in-memory data grids are based on an object-oriented NoSQL paradigm. This is one of the reasons customers are looking into in-memory data grids.”
Mike Allen, vice president of product management at Terracotta, thinks there are a few reasons behind the growth of in-memory data grids, but there’s one large factor he cited. “One is data volume, and now you can suddenly get machines with a lot of memory very cheaply,” he said. “You can stack up six servers with a half-terabyte of RAM each, and then keep all your data in memory, which was never really possible before. We scale that grid predictably and scale it to that capacity.”
Another reason for the growth of in-memory data grids, said Allen, is the new focus on analytics in business. With Terracotta, or any other in-memory data grid, analytics can be run in real time because all the information is stored in RAM.
“If I’m doing transactions on an e-commerce site, I don’t typically have a view into those transactions until after,” he said. “But I can now look at that data in-flight, and do real-time promotions or modify pricing. I can offer people incentives, or correlate current actual behavior with profile info about historical access.”
Uri Cohen, vice president of product management at GigaSpaces, said there are two major reasons for the growth of in-memory data grids. “The first is that the market for such solutions has definitely grown, with drivers such as the explosion of user-generated data and Web-scale deployments,” he said. “Whereas in the past most of the demand for these technologies came from high-end financial services and telecom applications, today it’s prevalent in almost every vertical.
“We’re seeing this demand in e-commerce, travel, fraud detection, homeland security, and SaaS implementations, to name a few. Some apps need to process tens or even hundreds of thousands of events per second, which is only feasible if you’re using a distributed architecture. The in-memory aspect of things is what allows you to do it at real-time latencies, meaning you can do this as the events are flowing into your system and not have to wait for a batch Map/Reduce job to get the processed data and insights.
“The second trend is that with the advent of NoSQL data stores and cloud technologies, which drive people toward distributed architectures, the market is much better educated about such technologies and understands the trade-offs associated with them, with terms like CAP and BASE being widely known and reasonably well-comprehended. This saves us a lot of work in explaining our technology and how to implement your applications on top of it.”