From the Strata + Hadoop World event in New York City, Cloudera, the category leader that sets the standard for Apache Hadoop in the enterprise, today unveiled the industry’s first true real-time query engine for Hadoop. This major evolution to the company’s Platform for Big Data, Cloudera Enterprise, makes this the first Big Data management solution that allows batch and real-time operations to be performed on any type of data — unstructured and structured — within one massively scalable system. This new approach of creating a single, centralized Big Data platform dramatically improves the economics and performance of large scale data management in the enterprise. For the first time, organizations can process data at petabyte scale and, on the same system, interact with that data in real time to deliver “speed-of-thought” insights.
“Mainstream enterprise adoption of Hadoop will inevitably raise expectations,” said Tony Baer, Principal Analyst for Ovum. “Enterprises have grown accustomed to interactive querying and on-the-spot analytics with their existing data warehousing and BI infrastructures and will expect no less of Hadoop. With a real-time query capability powered by its new Impala engine, Cloudera is striving to level the playing field in performance and accessibility with massively parallel SQL platforms.”
This latest innovation from Cloudera, which takes the market-leading Apache Hadoop platform decisively “beyond batch,” is Cloudera Impala, an Apache-licensed, real-time query engine for data stored in HDFS (Hadoop Distributed File System) and HBase, resulting from two years of in-house development. Cloudera Enterprise RTQ (Real-time Query) provides the management and support capabilities needed to effectively operate Cloudera Impala in production environments. Cloudera partners, including Capgemini Financial Services, Karmasphere, MicroStrategy, Pentaho, Qlikview, and Tableau, have already validated their solutions with Cloudera Enterprise RTQ powered by Impala.
“Today, the power of global business is increasingly fueled by the need to understand and make decisions in real-time, driving changes in the way enterprises analyze information and make decisions,” said Jojy Mathew, Vice President and Chief Solution Strategist, Business Information Management at Capgemini Financial Services. “As a leading provider of BIM services and solutions to the global FS market, we are confident that Cloudera Enterprise Real-Time Query powered by Impala is uniquely responsive to the Big Data management needs of organizations. To that end, to leverage the enhanced performance and scalability that Cloudera offers, we are currently integrating the product with a wide array of proven Big Data use cases that we have developed for such critical domains as product, customer, risk management and finance.”
“MicroStrategy is committed to helping enterprises excel when it comes to running smarter and faster Big Data queries,” said Sanju Bansal, Executive Vice President, MicroStrategy. “For Cloudera’s latest release, MicroStrategy has optimized its queries to take advantage of the analytical and performance benefits associated with Cloudera Impala. Now, business people can use MicroStrategy’s intuitive analytics and data visualizations to make better decisions from all their data, structured and unstructured.”
Enabling the Truly Data-Driven Enterprise Big Data represents a significant opportunity for businesses. According to a recent Cloudera survey of over 100 customers, more than 70% of enterprises are actively exploring how to extract value from Big Data as a major business imperative. The survey reported operational IT efficiency and competitive advantage as the main business drivers for adopting Hadoop. However, 78% of customers said they need faster queries on Hadoop.
“We have already seen high levels of interest in, and adoption of, Hadoop by enterprises for low-cost storage and transformational processing of large volumes of data, but have argued that for Hadoop to gain more adoption for analytic workloads we need to see analytic tools taking full advantage of Hadoop’s scalable parallel processing architecture,” said Matt Aslett, research manager, data management and analytics, 451 Research. “Cloudera Enterprise RTQ and Cloudera Impala look to be a significant step in enabling enterprises to take advantage of existing SQL skills and tools to realize the potential of real-time analytics against large volumes of structured and unstructured data stored in Hadoop.”
Apache Hadoop started as an offline, batch processing system. Subsequently, Hadoop was extended to service more interactive online workloads. First among these was HBase, the distributed, tabular data store. Impala, the new open source project for real-time workloads announced by Cloudera today, introduces a scalable, distributed query engine to the Hadoop ecosystem. The technology was developed by Lead Architect of the Impala project, Marcel Kornacker, who previously co-architected the query engine for the F1 project at Google. Cloudera Impala boasts a flexible data model so it can work over more complex data than a data warehouse and is efficient, with interactive queries expressed in industry-standard SQL. It can be used by IT and business analysts across a wide range of data types and data volumes to interact at the speed of thought with data stored in HDFS or HBase.
“Expedia manages over four petabytes of data using Cloudera Enterprise. With the addition of Cloudera Enterprise RTQ powered by Impala, we are able to work on one single platform for Big Data rather than many disparate systems for archiving, ETL and analytics. This evolution of Hadoop has enabled us to reduce our latency by 50% and produce a new real business insight service not previously viable,” said Jeff Prather, Director of Global Business Intelligence and Data Warehousing Platforms at Expedia.
“Informatica and Cloudera deliver a proven combination of enterprise-ready data integration on Hadoop. Now with the real-time query execution of Cloudera Impala plus the speed of Informatica’s real-time data integration, organizations can analyze data faster than ever by performing real-time analytics on Hadoop,” said James Markarian, Chief Technology Officer, Informatica.
Now Business and IT Can Interact with Data in Real-time within Hadoop The Cloudera vision is to enable enterprises to Ask Bigger Questions and get bigger answers from all their structured and unstructured business data. By introducing Cloudera Enterprise RTQ powered by Impala, Cloudera unveils a unique system for driving business insights. Moving beyond batch processing, Cloudera enables enterprises to simplify Big Data management to empower more users, avoid large costs for increased data insights, and allow business and IT to interact with data in real time in Hadoop.
“Apache Hadoop has already transformed the industry, unlocking value from Big Data for enterprises around the world,” said Mike Olson, CEO of Cloudera. “Until now, enterprises had to limit the work they did with Hadoop because batch-mode processing using MapReduce was just too slow for some business problems. With today’s release of Cloudera Enterprise Real-Time Query powered by Impala, we solve that problem. Cloudera Impala complements MapReduce and is the latest addition to our one hundred percent open source Big Data platform. You can now store all your data in Hadoop and use the same hardware to do both powerful analytics and run real-time queries using industry-standard tools and the SQL language. This groundbreaking new project delivers the crucial next step in realizing our vision — to let our customers Ask Bigger Questions of all their data. Cloudera Enterprise with Real-Time Query powered by Impala is a major advance on the Hadoop platform and opens up new possibilities for Big Data in the enterprise.”
Advancements in Cloudera Enterprise Include:
• Performance: Typically delivers 10x faster queries than Hive/MapReduce — and can be much higher depending on the workload — with very low latency and a flexible data model.
• Cost Savings: Saves up to 90% on the incremental infrastructure costs required to process, explore and analyze Big Data. Reduces cost of ownership by running open source community-based technology on standard commodity hardware.
• Real-time Interaction with Business Data – Cloudera Enterprise RTQ (Real-time Query): Improves usability of workloads in ways never previously possible on Hadoop by collapsing return times for queries from minutes in Hive/MapReduce to seconds powered by Apache-licensed Cloudera Impala.
