In a move that begins to reposition the company alongside HP, Oracle and other full-stack enterprise software suite vendors, Pivotal today made several announcements around its Big Data offerings. Chief among these was the Pivotal Big Data Suite, which comprises Pivotal’s Big Data tools, all available under a single license.
Not only is this suite now available to enterprises, but the tools it encompasses—SQL-on-Hadoop tool HAWQ, Pivotal GemFire, and EMC’s Greenplum analytics database—are now open source as well. Pivotal also announced a long-term partnership with Hortonworks, as well as the formation of the Open Data Platform with companies like GE, IBM, Infosys and SAS.
Michael Cucchi, senior director of product marketing at Pivotal, said that the company has “spent a lot of time in the last year listening, doing pattern recognition and making sure we’re there to support the journey. A lot of large enterprises are significantly threatened by Uber, Airbnb, Instacart, and others. These startups can create an offering from scratch because they can reinvent their infrastructure. They execute agilely and flexibly, and the next thing you know they’re toppling industries. A lot of startups are getting there by leveraging Hadoop. They turn to it as their cloud database of the future. When these enterprises are looking to make these transitions, we work with them. They are insisting this new stack be an open-source stack.”
The Pivotal Big Data Suite also includes new tools designed to help businesses increase velocity on top of Hadoop. The Suite includes Spring XD, popular NoSQL Redis, and RabbitMQ. VMware had previously purchased SpringSource, and these assets included both GemFire and RabbitMQ commercial support and services. The new addition to the former Spring portfolio is Spring XD, which is to Hadoop what Spring was to Java EE.
Put simply, that means Spring XD simplifies the configuration and boilerplate work that comes with building Map/Reduce and other YARN queries. Spring did this for Java EE, simplifying a process that, for years, had enterprise developers grinding their teeth over endless configuration and XML files.
Spring XD can link into Project Reactor Streams, RxJava Observables, and Spark Streaming to handle complex event processing. It’s also capable of easing Big Data application development by including a visual tool for building batch analytics jobs with a mouse and some previously built configurations.
The Pivotal Big Data Suite is licensed as a single entity, and all included tools are now open source. For Greenplum, this means the analytics and optimizations EMC built on top of POSTGRES are now open source for the first time, as is the HAWQ SQL tool for Hadoop.
Pivotal announced two Hadoop-related initiatives, as well. The first is the Open Data Platform, which is an effort by Pivotal, some of its key customers, IBM and other vendors, to standardize on a single version of Hadoop. Until now, Pivotal has offered its own distribution of Hadoop for enterprises, but today’s news was a sign of the company moving away from this tactic.
Instead, Pivotal will now work with Hortonworks to certify a single version of Apache Hadoop for use with its and other vendors’ products. Initially, this certified version of Hadoop will only include HDFS, YARN, Map/Reduce and Apache Ambari, the Hadoop management project.
The agreements between Hortonworks and Pivotal also mean that Pivotal’s tools will work on Hortonworks’ distribution of Hadoop, which includes more than the items being standardized on by the Open Data Platform.
Cucchi said that these announcements are indicative of what Pivotal is seeing in the markets. He said that enterprises are struggling to understand Big Data and deal with it in a timely fashion as their smaller competitors can do thanks to their leaner infrastructures.
Thus, Cucchi said that Pivotal has open-sourced HAWQ and made it a part of its Big Data Suite in order to give businesses a competitive advantage by allowing them to bring their SQL developers to bear upon Hadoop. He also said that HAWQ was previously tied to Pivotal’s Hadoop distribution, but that it now works on Hortonworks’ distribution.