The inaugural Spark Summit East conference is taking place in New York City this week, and many Big Data companies have seized the opportunity to announce new partnerships, product releases, integrations and Apache Spark services.
Databricks announces ‘Jobs’ cloud feature, partnership with Tresata
Databricks has announced a new feature for its Databricks Cloud called “Jobs” for automated scheduling and managing of Spark workload production pipelines.
The new Databricks Cloud feature ingests, transforms and processes new data streams periodically, enabling the creation of Spark cloud applications as well as apps on new or existing clusters running AWS. Jobs also includes a flexible job scheduler for application execution, as well as a notification service. More details about Databricks Cloud’s Jobs feature are available here.
Databricks also announced a new joint solution with real-time customer intelligence provider Tresata, integrating the Databricks Cloud with Tresata’s TEAK predictive anti-money-laundering solution engineered for Spark. More details can be found here.
Cloudwick announces performance assessment service for Spark
Enterprise Big Data consulting firm Cloudwick debuted Cloudwick Insight, an assessment service for Apache Spark to analyze performance by CPU, RAM, disk I/O and network utilization.
Cloudwick Insight uses Pepperdata and Spark performance optimization to provide analysis and recommendations for Spark jobs on Cloudera or Hortonworks clusters. More details are available here.
MemSQL rolls out geospatial capabilities for in-memory database
Real-time transactional database provider MemSQL has announced support for a full range of relational SQL and geospatial functions in its in-memory distributed database. The geospatial capabilities are designed to increase database efficiency by combining geospatial data in the primary datastore for shorter latency and more-optimized synchronization. More information can be found here.
ClearStory Data announces Hortonworks partnership for data processing
Enterprise data intelligence company ClearStory Data is partnering with Hadoop provider Hortonworks to tackle large-scale data processing on Hortonworks Data Platform, and data governance for Hadoop data lake initiatives. The partnership is targeted primarily at enterprise organizations in the healthcare and pharmaceutical industries to analyze large volumes of medical data for business intelligence. Additional details about the partnership are available here.
Qubole extends Big-Data-as-a-Service platform with Amazon Redshift
Big Data-as-a-Service company Qubole, independent of the Spark Summit, has announced that Qubole Data Service (QDS) now includes a connector to the Amazon Redshift cloud service for structured data management. Qubole’s integration with the cloud-based data warehouse in AWS enables running queries on data stored in Redshift directly within the QDS workbench. More information can be found here.
