0110-sdt-beam

Apache Beam goes top level

The Apache Foundation this morning announced the promotion of Apache Beam to the top level. That’s good news for the many contributors and users of this unified programming model, which allows them to write batch and streaming jobs at the same time, and to run the resulting artifact on various execution engines. Within the Hadoop … continue reading

dec16-1128-guestview-ledem

Guest View: The first release of Apache Arrow

Work on Apache Arrow has been progressing rapidly since its inception earlier this year, and now Arrow is the open-source standard for columnar in-memory execution, enabling fast vectorized data processing and interoperability across the Big Data ecosystem. Background Apache Parquet is now the de facto standard for columnar storage on disk, and building on that … continue reading

1101-sdt-redis

Redis adds Spark-based machine learning

Redis and Spark are coming a bit closer together. Redis Labs today announced a new project, Redis-ML, to bring Spark-based machine learning capabilities to the Redis database. The combination will provide a faster place to store a trained Spark machine learning model. Redis-ML is hosted on GitHub and will be demonstrated at the Big Data … continue reading

1003-sdt-news

Compuware’s bringing the mainframe to DevOps, Arduino IoT kit, and Yahoo open-sources deep learning model—SD Times news digest: Oct. 3, 2016

Compuware is extending its mainframe/DevOps initiatives with the availability of new REST APIs for ISPW, its source-code-management and release automation solution that Compuware acquired in January. The new APIs are architected as REST web services, and with these APIs, Compuware customers can begin to accelerate cross-platform apps, reduce IT costs, and improve the quality of … continue reading

0628.sdt-mongodb

MongoDB announces new database-as-a-service and real-time analytic solutions

MongoDB wants to empower developers to do more with its NoSQL database. The company unveiled MongoDB Atlas and MongoDB Connector for Apache Spark at its third annual MongoDB World conference in New York City today. “When people free their mind and focus on the problem or task at hand, you can do your best thinking,” … continue reading

XIN,REYNOLD

Examining Spark 2.0 with Reynold Xin

With the release of Databricks Community Edition today, and the kicking off of the second day of Spark Summit in San Francisco, the Apache Spark project is a hot topic. Underneath the hood, the update brings many changes and improvements to the platform, and at Databricks’ Summit, these were discussed at length. To get a … continue reading

0606.sdt-sparksummit

Spark Summit shows spread of software

Companies like Amazon, Baidu, IBM and Microsoft were all at Spark Summit this week to discuss how they used the engine. Some of those companies even had new Spark-based products to show off. IBM, for example, introduced a data-centric development environment, which will be able to support machine learning with the R language thanks to … continue reading

0512.sdt-databricks

Databricks announces Apache Spark 2.0 technical preview

It’s been two years since Apache Spark 1.0 was released, and today Databricks is giving everyone a preview for what is to come in version 2.0. According to the company, the upcoming version focuses on three major themes: easier, faster and smarter. “Spark 2.0 builds on what we have learned in the past two years, … continue reading

SQL

The building blocks of SQL

SQL is the prime building block of the modern enterprise. All those exciting applications, nifty mobile apps and massive back-end projects are, essentially, useless without the data behind them. That data may not be so important at runtime if the application is just saving logs or form information, but at the end of the day, … continue reading

0302.sdt-hortonworks

Hortonworks collaborates with HP to improve Spark

Hortonworks, one of the three major Hadoop vendors, announced yesterday that it has been collaborating with HP to improve Apache Spark. The work has already yielded faster sort and in-memory computation for the project, as well as improved performance and usage for scalability. Hortonworks also announced the inclusion of Apache Kafka and Storm in its … continue reading

0224.sdt-caffe

Yahoo brings deep learning framework to Spark

Yahoo has released CaffeOnSpark, which brings the fruits of two University of California, Berkeley projects together: vision-focused deep learning framework Caffe, and Big Data processing engine Apache Spark. Without the aid of Spark, Caffe can process up to 60 million images per day. Those numbers come from benchmarks on a single NVIDIA GPU, so the … continue reading

0217.sdt-arrow

Apache Arrow unifies in-memory columnar storage

The database revolution happened a few years back, when NoSQL options stormed the world. Today, however, a second wave of innovation in data storage has been unleashed in the form of Apache Arrow. Arrow builds a standard for columnar in-memory analytics, and will provide a unified data structure, algorithms and cross-language bindings. The overall goal … continue reading

Next Page »
HTML Snippets Powered By : XYZScripts.com