Databricks has introduced Spark Datasets, an extension of the DataFrames API that provides a type-safe, object-oriented programming interface. Spark 1.6 includes an API preview of Datasets, and they will be a development focus for the next several versions of Spark.
Databricks was founded out of UC Berkeley’s AMPLab by the creators of Apache Spark, and the team has been working on systems to extract value from Big Data. With Datasets, they hope to create a way to write more efficient Spark applications, and they designed Datasets to work alongside the existing RDD API.
A full list of features with Spark Datasets can be found on Databricks’ blog.
New Developer Community from Conviva
Conviva, a video optimization company, announced the release of its online Developer Community.
Because consumers are viewing their videos on larger screens, developers need to deliver optimized experiences, according to Conviva. The Developer Community is replacing the company’s existing Developer Resource Center, and it will provide developers with technical documentation, software development kits, sample source code, and reference guides for testing their new applications.
Survey shows lack of full DevOps adoption
CA Technologies conducted a global study that showed only 20% of organizations that have attempted to implement DevOps have fully deployed it. Its research also showed that these “advanced” DevOps adopters were more likely to report that their digital initiatives contributed to competitiveness.
The study, titled “Assembling the DevOps Jigsaw,” said that more than 80% of senior IT and business executives identified nine components that are key to maximizing DevOps effectiveness. These components include business stakeholder education, security and compliance measures, and IT processes.
The majority of DevOps adopters said there’s still work to be done in relation to infrastructure and tooling, according to the survey. More information can be found here.
