A new technology is coming to open source that promises an integrated, high-quality and consistent streaming analytics application framework. The technology, codenamed “jetStream,” will allow original equipment manufacturers, independent software vendors, system integrators, service providers and developers to create Big Data analytics and streaming apps that address business use cases such as real-time analysis for spam filtering, network intrusion detection and analytics, location analytics, and social media market analysis.
“When you look at the wealth of data being generated and processed, and the opportunity within that data, giving more organizations the ability to make informed, real-time decisions with data is critical,” wrote Nitin Motgi, cofounder of Continuuity, in a blog post. “We believe that the next commercial opportunity in Big Data is moving beyond ad hoc batch analysis to a real-time model where applications serve relevant data continuously to business users and consumers.”
(Related: IBM acquires Cloudant to boost its analytics)
The technology is an integration of Continuuity’s BigFlow framework for building real-time data-processing apps and AT&T’s streaming analytics tool.
According to Motgi, with jetStream, developers will be able to:
• “Direct integration of real-time data-ingestion and processing applications with Hadoop and HBase, and utilization of YARN for deployment and resource management
• Framework-level correctness, fault-tolerance guarantees, and application logic scalability that reduces friction, errors, and bugs during development
• A transaction engine that provides delivery, isolation and consistency guarantees that enable exactly-once processing semantics
• Scalability without increased operational cost of building and maintaining applications
• Develop pipelines that combine in-memory continuous query semantics with persistent, procedural event processing with simple Java APIs”
Other features include debugging, logging and monitoring tools; a simple programming model, tools and UIs; integration with HDFS and HBase; and stream processing using SQL-like language.
AT&T Labs and Continuuity are expected to bring jetStream to the market as an Apache-licensed open-source project by the third quarter of 2014.
“Putting this into open source makes available the required tools for developing data-intensive, complex applications accessible to a broader base of developers across businesses and partners of all sizes,” said Christopher W. Rice, vice president of advanced technologies at AT&T Labs, in a press release.
