Meta has recently announced its open-source project, Velox. This offering comes as an execution engine aimed at accelerating data management systems and streamlining their development.
Velox is currently under active development and is geared at helping to consolidate and unify data management systems in a beneficial way.
The open-source project aims to solve the challenge of fragmentation within systems with different feature sets and inconsistent semantics. With Velox, the data-intensive components of data computation engines are unified while still maintaining the extensibility and adaptability for different computation engines.
According to Meta, Velox democratizes optimizations that had been implemented only in individual engines, resulting in a framework that allows for the implementation of consistent semantics.
Velox also leverages several runtime optimizations including filter and conjunct reordering, key normalization for array and hash-based aggressions and joins, dynamic filter push down, and adaptive column prefetching.
The company also stated that this open-source project is designed to support more complex data types and extensively relies on dictionary encoding for cardinality- increasing and cardinality-reducing operations such as joins and filtering.
To learn more, read the blog post.
