Databricks made the announcements yesterday at its Data + AI Summit, which brought together experts from Databricks and across the industry as a whole to discuss data and AI.
One of the announcements was that the company added new features to Lakehouse AI, which is a data-centric platform for building generative AI applications.
One of the new features is Vector Search, which allows developers to use embedded search when developing generative AI solutions, which helps improve the accuracy of responses.
They have also added a low-code way to fine-tune LLMs and a curated list of open-source models for getting started with generative AI.
The company also announced new capabilities in Lakehouse Federation in Unity Catalog, which is a data and AI governance solution for data lakehouses. It now has new querying capabilities that allow customers to consolidate and map data assets from all the platforms they are using.
Customers can also now set up consistent access policies for their data assets and push those policies to other data warehouses.
“We’re giving organizations access to all of the data they need through one system, which will lead to more innovation — and the best part about that innovation is that it doesn’t sacrifice security,” said Matei Zaharia, co-founder and chief technologist at Databricks. “By enabling customers to easily apply the rules consistently across platforms and track data usage, we’ll help them meet compliance requirements while pushing their businesses forward.”
And finally, the company also announced the preview for LakehouseIQ, which is a natural language interface that learns about an organization’s unique data, culture, and operations using generative AI. This enables AI to provide responses that are specifically tailored to that organization because it understands their specific business jargon and context.
According to Databricks, LakehouseIQ learns from things like schemas, documents, queries, lineage, notebooks, and BI dashboards.
Databricks believes this new offering will enable every employee to gain insights from internal data, not just the data scientists. It uses Unity Catalog for governance to ensure that employees have access to the right data and don’t have access to data they shouldn’t.
“LakehouseIQ will help democratize data access for every company to improve better decision-making and accelerate innovation. With LakehouseIQ, an employee can simply write a question and find the data they need for a project, or get answers to questions relevant to their company’s operations. It removes the roadblocks inherent in traditional data tools and doesn’t require programming skills,” said Ali Ghodsi, co-founder and CEO at Databricks. “Every employee knows the questions to ask to improve their day-to-day work and ultimately their business. With LakehouseIQ, they have the power to quickly and accurately discover the answers.”
