Trifacta Inc., the global leader in data preparation, today announced a new set of capabilities specifically focused on making data quality assessment, remediation and monitoring more efficient. The new capabilities are designed to help organizations modernize their approach to addressing data quality issues that hinder the success of analytics, machine learning and cloud data management initiatives. With an increasing need to derive faster insights and predictions from disparate sources of data, organizations can no longer rely on legacy, siloed data quality processes to handle the speed, scale and diversity of today’s data.

The first, Active Profiling, is a new selection model that blends real time visual and interactive guidance with machine learning, helping the user discover and interact with data quality issues and resolve them with intelligent suggestions — all while sharing live previews to ensure that user validation is built into every step. Second, Smart Cleaning is a set of new features to address data quality issues around formatting and standardization. With Cluster Clean, Format Clean, and Target Clean, users can choose from a variety of different intelligent cleaning  approaches to resolve issues with mismatched formatting and miscategorizations.

“Having an intuitive tool enables us to massage and normalize the data very fast, said Litty Thomas, director of marketing, Malwarebytes.” “Historically it took us days to do it and now with Trifacta we are able to turn it around instantaneously,”

As the volumes and sources of data continue to expand, so do the number of advanced machine learning models and analytics tools available to help organizations maximize the value of their data. The trouble is, machine learning models and analytics tools are only as good as the data that feeds them, and many organizations struggle with data quality issues. The success of today’s machine learning and analytics initiatives requires a new approach to data quality that focuses on increasing the speed, scale and accuracy of cleaning and standardizing data. As organizations modernize data quality processes for the machine learning and analytics use cases of today, the success rate of these initiatives will rapidly improve compared to the anemic success rates currently seen.

“The need for modernizing data quality is exactly what we’ve seen with ETL and data integration. To improve the speed, scale and accuracy of these processes, they must transition from being managed as siloed, IT-led activities, to ones that are driven by analysts and data scientists,” said Wei Zheng, vice president of products at Trifacta. “Trifacta’s expansion into Data Quality with the introduction of Active Profiling and Smart Cleaning will help organizations democratize data quality remediation while maintaining governance. As a result, the efficiency and value of their analytics initiatives will significantly improve.”

Gartner Inc. has determined that 40% of all failed business initiatives are a result of poor quality data and data quality effects overall labor productivity by as much as 20%. Research indicates that organizations believe poor data quality is costing them an average of $9.7 million per year. In order to truly capitalize on the unprecedented business opportunity of machine learning and AI, organizations must ensure their data meets high standards of quality. Their algorithms and analytics tools will not provide value if they run on poor-quality data.

The new features from Trifacta to further support data quality initiatives include:

  • Active Profiling

    • New Selection Model creates a seamless experience around highlighting data quality issues and offers interactive guidance on how to resolve the issues.

    • Column selection provides expanded histograms, data quality bars, and pattern information to offer immediate insight to column distributions and data quality issues that update with every change and preview every step.

    • Interaction with profiling information drives intelligent suggestions and methods for cleaning that the user can choose from.

  • Smart Cleaning

    • Cluster Clean uses state-of-the-art clustering algorithms to group like values and resolve to a single standard value.

    • Format Clean will identify and correct  multiple date or phone number formats to a standard format.

    • Reference Clean gives Trifacta a target column of dates, phone numbers or entity names or formats as a target and Trifacta will align the values in your column to this target.

“Canopy’s ability to utilize raw, diverse data sources to help our customers in financial services succeed with machine learning and analytics initiatives is a real differentiator for us — and critical to that is ensuring data quality at every stage,” said Amit Prakash Gupta, CTO of Canopy PTE. LTD.  “With Trifacta’s new Active Profiling and Smart Cleaning functionality, we’re able to broaden data quality initiatives to span a greater  number of users and in the process improve the speed, scale and accuracy of our projects.”

Later in 2019, Trifacta will be focused on bringing data quality to the automation process. With the rollout of additional functionality to support flow orchestration, monitoring, and alerting, organizations will be able to set data quality specifications and isolate any data that doesn’t meet the data quality standards of the organization to re-evaluate and continue to improve. This will continue Trifacta’s strategy of expanding beyond data preparation by adding support for data quality and other aspects of a modern DataOps platform.