Syncsort has released the results of its fourth annual Big Data survey, detailing the top use cases and challenges organizations face with Big Data initiatives.
Based on the results, there are five key trends that organizations should monitor in 2018.
1. The composition of the data lake has been shifting. The amount of Relational Database Management Systems and NoSQL databases increased this year. Relational Database Management Systems were chosen as the top source, surpassing traditional enterprise data warehouses. It also found that cloud repositories are gaining popularity.
2. Legacy platforms will continue to make significant contributions to the data lake. There was a 27 percent increase from last year in the amount of people who believe it is important to access and integrate mainframe data into data lakes.
3. Data quality and regulatory compliance were top challenges for many organizations. Forty percent of respondents cited these as significant struggles that they will continue to try to improve upon in the coming year.
4. Another trend is that data lakes will be kept fresher in order to support data use. Seventy-one percent of respondents listed ETL as the most compelling data lake use case, with advanced/predictive analytics and real-time analytics coming in second and third, respectively. Even though these case require up-to-date data, over 75 percent of respondents reported challenges with keeping their data lake in sync.
5. Finally, organizations will continue their investment in Big Data. Ninety percent of companies found that using Hadoop and Spark instead of legacy systems was valuable in saving money and driving data insights.
“We are seeing increased adoption of data lake initiatives where organizations are very focused on governance of the data in the data lakes, increasing benefits through advanced analytics and machine learning and deployment of hybrid environments including cloud,” said Tendü Yoğurtçu, CTO, Syncsort. “But those benefits can only be unlocked if organizations have access to enterprise data, can create trusted data sets and establish effective data governance practices. This propels them to a place where they can not only adapt to digital disruption, but take advantage of it so their businesses thrive.”