Data is becoming more important than ever, and developers are beginning to realize they need better ways to harness and work with data. The problem, however, is that data isn’t handled the same way development is and therefore it can become a time-consuming and complex process.
“The rise of git, docker, and DevOps has created a new world where developers can easily build, test, and deploy right from their laptop. Despite these advances, developers still struggle to manage structured data with the same speed and simplicity. Techniques like SQL scripts, database dumps, and plain text exports still leave a lot of work for developers,” the Delphix Titan team wrote on a website.
To address this, Delphix open sourced Titan earlier this year. Titan is an open-source project that enables developers to treat data like code.
“The thinking behind Titan is today the way developers develop is locally on their laptop. They pull code from their git repository, they clone that code locally on their laptop, and they go to work. What do they do for data? They are actually copying databases around and they can’t copy a commercial database around. Even if they get that data, they can’t version it. If they do testing that changes the data, then they have to get another copy and it is all a manual process. There is no git for data, and there have been several attempts to make it so we decided we would make our own,” Sanjeev Sharma, vice president and global practice director for data modernization and strategy at Delphix, told SD Times.
Titan is not git for data, but it provides capabilities that help developers manage, version and branch databases locally on their laptops, Sharma explained. The project enables developers to clone, commit, checkout, push and pull data like code. In addition, they can rollback to a previous state, build a test data library and share structured datasets, according to the project’s website. Other features include data versioning, support for off-the-shelf Docker containers, and a command line tool.
“Setting up and tearing down databases for developers has been the bane of the dev workflow. Not only do developers have to decide WHERE and HOW to run the database but they have to struggle with the configuration,” Robert Reeves, CTO of Datical, said in a post. “Of course, containers are perfect for local development, but until Titan, applying the dev workflow to the data just didn’t happen.”
