But having all that data publicly available means there’s a lot of work to be done before an API can go live. Depending on the data source, the project to open the data behind an API can sometimes be very easy, and sometimes require a lot of hands-on work.
“Sometimes it works great; sometimes it requires manual work to make sure it works,” said Taveau. “It depends on the complexity of the API and the data you’re trying to reach. Location is very easy to automate: It doesn’t require the security. The data is not critical—just a pinpoint to a merchant location or restaurant location or ATM location. That’s not critical.”
When it comes to more delicate information, said Taveau, the effort requires more coordination and care. “When you talk about a merchant identifier API, that’s a completely different story. As you expect, the review process for the location API and the tokenization API or risk management are very different.”
Yaniv Yehuda, cofounder and CTO of DBmaestro, said that managing software is almost second nature to developers, but managing the data is another story entirely. “Tools are important, as well as human processes. But what we found out is that people are really challenged when they get to the database. Databases don’t follow the same processes the code does. They require special attention,” he said.
The reason for the difference in managing them is that they are created in different ways and have a different workflow, said Yehuda. “When you deal with traditional code, you have your development people working with version control. Then it gets built by a build server and pushed to the next environment,” he said.
“When you deal with a database, a database is not compiled in one area and then pushed to another. The database actually holds its internal structure of the data: the schema in each environment. If you have a development environment, it holds the structure code content as the application, but it still hosts the same data in the Q/A environment. In order to deal with that, you create transition code that changes the database from one state to the next.
“The database, in order to be promoted from one version to the next, it requires additional steps in order to deal with that transition code. Because people are really doing stuff with traditional tools, developers have to do this stuff manually. This code is really static, so if you have some changes in your development environment and you want them to push to another environment, then you write the code to deal with that transition. You have code override, and people are not using the database itself to manage version control because they have to extract the objects from the database and put them in a different version control from the code.
“What DBmaestro does first is to create a bulletproof version-control system. We created an enforced version control to the database. You cannot change the database unless you check out the object. You go to the database and say ‘add column,’ but only when you check it out. When you check it back in, it gets version-controlled. The second thing DBmaestro does is it safely automates deployments from one environment to another.”
Red Gate offers similar tools for data automation and management. Ben Rees, general manager at Red Gate, said that the company has been building tools around this problem for 15 years now. “People were using them to do database life-cycle management, but we didn’t call it that back then. Back then we called it agile database delivery and agile database development,” he said.