The need for the digital transformation of business processes, operations, and products is nearly ubiquitous. This is putting development teams under immense pressure to accelerate software releases, despite time and budget constraints. At the same time, compliance with data privacy and protection mandates, as well as other risk mitigation efforts (e.g., zero trust), often choke the rate of innovation by making it harder for development teams to acquire and use high-quality test data. Is it possible to achieve both of these seemingly opposing requirements, speed and protection?
The answer lies in a familiar tactic: automation. Development teams are increasingly adept at automating huge chunks of their work, from setting up the necessary infrastructure environments to building, integrating, testing, and releasing software. Call it DevOps or CI/CD, the tactic is the same: ruthlessly automate mundane or repetitive tasks. To ensure compliance requirements don’t hinder development, IT leaders must similarly prioritize automating data profiling and protection as a normal part of their development pipelines.
The growing impact of data privacy on software development
The regulatory landscape for data privacy and protection continues to grow, resulting in ever-increasing, and increasingly complex, compliance requirements. In fact, McKinsey found three quarters of all countries have adopted data localization rules, which have “major implications for the IT footprints, data governance, and data architectures of companies, as well as their interactions with local regulators.”
Existing data privacy regulations such GDPR in the EU and HIPAA in the US, updates to older mandates (e.g., the recently updated Federal Trade Commission (FTC) Safeguards Rule mandated by the Gramm-Leach-Bliley Act), and new and emerging laws (e.g., Virginia Consumer Data Protection Act, Canada’s Consumer Privacy Protection Act) all threaten to slow down software development and innovation by adding layers of security requirements onto the development process.
Even without the introduction of new privacy mandates, the impact of data privacy and security requirements on development is almost certain to grow. For one thing, development and testing environments have proven to be rich attack targets for threat actors. From source code management systems to infrastructure such as virtual test servers to the test data itself, all are attractive targets for bad actors seeking to compromise systems and data. Add in the many different cloud development platforms like Salesforce and SAP, and it’s clear there is plenty of opportunity for a hungry hacker with nefarious intentions.
Therefore teams must ensure the entire application lifecycle is secure, including development and test environments, whether on-prem or in the cloud. How do IT and security accomplish this without slowing development and release cycles? The answer lies in test data automation.
Test data management meets DevOps
The software development process is reliant on access to fresh test data. Traditional methods for managing and provisioning test data are typically manual and tremendously slow – think ticketing systems and siloed, request-fulfill models that can take days or even weeks. These processes are very much at odds with modern development methods such as DevOps and CI/CD, which demand fast, iterative release cycles.
This is where application innovation often grinds to a halt. DevOps and DevSecOps processes have automated quality assurance testing and security and compliance testing throughout the CI/CD pipeline. But data provisioning and governance has remained a manual and time-consuming practice. Enter DevOps test data management (TDM) which automates the “last mile” of DevOps and provides fast delivery of lightweight, protected data in minutes instead of days, weeks or months. With DevOps TDM, organizations can accelerate development and testing, and in turn, can increase compliance and innovation.
Just how much can DevOps TDM accelerate software innovation? Consider one example from Dell Technologies. The technology giant’s developers needed quick access to fresh test data, but, like many other organizations, manually provisioning the data was a slow, tedious process.
By automating DevOps test data management, Dell significantly increased the speed and efficiency of its test data provisioning and governance. Now, 92% of Dell’s ~160 global, non-production database environments are refreshed automatically on a bi-weekly basis. Developers can now initiate releases through their CI/CD pipelines in just 17 minutes. This has allowed the Dell team to run 6 million pipelines the first quarter of 2022, and more than 50 million since they implemented this standardized, automated approach.
Shrink the surface area of private data
Antiquated approaches to test data management often rely on scripts or otherwise poorly integrated processes that result in the proliferation of sensitive data throughout the enterprise. It’s not uncommon for each development environment to have its own copy of sensitive production data for testing purposes. And often developers maintain their own copies for coding and unit testing. Many enterprises end up with hundreds or even thousands of uncontrolled copies of sensitive data.
Privacy mandates and security policies treat these copies of sensitive data no differently than the production databases from which they were spawned. Sensitive data such as personally identifiable information (PII) or cardholder data must be secured to the same degree, whether or not it’s in production. This often translates into requiring encryption both at rest and in transit, as well as carefully managed access controls and other protections. And then there are the near-universal requirements for the right to be forgotten. Privacy mandates regularly require businesses to destroy personal data upon request. It does not matter where that data lives.
The solution is eliminating the replication of sensitive data through the use of data masking. To provide production-quality data to your teams and non-production environments without multiplying the burden of security and privacy protections, DevOps TDM approaches — when implemented properly — automate the masking of sensitive data. In effect, this step shrinks the surface area that you must protect. This reduces your compliance and security risks as well as the impact on your budget.
Quite simply, having less sensitive data strewn about your business means less you to protect. Automation can make that possible.
Starting small but thinking big
Automating with DevOps TDM may appear overwhelming at first. Where do you start? But this is one change where it is very easy to start small, automating test data delivery and masking for just one or a handful of applications. Many businesses begin by addressing their most sensitive environments and where CI/CD pipelines already exist, such as customer-facing apps. Here, the need for protection and the underlying automation framework (i.e., the DevOps toolchains) already exist.
But businesses should also think big as they evaluate solutions. The number of distinct data sources is likely to expand over time. You might have a SQL database on AWS today, but then add your Salesforce platform and mainframe DB2 into the mix in the future. Masking these data sources while preserving referential integrity across them may prove challenging but is essential to effective integration and user acceptance testing.
Ultimately, businesses centralize DevOps TDM while giving their development teams autonomy over the acquisition and use of test data. Centralization means you can apply policies for the masking of sensitive fields and use database virtualization to cost-effectively provision data.
The benefits of DevOps TDM are substantial. Not only do businesses improve compliance and mitigate risks, they also speed up development and reduce costs. It represents one of those rare instances where a tradeoff between faster, better (safer) and cheaper is no longer required.
