The ketchup may move slowly, but Kraft Heinz has been ramping up its software testing to breakneck speed over the past five years. Speaking at the STARWEST conference this week, Aaron Katz, group lead for IT global testing at the Kraft Heinz Company, detailed the path to an automated and more successful testing organization.
The path to success initially went through the creation of a testing center of excellence within Kraft Heinz, which Katz heads up. This team was built with experts and even received extra training in other areas of the development process to ensure their end-to-end relevance.
Over the course of three years, Katz said that the Kraft Heinz testing team pushed a major automation campaign to completion, and as a result, is now catching 95% of all defects before hitting production, up from 65% at the start.
Katz said that the process began with the automation of regression tests and of SAP ERP-specific performance tests. In 2014, its second year, the effort to automate integration testing and ERP upgrades was developed. Then, in 2015, the teams took on security and overall performance testing automation.
“Before, when we’d be involved in these big programs at the end, and we’d start building our integration test plan, but now we start really toward the end of the design phase,” said Katz, describing the shift-left effort inside Kraft Heinz.
“Our automation in our regression [test] inventory is 86%,” he said. “Gartner had industry standards around 60%, so this is a great achievement. We have over 2,000 automated scripts. We’ve met with business partners and have 100% critical process coverage.”
There was more at work than simply a shift to a new office for the testing center of excellence, said Katz. A lot of the changes came from being better embedded in the development process and earlier.
“We’ve been able to use [automation] going from development to QA,” he said. “We can run some sanity checks in a short window of time. Usually in two or three hours we can run several hundred automated scripts, so when it does go to integration testing the next day, we already know whether or not things are going to go smoothly.”
Katz’s team has also carved out enough time to do the work that ensures products are defect-free at launch. “When we do our integration and regression testing, we do a month in closed simulation. The finance people really love this. We use automation to generate transactional data, so when we do the month in closed, there’s a dynamic suite of data to go through the process,” he said.
Katz had more tips for saving time and sanity on testing teams. “We also want to focus on core business applications. We don’t want to spend time or money building automation for standalone systems or systems that don’t get much use. We also want to ensure these are going to be repeatable tests. If a business process is going to change or go away, you don’t want to spend the time automating that process. This seems like common sense, but it is a [danger].”
Katz said that before the testing teams were split out into a center of excellence, integration tests were only about 20% automated. Now that number is 97%. He said that once the center of excellence started hitting its stride, the questions from other project managers stopped being “Why are we doing this?” and started going the other way. “Really the question is, ‘How come you’re not using us?’ That’s what leadership asks,” he said.
As a leadership organization inside Kraft Heinz, the transformation process undergone by the testing team has not just vastly improved the quality of testing, it’s also made the center of excellence into a leader for the rest of the company’s software projects. That’s a major transformation from the typical enterprise roll for a room full of forgotten, interchangeable testers. Thus, instead of being asked to explain the use of testing, Katz and his team find themselves asking other projects why they haven’t yet taken advantage of their skills.