DevOps teams often struggle to meet speed and quality goals simultaneously. Intelligent test automation company mabl just released its 2020 State of DevOps Report based on a survey of 1,000 software professionals, including testers and QA professionals (59%) as well as developers and engineers (34%). The key takeaway is that manual testing doesn’t cut it.

“The teams doing manual testing are seeing no customer experience lift, even if they’re doing a lot of tests manually,” said Chou Yang, content and brand marketer at mabl. “The work it takes to implement a smart testing strategy and automation can be a time-consuming investment up front, and requires a skillset that’s hard to find, so they stick with manual testing to get by.”

There’s a trade-off between what’s easiest in the short-term versus what’s wise over the long-term. As the report illustrates, the highest performing DevOps teams have integrated many types of automated testing into their automated CI/CD pipelines. Those teams are less stressed and more confident than Waterfall, Agile, and Emerging DevOps teams. More importantly, the most mature DevOps teams are delivering better customer experiences.

DevOps Isn’t Just One Thing
DevOps would be easier to implement if there was a single definition beyond “dev and ops working together.” Some organizations make DevOps part of a single person’s role when effective DevOps really requires a team effort.
“You’re tearing down walls in order to deliver value faster to your customers,” said Chou.

mabl’s report underscores the importance of automated testing. A lot of teams have accelerated their development cycles so they’re delivering features faster, but there’s not enough time left for testing especially when they’re still doing it manually.
The most sophisticated DevOps teams have a definite speed advantage because they’re releasing weekly to multiple times per day. Waterfall teams are releasing monthly, Agile teams are releasing weekly or monthly, and Emerging DevOps teams are releasing weekly. While the speed gains are necessary from a competitive standpoint, without adequate testing, teams are just releasing dubious quality software faster. Yang said the biggest change from 2019 is the rise in once per day releases.

More Types of Tests Should Be Automated
mabl’s report also examines the types of tests being automated in the CI/CD pipeline. The most common are earlier stage tests including unit (41%), integration (34%) and API (33%) tests. Later stage tests including functional (30%), system (22%) and performance (19%) testing are more likely to be done manually because they’re harder to automate.
“Functional end-to-end tests are becoming more important,” said Yang. “The problem is that creating end-to-end test scripts is a specialty that you’re unlikely to find in a single person because that person needs to be skilled in both dev and QA. mabl democratizes testing, so developers, QA and even product managers can easily automate functional end-to-end tests.”

The report reveals that the more automated tests DevOps teams are doing, the happier their customers are. In addition, the most mature DevOps teams can roll back a release in less than an hour, which is faster than less mature teams. Faster rollbacks lower developers’ stress level (30% for DevOps, 14% for Agile and 7% for Waterfall). Similarly, the amount of application monitoring in production correlates with how confident respondents are in their team’s ability to assure product quality.

“About half of the managers and testers who had amazing customer happiness scores were able to use the monitoring tools for basic querying,” said Yang. “The majority of respondents with ‘terrible’ CX regardless of their role did not know how to use such tools.”

Application Quality is Everyone’s Job
The most sophisticated DevOps teams can deliver higher quality code faster than their peers. To achieve that, they’ve made speed and quality everyone’s job. Specifically, the speed and quality goals are baked into the team’s culture and facilitated by processes that align with those goals, including automating more types of tests.

“If you’re doing DevOps as a team, but then you’re failing to bring that value to your customers, then have you reached your goal? Is it the end of your transformation if you’ve automated everything but it’s still not affecting customer experience?” said Yang. “You haven’t hit your goals and you haven’t finished your journey unless automated tests are included in your CI/CD pipeline, not just integration and API tests but functional and performance tests.”

Learn more at www.mabl.com