Report: Test automation coverage has rebounded after a dip last year

Test automation coverage has rebounded after a dip last year, according to SmartBear’s State of Quality Testing 2022 report. 

SmartBear conducted a global online survey over the course of five weeks earlier this year. The findings are based upon aggregated responses from more than 1,500 software developers, testers, IT/operations professionals, and business leaders across many different industries.

Last year saw the amount of companies performing manual tests at 11%, while that number dwindled to 7% this year, almost returning to pre-pandemic levels of 5% of all tests being performed completely manually. 

This year also saw slightly higher numbers than ever before for respondents that said 50-99%  of their tests are automated across the board. The biggest jump happened in the 76-99% group which jumped over 10% to 16% over the last year. The amount of respondents that said their tests are all automated regained some ground to the pre-pandemic level of 4%.

When looking at the different types of tests and how they are performed, over half of respondents reported using manual testing for usability and user acceptance tests. Unit tests, performance tests, and BDD framework tests were highest among all automated testing. 

Another finding is that the time spent testing increased for traditional testers but decreased for developers. However, the average percentage of time spent testing remained the same as last year, at 63% across the organization.

QA engineers/automation engineers spend the most time testing, averaging 76% of their weeks on testing up from 72% last year. While the trend for developer testing inched up between 2018 to 2021, reaching 47%, it sank to 40% this year. Testing done by architects plummeted from 49% to 30% over the last year. 

This year, the most time-consuming activity was performing manual and exploratory tests, jumping to 26% from 18% last year as the most time-consuming task. In the same time period, learning how to use test tools as the most time-consuming challenge with testing fell from 22% to just 8%. 

The biggest challenges that organizations reported for test automation varied by company size.  Companies with 1-25 employees cite “not having the correct tools” as their biggest challenge, while companies with 501-1,000 employees cite “not having the right testing environments available” as their biggest challenge. These are different from the biggest problem that was cited last year “not enough time to test” at 37%.