SmartBear Software is gearing up to release its flagship testing survey tomorrow, and this year’s results paint the picture of the testing industry from end-to-end. The company shared its findings with SD Times in advance, and we dove into the results to learn how the disruption of software development is impacting testers, developers, and operations teams today.
SmartBear’s survey garnered a total of 5,219 responses, which included data from developers and QA professionals from a variety of job functions, levels, company sizes and industries globally.
One high-level takeaway is that over half of QA engineers and software developers in test are responsible for testing on both the front-end and back-end of their applications. In fact, 44 percent of testing teams have two to five people and more than half of development teams have two to 15 people that are also responsible for testing. This is partly because the growth of methodologies like Agile and DevOps blurs the lines between roles like testing and QA and development, according to SmartBear.
A majority of these responders also said that they are responsible for both API and UI testing (63 percent of respondents):
“Based on job title, there is a small shift in the focus of testing responsibilities. Developers, Project Managers, and Consultants are more likely to test only APIs, while Manual Testers and QA Managers are more likely to test only the UI of an application. Of the QA Engineers that participated in the survey, 68% said they were responsible for both back-end API and UI testing.” – SmartBear State of Software Testing Report
According to SmartBear, this forces teams to put the growth of trends like “shifting left” into perspective — testing needs to be performed earlier in the lifecycle, and teams need to keep the mantra “test early and often” in mind. (It’s also important to note that much of the data presented from SmartBear refers to the responses of the individuals whose job it is to conduct both API and UI testing, and in some key instances, this differs from the responses of those who focus strictly on one or the other, said the company).
State of software testing – What’s being tested?
Despite the progression of mobile application development and testing, SmartBear found that web applications are the most common type of application being tested today. That’s because web technologies have gained a significant amount of traction among businesses and consumers. To meet the demand, teams are ramping up their web application development and testing, with 85 percent of survey respondents saying they are developing and testing web applications today.
SmartBear also found that B2B software is tested more than B2C software. Seventy percent of developers and testers are working on software intended for businesses, whereas only 60% are working on applications intended for consumers, according to the company.
Agile still dominates.
Even though it seems like all companies have a strong grasp on DevOps and Agile methodologies, SmartBear found teams are not as agile as they think they are. A majority of teams are embracing the agile approach to software development, but many do not implement some agile best practices. Teams are definitely utilizing continuous integration and automation, but a key indicator of true agility is the ability to deploy on a frequent and regular schedule, which SmartBear finds isn’t really the case for these survey respondents.
“While teams that follow a DevOps model are more likely to release multiple times a day, more than half of the teams that labeled themselves as ‘Agile’ are releasing applications on a monthly, quarterly, or even yearly basis. Interestingly, there was no significant difference between release cycles for teams that follow an Agile model versus teams that follow a Waterfall model. In fact, teams that follow other models (including ‘It Depends on the Project,’ ‘Waterfall,’ and ‘Other’) are more likely to release daily or multiple times a day than agile-labeled teams.” – SmartBear State of Software Testing Report
The future of testing.
When it comes to the future of software and application testing, respondents had some interesting answers to the open-ended question, “What will the state of software testing look like two years from now?”
According to one QA engineer, it will be all automation. Another manual tester said that manual testing will “make a resurgence following a reality check that not all tests can or should be automated.” One QA manager said that testing will push back from the trend of automate everything due to “diminishing ROI.” Many stakeholders do not see automation solving all of ITs problems, and they want skilled humans back into the pipeline, according to his response.
“It would look more flexible and adaptable to changing software demands. Better techniques will be used to test a system. AI will be an important part in software testing tools and many common test strategies can be detected by tools themselves. Automated test script creation will become creative and less time consuming. Nevertheless, manually testing a software will not become obsolete.” – QA Engineer / SDET
It’s clear from this report that QA, developers, and managers still view software testing as a challenge. Test stability, object identification and test maintenance are the top three challenges respondents face when automating UI tests, and having APIs return the correct response is still a top concern for API testers. In summary, with the right technologies, teams can meet client demand, pursue agile methodologies, and choose the right testing approaches and techniques needed to deliver a positive user experience, according to SmartBear. The full software testing report can be viewed and downloaded here.
