Agile and DevOps change the game for software testing.It’s not just a matter of accelerating testing—it’s also about fundamentally altering the way that quality is measured. Agile requires teams to test faster and earlier. And DevOps demands a more deep-seated shift. The test outcomes required to drive a fully-automated release pipeline are dramatically different than the ones that most teams measure today.
“Sooner or later, Agile and DevOps teams realize they can’t achieve continuous delivery or continuous deployment without continuous testing,” said Wayne Ariola, chief marketing officer at Tricentis. “However, the path to that goal is often unclear—especially for enterprise organizations faced with complex legacy architectures,stringent compliance requirements and a long history of manual testing.”
To deliver the rapid feedback and proactive quality engineering that accelerates testing while protecting the business, the differences between automated testing and continuous testing must be understood and applied.
As a continuous testing leader, Tricentis is among the 2018 SD Times 100 for its outstanding contributions to the Testing category.
How automated testing and continuous testing differ
Continuous testing is the process of executing automated tests as part of the software delivery pipeline in order to obtain feedback on the business risks associated with a software release candidate as rapidly as possible.
“Test automation is designed to produce a set of pass/fail data points correlated to user stories or application requirements,” said Ariola. “Continuous testing focuses on business risk and whether the software should be released. To successfully shift from test automation to continuous testing, we need to stop asking, ‘Are we done testing?’ and instead concentrate on whether the release candidate has an acceptable level of business risk.”
Of course, continuous testing takes advantage of automated testing, but test automation alone cannot keep pace with changing business and technology requirements. For example, unlike traditional test automation, continuous testing requires the alignment of testing with business risk. continuous testing also applies service virtualization and stateful test data management to stabilize testing as necessary for continuous Integration. In addition, continuous testing enables exploratory testing that exposes “big block” issues early in each iteration which automated testing alone cannot do.
“Continuous testing is not just about more or different tools, it transforms processes and the way people work,” said Ariola.
How automated testing and continuous testing compare
Automated testing and continuous testing differ in three important ways: risk, breadth and time.
Business risk increases as organizations innovate faster for competitive advantage. The more functionality they expose to users, the greater the number, variety and complexity of potential failure points. Most automated tests provide low-level details about whether user stories correctly implement requirements. Continuous testing also ensures high-level assessments about the viability of a release candidate.
The breadth of testing also differs. Just knowing that a unit test failed or a UI test passed doesn’t reveal whether the overall user experience has been impacted by recent application changes. continuous testing ensures that the tests that are broad enough to detect when an application change inadvertently impacts functionality on which users rely.
Finally, time is of the essence. As software release cycles continue to diminish, fast feedback isn’t enough. To minimize the risk of faulty software reaching an end user, software teams must be able to process instantaneous feedback,which continuous testing enables.
“Continuous testing offers five clear advantages over automated testing alone,” said Ariola. “It allows you to assess business risk coverage, helps you protect the user from software defects and provides a stable test environment that’s available on demand. Continuous testing also seamlessly integrates into the software delivery pipeline and DevOps toolchain and delivers actionable feedback appropriate for each stage of the delivery pipeline.”
Why move to continuous testing?
Reinventing testing is an untapped opportunity to accelerate the delivery of innovative software. Most organizations today have already invested considerable time and resources in reinventing their development and delivery processes. However, testing is commonly overlooked in these transformations.
“Yes, this is a problem—but it’s also an opportunity,” explained Ariola. “You can further accelerate delivery speed by shifting your attention to testing. By transforming testing to an automated, continuous process, you better align your quality process with Agile, DevOps and other digital transformation initiatives. Plus, you also reduce business risk and free up significant budget that can be reallocated to underfunded innovation initiatives.”
Learn more at www.tricentis.com.
Content provided by SD Times and Tricentis.
