That leads us to Continuous Testing. It’s not a new phrase; one might call it a somewhat overloaded phrase. A short blog post by Scott Johnson at Puppet Labs, “No Nasty Surprises: Continuous Testing & Continuous Integration for Successful Release Management,” focuses on Continuous Delivery, and states that:
Whether you use simple testing tools or complex ones, the message is clear: Test early and test often, and make sure you incorporate load and performance tests into your continuous integration process.
#!That’s not what Parasoft’s Wayne Ariola and Cynthia Dunlop mean in their 40-page book, “Continuous Testing.” They say:
Continuous Testing provides a real-time, objective assessment of the business risks associated with an application under development. Applied uniformly, Continuous Testing allows both business and technical managers to make better trade-off decisions between release scope, time, and quality.
Generally speaking, Continuous Testing is NOT simply more test automation. Rather, it is the reassessment of software quality practices—driven by an organization’s cost of quality and balanced for speed and agility. Ultimately, Continuous Testing can provide a quantitative assessment of risk and produce actionable tasks that will help mitigate these risks before progressing to the next stage of the SDLC.
Continuous Testing can help your organization answer the following questions at the time of the critical “go/no-go” decision for a software release candidate:
• Are we done testing?
• Does the release candidate achieve expected quality standards?
• What are the quantifiable risks associated with the release candidate?
• How confident are we that we won’t end up in the news for software failures?
Ariola and Dunlop nail the target: It’s all about risk. That’s what insurance is all about, that’s what attorneys are all about, that’s the sort of decision that every business and technology manager makes all day, every day. We have to live with risk and make tradeoffs. More testing? At some point, indeed, we have to cut it off.
It’s difficult if not impossible to assess the business risk of software quality. Yes, software quality is expensive. The higher the quality, the more time it takes to deliver software, and the greater the resources you must spend on software quality. And yes, it is expensive to have software failures—you might lose money, lose customers, suffer lawsuits, damage your brand, end up on the front page of The Wall Street Journal. Not good.
Parasoft’s objective here is to build awareness for the company’s Service Virtualization API for test environment provisioning and test execution. As the company wrote in September 2013: