Testing-as-a-Service can provide a competitive edge

Test early and test often is a mantra circling around software development life cycles these days, but with the pressures to quickly deliver an application to market and constantly update it, testing doesn’t come easily. Not all organizations have enough employees or resources to keep up with their competition. It isn’t enough to just put an application out there; quality is important.

“Development and QA organizations have known for a long time that dedicated testing resources for individual development teams and projects [are] inefficient and leads to poor utilization of IT resources,” said Mughees Minhas, Oracle’s VP of product management.

So, how are organizations supposed to quickly get their products out there and ensure quality? For some, the answer is Testing-as-a-Service (TaaS).

What is Testing-as-a-Service?
Testing-as-a-Service, also known as on-demand testing, has different meanings depending on the solution provider.

SOASTA views TaaS as anytime an organization rents a test environment. According to SOASTA CEO Tom Lounibos, where the test is built doesn’t matter, nor does the actual person who builds or runs it.

“It’s the process of running a test that requires all the test servers,” he said.

Oracle sees TaaS as a cloud service where the customers do the testing themselves, but take advantage of cloud solutions to get the efficiency benefits.

“Depending on the type of TaaS solution used, customers can reduce hardware costs, testing efforts and time, while empowering application owners and IT operations teams to participate more in the testing process,” said Minhas.

(Related: The importance of testing)

TaaS often takes advantage of private and public cloud platforms in order to deliver automated application and software testing services to organizations when they need it. TaaS can be used for load and performance testing, functional testing, or security testing, or it can include all aspects of testing, including test lab management, test driver provisioning, and orchestration of test runs depending on a solution an organization chooses, according to Minhas.

TaaS can also imply outsourced testing, where a third party does testing on a customer’s behalf. “Testing-as-a-Service is a really strong way to enable the quality goals that users have without having to instantly produce a whole bunch of knowledgeable folks,” said Kelly Emo, director of product marketing for HP Software.
#!A deeper look into Testing-as-a-Service
The biggest factors that drive organizations to TaaS are ease of use, time, scaling and affordability. For a company to test an application, it would take up to six weeks or more to set up a test, build it and analyze it, according SOASTA’s Lounibos.

“Now you can build that test within minutes if not hours,” he said. “With Testing-as-a-Service, those six weeks are more like six hours for the complete process.”

Testing-as-a-Service becomes a huge asset to organizations that are launching a new product or offering two weeks down the line, for example, and don’t have the time to learn a new tool themselves or buy extensive software, according to Lounibos.

Some small organizations don’t necessarily have the skills, people or infrastructure required for testing, and they can’t afford to acquire them. TaaS provides experts on-demand.

“Testing-as-a-Service opens organizations up to a world with vendors and testers that have experience with tools that they don’t necessarily have experience with,” said Chris Riley, technology evangelist for CloudShare. “So organizations don’t have to integrate a testing tool into their process, they can defer that out to their service provider without changing anything in their flow.”

TaaS solutions can also fit into agile projects in order to help organizations in the agile process.

“TaaS allows an organization to become agile by enabling quick uptake of patches, new functionality or versions of the applications and all without sacrificing application quality,” said Oracle’s Minhas.

He explained that a large majority of an organization’s testing time can be spent on preparatory activities from organizing hardware for QA environment to installing operating systems, application servers and database servers.

“TaaS can streamline these activities by automatic deployment of the application under test, test tools and test scripts into the private testing cloud and can reduce the testing project cycle significantly, in some cases by more than half,” said Minhas.

TaaS can also work with continuous testing, according to Minhas, provided that the continuous integration system’s software builds are available to the TaaS platform.

Organizations are sometimes skeptical of TaaS because it forces them to share some private assets with a third party, like source code, design and more. Peter Klenk, software technology manager within IBM DevOps, describes it as a tradeoff. Organizations need to weigh the risk of potentially exposing those assets to the world versus building up their own test infrastructure. When choosing a TaaS provider, it is important to think about how established the provider is and the success it has had, he said.

But it is possible for companies to avoid sending their work to a third party with TaaS solutions. “TaaS has highly flexible deployment architectures,” said Minhas. “TaaS can be delivered in-house using a private cloud deployment model. In this deployment model, data will be kept on the organization’s own systems but still benefit from the self-service agility, automation efficiency and scalability benefits of TaaS.”

Another reason for skepticism regarding TaaS is that an organization might think its application is too complex. “Enterprises are afraid of Testing-as-a-Service because they argue that they can’t just take their library of business applications that all have to work together and package it up nicely for someone else to test,” said CloudShare’s Riley.

Organizations also need to make sure the developer and tester relationship is defined in TaaS.

“One of the big problems a lot of these Testing-as-a-Service solutions fail at is not the test part, but to test when there is a bug part,” said Riley. “So if something goes wrong, the tester needs to be able to communicate back to the developer efficiently because they might not have all the necessary information to fix it.”
#!What to look for when choosing a TaaS offering
TaaS is meant to be a solution for organizations to become more efficient with testing, and to decrease effort and costs. A good TaaS solution should include the following:
• Cloud: SOASTA’s Lounibos said the first thing organizations should look for when choosing a TaaS solution is whether it is cloud-based or not. According to him, cloud-based TaaS solutions enable easy access and availability to testing services.
• Self-service interfaces: These allow QA engineers and developers to request compute resources, fully functioning test assets and scripts, and test tools, according to Oracle’s Minhas.
• Outcomes: According to HP’s Emo, customers or organizations looking for TaaS solutions need to think about what their outcomes are and what their partner is going to deliver for them. “When we think about Testing-as-a-Service, we want to be able to deliver our customers the ability to define what their goals are from a business and quality perspective, and then very flexibly get the capabilities from us that deliver these outcomes,” she said.
• APIs: Fully automated cloud APIs enable test platforms to be quickly set up and torn down. Private and public cloud APIs allow test environments to be available internally or externally, according to Lounibos. Web service APIs enable integration with exciting QA procedures, according to Minhas.
• Rich service catalog: A catalog “where QA engineers and developers can select the exact application, scenario and platform they want to test,” said Minhas.
• Subscription access: Sometimes organizations only need to use aTaaS solution once, and sometimes they use it regularly. In order to make sure TaaS is affordable, organizations should see if the TaaS solution they are choosing offers business models that include hourly and annual rates.