Not so fast when it comes to testing in the cloud
January 31, 2012 —
(Page 1 of 4)
Related Search Term(s): cloud, testing
Signs of a “Testpocalypse” loom on the horizon, Alberto Savoia (costumed as the Grim Reaper) told the audience. It was the keynote at the Google Test Automation Conference last October, where the theme was “Cloudy with a Chance of Tests.”
“Hiring and recruiting for testers is waaaay down. Testers are being commoditized, and there’s an exodus of test leadership,” he said. Finally, “More and more companies are shifting to ‘FrAgile’ post-agile ‘testing.’ ”
Indeed, outsourcing quality assurance is now the norm for 70% of respondents to the June 2011 World Quality Report conducted by Capgemini, HP and Sogeti. But cloud adoption is causing new demand for testing and QA, according to the survey.
“The day of the performance tester is going to come back,” said Kelly Emo, director of application product marketing for HP Software. “When you think about what cloud brings to the equation of testing, there are a couple of levels. For teams developing apps where part or all of it will be running in the cloud, that’s a whole new level of concerns across all three aspects of testing: functional, performance and security. The other level is, how can you harness the cloud to make you more effective as a tester?”
Savoia isn’t really a pessimist about testing, but he does see it as a rapidly changing field. Even the name “tester” is suspect, he said, and should be replaced by something new to reinvigorate awareness of the value of people who break software for a living.
According to Emo, “I think that what’s happening is that people are categorizing too much. More testers are becoming developers, and developers are becoming testers. Testers today have to know the fundamental architecture, or whether there are pieces running on iOS or Android hitting their app in the cloud. And to be more responsive to business, developers need to focus on regression testing, unit testing and basic functional testing so that testers can focus on high-value fringe cases, integration or end-to-end, and exploratory testing.”
The cloud over testing
In keeping with its stepchild relationship to application development, testing was not on thought-leader lips when cloud hype thundered in a few years ago. Yet it’s now clear that testing may in fact be one of the best cases for cloud adoption, exceeding that for deployment.
“This is an interesting time to be in cloud computing," said Brian White, VP of products for Skytap, a cloud automation startup founded more than four yeas ago in Seattle.
"Because of the concern around security of data, enterprises are looking at cloud in the dev and test area, not for touching customer-sensitive data that’s in their production apps."
According to Brett Goodwin, Skytap’s VP of marketing and business development, the No. 1 use case for Skytap’s 150 customers is development and testing. “After that, the customer can deploy their tested apps in their own data center or on to some other cloud. But the more common scenario is they move those final apps into their own environments.”
Vendors such as Keynote Systems, LoadStorm, Parasoft, Skytap, SOASTA and more are the first to see where companies are placing their cloud bets. Private clouds are definitely popular.
“I’d say 15% of our customers are using private or hybrid cloud for load generation, but to deploy to private cloud, it’s way less,” said Dan Bartow, vice president and chief evangelist of SOASTA.
"But this year, for the first time, SOASTA has two or three paying customers that have private clouds that they use for deployment. It’s on our road map to integrate with a private cloud API, such as Nimbula’s."
It’s all part of the evolution of virtualization, according to Theresa Lanowitz, founder of the San Francisco consultancy Voke. She characterizes three stages of virtualization, starting with the now-mature server market, which targeted data centers and was dominated by VMware. Then there’s application and desktop virtualization, which Citrix dominates, whereby a private cloud service delivers desktops and apps to any user on any device. Here, the primary users are service centers. And at the earliest stage of adoption is life-cycle virtualization, where vendors are competing for primacy. The user base? Development, operations and testing.
Virtual lab management is the Holy Grail for application life-cycle management, Lanowitz posits. That ties in with the Skytap story. “If you’re doing a hard press on testing, it’s not feasible to give the test team multiple labs and environments. But with Skytap it’s possible to create these complex environments with multiple machines, load balancing, firewalls, etc. and simulate the production environment earlier in the development cycle,” said White.
Another challenge in the heterogeneous landscape of cloud computing is when the developer can’t reproduce the multi-tier test environment, he explained. “With Skytap, that entire environment can be quickly saved as a template," he said.
"We found our customers like the ability to quickly create a snapshot of an entire virtual data center, including all of its machines, data, memory state, network settings and application state. You can save that off, then a developer can start up their own copy. It’s a really efficient way—if the QA team finds a bug—for the dev team to reproduce the bug in its own environment.” One such feature is the “publish” button, which saves the configuration as a URL.
With Skytap, a development manager can create a template for multiple standardized development and test environments in the Skytap Cloud Library, from which developers and testers can quickly serve themselves to new environments. Destructive or security tests are no problem: The configuration is network fenced and can be redeployed from the source template to return to a clean slate. Further, the management console offers a host of administrative features such as seeing compute and storage consumption of teams broken down over time.
“A lot of companies worry that when they go to cloud, users will be using resources willy-nilly,” said White.