• Hybrid Tester/Architect: A test architect aligned with the application architect can help determine, based on the application footprint, what the next automated components and test assets should be, to better manage the overlying test framework and promote use of reusable automated assets whenever possible.
• Don’t sleep on metrics: Metrics ingrained within the automation process can create quality gates to maintain a well-defined quality state. Without measuring how automated tests are performing to make actionable improvements, testers run the risk of promoting defects faster through the testing pipeline.
• Practice, practice, practice: Virtualization is the testing equivalent of a flight simulator, allowing simulation of every possible user experience. The better understanding developers and testers have of where problems may occur, the more prepared they’ll be.
Subtle benefits and hidden obstacles
Everyone knows Continuous Delivery and testing speed up the development cycle. Everyone knows you need to automate. Neotys’ Hinds and HP Software’s Emo laid out a few of the advantages people wouldn’t immediately associate with continuous testing, and some of the more subtle challenges to doing it right.
Benefit: Avoiding late performance problems: “It’s always cheaper to make changes earlier in development than to have something deployed to production and going back to add a hotfix. It also allows people to make sure that whenever you’re releasing new features into production, you’re not allowing any sort of performance regression; not allowing old bugs to creep their way back in.” —Hinds
Benefit: Mitigating technical debt: “If you’re seeing load, performance or security issues early on, you’re likely not to let them propagate or let them get consumed in other composite applications. It also creates an interesting conversation between your tester, your developer and your product manager really pushing on those user story functions, really pushing on the requirements.” —Emo
Challenge: Shorter development cycles: “When moving performance testing to continuous environments, testers need to adapt. You’re getting a new build way more often than you are in a more traditional waterfall environment, though you’ve got to do basically the same number of tests you were doing before, except now you’ve got them every two weeks or less.” —Hinds
Challenge: Skill set: “Making sure you have the folks with the level of understanding needed to be able to do this kind of testing, but also to engineer the process, the infrastructure. There is a special skill in being a really good tester. They need to put in place the continuous integration process connected to your test automation suite and connect it back into your ALM system so you know the results the next morning and you’re able to act on it.” —Emo