Using agile development methods, continuous build and deployment, organizations can produce several builds in a day. But if it takes 30 hours to test a build, how do testers get it done by tomorrow? How do they keep up?
By leveraging the automation capabilities of the very tools that are threatening to bury testers, to be able to do more testing and increase test efficiency.
A couple of years ago, I was testing the customer-facing and internal-facing portals for a company that provided VoIP services. It took about 30 minutes for me to manually deploy a Git Tag to my virtual Unix/Tomcat test environment. After that, it took about 60 hours of testing to go through the major scenarios and validate the results.
Eventually I cut it down to 30 hours of testing by adding some automated scripts to prepare the database with the test accounts and profiles that my test scripts needed. This saved me hours of manual data entry in the portal interface. I also added some automated scripts to help quick-test the basic portal features, but the real testing had to be done manually.
Sales and Marketing were evolving the “perfect” feature package plans. As a result, the customer’s billing statement changed with every new build. The requirements were fuzzy and constantly changing as they tried out different ideas.
Also, as we neared the release date, I discovered that certain rapidly proliferating inventory items in the production database could not be deleted when they ceased to exist in the real world. So, we started testing database fixes that could not be completely verified in testing due to the differences between the QA and production environments, meaning it would have to be tested again in production.