Before testers try out new methods that may expose software to many risks, their companies are asking themselves a big question: to test or not to test in production?
As companies move to implement development processes such as DevOps or Continuous Integration and Delivery, testing in production can become an important piece of the equation.
To put it simply, testing in production (TiP) means performing various types of software tests in a production environment where it is live and accessible to the end user. It’s rare to find a test environment that completely replicates a production environment, so the scale is not the same and won’t put your software through the same variables that affect performance as “real life.”
So, as with most processes in software development, experts in the field suggest considering the risks, the rewards and best practices before you test in production.
What’s the point of TiP?
Despite the risks, there are several benefits, which is why a tester should consider testing in production. One major benefit is that testing in production allows you to see how your application works in the live environment in which it runs.
“It would be like testing in development and then saying, ‘Well it works in development, so we don’t have to test in QA,’” said Brad Stoner, senior performance engineer at test company Neotys.
Testing in production can become a necessity when a service becomes more complex, as it would become difficult to mimic a production environment and know how the service will perform or be utilized by users. The staging environment and the real environment are never the same thing. According to Stoner, some organizations might not have the budget to recreate this type of environment that closely resembles the production environment, which is why companies and consultants switch to testing in production.
Stoner also said that testing in production gives a tester confidence in the test results because “You are actually testing against the environment that will take users versus testing in the lower environment.”