Test environments can be a frustrating bottleneck to the testing process and the software development life cycle as a whole. Whether it be unavailable services, devices, or ever-elusive test data, ensuring the right environment for testing creates potential for barriers to shifting left at speed, and cutting corners can put application quality and your business at risk. 

A recent study of 1,000 software developers and startup employees found that at least 29% of organizations are using real customer production data in their testing environments. This poses numerous concerns because utilizing real customer data for testing opens the door for violation of GDPR regulations. This, in turn, can lead to loss of resources and reputation for companies. Furthermore, utilizing real data can be disastrous in the event of a data breach – which 45% of companies report experiencing.  

Luckily, there are several steps testers and dev teams can take to ensure they are both eliminating these concerns and testing efficiently.  

This article will take a closer look at some of the most common environment concerns facing testers and dev teams—including the acquisition of usable test data—and explore solutions to eliminating these concerns that fit seamlessly into your CI/CD pipeline. 

Intelligent mocks

Problem: Traditional mocks are too simplistic; legacy service virtualization is too complex 

Traditionally, technical teams have utilized mocks and stubs during the development and testing of their mobile apps. Mocks act as a response to external dependencies that are part of the application’s flow (databases, mainframes, etc.) but are not pertinent to the test at hand. Teams have used mocks so that developers can focus on their code’s functionality and not get sidetracked with these external dependencies. 

Traditional mocks and stubs are limited, however. They provide a simple response to the external dependency to keep the testing moving along. Mocks and stubs do not effectively test real-world scenarios because they do not consider the varied conditions that can arise outside of their particular response. 

But what if you want to test more “real world” conditions? 

Service virtualization allows more in-depth testing than traditional mocks and stubs. However, even if you have access to an expensive service virtualization solution, it will undoubtedly be complex and typically require specialized training or even on-site expertise to facilitate. As such, testers can be stalled in their testing process when waiting for virtual services experts to provide the required virtual services. 

Solution: Mock services are the shift-left answer 

Intelligent mocks, or mock services, are the ideal solution for teams looking for greater agility in their testing process. Intelligent mocks combine the capabilities of mocks and service virtualization to create a testing solution that emulates the behavior, data, and state of external dependencies. You can easily create a slow or garbled response to replicate unexpected real-world conditions, ensuring the application under test is ready for production.  

Mock services are simple to create – simply upload well-known industry specification files such as a Swagger file, WSDL file, or request-response pairs, and create a recording, use the template, or use one of the pre-built common services. Then, share services across the enterprise in an asset repository. These stored intelligent mocks can then be easily accessed for subsequent tests during all stages of the software development lifecycle.

Synthetic Test Data

Problem: Testers have incomplete, incorrect, or unavailable test data

Do any of the following problems sound familiar to you? 

  • Our tests often fail due to outdated or incorrect test data sets. 
  • I cannot test changes to my app early enough because I have outdated test data and no way to create new data sets. 
  • The test data I was provided does not contain unique IDs as expected and that broke my test and delayed my release. 

These problems are just a few of the many issues facing testers when it comes to locating test data for testing mobile apps. While some frequent concerns for mobile testing include requiring a testing environment that is not yet ready or other departments not prioritizing the resources you require, by far the most common concern for testers and dev teams is the lack of relevant or complete test data. 

Many organizations rely on test data management (TDM) systems to create and deliver data; however, this can often result in a wait of days or weeks while the agile testing team waits for the DBA to complete the data task. This often creates substantial delay in release cycles.  

Additionally, with the onset of regulations around personal identifiable information (PII), the challenge to testers lies in creating reliable test data that does not contain any PII. To get around this issue, organizations are trending toward using synthetic data.  

Solution: Having realistic, reusable test data on demand 

When adopting a continuous testing platform, the best options include the ability to generate realistic synthetic test data on the fly for various types of tests and synchronize that data across various components involved in testing. These include the test itself, the test environment, and external dependencies so that testers can work faster and more efficiently. Furthermore, testers can ensure that their app is being tested against relevant, real-world data while alleviating bottlenecks and dependencies in their CI/CD pipeline. 

Some points to consider: 

  • Ideally, a testing platform will be able to quickly generate synthetic data that mirrors real-world data.  
  • Test data generated will be usable across various tests (e.g., functional and performance) and can be reused for future tests.  
  • Synthetic data generation allows teams to be agile and save time and resources by focusing on the test itself—rather than waste resources generating test data.   
  • Testers will be able to work with comprehensive test data with desired variety to achieve better and more robust tests.  
  • Synthetic data generation eliminates PII concerns. 
  • A testing platform ensures that the test data that drives the test is consistent with data in the test environments and external services. 

The key to choosing the right source for your synthetic test data is to adopt a platform that allows you to produce synthetic data to your exact specifications on demand, but also synchronizes the data across tests, environments, and external or mock services. Synchronized data can be reused after your initial test is complete because it resets to its original format and remains referentially intact. For instance, names, addresses, and credit card numbers from the synthetic data set will reset to their original form and will be ready to use in subsequent tests. This process is very cost-effective and saves time since you only have to generate data once for use across multiple tests. With synchronized, synthetic data at your fingertips, testers can eliminate the biggest roadblock to effective testing. 

Virtual Devices  

Problem: Teams want to release high-quality applications more quickly  

While there is no replacement for testing on real devices—particularly during later stage functional and UI tests—testing on simulators and emulators in the early stages of development is an efficient and cost-effective way to speed up the mobile application testing process. Testing on virtual devices earlier in the development lifecycle allows testers to locate glitches and bugs sooner. Furthermore, utilizing virtual devices allows testers access to a broader range of devices as well as access to devices that might otherwise be reserved by another member of your organization.  

Solution: Virtual devices to augment your comprehensive real device lab  

Investing in virtual devices to augment your real device lab is a smart move for testing teams looking to create high-quality mobile apps faster.  

Virtual devices are well-suited for unit testing because simulators and emulators provide quick and relevant feedback in the early stages of development. In addition, a combination of real and virtual devices can perform integration testing, including performance and accessibility testing, quickly and efficiently.  

By testing on a combination of real and virtual devices utilizing the services of a supported virtual device lab—in tandem with your comprehensive real device lab—testing teams can test efficiently at all stages of the software development lifecycle. 

Bottom Line

When it comes to creating high-quality applications that compete in a global marketplace, testing teams must find ways to eliminate common environment concerns that stand in the way. Mock services allow teams to bridge the functionality gaps between traditional mocks and stubs—which are limited—and legacy service virtualization which creates barriers to shifting left—in order to become more agile. When combined with on-demand synthetic test data and complete with synchronization, testers will have the tools and data needed to perform tests throughout the SDLC. Finally, supplementing your real devices with virtual devices allows teams to speed up their testing process and test early and often.  

To learn more about eliminating barriers to application quality, request a free trial at Perfecto.io.