Applying TDM to microservices is quite challenging. This is due to the fact that an application may have many services, each with its own underlying diverse data store. Also, there can be intricate dependencies between these services, resulting in a type of ‘spaghetti architecture.’
For these systems, TDM for end-to-end system tests can be quite complex. However, it lends itself very well to the continuous TDM approach. As part of this approach, it is key to align TDM with the test pyramid concept.
Let’s look at the TDM approaches for tests in the various layers of the pyramid.
TDM Approach for Supporting Microservices Unit Tests
Unit tests test the code within the microservice and at the lowest level of granularity. This is typically at a function or method level within a class or object. This is no different than how we do unit testing for other types of applications. Most test data for such tests should be synthetic. Such data is typically created by the developer or software development engineer in test (SDET), who uses “as-code” algorithmic techniques, such as combinatorial. Through this approach, teams can establish a high level of test data coverage. While running unit tests, we recommend that all dependencies outside the component (or even the function being tested) are stubbed out using mocks or virtual services.
TDM Approach for Supporting Microservices Component or API Tests
This step is key for TDM of microservices, since the other tests in the stack depend on it. In these tests, we prepare the test data for testing the microservice or component as a whole via its API.
There are various ways of doing this depending on the context:
- Generate simple synthetic test data based on the API specs. This is typically used for property-based testing or unit testing of the API.
- Generate more robust synthetic test data from API models, for example, by using a test modeling tool like Broadcom Agile Requirements Designer. This enables us to do more rigorous API testing, for example for regression tests.
- Generate test data by traffic sniffing a production instance of the service, for example, by using a tool like Wireshark. This helps us create more production-like data. This approach is very useful if for some reason it isn’t possible to take a subset of data from production instances.
- Generate test data by sub-setting and masking test data from a production instance of the service, or by using data virtualization. Note that many microservice architectures do not allow direct access to the data store, so we may need special data access APIs to create such test data.
Regardless of the approach, in most cases test data fabrication for a microservice must be prepared by the developer or producer of the microservice, and made available as part of service definition. Specifically, additional APIs should be provided to set up the test data for that component. This is necessary to allow for data encapsulation within a microservice. It is also required because different microservices may have various types of data stores, often with no direct access to the data.
This also allows the TDM of microservices applications to re-use test data, which enables teams to scale tests at higher layers of the pyramid. For example, a system or end-to-end test may span hundreds of microservices, with each having its own unique encapsulated data storage. It would be very difficult to build test data for tests that span different microservices using traditional approaches.
Again, for a single component API test, it is recommended that all dependencies from the component be virtualized to reduce the TDM burden placed on dependent systems.
TDM Approach for Supporting Microservices Integration and Contract Tests
These tests validate the interaction between microservices based on behaviors defined in their API specifications.
The TDM principles used for such testing are generally the same as for the process for API testing described previously. The process goes as follows:
For contract definition, we recommend using synthetic test data, for example, based on the API specs, to define the tests for the provider component.
The validated contract should be a recorded virtual service based on the provider service. This virtual service can then be used for consumer tests. Note that in this case, a virtual service recording forms the basis of the test data for the consumer test.
TDM Approach for Supporting an X-service System Test or Transaction Test at the API Level
In this type of test, we have to support a chain of API calls across multiple services. For example, this type of test may involve invoking services A, B, and C in succession.
The TDM approach for supporting this type of test is essentially the same as that for a single API test described above—except that we need to set up the test data for each of the services involved in the transaction.
However, an additional complexity is that you also need to ensure that the test data setup for each of these services (and the underlying services they depend on) are aligned, so the test can be successfully executed. Data synchronization across microservices is largely a data management issue, not specific to TDM per se, so you need to ensure that your microservices architecture sufficiently addresses this requirement.
Assuming data synchronization between microservices is in place, the following approaches are recommended to make test management easier:
- As mentioned before, use model-based testing to describe the cross-service system tests. This allows you to specify test data constraints for the test uniformly across affected services, so that that initial setup of test data is correct. This is done using the test data setup APIs we discussed above.
- Since setting up test data definition across services is more time consuming, I recommend minimizing the number of cross-service tests, based on change impact testing. Run transaction tests only if the transaction, or any of the underlying components of the transaction, have changed. Again, this is a key principle of continuous testing that’s aligned with the test pyramid.
- If there have been no changes to a participating component or underlying sub-component, we recommend using a virtual service representation of that component. This will further help to reduce the TDM burden for that component.
TDM Approach for Supporting End-to-End Business Process or User Acceptance Tests
The TDM approach for these tests is similar to that for system tests described above, since user actions map to underlying API calls. Such tests are likely to span more components.
Many customers prefer to use real components, rather than virtual services, for user acceptance testing, which means that the TDM burden can be significant. As before, the key to reducing TDM complexity for such tests is to reduce the number of tests to the bare minimum, using techniques like change-impact testing, which was discussed above. I also recommend you use the change-impact approach to decide whether to use real components or their virtual services counterparts. If a set of components has changed as part of the release or deployment, it makes sense to use the actual components. However, if any dependent components are unchanged, and their test data has not been refreshed or is not readily available, then virtual services can be considered.
