A mobile testing strategy is not complete without testing the integration between the application and back-end system. This is especially true when the release cycles of mobile apps and back-end systems are very different, which they often are.
Manual or automated
A lot of basic compatibility and basic functional testing can be done efficiently with manual testing, but when it comes to testing lots of devices and applications that need to be retested frequently, automation can be an efficient way to scale. The efficiency gain will depend on the experience and skill of the automation team—the standard disclaimer “results may vary” is even more applicable to mobile test automation due to all the variables. Also, various test automation tools will impact your choices of emulators vs. real devices.
A big challenge of mobile testing is sourcing and then management of devices. Creating the initial matrix is just the beginning. It’s common for each manufacturer to introduce three or more new devices each year, and, on average, devices are upgraded every two years. For most companies this makes it impractical to maintain an inventory of devices. The growing numbers of cloud service providers make it possible to completely “outsource” device management, and are a good way to go most of the time. However, there are limitations to relying solely on device rental. An option is to own a manageable number of the key devices for a majority of testing and then utilize devices in the cloud for basic compatibility and functional testing. The knowledge and research for doing this is a big task.
Fully outsourced option
Completely outsourcing mobile testing is a strategy that works well for a lot of organizations. This eliminates the challenges and headaches of managing and maintaining an inventory of mobile devices. Firms with mobile specialists typically understand the unique device and emulator testing nuances, and likely have mobile automation expertise as well. Better firms, because of their experience, can also help develop the device and testing matrix that will provide the optimum test coverage at the lowest cost.
Mobile is rapidly becoming the primary user interface; with the Mobile First movement, it already is the primary interface, which means mobile testing will continue to increase in importance. Applying a thoughtful approach and rational analysis will go a long way in developing a mobile strategy that will provide the right level of testing.