Apple’s iOS 8 is here, and it will be a game-changer. No operating system on the planet gets adopted faster than a new iOS release. If iOS 7 was any indication (60% of users upgraded just a few days after release), iOS 8 may set new records.
Even before worrying about adding new capabilities to apps, this upgrade tsunami underscores the importance of ensuring existing iOS 7 apps run as expected on iOS 8. Adding to that, Apple is releasing two new iPhones with new form factors (which means that developers will have to ensure apps are optimized for each new device), and the release of iOS 8 quickly becomes the regression-testing event of the millennium.
(Related: Apple previews iOS 8)
Oh, and by the way, it will get even more complex with more than 4,000 new APIs. Developers will race to get these into their next release. Be ready for this. Some of these amazing new features include interaction with other devices and services (e.g., payment services and fitness wearables).
Whether dealing with the ultimate regression test of the new millennium, or new features that are being added to your new app, below are the key best practices to think about incorporating into your testing process for Apple’s new iOS 8 and beyond.
1. Think like a user mobile user. Mobile app testing is fundamentally different from traditional Web app testing. This is driven by user expectations.
At the core of these expectations is the belief that there is little to no tolerance for anything less than an amazing user experience. This means successful apps have a game-like experience. This results in an increased relevance for ad hoc, fuzzy and exploratory testing as a first step.
2. Real device testing or emulators. While developers typically leverage emulators for testing, traditional QA knows there is no substitute for real devices. This means both have their place as part of the process.
For early testing and distributed teams, nothing can replace the convenience of emulators. However, they typically don’t replicate many core capabilities that should be tested. Recent advances with test cloud libraries of online devices replaced many of the emulator advantages and eliminated the need to purchase and manage your own library of devices. The bottom line is that both emulators and real device testing should play a key role.
3. Create a set of smoke tests for your existing apps. Building on top of a bad foundation is just bad hygiene. Every mobile app release plan (and especially one for iOS 8) should perform a set of smoke tests (whether automated or not), covering basics such as app launch, login, key first common action, and logout.
4. Get your developers involved beyond just unit testing. There’s a long-held belief that whoever created the app shouldn’t be trusted to test it. The truth in mobile is there are many elements of the quality process, given the release velocity and relatively small size of development teams, that makes developers best positioned to participate in the testing process. Moreover, with advances in mobile APM solutions, developers can only leverage real-time feedback to developers from app users in the wild about crashes and exceptions.
5. Automate, automate, automate. By most estimates, organizations have small, single-digit percentages of test automation for mobile apps. This will only increase going forward.
With the introduction of iOS 8, integrations of third-party services and devices with smartphones and tablets (e.g., health bands, payment services, etc.) are increasingly complex. Chose your automation battles wisely. The old rule of thumb was that doing something simple seven times meant it was a good candidate for automation. Given the accelerated releases of mobile, there are many repetitive activities with apps and iOS 8 that are ripe for automation. Don’t dive into new complex use cases for automation.
6. Continuous Integration should be key in any test automation effort. If you’re going to automate, why not go all the way? Why spend all the time creating durable, reusable automated tests and not make the additional investment to integrate these into a build process to be kicked off upon a successful build? It makes no sense.
The mobile iOS 8 party is just starting. Adapting some of these best practices can help expedite the adoption of iOS 8 without compromising your users’ experience when they upgrade.
