“The app ecosystem has transitioned to where it no longer matters how many apps each OS has, but rather the satisfaction the user gets out of them. Going forward, we can compare where the apps were and where they currently stand from a user’s perspective.”—Piper Jaffray analyst Gene Munster, January 2014
According to a recent Google study, 52% of mobile users say a bad mobile experience will make them less likely to engage with a company. No longer is it a question of who has the most apps in their app store, now the question has become: “Who has the best apps?” Thankfully, mobile performance testing and monitoring has evolved to help ensure the highest levels of mobile quality.
In the early days, testing on mobile devices involved visual validation testing of onscreen images. This worked up to a point, but fell far short of providing the kind of insights testers and QA teams need to triage any problems that might arise, and to get a sense for what real users were experiencing. Today, a range of cloud-based, automated testing solutions are available that allow for object-based, image-based and optical character recognition scripting, providing testers with an efficient way to achieve a multi-dimensional view into everything that might negatively affect the mobile user experience.
The early days of mobile quality
Going back to the beginning of the mobile era, mobile quality referred strictly to call quality. This was back before 4G, 3G or even 2G, when coverage was spotty and national networks were still in the initial build-out phase. (Who doesn’t remember the “Can you hear me now?” commercials?) Before the mobile Web began to fulfil its promise and mobile apps expanded with more robust offerings in areas like financial services, travel and entertainment, near-ubiquitous wireless broadband infrastructure had to be developed, and handsets had to be optimized to manage the newly expanded offerings. Once the carrier networks completed the build out of their 3G networks, and the newer mobile broadband standards like UMTS replaced the earlier limiting 2G infrastructure, the app era was in full swing.
(Related: Mobile testing is no monkey business)
Mobile app quality testing has been around since the early days of the mobile Web as well. Companies like Nokia, Siemens, Sun and Motorola all believed in the importance of establishing agreed-upon standards for the Java ME apps that were running on mobile phones as early as 2004. While they weren’t as robust or plentiful as the mobile apps that populate our smartphones today, Java MIDlets (as they were known back then) were the forerunners of today’s mobile apps.
Even then it was clear that ensuring a quality user experience across the wide variety of handsets, networks and platforms would be key to increased user adoption and continued growth, so handset manufacturers banded together and created the first widely accepted set of mobile quality standards under the name Java Verified. While Java apps have since mostly disappeared, mobile quality standards and the importance of ensuring a high-quality user experience were ideas that are still with us today.
The app store and the rating revolution
The opening of Apple’s iOS App Store on July 10, 2008, marked a new era in mobile quality. While earlier versions of an “app store” had existed on some carriers’ older feature phones to allow downloads of Java ME and Brew apps (such as wallpaper, games and messaging), never before had so many apps been available to so many people—and all with the ability to rate their performance!