“The app ecosystem has transitioned to where it no longer matters how many apps each OS has, but rather the satisfaction the user gets out of them. Going forward, we can compare where the apps were and where they currently stand from a user’s perspective.”—Piper Jaffray analyst Gene Munster, January 2014

According to a recent Google study, 52% of mobile users say a bad mobile experience will make them less likely to engage with a company. No longer is it a question of who has the most apps in their app store, now the question has become: “Who has the best apps?” Thankfully, mobile performance testing and monitoring has evolved to help ensure the highest levels of mobile quality.

In the early days, testing on mobile devices involved visual validation testing of onscreen images. This worked up to a point, but fell far short of providing the kind of insights testers and QA teams need to triage any problems that might arise, and to get a sense for what real users were experiencing. Today, a range of cloud-based, automated testing solutions are available that allow for object-based, image-based and optical character recognition scripting, providing testers with an efficient way to achieve a multi-dimensional view into everything that might negatively affect the mobile user experience.

The early days of mobile quality
Going back to the beginning of the mobile era, mobile quality referred strictly to call quality. This was back before 4G, 3G or even 2G, when coverage was spotty and national networks were still in the initial build-out phase. (Who doesn’t remember the “Can you hear me now?” commercials?) Before the mobile Web began to fulfil its promise and mobile apps expanded with more robust offerings in areas like financial services, travel and entertainment, near-ubiquitous wireless broadband infrastructure had to be developed, and handsets had to be optimized to manage the newly expanded offerings. Once the carrier networks completed the build out of their 3G networks, and the newer mobile broadband standards like UMTS replaced the earlier limiting 2G infrastructure, the app era was in full swing.

(Related: Mobile testing is no monkey business)

Mobile app quality testing has been around since the early days of the mobile Web as well. Companies like Nokia, Siemens, Sun and Motorola all believed in the importance of establishing agreed-upon standards for the Java ME apps that were running on mobile phones as early as 2004. While they weren’t as robust or plentiful as the mobile apps that populate our smartphones today, Java MIDlets (as they were known back then) were the forerunners of today’s mobile apps.

Even then it was clear that ensuring a quality user experience across the wide variety of handsets, networks and platforms would be key to increased user adoption and continued growth, so handset manufacturers banded together and created the first widely accepted set of mobile quality standards under the name Java Verified. While Java apps have since mostly disappeared, mobile quality standards and the importance of ensuring a high-quality user experience were ideas that are still with us today.

The app store and the rating revolution
The opening of Apple’s iOS App Store on July 10, 2008, marked a new era in mobile quality. While earlier versions of an “app store” had existed on some carriers’ older feature phones to allow downloads of Java ME and Brew apps (such as wallpaper, games and messaging), never before had so many apps been available to so many people—and all with the ability to rate their performance!

As of this time last year, Apple announced that there have been more than 40 billion apps downloaded from their App Store, with almost half of those downloads coming in 2012. What this has meant for mobile quality is that developers have been forced to focus on the quality of their apps as a differentiating factor lest poor quality have the damaging effect of consistently low ratings—a sure kiss of death for any app.

The emergence of universal app quality standards
Recently, a group with roots in the field of mobile app quality extending as far back as Java Verified launched a new standards organization known as the App Quality Alliance (AQuA). Based on their years of experience, AQuA created a set of testing criteria and best practices for developers of Android apps that became the standard for app quality assurance. AQuA has since gone a step further, releasing in October 2013 a set of testing criteria for Apple iOS applications. The emergence of a widely-accepted standards-based testing regimen has been another important step in the evolution of mobile quality.

But perhaps AQuA’s main contribution, as its chairman Martin Wrigley noted in a recent interview, is in promoting a consistent message that app quality matters. “There’s a huge benefit in having a consistent message from the quality and testing industry, the handset manufacturers, the network operators and the platform owners, to actually push back on the developers and say, ‘You DO need to do testing. Here’s a minimum set of testing that you should do in order to make users happy… [so] you can then be sure that your app is going to work and you’re not going to disappoint your users, because there’s nothing worse than a disappointed user.’ ”#!Riding the cloud to better performance
Thanks to the emergence of cloud-based testing solutions, it is no longer necessary to equip QA and testing teams with a closet full of mobile devices to test for every variation in performance across networks, platforms and devices. Cloud-based testing on real devices has given testers access to the full functionality of each mobile device being tested and the full range of devices, all from the desktop. This has allowed developers to streamline their testing, reduce costs, and accelerate time to market.

Some testing providers have also recently begun offering pay-as-you-go pricing plans as a compliment to their premium testing offerings. This flexible pricing model allows smaller development shops and organizations whose primary focus is not on mobile to effectively test mobile apps and websites on real devices without a major commitment of resources, engendering greater levels of innovation and diversity in the app market.

Better testing and better quality through automation
With the rise of new automated testing technology, QA and testing teams can now quickly create and automate test scripts to capture, verify and replay real user interactions on live mobile devices. This allows for the creation of complex test scripts that include touch events, using device peripherals, making phone calls on the network, SMS texting, HTML5 capabilities, and other features that simulate real-life user experiences with only a limited commitment of resources.

When combined with cloud-based testing solutions, automation further empowers businesses by allowing developers, IT and QA teams to perform testing more efficiently from anywhere, on any advanced mobile device. Greater automation also frees up more QA and testing personnel to conduct more robust situational, exploratory testing.

Mobile quality today… and beyond
Since Japan’s NTT DoCoMo introduced the first full Internet service on mobile phones in 1999, the mobile Web has grown exponentially. As the mobile platform has matured, user expectations for an error-free experience have risen in tandem.

Users have also now come to expect a seamless transition among their three screens (desktop, mobile and tablet) with equal levels of performance in all three environments. This new three-screen reality has created unique challenges for developers and testers alike, and Responsive Design has arisen accordingly to address some of these challenges.

(Related: Responding to changes in Responsive Web Design)

On the testing and monitoring side, it is important when considering any provider of testing services to make sure that that it is capable of handling this rapidly changing environment with the infrastructure and capabilities to accurately test across all three screens.

With the projected growth of wearable technology and the Internet of Things, embedded sensors measuring motion, location and other environmental factors will add another layer to the challenges of mobile quality assurance. But as with each preceding wave of development and innovation, it is clear that broad-based adoption of new mobile technologies will not occur unless a high-quality user experience is delivered and maintained.

If we can learn anything from the evolution of mobile quality standards and testing practices, it is that performance matters, and the ability to flexibly adjust to ever-evolving technological and market conditions will help to ensure an organization’s success in reacting to and staying one step ahead of coming changes.

Josh Galde is mobile evangelist at Keynote. He has more than 13 years of experience in the wireless industry and has worked for companies ranging from startups to multinational corporations.