As the world goes mobile at increasing speed, the current process for testing mobile applications simply does not work. Despite increased investments in application performance monitoring, load testing and functional testing, users continue to be frustrated by application failures.
Over the last few months, business and technology news grabbed our attention with headlines highlighting spectacular website failures, costing the offending organizations millions of dollars per incident. The untold story is the contributing effect mobile devices had on those failures. I have observed that shifting as few as 15% of interactions to a mobile device population can overwhelm server resources. Whether with a browser or a purpose-built mobile app, the same back-end resources support functions like accessing user accounts, payment processing and logistics. The websites and processes were simply not designed or tested to support the increased load caused by mobile access.
According to the National Institute of Standards and Technology, failed application deployments cost companies nearly US$60 billion per year in the United States alone. These failures are not acceptable in the on-demand world. Thoughtful and accountable businesses and organizations are adopting a new, proactive approach to mobile application testing, one that incorporates the concepts of application performance and real-world network conditions into every phase of the application development life cycle.
I have seen firsthand the economic impact enjoyed by companies that design, develop, test and manage their applications for performance. The impact in many cases easily reaches millions of dollars saved in reducing the continuous break/fix cycle. I have also seen revenue increase dramatically as mobile application performance increases. Perhaps the most important impact of all is customer retention. Simply stated, a bad mobile experience threatens your brand.
Rightly or wrongly, end users now expect faster performance levels from mobile applications than from the desktop or laptop. Evidence reveals that an end user that was satisfied with a seven-second page load last year expects the same page load in three seconds now. Users don’t care about the added complexity of mobile networks, devices and geographies. Users don’t care how it gets done, they just want apps that load quickly and perform perfectly.
And as everyone knows, users aren’t planning to do less on smaller screens; if anything, they expect more, like clear VoIP, streaming video and 24/7 access to social media applications.
How can we improve end-user satisfaction? With a new approach to performance testing, companies will eliminate underperforming apps that frustrate users. We are witnessing dramatic results to this end in the testing of applications under real-world network conditions.
In order to truly understand how an application will perform and address shortcomings pre-deployment, or to troubleshoot failures post-deployment, actual network and usage conditions such as latency, jitter, packet loss and bandwidth need to be captured, analyzed, and then replayed or emulated throughout the development process. This is the process referred to as network virtualization.
Network virtualization helps organizations understand their end users more completely, determining what devices they are using, where they are located, and what networks are being accessed. This critical information is communicated throughout the software life cycle during design, development, QA, performance test and operations. Then, when applications fail to perform as they did in the test environment, the information available includes the fact that a performance issue occurred when a user tried to execute a specific function, and includes the conditions that user was operating under. A reliable and reproducible test case for troubleshooting exists with a holistic view of the user’s actions and environment.
Just as performance testing benefits from increased location and network specificity, performance analysis is also location-specific so that performance levels are identified and established per user and per location. Insight into the impact of the network on users in Japan versus New York, or offices in Los Angeles versus London, is critical to understanding acceptable levels of performance. This helps companies by predicting and validating performance levels, determining compliance with stated service level objectives or agreements, and identifying and diagnosing the root causes of performance bottlenecks.
Why is this important? Finding and fixing problems before end users are affected is critical because businesses might spend as much as 100 times more to fix errors after deployment. And, if a performance-focused practice is employed early in the development life cycle, then organizations have a reliable means for evaluating performance optimization initiatives like WAN acceleration. A cost-per-performance-gain analysis can help organizations make the right investment in ensuring performance. Perhaps most importantly, deploying an application with confidence in performance means that businesses and end users will not suffer, and businesses will avoid the negative and extremely painful affects such as lost revenue, negative publicity, decreased employee productivity, and lost customers.
Organizations will also benefit from a greater ability to hit targeted SLOs or SLAs, and deliver a smoother and more enjoyable experience for end users.
While incorporating and considering real-world network conditions on a user-by-user basis may not seem like a significant change to application testing methodology, implementing and integrating the approach throughout the application life cycle requires a new mindset. Let’s treat performance as a feature of the application that gets tested and validated in each development cycle.
The traditional approach to application testing currently in use simply isn’t working. It costs businesses and end users too much time and money. It’s time for a new mindset. It’s time to include real-world network conditions into all aspects of the application development process.
Gary Jackson is CEO of Shunra Software, which sells tools for using network virtualization in mobile apps testing.
