Mobile devices have changed our world. Smartphones and tablets allow employees and customers to be available and active online almost 24z7—something unimaginable in the PC era. Mobile devices’ touch-screens support rich media and application functionality and deliver a far more powerful user experience than ever thought possible.
The impact is largely felt in the testing and development community. Consider a few decades ago when the traditional IT environment comprised the standalone PC and the situation was far simpler. Developers only had two browsers to deal with, and there was only one platform to test them on. Developers would simply set the browsers up on one machine, install the application and run through a few functions to make sure everything was working. At that time, manual testing was the viable and effective option.
The proliferation of mobile devices and applications has dramatically altered the development landscape. For instance, today’s testing teams have six or more browsers to contend with. Agile increases the relentless rate at which new application releases and updates are generated, which must be tested against a plethora of new mobile devices continually being introduced to the market. The browser/device configurations are multiplying at an exponential rate, and complexity is reaching alarming levels.
Any organization delivering applications has a number of complex testing and delivery challenges to overcome, which are caused by the mobile multiplier effect. If left unchecked, these challenges will cause product delays, end user frustration and, worst of all, loss of revenue.
Functional testing and performance testing
Being assured that applications meet requirements for end users across devices, and ensuring that the applications perform adequately even as demand scales, is made even more difficult in an agile environment. Agile has actually made software testing a more difficult challenge as it increases the rate at which new releases and updates are generated, which, by default, increases the pace at which testing must take place.
Functional testing has been able to keep up reasonably well with agile as the tools have matured to allow manual testing to develop into automated testing. Factor in the need to perform functional tests across multiple mobile devices, as well as the need to have rigorous functional testing on all devices on which the application is supported becomes even more critical.
Performance testing has become even more important in an agile environment, because if applications don’t perform well and scale under demand, large business impacts can occur. This is often the result of the way distributed server application architectures are implemented. The use of virtual machine technology and cloud resources has, in many cases, changed application resource allocation from static to dynamic. However, allocation isn’t the only concern. What has been found in the APM world is that other root cause issues can lead even the most functionally and transactionally sound applications to degrade if problems are not found and fixed during testing.
Organizations need to work harder than ever to be sure that the application releases they generate with agile methods will perform satisfactorily under all operating conditions, including across a wide range of devices.
Prepare and plan for the mobile multiplier
Many organizations do not have an effective mobile strategy in place, and so they start out by focusing on a couple of their most common use cases. Then, as soon as they attempt to deliver real business functionality to their users, they realize they need to accommodate testing of multiple use cases based on the operating systems and device configurations that their audiences are using. In addition, the cost of testing multiple use cases grows exponentially as the complexity of the testing environment increases.
Finance departments are naturally skeptical about footing the bill for thorough testing of complex mobile environments, which requires different levels of domain knowledge and expertise, as well as increased time and manpower to validate applications (if costs are reaching excessive levels and dramatically impacting return on investment). Companies certainly do not want to spend more time and money on testing an application than they did developing it.
Delivering performance and capacity
In light of the mobile multiplier effect, performance and capacity are key areas for consideration. Application quality can be defined as performance plus capacity, and as with quality and cost, there will usually be a trade-off. If performance levels are poor and an application is ridiculously unresponsive, users will quickly seek an alternative. The competition is fierce; today’s users are fast to dismiss a brand, and their loyalty will quickly evaporate when faced with an app that’s taking an age to respond.
Capacity refers to an application’s range and scope (what is does and how extensive it is), and this has a major impact on whether or not an excellent user experience is delivered. If an application is seriously limited in scope, it will be simple to test but will also be of little use to the end user. The more developers put into an application, the more engaging and valuable it becomes for the person using it, and the more difficult and expensive it becomes to test it. The bottom line is all about prioritizing performance and capacity levels to suit your application.
Mobile and agile
Agile development, with its emphasis on working software and regular testing, needs test automation, which enables levels of code coverage and confidence impossible through other means. Not only does automated testing help catch defects earlier in the cycle, it essentially builds quality into the process.
Today’s application developers and testers need a mobile strategy with an integrated and automated approach. This allows them to effectively and efficiently develop and test apps, while taking into account the plethora of operating systems and devices, and managing the workload generated by the mobile multiplier effect.
Cloud-based performance testing will ensure capacity even in the most extreme performance scenarios. By allowing test teams to instantly deploy existing performance test scripts to cloud-based load generators, the load is created on preconfigured systems provisioned in the cloud. This eliminates the effort and cost related to extending the on-premise test infrastructure, which only the highest-load scenarios would need.
In addition, cloud-based services can provide a diagnosis of any performance related issues when they arise, giving teams the detailed diagnostics they need to pinpoint the nature and location of the problem in order to remediate quickly. Combined with an on-premise performance monitor, it’s straightforward to understand the demands on the server infrastructure in the data center, providing end-to-end transparency.
By harnessing the power of the cloud, developers can execute mobile testing quickly across multiple platforms and devices, ensuring applications work exactly as intended and generate an exceptional user experience for everyone.
