Mobile devices require new strategies for testing applications. One can run all the emulators and in-house tests, but sooner or later, someone has to run tests in the field. And even with those bases covered, an emulator and a mouse are very different from a real device and a finger.
With so many new requirements coming along with mobile testing, the somewhat mundane world of QA is now inundated with new ideas, approaches and tools. Gorilla Logic recently released version 5 of FoneMonkey, its open-source functional testing tool. uTest is pushing crowdsourced testing regimes. Selenium can now be tweaked to work in mobile browsers, thanks to a browser plug-in model introduced in version 2.0.
And yet, there are still gaps in the testing regime, said Ed Schwarz, vice president of engineering at Gorilla Logic. “Supporting device-based testing, as opposed to just simulator-based testing, continues to be a hole,” he said. “Even with our tool-automating tests, the device-based testing is pretty complicated. The provisioning for that can be complicated.
“A lot of our customers are not happy with only simulator-based testing. [Device-based testing] requires going old-school. You can’t virtualize the platform the way you can on the desktop. You have to have a test lab where you walk in and find a bank of 20 or 30 devices, and a lot of the provisioning is ad hoc by the development team, to cover different versions of the OS and devices.”
Part of the problem, said Jason Huggins, creator of the Selenium testing framework, is that testing is a lagging indicator in the broad spectrum of software development trends.
“The mobile industry started with the iPhone. 2007,” he said. “We’re now four years in. Realizing that it’s a lagging indicator, if you look for testing projects related to mobile, 2010 was a tipping point.
“Right about now is a great time. People have to care about testing. It’s an 8 a.m.-Monday-morning moment for developers and testers. When boring companies say, ‘We’re going to do our next time-and-expenses application as a mobile app,’ that’s where testing comes in. We’ve had four years of silly games, but now it’s for real.”
Matt Johnston, vice president of marketing and community at uTest, said that mobile hasn’t just been of interest within that crowdsourcing QA community; it’s been evolutionary.