Mobile devices require new strategies for testing applications. One can run all the emulators and in-house tests, but sooner or later, someone has to run tests in the field. And even with those bases covered, an emulator and a mouse are very different from a real device and a finger.

With so many new requirements coming along with mobile testing, the somewhat mundane world of QA is now inundated with new ideas, approaches and tools. Gorilla Logic recently released version 5 of FoneMonkey, its open-source functional testing tool. uTest is pushing crowdsourced testing regimes. Selenium can now be tweaked to work in mobile browsers, thanks to a browser plug-in model introduced in version 2.0.

And yet, there are still gaps in the testing regime, said Ed Schwarz, vice president of engineering at Gorilla Logic. “Supporting device-based testing, as opposed to just simulator-based testing, continues to be a hole,” he said. “Even with our tool-automating tests, the device-based testing is pretty complicated. The provisioning for that can be complicated.

“A lot of our customers are not happy with only simulator-based testing. [Device-based testing] requires going old-school. You can’t virtualize the platform the way you can on the desktop. You have to have a test lab where you walk in and find a bank of 20 or 30 devices, and a lot of the provisioning is ad hoc by the development team, to cover different versions of the OS and devices.”

Part of the problem, said Jason Huggins, creator of the Selenium testing framework, is that testing is a lagging indicator in the broad spectrum of software development trends.

“The mobile industry started with the iPhone. 2007,” he said. “We’re now four years in. Realizing that it’s a lagging indicator, if you look for testing projects related to mobile, 2010 was a tipping point.

“Right about now is a great time. People have to care about testing.  It’s an 8 a.m.-Monday-morning moment for developers and testers. When boring companies say, ‘We’re going to do our next time-and-expenses application as a mobile app,’ that’s where testing comes in. We’ve had four years of silly games, but now it’s for real.”

Matt Johnston, vice president of marketing and community at uTest, said that mobile hasn’t just been of interest within that crowdsourcing QA community; it’s been evolutionary.

“Evolution is putting it kindly,” said Johnston. “Revolution is probably closer. In 2008, we cut our teeth on Web and desktop. Not only is mobile the fastest-growing segment of our business, it’s catching up to Web.

“There are a lot of lines of code written for mobile. You can’t just test mobile in the lab. The testing matrix for Web and desktop is simpler than mobile. You can’t keep up with the testing matrix when you add mobile OS and mobile browser and hardware. Even if you build a million-dollar lab and cover all that, you’d still be in the tall grass because of location.”

Different solutions abound. Huggins’ company, Sauce Labs, offers clouds that run Selenium tests, and thus fixes the provisioning and management problem through commodity hardware as a service. uTest offers actual humans on demand, who can wander over to the nearest mall and actually test a mobile app in the field. Gorilla Logic gives testers more traditional record-and-playback testing software for purely functional problems, like wireless networks, carrier coverage gaps and human involvement in the testing regime.

While traditional enterprise applications get battered nightly or even hourly in continuous integration build cycles, it can be tough to crank mobile tests out at a CI pace. It’s a vibrant ecosystem of options, thanks to a complicated set of problems. “A native app doesn’t have the same breadth of functionality as a desktop or Web app,” said Johnston.

“Imagine someone gives you a 20-step test case and you’re at your computer. You can hammer through it quickly. You’re on a T-1 with no latency. Now, let’s say you’re in a good coverage place with an iPhone 4. It’s tough to get through those 20 clicks, and you’re going to have some latency. There’s less testing done per hour. We’ve gotten that feedback from our testers.”

But with so many new companies taking a crack at the market, 2011 should be a busy year for mobile testers and tests.