Test automation is critical for organizations looking to get faster and better at delivering quality software. However, as with many issues surrounding technology, just because you can doesn’t mean you should.

Phil Soffer has spent the last 20 years developing and shipping software, as a developer, marketer and now CEO of crowdsource testing company test IO. All those years inform his perspective on the industry, and he firmly believes that getting faster and better does not tell you what it means to ship software.

“Fundamentally, the decision to put something in front of a customer is a business decision,” he stated flatly. “But there are lots of different inputs [into that decision]. It’s not a decision to make by looking at a dashboard of software that ran.”

Much of what we hear about test automation is from big companies with large user bases that in many cases don’t pay for the software, Soffer said. Think Google, Facebook or YouTube. “YouTube can roll out to 10,000 people and get that feedback. Then they ship to one-tenth of 1 percent of the user base and they can get a lot of telemetry from that. But most people who ship software don’t have the luxury of a billion users.”

Automation isn’t a plug-and-play type deal.  Companies have to mesh test automation metrics with data from people—manual testing, exploratory testing, user acceptance testing, and testing in real-world customer environments—to form a comprehensive view about the quality of the product. Only then, to Soffer’s way of thinking, can organizations even begin to consider if software is ready to be released.

Soffer said that the development organizations he deals with here in the United States are all trying to figure out how to go a little bit faster. “Speed is measurable, and people like to do things that are measurable,” he said, noting that the agile movement was a reaction to development that was complex, late and wrong for the user when it shipped. “Being responsive to users is right, is true, but caring about shipping 10 times a day isn’t true.”

Soffer said automated testing is important, but he believes that in the agile, DevOps world, “you need some input into business decisions that is human. It’s great to say we should be doing continuous delivery, but the idea that you can ship to a million people and get it wrong is fundamentally wrong-headed,” Soffer said.

A lot of work in the industry these days is going into developing out artificial intelligence systems, designed—we’re told—to help humans work and live more efficiently and safely. We’re seeing much money poured into developing self-driving cars, drones and all manner of workplace and lifestyle robots (Hey Google, Alexa, Siri and Cortana).  

(Cute aside) … I remember as a kid listening to conversations my parents would have as they began to get older. “You know who I saw on TV yesterday?”

“No, who?”

“You know, that redhead, she was in that movie that suave British actor you like.”

“Lucille Ball?”

“No, she was that western with John Wayne and she gets shot at the end.”

“Rita Hayworth?”

“No. She was married to the director who made that film about blowing up a bridge during World War II.”

“I have no idea what you’re talking about.”

Then the conversation would invariably end with, “I’ll think of it at 2 o’clock in the morning.”

“Don’t wake me.”

Today, of course, we can simply sit with our cellphones and search for the answers. We never really have to think or recall information anymore. Devices are doing that for us. Yes, they can answer simple questions (and complex ones as well), but they can’t deliver the empathy, or sarcasm, or come at things from seemingly out of left field—all of which makes the human experience what it is… human!

Industry analysts see a role for artificial intelligence in test automation, creating tools that can adapt to software changes as well as user behaviors. As with all bots, though, Soffer said you still need the human touch. “AI could make tests better, but it can’t replace the random chaos-monkeyness that is humanity,” he said.

Beautiful, messy, blissful humanity.