In an ideal world, software testing is all about bringing vital information to light so our teams can deliver amazing products that grow the business (to paraphrase James Bach). Investigation and exploration lie at the heart of testing. The obsession with uncovering critical defects before they unfurl into business problems is what gets under our skin and makes us want to answer all those “what if…” questions before sending each release off into the world. 

But before the exploring can begin, some work is required. If you’re tracking down literal bugs in the wilderness, you’re not going to experience any gratification until after you check the weather forecasts, study your maps and field guides, gear up, slather on the sunscreen and mosquito repellant, and make it out into the field. If your metaphorical hunting grounds are actually software applications, these mundane tasks are called “checking.” This includes both the rote work of ensuring that nothing broke when you last made a code change (regression testing) and that the basic tenets of the requirement are actually met (progression testing).

RELATED CONTENT: Testing in a complex digital world

This work is rarely described as “fun.” It’s not what keeps us going through those late-night bug hunts (along with pizza and beverages of choice). So, what do we do? We automate it! Now we’re talking… There is always primal joy in creating something, and automation is no different. The rush you get when your cursor moves, the request is sent, the API is called…all without you moving a finger…can make you feel fulfilled. Powerful, even. For a moment, the application is your universe, and you are its master.

You now breathe a sigh of relief and put your feet up, satisfied with your efforts. Tomorrow is now clear to be spent exploring. Back to the bug hunting! The next day, you flip open your laptop, ready to roll up your sleeves and dive into the fun stuff. But what’s that? Build failed? Awesome! Your work is already paying off. Your automated checks have already surfaced some issues…or have they?

No… not really. It was just an XPath change. No problem; you won’t make that mistake again. You fix it up, and run the tests again. Wait, that element has a dynamic ID? Since when? Ok ok ok…fine! You utter the incantation and summon the arcane power of Regex, silently praying that you never have to debug this part of your test again. At some point, you glance at the clock. Another day has passed without any time for real exploration. This work was not fun. It was frustrating. No longer are you the master of this universe, but an eternal servant at the whims of an ever-growing list of flaky, capricious tests. 

Turns out, the trick to getting past all the mind-numbing grunt work isn’t outsmarting the traditional script-based UI test automation that everyone’s been battling for years. It’s enlisting innately smarter automation—automation that understands what you need it to do so you can focus on what you actually want to do: explore!  

With the latest generation of AI-driven test automation based on optical recognition, you can delegate all the automation logistics to a machine—so you can focus on the creative aspects that truly make a product successful. (Full disclosure: Several companies offer AI-driven UI test automation based on optical recognition…and I’m leading the development of this technology at one of them.)

The idea behind this approach is to tap an engine that can understand and drive the UI like a human would. This human behavior is simulated using various AI and machine learning strategies—for example, deep convolutional neural networks combined with advanced heuristics—to deliver stable, self-healing, platform-agnostic UI automation. 

From the tester perspective, you provide a natural language description of what actions to perform, and the engine translates that to the appropriate UI interactions. UI elements are identified based on their appearance rather than their technical properties. If some UI element is redesigned or the entire application is re-implemented using a new technology, it doesn’t matter at all. Like a human, the automation will simply figure it out and adapt.

Making sure this works with the necessary speed and accuracy across all the technologies you need to test is the hard part—but that’s our job, not your problem. You can just roll up your sleeves, tell it what you want to test, and let the automation handle the rest. Then the fun can begin. 

Here are two core ways that this “AI-driven UI test automation” approach helps you get back to the fun part of testing…

Automation without aggravation
I’m no stranger to automation. I’ve worked in automation for over a decade, managing automation teams, implementing automation, and even doing some work on the Selenium project. Building stable automation at a technical level is invariably tedious, no matter what you’re automating. And aside from some very high-level guiding principles like separation of concerns, data abstraction, and design patterns, your mastery of automating one technology doesn’t really translate when it’s time to automate another. 

Automatically driving a browser or mobile interface is a lot different than “steering” a desktop application, a mainframe, or some custom/packaged app that’s highly specialized. Technologies like model-based test automation remove complexity, adding an abstraction layer that lets you work at the business layer instead of the technical layer. However, it’s not always feasible to apply model-based approaches to extremely old applications, applications running on remote environments (e.g., accessed via Citrix), highly-specialized applications for your company/industry, etc.

With image-based test automation, the underlying technology is irrelevant. If you can exercise the application via a UI, you can build automation for it. No scripting. No learning curve. Just exercise it naturally—like you probably already do as you’re checking each new user story—and you can get all the repeatability and scalability of automation without any of the work or hassle. 

Technology Stockholm Syndrome
Back when I was a university student, I “learned” that nothing would ever be developed that wasn’t a big heavy C thick client. There was some talk of thin clients, but of course those wouldn’t last. After I graduated, everyone was scrambling to rewrite their thick clients using the shiny new service-oriented architecture. Then came mobile. And containerization and Kubernetes.

By this time, I figured out that I have a problem: let’s call it Technology Stockholm Syndrome. I was held captive by the ever-changing landscape of technology. And the strange thing was that I kind of liked it because this ever-changing, ever-shifting set of goalposts was so much fun.

This is a good problem to have in terms of ensuring the continued value and viability of your organization’s applications. You want your dev teams to stay on top of the latest trends and take advantage of new approaches that improve application flexibility, reliability, security, and speed.  But if you’re responsible for building and maintaining the test automation, each change can be torture. In most cases, a technical shift means you need to scrap the existing test automation (which likely represents a significant investment of time and resources) and start over—rebuilding what’s essentially the same thing from scratch. Not fun.

 Also, not really required anymore. You’d be surprised at how few fundamental changes are introduced into an application’s UI from generation to generation. (Don’t believe it? Just take a trip back in web history on The Wayback Machine and see for yourself.)

Although the underlying implementation and specific look and feel might shift dramatically, most of the same core test sequences typically still apply (for example, enter maxmusterman under username, enter 12345 under password, and click the login button). If those same general actions remain valid, then image-based test automation should still be able to identify the appropriate UI elements and complete the required set of actions.

Remember Java’s mantra of “write once, run anywhere”?” (Go ahead, insert your own Java joke here). Done right, this testing approach should actually deliver on that promise. Of course, you’ll probably want to add/extend/prune some tests as the app evolves from generation to generation—but you certainly don’t need to start with a blank slate each time the application is optimized or re-architected.

Deep down, testing is truly fun
UI test automation is undeniably a critical component of a mature enterprise test strategy.  But wrestling with all the nitty gritty bits and bytes of it isn’t fun for anyone. Go search across engineers, professional testers, business users, and all sorts of other project stakeholders, and I guarantee you won’t find anyone with a burning desire to deal with it. But testing as a discipline, as a creative, problem-solving process, is truly fun at its core. 

Peel away the layers of scripting, flakiness, and constant failures, and you can (re)focus on the experimentation, exploration, questioning, modeling… all the things that make it fun and fulfilling.