UX design and testing continue to evolve with the emergence of new technologies that enable new types of experiences. Mobile and web apps changed the conversation from UI to UX. Now, UX involves much more than graphical UIs and app performance. Organizations must deliver omnichannel experiences that contemplate voice interfaces, virtual elements and more.

“We’re just scratching the surface of UX design as a discipline. It’s not UX/UI. UX is a much broader discipline than UI,” said Jason Wong, research VP at Gartner. “UI looks at how the user interacts with a given application. UX includes user research, content, performance and back-end so you need a team to help execute that.”

Beyond graphics, UX designers and testers must contemplate conversational interfaces, immersive experiences, and the IoT.

“It’s not just about graphics, it’s about tones,” said Wong. “We’ve seen companies hire copywriters from the movies and TV to help script responses for chatbots and hone what that voice response should sound like in terms of a voice interaction. Very complex things like that are still emerging which UX is still trying to grasp because they’re so new and they’re constantly changing.”

UX requires a holistic view
UX designers need to think about how their brand translates to different types of experiences and how the brand translates across those experiences.

“At a meta level, the biggest shift has been toward recognizing that experience design is multichannel,” said Martha Cotton, managing director of Design Research at Fjord, the design arm of Accenture. “You need to think about the channel, but when you’re designing an experience, [you have to realize that] people don’t interact on a channel-by-channel basis.”

Instead, they think about the tasks they want to accomplish. For example, noisy airports aren’t the best settings for phone calls or voice interfaces, so an individual might choose to check a credit card balance using a smartphone or laptop. However, while driving, the same person might call the credit card company and check her balance using an interactive voice response (IVR) system.

“When you’re a designer, you need to think about the omnichannel world and how people engage overall with a service. That can include any number of touch points, some of which may not have been invented yet,” said Cotton.

In order to get those experiences right, UX designers are working closer than ever with end users in the form of participatory design, which involves the user in the design of an interface early on. For example, Cotton said when she helped a printer company develop a new app, the project included a panel of approximately 40 users who were engaged over the course of several months, albeit not continuously.

“I’m seeing this notion of explicit co-creation,” said Cotton. “Another interesting trend is the integration of data scientists and UX researchers. [We’re seeing] UX teams made up of a multidisciplinary set of folks and they’re working to integrate data science with qualitative UX design. Uber did exactly that. It was this constant interplay between what they were learning from the data scientist’s perspective and out in the field understanding the rationale for people using the service. They were testing a series of new features and able to pivot based on the data that was coming through the data scientist’s effort and user research insights.”

The implication is that designers are spending less time in front of a screen and more time out in the field trying to understand what an experience should be from an end user’s point of view.

Agile applies to UX
Not surprisingly, Agile ways of working are impacting UX design and testing in the form of iteration. LinkedIn founder Reid Hoffman once said, “If you are not embarrassed by the first version of your product, you’ve launched too late.” The quote, though, reflects a common misperception about agile, which is that the methodology is all about speed.

“It’s a ridiculous quote because there’s only one opportunity to make a first impression. Do you really want to make a poor impression with a crappy app?” said Gartner’s Wong.  “When I talk to clients and they say we just want to reskin our website or take portions of our website and put it on our mobile app I tell them that will create a bad impression. If a user is excited enough to download your app and finds it’s a crappy reskin of your website, they’ll probably delete it. They won’t reload it and they won‘t give you the opportunity to understand how they’re using it, so design is really integral to the whole Agile and DevOps process.”

 

While speed is a benefit of Agile, two other important benefits are the alignment of business and IT and improved quality.

“Businesses are adopting Agile methodologies such as LeSS and SAFe that help entire businesses think about how they should move to continuous, iterative work and improvements,” said Wong. “The preconceptions of Agile from a development and IT perspective is we can do things faster.”

In fact, in a recent note, Gartner stated that achieving Agile and DevOps success requires setting business expectations and measuring success in terms of business value.  That requires aligning the business and IT around Agile and DevOps.

“Digital design and techniques like design thinking, human-centered design and then going to prototyping techniques like Lean Startup and Lean UX help refine those ideas and uncover what the problems are because the problems and issues faced by the users are not explicit,” said Wong. “Since they’re not visible, you have to uncover the hidden challenges. Empathy is a big thing in design, stepping into the shoes of your users to understand what they need even if they don’t recognize it themselves.”

Through 2023, Gartner expects human rather than technical factors will cause 90% of Agile/DevOps initiatives to miss expectations of higher customer value and faster product delivery.

“A lot of companies think they’re doing Agile, but they’re not actually being successful at it, so they’re going through the motions and they’re not wholly embracing the concepts and taking it to its fullest and changing the culture entirely across their organization,” said Wong. “We see companies trying to do a wholesale shift to Agile, and that typically fails. It typically takes two to three years before the Agile mindset is fully indoctrinated within any large development organizations so within the first year there should be progress but by no means should companies expect in the first year to become fully Agile.”

Another issue is that management doesn’t understand customer value, according to Gartner.

“Before, the business would bring its projects and ideas to IT on an annual basis. All sorts of executives would get together, pick and choose different projects, and then bless them. You’d start building it and then there would be scope creep and budget overruns and the thing would get pushed out. That’s waterfall. Agile is really about being responsive so you can see what’s happening in a shorter period of time and responding to market demands, customer demands, and allowing the business to adjust its strategy to whatever disruption is out there.”

Synonymous with the brand is the experience it delivers, which is why Agile ways of working must extend to design professionals as well as developers, DevOps and the business.

“Agile without design thinking is still a form of order-taking,” said Wong. “Ultimately, Agile is about iterative work. If you don’t do an upfront design/discovery exercise, how do you know that MVP is actually viable?”

More design expertise is necessary
UX design is not a one-size-fits-all proposition because a product or service may span multiple touchpoints that are navigated differently by different users.

“It’s not so much application development organizations taking requirements now or the business going to IT and saying we need to build a mobile app or a chatbot and we want it to do this,” said Wong. “Upfront design means solving the customer problems in the most efficient and effective manner for usability, engagement, [and] stickiness.”

However, the ratio of designers and developers is very poor, particularly in enterprises, according to a Gartner survey. In trailing enterprises, the ratio of developers to UX professionals is 21:1. At the leading companies where building mobile apps is central to the business, the ratio is 2:1. At design agencies, it’s 4:1, and at ISVs, it’s 9:1.

“If you have one designer and 21 developers, that’s one designer across five teams potentially,” said Wong. “Developers should understand the principles behind design and UX but they themselves are not fulfilling the roles of UX design. UX design encompasses many different areas of expertise like user research, content and performance.”

An emerging role is the UX architect, who considers the digital landscape beyond the UI level, such as performance requirements and the services that will be necessary to deliver experiences across multiple touch points.

“The UX architect is an important role for bridging IT and business,” said Wong. “Design as a competency should not reside within IT because it becomes an execution arm. [Instead,] design really should be the glue between the business and IT.”

In line with the asymmetric balance of designers versus developers, Gartner found that only 28% of organizations are mature at UX design. One problem is that UX design as a function can reside in different places in an organization such as marketing, an innovation center or IT. The function also tends to lack a charter.

“The question we get from the mainstream and conservative companies is standardizing design for their apps because they’ve got old apps and SaaS apps from all kinds of vendors,” said Wong. “The question is how do they standardize that so employees are not confused by all the different UI elements and they don’t have to jump from one app to another app to fill in data and do repetitive things.”

Increasingly, users expect to accomplish tasks from within an app rather than jumping from one app to another. So, instead of a salesperson logging into a customer relationship management (CRM) app, a customer service ticketing system and an expense reporting system, there’s a new responsive web app or new mobile app that interfaces with various systems.

UX testing Is evolving too
UX testing is becoming more iterative, and the tests themselves extend past the UI.

“I’m seeing a more agile approach to testing [in the form of] much more fluid, rapid bursts of testing,” said Fjord’s Cotton.  “I’m seeing a lot more of what we call ‘micro pilot testing,’ so rather than testing an entire interface, taking small pieces and launching them maybe even in the real world to get a sense of how they’re doing and making tweaks as necessary.”

To improve the success of digital products, Gartner recommends organizations instill a continuous quality mentality across development and operations.

“Now I’m thinking about how can deploy my application intelligently so that I can learn about its effect on my business, as well as mitigate some risk, so I’m looking at things like a canary release, or a blue-green release or using feature flags, coupled with launching darkly,” said Jim Scheibmeir, associate principal analyst at Gartner. “One thing we need to understand is reliability is an aspect of user experience.”

Few organizations are using hypothesis-driven chaos engineering (a modern form of testing) to understand how complex environments and dependency degradation impact their applications. According to Scheibmeir, organizations that embrace that type of practice will achieve a competitive advantage and retain customers even on bad days when the application is behaving at its worst.

“There’s a cultural aspect to this which is it’s my job and we don’t throw quality or testing over the wall to another team,” said Scheibmeir.  “Part of this is about architecture and building applications in a way that enables us to blue green releases or canary releases.”