“How do you search for things?” asked the user. She stared at the screen for many seconds.
“Okay,” I said, “Thank you for—”
“How do you search for things?” she asked again, irritated.
I answered truthfully: “You use the ‘search’ button.”
Without the slightest hesitation she responded, “I never would have guessed that.” She then went on to explain how it should have been designed.
We all think we’re designers. Once I had a client who was obsessed with what he called “wasted space,” which a designer would refer to as “white space.” As he kvetched about the horrors of using up screen space with a rocker button that cycled through color options, I said, “Well, we could have it so that when they clicked directly on the product, it cycled the colors.”
“Yes!”
“Well… of course people aren’t going to know that when they first come to the site.”
He responded instantly, with a voice as ominous as Darth Vader: “They will learn.”
There’s a part of me that loves user testing; the same part that enjoys visiting ThereIFixedIt.com and watching videos of people destroying their trucks by felling trees on top of them. I take comfort in believing, however briefly, that there are people more foolish than myself. My source code often makes me despair of my own sentience, but I like to tell myself that I wouldn’t need to be prompted to use a button marked “search.”
My major reaction to user testing, though, is often resignation. User testing invariably reveals distasteful work—layouts that need total reworking, dead-end navigation paths, and corner cases that the library API doesn’t cover. The interesting algorithmic stuff that you developed while surrounded by a stack of heavily annotated journal articles and intently pored over on a statement-by-statement basis? That stuff works fine.
Unfortunately, the temptation to skip user testing is often encouraged by clients. While experienced developers know that user testing will lead to a certain amount of dismay, inexperienced clients dismiss user testing because they’re invariably overconfident. I don’t think many are as bad as my above-mentioned client, who was confident that word would spread like wildfire that one could cycle colors by directly clicking on the product. More generally, the problem is the opposite: Clients have seen so many mockups and prototypes and test versions that they cannot see it with fresh eyes.
The days are gone when user testing was an expensive proposition involving one-way mirrors, dedicated video cameras and a shiny binder of results delivered in just six weeks. Although such facilities still exist, a webcam and screen-recording software can provide similar data. More importantly, quantitative formal research is no longer considered the only valid, or even most valuable, approach.
Steve Krug’s “Rocket Surgery Made Easy” (my favorite book title of the year) is a recent companion to his previous book, “Don’t Make Me Think!” Where his earlier work advocated for usability as a primary concern, “Rocket Surgery” is a (very) brief primer on qualitative user testing.
Qualitative user testing differs from formal research in a number of ways. I think the two most important are that it’s invariably cheaper (just the logistics needed for valid research drive up costs significantly), and that it’s much more pleasant for both the user and tester. The tester is allowed to prompt the user or change an obvious trouble spot without forcing a statistically significant number of failures. The goal is not to withstand the scrutiny of peer review, but to create a better user experience.
Krug also focuses on doing as little as possible, which endears him to my heart. The distasteful work that user testing invariably reveals is still going to show up, but he argues that “the least you can do” is often the best approach, an argument that I’m certainly willing to listen to.
Another type of testing that I’m fond of is “A/B” testing, in which you offer up alternative UIs and see which one produces better results. This is especially easy on the Web, at least for pages without extensive JavaScript dynamics. It’s immensely enjoyable to head off one of those endless discussions about “What if the sidebar were wider?” with “Well, let’s try them both and see what we get.”
A/B tools are rapidly proliferating; one that’s broadly useful is Google’s Website Optimizer, but development teams may prefer a platform-specific one, such as A/Bingo for Ruby on Rails.
By the way, I put a counter in that color-cycling code and was shocked when I saw how frequently it was used. Then I actually crunched the logs and saw that more than 90% of the color-cycling clicks were coming from three internal VPN addresses: the boss’ desktop, his home machine and his laptop. Some people never learn.
Larry O’Brien is a technology consultant, analyst and writer. Read his blog at www.knowing.net.
