The feedback and the insight you can gain from usability testing are invaluable. To actually see people kicking your tires, or using your website or software in the real world, can bring to light a host of issues that would never crop up in the Q/A stage. Or, worse, that would irritate end users when you least expected.
The good news is that usability testing no longer needs to break the bank. Usability testing services exist that follow the crowdsourcing model. These can provide quick feedback of how your software works in the real world.
Old-school labs
Usability testing has always been a useful but cumbersome exercise reserved for those with deep pockets. In the past, it might have involved engaging the services of a usability lab. A group of testers would be gathered, given instructions, and then filmed with a camera or observed through a one-way mirror to see how they interacted with your product. This was usually followed by extensive interviews, analysis and recommendations that pushed the product back to the beginning of its design cycle. The whole process was time-consuming, contrived in an artificial setting, and very expensive.
Today, usability testing services are offered online and for cheap. These online companies sign up potential testers and match your testing requests with testers who meet your criteria. For each testing engagement, you create a set of test instructions, which are then forwarded to the selected testers.
What you get back is a 20-minute video of the testers actually performing your instructions, with audio of their running commentary as they perform the assigned tasks. The videos you get back are segmented, so you can analyze each step of the instructions you provided.
Intrigued by this new usability testing model, their cost, and our urgent need to validate a new account registration process, we tried out two services: UserTesting and TryMyUI.
We were quite impressed and pleased with the results. By providing a deliberately vague set of instructions to visit our website and download our qTrace screenshot and bug-capture tool, we were able to identify a number of issues and make immediate changes to make the process easier for visitors.
One of the advantages is that the testers are sitting comfy working on their own computers. They come from all walks of life, and they interact with your product in their home or office setting, so you know they are not under too much pressure, unlike a monitored lab setting.
You get a stream-of-consciousness, close-up view on what your end users are doing and how they approach your product. You can see which instructions are not clear, what steps in a process create confusion and frustration, and which steps point to things that require improvement. Best of all, usability testing exposed unexpected behaviors that may not have been uncovered or predicted in a more controlled environment.
Besides the low cost, we were also impressed with the speed of these services. Within a few hours of placing the request for testing, we started receiving test results. After 24 hours, we received all the feedback and we could perform our analysis. That allowed us to make some tweaks, retest and improve our usability in a matter of days.
Drawbacks to crowdsourced usability
For starters, you are working with a generic user base, and there is no screening for demographics. If your product is aimed at a very specific segment of the market, this might not be so useful for you. The majority of testers are presumably work-at-home types or students looking to earn extra cash.
Because suggested run times are limited to around 15 minutes with a hard limit of 20 minutes, you can’t make the tests too complicated or in-depth. This can be a good thing, though, because you can garner extremely focused results on specific areas, and you aren’t asking testers to concentrate for too long.
Overall, we found crowdsourcing for usability testing to be an exciting new concept that is well worth exploring. It produced actionable results that helped improve our end-user experience, it was fast, and the fees are low. With the services we used, we engaged 10 testers for US$200. We found this a great way to bolster traditional Q/A and get some valuable insights.
Vu Lam is CEO/founder of QASymphony, developer of defect-documentation tools that track user interactions with applications.
