For software development teams, moving fast is key. But so is the confidence that you’re building the right thing in the first place. To reduce uncertainty, product and engineering teams are running more experiments across websites, apps, and every level of the stack to gain valuable data and real user feedback, enabling them to deliver the right features to end users even faster. So when is the right time in the development process to discuss accessibility?

As a designer, accessibility might mean ensuring every input has a label or verifying the contrast between colors in the user interface is consistently strong enough to pass Web Content Accessibility Guidelines (WCAG) standards. As a developer, one aspect of building accessible products is testing for keyboard navigation and checking focus states. However, whether in design, development, or product management, as an experiment creator, accessibility means guaranteeing — regardless of the experiment variation — users will receive a consistent and usable experience that won’t interrupt their regular workflow. 

RELATED CONTENT: Getting started on the accessibility track

When creating new experiments for your product, it can be tempting to overlook or postpone accessibility until after you’ve identified the winning variation. However, accurate results depend on your users interacting with the experiment in its intended form. Otherwise, how will you know if a particular variation was truly less ideal compared to other experiences, or if it was simply inaccessible to a large portion of users? Plus, presenting inaccessible experiments can make the future rollout more difficult to implement.

Accessible experiments aren’t just a “nice to have.” It’s a common misconception that accessibility is only meant for people with specific disabilities. But in reality, accessible products benefit people who are experiencing temporary or situational changes that impact their abilities as well. Accessible products are usable by everyone — no matter their physical or cognitive ability. Likewise, your experiments should give your users an enjoyable, productive experience.  As more elements of daily life move online — from shopping, to working, to socializing — it’s important to ensure that every user can accurately consume the information presented through your experiments.

To the user, it’s not an experiment
While it’s natural for experiment creators to view an experiment as temporary, the reality is that for the end users, the variant experience is just like any other day using the product. Many experiments go unnoticed because they seamlessly fit into an established workflow. 

Imagine you’re a user of assistive technology like a screen reader and going about your usual business in a product — only to be blocked from logging in. Yesterday, you could log in easily. But today, no luck. It turns out the log-in flow is now part of an experiment, but the code behind this particular variation is not accessible with your screen reader. Suddenly your entire workflow is thwarted and you have no control over it. 

This sample scenario is the type of experience experimentation practitioners want to avoid, and the best way to do that is to ensure each experiment variation is fully accessible before it goes live. 

Easily implement the winner
Creating an accessible experiment up-front enables rollout of the successful variation faster so you can start bringing in those new users or increasing revenue even sooner. In experimentation, once you identify a winning variation, the last thing your stakeholders want to hear is that they have to postpone the rollout because the implementation is not yet accessible. Plus, while the experiment is running, you may receive feedback about accessibility issues from your users, thereby giving you more time and concrete information to improve the experience before it is rolled out to all users. 

Trust your results
Whether you’re experimenting with new links to documentation or variations of video content, ensuring that every user can equally access the information gives you confidence in the results. Otherwise, you’ll never know if it was the video content that made your page less engaging or the lack of captions. Creating accessible experiments from the start eliminates this uncertainty, so you can focus on the experimental differences that really matter.

Next steps
To learn more about accessibility and inclusive design, I recommend referencing the W3C Web Accessibility Initiative documentation, which includes handy resources around getting started for both designers and developers. Similarly, the A11Y project provides helpful WCAG checklists to highlight areas that have specific accessibility requirements. 

More than anything, as you create experiments and features, one of the best ways to ensure accessible experiences is to test the workflow with real people. Services like Fable allow you to meet with people with disabilities and engage them in research and user testing.