Mention “metrics” and “digital experience,” and people immediately think of customer analytics: measures of the way customers engage with an organization via digital channels.
Marketing uses a host of customer-centric measures to shape how it creates digital customer experiences and translates them into value for the business; counting page views, session times and completed sales are just the beginning. Analytics specialists track user engagement across platforms to build a complete picture of a customer’s journey, from discovery via a search engine to reading reviews on a mobile phone to making the purchase in a traditional Web browser.
In contrast, metrics for digital experience delivery measure development’s delivery of customer-facing applications. Marketing scores developers on two counts:
• On-time delivery: The rapid pace of innovation in customer-facing applications drives business stakeholders to demand shorter timelines for digital experience projects. Organizations introduce new mobile or tablet apps in an effort to lead (or at least keep up with) the competition.
• Within-budget delivery: Budget is the second concern for the project stakeholders we interviewed. Development teams compete with agencies and contractors to deliver the most value per dollar spent.
Compared with the advanced analytics marketing pros use to measure customer behavior, this method of measuring development’s delivery of digital apps falls short.
Typically, developers and business stakeholders negotiate an unrealistic timeline, developers deliver the project behind schedule, and the features don’t match the business need. Measuring on-time delivery doesn’t work for digital experience projects, which tend to be unpredictable. Development teams don’t know at the outset what they’ll build, so they must be able revise designs based on constant feedback.
Rather than adhering to rigid timelines established before development begins, development teams should constantly deliver value to the business by releasing frequently, soliciting feedback, and adjusting priorities after each release—like agile teams do. Instead of on-time delivery of the project as a whole, the key metric is velocity: how quickly and how often teams deliver software. Teams can also track cycle time, which is the time it takes to complete one software development cycle—in this case, from the definition of a release to its production. This approach has two benefits:
• Flexibility: Development teams become flexible because they can quickly get feedback on new features included in the frequent releases and adapt their plans accordingly. At the end of each development cycle, teams re-evaluate priorities and set the timeline for the next few weeks.
• Transparency: Development teams become transparent because business stakeholders see consistent progress. Stakeholders collaborate with developers to decide which features to build next. Keith Mann, a group architect at 407 ETR, told us that delivering often gives the business insight into development’s activities: “The business has a gut feel for how IT is doing. They’re happy to be getting new functionality every two weeks.”
Consistent velocity is an indicator that a project is on track, but blindly pushing to increase velocity without attention to context will cause problems. Developers will push to deliver the wrong features faster, leading to poor quality, unstable code, and products that don’t meet stakeholder needs—not to mention burnout at the end of each sprint. So, to ensure that developers deliver the right features at the right time, velocity must be paired with business value metrics.
The first step to measuring business value is ensuring that developers understand the business key performance indicators for the project up front. Developers need to understand what “value” means in the context of the app they’re working on. Understanding business value allows developers to partner with designers, suggesting improvements and finding opportunities for a more consistent look and feel. Justin Williams, director of development at Cynergy, told us that he measures developers on empathy: their understanding of the business problem and the business value of the solution they’re building.
Vivian Brown is a researcher at Forrester Research.
