When it comes to data-driven decision-making, quality control, goal alignment, and accountability, establishing and adhering to key performance indicators (KPIs) is the industry standard for creating and maintaining exceptional engineering teams.
Unfortunately, there are a few engineering KPI anti-patterns that reduce the organizations’ ability to understand how their performance relates to business outcomes.
When teams start measuring for the first time, they tend to use those metrics to drive accountability and predictability. But according to Jeremy Freeman, co-founder of value stream intelligence platform provider Allstacks, failing to trace those metrics back into business outcomes is the first of three KPI anti-patterns he discussed in a recent call with SD Times.
The message from executives is that organizations need to be more data-driven, and there are countless metrics that can offer some visibility into the activities and performance of the product development team. “If you’ve not done engineering KPIs before, you might grab the first six or seven that pop up,” Freeman explained. He highlighted metrics like average downtime, churn, and on-time delivery, then warned that engineering leaders often neglect to do the next step: “You don’t actually ever really go back and connect those back to the business value.” And that’s where things get dicey with the business side of the house. Freeman elaborated, “Would the CEO go to the company board and say, ‘we improved our Scrum velocity by 50%?’ Because the board would say, ‘Why do we care? Does that move the needle on our revenue? Does it somehow reduce our costs?’” Engineering leaders often don’t know how to use their data to answer these questions.
Keep in mind that the engineering team in a lot of organizations is responsible for multiple different outcomes. The team could be responsible for working against a roadmap developed in another part of the business, but also responsible for delivering that software to the user, and ensuring that it’s running and users have a good experience, which impacts the business. “What ends up happening is leaders tend to search for one key metric for engineering performance, and tend to forget that there are multiple outcomes the org is responsible for,” he added. “It’s not just to be the most efficient development organization possible.”
The second anti-pattern Freeman discussed has to do with the term “cargo culting.” He said he has heard the term used to describe people adopting Agile practices for the sake of adopting Agile. “A team will read a bunch of information or hear talks at a conference and say that ‘Agile seems great. And all we’ve got to do is have a standup once a day.’ But they don’t see any of the benefits. They’re pulling in these practices, hoping that they’ll get the benefits, but don’t really understand the connection between actions and outcomes. That ends up actually being a big anti-pattern.”
In this scenario, engineering KPIs become checkboxes like “What’s our PR cycle time” instead of “Will improving the our PR cycle time improve our organization’s delivery ability?” Without seeking to answer the second question, any measure gets clunky, hindering instead of helping teams achieve efficiency.
The third anti-pattern? Freeman readily identified “using metrics for evil.”
The Allstacks CTO explained that as the misuse of data and losing sight of the overarching goal it’s trying to track. “If you are so focused on the metric, it can lead to some really toxic behaviors,” he said. Freeman used the example of an organization’s SLA that requires all bugs to be finished within two weeks. The team might be tempted to put in hasty fixes, or try to skirt the metric by creating a new ticket that resets the clock. “For junior leaders, it can be tempting to tie metrics like that to performance reviews and potentially even compensation or bonuses. If that happens, people are going to get real focused on hitting the metric at the expense of the business objectives and the job that they’re supposed to be doing.”
If people are incentivized to game a metric, Freeman concluded, you’re using metrics incorrectly. “You should be using your KPIs and metrics to point people in the right direction and drive the right behaviors, not punish people for getting off-track.”
Content provided by SD Times and Allstacks
