KPIs (key performance indicators) are crucial to advancing DevOps in a world where business competitiveness depends on our ability to measure how well we are producing compelling, innovative software. But “KPI” is a term that gets thrown around haphazardly at times, and can get overlooked as simply a buzzword.

This lack of focus was apparent in a Forrester survey of large organizations that showed effective measurement of critical KPIs is often absent and the goal of continuous improvement elusive. The head of application development for a major insurance company recently told us, “You can’t improve what you don’t measure.” This simple truth (originated by legendary management guru Peter Drucker) leads DevOps-focused organizations to a series of complex workflow decisions. What exactly are you measuring? Is it connected to business outcomes? What are we missing?

What to measure?

There are many possible KPIs, and they will differ depending your company, but the survey identified quality, velocity and efficiency as the most critical ones for the majority of organizations. How can software development processes be advanced in each of these areas? With bug-free, elegant digital user experiences such a priority, it’s no surprise the respondents specified quality as the most important indicator.

The problem: two-thirds say they are dissatisfied with release velocity and nearly a third don’t measure velocity and efficiency at all. The survey hints at several overriding recommendations on how to strengthen these primary KPIs and bring them into the appropriate balance within an organization, including consistently reviewing and reassessing what is being measured (making sure that quality is not over-emphasized at the expense of velocity and efficiency).

It is also vital to measure KPIs across all enterprise platforms, as many applications are now hybrid in nature and involve multiple systems. For example, does the application ultimately reach to the back-end mainframe for transaction processing?  If so, optimizing developers’ work there is critical, especially since that platform is facing a skills shortage.

Beyond these high-level recommendations, here are several practical tips for moving the needle on each of the KPIs:

  • Increasing quality: Quality can be improved by automating repetitive manual testing. Building a replicable test bed will refine the process going forward. Increase debugging and use code coverage to ensure code is executing correctly. Also, look holistically at the application to identify potential code-changing impacts early in the development process.
  • Increasing velocity: Start with a baseline of how fast your teams currently work to deliver a specific feature. Then segment development cycles into short “sprints,” as deploying features containing fewer modules lowers the risk of introducing a bug. These more frequent team feedback loops will speed the process. Automation of unit testing and code coverage of unit tests will help developers more quickly isolate the code that requires attention.
  • Building efficiency: This is another area where sprint planning will help. Scrum meetings can provide a more disciplined approach to prioritizing workloads. Analyze the pipeline life cycle to identify where the process might be slowing down. And, as this bears repeating, automate manual processes wherever possible.

The missing KPI?

Once you’ve reexamined these KPIs, what you discover may organically lead you to a fourth one: driving engagement. The goal is turning unengaged or disengaged staff into innovators and explorers. This could be seen as a mega-KPI that supercharges all the others. And yes, it involves monitoring and measuring developer behaviors to find the connections between those behaviors and your core KPIs.

Obviously this needs to be handled appropriately, as no employee likes someone constantly looking over his or her shoulder. Consistent with DevOps transparency, we suggest bringing this new KPI out into the open as a team project. Let them know this isn’t about pointing out mistakes, rather it’s about freeing them up to innovate.

A simple way to start is with staff surveys that would regularly query them on their level of satisfaction with the team and its processes, whether they feel empowered to take risks, whether they feel bogged down in current processes, etc. Pairing this feedback with data captured from the three main KPIs will shed light on how the system can be improved.

Software can help too. There may be useful data found within DevOps tools that can identify patterns in your team’s development process that are hindering its delivery pipeline. For example: managers can determine if a high defect rate in an application immediately following an update correlates to developers who haven’t maximized their use of specific tools. The team can act on that insight to make optimal use of specific features in those tools to protect applications against future defects.

Managers can also review data to see if increases in velocity lead to decreases in application quality. It might also provide high-level insights into other DevOps KPIs, such as time spent on defect management vs. innovation.

Making all these changes as well as a commitment to measuring and improving KPIs may shake up your team for a while, but it’s a small price to pay for the reward of continuous improvement and overall improved software delivery. One not insignificant side benefit – happier, more productive, more engaged developers, and a more self-sustaining approach to improved application delivery.