ATLANTA — You’ve heard the benefits of DevOps, and you’ve decided to move your teams to this way of working. But how do you know if you are doing it right? Metrics are key indicators for businesses to figure out whether or not they are making the right decisions, but often they aren’t choosing the right metrics to look at, according to Anders Wallgren, CTO of Electric Cloud.

He spoke at the Agile 2016 conference in Atlanta today about the key DevOps metrics businesses should look into to determine failures or successes. According to him, the three types of measurements developers should be looking at are efficiency, effectiveness, and culture. “You want to make sure you are looking at all three of them. It makes no sense to just look at the cost part,” he said.

(Related: cPrime reveals a new approach to agile)

According to Wallgren, efficiency refers to internal metrics such as the costs, time and resources being used. Effectiveness refers to value delivered: Did you build a good app, or a great app? And while culture is a difficult thing to measure, it relates to the process overhead, trustworthiness, the type of environment you are working in, and team motivation.

Wallgren explained that businesses should pay attention to culture in particular, because if developers don’t, it is going to difficult to be good at agile or DevOps. Key performance indicators of culture include satisfaction and retention. Businesses can measure this through DevOps investments, team leader effectiveness and experience, Continuous Delivery practices, deployment pain, management practices, and “win-win” outcomes, he explained.

The best practices for measuring effectiveness and efficiency include using objective measurements, focusing on outcomes, identifying predictor metrics, keeping the signal-to-noise ratio, measuring team performance instead of individual performance, and avoiding local optimizations.

In order to successfully apply these best practices, metrics must be automatic, unobstructive to the app and end users, be applied to all areas of the application (not just production), easily visible, measurable, and supplemented with technology.

A major pitfall for businesses is tracking vanity metrics such as number of tickets closed, number of users, or raw page views. But those numbers don’t really tell them anything, according to Wallgren. The metrics businesses measure should be actionable, accessible and auditable. “Vanity metrics are the quicksand of metrics. Stay away,” he said.

So when it comes down to it, Wallgren said businesses should measure things like lead time, stories delivered, customer satisfaction, and retention cost. “It is important to be a little bit balanced in the kind of metrics you look at,” he said.