It’s a new year, and organizations around the world are giving developers goals for the new year and reviewing their past year’s efforts.
A question I often hear is, ‘How do you assess a developer’s work, and his/her worth to the organization?’
Some organizations still cling to the metric of lines of code produced by a developer, which — given the extra responsibilities of testing, ensuring security, adhering to policies and regulations, and more — might not be a fair valuation in today’s complex world.
This method is entrenched in the finger-pointing of the past, which modern development organizations have largely eschewed as they look to create a blameless culture.
Forward-thinking companies will look at the role of the team around development, assembled with software engineers, testers, security experts and people from the business side, and look holistically at how that team performs.
“Line counts is a terrible metric, and I think we all agree on that,” said Chris Downard, vice president of engineering at Gigsmart, a website for hiring gig workers. “There are times … when it could be useful as an additional data point, but not necessarily for information.”
When you’re managing humans, he said, reducing every action to data points is not good. Time must be spent building context, as data can often misrepresent things. At Gigsmart, Downard said they don’t use sprints, instead taking what he called “an ongoing, non-stop kind of combat approach.” But they do use sprint reports, from metrics captured every two weeks, to communicate what happened in that time period.
He pointed out he knew what his team was doing between the sprint reports — they were working hard, pairing up, and he saw the number of merge requests going up. “But one of the normal indicators of productivity is, ‘are we moving things across the line to delivered,’ as points completed,” he said, and that number was going down. But based on their knowledge of the team and of the context of everything else going on, they discounted the number, knowing the team’s productivity was very, very high. “It’s just the way the ticketing shook out, producing a data point that was not necessarily indicative of what was accurate,” he noted.
As an organizational leader, Downard said, you need to think about the things you want the organization to produce, and then think about the measurements that will indicate that you’re having success or struggling. Different teams, of course, have different goals.
“If you’re running a DevOps team, you might care about time to resolution, and if you’re tracking the development portion of an IT department, it might be turnaround time for customizing reports and data stuff. You need to track the things that matter to your organization’s success. So for us, I track merge request counts for a week. And we don’t necessarily do anything with that data. It’s not a carrot-and-stick thing. It’s just, it gives me additional information. Kind of like a doctor would be diagnosing a patient.”
But data points often don’t align with assessing developer productivity because while much programming involves the logical reasoning side of the brain, it also involves the creative side. So for Downard, raw data points are “typically terrible. But what we do get is a lot of soft indicators. You get information out of standup updates of people communicating how they feel about what they’re doing. You get hard data points in the sense that you can see their commit activity, but you have to keep context.” As a leader, he said, you have to advocate for developers and translate what they’re running into, to every other organization around development.
Downard said Gigsmart uses Bushido, the samurai code of conduct that defines the values of how you should act and conduct yourself as an individual, as its organizational ethos. “Jason Waldrip, our CTO and I sat down and crafted it into a set of ideals to drive the organization, and I use that as the core for everything we do. So if I’m going to start tracking something, it has to map to some sort of value from there, because if I try to track things that don’t map well to those values, I can’t advocate for those values with the team. It’s not gonna stick, it’s going to become hollow.”
Data points, he said, are nothing more than signals to go look into something and start asking questions. “And it should always be exploratory, not accusatory. That’s important to us.
