Is it worth measuring software developer productivity? CIOs weigh in
“The purpose of the measurement isn’t to determine whether one developer is better or worse than another by comparing their metrics,” says Mann. “Instead, the purpose is to diagnose which factors might lead to higher or lower productivity for the developer in question.” For instance, he recounts how a client using the SPACE framework unveiled a communication breakdown, which was successfully mended to reduce quality problems and rework.
These sorts of small fixes, exposed by intelligent productivity monitoring, can enable quicker turnarounds that pay out dividends. “When it comes to productivity, it’s about how quickly businesses can progress from conceptualizing an idea and defining its specifics, to planning the architecture, says Gibson. “Productivity directly translates to speed in both market entry and innovation, ultimately impacting the bottom line.”
Teamwork yields productivity gains
Improving software development productivity doesn’t have to be encouraged through metrics alone. Another significant contributing factor to overall productivity is the sense of ownership and commitment a developer has with their team.
“Team connectivity is a cornerstone of productivity,” says Gibson. “In order to have highly productive teams, people need to feel connected, and have a sense of belonging and cohesion with the teams they’re working with,” she says.
Getting a better grasp on productivity could also mean reimagining the concept in its entirety, since the typical industrial definition of what it means to be ‘productive’ simply doesn’t transfer well to the fluid software design and development process. Software isn’t like producing some mechanical widget, whose manufacturing process remains identical for each unit, says Mann. Software is more nuanced and ever-changing, and the end value differs from component to component, complicating traditional productivity measurement techniques.
“Every piece of software is unique and has a unique value,” says Mann. “It’s meaningless to say, ‘We produced twice the number of pieces of software as we did last week, so we were twice as productive,’ because this week’s software may be half the value.” Therefore, productivity measurements can often be illusions without real tangible benefits. “What we need to do is understand productivity as the amount of value we’re delivering per unit of time or cost,” he says.
The other implication is the realization that software is not created in isolation — it’s a collaborative process with many stakeholders involved in each sprint. “Most software is produced by teams of developers, not individual developers,” says Mann. Therefore, leaders should seek to evaluate the team’s productivity — Mann describes productivity as ‘value per unit time’ — over extended periods to truly gauge whether productivity improvements are effective.
“If you can assess value consistently across teams, then you can even compare their productivity,” notes Mann. However, that’s a big ‘if,’ he adds, since value depends highly on the business domain in question and varies greatly across stakeholders.
Of course, value isn’t always easy to measure, underscoring the need for a flexible approach, especially when comparing team dynamics. Therefore, instead of relying on a specific universal metric, it may be more beneficial to reveal trends relative to the teams in question.
“It’s more meaningful to compare and understand trends and use those as the basis for deeper questions,” says Mann. “For example, if one team’s productivity is trending upward while a similar team’s isn’t, we might ask what the first team is doing differently.” Asking such questions could expose knowledge to be shared throughout the company, which would help other teams improve.
In the context of developer experience, key areas to focus on take on slightly different characteristics. “When we talk about development outputs anecdotally, it’s critical to assess the key components of a developer’s experience with feedback from teams,” says Campbell. He groups these components into Clarity (How do I deploy), Ease of use (What are the minimal steps to deploy), Functionality (Is there an existing workflow, API, or SDK I can extend), and Stability (Can I be sure this won’t break in the middle of the night if I deploy it now).
By listening to the feedback from engineers in these areas, leadership can “quickly develop a level of empathy for what areas they need support to do their best work,” says Campbell. With this in hand, IT can best invest in improvements that enhance productivity and positively affect the business.
The developer and customer experience
Technical leaders should be cautious about measuring developer productivity, and if they do attempt it, results must based on tangible value to end consumers.
“Executives should ensure that productivity measures focus on customer experiences and outcomes, and that teams are agile while supporting new opportunities as they arise,” says Titcombe. “We want to prioritize ways in which technology can help us take care of patients now and in the future,” he adds.
Leaders should also remember that mental energy has limitations, and burnout is a real possibility for knowledge workers. As such, when measuring performance, Gibson says it’s essential to focus on processes rather than individuals to avoid instilling fear. “By emphasizing the effectiveness of the overall process and assessing the efficiency of the measurement process itself, the focus shifts to how well individuals are operating within that framework,” she says.
For others, measuring developer productivity alone can be a red herring. Instead, Campbell encourages fostering a culture of continuous improvement and discovering strategies to better instrument developer workflows, and from there, measure this toolchain to garner actionable development insights. “Just as we measure the impact our software has for end users trying to achieve a goal, we can also measure the impact our internal tools have for our objective, too,” he says.
