Is Measuring Developer’s Performance
Feasible & Objective?
A Thoughtful Brainstorming!
When discussing with people from management to developers, the same questions come up all the time but with different words from different perspectives: “Does my software development team perform well?” or “How should I measure their performance?”
These questions derive from different perspectives depending on the position of the person making them. For example, the management wants a well-performing development team in order to increase the time-to-market which results in increased profitability. The product owner asks such questions in order to make better planning and increase the pace of delivery for new features or bug fixing. The Lead developer wants to know the capabilities, the knowledge sharing and the satisfaction of his team.
In order to answer the questions above, multiple KPIs have been used with most of them might be familiar to you, if you are working in software development. Below you can see listed the most common and popular.
While you were reading them, you were arguing with yourself if each one of them is suitable or not to answer questions related to performance and I can guess that you have found flaws in each one of them. However, the real question here is how they can be combined to provide a qualitative output of developers’ performance quantitatively. To answer this question, we should define what these KPIs serve individually.
The most popular propositions for acquiring insights into developers’ performance are “DORA” metrics and “SPACE” framework. DORA metrics are used by the DevOps team to measure their performance and include the Deployment Frequency, Lead Time for Changes, Mean Time to Recovery and Change Failure Rate. The goal of the “SPACE” framework is to shape a landscape for developers to perform better and basically understand developers’ productivity, not measure it. One can say that the DORA metrics are an implementation of the “SPACE” framework.
However, in order to understand why these metrics are used by “DORA” and how the “SPACE” framework works, we should define what each KPI serves in terms of Developers’ Productivity, Efficiency, and Quality. The reason for selecting these attributes is that the first sentence on “SPACE” framework is “A clear understanding of defining, measuring, and predicting developer productivity could provide organizations, managers, and developers with the ability to make higher quality software—and make it more efficiently” where the main words which should be quantified in order to make better predictions and measurements are the ones marked with bold.
It is obvious that none of these KPIs is allocated to all the properties of Developers’ Performance and this is the point where“DORA” metrics and the “SPACE” framework come into the game of Developers’ Performance quantification. As you can see the “DORA” metrics, which are used by many organizations to measure Developers’ performance, cover Productivity & Efficiency with 3 different KPIs and Quality with one KPI.
However, we are going to focus on the “SPACE” framework which provides a way of thinking, considering the fact that AI is becoming part of Developers’ daily life. First of all, the “SPACE” framework has 5 dimensions, and each one of them has 3 different levels. Specifically, the dimensions are:
- Satisfaction & Well-being (How fulfilled, happy, and healthy one is?)
- Performance (An outcome of a process)
- Activity (The count of actions or outputs)
- Communicate and Collaboration (How people talk and work together)
- Efficiency and Flow (Doing work with minimal delays or interruptions)
Considering the fact that the organizations are not full of individuals without cooperate with each other, the “SPACE” framework dimensions have 3 different levels which are:
- Individual (One person)
- Team or Group (People that work together)
- System (End-to-end work through a system. [e.g. development pipeline])
The above way of thinking can shape a totally different way of measuring Developers’ Performance, adaptable to every organization’s specific needs and requirements. In the following table, I have provided an illustrative example that serves as a starting point for further consideration as to how a more contemporary and holistic evaluation system for Developers’ Performance could be built. As mentioned, this approach is subject to variations depending on the structure, philosophy and needs of each organization.
Level
Individual
Team
System
Satisfaction & Well-being
Retention Rate
Retention Rate
Satisfaction with engineering system (CI/CD pipeline)
Performance
Code Review Velocity
Story Points Shipped
Code Review Acceptance Rate
Activity
-Commits
-Lines of Code
-Lines of Code
Story Points Completed
Frequency of deployments
Communication & Collaboration
PR merge times
PR merge times
Knowledge Sharing, discoverability
Efficiency & Flow
Code Review Timing
Code Review Timing
Code Review Timing
Conclusion
It is evident that you cannot find a magic number that answers all the questions. Think about it, does age say everything for a person? I don’t think so.
For example, let’s imagine a person who is 28 years old and another person who is 30 years old, has been born in Australia, with a Master’s Degree in Finance, currently works for an Investment Fund and his hobby is basketball. Obviously, you have limited information for the first one and cannot shape a figure. On the other hand, the qualitative characteristics provided in the second case combined, they can provide a better image of the person described.
In software development, if we measure only code “Code Churn” we might draw false conclusions about the performance of the team. However, if we combine the “Code Churn” with “Code Complexity”, “Bug Rates” and “Test Coverage”, someone can make a safe assumption regarding the quality of the software and the pace of the team. Imagine a software team that delivers plenty of code in terms of volume (Code Churn) without taking into consideration the quality of the software. In the long term, the volume may remain at a high level, but instead of delivering new features, it is aimed at fixing bugs, due to the technical debt constantly created because of code “mass production”.
In short, in order to assess Developers’ Performance in an organizationally effective yet people supportive way, you have to define a system that covers all relevant dimensions adequately, extending from the individual to the system as a whole. Secondly and most importantly, you should treat KPI scores as indexes of a living organization and not as absolute numbers that can be interpreted out of context.
Sources: