Developer Productivity vs Effectiveness
This was posted by Andrew Boyagi the other day on LinkedIn.
There, I fixed it for you. (referring to the image above)
There's an unhealthy obsession with measuring developer productivity. Imagine the possibilities if the same amount of time and energy went into helping developers be more productive.
I speak to a lot of developers; I'm yet to meet one who doesn't want to be productive.
You already know if your developers are as productive as they could be; they've probably already told you what they need.
Give them what they need, and they will be more productive.
I'm not saying don't measure, but measuring isn't going to get you closer to the outcome you're after. Balance where you put your energy and time.- Andrew Boyagi
While its not the first this has ever been said, it needs to be echoed as many times as possible. While I may not agree with every word, I definitely agree with the overall sentiment of it.
Over the past 20 years or so, almost every company has become a software development company. They have had to in order to survive. You can no longer be just a bank, hardware store, or an engineering company.
I recently worked with a civil engineering company that was really coming to this realization. They had done an analysis, and ~55% of their new contract margin was in software services provided alongside the actual engineering work.
This puts a very new but real set of pressure on an organization they are not yet familiar with… The majority of their business is about their ability to deliver software and not engineering. So what is the natural thing to do? Try and improve their ability to deliver software in the most efficient way possible.
For most other types of work, this would begin with trying to figure out ways of measuring productivity. This makes a ton of sense in a lot of cases. For the traditional engineering company, you might look at how many electrical transmission poles they install over a period of time or how well a project is staying on track.
This works okay in these cases since there is a well-defined set of tasks with small amounts of uncertainty or variation. This is rarely the case for most software development, especially in the “agile” use cases. I put quotes there, considering most really aren’t very agile; they claim to be… but that’s a whole different article.
This really is the root of the problem when it comes to measuring developer productivity. The best you can do is measure the output of certain types of things like stories, story points, commits, PR reviews, etc. These are all arbitrary and really don’t provide any tangible insights into how well a team is actually doing.
You can use some of these to compare one team to another, but these are also only relative comparisons. You might be able to say that one team, on paper, looks better than another, but once the folks know you measure something one way… then it becomes very easy to game the system.
For example, if you are measuring commits to try and identify who is more productive on a team, then what would stop me from just committing once an hour with whatever I had completed? It would make my numbers look good but have zero impact on how well the team might be performing.
In that same scenario, if you have an excellent tech lead who is working on a new major feature design with the product lead. She might clearly be the best on the team, but based on your metric, she would fall to the bottom quickly.
“You can’t (effectively) measure engineering productivity directly, but you can measure & eliminate waste.” (paraphrased)
-Google, Nicole Forsgren, and others
This really is the root of trying to measure productivity. It becomes a very, very difficult problem with a murky, at best, answer at the end of it.
To be super clear - This is NOT me saying not to measure these things and gather quantitative information. This is really important data; you can’t try and use it to define the productivity of a team or especially an individual.
What you can do is use the type of information to identify and measure waste and friction that gets in the way of developers and development teams. Or, to put it differently;
Directly measuring productivity is a fool’s errand. Instead, focus on abstracting away complexity and measuring impact through qualitative and quantitative metrics.
I get frustrated with my friends over at Google sometimes as they seem to always talk about developer productivity… but when you look at what they are actually doing, it is all about driving efficiency. They have a developer productivity team of ~2,000 people, but they really focus on improving developer tooling and processes more efficiently. They do a ton of qualitative and quantitative research to figure out where the friction points are and eliminate them. They have a significant amount of UX developers on the team as well which is another key element of driving out inefficiencies.
Whatever words you want to use, productivity or effectiveness, I really don’t care, but the goal is the elimination of friction across the system. This should be done from local environments all the way to the validation of code in a production environment.
Here is an interesting metric that I like to ask people when we sit down to talk about DevEx: how long does it take someone to validate local code changes? In a lot of companies, there seems to be little care for local environments or for the machines that a dev might use on a day-to-day basis.
This is the closest source of friction for any dev in almost any environment and can have a serious impact to their productivity and their overall happiness. If you are constantly frustrated all day with your machine or local environment, this could flow over into other areas of your day.
If you’re a leader looking to increase “productivity,” then focus on the removal of friction across your org. This isn’t hard… I guarantee most of your devs know where the major problems are, all you need to do is ask them. Let them know that the light at the end of the tunnel isn’t a train… its the sun peaking out from the clouds.