Estimating the work of developers is more art than science. Developers usually use story points or hours for their estimates, or sometimes they use a “planning pokerexercise to determine what they can accomplish in a given sprint. But these estimates are not an accurate measure of the development team’s velocity.
Take story points, for example. How many can you complete in a given sprint? You decide on a number, plug it into a burndown chart, and it becomes your velocity. The problem is that story points can mean something different to each person. Consider two developers working on the same project. One may estimate three story points for a task while the other estimates five. How to compare the two?
It is not only difficult to compare individuals but also teams. As soon as you hire a new developer, you revert to the assault and standardization stages of team development, and your previous vision of team speed falls apart. Story points are just too subjective to give you a clear directional picture. Even when estimated with hours or a poker schedule, velocity estimates remain subjective.
I’ve thought a lot about how development teams measure velocity, and it’s clear to me that we need a better way. To that end, I started experimenting with new definitions of velocity – something that would make more sense to me and development teams in general. Here is what I found.
A new way to measure product velocity
I recently started a new role as CTO of Higher level, an engineering analytics organization that helps development teams and executives be driven by data. But before joining Uplevel, I was a customer of the company. That’s when I started using his deep work insights to measure the uninterrupted working time my teams were getting each day. I was already familiar with the concept of Deep Work, but it resonated even more with me in a development context.
Then I looked at their pull request information. Although this information is not new to me, the data used to circulate without really being put together in an actionable format. Together, these ideas of Deep Work and PR are what ultimately led me to a new definition of velocity. I now think of velocity as having two main components:
- The first is abilitythis is where the information about Deep Work comes in. The more Deep Work hours you can free up for your team, the more capacity they will have to design, write code, and deliver new products.
- The second component is efficiency, which is necessary to work within this expanded capacity of Deep Work. Efficiency is about doing the right things in the right way. To measure efficiency, we first look at cycle time information related to pull requests, i.e. the time it takes to process the request. The other insight we look at is PR complexity, another indicator of cycle time.
Here’s what it looks like in action. A developer sits down to write a small logical unit containing up to several dozen lines of modified code, with few files affected, behind a feature or environment flag. As this is a minor change, the code review will be faster. It’s a different story when developing an entire feature, submitting PRs with 50 affected files and hundreds of lines of code changed. In this case, the code review process will take longer and involve a lot more back and forth.
By the time you complete the PR and are ready to return to the main vein of code, the vein itself will have progressed. Consequently, you will encounter merge conflicts, which you will need to resolve as other people work on the same area of code and progress. To be as efficient as possible, it is best to work in small logical units of code using security flags for production to prevent it from running in production. This allows you to move code into the main stream faster than you could write larger chunks of code changes.
In summary, I think speed is best measured by looking at your teams uninterrupted focus time (capacity) and how effective they are during those times, with a focus on cycle time and PR complexity.
You can’t predict future sprints
While I view the above approach as a better way to measure velocity, it neglects one of the main promises of a story point approach: predicting future sprints.
When you use story points to measure velocity, you predict what future sprints will look like based on previous trends. For example, if you worked on 10 story points over 10 sprints, you can expect the same number in the next sprint. In reality, story points do not actually correlate with the prediction of future sprints due to their subjective nature.
So while you can’t predict future sprints when you use my approach to measuring speed, you’re not missing out. I’d say you never really had a way to predict sprints anyway, so you don’t lose anything by trying to create more capacity and efficiency for your developers. That doesn’t mean you still can’t use story points. They can still be useful for planning, but not as a measure of speed.
I don’t claim this alternative measure of velocity as my own, as other development teams may already be taking a capacity/efficiency approach. But I see it as a more objective way to measure speed versus subjectively assigning story points to a task.