With Scrum projects, we have a few units of measurement. User stories, or features, are measured in story points, a relative estimation technique. Tasks are measured in hours. User stories can be estimated well in advance of the actual work being done, and do not depend on the team member performing the work. Tasks are estimated during sprint planning, and are more dependent on who is doing the work. So how do we use these to provide metrics for projects using Scrum?
The end-all-to-be all metric is velocity. Velocity is the number of story points a team is completing each sprint. We use velocity for forward planning. One argument I’ve heard against velocity is that a team needs a few sprints to determine their velocity. I’ve found this to be true, and not too difficult to explain to management. It’s much easier if your cycle times (sprints) are shorter. We develop in one week increments.
Another metric is the number of hours it’s taking to complete tasks, or as a ratio: actual:estimate.
And one more item we can track is impediments – how many impediments are showing up each sprint, how many are we overcoming each sprint, and how many come up over and over again (showing perhaps systemic problems in the organization).
With all of these, the point is to improve our estimates over time. Kaizen – continuous improvement – is a continuous goal in Agile.
But I can’t help to feel something is missing. Are there only two things we can track on Scrum projects?
Nope. There are many more.
Borrowing a page or two from Kanban, I propose that we can use the following additional metrics on our Scrum projects:
- Quality – number of bugs opened and closed per sprint
- Cost – the amount of money it cost to produce a feature or user story
- Lead Time – the amount of time it takes a feature to go from planned to implemented
- Feature Complexity – if you are breaking features into stories, you can measure how many stories it takes to complete a feature
What other metrics are you using on your Scrum projects?
Other Posts That Might Interest You
I would add something like Time To Market, which is Lead Time but defined a bit differently. If you work on living system it doesn't really matter how fast you can develop a feature. What does matter is how fast you can deliver a feature to users. What you want to know is not only how much time it takes to develop some code, test it or fix it. You also think about deployment, possibly some integration tests and sanity checks.
With short iterations in development and established working system it may appear that it is not development but deployment part which is the most costly.
By the way: we faced this issue as we changed final state for our features from 'ready to ship' to 'shipped'. Deployment part adds significantly to lead times.
On a side note: I believe the less measures you have the better, as long as they give you complete information you need. I see velocity and lead time as interchangeable metrics delivering almost the same information: how fast you can run.
Hi,
Interested in the feature complexity metric. How would you measure it ?
-kaustabh
Other important but easy to gather metrics are
1. Planned Vs Actual user stories
2. Total planned Vs Total accepted at project level (product backlog+sprint backlog)
Two great additions, thanks.
In Kanban, the goal is to have every feature that goes through the system be of the same size, so you can achieve a continuous flow. In Scrum our user stories are of varying sizes, so using story points rather than hours takes the complexity of a feature into account.