(Agile && metrics) ? Can agile metrics help developers and teams improve?
The journey to becoming Agile can sometimes be tricky. In this article, discover nine critical success factors that make Agile metrics work for teams. What questions should you and your team be asking yourself in order to focus on self-improvement, reliability, efficiency, and high-quality code delivery?
Inevitably, most teams get to the stage where they need to adopt a more Agile delivery process. This is not just a sign of maturity. It’s a sign that the software they are developing is being used, is deemed useful, and is receiving feedback and change requests so that it continues to improve.
My team is in a unique position. We are striving to improve delivery as we develop a tool that strives to help teams do the same. In other words, we use our own tool to improve the delivery of it!
In my experience, the journey to becoming more Agile can be tricky. Each team has its own goals and ideas about how to get there. All teams, however, need to be able to reflect on their progress, measure how effective their current strategy is, and gain more visibility of the wider landscape. Of course, this is easier said than done.
Bottom-up is best
The topic of which metrics Agile teams can trust to reliably help them measure progress – or whether to use them at all – is both fascinating and contentious. Many people associate metrics with a top-down management style, which is the opposite of the decentralised, empowered and self-determining team philosophy that Agile promotes.
During a one-to-one meeting with my team lead, I asked him which metrics he felt I should focus on. He explained that the only ones worth looking at were those that the whole team agreed would help improve delivery. When it came to my own self-improvement goals, he said I should select metrics myself.
As a rule, whenever metrics are applied from the top down, the less effective they are. (This is not to say, however, that there aren’t valuable metrics that can indicate progress at a higher level.)
Using Agile metrics for team improvement
Self-improvement is a key Agile principle. On the face of it, it’s a pretty simple process. First, you identify what you want to improve. Next, establish ways to measure the attributes that contribute to improvement. Then measure and reflect. Therefore you will always need a reliable way to track progress.
My team chose Agile metrics that focus on various attributes of delivery, quality and value. For example, we measure Lead Time from the time the ticket is created in Jira to its production deployment and number of escaped bugs. We’ve created a dashboard in our own software around these attributes so we can measure, integrate and affect them daily, or as part of a retrospective. Our dashboards help guide us and qualify decisions we make around team, process, and delivery improvement to ensure we continually head in the right direction. We can also opt to see individual contributions to these metrics. For instance, I have chosen to create a view of metrics that only I can see, so I can measure my own personal output.
Using Agile metrics for delivery cycle improvement
As part of our cycle rituals, our team is responsible for making sure our scope is realistic. To support this, we use Agile metrics to ensure that the sum complexity, time and effort of our tasks match the overall time available and the team’s abilities. We measure the scope using story points. We also built and now use a ‘Sprint Report’ facility. This allows us to see a breakdown of the sprint’s overall completion, including the target completion and work added to the sprint after it started. It also includes ‘Sprint-specific dashboards’ that use metrics like ‘Completed Tickets’ to calculate the amount of work developers can reliably complete during a sprint (aka their ‘velocity’).
Nine critical success factors that make Agile metrics work for teams
As I said before, Agile metrics for team improvement can be contentious. They open up a lot of heated discussions and to varying degrees benefit from a wider understanding of context and narrative. So we discuss them and apply the following tenets to help find common ground:
Complicated metrics run counter to the Agile spirit. We like to define our journey as specifically as we can, answering simple questions with easy to understand metrics which support them like:
- Are we effective at delivering value?
- What is our lead time?
- How long is our bug backlog?
- Are our sprints going well?
- Are we completing all tickets in a sprint?
- Are we adding lots of scope to sprints?
- Is team morale high?
- Does the team understand the business value of the work?
- Does the team feel autonomous?
The metrics need to be selected by the development team and serve a common aim shared by project members from the Scrum Master to the technology leader.
You shouldn’t measure anything unrelated to your journey’s destination. Each project follows a different set of milestones so may need different metrics. However, as there is only one final destination, some carefully-selected metrics should be applicable across all teams. Less can be more, so when we build out a dashboard together in our team meetings, we try to concentrate on only a handful of metrics at a time.
Software delivery metrics are often outcome-based. Although legitimate, there’s a risk of tracking only symptoms and not root causes. The ‘Cycle Times’ metric, for example, shows how long work is taking rather than why. Descriptive metrics like these should also include details of the variables that impact the outcomes. For example, alongside Cycle Time you could show an analysis of the bottlenecks. To improve we want to uncover root causes and identify behaviour gains we can make together – we need to tell a full story.
Right sources – we need to analyse data from those sources that our developers genuinely engage within their everyday work. These include workflow management software like Jira; code repositories like GitHub and Bitbucket, TFS or Gitlab; code quality tools like Sonarqube; time tracking systems like Harvest or Tempo; and continuous delivery tools like Jenkins and GoCD.
If analysing metrics takes significant cognitive effort or time to collate, we tend to lose patience and abandon the effort. The metrics need to complement processes, not slow them down.
Agile metrics delivered in near real-time fundamentally drive improvement as they can be discussed in daily stand-ups and sprint retrospectives.
The human factor
Software development is a process (almost) completely driven by people. This means it should be possible to source information and get to the root-cause of issues very fast. Typically feedback is collected in person, in stand-ups and retros. In theory this should work well, but it can also hide issues that participants don’t want to openly communicate. This is especially true in changing, distributed teams with a mix of full-time employees and contractors. To address this and provide us with context and narrative around our metrics, we incorporate feedback into our tool. For example, when tickets get closed – we get the chance to provide feedback on how the ticket went and its requirements via Slack. These prompts also give us a feel for how a ticket has performed post dev as it continues (hopefully!) past QA.
Metrics only make sense if teams can act upon and improve them. Without constant support, many teams will not have the time or inclination to drive their own self-improvement as they strive to meet their short-term delivery objectives. Active stewardship by the technology leadership team can make a huge difference.
The limits of home-grown solutions
Since for many developers (Agile && metrics) don’t seem to get along, it’s no surprise that few analytics tools are available to measure Agile delivery effectiveness. However, now that Agile is mainstream, there is urgency to measure. Since few analytics tools were available, many teams started to build their own. This worked well on a small-scale but hit the wall when projects and teams grew.
There are several other problems with home-grown tools. Most notably, they allow teams to tweak calculations and tell an overly flattering story. Also, the time it takes to build your own tool can be a big distraction from planned work. Fortunately, new solutions are now emerging that work in line with the principles listed above.
Agile metrics for self-improvement
If you are still not convinced about using Agile metrics for teams, I recommend testing them on yourself. Most find that when they do this, the metrics become a reassurance or even a confidence boost. For example, a younger colleague of mine was struggling with his programming confidence. He found metrics to be very helpful because they showed him objective proof of improvement.
For my part, one way I often use Agile metrics is to provide insights during a retrospective. To measure how I’m improving over time, I track metrics for the tickets I’ve completed, the amount of story points completed, and the amount of returns I’ve had from QA. Crucially, this also helps me remember the tickets I’ve worked on and how they went. Like most developers, I tend to switch focus once a ticket passes and can find it hard to retain the details when it’s time to review a cycle or perform a project post mortem.
You will of course come up with your own, but I have found these example questions (and related Agile metrics) can help self-improvement:
- Where has my time been spent? I find it interesting to look back on this. The time I actually spend on high-priority work drives velocity, productivity and helps me estimate delivery dates with more confidence.
- How actively am I contributing towards ticket creation to improve quality?
- How much impact have I had? How much work did I get done and what was it?
- Am I delivering high-quality code?
- How reliable am I?
- How efficient am I at delivering value?
Whether you decide to use them yourself or for your team, (Agile && metrics) return true. In my experience, people want similar things and work well together in helping deliver on the key Agile principle of self-improvement. Try it out!