“Make sure you think about the damage you can do by blindly optimizing for a metric”
How do we know what’s important enough to measure? How do we get a holistic picture of organizational health? How do we ensure our metrics help us more than they hurt us? JAXenter editor Gabriela Motroc caught up with Julia Wester at DevOpsCon 2018 to talk about all this and more.
Data-driven decision-making is a goal of most leaders today. Unfortunately, between the overwhelming amounts of data available to us and the ease of falling prey to misleading vanity metrics, how to begin this journey can feel extremely elusive.
How do we know what’s important enough to measure? How do we get a holistic picture of organizational health? How do we ensure our metrics help us more than they hurt us? There are many important questions to consider before doubling down on your inventory of metrics. JAXenter editor Gabriela Motroc caught up with Julia Wester, co-founder and Principal Consultant at Lagom Solutions to talk about all this and more.
Here are some quotes from the interview:
- We have a lot of metrics and not all of them matter, therefore we can cause a lot of damage without knowing it
- We need to make sure we can map every metric to an outcome we want to achieve
- We need to anticipate all the behaviors metrics can drive
- Look at trends over time instead of looking at individual data points
- Metrics should be interactive but each should be shown in a different way. Don’t use pie charts though!
- Make sure you think about the damage you can do by blindly optimizing for a metric (think of the unintended consequences). Go into this with your eyes wide open.
In her session at DevOpsCon 2018, Julia Wester discussed how sometimes too much information is as bad as not enough information in her session “Finding Metrics that Matter and Using them Safely”. There are no one-size-fits-all metrics, although there may be some common, good baseline metrics. Wester spoke about how to do metrics the safe way, how to know when to discard metrics, and what missteps might be taken.
Some key points on how to do metrics the safe way:
- Start with a clear outcome in mind
- Walk the path to discover your metrics: ODIM (outcome, decision, insight, measure)
- Know what each metric really tells you and evaluate your metrics with a “so what” test: Does this matter to my customer?
- Choose metrics that compete with each other. We don’t always see the consequence of our actions, but this helps us see the negative consequences.
- A balanced dashboard should include: quality, responsiveness, productivity, and sustainability
To avoid missteps, Julia cautioned that it’s easy to over-optimize things and that superstars aren’t always the best team members. She stressed that using shame as an incentive is a misstep and that teams should care more about trends than numbers. Wester left her audience with the takeaway that you don’t need perfect data, you just need more info than you have right now.
Julia Wester is the co-founder and Principal Consultant at Lagom Solutions, a Lean/Agile consulting and training company that leverages her 17 years of experience working in and managing teams at Turner Broadcasting, F5 Networks and LeanKit. Julia is passionate about teaching others how to tame the chaos of everyday work by embracing transparency, continuous improvement and a lagom mindset, as well as talking about how management doesn’t have to be a dirty word. Julia blogs at everydaykanban.com and tweets at @EverydayKanban.