Easy to say, harder to measure

The real ROI of DevOps

Justin Arbuckle
Measuring tape image via Shutterstock

Measuring the Return on Investment (ROI) from DevOps is critical to convince organizations which are falling behind that they can still shift course, transform and head to the IT future. It is also essential for companies that are embracing the industry’s forward movement, so they know if they’re on the right track and can keep turbo-charging their DevOps efforts.

Many established enterprises are stuck in reverse. Dragged down by legacy IT systems and sclerotic software work streams, these businesses simply cannot adapt and get up to speed in the continuously changing, high-velocity global marketplace of the 21st century. Delta Airlines’ technology crash, for example, dramatically illustrates how aging and inflexible IT systems can cripple a company in 2016, and lead to millions of dollars in lost revenue, as well as the dissipation of a hard-won reputation among customers.

But an increasing number of organizations are accelerating their cultural transformations in order to become software-led companies that can consistently deliver new products and services as a result of automated and collaborative workflows. Alaska Airlines, which focuses relentlessly on measuring the innovation velocity of its business, is a powerful case in point. This is DevOps at work — and it’s how IT must operate to be the business engine of our quicksilver digital economy.

Measuring the Return on Investment (ROI) from DevOps is critical to convince organizations which are falling behind that they can still shift course, transform and head to the IT future. Measuring DevOps ROI is also essential for companies that are embracing the industry’s forward movement, so they know if they’re on the right track and can keep turbo-charging their DevOps efforts.

The most proactive approach for measuring DevOps ROI is to gauge the impact of products and services on customers. But, to get there, we must also assess and analyze the efficiency and effectiveness of the software development process.

Which brings us to the one metric that’s truly relevant in business today — velocity. Without velocity, you can’t quickly recognize and remediate problems. More significantly, you’ll fall behind in the marketplace because you can’t innovate fast enough to keep pace with competitors and customer demand.

Enter DevOps: The culture of innovating at velocity

Easy to say, harder to measure. The real ROI of DevOps is how rapidly you can ship your new ideas to customers, and importantly, understand if those ideas meet customer needs.

Many may look at cost savings and headcount as their first DevOps metrics. These are important barometers but don’t measure long-term value back to the business. Two places to start for value-oriented metrics are time to delivery and quality of release.

Measuring time to delivery shows how long it takes to get new product from code checking to production. Ideally, you want to see your time to delivery drop from months, to weeks, to days and then literally to hours and minutes.

Quality of release reveals how many bugs and errors are found in each iteration of the product. Companies should look to remedy these immediately, and to develop a pre-production environment and workflow of automated testing for issues.

It’s equally important to measure how rapidly you can get feedback from customers and incorporate that into the next development cycle. Development teams must get clear, direct and actionable feedback about whether their products are moving the business forward. For example, did the two-week sprint to build a new landing page for a bank’s website increase customers online banking sessions? Did it generate more cross-sell for the bank’s product portfolio? How soon can we see an uptick in customer satisfaction? This is significant information, and the question is how fast do the developers respond to this feedback?

SEE ALSO: Does DevOps have value in finance?

The bottom line is that DevOps metrics should always connect back to the business; they need to show a measurable improvement in the customer or staff experience. This could be more quality touchpoints with customers, an improved customer-satisfaction score (such as a Net Promoter Score), or increased customer activity in areas such as online billing or account management.

“Delivering on the final product is key,” says Adam Mikeal, Director of Information Technology at the College of Architecture, Texas A&M University. “Your job’s not done if the customer can’t accomplish what they want. If the customer can’t do it, it doesn’t matter if your section of the code was good. The customer experience is all that matters.”

Another important part of this equation is measuring your team’s efficiency on the road to velocity. To truly ship ideas at speed, and positively impact your business and its customers, you also need to understand how smoothly your teams are working — in addition to how quickly they’re delivering.

Banks cash in on DevOps

Westpac Bank in New Zealand fully grasps this imperative, for example, and it enacted DevOps practices to deploy a new pricing engine for mortgages in just three months — from concept to customer. In the past, this project would have taken Westpac 18 months.

Additionally, Westpac measures how well it’s using resources. For every $1 spent on a problem, how much is chewed up by politics, and how much actually goes to solving the problem? Before adopting DevOps practices, Westpac was delivering a return of about 20-22 cents. Now, that number is just shy of 40 cents. That’s a near-2x increase in the value delivery. Westpac’s goal is 3x. In addition, Westpac has increased the amount of work its development teams accomplish by more than 40 percent over the previous year.

Like Westpac, Standard Bank in South Africa has boosted efficiency, thanks to a thriving DevOps culture. The cost of production at Standard, for example, has dropped from about $4,000 to $700 – a 300 percent improvement.

The goal for companies in every industry and economic sector around the world today is clear – accelerate your business so that you create and provide greater, and faster, value for the customer. But, in order to drive a robust ROI on this journey, it’s crucial that you carefully measure the true velocity of your organization, whether you’re just beginning with DevOps or you’re well along in the transformation.



To read more about DevOps, download the latest issue of JAX Magazine:

People like Patrick Debois, Andrew Shafer and John Allspaw (to name a few) have given us the means to transform this group of concepts into a movement but now it’s time to focus on some of its facets: continuous delivery, business culture, cloud platforms, microservices and container technology. Let’s allow our authors to convince you that these ingredients are equally important to DevOps. Mark Hinkle, VP at The Linux Foundation will tell you what’s next for the effort to improve coordination between software developers and operations personnel while Justin Arbuckle, CTO & Agile, Lean and DevOps Transformation Leader for Sococo, Inc. will explain why measuring DevOps ROI is essential for companies that are embracing the industry’s forward movement.

Thirsty for more? Open the magazine and see what we have prepared for you.


Justin Arbuckle

Justin Arbuckle is Chief Technology Officer at Leonteq. He was previously Vice President of Worldwide Transformation and Chief Enterprise Architect at Chef Software, an IT automation company that helps enterprises use DevOps to build and deliver quality software quickly. He is a passionate proponent of the transformative potential of Agile, Lean and DevOps in large enterprises. Previously, he was Chief Architect at GE Capital where he introduced and sponsored DevOps.

Inline Feedbacks
View all comments