DevOps at HashiCorp: “With independence comes ownership”
“Software organizations are composed of people, processes, and technology. While it’s easy to focus on just the technology, it is ultimately just an enabler of successful people and process.” We invited Armon Dadgar, co-founder and CTO at HashiCorp to weigh in on the company’s road to DevOps and to talk about DevOps Defined, their guide for adopting DevOps to accelerate application delivery.
JAXenter: What is DevOps Defined and how can it convince people to jump on the DevOps bandwagon?
Armon Dadgar: DevOps Defined is a guide HashiCorp has published to help explain what DevOps is. While the term is often used, most customers and vendors alike struggle to define it. Our goal in publishing the DevOps Defined was to provide our definition and to frame the broader challenges of delivering a modern application. Every organization consists of people, processes, and tools. The tools support the processes, and those processes are in service of the people. The process by which an application is delivered is changing to optimize for agility, and that is the essence of DevOps.
Instead of a traditional waterfall, DevOps is about decoupling the teams and empowering individuals.
DevOps changes the application delivery process to empower individuals, remove coordination bottlenecks, and improve the velocity of developers, operators, and security teams. Almost any organization that builds software and wants to go faster can benefit from DevOps. There is no need to convince people, it is more effective to show them how their existing processes can be improved by DevOps and new tools since the challenges are familiar to any practitioner.
JAXenter: How does HashiCorp define DevOps? More importantly, how does it do DevOps?
Armon Dadgar: We think of DevOps as a process, not a collection of tools. It starts by looking at the various people and teams, primarily developers, operators, and security and figuring out how they should work together. When we look at the entire development to production lifecycle, there are seven steps. You have to write the app, test, package, provision infrastructure, deploy, monitor, and secure. If you try to remove any step, it is quickly obvious why they are all necessary. Depending on the industry there may be additional steps required for compliance or other requirements, but in general, these steps are both necessary and sufficient.
Our definition of DevOps is about organizing the process by which all seven steps are done focusing on overall throughput. Instead of a traditional waterfall, DevOps is about decoupling the teams and empowering individuals. Allowing developers to deploy and scale their services without talking to operators for example. This is where the tools come in. Vagrant allows developers to quickly set up their development environments to start or switch projects independently. Nomad allows developers to deploy their applications and manage the lifecycle and capacity independently of operators. Operators are responsible for providing the platform developers are consuming, using Terraform allows them to delegate responsibility and work in parallel managing a common infrastructure. Vault allows security teams to define policy and manage access to secret materials, while other teams can consume secrets programmatically without filing tickets against security.
below: Interactive resource monitor for modern Linux systems
with Daniel XU | Facebook
“I Work Better Alone” and Other Lies You Tell Yourself
with Casie Siekman | soona
We like to separate the problem into three discrete layers: infrastructure, security, and runtime. The infrastructure is provisioned by operators and provided to the rest of the organization. The security team uses the infrastructure provided to manage secrets and delegate access. Lastly, the runtime is the set of services and middleware provided to developers to allow them to deploy and manage their applications.
As you would imagine, this view of DevOps informs how we do things at HashiCorp. Internally, our platform team is responsible for providing the core services other teams need to deploy and manage their apps independently. The platform team provides Consul, Vault, and Nomad as a service to the rest of the organization and manages that infrastructure using Terraform. Application teams deploy with Nomad, monitor with Consul, and access secrets from Vault without coordinating with our platform team, which allows them to work faster.
JAXenter: DevOps also deals a lot with company culture. Would you say there is something like a coherent culture at HashiCorp?
Armon Dadgar: DevOps definitely requires a certain company culture to make it work. If you adopt tools and processes which empower teams, it has been said that “with great power, comes great responsibility.” What this means in practice is that if you are empowering teams to deliver independently, then with that must come ownership. Those same teams must take responsibility for the performance, scalability, security, and availability of those applications.
Without DevOps, those were the concerns of other teams, but with that came a loss of autonomy and empowerment of individual teams. For DevOps to be effective, the culture must encourage taking ownership of applications and the tools must allow teams to manage their applications effectively.
JAXenter: What are the DevOps tools HashiCorp offers and how do they help professionals deliver better applications, faster?
Armon Dadgar: HashiCorp offers a suite of tools, each of which helps with a discreet part of the application delivery process. Vagrant helps with building and testing applications, Packer to package VMs or containers, Terraform to provision infrastructure, Nomad to deploy applications, Consul to provide service discovery and monitoring, and Vault to secure secrets and infrastructure. Each of the tools solves a specific set of problems, but they are built with a common philosophy and approach.
We talk about our design philosophy in the Tao of HashiCorp, and this uniformity makes it easier for users to expand their usage from a single tool to the entire HashiCorp suite without feeling like they are starting at ground zero. Organizations typically start by bringing in one of these tools to solve a specific problem, and then adopt more tools incrementally as they have gain comfort and solve their original problem.
SEE ALSO: Mitchell Hashimoto: “It’s safe now to move to the cloud”
JAXenter: How can we use technology to empower dev and ops professionals within an organization?
Armon Dadgar: It’s important to remember that tools ultimately support processes in the service of people. There are no silver bullets, so no tool will be able to fix an inefficient delivery process. To empower developers and operators, the technology we use should focus on decoupling the responsibilities between the groups to minimize coordination and allow them to work in parallel. In practice, this means allowing developers to focus on their applications and services independently from the infrastructure and platforms which are the concern of our operators. The best way to do that today is with tools like Nomad, which allow machines to be decoupled from applications. Developers write Nomad jobs and manage lifecycles, while operators add and remove capacity from the fleet. We make it Nomad’s problem to coordinate placing applications on machines and handling failures so that we don’t need our developers and operators to coordinate.
For DevOps to be effective, the culture must encourage taking ownership of applications.
JAXenter: In your view, what are the steps to DevOps success?
Armon Dadgar: To be successful with DevOps, organizations need to be explicit about making it a goal to promote ownership and improve application delivery speed. When those goals are clear, the metrics to measure and the pain points become more obvious. One temptation is to change the entire delivery process and the tools used all at once. This almost invariably fails, as it’s very hard to fully scope out the changes while retraining and retooling overnight.
Instead, the most successful organizations start by picking the top pain points and changing their process and tools incrementally. This reduces the risk and impact to the business, and these small wins give the teams more momentum and organizational buy-in. When teams are getting started with HashiCorp tools, we have them pick an area of focus like enabling microservices, adopting cloud, or improving security. Then we help them adopt Consul, Terraform, or Vault respectively. Once the teams are proficient in the tool, that pain point solved, and their workflow normalized, then we look to the next set of challenges. This also allows customers to build trust in HashiCorp without having a big bang adoption of our entire product suite.
JAXenter: Why is cloud important for DevOps?
Armon Dadgar: Agility is the primary focus of DevOps, and cloud plays an important role there. With private data centers, we need to worry about capacity planning, procurement, long lead times, and up-front capital expenditures. This forces planning to be done months to years in advance of actual need, and the up-front costs typically force companies to take a more conservative approach. The expertise required to run data centers means there are more teams and more coordination in the application delivery process. The impact of all this is to reduce the overall agility by adding lead times, more coordination, and removing the ability to experiment.
The role of the cloud is to remove the need for capacity planning, procurement, and to switch CapEx to OpEx. In addition, we no longer need expertise in running datacenters. This allows organizations to quickly add capacity in response to growth, to run experiments cheaply, and to avoid the long-term planning required when you have private data centers. To be clear this means the cost is not usually a primary driver, and in many cases, the cloud is more expensive at scale, but for organizations adopting DevOps, the focus on agility is the priority.
Agility is the primary focus of DevOps, and cloud plays an important role there.
An interesting new trend is the use of application schedulers like Nomad in the private data center. Schedulers like Nomad decouple the hardware from the applications so that developers can scale up and down or run short lived experiments without having dedicated hardware. For organizations with existing data centers and expertise in running them, this is an attractive way to gain the agility benefits of the cloud without the additional cost or regulatory concerns.
JAXenter: What’s next for HashiCorp?
Armon Dadgar: HashiCorp has always been focused on the application delivery process with a suite of products to focus on the various challenges. We started with purely open source tools, and what we’ve seen is an incredible demand from our enterprise customers to help them adopt these tools as well. Our current focus is to continue maturing the tools and growing our community while introducing enterprise versions of the tools to help our Fortune 2000 customers be successful in their use. Within the domain of application delivery, we still believe there are many challenges that we can solve by extending our current tools and making them more accessible to a broader audience.
Thank you very much!
In my experience, Devs and Ops work best together if there is a culture of shared ownership and goals. The teams cannot throw work over the fence and point fingers. Ops teams should empower developers, who then need to take ownership.
The biggest obstacle for DevOps is attempting to adopt new tools without recognizing the cultural and process changes that need to accompany the tools to be successful.
What promotes employee satisfaction the most is being empowered to make a change. Most people in technical fields are motivated by solving problems, and empowering them to make change creates a positive cycle of identifying a problem, designing a solution, and implementing it.
The biggest advantage of autonomously-working teams is removing coordination required and allowing them to move quickly. Amdahl’s law is an important theorem that states the throughput of a system is indirectly proportional to the coordination it requires. A software organization is no different, it’s just a system composed of people.
It is important for a positive company culture to avoid blaming individuals and to understand the organizational problems. It’s often easy to blame an individual when the problem is larger and more nebulous. Having a disciplined post-mortem process and repeatedly asking why to understand root causes allows the underlying problem to be fixed instead of blaming individuals for symptoms.
Have you taken our DevOps survey yet? Help us identify the latest trends in modern DevOps
Check out our ‘Demystifying DevOps’ series:
