DevOps at HashiCorp: “With independence comes ownership”

JAXenter: What is DevOps Defined and how can it convince people to jump on the DevOps bandwagon?



Armon Dadgar: DevOps Defined is a guide HashiCorp has published to help explain what DevOps is.
While the term is often used, most customers and vendors alike struggle to define it.
Our goal in publishing the DevOps Defined was to provide our definition and to frame
 the broader challenges of delivering a modern application. Every organization consists of people, processes, and tools. The tools support the processes, and those processes are 
in service of the people. The process by which an application is delivered is changing 
to optimize for agility, and that is the essence of DevOps.

Instead of a traditional waterfall, DevOps is about
 decoupling the teams and empowering individuals.

DevOps changes the application delivery process to empower individuals, remove coordination
 bottlenecks, and improve the velocity of developers, operators, and security teams. Almost 
any organization that builds software and wants to go faster can benefit from DevOps. 
There is no need to convince people, it is more effective to show them how their existing
 processes can be improved by DevOps and new tools since the challenges are familiar to
 any practitioner.

JAXenter: How does HashiCorp define DevOps? More importantly, how does it do DevOps? 

Armon Dadgar: We think of DevOps as a process, not a collection of tools. It starts by looking at the various
 people and teams, primarily developers, operators, and security and figuring out how they should work 
together. When we look at the entire development to production lifecycle, there are seven steps.
 You have to write the app, test, package, provision infrastructure, deploy, monitor, and secure.
 If you try to remove any step, it is quickly obvious why they are all necessary. Depending on the 
industry there may be additional steps required for compliance or other requirements, but in
 general, these steps are both necessary and sufficient.

Our definition of DevOps is about organizing the process by which all seven steps are done focusing on overall throughput. Instead of a traditional waterfall, DevOps is about
 decoupling the teams and empowering individuals. Allowing developers to deploy and scale their services
 without talking to operators for example. This is where the tools come in. Vagrant allows developers 
to quickly set up their development environments to start or switch projects independently. Nomad allows
 developers to deploy their applications and manage the lifecycle and capacity independently of operators.
 Operators are responsible for providing the platform developers are consuming, using Terraform 
allows them to delegate responsibility and work in parallel managing a common infrastructure.
 Vault allows security teams to define policy and manage access to secret materials, while other teams 
can consume secrets programmatically without filing tickets against security.

We like to separate the problem into three discrete layers: infrastructure, security, and runtime.
 The infrastructure is provisioned by operators and provided to the rest of the organization. The
 security team uses the infrastructure provided to manage secrets and delegate access. Lastly, the 
runtime is the set of services and middleware provided to developers to allow them to deploy and
manage their applications.

As you would imagine, this view of DevOps informs how we do things at HashiCorp. Internally,
our platform team is responsible for providing the core services other teams need to deploy
 and manage their apps independently. The platform team provides Consul, Vault, and Nomad as
 a service to the rest of the organization and manages that infrastructure using Terraform.
 Application teams deploy with Nomad, monitor with Consul, and access secrets from Vault without
coordinating with our platform team, which allows them to work faster.

JAXenter: DevOps also deals a lot with company culture. Would you say there is something like a coherent culture at HashiCorp? 

Armon Dadgar: DevOps definitely requires a certain company culture to make it work. If you adopt tools
 and processes which empower teams, it has been said that “with great power, comes great responsibility.”
 What this means in practice is that if you are empowering teams to deliver independently,
 then with that must come ownership. Those same teams must take responsibility for the performance,
scalability, security, and availability of those applications.

Without DevOps, those were the
 concerns of other teams, but with that came a loss of autonomy and empowerment of individual teams.
 For DevOps to be effective, the culture must encourage taking ownership of applications and 
the tools must allow teams to manage their applications effectively.

JAXenter: What are the DevOps tools HashiCorp offers and how do they help professionals deliver better applications, faster?


Armon Dadgar: HashiCorp offers a suite of tools, each of which helps with a discreet part of the application delivery process.
 Vagrant helps with building and testing applications, Packer to package VMs or containers, Terraform to provision infrastructure,
 Nomad to deploy applications, Consul to provide service discovery and monitoring, and Vault to secure secrets
and infrastructure. Each of the tools solves a specific set of problems, but they are built with a common philosophy
and approach.

We talk about our design philosophy in the Tao of HashiCorp,
 and this uniformity makes it easier for users to expand their usage from a single tool to the entire HashiCorp suite
 without feeling like they are starting at ground zero. Organizations typically start by bringing in one of these tools 
to solve a specific problem, and then adopt more tools incrementally as they have gain comfort and solve their original problem.

SEE ALSO: Mitchell Hashimoto: “It’s safe now to move to the cloud”

JAXenter: How can we use technology to empower dev and ops professionals within an organization?

Armon Dadgar: 

It’s important to remember that tools ultimately support processes in the service of people.
 There are no silver bullets, so no tool will be able to fix an inefficient delivery process.
 To empower developers and operators, the technology we use should focus on decoupling 
the responsibilities between the groups to minimize coordination and allow them to work in parallel.
 In practice, this means allowing developers to focus on their applications and services independently
 from the infrastructure and platforms which are the concern of our operators. The best way to do 
that today is with tools like Nomad, which allow machines to be decoupled from applications.
 Developers write Nomad jobs and manage lifecycles, while operators add and remove capacity from 
the fleet. We make it Nomad’s problem to coordinate placing applications on machines and handling
 failures so that we don’t need our developers and operators to coordinate.

For DevOps to be effective, the culture must encourage taking ownership of applications.

JAXenter: In your view, what are the steps to DevOps success?



Armon Dadgar: To be successful with DevOps, organizations need to be explicit about making it a goal
 to promote ownership and improve application delivery speed. When those goals are clear, 
the metrics to measure and the pain points become more obvious. One temptation is to change 
the entire delivery process and the tools used all at once. This almost invariably fails, as
 it’s very hard to fully scope out the changes while retraining and retooling overnight.

Instead, the most successful organizations start by picking the top pain points and changing
 their process and tools incrementally. This reduces the risk and impact to the business, and 
these small wins give the teams more momentum and organizational buy-in.

 When teams are getting started with HashiCorp tools, we have them pick an area of focus
 like enabling microservices, adopting cloud, or improving security. Then we help them 
adopt Consul, Terraform, or Vault respectively. Once the teams are proficient in the tool, 
that pain point solved, and their workflow normalized, then we look to the next set of
 challenges. This also allows customers to build trust in HashiCorp without having a big bang
 adoption of our entire product suite.

JAXenter: Why is cloud important for DevOps?



Armon Dadgar: Agility is the primary focus of DevOps, and cloud plays an important role there.
 With private data centers, we need to worry about capacity planning, procurement,
 long lead times, and up-front capital expenditures. This forces planning to be done 
months to years in advance of actual need, and the up-front costs typically force
companies to take a more conservative approach. The expertise required to run 
data centers means there are more teams and more coordination in the application 
delivery process. The impact of all this is to reduce the overall agility by
 adding lead times, more coordination, and removing the ability to experiment.

The role of the cloud is to remove the need for capacity planning, procurement,
and to switch CapEx to OpEx. In addition, we no longer need expertise in running
datacenters. This allows organizations to quickly add capacity in response to
growth, to run experiments cheaply, and to avoid the long-term planning required
when you have private data centers. To be clear this means the cost is not usually a
 primary driver, and in many cases, the cloud is more expensive at scale, but for 
organizations adopting DevOps, the focus on agility is the priority.

Agility is the primary focus of DevOps, and cloud plays an important role there.

An interesting new trend is the use of application schedulers like Nomad in
the private data center. Schedulers like Nomad decouple the hardware from the 
applications so that developers can scale up and down or run short lived experiments
 without having dedicated hardware. For organizations with existing data centers
 and expertise in running them, this is an attractive way to gain the agility 
benefits of the cloud without the additional cost or regulatory concerns.

JAXenter: What’s next for HashiCorp?

Armon Dadgar: 

HashiCorp has always been focused on the application delivery process with
a suite of products to focus on the various challenges. We started with purely
 open source tools, and what we’ve seen is an incredible demand from our
 enterprise customers to help them adopt these tools as well. Our current 
focus is to continue maturing the tools and growing our community while 
introducing enterprise versions of the tools to help our Fortune 2000 customers 
be successful in their use. Within the domain of application delivery, we still 
believe there are many challenges that we can solve by extending our current 
tools and making them more accessible to a broader audience.

Thank you very much!

Have you taken our DevOps survey yet? Help us identify the latest trends in modern DevOps