Building a better DevOps-enabled cloud
As the number of applications utilizing cloud technology grows, it is more expensive to have DevOps teams build each environment as a wild west project. The answer is to implement a cloud architecture that standardizes on stack-level operations. In this article, John Mathon gives some tips on building a better DevOps cloud.
There’s no way around it. Building a DevOps cloud is complex. For starters, a typical cloud deployment can contain 30-plus product components that need to be integrated to run together smoothly. At any time, an update to one of these products can break processes that rely on the other components.
DevOps compounds the complexity since teams need to ensure consistency across development, test, and production environments. When enterprises have one project, it’s fine to have a dedicated DevOps team in the group handling these environments. However, as the number of applications utilizing cloud technology grows, it is expensive to have DevOps teams build each environment as a wild west project.
The answer is to implement a cloud architecture that standardizes on stack-level operations. Implemented effectively, a stack serves as the reference architecture for all the components, an automation needed to create a replica of your production environment. This “full stack” is in essence similar to the configuration management database (CMDB) once thought to be the holy grail of enterprises.
Having a meta-description of your full stack for a service or application makes it possible to deploy identical copies for development, test and production, as well as automate DevOps across your application. This ability to duplicate stacks removes the barriers to, for example, deploying multiple test environments or multiple copies of the stack for security compliance or disaster recovery.
The stack is an important concept for understanding how you are building and running your software. If something goes wrong, it also provides visibility into what is running, how it was deployed, and what the problem might be, facilitating the ability to systematically correct the problem and deploy the fix to all needed replicas. Moreover, developing a stack concept lets you track applications—plus the associated tools, development practices, cloud practices, and vulnerabilities—across your organization, providing a critical step in the maturity of your enterprise’s cloud development.
A DevOps cloud architecture will consist of at least two main stacks: the core infrastructure stack and the DevOps stack. Increasingly, these stacks run in containers, which provide both greater agility through the ability to spin up apps faster and more cost-effective use of resources through operating system-level virtualization.
The core infrastructure stack supports a service or application with services that ensure security, reliability, and scalability. It includes cloud infrastructure, such as networking and storage, which may be handled by a public cloud platform or a private cloud, as well as container-specific functionality, such as a Docker runtime with orchestration provided by Docker Swarm or Kubernetes.
Next is a layer that handles performance and management, including networking, load balancing, service directory, log management, monitoring, backup and recovery, and vulnerability scanning, which are supported by a range of different products.
At the top of the core infrastructure stack is an application layer or stack that includes development tools, such as Java Stack or Node.js Stack. More advanced application stacks will go beyond tools to provide the blueprints for migrating from monolithic apps to containers and microservices.
The DevOps stack supports continuous integration/continuous deployment (CI/CD). It runs across multiple infrastructure stacks and codifies the tools, processes, and procedures to test and deploy or rollback changes to the infrastructure stacks. It usually incorporates DevOps instructions using Chef or Terraform, Ansible, Puppet, Swarm, Helm, and half a dozen other tools. Because automated testing is critical, DevOps stacks also should include tools that help with development, building, integration, automated testing for functionality—such as WireMock, Selenium, performance testing using Artillery, and security testing using Blackduck or Clair.
Additionally, there are key functions within the cloud platform that extend across development, test and production stacks. These include security, a catalog of stack templates, a registry for certified containers, and usage data collection to apply predictive analytics for performance and cost optimization.
Standardizing on stacks enables development teams to focus on building new applications and services rather than spending 6 to 18 months implementing the cloud architecture. At the same time, different groups may require different capabilities, making a one-stack-fits all-approach impractical. The medium ground is to provide a set of “sanctioned stacks” or a portal for configuring stacks, which limits options to products that have been pre-tested to confirm they work together. In doing so, you can significantly cut the complexity of implementing DevOps clouds while providing an agile platform to support your development team’s innovations.