The 7 core principles of Continuous Integration
Organizations doing continuous integration correctly are all following a few basic rules. In this article, Brian Dawson, DevOps evangelist at CloudBees presents the 7 core principles of Continuous Integration.
Many organizations are dedicated to DevOps, recognizing the benefits from both continuous integration (CI) and continuous delivery (CD). Yet, it is surprising how few truly understand the core principles of Continuous Integration.
A large percentage of people think they’re doing continuous integration, but they really aren’t. Are you really doing continuous integration? It’s an important question. The key questions to know whether your organization is carrying out continuous integration include:
- Do you feed builds into your organization’s main pipeline regularly?
- Can you fix a software issue in under 10 minutes?
Continuous delivery (CD) and DevOps, after all, are disrupting the market and providing businesses with a huge competitive advantage. Continuous delivery is built on the tried-and-true practices of continuous integration. Organizations that seek to recognize the benefits of continuous delivery have often failed to fully understand the concepts of continuous integration.
Organizations doing continuous integration correctly are all following a few basic rules. Developers’ working copies are synchronized with a shared mainline at least daily, preferably several times a day. Each integration gets verified by an automated build to detect errors as quickly as possible. Continuous integration itself is a process designed to embrace failure, where developers can fail often and fail fast, so they can find and fix errors early and quickly. Organizations that aren’t following these steps aren’t really doing continuous integration properly.
Continuous integration and your DevOps team
Continuous integration is a development team practice that generates real benefits for an entire organization. Engineers in charge of implementing continuous integration practices want to achieve those benefits and follow the modern practices their peers follow. Often, organizations hear how other teams implement continuous integration, then make decisions based on this. However, one fundamental matter is that every organization is different. Some DevOps team may have a vision for what continuous integration looks like in their organization, but it might not fit precisely inside commonly accepted definitions of it.
Organizations that don’t follow the core principles of continuous integration are likely to run into problems delivering crisp, functioning builds on a regular basis. Over time, the initiative will lose momentum and team members will become disenchanted. People who are resistant to change (which is most of us) will revert back to their old practices if they don’t see evidence of the benefits of change. In this case, you have multiple problems: your builds are still riddled with errors, your team has lost faith in the implementation, you’ve lost critical time, and now you need to restart the project.
The cultural shift needed to implement continuous integration
Organizations mis-implementing continuous integration often face a cultural barrier. Engineers are great at solving technical problems, but CI requires a cultural shift and culture is hard to change. You can bring in a continuous integration tool and check most of the boxes that apply to what CI is supposed to represent, but succeeding with CI requires a change in how you work and how you work together. If the culture of the team doesn’t change, they’re going to have a hard time implementing continuous integration.
The principles of Continuous Integration
The core principles and practices of continuous integration were formed at least 15 years, by Martin Fowler and they have not changed. Here are the practices organizations must follow to truly do CI correctly.
- Commit to the mainline: This is table stakes for continuous integration. A developer can set up an automated build and have the build run on every commit. But if the culture is to not commit frequently, it won’t matter. If a developer waits three weeks to commit or branches off for three weeks, he has delayed the integration and broken the principles. If a build breaks, the team has to sort through three weeks of work to figure out where it broke.
- Maintain a single-source repository: In complex applications, developers often branch and maintain changes off of a trunk (branch) or main. The branching creates complexity and prevents everyone working with a single source of truth. Teams need to commit/merge to trunk or main at least once per day, or even better for every change.
- Automate the build: This is a practice most organizations tend to do well. However, some who claim to practice CI are simply doing is scheduled builds (i.e. nightly builds), or continuous builds but they are not actually testing or validating each build. Without validation of the build, you are not doing continuous integration.
- Make builds self-testing: The first step of the validation process is to know that a build with problems actually failed. The next step is to determine if the product of the build is operational and that the build performs as we expect it to. This testing should be included as part of the build process. This consists of fast functional and non-functional testing.
- Build quickly: If it takes too long to build an app, developers will be reluctant to commit changes regularly or there will be larger changesets. In either case, no one will spot a failure quickly. By building quickly and integrating quickly, you can isolate changes quickly. If it takes hours to run, you might have 20 to 30 more changes during that time and it will be difficult to quickly spot problems.
- Test in a clone: Validation processes verify that software performs as expected in its intended environment. If you test in a different kind of environment, it may give you false results.
- Fix broken builds immediately: It’s critical for development teams to find problems fast and fix them immediately, so they don’t move downstream. Years ago, Toyota instituted a “stop-the-line” approach where workers could pull a rope and stop the manufacturing process if they spotted a problem. CI sets up a process where builds are validated and committed continuously, so if something goes wrong, it’ll be easy to fix.
Despite all the challenges organizations face implementing true continuous integration, it’s important to note how far the software development community has come in following modern processes that create true value for their operations. Many are working hard to make changes and improve their DevOps practices. The biggest obstacle organizations face is employees’ cultural, emotional and technical attachments to legacy technologies. It is once we overcome the “culture of can’t” that we can deliver pristine Continuous integration.