Comparing CI servers: Jenkins vs. CruiseControl vs. Travis
3D Sparkling Lightning Bolt image via Shutterstock
Joel Spolsky’s „Joel Test,“ which was published 16 years ago, delved into the daily generation of team-wide builds. As Continuous Integration server designated systems solve the issue automatically, they compile their administered elements – the name says it all – permanently. But what can an individual CI server do and which is the right one for my project?
The topic of CI servers has grown immensely since CruiseControl was published in 2001. Nowadays developers are able to choose between a variety of different products —in practice, you have to decide between the following three candidates:
- Travis CI
The forefather of Jenkins, Hudson, isn‘t mentioned at this point because the developer’s interest has almost completely collapsed after the fork of Jenkins — the practical use in new systems can only be recommended at the present time.
CI of mediocrity
As so often happens in IT, there is no magic bullet: those who want to discuss a goal successfully need to choose the correct tool. Let‘s start our discussion with Travis CI – it‘s a CI server from Github, the company which openly shows its preference for mediocrity. This logic has a consequence though; the product is only available if you trust the company with your code – a China critical quote from the US-residing UnME2-CEO summarizes the usefulness of that approach regarding non open-source software.
If you are considering storing your code, possibly your most valuable asset, on GitHub (or anywhere else on the ‘cloud’), you might consider sending it directly to China in the service of efficiency.
If you want to use Travis, the first step is to register at https://travis-ci.org/. The page’s backend allows you to select all those GitHub projects which will be co-managed by Travis. The configuration is performed via a file called travis.yml – the .yml file must migrate to the repository since Travis wouldn’t work otherwise.
The exact compilation process varies from platform to platform. In C’s case, the engine relies, for instance, on a dedicated VM, which is equipped with some tools.
Most importantly in all .yml-files is that the CI System provides information on the type of operation to be performed. A good example would be the one shown in the .yml data, which encourages Travis to compile an Android project:
language: android android: components: – build-tools-19.1.0 – android-19 – sys-img-armeabi-v7a-android-19
At the time of writing, Travis supports two dozen languages. Sometimes the environment is not something you would normally expect, so be cautious about that. The .Net code on Linux is, for instance, tested using Mono: Those who want to include Windows-specific libraries in a test are missing out.
Information on success and failure are not limited to the Web Interface. In addition to sending emails, Travis can also be used for placing messages to any websites – the GitHub Lingo feature identified as WebHook retrieves any URL and transmits a JSON payload with further information. These can later be used in any process.
Anyone who wishes to use Travis in a commercial product must use one of the commercial contracts. The monthly rate starts at $129 (see: https://travis-ci.com/plans).
Golden Oldie in use
Although first papers on continuous integration appeared before CruiseControl, it was this product that launched the „CI-Craze“. Unfortunately, it is no longer in active development; the last release took place in 2010. This can also be seen in a published switch:gears study, which – among other things – includes the diagram, shown in Figure 1, with the activity of the developer mailing lists:
Whoever can make peace with the metaphor „Never change a running system“ found a reliable partner in CruiseControl regarding the realization of a robust CI system for Java developers. Download the binary distribution and run the installation script – after the CruiseControl daemon kicks in, the installation is already complete.
For the first compilation process, the only thing missing is a configuration file. CruiseControl is integrated into the Eclipse IDE: those who don‘t want to assemble a configuration through a Texteditor don’t have to put much effort into it.
CruiseControl.net and CruiseControl.rb are two new implementations tailored to the needs of .Net or Ruby developers. The former is particularly interesting, because it can be used in Microsoft‘s .Net runtime – the test can thus be carried out in the same VM, which is also responsible for program execution.
Greetings from Lusakka
Although the name Jenkins inspires funny connotations in Angelo-Saxon regions, [it was formed from a fork of the Oracle-managed Hudson product], it has become a quasi standard in CI.
This, as .war-file available, project implements a full server, which can be equipped with various plugins with extra features. A particularly interesting feature is the establishment of „test clusters“: A Jenkins server can, in this way, take Unix, Windows and MacOS machines to automatically work through unit tests on multiple platforms.
For interactions between CI server and environment one usually resorts to the web interface, which is offered on Port 8080. Unlike Travis, this one the advantage of comfort – configuration files are not for everyone. One doesn’t need more than a few clicks to set up jobs; for more complex Java environments like Android, there are some tutorials readily available.
Although Jenkins is written in Java, it doesn‘t see itself as a build system for Java-based applications. The target language is determined by plug-ins. Those who operate Jenkins servers on Windows can test .Net apps natively. Thanks to flexible triggering capabilities, a Jenkins server can be provided with new code – even from Visual Studio Team Services.
Another interesting feature – CruiseControl has it, although in a limited way – is the possibility to equip the CI server with various static analyzing tools. In this manner, uploaded code is automatically scanned for bugs. Some static analyzing systems display a downright impressive built-in intelligence nowadays.
Jenkins doesn‘t necessarily have to be hosted itself. Various service providers offer company-hosted Jenkins instances: Despite the growing number of competitors, the prices are quite low. Consulting companies also offer support – those kinds of offerings, known from Linux environments, are worthwhile because you can skip the initial learning step.
The author claims that Jenkins goes hand in hand with commercial projects: Companies who don’t have a certified expert for CruiseControl can use this widely used and very popular CI server. He also opines that CruiseControl is the best option only when other products are only available for the Oldie.
Travis shows its strength when used to manage code that is hosted on GitHub. If the project can handle the peculiarities of the test runtime, one should give it a chance. However, as far as commercial projects are concerned, the product is a rather poor option.