“Data persistence is the most misunderstood element by Docker users”
Docker is revolutionizing IT — you’re probably hearing this phrase quite often. Still, these questions linger: If we were to look beyond the hype, what’s so disruptive about Docker technology? What are the differences between Docker and a virtual machine? What is hype and where does the real added value lie? We talked with Nicolas De Loof about all this and more.
Docker manages to insert itself into all our conversations — why? Because it is extremely helpful and everyone loves it. There’s a lot going on in the Docker world (for example, the Docker platform and Moby Project are now integrating support for Kubernetes) but this is not why we’re doing this interview series with Docker Captains.
Don’t miss our Docker Captains interview series
- “Docker doesn’t want to be replaced as the container runtime of choice”
- Docker vs. VM: What’s the difference?
- Solving Docker confusions one by one — Docker Captains share their tricks
- “It is crucial for Docker to be the industry-wide accepted standard”
- Docker Captains speak bluntly: “Containerd is basically the real engine behind Docker”
- “Making containers usable with nice tooling was the only thing missing — Docker provided that”
We’d like to hear more about their love stories with Docker, their likes and dislikes, their battle scars and more. Without further ado, we’d like to introduce Nicolas De Loof, Docker Captain and Hacker at CloudBees.
JAXenter: Can you tell us a little bit about your first contact with Docker? Was it love at first sight?
Nicolas De Loof: The company that I work for, CloudBees was a dotCloud competitor. My first contact with Docker container was mostly negative as this was very inflexible compared to what we were running at that time, but I later understood the benefits of immutable infrastructures and how Docker can just make this easier to implement.
In the meantime, we shut down our PaaS offer and I was able to experiment more, then definitively adopted Docker in my developer toolbox.
JAXenter: Docker is revolutionizing IT — that is what we read and hear very often. Do you think this is true? If we were to look beyond the hype, what’s so disruptive about Docker technology?
Nicolas De Loof: Containers are not a new thing, Google has been using them for a decade and many ops teams have already adopted LXC before. What makes Docker disruptive is that it defines a high level, user-focussed abstraction for “distributing and running stuff”.
The main value of Docker is the image format and plumbing to distribute them. Runtime is also great as it offers reasonable defaults so things just work out-of-the-box, but experienced users (aka production) can also tweak runtime for fine grained control.
Containers and Docker are not opposed to virtual machines, they are complementary technologies for distinct usages.
JAXenter: How is Docker different from a normal virtual machine?
Nicolas De Loof: They differ from a technical perspective but that’s not the point. One could implement Docker using virtual machines (actually, Intel’s clearcontainers and hyper.sh do). But for most users, VMs are created and managed as plain machines that never get replaced. One has to upgrade them, fix them, etc.
But a VM is still a full system, so when something goes wrong, it’s hard to tell who’s guilty. Containers and Docker are not opposed to virtual machines, they are complementary technologies for distinct usages. VMs allow users to manage hosts by APIs and offer infrastructure elasticity. Meanwhile, Docker allows to define software as small lego blocks to be assembled, so they embrace modern architectures such as immutable infrastructures, microservices, distributed software.
JAXenter: How do you use Docker in your daily work?
Nicolas De Loof: For my personal use, I rely on Docker for various tests, so I make sure that I have a reproducible environment that I can share with others, as well as prevent impacts on my workstation. My company also offers a Docker-based elastic CI/CD solution “CloudBees Jenkins Enterprise” and as a Docker expert, I try to make it adopt best Docker features.
JAXenter: What issues do you experience when working with Docker? What are the current challenges?
Nicolas De Loof: Data persistence is the most misunderstood element by Docker users. Some say one can’t run a database in Docker, maybe they just missed the “volumes” chapter in the doc? But volumes introduce permissions issues when one tries to access them from multiple containers, and things get worst when people try to manage them manually as “bind mount”, without letting docker daemon do its voodoo setup.
This is far from being a trivial issue, but I expect some upstream changes in Linux kernel / filesystem drivers to make this easier to use in the future.
JAXenter: Talking about the evolution of the Docker ecosystem: How do you comment on Docker’s decision to donate containerd runtime to CNCF?
Nicolas De Loof: As I said, Docker’s technical value is not in container runtime, which is not a huge engineering effort to replicate. But for ecosystem health they need to ensure confidence and interoperability, so an open standard and a reference implementation is a must have.
As a Java developer, I’m used with such approach to have a standard to document API and reference implementation, which doesn’t prevent alternate implementations or innovation. Docker does the same with a distinct approach: it proves things can work, then extracts an open source component and makes sure it becomes part of the standardized container ecosystem driven by OCI.
Docker’s technical value is not in container runtime, which is not a huge engineering effort to replicate.
JAXenter: Is there a special feature you would like to see in one of the next Docker releases?
Nicolas De Loof: First of all, I’d like to get unprivileged nested containers “Docker in Docker” support. DinD requires some extra privileges that makes it hard to secure, but on the other side for my CI/CD use case I need to let user run docker stuff inside a dockerized build environment. This is technically feasible as LXD can do it, relying on a recent Linux kernel.
Another feature I’d like to see is user namespace implemented at container level, so I can configure user ID mapping on a per-container basis. With filesystem support, this would just make it trivial to assemble containers together with full support for shared volumes. But this is a longer-term effort as this still requires some fixes in Linux kernel filesystem (work in progress).
JAXenter: Could you share one of your favorite tips when using Docker?
Nicolas De Loof: Considering immutable infrastructure, there’s a lot of middleware that uses filesystem as a cache, and one might want to avoid making this persistent. So I like to constrain them running as a read-only container (docker run –read-only) to know exactly where they need to access filesystem, then to create a volume for the actual persistent data directory and a tmpfs for everything else, typically caches or log files.