Docker Tooling in Eclipse
Ask anybody these days about hot software technology topics and you’re bound to hear the word Containers which are light-weight virtual machines that operate on the process level. They can be instantiated very quickly and are easy to configure. If you are going to enter the world of Containers, then you will undoubtedly run into the name Docker which is quickly becoming the ubiquitous standard. Docker is the trademarked company name but also used when referring to their implementation of container technology.
Some brief terminology is in order. An Image is a template for creating a Container. A Container is an instance of an Image with some process or application to run along with an environment specification. There are a large assortment of pre-formed Docker Images out there for various Operating Systems (e.g. Fedora, Ubuntu, Solaris). You can take an existing Image, modify it, and save that as a new Image to use later. For example, you might take a base Fedora 22 Image, install ssh on it, and then store that as a new Image to use whenever you need to run an application that uses ssh in a Container.
Images are stored in a registry and have an identifying id which includes a repository and a tag. For example: fedora:22 indicates the base Fedora 22 OS from the fedora repository. To use an Image locally, it first needs to be pulled from a registry. To share it with others, it is pushed to a registry. There is a default Docker registry known as Docker Hub where you can find base OS Images. These base OS Images are available to all. If you wish to modify these Images, or create your own, you need to sign up for a personal Docker Hub account and store Images under a personal repository. A personal repository is specified by prefixing the repository with your username in the Image id; for example, jjohnstn/fedora:22. These Images in turn can be shared with others, modified, etc … You may also create a private registry, if desired. In such a case, the registry name needs to be specified as part of the Image id: registry/repository:tag means you are not using the default Docker Hub registry.
Modifying an existing Image can be done two ways. The first way is to use a special file called a Dockerfile which contains instructions on which Image to base upon and what modifications are desired (e.g. installing various packages, configuring the system, setting environment variables). The other way to modify an Image is to create a Container that runs a shell and then perform actions in the Container such as installing packages, configuration, etc … Once all modifications are made, commit the Container as a new Image.
To use Docker, you must first start a Docker daemon. The Docker daemon knows about Docker Hub and is used to manage the set of local Docker Images and Containers. The daemon can be set up to start up as a system service (Linux) or can be started manually using the docker command (note that lower-case docker is used when referring to the command). The docker command is also used for performing various tasks for Docker Images and Containers as well as registry support. On some systems such as Windows, the Docker daemon cannot be started locally. In these instances, one can use Docker Machine which starts the Docker daemon in a VM (Virtual Machine) for you, provides the Host with any certificates required to connect with the Docker daemon, and exports environment variables that contain all the information needed to communicate with it (e.g. TCP address). Formerly, one would use Boot2Docker to do all of this, but this has been deprecated.
The Docker Tooling Project
Starting in the fall of 2014, members of the Eclipse Linux Tools Project at Red Hat Inc (Jeff Johnston, Roland Grunberg, and Xavier Coulon) developed plug-ins to allow management of Docker Images and Containers within Eclipse to mimic some of the functionality provided by the docker command and enhance its functionality through a robust UI. To communicate with the Docker daemon, a Java API library was used. At the time, there were two such public Java API Libraries available (docker-java and docker-client), but only docker-client (from Spotify) had Unix socket support which is the default configuration for the Docker daemon on Linux. Although the Docker daemon also supports setting up TCP sockets, this would have required the end-user being forced to add a special option on the daemon start-up and this was deemed too cumbersome. It should be noted that the other Java API Library at the time (docker-java) has since added Unix socket support.
While docker-client had Unix socket support, it had implemented this using a package with little support unix-socket-factory, that made use of the socket functionality from another poorly supported package, junix-socket. A patch to convert docker-client to use JNR Unix sockets was submitted to github and this was accepted upstream.
The Linux Tools Docker Tooling plug-ins were successfully shipped as part of the Linux Tools 4.0 release which was shipped in Eclipse Mars in June/2015 and updated as part of Linux Tools 4.1 which was shipped as part of the Eclipse Mars.1 update in Sept/2015.
If running on Linux, install the appropriate Docker package on your system. If running on Windows, you can use Docker Machine to start a virtual machine running the Docker daemon. On Eclipse install the Mars.1 update version of Eclipse and then install the Docker Tooling feature which can be installed from the default Mars update site (Help->Install New Software…).
Before starting Eclipse, ensure the Docker daemon is running.
When you start Eclipse, you will see a new perspective has been added, called Docker Tooling. Switch to this perspective. It consists of three new views: Docker Explorer, Docker Images, and Docker Containers along with the standard Eclipse Console and Properties views.
The first time you start your workspace, there will be no Connection to a Docker daemon, so you should see the following:
Clicking on the message found in the Docker Explorer View will bring up the New Connection Wizard:
The New Connection Wizard upon opening will attempt to fill in defaults on behalf of the end-user. On a Linux system, the wizard looks for /var/run/docker.sock being readable in which case it defaults to the unix socket. Otherwise, the wizard will look for the environment variables: DOCKER_HOST, DOCKER_TLS_VERIFY, and DOCKER_CERT_PATH to initialize default TCP settings. The wizard will present a default name but you may rename the connection as desired. The Search button can be used to find a local running Docker Machine instance and/or you can test the connection using the Test Connection button to ensure that the daemon is running and responding correctly. A new connection can be created any time from the Docker Explorer View toolbar.
Once a connection is created, it becomes the default active connection. This results in the following:
To change the active connection, simply click on any of the nodes in the Docker Explorer View. The Docker Image, and Docker Container views will display content based on the selected connection. These lists are also available in the Docker Explorer tree view by expanding a connection node and clicking on either Images or Containers.
By default, the lists of Images and Containers are filtered. Untagged and intermediate Images as well as stopped Containers are not shown. These settings may be changed from the Docker Explorer View’s “Customize view…” pull down menu.
The next step is to ensure you have some Images to run. When you first start the Docker daemon, there will be no Images. Images need to be pulled from a registry. By default, the Docker daemon points to the Docker Hub registry, but you may have a private registry set up that you wish to use.
To pull an Image, you can right-click in the Docker Explorer View on the Images node or else you can use the Docker Images View toolbar which has the following actions:
|– pull an Image from the Repository|
|– push an Image to the Repository|
|– run an Image and create a Container|
|– build an Image from a Dockerfile|
|– delete one or more Images (requires confirmation)|
|– tag an Image|
|– refresh Images list|
Choose to pull and Image and then specify the Image name in one of the following forms:
- <repository>[:<tag>] where <repository> may be <username/reponame> or <reponame>
- <registry>:<repository>:<tag> where <registry> contains a “.” or ends with a port specifier (“:port”)
You can also do a search of the Docker Hub registry by hitting the Search button. Private registries cannot be searched at this time. For this example, specify “fedora:latest” for the image tag.
Note that pulling an Image may take a long time. This is because an Image may use several intermediate Images each of which may be quite large. The Linux Tools Docker Tooling provides status jobs to monitor the download of the various Images needed and has an overall job created to monitor the status of the pull itself. When complete, refreshing of the Docker Explorer View and Docker Images View is automatic.
Running the Image
Now that you have one or more Images, you can create and start a Container. To do this, use the Run Image action which can either be triggered by right-clicking the desired Image in the Docker Explorer View or using the Docker Images toolbar. Select the fedora:latest Image you just downloaded in the Docker Explorer View and select Run Image from the right-click context menu. This will bring up the Run Image Wizard:
The first page of the Wizard allows a number of common settings:
- Image – this field is filled in based on the user’s selection
- Other images are available from the pull-down or the user can type in an Image not currently loaded and click on the Pull Image link.
- Name – this field must be filled in as this is the name of the new Image
- EntryPoint – this allows configuring the Container to run as an executable
- The entry point is an executable to run plus arguments. On the command-line, additional arguments can be specified.
- Command – this is the command to run in the Container when it starts
- This field may be left blank if the Image has a default command to run.
- Ports – the user can choose to expose ports from the Container (self-explanatory)
- Links – the user can choose to link to other Containers
- Keep Stdin Open – used to allow input from the Console
- Allocate pseudo-tty – used to allocate a TTY for the Container (needed if running a shell)
- Automatically remove the Container on exit – this is used to remove the Container when finished
The second page of the Wizard has the following settings:
- Data Volumes – the user can mount Host volumes in the Container or mount other Container volumes
- This is useful for copying data from the Host into the Container (e.g. an executable).
- Environment Variables – way of specifying Env variables to use in the Container
- Enable Resource Limits – this is used to restrict memory or CPU priority for the Container
Deselect the “tty” and “interactive” options and specify:
echo “!!!Hello World!!!”
in the Command text box. Hit the Finish button. This will create and start the Container.
Once the Container is running, the logs will be displayed in the Console View.
A new console opens by default when a Container is started using Docker Tooling’s Run Image Wizard or the Display Log action is used on a running or stopped Container from either the Docker Explorer View or Docker Containers View.
In our example, each entry in the log is preceded by a time-stamp. This is default except when the “tty” option is specified and can be controlled using Window->Preferences->Docker->Logging. The timestamp is useful when starting the Container multiple times to see the logs from each run. Go to the Docker Containers View and use the view pull-down to ensure all Containers are shown. Once done, find the Container you just ran, select it, and hit the start button in the toolbar. The Container will run again and you will see two messages each with a different timestamp.
Each Container log is given its own separate console under the “Console View”. To delete a console, use the Remove Console context menu action from for the specific Container being displayed either from the Docker Explorer View or from the Docker Containers View. This cannot be done from the Console View. The console can be shown again later using the Display Logs context menu action for any Container whether running or stopped.
Let’s say that the command you wanted to run in your Container requires valgrind be installed. The default fedora:latest Image you used above does not have valgrind installed. How to handle this? Build your own Image based on fedora:latest and install valgrind as part of the build process.
Building an Image takes an existing Image and modifies it to create a new Image. Typically modifying an Image involves installing new packages. The specification of the new Docker Image is done via a special file which is always named: “Dockerfile”.
There are two ways to kick off an Image build.
Clicking on the Build Image icon starts the Build Image Wizard where you can specify the tag of the new Image and the directory containing the “Dockerfile”. Note that building an Image requires full read access to the directory and it’s sub-directories so it is best to use a leaf directory. If editing or creation of a Dockerfile is required in the directory, the Edit button in the Build Image Wizard will bring up a rudimentary Dockerfile Editor with clipboard support (copy/cut/paste).
Alternatively, you can also create a Image Build launch from the Run->Run Configurations… dialog. From there, double-click on the “Build Docker Image” launch category and use the Main tab to specify a Docker connection plus a source location (workspace or file system) where an existing Dockerfile is located. You can use the default Eclipse text editor to create/modify such a Dockerfile.
The Dockerfile itself contains a set of commands used to build the Image. For details on what goes in the Dockerfile, see the Docker documentation: https://docs.docker.com/reference/builder/.
In our example, we installed valgrind using the yum command. For a different base OS, this might be something different (e.g. apt-get). We also specified a default command. If you were to run this Image, you could opt to leave out the specification of a command and the Container would simply run: valgrind -v which would display the version of valgrind installed.
Another way to accomplish the same thing would be to start a/bin/sh shell running as the command for running the base fedora:latest Image. Specify the interactive and tty options and this will bring up an interactive console. In the console, install valgrind using the yum command above. Finally, select the Container in either the Docker Explorer View or Docker Containers View and right-click. Choose the Commit Container option and this will bring up the Commit Container Wizard which will ask you to choose a tag name. The Container with the package you just installed will be saved as an Image with the tag you gave it (e.g. fedora:latest-with-valgrind).
Running C/C++ Application in a Container
Added as part of the CDT (C/C++ Development Tools) 8.7.0 and Linux Tools 4.0.0 in Eclipse Mars is the ability to start the binary target of a C/C++ Project in a Container. To try this, make sure you have the C/C++ Development Tools feature installed along with the optional C/C++ Docker Container Launch feature. Both can be found from the Eclipse Mars update site (Help->Install New Software…)
Switch to the C/C++ Perspective using Open Perspective (just left of the list of perspectives top-right). Create a new C Project (File->New…->C Project). This brings up the New C Project Wizard. Open the Executable node and choose the Hello World Ansi C Project. Choose an appropriate tool-chain on the right (e.g. Linux gcc if you are running Linux), name your project “helloworld” and then hit Finish.
Go to Window->Preferences->C/C++->Docker Container and in the Default Image window, type “fedora:latest”. This instructs the C/C++ launcher to use fedora:latest as the default Image for a Container launch. If you want the Container to exist after running, select the “Keep Container after launch” option. Hit OK and exit the preferences dialog.
Select your new project in the Project Explorer and then select the hammer icon in the Eclipse tool-bar. This will build your new C project.
Expand your project in the Project Explorer and then expand the Binaries folder. This folder is a virtual folder showing you the binaries created by your C project. Right-click on the “helloworld” executable within the Binaries folder and then choose:
Run as->C/C++ Container Application
Your helloworld application will run in a Container based off the fedora:latest Image.
If you need special options such as “interactive” or you want to have access to data on the host, you can go to the Run->Run Configurations … dialog and create a launch configuration for a C/C++ Container Application (double click the launch category). Under the Containers tab there are a number of options:
Here you can specify to keep the Container after launch or to support input in the Console. As well, you can specify host directories that are accessed by the application and these will be bound into the Container.
At the time of this article, a number of known issues exist with the Docker Tooling plug-ins. These are:
- An action with a long wait time can cause a timeout to occur and then no more output is shown in the Display Log. This is due to the upstream docker-client not specifying notimeout and a fix has been supplied which will appear in Linux Tools 4.2.0.
- Specifying -i and -t in the Run Image Wizard for a Container that quickly finishes may result in no output being shown in the Display Log. This is due to an issue in the underlying apache package being used. To support tty, the Docker Tooling code attempts to open a new stream but this can only be done once the Container is running so a timing issue sometimes occurs where the output has already been displayed. This is an open bug that is being researched.
- Private registries cannot be searched for Images. This is an underlying problem with the remote Docker API and efforts are being made to suggest enhancements directly to Docker.
- Varying release cycles of Eclipse and Docker can cause incompatibility between docker-client and the “Docker Remote API”. Our Docker Tooling releases in line with Eclipse, which in the past has had one major release per year, with 2 service releases in between. The Docker Remote API, on the other hand can change as often as every few months, and given that many will adopt the newer version, our releases can become incompatible. As a result we have begun to consider releasing more often to line up with the quick rate at which the Remote API is evolving. Supporting older versions of the Docker Remote API is another issue that may become more relevant in the future as the tooling gets adopted by other products.
For further details on the Docker Tooling plug-ins, either consult the on-line user-guide documentation provided with the Docker Tooling Feature (Help->Help Contents) or see the Docker Tooling Wiki page.