Lighting the fuse

Tutorial: Managing Apache ServiceMix clusters with Fuse Fabric


Torsten Mielke introduces some powerful OSGi and ESB concepts through the former FuseSource, now Red Hat led project

Torsten Mielke introduces some powerful OSGi and ESB
concepts through the former FuseSource, now Red Hat led

Managing a large number of
ServiceMix instances with dozens of applications deployed is a non
trivial task, but open source project ServiceMix from Red Hat can
help reduce the complexity of your application

Apache ServiceMix is
quite a popular open source ESB that is best suited for Integration
and SOA projects. It offers all the functionality one would expect
from a commercial ESB – but in contrast to most commercial
counterparts, at its core it is truly based on open standards and
specifications .

ServiceMix leverages a number of very popular open source
projects. Its excellent message routing capabilities are based on
the Apache Camel framework. Apache Camel is a lightweight
integration framework that uses standard Enterprise Integration
Patterns (EIP) for defining integration routes using a variety of
domain specific languages (DSL).

The majority of integration projects require a reliable
messaging infrastructure. ServiceMix supports reliable messaging by
embedding an Apache
message broker, which is one of the most popular,
fully JMS 1.1 compliant open source message brokers. It offers a
long list of messaging features, can be scaled to thousands of
clients and supports many Clustering and High Availability broker

Support for Web Services and RESTful services is achieved by
integrating Apache CXF. CXF is
perhaps the most well known open source Web Services framework and
has been fully integrated into ServiceMix. CXF supports both the
JAX-WS and JAX-RS standard and all major WS-* specifications.

At the heart of ServiceMix is an OSGi container runtime.
The OSGi Framework
is responsible for loading and running truly dynamic software
modules – so called OSGi bundles. An OSGi bundle is a plain Java
jar file that contains additional OSGi specific meta data
information about the classes and resources contained inside the

The OSGi runtime used in ServiceMix is Apache Karaf, which offers many
interesting features like hot deployment, dynamic configuration of
OSGi bundles at runtime, a centralized logging system, remote
management via JMX and an extensible shell console that can be used
to manage all aspects of an OSGi runtime. Using Karaf one can
manage all life cycle aspects of the deployed application modules
individually. Karaf not only supports deploying OSGi bundles, but
also plain Java jar files, Blueprint XML, Spring XML and war files.
The flexible deployment options ease the migration of existing Java
applications to OSGi.

ServiceMix deploys these open source projects out-of-the-box on
top of the Karaf OSGi runtime. ActiveMQ and Camel register
additional shell commands into the Karaf shell that can manage the
embedded JMS broker and Camel environment at runtime. It’s also
possible to only deploy those ESB functions that are needed for a
particular project. If support for a certain element, for example
Web Services, is not needed, the CXF related OSGi bundles can all
be uninstalled. This further reduces the already small runtime
memory footprint of ServiceMix. Figure 1
summarizes the technologies and standards that Apache ServiceMix is
built on.

Figure 1: ESB enabling technologies
in ServiceMix 

ServiceMix leverages a number of very successful open source
projects. Each of these projects is based on open standards and
industry specifications and designed to provide a maximum level of
interoperability. All of these aspects make ServiceMix a very
popular ESB that is deployed in thousands of customer sites today
and in many mission critical applications. There is also
professional, enterprise level support available from companies
like Red Hat (who acquired FuseSource in 2012) and Talend.

Introduction to Fuse Fabric

It’s no surprise that some companies have dozens and even
hundreds of ServiceMix instances deployed in their IT
infrastructure. Larger projects may spawn multiple ServiceMix
containers as one single JVM instance would not fit the entire
application. In addition, the same application may be deployed to
multiple ServiceMix containers for load balancing reasons. Each
ServiceMix instance is an independent OSGi container with its own
OSGi runtime and an individual set of deployed applications.
However, managing a larger number of ServiceMix instances with
dozens of applications deployed becomes a non trivial task as
ServiceMix itself does not provide any tools to manage multiple ESB
instances centrally.

Installing updates of an application deployed to multiple
independent OSGi containers becomes a tedious and error-prone task.
It is necessary to manually log into each OSGi container (e.g.
using an ssh client session), stop the existing application,
install and perhaps reconfigure the new version of the application
and finally start the new application. These steps then need to be
repeated on all the remaining ESB instances that run the same
application. If anything goes wrong during such upgrade, changes
need to be reverted back manually. This manual approach is
cumbersome and chances are high that mistakes are made along the

Here, we can use Fuse Fabric, an
open source Integration Platform under Apache license which began
as a project within FuseSource. With Fuse Fabric you can group all
ServiceMix container instances into one or several clusters, so
called Fabrics. All instances of this cluster can
then be managed from a central location, which potentially may be
any ServiceMix instance within the Fabric. This includes both the
configuration of all ESB instances in a cluster as well as the
deployment of applications to each ServiceMix container.

Fabric extends the Karaf shell with additional commands for
managing the cluster of OSGi containers, so users don’t need to use
another tool for managing the Fabric. It also supports deploying
applications to both private and public clouds. Using the jclouds library, all major cloud
providers are supported. Applications may be deployed to the cloud
with a single Karaf shell command and even the virtual machine in
the cloud can be started by Fabric.

Fabric can also create ESB containers on-demand. Not only can it
create new ESB containers locally (sharing the existing
installation of ServiceMix) but it can also start new ESB
containers on remote machines that do not even have ServiceMix
pre-installed. Using ssh, Fabric is capable of streaming a full
ServiceMix installation to a remote machine, unpacking and starting
that ServiceMix installation and provision it with pre-configured

To better understand these features, let’s have a look into the
mechanisms and concepts used by Fabric.

Fabric concepts

Fabric defines a couple of components that work together to
offer a centralized integration platform.

Each Fabric contains one or more Fabric Registries. A
Fabric Registry is an Apache
-based, distributed and highly-available configuration
service which stores the complete configuration and deployment
information of all ESB containers making up the cluster in a
configuration registry.

The data is stored in a hierarchical tree-like structure inside
Zookeeper. ESB containers get provisioned by Fabric based on the
information stored in the configuration registry. There is also a
runtime registry that stores details of the physical ESB instances
making up the Fabric cluster, their physical locations and the
services they are running. The runtime registry is used by clients
to discover available services dynamically at runtime. The Fabric
Registry can be made highly available by running replica instances.
The example cluster in Figure 2 consists of three
ESB instances that each run a registry replica.

Figure 2: A Fabric cluster
consisting of 3 ESB instances, all running a Fabric

Fabric Registries store all configuration and deployment
information of all ESB instances. This information is described in
Fabric Profiles, where users fully describe their
applications and the necessary configuration in these profiles.
Profiles therefore become high level deployment units in Fabric and
specify which OSGi bundles, plain Java jar or war files, what
configuration and which Bundle Repositories a particular
application or application module requires.

A Profile can be deployed to many ESB containers and each ESB
container may deploy multiple profiles. Profiles are versioned,
support inheritance relationships, and are managed using a set of
Karaf shell commands. It is possible to describe common
configuration or deployment information in a base profile that
other more specific profiles inherit from.

Figure 3 shows some example profiles that are
provided out-of-the-box. It defines a common base profile called
default that all other profiles inherit from. The example also
lists profiles named camel, mq or cxf. These profiles define the
OSGi bundles and configuration for various ESB functions like
message routing (based on Camel), reliable messaging (based on
ActiveMQ) and Web Services support (based on CXF). Users are
encouraged to create their own profile that inherit from these
standard profiles.

Figure 3: Sample profiles

Profiles can be easily deployed to one or more ESB containers.
Deploying a profile to a particular container is the task of the
Fabric Agent. There is an agent running on each ESB
container in the Fabric cluster. It connects to the Fabric Registry
and evaluates the set of profiles it needs to deploy to its
container. The agent further listens for changes to profile
definitions and provisions the changes immediately to its

Finally Fabric defines the component of a Fabric Server
or Fabric Container. This is every ESB container that is
managed by Fabric. Each Fabric Server has a Fabric Agent

For true location transparency Fabric also defines a number of

Fabric Extensions
 Each CXF based Web Service, each
Camel consumer endpoint (the start endpoint of a Camel integration
route) and each ActiveMQ message broker instance can register its
endpoint address in the Fabric runtime registry at start up.
Clients can query the registry for these addresses at runtime
rather than having the addresses hard-coded. This allows you to
move endpoints to different physical machines at runtime, running
replicas of endpoints for load balancing reasons, or even creating
master/slave topologies where a slave endpoint (e.g. a slave
message broker) waits on standby for the master endpoint to become
unavailable. Fabric Extensions are outside the scope of this
article but the link above explains them in full detail.

Fabric defines some really powerful concepts. All provisioning
information is stored in a highly available Fabric Registry in form
of Fabric profiles. These profiles can then be deployed quickly to
any number of ESB instances inside the cluster thanks to the Fabric
Agents. Also, Fabric is capable of creating new local and remote
ESB instances on demand. Together with the Fabric Extensions this
allows for very flexible deployments. If the load of a particular
ESB container increases it is possible to start up another ESB
container instance (perhaps in the cloud) that deploys the same set
of applications and then load balance the overall work across all
instances. Furthermore ESB instances can be moved to different
physical servers if there is a need to run on faster hardware while
clients automatically get rebalanced. With Fuse Fabric it is
possible to quickly and easily adapt on any changes to your runtime
requirements and have a fully flexible integration platform.



Having introduced the concepts of Fabric, this last section aims
to provide a quick introduction on how to practically use Fuse
Fabric for deploying an integration project. Although one could
download and run Fuse Fabric from its project web site, this part
Fuse ESB Enterprise 7.1
as released by Red Hat. Fuse ESB
Enterprise is based on Apache ServiceMix and already includes
Fabric out-of-the-box. It is fully documented
. The default workflow when working with Fabric is as

  1. Create a new Fabric. This starts the Fabric Registry and imports
    the default profiles.

  2. Create the Integration or SOA application using the technologies
    offered by ServiceMix.

  3. Define the deployment of the application plus its configuration
    in one or more Fabric Profiles.

  4. Create the required number of ESB containers and configure these
    containers for one or many profiles.

  5. Test or run the deployed application.

Lets go through these steps one by one.

Create a new Fabric

After installing Fuse ESB Enterprise 7.1, it can be started
using the script ‘bin/fuseesb’. A few seconds later the welcome
screen of the shell console is displayed.


|  ___|                 |  ___|/  ___|| ___  
| |_  _   _  ___   ___  | |__   `--. | |_/ / 
|  _|| | | |/ __| / _  |  __|  `--. | ___  
| |  | |_| |__ |  __/ | |___ /__/ /| |_/ / 
_|   __,_||___/ ___| ____/ ____/ ____/ 

  Fuse ESB (7.1.0.fuse-047)

Hit '<tab>' for a list of available commands 
and '[cmd] --help' for help on a specific command. 
Hit '<ctrl-d>' or 'osgi:shutdown' to shutdown Fuse ESB. 



Tip: All Karaf shell commands take the –help
argument which displays a quick man page of the command.

On its first start up this
ESB container does not have a Fabric pre-configured. Its only a
standalone ServiceMix installation with a number of OSGi bundles
deployed. It is necessary to create a Fabric first using the Karaf
shell command
This reconfigures the current
ESB container, deploys and starts the Fabric registry and imports
the default profiles into the registry. Alternatively a container
can join an existing fabric cluster using the command
fabric:join’ providing the URL of the already running Fabric
registry. This Fabric enabled ESB container does not deploy any ESB
functionality by default (use the command ‘
osgi:list’ to verify). ESB functions get enabled by
deploying the relevant profiles.

Create the Integration or SOA Application

Fuse ESB Enterprise 7.1 also comes with a couple of demos from
which this article picks the examples/jms demo. It demonstrates how
to connect to an ActiveMQ broker and use JMS messaging between two
Camel based integration routes. The demo works in a plain
ServiceMix environment but in this part it will be deployed to a
Fabric enabled ESB container. This demo has only one interesting
file, which is the Camel route definition located in


<camelContext xmlns="" 

  <route id="file-to-jms-route"> 
    <from uri="file:work/jms/input" /> 
      <log message="Receiving order ${file:name}"/> 
      <to uri="activemq:incomingOrders" /> 

  <route id="jms-cbr-route"> 
    <from uri="activemq:incomingOrders" /> 
          <xpath>/order:order/order:customer/order:country = 'UK'</xpath> 
          <log message="Sending order ${file:name} to the UK"/> 
          <to uri="file:work/jms/output/uk"/> 
          <xpath>/order:order/order:customer/order:country = 'US'</xpath> 
          <log message="Sending order ${file:name} to the US"/> 
          <to uri="file:work/jms/output/us"/> 
          <log message="Sending order ${file:name} to another country"/> 
          <to uri="file:work/jms/output/others"/> 
    <log message="Done processing ${file:name}"/> 


This Camel context defines two Camel routes. The first route
with the id=file-to-jms-route consumes a message from a file
location on the local file system (directory work/jms/input). It
then logs the file name and sends the content of the file to the
incomingOrders queue on an external ActiveMQ broker.

The second Camel route with id=jms-cbr-route consumes messages
from the incomingOrders JMS queue and runs a content-based routing.
Depending on the XML payload of the message it gets routed to
different target directories on the local file system. This is a
simple yet fairly common integration use-case. Some small
additional configuration is needed to tell Camel how to connect to
the external ActiveMQ broker.


<!-- connects to the ActiveMQ broker --> 
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent"> 
  <property name="brokerURL" value="discovery:(fabric:default)"/> 
  <property name="userName" value="admin"/> 
  <property name="password" value="admin"/> 

Notice the brokerURL property. Rather than using a hard coded
url like tcp://localhost:61616 the real broker address is queried
from the Fabric registry at runtime using the Fabric MQ Extension.
That way the broker can be moved to a different physical machine
and clients automatically reconnect to the new broker address.

The demo can be build by running ‘mvn
. This will install the generated OSGi bundle to
the local Maven repository.

Define deployment

Now it’s time to create the Fabric profiles that will deploy
this integration project. Let’s assume there is a requirement to
run the ActiveMQ broker externally in its own ESB container, which
can be useful for various reasons like providing a common messaging
infrastructure to a number of deployed applications. Therefore two
ESB containers are required: one running the ActiveMQ broker, the
other running the Camel integration route.

For running an ActiveMQ broker there is already a profile with
the name mq provided out of the box. That ActiveMQ broker has a
default configuration, which is sufficient for running this demo.
The mq profile can simply be re-used so there is no need to create
a new profile. The command ‘fabric:profile-list
lists all available profiles. ‘fabric:profile-display
’ shows the content of a profile.

For running the Camel integration demo, the Camel runtime needs
to be deployed to the ESB container. Furthermore, both Camel routes
connect to the external ActiveMQ broker. So it’s also necessary to
deploy the ActiveMQ client libraries to this ESB container.
lists the following three profiles among others


FuseESB:karaf@root> profile-list 
[id]                                     [# containers] [parents] 
activemq-client                          0              default 
camel                                    0              karaf 
camel-jms                                0              camel, activemq-client

The profile activemq-client deploys the ActiveMQ client
libraries needed for connecting to an ActiveMQ broker. The profile
camel deploys the core Camel runtime (but not the many Camel
components). Finally the profile camel-jms has two parent profiles
named camel and activemq-client, so it deploys both of the ActiveMQ
client libraries – the Camel core runtime and camel-jms component.
When using the profile camel-jms as a parent, it will automatically
deploy the Camel runtime and ActiveMQ client runtime.

–parents camel-jms camel-jms-demo

This command creates a new profile called camel-jms-demo and
uses the profile camel-jms as its parent., which also needs to
deploy the OSGi bundle of the ServiceMix demo. This can be added
using the demo’s Maven coordinates (the demo was previously built
and installed to the local Maven repository) by invoking

–bundles mvn:org.fusesource.examples/jms/7.1.0.fuse-047

It modifies the
camel-jms-demo profile and adds the demo’s OSGi bundle that is
identified by its Maven coordinates
org.fusesource.examples/jms/7.1.0.fuse-047. That’s all! Thanks to the out-of-the-box
profiles it took only two Fabric shell commands to create a profile
that fully deploys the Camel integration demo.

Create ESB containers

The last step is to create the two ESB containers that run the
ActiveMQ broker and the Camel demo. For running the ActiveMQ broker
in its own ESB container this command is all that is needed:

–profile mq root activemq-broker

It creates a new local ESB
container called ‘activemq-broker’ (using the existing installation
of Fuse ESB Enterprise) with the parent container being the root
container. It also deploys the mq profile, which runs the ActiveMQ
broker. The ESB container could be created on a different machine,
using the command ‘
fabric:container-create-ssh. Running
fabric:container-list’ verifies
that the new ESB container got started. It is possible to connect
to that container using ‘
and check the log file using
log:tail’. If the ActiveMQ broker got started
successfully, the log will contain a line like:

ActiveMQ 5.7.0.fuse-71-047 (activemq-broker,
ID:XPS-49463-1357740918210-0:1) started.

With the broker running its time to deploy the camel-jms-demo
profile to another ESB container. The existing root container only
runs the Fabric Registry so the demo can be deployed to the root
container using the command

camel-jms-demo root

This reconfigured the root container to also deploy the
camel-jms-demo profile (the jms demo).

Test the application

The demo can finally be tested by copying a sample XML message
to the work/jms/input folder that the first Camel route listens on.
Fortunately some sample messages are provided with the demo. On a
plain Unix or Windows shell run


Right after copying the file will be picked up by Camel, get
routed through the two Camel routes via JMS and is finally put into
the target directory instances/camel-jms-demo/work/jms/output/uk/order2.xml.
This verifies that the demo works correctly.

For users that aren’t fans of command line tools, it is
also possible to manage all aspects of a Fabric using

Fuse Management Console (FMC
The FMC is a graphical, browser based management tool for Fabric
and a full alternative to using the Karaf Shell. It can be
installed directly to the root ESB container using the

fmc root

Thereafter, it can be accessed from a browser using the
(see Figure 4). Discussing the details of the Fuse Management Console is
outside of the scope of this article.

Figure 4: Fuse Management


Anyone who needs to manage multiple instances of ServiceMix
should look into Fuse Fabric. The ability to describe all
deployments centrally and roll them out to any number of ESB
instances can greatly increase productivity and reduce management

Author Bio: 

Torsten Mielke works as a Senior Technical
Support Engineer at Red Hat. He is part of the global professional
support team at Red Hat and a specialist in open source enterprise
integration and messaging systems. Torsten actively works on open
source projects such as Apache ActiveMQ, Apache ServiceMix, Apache
Camel and Apache CXF and is a committer on the Apache ActiveMQ

This article appeared in JAX Magazine: Socket to them!. For
other articles and issues, click here.

Image courtesy of Razor512

Torsten Mielke works as a Senior Technical Support Engineer at Red Hat. He is part of the global professional support team at Red Hat and a specialist in open source enterprise integration and messaging systems. Torsten actively works on open source projects such as Apache ActiveMQ, Apache ServiceMix, Apache Camel and Apache CXF and is a committer on the Apache ActiveMQ project.
comments powered by Disqus