Bundles, Bnd, PAX, Maven and More

OSGi à la Carte

ValentinMahrwald
OSGi-a-la-Carte

A tour through the OSGi tool chain with Holly Cummins and Valentin Mahrwald.

 

A web search on OSGi will yield a bewildering number of open
source projects. Beyond the familiar Equinox and Felix, as OSGi
moves into the Enterprise Java space the number of OSGi related
projects is increasing. If you are interested in OSGi, many of
these projects will be useful to you but it may not be immediately
obvious how. This article hopes to alleviate this shortcoming with
a whirlwind tour through the most useful open source tools the
authors have encountered.

OSGi as a technology has been around since the late ’90s,
originally in the embedded space. More widespread interest has
arisen after Eclipse adopted OSGi as the integration and plugin
technology underlying the Eclipse IDE. This move was followed by
the adoption of OSGi as the modularization technology at the heart
of most JEE application servers. Even then OSGi was much rarer in
the space of individual applications. Only with the advent of a
dedicated enterprise OSGi programming model with Spring dmServer
have we seen widespread interest in OSGi.

Historically, OSGi has often been perceived as a hard technology
to get started with – especially since OSGi certainly is an
invasive technology. Correctly using OSGi and its modularity model
requires discipline and forethought from developers and framework
authors. Especially assumptions made concerning classloaders often
no longer hold in an OSGi environment. However, the benefits of
using OSGi for modularization as well as its inherent support for
dynamics far outweighs the initial challenges in using it.

This article hopes to remove some of the biggest hurdles of
using OSGi by introducing an appropriate set of tools, many of
which are of recent vintage. The first set of tools is intended to
help with the task of developing bundles. Next, we look at the
tools that facilitate integration (i.e. in-container) testing with
OSGi. Finally, we have a look at deployment and distribution
options.

It is assumed that the reader has some familiarity with basic
OSGi concepts like bundles, bundle manifests, services and OSGi
lifecycle. Readers completely new to OSGi are referred to
introductory resources such as Neil Bartlett’s “OSGi in
practice
” as well as the OSGi home
page
.

Developing bundles

At the very simplest level an OSGi bundle is just a traditional
JAR file whose manifest has some additional headers that define the
identity of the bundles as well as its (package) capabilities and
requirements – its imports and exports. However, the often cited
bold claim “No more NoClassDefFoundErrors with OSGi” relies on
correct metadata. A missing import for a needed package can easily
mean that OSGi at the very first usage actually introduces more
missing classes rather than less.

So at the minimum tools for OSGi
development should help the developer to produce correct metadata
and well-factored bundles. Beyond that development tooling should
help with test running the bundles and ideally have support for the
OSGi programming models such as the component models, Web
Application Bundles, persistence bundles and the like.

 

 

 

Eclipse PDE and
Libra

 

Perhaps the most well-known tooling
for OSGi is the Eclipse Plugin Development environment, which comes in
many common distributions of the Eclipse IDE (for example the
distribution for JEE development). It provides all the necessities
of OSGi development such as a content aware bundle manifest editor
and a compiler plugin that enforces OSGi visibility rules (unless
side-stepped by a custom build path) and reports errors for missing
imports. On top of that a collection of bundles can be launched
directly from the IDE via an OSGi framework definition. PDE’s
greatest asset is also its greatest weakness: explicitness. A
developer needs to carefully craft the OSGi manifest and update it
whenever the code changes.

 

On the other hand, the resulting
manifest will never contain any surprises. Also, some of the
default suggestions regarding version ranges and the usage of
Require-Bundle do not conform to widely accepted OSGi best
practices.

 

 

For developing more enterprise
focussed OSGi applications, for example using JPA end Servlet
functionality in bundles, Eclipse Libra promises to be an interesting
option. Loosely related to IBM’s free (but not open sourced)
OSGi development tools this project aims to
integrate existing high quality Eclipse tools for JEE development
with OSGi specific functionality as defined in the corresponding
chapters of the OSGi Enterprise Specification.

 

Bnd and
BndTools


While PDE’s explicit approach to
authoring manifests certainly works, there is another more
convenient option, which is to let most of the hard stuff
(particularly the Import-Package) be generated.

 

After all, the source code, as well
as the byte code, have most of the information needed to determine
dependencies. Peter Kriens’ Bnd tool is built on that premise that most of
the manifest can be described in a terse domain specific language
(DSL) and the rest can be filled in at the time when the bundle is
assembled by introspecting the classes. With this approach it also
becomes trivial to convert third-party JARs into functional bundles
(assuming they do not make assumptions about classloaders in the code,). Bnd also honors OSGi best
practices such as generating uses directives for exported
packages.

 

More than for just converting JAR
files, the Bnd tool provides a holistic solution for bundle
development, supporting complex bundle assemblies, which pull
classes from multiple sources, bundle verification as well as
launching a framework for running an application or running
integration tests. The invocations below show very basic usage for
converting a JAR to a bundle (Listing 1).

 

 

Although the above example shows Bnd
as a stand-alone command line tool, it can be integrated into Ant
or Maven builds. Neil Bartlett has even created a complete Eclipse
development environment, BndTools, based on Bnd. The complete package includes
very capable editors for defining bundles (even several for a
single Eclipse project), support for running a collection of
bundles and even hot swapping code changes into the running
framework. On top of that there are prebuilt templates for defining
Declarative services components (Figure 1) and integration
tests.

 

 

Maven with Felix Bundle
plugin and Pax Construct

 

The above projects are IDE centric
or build agnostic, like Bnd. Some readers may be looking for a very
mature build solution, especially when confronted with large OSGi
applications. This can be found in the Maven build
infrastructure that is used across almost all Apache OSGi projects.
Maven as a build choice for OSGi projects offers significant
freedom for choosing an IDE as all major IDEs feature Maven
plugins. Also Maven’s dependency management aligns advantageously
with OSGi concepts of modularity. Maven itself does not come with
direct support for OSGi packaging, however the Apache Felix Bundle
plugin adds that capability and is widely used across Apache
projects. The Felix bundle plugin uses Bnd under the covers, so the
configuration options for defining bundles are the same as for
using Bnd directly or through the BndTools.

 

Despite many advantages of Maven,
setting up the right infrastructure for OSGi is a time-consuming
process, which especially to a Maven novice can seem daunting.
Fortunately, the Pax Construct project provides a very useful starter
kit. It helps to create Maven 2 OSGi projects and define individual
bundles. Pax Construct also has excellent support for launching a
test OSGi environment from Maven.

 

     
   

Component
Models

 

Finally, no section on OSGi
development would be complete without mentioning the component
models, which greatly simplify using the OSGi dynamism and in
particular OSGi services (or μServices as Peter Kriens has dubbed them). These two
models absolve the developer from the responsibility of using the
complicated OSGi API for services and in part can also hide some of
the dynamism where it is not desired. Two component models are
standardized as part of the OSGi compendium specification and are
in wide use.

 

The older component model,
Declarative Services, allows the definition of service components.
These components are plain Java classes that are instantiated as
well as activated and deactivated by the Declarative Services
extender. Components can have dependencies on other services, which
are injected by the extender. At runtime components are defined in
XML. However, the Bnd tool also allows a simpler property syntax as
well as annotations. Built on top of that the BndTools have a set
of nice editors and project templates for working with Declarative
Services. The code sample below shows an example of an annotated
component and the equivalent (generated) XML. The sample component
uses zero or more StockProvider services, which can dynamically be
added or removed (Listing 2).

 

 

The newer component model,
Blueprint, which was introduced with OSGi Compendium Specification
revision 4.2, offers a full dependency injection framework,
syntactically based on the popular Spring framework, together with
first class support for interacting with the OSGi service registry.
Apart from the syntax of the XML component descriptors, perhaps the
most notable difference between Declarative Services and Blueprint
lies in how both expose OSGi service dynamics.

Declarative services expose dynamics
directly to the client code. A service that is injected via one
method (addProvider in the example), is removed via another method
(remove-Provider in the example). A component is deactivated when a
mandatory service dependency goes away and re-activated when a
replacement becomes available. Blueprint, on the other hand,
attempts to hide some of these dynamics. For example if a required
service dependency becomes unavailable, a Blueprint bean is not
stopped or notified. In fact it will continue to operate until the
component tries to use the required service in some way. At that
point the Blueprint container will pause to wait for a replacement
service to become available and then continues the suspended method
call on the service. Through this service damping OSGi bundles
built on Blueprint can remain oblivious to short outages (due to
for example live updates) of their service dependencies while still
enjoying the benefits of potentially continuous updates. Of course
Blueprint also has support for letting components listen and react
directly to service lifecycle events. The Blueprint snippet in
Listing 3 shows the Blueprint descriptor equivalent to the
Declarative services descriptor shown before.

 

Testing with Pax
Exam

 

Sooner or later in the life of an
OSGi the question arises of how to test integration between
different bundles, especially projects that use OSGi services (for
example through one of the component models discussed above). The
components can be quite thoroughly unit tested but some level of
in-container testing is necessary, for example to ensure that
components are defined correctly.

 

Several options exist for running
in-container tests, each offering a different way to accomplish the
basic goal: launching a bundle with a number of JUnit tests in an
OSGi framework with specific runtime bundles. Available choices
include Bnd itself, the BndTools integration test configuration
based on it and PaxExam. Of these, the authors have found PaxExam
with its annotation based configuration and stream-lined
configuration DSL particularly elegant. A basic PaxExam test (using
the 2.0.0 branch) can look like Listing 4.

 

The code snippet shows a simple
integration test, which in this case test drives a Blueprint
exposed service. Note that even though there is only one @Test
annotated method the given test class actually executes two tests:
one on Equinox and one on Felix. As shown in the code, Pax Exam
works particularly well in conjunction with a Maven build
infrastructure. However, it can also be used with plain file
URLs.

 

Deployment


The question of deployment arises
relatively early for OSGi projects. With OSGi it is no longer
enough to run the Java command with an appropriately long classpath
and point to some main class. Instead the task of deploying an OSGi
application is complicated by the need first to bootstrap an OSGi
framework and then to install and activate the application bundles
plus all their dependencies.

 

At the simplest one can start each
of the three common open source frameworks (Apache
Felix
, Eclipse Equinox and Knopflerfish)
and then use the default console of the framework – all of which
are different! – or the OSGi framework API to install further
bundles. This approach soon becomes highly inconvenient and does
not help the accessibility of OSGi frameworks. Especially, for
complex applications that use one or more of the compendium or
enterprise specifications, this way clearly is not an option.

 

Fortunately there are various
options both for easily bootstrapping custom slim OSGi runtimes as
well as full enterprise strength pre-built runtimes. In almost all
cases the various runtime bits are completely runtime-independent –
as one would expect from the components of a technology focussed
primarily on modularity – and can be reused in any other of the
runtimes.

 

Pax Runner


As an alternative to the basic
native launch facilities of the individual OSGi frameworks, the Pax
Runner project provides a unified way to launch any of three open
source frameworks (always with the default console installed) and
to provision application and support bundles into that framework.
This makes starting a simple OSGi application a breeze.

 

Bundles can be provisioned from a
variety of sources: files on disk, URLs as well as simple
provisioning text files that list bundle resources. In particular
there is a lot of flexibility in the URL scheme (through the
embedded use of the Pax URL projects); bundles can be sourced
directly from a Maven repository by its Maven coordinates. Finally,
profiles provide a number of very useful pre-built bundle
selections for common capabilities such as a Declarative Services
runtime. The command below shows one particular invocation.

 

pax-run –platform=equinox
–profiles=felix.ds <bundles …>

 

This command bootstraps an Equinox
framework, then installs all the bundles required for the Apache
Felix Declarative Services implementation and finally installs all
the listed files as bundles into that runtime.

 

Karaf


One step up from Pax Runner comes
Karaf, which
provides a much more full-fledged proto-runtime while still being
very light-weight and fast to start. By default Karaf comes with
excellent built-in support for the Blueprint component model,
remote access, JAAS based security and provisioning – not to
mention an excellent and extensible shell (based on the Felix Gogo
shell), all of which make working with Karaf a real pleasure.

 

Similar to the profiles that Pax
Runner offers are Karaf features, which capture commonly needed
functionality. For example, to deploy a Web Application Bundle
(WAB) one can simply install the ‘war’ feature. There is even a
feature to deploy a full-fledged Spring dmServer runtime (see
below). With a plentitude of predefined EE features, a custom
enterprisey runtime that contains just the needed features can be
bootstrapped in a breeze. Karaf features can also be used from Pax
Runner through the custom ‘scan-features’ URL scheme. Note that in
some cases prerequisites that are available by default in Karaf
need to be added manually in a blank Pax Runner framework.

 

One particularly nice option that
deserves separate mention is the possibility of provisioning a web
console, based on the Apache Felix Web Console, which of course can
also be installed separately in other runtimes.

 

Provisioning


Pax Runner profiles and Karaf
features rely on pre-defined collections of bundles. This has the
benefit that the concrete selection of bundles that make up a
feature can be tested reliably. However, this provisioning
mechanism is not able to exploit the existence of capabilities in
the runtime. For example, the war feature needs the servlet API and
will install the Geronimo servlet API bundle regardless of whether
the servlet classes are already available (at the right package
version) in the runtime. The key challenge for features and
profiles are that they are oblivious to the declared bundle
dependencies (Import-Package, Require-Bundle etc) and hence the
provisioning system cannot avoid duplication.

 

An interesting alternative that does
not suffer from this problem is the OSGi Bundle repository
technology (OBR), available from Apache Felix OBR. An OSGi bundle repository
stores not just bundle binaries but also their capabilities and
requirements as extracted from the bundle manifest and possibly
other sources (such as component descriptors etc). The provisioner
can use these and information about the current runtime to
provision exactly what is needed. So provisioning bundle A, which
depends on capabilities X and Y where Y is already available in the
runtime, will install A and a bundle that provides X (plus any
additional transitive dependencies of that bundle). This is a
powerful provisioning model that eliminates the need to exactly
specify bundle combinations once and for all and shifts the brunt
of the work to the provisioning system.

 

For playing around with OBR, Karaf
offers an OBR provisioner in the OBR feature. Generating bundle
repositories can either be done via the Bindex
tool
, which supports just package and bundle dependencies
declared in the bundle manifest, or the Apache Aries repository generator, which also
generates capabilities and requirements for Blueprint defined
services and references – this allows service based provisioning. A
collection of public OBR repositories can be found at Apache Felix Sigil.

 

Pre-built enterprise
stacks

 

The runtime options presented so far
were all concerned with simplifying the task of managing custom
runtimes with just the features needed for the desired OSGi
application and no excess baggage. However, in some cases the
benefit of a smaller runtime and faster startup time will not
outweigh the extra effort required to manage what is essentially a
light-weight app server runtime rather than taking a pre-defined
all inclusive package.

 

There are three open-source options:
Apache
Geronimo 3.0
, Eclipse Virgo and Glassfish. Each of these supports different
types of applications. Underlying those full fledged options are a
number of Enterprise specifications, amongst others Blueprint,
JNDI, Web and JPA. These are available as individual components
from Eclipse
Gemini
and also Apache Aries – with the exception of the Web
Application Bundle support, which is available from Pax Web – for
separate consumption (for example via Karaf, which has pre-defined
features for the Aries components). Even though the application
models are syntactically different, the core idea is the same
between Geronimo and Virgo.

 

An OSGi application, defined via an
application manifest in Geronimo, or a plan in Virgo, defines the
collection of bundles (and even configuration artifacts in Virgo’s
plans) that logically define the application. As an example, the
descriptor for the Apache Aries Blog sample application is shown
above.

 

Distribution

 

Soon after the first steps of OSGi
development are taken, many people wonder how to connect things up.
For example how can I make two OSGi applications running in two
different frameworks talk to each other? How easily can OSGi
applications be distributed across multiple deployment systems?

 

How can I connect an OSGi
application to a legacy JEE application? Apache CXF and Tuscany
provide some answers. Apache CXF is the reference implementation of the
Distributed OSGi specification. With very simple
metadata additions it allows two OSGi frameworks to be connected on
the OSGi service levels. This method of distribution fits very
nicely with the OSGi service model, which is dynamic from the
start. As with most OSGi specification implementations there is an
Eclipse equivalent for distributed OSGi in the ECF project.

 

Apache Tuscany can be used to connect OSGi
services to non-OSGi artifacts such as traditional Java, Spring or
Web components. For this Tuscany supports SCA (Service component
architecture) components to be implemented by OSGi services. In
this way such services can be consumed by other SCA components,
which are not necessarily implemented in OSGi. Similarly OSGi
services defined as SCA components can declare references to, and
thus consume, other SCA components as services, which again need no
longer be implemented in OSGi.

 

 Conclusion

 

This brief tour has hopefully
inspired the reader with the sense of how much the OSGi ecosystem
has to offer. In the authors’ experience the amount of reuse
between the various different tools and frameworks is staggering.
This only goes to show the maturity of the components and benefits
of OSGi as a platform for modularity and reuse.

Article taken from
Java Tech
Journal
.

Author
ValentinMahrwald
Valentin Mahrwald is a software engineer at the IBM Hursley Development Laboratory in Hampshire. He has over two years of experience working on WebSphere Application Server, most of this time spent on the OSGi Applications feature, for which he has also authored the accompanying IBM RedBook. He is an active Apache Software Foundation committer in the Apache Aries project. Valentin holds a Masters degree in Mathematics and Computer Science from the University of York.
Comments
comments powered by Disqus