OSGi à la Carte
A tour through the OSGi tool chain with Holly Cummins and Valentin Mahrwald.
A web search on OSGi will yield a bewildering number of open source projects. Beyond the familiar Equinox and Felix, as OSGi moves into the Enterprise Java space the number of OSGi related projects is increasing. If you are interested in OSGi, many of these projects will be useful to you but it may not be immediately obvious how. This article hopes to alleviate this shortcoming with a whirlwind tour through the most useful open source tools the authors have encountered.
OSGi as a technology has been around since the late ’90s, originally in the embedded space. More widespread interest has arisen after Eclipse adopted OSGi as the integration and plugin technology underlying the Eclipse IDE. This move was followed by the adoption of OSGi as the modularization technology at the heart of most JEE application servers. Even then OSGi was much rarer in the space of individual applications. Only with the advent of a dedicated enterprise OSGi programming model with Spring dmServer have we seen widespread interest in OSGi.
Historically, OSGi has often been perceived as a hard technology to get started with – especially since OSGi certainly is an invasive technology. Correctly using OSGi and its modularity model requires discipline and forethought from developers and framework authors. Especially assumptions made concerning classloaders often no longer hold in an OSGi environment. However, the benefits of using OSGi for modularization as well as its inherent support for dynamics far outweighs the initial challenges in using it.
This article hopes to remove some of the biggest hurdles of using OSGi by introducing an appropriate set of tools, many of which are of recent vintage. The first set of tools is intended to help with the task of developing bundles. Next, we look at the tools that facilitate integration (i.e. in-container) testing with OSGi. Finally, we have a look at deployment and distribution options.
It is assumed that the reader has some familiarity with basic OSGi concepts like bundles, bundle manifests, services and OSGi lifecycle. Readers completely new to OSGi are referred to introductory resources such as Neil Bartlett’s “OSGi in practice” as well as the OSGi home page.
At the very simplest level an OSGi bundle is just a traditional JAR file whose manifest has some additional headers that define the identity of the bundles as well as its (package) capabilities and requirements – its imports and exports. However, the often cited bold claim “No more NoClassDefFoundErrors with OSGi” relies on correct metadata. A missing import for a needed package can easily mean that OSGi at the very first usage actually introduces more missing classes rather than less.
So at the minimum tools for OSGi development should help the developer to produce correct metadata and well-factored bundles. Beyond that development tooling should help with test running the bundles and ideally have support for the OSGi programming models such as the component models, Web Application Bundles, persistence bundles and the like.
Eclipse PDE and Libra
Perhaps the most well-known tooling for OSGi is the Eclipse Plugin Development environment, which comes in many common distributions of the Eclipse IDE (for example the distribution for JEE development). It provides all the necessities of OSGi development such as a content aware bundle manifest editor and a compiler plugin that enforces OSGi visibility rules (unless side-stepped by a custom build path) and reports errors for missing imports. On top of that a collection of bundles can be launched directly from the IDE via an OSGi framework definition. PDE’s greatest asset is also its greatest weakness: explicitness. A developer needs to carefully craft the OSGi manifest and update it whenever the code changes.
On the other hand, the resulting manifest will never contain any surprises. Also, some of the default suggestions regarding version ranges and the usage of Require-Bundle do not conform to widely accepted OSGi best practices.
For developing more enterprise focussed OSGi applications, for example using JPA end Servlet functionality in bundles, Eclipse Libra promises to be an interesting option. Loosely related to IBM’s free (but not open sourced) OSGi development tools this project aims to integrate existing high quality Eclipse tools for JEE development with OSGi specific functionality as defined in the corresponding chapters of the OSGi Enterprise Specification.
Bnd and BndTools
While PDE’s explicit approach to authoring manifests certainly works, there is another more convenient option, which is to let most of the hard stuff (particularly the Import-Package) be generated.
After all, the source code, as well as the byte code, have most of the information needed to determine dependencies. Peter Kriens’ Bnd tool is built on that premise that most of the manifest can be described in a terse domain specific language (DSL) and the rest can be filled in at the time when the bundle is assembled by introspecting the classes. With this approach it also becomes trivial to convert third-party JARs into functional bundles (assuming they do not make assumptions about classloaders in the code,). Bnd also honors OSGi best practices such as generating uses directives for exported packages.
More than for just converting JAR files, the Bnd tool provides a holistic solution for bundle development, supporting complex bundle assemblies, which pull classes from multiple sources, bundle verification as well as launching a framework for running an application or running integration tests. The invocations below show very basic usage for converting a JAR to a bundle (Listing 1).
Although the above example shows Bnd as a stand-alone command line tool, it can be integrated into Ant or Maven builds. Neil Bartlett has even created a complete Eclipse development environment, BndTools, based on Bnd. The complete package includes very capable editors for defining bundles (even several for a single Eclipse project), support for running a collection of bundles and even hot swapping code changes into the running framework. On top of that there are prebuilt templates for defining Declarative services components (Figure 1) and integration tests.
Maven with Felix Bundle plugin and Pax Construct
The above projects are IDE centric or build agnostic, like Bnd. Some readers may be looking for a very mature build solution, especially when confronted with large OSGi applications. This can be found in the Maven build infrastructure that is used across almost all Apache OSGi projects. Maven as a build choice for OSGi projects offers significant freedom for choosing an IDE as all major IDEs feature Maven plugins. Also Maven’s dependency management aligns advantageously with OSGi concepts of modularity. Maven itself does not come with direct support for OSGi packaging, however the Apache Felix Bundle plugin adds that capability and is widely used across Apache projects. The Felix bundle plugin uses Bnd under the covers, so the configuration options for defining bundles are the same as for using Bnd directly or through the BndTools.
Despite many advantages of Maven, setting up the right infrastructure for OSGi is a time-consuming process, which especially to a Maven novice can seem daunting. Fortunately, the Pax Construct project provides a very useful starter kit. It helps to create Maven 2 OSGi projects and define individual bundles. Pax Construct also has excellent support for launching a test OSGi environment from Maven.
Finally, no section on OSGi development would be complete without mentioning the component models, which greatly simplify using the OSGi dynamism and in particular OSGi services (or μServices as Peter Kriens has dubbed them). These two models absolve the developer from the responsibility of using the complicated OSGi API for services and in part can also hide some of the dynamism where it is not desired. Two component models are standardized as part of the OSGi compendium specification and are in wide use.
The older component model, Declarative Services, allows the definition of service components. These components are plain Java classes that are instantiated as well as activated and deactivated by the Declarative Services extender. Components can have dependencies on other services, which are injected by the extender. At runtime components are defined in XML. However, the Bnd tool also allows a simpler property syntax as well as annotations. Built on top of that the BndTools have a set of nice editors and project templates for working with Declarative Services. The code sample below shows an example of an annotated component and the equivalent (generated) XML. The sample component uses zero or more StockProvider services, which can dynamically be added or removed (Listing 2).
The newer component model, Blueprint, which was introduced with OSGi Compendium Specification revision 4.2, offers a full dependency injection framework, syntactically based on the popular Spring framework, together with first class support for interacting with the OSGi service registry. Apart from the syntax of the XML component descriptors, perhaps the most notable difference between Declarative Services and Blueprint lies in how both expose OSGi service dynamics.
Declarative services expose dynamics directly to the client code. A service that is injected via one method (addProvider in the example), is removed via another method (remove-Provider in the example). A component is deactivated when a mandatory service dependency goes away and re-activated when a replacement becomes available. Blueprint, on the other hand, attempts to hide some of these dynamics. For example if a required service dependency becomes unavailable, a Blueprint bean is not stopped or notified. In fact it will continue to operate until the component tries to use the required service in some way. At that point the Blueprint container will pause to wait for a replacement service to become available and then continues the suspended method call on the service. Through this service damping OSGi bundles built on Blueprint can remain oblivious to short outages (due to for example live updates) of their service dependencies while still enjoying the benefits of potentially continuous updates. Of course Blueprint also has support for letting components listen and react directly to service lifecycle events. The Blueprint snippet in Listing 3 shows the Blueprint descriptor equivalent to the Declarative services descriptor shown before.
Testing with Pax Exam
Sooner or later in the life of an OSGi the question arises of how to test integration between different bundles, especially projects that use OSGi services (for example through one of the component models discussed above). The components can be quite thoroughly unit tested but some level of in-container testing is necessary, for example to ensure that components are defined correctly.
Several options exist for running in-container tests, each offering a different way to accomplish the basic goal: launching a bundle with a number of JUnit tests in an OSGi framework with specific runtime bundles. Available choices include Bnd itself, the BndTools integration test configuration based on it and PaxExam. Of these, the authors have found PaxExam with its annotation based configuration and stream-lined configuration DSL particularly elegant. A basic PaxExam test (using the 2.0.0 branch) can look like Listing 4.
The code snippet shows a simple integration test, which in this case test drives a Blueprint exposed service. Note that even though there is only one @Test annotated method the given test class actually executes two tests: one on Equinox and one on Felix. As shown in the code, Pax Exam works particularly well in conjunction with a Maven build infrastructure. However, it can also be used with plain file URLs.
The question of deployment arises relatively early for OSGi projects. With OSGi it is no longer enough to run the Java command with an appropriately long classpath and point to some main class. Instead the task of deploying an OSGi application is complicated by the need first to bootstrap an OSGi framework and then to install and activate the application bundles plus all their dependencies.
At the simplest one can start each of the three common open source frameworks (Apache Felix, Eclipse Equinox and Knopflerfish) and then use the default console of the framework – all of which are different! – or the OSGi framework API to install further bundles. This approach soon becomes highly inconvenient and does not help the accessibility of OSGi frameworks. Especially, for complex applications that use one or more of the compendium or enterprise specifications, this way clearly is not an option.
Fortunately there are various options both for easily bootstrapping custom slim OSGi runtimes as well as full enterprise strength pre-built runtimes. In almost all cases the various runtime bits are completely runtime-independent – as one would expect from the components of a technology focussed primarily on modularity – and can be reused in any other of the runtimes.
As an alternative to the basic native launch facilities of the individual OSGi frameworks, the Pax Runner project provides a unified way to launch any of three open source frameworks (always with the default console installed) and to provision application and support bundles into that framework. This makes starting a simple OSGi application a breeze.
Bundles can be provisioned from a variety of sources: files on disk, URLs as well as simple provisioning text files that list bundle resources. In particular there is a lot of flexibility in the URL scheme (through the embedded use of the Pax URL projects); bundles can be sourced directly from a Maven repository by its Maven coordinates. Finally, profiles provide a number of very useful pre-built bundle selections for common capabilities such as a Declarative Services runtime. The command below shows one particular invocation.
pax-run –platform=equinox –profiles=felix.ds <bundles …>
This command bootstraps an Equinox framework, then installs all the bundles required for the Apache Felix Declarative Services implementation and finally installs all the listed files as bundles into that runtime.
One step up from Pax Runner comes Karaf, which provides a much more full-fledged proto-runtime while still being very light-weight and fast to start. By default Karaf comes with excellent built-in support for the Blueprint component model, remote access, JAAS based security and provisioning – not to mention an excellent and extensible shell (based on the Felix Gogo shell), all of which make working with Karaf a real pleasure.
Similar to the profiles that Pax Runner offers are Karaf features, which capture commonly needed functionality. For example, to deploy a Web Application Bundle (WAB) one can simply install the ‘war’ feature. There is even a feature to deploy a full-fledged Spring dmServer runtime (see below). With a plentitude of predefined EE features, a custom enterprisey runtime that contains just the needed features can be bootstrapped in a breeze. Karaf features can also be used from Pax Runner through the custom ‘scan-features’ URL scheme. Note that in some cases prerequisites that are available by default in Karaf need to be added manually in a blank Pax Runner framework.
One particularly nice option that deserves separate mention is the possibility of provisioning a web console, based on the Apache Felix Web Console, which of course can also be installed separately in other runtimes.
Pax Runner profiles and Karaf features rely on pre-defined collections of bundles. This has the benefit that the concrete selection of bundles that make up a feature can be tested reliably. However, this provisioning mechanism is not able to exploit the existence of capabilities in the runtime. For example, the war feature needs the servlet API and will install the Geronimo servlet API bundle regardless of whether the servlet classes are already available (at the right package version) in the runtime. The key challenge for features and profiles are that they are oblivious to the declared bundle dependencies (Import-Package, Require-Bundle etc) and hence the provisioning system cannot avoid duplication.
An interesting alternative that does not suffer from this problem is the OSGi Bundle repository technology (OBR), available from Apache Felix OBR. An OSGi bundle repository stores not just bundle binaries but also their capabilities and requirements as extracted from the bundle manifest and possibly other sources (such as component descriptors etc). The provisioner can use these and information about the current runtime to provision exactly what is needed. So provisioning bundle A, which depends on capabilities X and Y where Y is already available in the runtime, will install A and a bundle that provides X (plus any additional transitive dependencies of that bundle). This is a powerful provisioning model that eliminates the need to exactly specify bundle combinations once and for all and shifts the brunt of the work to the provisioning system.
For playing around with OBR, Karaf offers an OBR provisioner in the OBR feature. Generating bundle repositories can either be done via the Bindex tool, which supports just package and bundle dependencies declared in the bundle manifest, or the Apache Aries repository generator, which also generates capabilities and requirements for Blueprint defined services and references – this allows service based provisioning. A collection of public OBR repositories can be found at Apache Felix Sigil.
Pre-built enterprise stacks
The runtime options presented so far were all concerned with simplifying the task of managing custom runtimes with just the features needed for the desired OSGi application and no excess baggage. However, in some cases the benefit of a smaller runtime and faster startup time will not outweigh the extra effort required to manage what is essentially a light-weight app server runtime rather than taking a pre-defined all inclusive package.
There are three open-source options: Apache Geronimo 3.0, Eclipse Virgo and Glassfish. Each of these supports different types of applications. Underlying those full fledged options are a number of Enterprise specifications, amongst others Blueprint, JNDI, Web and JPA. These are available as individual components from Eclipse Gemini and also Apache Aries – with the exception of the Web Application Bundle support, which is available from Pax Web – for separate consumption (for example via Karaf, which has pre-defined features for the Aries components). Even though the application models are syntactically different, the core idea is the same between Geronimo and Virgo.
An OSGi application, defined via an application manifest in Geronimo, or a plan in Virgo, defines the collection of bundles (and even configuration artifacts in Virgo’s plans) that logically define the application. As an example, the descriptor for the Apache Aries Blog sample application is shown above.
Soon after the first steps of OSGi development are taken, many people wonder how to connect things up. For example how can I make two OSGi applications running in two different frameworks talk to each other? How easily can OSGi applications be distributed across multiple deployment systems?
How can I connect an OSGi application to a legacy JEE application? Apache CXF and Tuscany provide some answers. Apache CXF is the reference implementation of the Distributed OSGi specification. With very simple metadata additions it allows two OSGi frameworks to be connected on the OSGi service levels. This method of distribution fits very nicely with the OSGi service model, which is dynamic from the start. As with most OSGi specification implementations there is an Eclipse equivalent for distributed OSGi in the ECF project.
Apache Tuscany can be used to connect OSGi services to non-OSGi artifacts such as traditional Java, Spring or Web components. For this Tuscany supports SCA (Service component architecture) components to be implemented by OSGi services. In this way such services can be consumed by other SCA components, which are not necessarily implemented in OSGi. Similarly OSGi services defined as SCA components can declare references to, and thus consume, other SCA components as services, which again need no longer be implemented in OSGi.
This brief tour has hopefully inspired the reader with the sense of how much the OSGi ecosystem has to offer. In the authors’ experience the amount of reuse between the various different tools and frameworks is staggering. This only goes to show the maturity of the components and benefits of OSGi as a platform for modularity and reuse.
Article taken from Java Tech Journal.