days
1
7
hours
0
3
minutes
1
4
seconds
2
8
search
Java 9 & Java EE 8: They are finally here!
Modular rules

OSGi enRoute – a new framework for OSGi applications

Peter Kriens
© OSGI Alliance

The OSGi μservice model is a paradigm shift, argues OSGi proponent Peter Kriens. But much of its audience remains blind to its advantages.

When we started with OSGi, we were given the opportunity to develop a world class framework for small network gateways. Today’s Raspberry Pi is huge in comparison. Now, 16 years later, OSGi is the primary modularity solution in the market for Java applications. A modularity solution that we believe is inevitable for every sophisticated Java application. That said, some blogs demonstrate that there are some of mistaken beliefs about OSGi out there.

OSGi is the only solution in the market that encompasses the seismic waves in our industry such as Object Oriented and Inversion of Control/Dependency Injection while also providing the next paradigm shift: service oriented programming. Not as a way to catch a hype wave, no, service oriented programming was not an afterthought in OSGi. True Service Oriented Systems (SOS) have been the cornerstone of OSGi since its creation in 1998. (We started calling our services micro-services after the SOA stole the term service. We now use μservice since micro-services are becoming used for the much heavier model of REST services.)

Though the μservice model is the most important feature of OSGi, it is also its biggest challenge to adoption because it requires developers to think different. Java, and especially Java EE, made a number of bad habits fashionable. For example, class loading was never intended to be used as an extension mechanism and statics in an API have the same disadvantages as global variables: they both create nasty context and singleton problems. Unfortunately, Java does not provide a more powerful extension model that allows the construction of applications out of (reusable) modules. OSGi does provide such a solid solution to modularity. However, avoiding the bad habits that are so anti-modular in Java has made it hard to connect to mainstream Java developers. That is, only when those developers are sunken so deep in the software swamp that OSGi becomes inevitable are the solutions finally understood.

The OSGi μservice model is a paradigm shift

The big problem with a paradigm shift is that the audience is often blind to the advantages of the new paradigm. This is beautifully depicted in Flatland, a book about living in different dimensions. It explains how a point living in line land can not fathom a plane, and how a plane cannot fathom a sphere. And how we are lost ourselves when thinking about a 4th (or 5th) dimension. We can understand this rationally, we can calculate with thousands of dimensions, but we will never ‘feel’ something we lack deep experience of. As an example, in the Sun Java issue tracker someone asked for a Multi Map, a surprising gap in the Java collections API. The issue was closed by Sun because the author never had felt the need (unlike the many ignored upvoters). In similar vein, as an old Smalltalker, it was interesting to follow the discussions about adding lambdas to Java over the past few years. Lots of developers were skeptical but in a few years these same developers will not be able to imagine a world without lambdas anymore. The world would be a quiet place if we only talked about technologies we actually practiced in anger.

So, at the OSGi Alliance, we have this wonderful new dimension to software but find that it is a really hard sell. That is, only once someone has experienced the benefit, will they look up annoyed and ask us why nobody had told them this earlier! They usually then write the 1023rd misunderstood OSGi “Hello World” blog. As a salesman of Object Oriented technology in the previous century, it is an oddly deja vu feeling. Yes, there was a time when the the industry was very skeptical about objects.

So what is so special about OSGi?

Let’s take a short tour through our recent history to provide some context.

In OSGi, when you create a component, you create a class that will act as a μservice and will express its dependencies as μservices. For example, the following code shows a component that provides a Speak μservice

    public interface Speak { void say(String text); }

    @Component
    public class SpeakLogImpl implements Speak {

        Logger logger;

        public void say(String text) {

          logger.info(text);
        }

@Reference
        void setLog( Logger log) { this.logger = log; }
   }

This is where the first attack usually comes in because it is likely that the first reaction is: “why doesn’t it use CDI’s @Inject?” Well, the problem with novel dimensions is that there is so much more than meets the eye. First, though not visible here, this component is actually fully dynamic with a lot of before and after guarantees to handle concurrency. For example, the Speak service is only registered after the Logger service is registered.

SEE ALSO: A gentle introduction to OSGi

Who uses the Speak μservice and where does the Logger come from? This irrelevant for the component we created. The component has expressed the capabilities (the Speak μservice) that it provides when its requirements (the Logger service) are met. It is the task of the deployer/ assembler to create an environment where the requirements and capabilities match up. Since the component is oblivious of all these details it can, and should only, focus on its domain function (which is arguably quite trivial in our example).

In this example we can ignore the dynamics because the system will activate a SpeakLogImpl instance when the dependencies are met and deactivate it when the dependencies are gone. The instance is not really aware of its life cycle but it does behave correctly in a dynamic world. This is a bit like eating your cake and having it too!

The fact that it is dynamic scares people off but that is like being scared of time when you live in a 3 dimensional world. Things do change in the real world, and we tend to handle those dynamic changes with domain specific code. With the μservice primitive, we have an excellent generic foundation that handles a surprising large number of life cycle, initialisation, ordering, failure, and change scenarios without, or hardly any, domain specific code. The result is that μservice based systems tend to be very robust and reliable.

One of the most shiny examples of the use of the μservice dynamics is in distributed systems. Since a component registers a service without claiming who can use that service it is possible for any other component to observe this registration. Depending on the service characteristics (and authorisation) such a service can therefore be registered as a communication endpoint and made available over a network service discovery layer. Another OSGi system can then pickup this service and register a local proxy μservice. Since services are dynamic, we can correctly handle the case when the communications break down or the provider of the original service decides to unregister this service, potentially because it was stopped or its dependencies were no longer met. Anybody that has programmed with this model has fallen in love with it for its simplicity in handling a large variety of failure cases as well as its elegance.

The second part not visible in the example is the configuration. Through the OSGi Configuration Admin μservice it is possible to define one instance of the Speak μservice or many, all with different configuration properties, dynamically and in real time.

Maybe it does not sound much to you (I received this reaction a lot when explaining objects in the eighties) but I can ensure you that the overall reduction in the complexity of a system is quite large. The key reason for the reduction in complexity is the effect that the μservice model has on the API of these μservices. Since these APIs do not have to concern themselves with life cycle, dynamics, initialisation, configuration, and a myriad of other details unrelated to their domain, they tend to be much, much, smaller and simpler.

In OSGi, the only concern of the μservice’s API is the collaboration because configuration and life cycle issues are taken care of. If the components are the actors, then the services are the role(s) they play. A μservice API describes a collaboration between those roles, just like a scenario of a play. A μservice API is therefore not just one interface since in general we need to define multiple roles, a.k.a. interfaces. For example, in the Event Admin μservice specification we have the role of Event Handler and the role of Event Admin. A μservice API is therefore specified in a Java package. This package can then also hold helper classes, like for example the Event object in the Event Admin μservice. A good μservice API defined in a package is very cohesive and not coupled to other APIs.

Components need to be deployed in a runtime environment, this is done through bundles. Bundles contain the classes, resources, and metadata in a JAR file. Bundles are reified in runtime. That is, components can watch for bundles and based on information in this bundle, extend those bundles. For example, in OSGi enRoute there is the Web server extender that maps contents of the bundle’s ‘/static’ directory to the current web server while providing caching management and advanced HTTP features like (cached) compression and ranges. This pattern is called the extender model and is one of the most popular patterns in OSGi.

Modularity

Bundles are modules. Modules work because they provide a fence: creating a private and a public space. This reduces overall complexity because there are less moving parts. Modularity is quite generally recognised and practiced. Where OSGi modularity differs from traditional Java is its dependency model. We learned from using Object Oriented technology that directly depending on other classes is not such a good idea. It creates systems that resembles bubble gum in your hair: very sticky. Java’s innovation was that it added interfaces to break the transitive type dependencies that killed or crippled so many Object Oriented projects in the nineties. Interfaces break the transitive type dependencies because both the implementer and the user depend on the same interface but they no longer depend on each other. Now it was possible to reuse that library in another context by just implementing those interfaces without dragging in other users/ implementers.

If this problem feels vaguely familiar then you probably think of Maven. Maven uses the exact same identity dependency model that failed so spectacularly with classes. Though it is true that Maven does not download the Internet, people download the Internet, it is however the use of direct module identity dependencies combined with transitivity that makes the model so eager to download the Internet. What is clearly needed is something that provides to modules what Java interfaces provided to objects.

In OSGi, we used the package as the interface for modules since it was already the unit for the μservice API. A bundle could therefore export (provide) a package and a bundle could import (require) a package. If a bundle imported a package then another bundle should export that package. In this model, there could be many bundles providing the same package, which obviously raises the questions: how do I find the bundles that exports my needed packages?

With Maven, if you have a repository URL, a group id, an artifact id, and a version then you can download that artifact. However, you’re also locked in to that single bag of bits, just like requiring an implementation class locks you in. In OSGi, you’re not locked in and bundles can be reused in widely different contexts but the price is that it does require an extra assembly step before you can run.

Over time we learned that packages were actually an instance of a generic capability model. We now define all our dependencies in the terms of capabilities and also allow user defined capabilities. By semantic versioning those capabilities, we can make sure that we can also detect incompatibilities. All of this could raise the fear that maintaining those capabilities is a lot of work: we can assure you that there are tools that can extract most of it from the class files, there are even annotations that generate the necessary metadata. The advantage is that bundle are now self describing and we can leverage tools to assemble applications out of bundles.

Fast Forward

Over time the OSGi Alliance created a solid foundation for component based systems with μservice as the links between the components. Bundles are the unit of deployment and are self describing and can therefore be assembled into applications with tooling. The foundation is solid because the OSGi specifications contain surprisingly few shortcuts and are extremely well documented to ensure interoperability. It is hard to envision a better foundation for a truly software engineering component model.

That said, we learned we did not pay proper attention to application developers. We focused on the specification implementers. We basically told the application developers that they now had a fantastic Lego box and the rest they had to figure out themselves. And if they could not, it was their own fault.

The other aspect I see as a failure is the API design of some of the services we specified in the OSGi Enterprise specifications. Most of these specifications were derived or identical to the corresponding Java EE API. Though I do understand the reasoning, Java EE APIs had a large following and many implementations, I think this was a mistake. Having to reuse an API developed for another world makes it very hard to leverage the unique capabilities that OSGi provides. The result is too much of a me-too specification that will trail the original, having annoying deviations, and will almost by definition have fewer implementations available. That said, untold companies are using it and it is always heartwarming to hear about the surprising places where people are using this technology.

SEE ALSO: Using JPA in an OSGi environment

However, we do believe that having proper OSGi APIs is worth it. We actually think that the increased productivity with proper APIs could be so high that we could convince many mainstream developers to come in our direction before they are almost drowning.

Obviously this is not an easy task, competing with mainstream is often a losing fight (until it isn’t of course). One big issue is how to get implementations since most open source projects are focused on Java EE or SE. Fortunately, experience shows that since the OSGi APIs tend to be much smaller, it is usually not hard to use existing (open source) implementations since a surprising large number of the implementation details are hidden from view. Just put all their classes (and often dependencies) in a bundle and not export any of it, just the small collaboration API. It is one of those things that looks like a steep curve to climb but in practice is not bad while truly providing the benefits of OSGi and thus modularity.

Why OSGi enRoute?

The desire to have real OSGi services and to show the world what OSGi is capable of was the reason that the OSGi Alliance started the OSGi enRoute project. The vision is to create an open

source environment for OSGi Applications that are done 100% the OSGi way. Leveraging what is good about OSGi and doing development for what is missing in OSGi, collaborating with existing open source foundations when possible. The OSGi enRoute project attempts to provide a one stop shop that includes best practices, full toolchain, profiles, documentation, tutorials, and a runtime distribution. With OSGi enRoute you can be up and running a web application project with a REST backend in minutes (ok, excluding download time).

What is OSGi enRoute?

The toolchain is complete. It provides an IDE (Bndtools based on Eclipse), continuous integration (Gradle with bnd) on Travis, integration with GitHub, and a repository indexing Maven Central artifacts and more called JPM4J. Projects created in the IDE can be pushed to GitHub and will automatically build on Travis without extra effort.

Bndtools is the premier OSGi development environment. OSGi enRoute provides a base workspace template on GitHub and templates to create different project types for API, providers of an API, test bundles, and web applications. The web application creates a small application showing a Bootstrap/Angular user interface and a REST backend. Bndtools provides for a very short development cycle between a save operation and, for example, a new web page that demonstrates the modifications in some Javascript or calls into the backend REST server. All without compromising fidelity. Since the toolchain is based on bnd, errors and bad practices are flagged before you release. It even integrates baselining, continuously comparing the current API against the previous release, flagging semantic version violations while you make them. And when you’re ready to release, you can release to a number of repositories or only release after continuous integration. (As a committer on bnd I’m obviously biased but I’ve yet to see an environment in which it is so easy to develop not only OSGi and Java code, but also with Javascript or other web languages.)

An OSGi enRoute profile is a set of service APIs, extenders, and web resources. They are collected in JAR file that is specification only. This JAR file can be placed on your build path and allows you to develop applications without vendor lock in. External dependencies can be added through the JPM repository.

Currently the OSGi enRoute Base Profile includes the following services:

  • org.osgi.service.cm – Provides a push and pull model to configure components.
  • org.osgi.service.component – An extender for Declarative Services components.
  • org.osgi.service.coordinator – A coordinator for thread based requests with the possibility to get acallback at the end of a request.
  • org.osgi.service.event – A simple publish and subscribe mechanism for generic events.
  • org.osgi.service.http – An HTTP server framework.
  • org.osgi.service.log – A service to log errors, warning, and information to.
  • org.osgi.service.metatype – A standard to describe data structures with the intent to create userinterfaces from.
  • org.osgi.service.url – A facility to register URL handlers via the service registry.
  • org.osgi.service.useradmin – A service to manage user information like groups, credentials, andgeneral properties.
  • org.osgi.util.promise – A utility to do asynchronous processing using Promises.
  • org.osgi.util.tracker – Utilities to reliably track services and bundles.
  • osgi.enroute.authentication.api – Providing an authenticated id based on variable usercredentials.
  • osgi.enroute.authorization.api – Authorising applications to execute their actions by providingcurrent user based permissions.
  • osgi.enroute.capabilities – All specific non-code capabilities found in an enRoute Base profile.
  • osgi.enroute.configurer.api – An extender that gets configuration information from a bundle.
  • osgi.enroute.debug.api – Constants and helpers for debugging.
  • osgi.enroute.dto.api – Support for converting objects to other objects or JSON.
  • osgi.enroute.jsonrpc.api – A white-board approach to JSON RPC.
  • osgi.enroute.launch.api – API to interact with the launcher, including getting access to the startuparguments.
  • osgi.enroute.logger.api – A facility to manage logging on a platform when not only OSGi loggingis used.
  • osgi.enroute.rest.api – Provides REST end-points that are based on method naming pattern withtype safe use of the pay-load, parameters, and result body.
  • osgi.enroute.scheduler.api – Provides time based functions like delays, (cron and period based)schedules, and timeouts on Promises.

The following web resources:

  • Angular Javascript
  • Angular UI Javascript
  • D3 Javascript
  • Bootstrap CSS
  • Pagedown Javascript Markdown editor

The following support bundles:

  • A web server that serves static resources from bundles.
  • A configurator that reads configuration from bundles
  • An OSGi Event Admin μservice to Server Sent Events (Javascript) bridge

The Base Profile and its services are OSGi Core Release 6 and Java 8 based. The new APIs leverage lambdas to the hilt.

Workflow

The profile is sufficient to make sophisticated web applications. Under heavy development is the service catalog on the enRoute web site. This catalog will be the currently missing manual of how to use a service. It is the intention that each service will have an accompanying application that allows a developer to copy and paste the usage pattern and play with the service in a debugger.

Service Oriented Systems are composed of bundles that try to be reusable. Even if it is a one-off, it is usually sensible to develop as if it could potentially be reused as this does tend to make life simpler already during the development phase. Define proper APIs and make sure that only API packages are exported. After the components are developed or selected from the repository they need to be packaged into an application. In classic environments it can be painful to create this application since it must contain all transitive dependencies and without help this can be quite painful. For this reason, Bndtools has a resolve facility that can aid in quickly creating a closure of bundles that provide all necessary capabilities.

When the application is defined, Bndtools provides a debug and test environment that is very responsive. You create a bndrun file that specifies the details like the framework, properties, etc. This bndrun file can be launched and debugged. Virtually all changes in the IDE are immediately effectuated; you rarely have to restart the framework.

The output packaging depends on the target environment. Currently, by default, we package as an executable JAR that contains the framework, properties, and all bundles. The JAR runs anywhere you have a Java 8 runtime. However, we’re working with IBM to make it also deploy (and debug) on IBM WebSphere Application Server Liberty Core and with Paremus to deploy it on the Paremus Service Fabric. We’re also looking into Karaf Kars and others. Any other targets are also more than welcome. These targets must provide a distribution; a distribution must provide every capability defined in the profile.

Documentation and tutorials (where currently all the work is focused) are available on enroute.osgi.org. Everything we do is on GitHub. If you want to find out where all the parts are, a good place to start is in the Enroute Handbook.

Where are we?

So where is OSGi enRoute today? We are basically function complete and we’re ready for integration tests. This means that we need eager and willing early adopters that are prepared to help bring OSGi enRoute further along and can stand some hurdles, jumps, and hoops. The Version 1.0 release date is targeted for this Summer, aligned with the planned OSGi Enterprise Release 6 release date so that it can include some of the necessary updates.

The majority of the outstanding work is writing documentation, creating examples, making tutorials, and evangelising about OSGi enRoute.

Working on OSGi enRoute has made it clear that we did provide a wonderful solid box of Lego to the world although we forgot assembly instructions. It is clear that OSGi modularity really works; it provides a sophisticated dependency model that can truly build applications out of reusable components. However, it also became clear that we dearly need those proper OSGi services for those common tasks that every developer faces. We had way too few of those. Already, OSGi enRoute work has resulted in ten OSGi Requests For Proposal (163-172), the first step in the specification process.

Therefore, if you’re interested in being one of those early adopters or if you have time to help out, do not hesitate to contact me. For the rest, pay attention because once we’re fully live, OSGi will be mind blowing!

Author
Peter Kriens
Peter Kriens is an independent consultant since 1990.He currently works for the OSGi Allianceand jpm4j. During the eighties he developed advanced distributed systems for newspapers based on microcomputers based on, at the time very novel, object oriented technologies. For this experience in Objects he was hired by a number of international companies, including Adobe, Intel, Ericsson, IBM, and many others. During his work at Ericsson Research in 1998 he got involved with the OSGi specification; Later he became the primary editor for these specifications. In 2005 he was awarded the OSGi Fellows title. After taking a sabbatical in 2012 to develop jpm4j he returned to the OSGi Alliance to help increasing adoption. He is Dutch but decided to live in France.

Comments
comments powered by Disqus