days
0
6
hours
1
5
minutes
2
4
seconds
1
3
search
The framework for Microservices and Domain-Driven Design

The Lagom Framework

Lutz Hühnken

Radically different, but nonetheless easy — that is the dichotomy the new open source microservice framework Lagom is trying to create. What are the features that differentiate it from other frameworks? How easy is it to handle? What does the name actually mean?

The question regarding the meaning of the name is not easy to answer since one cannot literally translate the Swedish idiom Lagom. According to Wikipedia, the meaning is: “Enough, sufficient, adequate, just right.” In our case, this is not supposed to be a self-praise but a critical statement to the concept of microservices. Instead of focusing on “micro” and stubbornly following a “the less code, the better” concept, Lagom suggests that we think of a concept of “Bounded Context” from the Domain-Driven Design to find the boundaries for a service. The conceptual proximity of domain driven design and microservices can be found in different locations in the Lagom framework.

Getting started with Lagom

The easiest way to develop an application with Lagom is with the help of a Maven project template.

$ mvn archetype:generate -DarchetypeGroupId=com.lightbend.lagom \
-DarchetypeArtifactId=maven-archetype-lagom-java \
-DarchetypeVersion=1.1.0

After the questions regarding names have been answered and you switch into the newly-created directory, you will find the directory structure as displayed here.

cassandra-config
hello-api
hello-impl
integration-tests
pom.xml
stream-api
stream-impl

As it should be for microservices, not one, but already two services were generated. After all, the interaction and communication between services are at least as important as the implementation of a single on (and frequently the bigger challenge). Here are the services “hello” and “stream”; each implementation is divided into two subprojects (“api” and “impl”).

To launch the application, a simple mvn lagom:runAll .

After a few downloads, it should be running at Port 9000. This can be easily checked with a command line tool like HTTPIE:

$ http localhost:9000/api/hello/Lagom
HTTP/1.1 200 OK
Content-Type: text/plainHello, Lagom!

One particularity that all components needed in the development have is that they —the project’s services, a service registry, an API gateway, and even the database Cassandra (in the embedded version)— are launched through the Maven plug-in. It is not necessary to set up services or a database outside of the project. Lagom stresses the importance to offer the developer an environment which feels interactive – check out the project and get going. This includes the fact that code changes will come into effect right after a reload, without the need for a build/deploy/restart cycle.

The services API — typesafe and asynchronous

As it can be seen from the folder structure, every service is divided into an implementation (“-”) and an API definition (“-”). The latter defines the HTTP interface of the service programmatically.

public interface HelloService extends Service {

  ServiceCall<NotUsed, String> hello(String id);

  default Descriptor descriptor() {
    return named("hello").withCalls(
        pathCall("/api/hello/:id",  this::hello),
      );
  }
}

With the help of a builder, the service description will be created, in which the requested path will be mapped on a method call.

This interface is not only the template for implementation; Lagom also generates an appropriate client library. In other Lagom services, this can be injected via dependency injection with Google’s Guice. This way, a type-safe interface is provided  when the respective service is selected. The manual construction of an HTML request and the direct use of a generic http client can be omitted.

Still, it is not mandatory to use the client library because the framework maps the method calls on HTTP calls, which may also be called directly, especially by non-Lagom services.
By the way, our little „hello“ method doesn’t deliver the response directly, but a ServiceCall. This is a functional interface. That is to say we do not create a simple object but a function – the function which shall be executed by the corresponding request. We deliver the types as type parameters for the request (since user GET call doesn’t submit any data, in this case “NotUsed”) and the response (in our case a simple String).  The processing of the request is always asynchronous – the outcome of our function must be a CompletionStage. Lagom extensively uses Java 8 features. A simple implementation would look like this:

public class HelloServiceImpl implements HelloService {
  @Override
  public ServiceCall<NotUsed, String> hello(String id) {
    return request -> {
      CompletableFuture.completedFuture("Hello, " + id);
    };
  }
}

For a simple GET request, the gain of the service descriptors is limited. It gets more interesting when we want to send events between services asynchronously. We can achieve this in Lagom by choosing different type parameters for the ServiceCall. If our request and response types are defined as source (a type from the Akka streams library), the framework will initialize a WebSocket link. Here the service abstraction can score since it simplifies working with the WebSockets. As far as future versions are concerned, there are plans to support the additional “publish/subscribe” pattern so that messages can be placed on a bus and other services can subscribe to it.

public interface StreamService extends Service {

  ServiceCall<Source<String, NotUsed>, Source<String, NotUsed>> stream();

  @Override
  default Descriptor descriptor() {
    return named("stream").withCalls(namedCall("stream", this::stream));
  }
}

Circuit breaker built-in

Let us assume that our service requests information per HTTP request at another service. This doesn’t respond within the expected timeframe, which means there will be a timeout. Requests to this server shouldn’t be repeated constantly because we ask an unnecessary amount of idle time from our application: If it’s likely that we won’t be getting a response, why should we wait for a timeout? Furthermore, there would be requests accumulating to the service. As soon as it becomes available again, it will be bombarded with pending requests to such an extent that it will be brought to its knees immediately.

A reliable solution for this problem is the circuit breaker pattern [6]. A circuit breaker knows three states:

  • As long as everything is running without errors, it is closed
  • If a defined limit of errors (timeouts, exceptions) is reached, it will be open for a defined period of time. Additional requests will fail with a “CircuitBreakerException”. For the client there won’t be additional waiting time and the external service won’t even notice the request.
  • As soon as the set time period runs out, the circuit breaker will switch into the state “half open”. Now there will be one request passed through. If it is successful, the circuit breaker will be closed- the external system seems to be available again. If it fails, the next round with the state “open” begins.
    Such circuit breakers are already integrated into the Lagom service client. The parameters are adjustable with the configuration file.

Lagom persistence

One aspect which proves that Lagom is very different from other micro frameworks is the integration of a framework for Event Sourcing and CQRS.
For many developers, working with a relational databaseis still the “default case”, possibly in connection with an ORM tool. Even this can be implemented in Lagom, but the user is steered into another direction. The standard in Lagom is the use of “Persistent Entities” ( corresponding to “Aggregate Roots” in Domain-Driven design). These Persistent Entities receive messages (commands).

public class HelloEntity extends PersistentEntity<HelloCommand, HelloEvent, HelloState> {

  @Override
  public Behavior initialBehavior(Optional<HelloState> snapshotState) {

    /*
     * Das Behavior definiert, wie die Entity auf Kommandos reagiert.
     */
    BehaviorBuilder b = newBehaviorBuilder(
        snapshotState.orElse(new HelloState("Hello", LocalDateTime.now().toString())));

    /*
     * Command handler für UseGreetingMessage.
     */
    b.setCommandHandler(UseGreetingMessage.class, (cmd, ctx) ->
    ctx.thenPersist(new GreetingMessageChanged(cmd.message),
        evt -> ctx.reply(Done.getInstance())));

    /*
     * Event handler für GreetingMessageChanged..
     */
    b.setEventHandler(GreetingMessageChanged.class,
        evt -> new HelloState(evt.message, LocalDateTime.now().toString()));

    return b.build();
  }
}

You can see exactly how this is presented in the code. Our quite simple entity allows us to change the welcome text for our service. We extend the superclass PersistentEntity which expects three type parameters: the command type, the event type, and the type of the state. In our case we define the command as a class UseGreetingMessage, which implements the interface HelloCommand and its instances are immutable. For type-saving purposes, one can go back to commands, events and states from the library Immutables. To save yourself some keystrokes, you can leverage a library such as Immutables for your commands, events and states.

The way our entity responds to commands is defined by a behavior. This can change at runtime. This way the entities can implement finite-state machines – the replacement of one behavior with another at the runtime correlates with the transition of the machines into another state.

The framework obtains the initial behavior via initialBevahior. To construct this, we will make use of the builder pattern.

First, we define a CommandHandler as our command. If a command is valid and demands the entity to be changed, for example, in case it sets an attribute to a new value, the change won’t occur immediately. Instead, an event will be created, saved and emitted. The EventHandler of the persistent entity which we also added with the builder to the behavior, reacts to the event and executes the actual change.

A significant difference to an “Update” in a relational database is that the current state of the persistent entity does not necessarily have to be saved. This will be merely held in memory (Memory Image). In case it becomes necessary to restore the state, e.g. after a restart of the application, this will be reconstructed through a playback of the events. The optional saving of the current state in called “Snapshot” in the model and does not replace the Event history, but solely represents a “pre-processing”. If an entity experienced thousands of changes of state during its lifetime, there is no need to play back all the events from the very beginning. It is possible to shortcut by starting with the latest snapshot and repeating only the following events.

The strict specifications that Lagom gives for the types and the structure of the behavior are meant to  ease the conversion to this principle, called Event Sourcing, for developers. The idea is that I am forced to specify a clear protocol for each entity: Which commands can be processed, which events can be triggered and which values define the state of my class?

Clustering included

The number of Persistent Entities that I can use is not limited by the main memory of a single server. Rather, every Lagom application can be used as a distributed application. During the start of an additional instance I only have to add the address of an already running instance, after that it will register there and form a cluster with the present instances. The Persistent Entities are administered by the framework and will be distributed automatically within the cluster (Cluster Sharding). If nodes are added to or removed from the cluster, the framework will redistribute the instances. Likewise, it can restore instances which were removed from the memory (Passivation).

By the way, the built-in feature to keep the application state in the memory this way and also to scale this hasn’t been developed for Lagom originally. For this, Lagom relies on Akka. This has definitely been used in mission-critical applications , therefore any concerns regarding the reliability of the young framework are not well-founded.

Separate writing and reading

While it is easy in SQL databases to request any information from the data model, it is impossible in the case of Event Sourcing.  We can only access our entity and request the state with the primary key. Since we only have an Event Log and not a relational data model, queries through secondary indices are impossible to make.

To enable this, the CQRS architecture (Command Query Responsibility Segregation, for further reading: A CQRS Journey) is applied. The basic principle here is that different data models are used for reading and writing. In our case this means that our Event Log is the write side.. It can be used to reconstruct our entities, but we won’t perform any queries on this. Instead, we also generate a read sidefrom the events. Lagom is already offering an ReadSideProcessor. Every event which occurs in combination with a class of PersistentEntities will also be processed and used to create the read side. This is optimized for reading and doesn’t allow for direct writing.

This architectural approach does not only offer technical advantages, since in many application cases the read and writing frequency are very different and they are scaled independently with this method. It also enables some new possibilities. As a consequence of never deleting the saved events, it is possible to add new structures on the read side, the so-called projections. These can be filled with the historical events and thus can give information not only in the future but also from the past.

CQRS allows the use of different technologies on the read side, adjusted to the Use Case. It is conceivable while not supported by Lagom yet, that one can build an SQL read sideand continue the use of available tooling, but simultaneously feeding an ElasticSearch database for the quick search and to send the events for analysis to Spark Streaming.

It is important to keep in mind that the read sidewill be refreshed asynchronously, with latency (“Eventual Consistency” between the write and the read side). Strong consistency is only available in this model on the level of the PersistentEntity.

Finally, it is also possible to code Lagom without Lagom Persistence. It is not mandatory to use Event Sourcing; the development of “stateless” – Services, or “CRUD” applications (Create, Read, Update, Delete) with a SQL database in the backend is also possible. But if someone is interested in Event Sourcing and CQRS, in scalable, distributed systems, Lagom can help them gain access into the topic.

Immutable values — Immutables

As mentioned earlier, the single commands, events and instances of the state must be immutable. Immutable data structures are an important concept from the functional coding, especially in the area of concurrency. Let us assume a method gets passed a list of numbers. The result is a value that is calculated from the list (maybe a meridian of the numbers of the list). By reasoning about this or maybe in some cases even through mathematical proof, you may state that a function is correct and will always deliver the same output for the same input.

But what if the delivered list is e.g. an ArrayList – how can we be sure? Fix is only the reference that is delivered. But what if another part of the program that is executed in parallel has the same reference? And adds some values to the list? In asynchronous systems that are based on sending the commands, it is essential that a command must not be changed after it has been sent. To rely on the fact that the developer will be careful would be negligent.

Lagom uses third party libraries for this. For the commands it binds Immutables, for collections pCollections. If I add a value to a collection from this library, the original collection will remain unchanged and I will receive a new instance with an additional value.

Deployment

Microservices provide a challenge not just for the developer but also for the ongoing operation. In many companies the deployment processes are still set up for the installation of .war or .ear files for application servers. But microservices are running standalone and are often packed into (Docker) containers and administered by the so-called service orchestration tools like Kubernetes or Docker Swarm.

Lagom requires such an environment, too. But it does not depend on a certain container standard (like Docker). It requires the runtime environment to have a registry which is searchable through other services. To be accessible, it must make an implementation of the Lagom ServiceLocator API available.

Unfortunately, at the moment it is only available for the commercial closed-source product ConductR. The open source community is working on the implementation for Kubernetes and Consul. Alternatively, a ServiceLocator based on static configuration can be used, but this is not recommended for production use.

Conclusion

Lagom follows an interesting path and is a remarkable framework. It’s fundamentally different in its technical base: Everything is asynchronous, it is based on sending commands and persisting is done per Event Sourcing. This brings tremendous advantages for the scalability of services – but for most developers (including everybody from the Java EE area), this means rethinking. With the change of a programming language always comes the fear of a temporary decrease in productivity because developers cannot revert to familiar practices and resources. It is the same in our case.

Lagom is trying to prevent this by giving the developer a clear path. If I follow the documentation of the textbook approach for service implementation and persistence in Lagom, I will be able to build a reactive system – completely based on messaging and being able to cluster, maybe even without realizing it.

In the relatively new area of microservices, standards are yet to be established. We will have to see which frameworks can stand the test of time. In contrast with old acquaintances from Java EE and Spring, Lagom instills new life into this and is putting a whole different architecture in the balance. Those who wish to try something new and are interested in scalable distributed systems will find Lagom helpful.


ReactiveTo read more about reactive programming, download the latest issue of JAX Magazine:

Reactive programming means different things to different people and we are not trying to reinvent the wheel or define this concept. Instead, we are allowing our authors to prove how Scala, Lagom, Spark, Akka and Play coexist and work together to create a reactive universe.

If the definition “stream of events” does not satisfy your thirst for knowledge, get ready to find out what reactive programming means to our experts in Scala, Lagom, Spark, Akka and Play. Plus, we talked to Scala creator Martin Odersky about the impending Scala 2.12, the current state of this programming language and the technical innovations that await us.

Thirsty for more? Open the magazine and see what we have prepared for you.

Author
Lutz Hühnken
Lutz Hühnken is an independent IT consultant and software developer.. He is an experienced software architect, project manager and team leader, with a history of successful projects in systems integration, internet/e-commerce and server-side application development in general.

Comments
comments powered by Disqus