Tutorial – EMFStore – A Model Repository

The Eclipse Modeling Framework (EMF) allows the generation of Java entities from a very focused and compact model language called Ecore. We like to think of the generated entities as ‘Java Beans on steroids.’ They provide additional functionality, such as change notification and reflective access and, furthermore the generated entities follow the same template, very much like Java Beans (more information about EMF here). This allows to provide frameworks for different use cases, which work for all EMF entities.

An example of such a framework is EMFStore, a server solution for EMF entities or ‘model repository’. EMFStore allows parallel and distributed editing of entities, supporting continuous offline work. This means that a client can modify the available entities until it decides that a set of changes is consistent and should be shared with the other clients. The potentially conflicting changes between two clients is resolved through EMFStore’s support for interactive merging of EMF model entities.

Try it!

The EMFStore model repository is a headless framework. It is usually integrated with custom applications to provide collaboration and versioning for EMF entities. But, as method calls are not helpful visually in understanding the use of a framework, EMFStore also offers an example application out of the box. This application is based on the EMF Client Platform, a generic UI framework for EMF. The example client provides the ability to create model entities, modify them and synchronize all changes with the EMFStore server. With this example client, all you need in order to try out the EMFStore is your own EMF model and the generated entities.

To install and run EMFStore, EMF Client Platform and find an example model, there’s documentation for both here and here. The example model used for this article contains entities from the game bowling, such as Player or League. However, the entities could be from any kind of domain.

The first thing to try out in EMFStore is creating some entities for your model. An easy way to do that is to use the example client provided. EMFStore organizes entities into projects. The example client provides a navigator view, which shows all projects as well as the contained entities. The editor view allows you to modify all attributes and references for the created entities. Using the example client, you are able to create a project locally, before it gets shared on the EMFStore server (see Figure 1 below).

Figure 1: The example client allows you to create and modify entities

Once a project is created locally, it can be shared on the EMFStore server. Shared projects can be checked out by other clients. It is even possible to check out the same project on the same client multiple times, for example, in order to access different versions of it (see Figure 2 below).

Figure 2: EMFStore allows you to create multiple check-outs of the same project

After sharing a project with the EMFStore, all changes you apply to model entities will be tracked. To see how this works, you can add a new model entity, or change an attribute value. These changes can be made via the example client, by a custom client or via the API of the entities (using setter methods). By default, the changes are only visible in the local project. Once a consistent set of changes is complete, they can be committed to the EMFStore server. The example client will use a default dialog that prompts the user to enter a commit message, describing the changes applied. Furthermore, the commit dialog allows a detailed review of the changes before they are transferred to the EMFStore server. A commit can also be triggered without any UI interaction (see below).

Figure 3: The default commit dialog allows reviewing the applied changes and adding a comment describing them

Once the commit is finished, other clients can trigger an update to receive the latest changes from the EMFStore server. Again, the example client will show a dialog to describe the applied changes in detail. This workflow will automatically produce a very detailed history of all entities. The example client provides a view to review all previous versions of a project (see Figure 4 below). In this view you can also check out a specific version of the project.

Figure 4: The history view allows reviewing and re-creating all versions of the project

Using this workflow (commit/update), an arbitrary number of clients can collaborate on the entities (an application’s data). All changes can be done locally, even without a server connection. As mentioned earlier, there could be conflicting changes among clients. For example, two clients could set a different name for an entity at the same time. To resolve this conflict, EMFStore server supports interactive merging for model entities. The example client includes a default UI for conflict resolution and interactive merging (see Figure 5).

Figure 5: In case of a conflict between clients, the merge dialog allows to select the changes to apply

The example client provides additional features such as a repository browser to manage access to different servers. All UI components of the example client can be re-used in a custom application. This way, it is possible to efficiently create an initial running version supporting all features. All the features described here can also be used without the default UI components using the API of the EMFStore server. We describe this API more in detail in the section “API”.

Behind the scenes

The use cases of the EMFStore are quite similar to a Source Code Management System (SCM) such as SVN, CVS or Git. However, from a technical point of view, the EMFStore fundamentally works differently. If you applied a versioning system such as Git for EMF model entities, they would need to be serialized to files, for example, to XMI. During a commit or update, these files would be compared to calculate the applied changes (diffing). In SCMs, this process is line-based. While it works great for the original use case, managing code, this technique has significant shortcomings for models and model entities (see Operation-based Model Evolution, Maximilian Kögel, Dr. Hut Verlag, ISBN: 978-3843900812).

The reason is essentially that an SCM manages files and text, but it does not consider the model itself. If, for example, two clients create references to the same model entity, a SCM might detect a conflict, as the same lines in the serialization have been changed. However, from a model point of view, adding references might not be a conflict at all. A second problem with SCM’s is the interactive merge support, which is again text-based rather than model-based. As changes on the entities, such as adding a reference, might change totally different locations within the serialization, text-based merging of model entities is very difficult and error prone. For many end user applications, text-based merging is not an option anyways.

To solve these issues, the EMFStore uses one of the features of EMF model entities. EMF provides the ability to be notified about all changes on entities, for example, changing an attribute or adding a reference. The client component of the EMFStore records all these changes locally. Once a commit is triggered, no diffing is even required. The client already knows exactly what changes were applied locally. Therefore, only the changes have to be transferred to the server. The same applies in the case of an update.

Recording changes creates another major advantage in that it is much more precise than diffing files. In case of a conflict, the EMFStore can consider information on the model entity level. An applied change is not a change in an arbitrary line in a file – it is an operation on an entity, e.g. “Added reference Z between Y and X”. This additional information allows more precise detection of conflicts (see book reference above). In case of a conflict, the user can be supported by a UI displaying the changes on the model entity level, which is much more informative than merging text-files. The conflict detection and resolution strategies can even be adapted, for example, to include domain-specific rules. Finally, EMFStore also offers a history, which is much more detailed than in standard SCM’s. (For example, it is possible to track the order in which changes were applied.)

The API

In the section „Try it!”, we describe the most important use cases based on an extensible example client. However, EMFStore can also be used independently of the example client and can be integrated into existing applications. To show you how we do this, we’ll describe some examples from the EMFStore API. Check out detailed documentation of the EMFStore API to find out more about it.

Projects on the client are managed in a Workspace that allows you to access and create projects. A project is managed by a ProjectSpace whose entities contain meta-information, such as the name of a project or tracked changes. The following code example retrieves the current workspace and creates a new local project:

Workspace workspace = WorkspaceManager.getInstance().getCurrentWorkspace();
ProjectSpace projectSpace = workspace.createLocalProject("ProjectName", "Project Description");

The next step is to fill the local project with model entities. In EMF, entities are structured by a containment hierarchy (see earlier tutorial example). The following call adds a new entity into a project:

projectSpace.getProject().getModelElements().add(myEntity);

Regular changes on the entities are made using the getter and setter methods, making EMFStore transparent for the client. If new entities are added to other entities, they are automatically added to the project as well. The following code example changes an entity’s attribute and adds a new entity to another one.

myEntity.setName(“xxx”);
myEntity.getChildren().add(newObject);

The project created is only locally available, but with the following code example, it is shared with the EMFStore server and therefore available for other clients. To manage the connection with the EMFStore server a UserSession is created, containing credentials for access control, the server address and port.

Usersession usersession = EMFStoreClientUtil.createUsersession("super", "super",
 "localhost", 8080);
projectSpace.shareProject(usersession);

Once a project is shared on the EMFStore server, it can be retrieved by other clients via checkout. In this case, the user usually selects from a list of available projects retrieved from the server. The following code example retrieves the list of available projects and selects the first one for check-out.

List<ProjectInfo> projectList =
workspace.getRemoteProjectList(usersession);
workspace.checkout(usersession, projectList.get(0));

Once projects are shared on multiple clients, changes can be synchronized. The first step is to apply changes to the model entities, e.g. someElement.setName(„newName“). These changes are automatically recorded and can be sent to the EMFStore server:

projectSpace.commit(logMessage, null, new ConsoleProgressMonitor()); 

In turn, changes can be retrieved from the server triggering an update:

projectSpace.update();

Using the features share, checkout, commit and update allows you to create a custom client which synchronizes its data (model entities) through the EMFStore server. The API offers many additional features such as retrieving a list of current changes or resetting the state of a project to a specific version.

Conclusion

The EMFStore model repository is a runtime technology that enables you to collaboratively modify EMF model entities. Using an example client provided by the EMF Client Platform, it assists in testing the most important use cases. In contrast with existing Source Code Management Systems, the EMFStore model repository does not work directly on files, but instead versions entities on a model level. This enables precise conflict detection for EMF model entities as well as interactive merging. All features of the EMFStore can also be triggered via an API. This allows the integration of EMFStore into custom clients, with or without a UI. EMFStore is an open source framework (Eclipse Public License). Source code, releases and documentation are all available online.