GEF goes 3D

Jens von Pilgrim

In his article, “Quo vadis Eclipse?”, Holger Voorman ventures a glimpse into the future. He describes, how model changes are visualized in a three-dimensional manner and how he can fly over layers of abstraction by his finger tips. The article explains how this is possible with GEF3D today – although we have to use the mouse for navigation.

In the area of model driven development, models play an important role. Usually models are visualized by two-dimensional diagrams, UML diagrams may be the most prominent examples. However, in some cases two dimensions are simply not enough. Sometimes, models require a three-dimensional representation, or multiple two-dimensional diagrams should be displayed simultaneously. The latter case enables the visualization of so called inter-diagram relationships, that is connections between elements of different diagrams. The GEF3D [2] framework enables the creation of three-dimensional diagram editors in a very simply way.

GEF3D has been started as a tool in the context of the author’s Ph.D. project, and it has become an Eclipse project in the autumn of 2008. At the moment the project has only two committers – we are always looking for more help from the open source community! Figure 1, an editor implemented by the author, gives an impression of the capabilities of GEF3D. As foreseen by Holger Voorman, in the future, two layers of abstraction are visible here, more precisely a use case diagram which has been transformed into a robustness diagram (cf. [3]). The elements of the diagrams are connected via so called transformation traces. Traces are a typical example for inter-diagram relationships, in this case they connect the source and target elements of a transformation.

In the world of Eclipse, graphical editors are (almost always) created using GEF [4] or GMF [5] (whereby GMF is based on GEF). The basic idea of GEF3D is to extend GEF in order to support three-dimensional editors. Thus, 3D editors can be implemented using the same techniques and concepts that are used for 2D editors – special knowledge in 3D programming is not required in most common cases. Moreover, GEF3D provides techniques for adapting existing 2D editors in order to use them within a 3D scene – we call this phenomenon as “3D-fication” of 2D editors. This can be achieved with a little effort. The overall strategy is to project the output of the 2D editor onto a plane in 3D space, and to tweak them a little bit for enabling 3D editing. A few example editors are available in GEF3D to demonstrate this: The ecore editor (from the EMF tools [6]) or a class-, activity-, and use-case editor from the UML2 tools [7]. Actually Figure 1 shows the 3D-fied version of the UML2 tools editor.

Below, we are going to present GEF3D in more detail. After a brief explanation of GEF we will describe how GEF3D extends GEF. We will then create a small 2D editor with GEF – and subsequently 3D-fy that editor.

Some basic knowledge of GEF is required for understanding GEF3D. Hence we firstly highlight some design features of GEF and secondly describe how these features can be used for 3D-fication. Figure 2 serves as a guidance, illustrating the main classes and patterns used in GEF and GEF3D. Don’t worry, the figure looks a little bit overwhelmed at the first glance, but things will get clearer soon. We are going to describes the elements shown in the figure from inside out.

In the center of GEF we can identify the model-view-control (MVC) pattern, hence it is shown in the center of Figure 2 as well (light blue background). The implementation of the MVC is very fine grained, that is for each editable item, an MVC triple is required. E.g., you will need an MVC triple for the diagram, for each node and even for each label which should be editable. The model containing the data can be implemented by simple Java classes, you do not need to implement any interfaces or extend special classes. However, the model is often implemented using the Eclipse Modeling Framework (EMF [8]), and if GMF is used, EMF is even required. The view part is implemented by a separate plugin which comes with GEF, called Draw2D. Elements are visualized by so called Figures. Figures are structured as trees, and its root is contained within a class called LightweightSystem. The LightweightSystem is a kind of adapter between the figures and the SWT. It paints figures onto an SWT canvas and delegates events from SWT to GEF. As mentioned earlier, a controller has to be provided for each editable element. Controllers are implemented by so called EditParts, which do not only handle the event, but create the figures as well. In contrast to other MVC implementations, the controller is the most important part of the MVC triple in GEF, acting as a mediator between the figure and the model. In particular, the figure does not listen to the model, it even does not have any dependency to the model elements at all. Just like the figures, the EditParts are organized as a tree as well.

We now focus on the next level, the elements painted on the blue background in Figure 2. These elements are still part of GEF, we start at the right with the EditPolicy and wander around the circle counter clockwise. EditParts (playing the role of controllers) have to fulfill many tasks: from managing the selection of items, to creating elements, to more application specific tasks. To avoid monster classes (with spaghetti code), the policy pattern (also known as strategy pattern) is used to extract functionality into separate classes, called policy classes. A policy is “installed” and called to manage a certain task if needed. Besides reducing the size of the EditPart, its behavior can be changed dynamically. This can be done by simply installing new policies, or by replacing existing policies during runtime. We will do the latter in order to add 3D functionality to already created EditParts. Apropos creation: As you can see in the figure, EditParts are created using the factory pattern, the factory is called EditPartFactory here. The EditPartViewer is a container bringing all parts together: it holds a reference to the factory, it contains the already explained LightweightSystem, and it also contains the root of the edit part tree (not visualized in the figure). The viewer itself is part of the GraphicalEditor, which serves as the interface to the Eclipse platform, it is registered as editor via Eclipse plugin mechanism.

To summarize, the MVC implements a separation of concerns, especially the visualization is separated from the content. The behavior (of the controller) can be dynamically modified by using policies, the controllers themselves are created using a factory. As we will see later, these design patterns are used later in order to 3D-fy existing editors.

GEF3D Overview

Last but not the least, we have a look at the outer circle (red), showing classes provided by GEF3D. This time we start at the bottom of Figure 2 and move clockwise. Most essentially, we have to add a new dimension to the two-dimensional view. This means we have to replace the view element of the MVC triple. Since the controller should be able to use the new three-dimensional view, we simply extend the original Figure, our 3D enabled subclass is called Figure3D. Instead of painting onto an SWT-Canvas, we have to use a 3D canvas now, called GLCanvas as we use OpenGL for rendering. As a matter of fact GEF3D does not depend on any particular 3D render library. Instead, a generic interface has to be implemented to adapt a concrete render library. Currently, an adapter for LWJGL [10] exists, and an adapter for JOGL [11] is planned. Accordingly a 3D version of the lightweight system, LightweightSystem3D, and a corresponding GraphicalViewer3D are provided. The viewer is created and configured by the editor, a basic version of a 3D editor is provided as well, although in most cases existing 2D editor classes are subclassed, as we will see later on.

Basically these are the most important classes of GEF3D and it would be possible to construct a 3D editor with these classes. Most 3D specific problems are handled by GEF3D in the background, programming a 3D editor is quite similar to programming a 2D editor. E.g., GEF3D handles the selection of elements in a 3D scene (called “picking”) or provides a camera for navigation. Coordinates are converts from 2D to 3D or vice versa transparently for the client code. This is especially useful when existing 2D code is to be reused, as we will see later.

Tweaking with Patterns

Lot of effort was spent in order to make use of most of the concepts and techniques of GEF for 3D editors. As a result, existing 2D editors can be 3D-fied with very little changes, most parts of an editor can be reused unchanged. Especially for adapting existing editors, GEF3D provides new patterns and makes use of existing patterns. The policy pattern plays a key role in all these approaches, because it enables the modification of behavior during runtime, without the need to alter the actual code. The idea is pretty simple: Existing policies for two-dimensional applications are to be replaced by appropriate 3D versions. Fortunately, only a few policies have to be exchanged, since most of the functionality is independent from the graphical representation.

In general, existing 2D code should not be altered during 3D-fication. Instead, the complete 3D related code required to enable the usage of an original 2D editor in a 3D scene is to be placed into an independent plugin, which depends on the original 2D plugin. The original classes are reused as much as possible, and, since the code is not changed, their behavior is altered during runtime. The central pattern used to alter the behavior of existing EditParts is called Borg Factory – if this reminds you of Start Trek, you’re not completely wrong. The classes involved in the Borg Factory pattern is shown in the right-upper area of Figure 2. First of all, a so called BorgFactory replaces an original EditPartFactory, the original factory is nested into the BorgFactory – this is very similar to the proxy pattern. So called Assimilators can be registered at the BorgFactory, in order to perform certain modifications. When a new element is to be created, the sequence is as follows: First, the nested original factory creates the EditPart, based on a given model element. After the EditPart has been created, it is passed to the registered Assimilators. They can now modify the created EditPart, e.g., policies can be added or removed. If it is too hard to modify the EditPart, it is even possible to replace it completely – “resistance is futile”.

The other patterns shown in Figure 2, the Multi Factory pattern and the Connected Element pattern, are needed when a modified editor is combined with other editors. Both patterns will be explained in a follow up article.

Graph Editor Example

We will now have a look at a concrete example, in order to illustrate how GEF3D and the new patterns are to be used. In the following, we are going to implement a very simple editor for graphs with nodes (called vertices) and connections (edges). First, we will implement an 2D editor with GEF, and then 3D-fy that editor. The final result is shown in Figure 3. The screenshot shows how two nodes are moved, you can see that GEF3D displayes 3D handles and a 3D feedback figure.

Whether or not we create a 2D or 3D editor, we initially have to create a model. Most simply this is done with EMF. The EMF model used for our example editor is shown in Figure 4, the model has been created with the ecore diagram editor [6].

From that model a Java implementation can be automatically generated by EMF. Since we want to use a wizard later for creating a model, the edit and editor code are to be generated as well.

Now we have to implement the 2D editor, which is created as a plugin. Broadly speaking, we have to implement an editor class, the edit part factory, and the view and controller classes for our model elements. Elements are modified by GEF using the command pattern. That is, elements are not directly changed by the edit part of policy classes, instead the policies create appropriate commands which then execute the modifications. Accordingly we have to create some commands for creating or modifying elements (we omit commands for deleting elements here), and implement corresponding policies which create these commands. The following table lists all the classes we need for the example editor. More information about GEF can be found at the GEF website [3], about EMF at EMFs website [8], the sources of the example editor are part of the GEF3D examples (the GEF3D graph example contains a non-EMF model for simplicity, the 3D-fied graph editor can be found in

org.eclipse.gef3d.examples.graph.editor.GraphEditor2D_3Dfied).
Class(es) / Files Description

GraphEditor2D

Editor class plugged into Eclipse, the editor with its classes (e.g., EditPartFactory and palette) are configured here and the model is loaded

GraphEditPartFactory

Factory for creating EditParts according to given model elements.

GraphEditPart, VertexEditPart, EdgeEditPart

Controllers of displayed elements. The controllers serve their model elements and refresh the view accordingly. Besides, they are responsible for mapping the model to the displayed figure elements.

GraphFigure, VertexFigure, EdgeFigure

Figures displaying the model elements.

VertexCreateCommand, VertexResizeCommand, EdgeCreateCommand

Commands for creating or changing (model) elements. These commands are created themselves by the policies listed below.

GraphLayoutPolicy, VertexEditPolicy

Policies for creating the commands above. The vertex commands are created by the GraphLayoutPolicy, and the edge command by the VertexEditPolicy. The policies are installed at the edit parts.

plugin.xml

The editor must be registered at the Eclipse platform, the extension point used for that is org.eclipse.ui.editors.
Table 1: A 2D graph editor with only 14 files. 3D-Fication

In order to use the previously created 2D editor in a 3D scene, only a few modifications are necessary: no more then three classes are needed. In GEF3D, 2D content is projected onto planes in the 3D scene. Thereby it is possible to reuse almost all parts of the 2D editor without any modifications. Only the figure displaying the diagram, in the example the GraphFigure, has to be replaced, since we want to use a plane instead of a 2D area. The GraphEditPart has to be changed as well, because we have to create the 3D graph figure now instead of the 2D one. We call the 3D classes GraphFigure3D and GraphEditPart3D respectively. GraphEditPart3D is subclassed from GraphEditPart, the modification is minimal: Instead of creating a 2D figure, GraphFigure3D is created. Table 2 lists all three classes (and the plugin file) required to 3D-fy the editor.

Classes / Files Description

GraphEditor3D

The 3D editor subclasses the 2D editor. The edit part factory is modified, and a new camera tool is created in order to enable the user to navigate in the 3D scene.

GraphEditPart3D

The diagram edit part for the 3D plane is subclasses from the original 2D version (here GraphEditPart), the only modification is to create a 3D figure instead of a 2D one.

GraphFigure3D

Instead of a 2D figure, a 3D plane (i.e., a cuboid) is to be created. The elements of the diagram are projected onto that plane.

plugin.xml

The 3D editor has to be registered at the Eclipse platform as well, again the extension point org.eclipse.ui.editors is used.
Table 2: The 3D-fication requires only 4 files!

The figure displaying the diagram itself, that is GraphFigure3D, has to be created from scratch, the original 2D figure cannot be reused. However, GEF3D provides some base figures which makes it very simple, in most cases no 3D programming knowledge is required. In order to give an impression of how 3D elements are to be implemented with GEF3D, the whole code of GraphFigure3D is listed below.:

public class GraphFigure3D extends ShapeFigure3D {
 
  private ISurface m_surface = new FigureSurface(this);
     
  public GraphFigure3D() {
          SurfaceLayout.setDelegate(this, new FreeformLayout());
            getPosition3D().setLocation3D(IVector3f.NULLVEC3f);
               getPosition3D().setSize3D(new Vector3fImpl(400,300,20));
          setBackgroundColor(ColorConstants.white);
         setAlpha((byte) 0x44);
    }
 
  @Override
 public ISurface getSurface() {
            return m_surface;
 }

   @Override
 protected Shape createShape() {
           return new CuboidFigureShape(this);
       }

}
Due to the usage of helper elements, so called Shapes, the implementation of a 3D figure looks quite similar to the one of 2D figures. As shapes are often used to actually render a figure, GEF3D provides a special figure class ShapeFigure3D, which uses a shape internally. As seen in the example, a subclass of ShapeFigure3D must only implement the abstract method createShape(). GEF3D comes with a bunch of shapes, such as cuboid, sphere, or cylinder. In order to be able to project the 2D content onto the figure of the graph (GraphFigure), the 3D figure needs a surface (see method getSurface()). The actual rendering is now hidden in the base class. In contrast to GEF, figures are not simply rendered by (recursively) calling a render-method. Instead, in a first step so called RenderFragments are collected. In a second step, these fragments are rendered, producing the actual output. This is done due to restrictions of OpenGL when rendering transparent objects. As a matter of fact, OpenGL does not support transparent objects directly, thus GEF3D has to take care that these objects are rendered correctly. This is achieved by using depth-ordered RenderFragments – in most cases, this is not visible at the client code level as shown in the example. The most work left to the client is to initialize the 3D editor correctly. If no 2D editor class has to be reused, one can simply rely on 3D editor classes provided by GEF3D, such as GraphicalEditor3D. However, in many cases, an existing 2D editor already exists, containing code for loading or saving the model and other application specific things. For that reason, the 2D editor has to be used as a base class, and the code for setting up the 3D editor has to be implemented in the client code. Hence we reuse the 2D editor here as well, that is we derive GraphEditor3D from GraphEditor2D. Now all 3D specific initialization code has to be implemented her, roughly spoken the following things have to be done:
  1. Instead of a GraphicalViewer, a GraphicalViewer3D has to be created.
  2. In order to navigate in the 3D scene, a camera has to be installed. This is done by adding a special tool (CameraTool).
  3. Instead of the 2D diagram element, the previously created 3D graph figure (and its editor part) has to be created. Moreover, all EditParts are to be adapted to be editable in the 3D view. E.g., instead of two-dimensional handles and feedback figure, 3D versions of these items are to be used as shown in Figure 4.
In the following we will address all these issues. That is, first of all we have to create a 3D viewer. For that, we have to override createGraphicalViewer() as follows:
@Override
protected void createGraphicalViewer(Composite i_parent) {
  GraphicalViewer3DImpl viewer = new GraphicalViewer3DImpl();
       Control control = viewer.createControl3D(i_parent);
       setGraphicalViewer(viewer);
       configureGraphicalViewer();
       hookGraphicalViewer();
    initializeGraphicalViewer();
      control.addDisposeListener(viewer.getLightweightSystem3D());
}
  The camera is simply installed as a tool. In order to reuse the original tools (which can still be used in the 3D version), the original palette initialization is called in the first line:
protected PaletteRoot getPaletteRoot() {
   PaletteRoot root = super.getPaletteRoot();

  PaletteDrawer drawer = new PaletteDrawer("GEF3D");
        drawer.setDescription("GEF3D tools");
     drawer.add(new ToolEntry("Camera", "Camera Tool", null, null,
             CameraTool.class) {});
    root.add(0, drawer);
      return root;
}
  These two first steps are pretty simple. Modifying the edit parts is more tricky, and we will need Borg technology to solve that problem. In other words, we have to use the borg factory pattern here:
protected BorgEditPartFactory createBorgFactory(
           EditPartFactory originalFactory) {
        BorgEditPartFactory borgFactory = new BorgEditPartFactory(originalFactory);
       
  // replace diagram edit part
      borgFactory.addAssimilator(new EditPartReplacer(GraphEditPart.class,
              GraphEditPart3D.class));

    // modify diagram edit part's policies 
   borgFactory.addAssimilator(new AbstractPolicyModifier() {

           public boolean match(EditPart part) {
                     return part instanceof GraphEditPart3D;
           }

           public void modifyPolicies(EditPart io_editpart) {
                        // feedback when creating a node:
                 io_editpart.installEditPolicy(
                            ShowLayoutFeedbackEditPolicy3D.ROLE,
                              new ShowLayoutFeedbackEditPolicy3D());
                    // handles and feedback when moving or resizing a node
                    io_editpart.installEditPolicy(
                            Handles3DEditPolicy.CHILD_DECORATOR,
                              new Handles3DEditPolicy());
               }
 });

 // modify node edit part's policies
       borgFactory.addAssimilator(new IAssimilator.InstanceOf(
           NodeEditPart.class) {

               public EditPart assimilate(EditPart io_editpart) {
                        // feedback when drawing a connection
                     io_editpart.installEditPolicy(
                            ShowSourceFeedback3DEditPolicy.ROLE,
                              new ShowSourceFeedback3DEditPolicy());
                    return io_editpart;
               }

   });
       return borgFactory;
}
The code is almost self-explanatory. First, the original edit part factory is nested inside the BorgEditPartFactory. Then appropriate assimilators are registered at the borg factory. The first assimilator is the EditPartReplacer, which replaces the 2D edit part by a 3D edit part. In the example, edit parts of type GraphEditPart are to be replaced by their 3D counterparts, GraphEditPart3D. The second assimilator, an AbstractPolicyModifier, does not change the edit part itself, but it adds two new policies to the 2D edit part: ShowLayoutFeedbackEditPolicy3D creates a 3D feedback figure when a node is to be created, and Handles3DEditPolicy takes care of adding 3D handles when nodes are to be resized or moved. The third assimilator also adds a new policy: ShowSourceFeedback3DEditPolicy creates a 3D feedback figure when a new connection is to be created. Last but not least, the BorgEditPartFactory must be set as factory, and a RootEditPart3D is to be installed which is required in the 3D editor:
protected void configureGraphicalViewer() {
        super.configureGraphicalViewer();
 EditPartFactory originalFactory = 
                getGraphicalViewer().getEditPartFactory();
        BorgEditPartFactory borgFactory = createBorgFactory(originalFactory);
     getGraphicalViewer().setEditPartFactory(borgFactory);

       ScalableFreeformRootEditPart root =     new ScalableFreeformRootEditPart3D();
     getGraphicalViewer().setRootEditPart(root);
}
That’s it! These are the changes required to 3D-fy an existing editor. We have to register the new 3D editor using the well-known extension point org.eclipse.ui.editors, then we are able to open, view and edit the very same diagrams in 3D as we can edit with the 2D editor. Conclusion As demonstrated with the small example, creating 3D editors with GEF3D is surprisingly simple. At the moment, no official release of GEF3D is available, more about that and current restrictions can be found in the box “Incubator”. By 3D-fying exiting 2D editors, it is a good way for taking first steps into the world of 3D programming. Although, GEF3D supports full 3D editors, especially when predefined elements (such as diagram elements) and connections between them are to be edited. If you need to create your own CAD system, GEF3D is not recommended. However, the design and future development of GEF3D is focused on 3D-fying existing 2D editors since it is an interesting technique for visualizing models (and model chains) in the area of model driven approaches. In the planned second article, we will explain more features of GEF3D, e.g., how to combine multiple editors in a single 3D scene, and how to 3D-fy GMF-based editors.
Incubator At the moment, GEF3D is in the Eclipse incubator. There is no official release of GEF3D available yet (on how to install GEF3D at the moment, see box “Installation”). Known problems are documented in bugzilla. Currently, the most important restriction is the absence of 3D handlers for moving a 3D figure in 3D space (e.g., for rotating a 3D figure). Rotation and positioning of figures in 3D space is fully supported, it is simply not possible to do that interactively at the moment. Besides, the incubator does not only mean that the software is not stable yet, but also that the project and the team has to become more mature. At the moment, there are only two committers, and help is highly appreciated! GEF3D provides a website with a wiki, questions are answered at the newsgroup or developer mailing list [2].
Installation of GEF3D The official release of GEF3D is not yet available. In order to use GEF3D, Eclipse (and GEF) 3.5 are required. GEF3D has to be imported into the workspace from its subversion repository ( http://dev.eclipse.org/svnroot/technology/org.eclipse.gef3d/trunk). The following projects are to be imported:
  • org.eclipse.draw3d
  • org.eclipse.draw3d.geometry
  • org.eclipse.draw3d.graphics3d
  • org.eclipse.draw3d.lwjgl
  • org.eclipse.draw3d.ui
  • org.eclipse.gef3d
  • org.eclipse.gef3d.ext
All examples are found under org.eclipse.gef3d.examples.*, for 3D-fying GMF based editors, some helper classes are provided in org.eclipse.gef3d.gmf. Besides the GEF3D (and Draw3D) code, a rendering library is required in order to actually draw the 3D scene. For creating 3D output on the screen, OpenGL is used. Since OpenGL only provides a C API, a Java wrapper library such as LWJGL [10] is required. At the LWJGL update site (http://lwjgl.org/update), an Eclipse plugin can be downloaded, which was created in the context of GEF3D. An up-to-date installation instruction can be found in the GEF3D-wiki (http://wiki.eclipse.org/GEF3D)
Links & Literature [1] Voormann, H.: Quo vadis Eclipse? Eclipse Magazin 4 (2008), April, S. 20–27 [2] Eclipse GEF3D: www.eclipse.org/gef3d [3] Jacobson, I.: Object-Oriented Software Engineering: A Use Case Driven Approach. Addison-Wesley Professional, 1992 (acm press) [4] Eclipse GEF: www.eclipse.org/gef [5] Eclipse GMF: www.eclipse.org/gmf [6] Eclipse Modeling Framework Technology (EMFT): www.eclipse.org/modeling/emft [7] Eclipse Model Development Tools (MDT): www.eclipse.org/modeling/mdt [8] Eclipse EMF: www.eclipse.org/modeling/emf [9] Bokowski, B. ; Gerhardt, F.: GEF: Maßgeschneiderte grafische Editoren selbst erstellen. In: Eclipse Magazin 2 (2005), Februar, S. 85–90 [10] Lighweight Java Game Library (LWJGL): www.lwjgl.org [11] Java Bindings for OpenGL API (JOGL): jogl.dev.java.net
Author
Jens von Pilgrim
Jens von Pilgrim is a research assistant at the Software Engineering Group at FernUniversit
Comments
comments powered by Disqus