GEF goes 3D

Jens von Pilgrim

In his article, “Quo vadis Eclipse?”, Holger Voorman ventures a glimpse into the future. He describes, how model changes are visualized in a three-dimensional manner and how he can fly over layers of abstraction by his finger tips. The article explains how this is possible with GEF3D today – although we have to use the mouse for navigation.

In the area of model driven development, models play an
important role. Usually models are visualized by two-dimensional
diagrams, UML diagrams may be the most prominent examples. However,
in some cases two dimensions are simply not enough. Sometimes,
models require a three-dimensional representation, or multiple
two-dimensional diagrams should be displayed simultaneously. The
latter case enables the visualization of so called inter-diagram
relationships, that is connections between elements of different
diagrams. The GEF3D [2] framework enables the creation of
three-dimensional diagram editors in a very simply way.

GEF3D has been started as a tool in the context of the author’s
Ph.D. project, and it has become an Eclipse project in the autumn
of 2008. At the moment the project has only two committers – we are
always looking for more help from the open source community! Figure
1, an editor implemented by the author, gives an impression of the
capabilities of GEF3D. As foreseen by Holger Voorman, in the
future, two layers of abstraction are visible here, more precisely
a use case diagram which has been transformed into a robustness
diagram (cf. [3]). The elements of the diagrams are connected via
so called transformation traces. Traces are a typical example for
inter-diagram relationships, in this case they connect the source
and target elements of a transformation.

In the world of Eclipse, graphical editors are (almost always)
created using GEF [4] or GMF [5] (whereby GMF is based on GEF). The
basic idea of GEF3D is to extend GEF in order to support
three-dimensional editors. Thus, 3D editors can be implemented
using the same techniques and concepts that are used for 2D editors
– special knowledge in 3D programming is not required in most
common cases. Moreover, GEF3D provides techniques for adapting
existing 2D editors in order to use them within a 3D scene – we
call this phenomenon as “3D-fication” of 2D editors. This can be
achieved with a little effort. The overall strategy is to project
the output of the 2D editor onto a plane in 3D space, and to tweak
them a little bit for enabling 3D editing. A few example editors
are available in GEF3D to demonstrate this: The ecore editor (from
the EMF tools [6]) or a class-, activity-, and use-case editor from
the UML2 tools [7]. Actually Figure 1 shows the 3D-fied version of
the UML2 tools editor.

Below, we are going to present GEF3D in more detail. After a
brief explanation of GEF we will describe how GEF3D extends GEF. We
will then create a small 2D editor with GEF – and subsequently
3D-fy that editor.

Some basic knowledge of GEF is required for understanding GEF3D.
Hence we firstly highlight some design features of GEF and secondly
describe how these features can be used for 3D-fication. Figure 2
serves as a guidance, illustrating the main classes and patterns
used in GEF and GEF3D. Don’t worry, the figure looks a little bit
overwhelmed at the first glance, but things will get clearer soon.
We are going to describes the elements shown in the figure from
inside out.

In the center of GEF we can identify the model-view-control
(MVC) pattern, hence it is shown in the center of Figure 2 as well
(light blue background). The implementation of the MVC is very fine
grained, that is for each editable item, an MVC triple is required.
E.g., you will need an MVC triple for the diagram, for each node
and even for each label which should be editable. The model
containing the data can be implemented by simple Java classes, you
do not need to implement any interfaces or extend special classes.
However, the model is often implemented using the Eclipse Modeling
Framework (EMF [8]), and if GMF is used, EMF is even required. The
view part is implemented by a separate plugin which comes with GEF,
called Draw2D. Elements are visualized by so called
Figures. Figures are structured as trees, and its
root is contained within a class called LightweightSystem.
The LightweightSystem is a kind of adapter between the
figures and the SWT. It paints figures onto an SWT canvas and
delegates events from SWT to GEF. As mentioned earlier, a
controller has to be provided for each editable element.
Controllers are implemented by so called EditParts, which
do not only handle the event, but create the figures as well. In
contrast to other MVC implementations, the controller is the most
important part of the MVC triple in GEF, acting as a mediator
between the figure and the model. In particular, the figure does
not listen to the model, it even does not have any dependency to
the model elements at all. Just like the figures, the
EditParts are organized as a tree as well.

We now focus on the next level, the elements painted on the blue
background in Figure 2. These elements are still part of GEF, we
start at the right with the EditPolicy and wander around
the circle counter clockwise. EditParts (playing the role of
controllers) have to fulfill many tasks: from managing the
selection of items, to creating elements, to more application
specific tasks. To avoid monster classes (with spaghetti code), the
policy pattern (also known as strategy pattern) is used to extract
functionality into separate classes, called policy classes. A
policy is “installed” and called to manage a certain task if
needed. Besides reducing the size of the EditPart, its behavior can
be changed dynamically. This can be done by simply installing new
policies, or by replacing existing policies during runtime. We will
do the latter in order to add 3D functionality to already created
EditParts. Apropos creation: As you can see in the figure,
EditParts are created using the factory pattern, the
factory is called EditPartFactory here. The EditPartViewer
is a container bringing all parts together: it holds a reference to
the factory, it contains the already explained LightweightSystem,
and it also contains the root of the edit part tree (not visualized
in the figure). The viewer itself is part of the GraphicalEditor,
which serves as the interface to the Eclipse platform, it is
registered as editor via Eclipse plugin mechanism.

To summarize, the MVC implements a separation of concerns,
especially the visualization is separated from the content. The
behavior (of the controller) can be dynamically modified by using
policies, the controllers themselves are created using a factory.
As we will see later, these design patterns are used later in order
to 3D-fy existing editors.

GEF3D Overview

Last but not the least, we have a look at the outer circle
(red), showing classes provided by GEF3D. This time we start at the
bottom of Figure 2 and move clockwise. Most essentially, we have to
add a new dimension to the two-dimensional view. This means we have
to replace the view element of the MVC triple. Since the controller
should be able to use the new three-dimensional view, we simply
extend the original Figure, our 3D enabled subclass is called
Figure3D. Instead of painting onto an SWT-Canvas, we have to use a
3D canvas now, called GLCanvas as we use OpenGL for rendering. As a
matter of fact GEF3D does not depend on any particular 3D render
library. Instead, a generic interface has to be implemented to
adapt a concrete render library. Currently, an adapter for LWJGL
[10] exists, and an adapter for JOGL [11] is planned. Accordingly a
3D version of the lightweight system, LightweightSystem3D, and a
corresponding GraphicalViewer3D are provided. The viewer is created
and configured by the editor, a basic version of a 3D editor is
provided as well, although in most cases existing 2D editor classes
are subclassed, as we will see later on.

Basically these are the most important classes of GEF3D and it
would be possible to construct a 3D editor with these classes. Most
3D specific problems are handled by GEF3D in the background,
programming a 3D editor is quite similar to programming a 2D
editor. E.g., GEF3D handles the selection of elements in a 3D scene
(called “picking”) or provides a camera for navigation. Coordinates
are converts from 2D to 3D or vice versa transparently for the
client code. This is especially useful when existing 2D code is to
be reused, as we will see later.

Tweaking with Patterns

Lot of effort was spent in order to make use of most of the
concepts and techniques of GEF for 3D editors. As a result,
existing 2D editors can be 3D-fied with very little changes, most
parts of an editor can be reused unchanged. Especially for adapting
existing editors, GEF3D provides new patterns and makes use of
existing patterns. The policy pattern plays a key role in all these
approaches, because it enables the modification of behavior during
runtime, without the need to alter the actual code. The idea is
pretty simple: Existing policies for two-dimensional applications
are to be replaced by appropriate 3D versions. Fortunately, only a
few policies have to be exchanged, since most of the functionality
is independent from the graphical representation.

In general, existing 2D code should not be altered during
3D-fication. Instead, the complete 3D related code required to
enable the usage of an original 2D editor in a 3D scene is to be
placed into an independent plugin, which depends on the original 2D
plugin. The original classes are reused as much as possible, and,
since the code is not changed, their behavior is altered during
runtime. The central pattern used to alter the behavior of existing
EditParts is called Borg Factory – if this reminds you of Start
Trek, you’re not completely wrong. The classes involved in the Borg
Factory pattern is shown in the right-upper area of Figure 2. First
of all, a so called BorgFactory replaces an original
EditPartFactory, the original factory is nested into the
BorgFactory – this is very similar to the proxy pattern. So called
Assimilators can be registered at the BorgFactory, in order to
perform certain modifications. When a new element is to be created,
the sequence is as follows: First, the nested original factory
creates the EditPart, based on a given model element. After the
EditPart has been created, it is passed to the registered
Assimilators. They can now modify the created EditPart, e.g.,
policies can be added or removed. If it is too hard to modify the
EditPart, it is even possible to replace it completely –
“resistance is futile”.

The other patterns shown in Figure 2, the Multi Factory pattern
and the Connected Element pattern, are needed when a modified
editor is combined with other editors. Both patterns will be
explained in a follow up article.

Graph Editor Example

We will now have a look at a concrete example, in order to
illustrate how GEF3D and the new patterns are to be used. In the
following, we are going to implement a very simple editor for
graphs with nodes (called vertices) and connections (edges). First,
we will implement an 2D editor with GEF, and then 3D-fy that
editor. The final result is shown in Figure 3. The screenshot shows
how two nodes are moved, you can see that GEF3D displayes 3D
handles and a 3D feedback figure.

Whether or not we create a 2D or 3D editor, we initially have to
create a model. Most simply this is done with EMF. The EMF model
used for our example editor is shown in Figure 4, the model has
been created with the ecore diagram editor [6].

From that model a Java implementation can be automatically
generated by EMF. Since we want to use a wizard later for creating
a model, the edit and editor code are to be generated as well.

Now we have to implement the 2D editor, which is created as a
plugin. Broadly speaking, we have to implement an editor class, the
edit part factory, and the view and controller classes for our
model elements. Elements are modified by GEF using the command
pattern. That is, elements are not directly changed by the edit
part of policy classes, instead the policies create appropriate
commands which then execute the modifications. Accordingly we have
to create some commands for creating or modifying elements (we omit
commands for deleting elements here), and implement corresponding
policies which create these commands. The following table lists all
the classes we need for the example editor. More information about
GEF can be found at the GEF website [3], about EMF at EMFs website
[8], the sources of the example editor are part of the GEF3D
examples (the GEF3D graph example contains a non-EMF model for
simplicity, the 3D-fied graph editor can be found in

org.eclipse.gef3d.examples.graph.editor.GraphEditor2D_3Dfied).

Class(es) / Files Description

GraphEditor2D

Editor class plugged into Eclipse, the editor with its classes
(e.g., EditPartFactory and palette) are configured here
and the model is loaded

GraphEditPartFactory

Factory for creating EditParts according to given model
elements.

GraphEditPart, VertexEditPart,
EdgeEditPart

Controllers of displayed elements. The controllers serve their
model elements and refresh the view accordingly. Besides, they are
responsible for mapping the model to the displayed figure
elements.

GraphFigure, VertexFigure, EdgeFigure

Figures displaying the model elements.

VertexCreateCommand, VertexResizeCommand,
EdgeCreateCommand

Commands for creating or changing (model) elements. These
commands are created themselves by the policies listed below.

GraphLayoutPolicy, VertexEditPolicy

Policies for creating the commands above. The vertex commands
are created by the GraphLayoutPolicy, and the edge
command by the VertexEditPolicy. The policies are
installed at the edit parts.

plugin.xml

The editor must be registered at the Eclipse platform, the
extension point used for that is
org.eclipse.ui.editors.

Table 1: A 2D graph editor with only 14 files.

3D-Fication

In order to use the previously created 2D editor in a 3D scene,
only a few modifications are necessary: no more then three classes
are needed. In GEF3D, 2D content is projected onto planes in the 3D
scene. Thereby it is possible to reuse almost all parts of the 2D
editor without any modifications. Only the figure displaying the
diagram, in the example the GraphFigure, has to be replaced, since
we want to use a plane instead of a 2D area. The GraphEditPart has
to be changed as well, because we have to create the 3D graph
figure now instead of the 2D one. We call the 3D classes
GraphFigure3D and GraphEditPart3D respectively.
GraphEditPart3D is subclassed from GraphEditPart, the modification
is minimal: Instead of creating a 2D figure, GraphFigure3D is
created. Table 2 lists all three classes (and the plugin file)
required to 3D-fy the editor.

Classes / Files Description

GraphEditor3D

The 3D editor subclasses the 2D editor. The edit part factory is
modified, and a new camera tool is created in order to enable the
user to navigate in the 3D scene.

GraphEditPart3D

The diagram edit part for the 3D plane is subclasses from the
original 2D version (here GraphEditPart), the only
modification is to create a 3D figure instead of a 2D one.

GraphFigure3D

Instead of a 2D figure, a 3D plane (i.e., a cuboid) is to be
created. The elements of the diagram are projected onto that
plane.

plugin.xml

The 3D editor has to be registered at the Eclipse platform as
well, again the extension point org.eclipse.ui.editors is
used.

Table 2: The 3D-fication requires only 4 files!

The figure displaying the diagram itself, that is
GraphFigure3D, has to be created from scratch, the
original 2D figure cannot be reused. However, GEF3D provides some
base figures which makes it very simple, in most cases no 3D
programming knowledge is required. In order to give an impression
of how 3D elements are to be implemented with GEF3D, the whole code
of GraphFigure3D is listed below.:

public class GraphFigure3D extends ShapeFigure3D {
 
  private ISurface m_surface = new FigureSurface(this);
     
  public GraphFigure3D() {
          SurfaceLayout.setDelegate(this, new FreeformLayout());
            getPosition3D().setLocation3D(IVector3f.NULLVEC3f);
               getPosition3D().setSize3D(new Vector3fImpl(400,300,20));
          setBackgroundColor(ColorConstants.white);
         setAlpha((byte) 0x44);
    }
 
  @Override
 public ISurface getSurface() {
            return m_surface;
 }

   @Override
 protected Shape createShape() {
           return new CuboidFigureShape(this);
       }

}

Due to the usage of helper elements, so called Shapes,
the implementation of a 3D figure looks quite similar to the one of
2D figures. As shapes are often used to actually render a figure,
GEF3D provides a special figure class ShapeFigure3D, which
uses a shape internally. As seen in the example, a subclass of
ShapeFigure3D must only implement the abstract method
createShape(). GEF3D comes with a bunch of shapes, such as
cuboid, sphere, or cylinder.

In order to be able to project the 2D content onto the figure of
the graph (GraphFigure), the 3D figure needs a surface (see method
getSurface()). The actual rendering is now hidden in the
base class. In contrast to GEF, figures are not simply rendered by
(recursively) calling a render-method. Instead, in a first step so
called RenderFragments are collected. In a second step,
these fragments are rendered, producing the actual output. This is
done due to restrictions of OpenGL when rendering transparent
objects. As a matter of fact, OpenGL does not support transparent
objects directly, thus GEF3D has to take care that these objects
are rendered correctly. This is achieved by using depth-ordered
RenderFragments – in most cases, this is not visible at
the client code level as shown in the example.

The most work left to the client is to initialize the 3D editor
correctly. If no 2D editor class has to be reused, one can simply
rely on 3D editor classes provided by GEF3D, such as
GraphicalEditor3D. However, in many cases, an existing 2D
editor already exists, containing code for loading or saving the
model and other application specific things. For that reason, the
2D editor has to be used as a base class, and the code for setting
up the 3D editor has to be implemented in the client code. Hence we
reuse the 2D editor here as well, that is we derive
GraphEditor3D from GraphEditor2D. Now all 3D
specific initialization code has to be implemented her, roughly
spoken the following things have to be done:

  1. Instead of a GraphicalViewer, a
    GraphicalViewer3D has to be created.
  2. In order to navigate in the 3D scene, a camera has to be
    installed. This is done by adding a special tool
    (CameraTool).
  3. Instead of the 2D diagram element, the previously created 3D
    graph figure (and its editor part) has to be created. Moreover, all
    EditParts are to be adapted to be editable in the 3D view.
    E.g., instead of two-dimensional handles and feedback figure, 3D
    versions of these items are to be used as shown in Figure 4.

In the following we will address all these issues. That is,
first of all we have to create a 3D viewer. For that, we have to
override createGraphicalViewer() as follows:

@Override
protected void createGraphicalViewer(Composite i_parent) {
  GraphicalViewer3DImpl viewer = new GraphicalViewer3DImpl();
       Control control = viewer.createControl3D(i_parent);
       setGraphicalViewer(viewer);
       configureGraphicalViewer();
       hookGraphicalViewer();
    initializeGraphicalViewer();
      control.addDisposeListener(viewer.getLightweightSystem3D());
}

 

The camera is simply installed as a tool. In order to reuse the
original tools (which can still be used in the 3D version), the
original palette initialization is called in the first line:

protected PaletteRoot getPaletteRoot() {
   PaletteRoot root = super.getPaletteRoot();

  PaletteDrawer drawer = new PaletteDrawer("GEF3D");
        drawer.setDescription("GEF3D tools");
     drawer.add(new ToolEntry("Camera", "Camera Tool", null, null,
             CameraTool.class) {});
    root.add(0, drawer);
      return root;
}

 

These two first steps are pretty simple. Modifying the edit
parts is more tricky, and we will need Borg technology to solve
that problem. In other words, we have to use the borg factory
pattern here:

protected BorgEditPartFactory createBorgFactory(
           EditPartFactory originalFactory) {
        BorgEditPartFactory borgFactory = new BorgEditPartFactory(originalFactory);
       
  // replace diagram edit part
      borgFactory.addAssimilator(new EditPartReplacer(GraphEditPart.class,
              GraphEditPart3D.class));

    // modify diagram edit part's policies 
   borgFactory.addAssimilator(new AbstractPolicyModifier() {

           public boolean match(EditPart part) {
                     return part instanceof GraphEditPart3D;
           }

           public void modifyPolicies(EditPart io_editpart) {
                        // feedback when creating a node:
                 io_editpart.installEditPolicy(
                            ShowLayoutFeedbackEditPolicy3D.ROLE,
                              new ShowLayoutFeedbackEditPolicy3D());
                    // handles and feedback when moving or resizing a node
                    io_editpart.installEditPolicy(
                            Handles3DEditPolicy.CHILD_DECORATOR,
                              new Handles3DEditPolicy());
               }
 });

 // modify node edit part's policies
       borgFactory.addAssimilator(new IAssimilator.InstanceOf(
           NodeEditPart.class) {

               public EditPart assimilate(EditPart io_editpart) {
                        // feedback when drawing a connection
                     io_editpart.installEditPolicy(
                            ShowSourceFeedback3DEditPolicy.ROLE,
                              new ShowSourceFeedback3DEditPolicy());
                    return io_editpart;
               }

   });
       return borgFactory;
}

The code is almost self-explanatory. First, the original edit
part factory is nested inside the BorgEditPartFactory. Then
appropriate assimilators are registered at the borg factory. The
first assimilator is the EditPartReplacer, which replaces the 2D
edit part by a 3D edit part. In the example, edit parts of type
GraphEditPart are to be replaced by their 3D counterparts,
GraphEditPart3D. The second assimilator, an
AbstractPolicyModifier, does not change the edit part
itself, but it adds two new policies to the 2D edit part:
ShowLayoutFeedbackEditPolicy3D creates a 3D feedback
figure when a node is to be created, and
Handles3DEditPolicy takes care of adding 3D handles when
nodes are to be resized or moved. The third assimilator also adds a
new policy: ShowSourceFeedback3DEditPolicy creates a 3D
feedback figure when a new connection is to be created.

Last but not least, the BorgEditPartFactory must be set
as factory, and a RootEditPart3D is to be installed which
is required in the 3D editor:

protected void configureGraphicalViewer() {
        super.configureGraphicalViewer();
 EditPartFactory originalFactory = 
                getGraphicalViewer().getEditPartFactory();
        BorgEditPartFactory borgFactory = createBorgFactory(originalFactory);
     getGraphicalViewer().setEditPartFactory(borgFactory);

       ScalableFreeformRootEditPart root =     new ScalableFreeformRootEditPart3D();
     getGraphicalViewer().setRootEditPart(root);
}

That’s it! These are the changes required to 3D-fy an existing
editor. We have to register the new 3D editor using the well-known
extension point org.eclipse.ui.editors, then we are able
to open, view and edit the very same diagrams in 3D as we can edit
with the 2D editor.

Conclusion

As demonstrated with the small example, creating 3D editors with
GEF3D is surprisingly simple. At the moment, no official release of
GEF3D is available, more about that and current restrictions can be
found in the box “Incubator”. By 3D-fying exiting 2D editors, it is
a good way for taking first steps into the world of 3D programming.
Although, GEF3D supports full 3D editors, especially when
predefined elements (such as diagram elements) and connections
between them are to be edited. If you need to create your own CAD
system, GEF3D is not recommended. However, the design and future
development of GEF3D is focused on 3D-fying existing 2D editors
since it is an interesting technique for visualizing models (and
model chains) in the area of model driven approaches.

In the planned second article, we will explain more features of
GEF3D, e.g., how to combine multiple editors in a single 3D scene,
and how to 3D-fy GMF-based editors.

Incubator

At the moment, GEF3D is in the Eclipse incubator. There is no
official release of GEF3D available yet (on how to install GEF3D at
the moment, see box “Installation”). Known problems are documented
in bugzilla. Currently, the most important restriction is the
absence of 3D handlers for moving a 3D figure in 3D space (e.g.,
for rotating a 3D figure). Rotation and positioning of figures in
3D space is fully supported, it is simply not possible to do that
interactively at the moment.

Besides, the incubator does not only mean that the software is
not stable yet, but also that the project and the team has to
become more mature. At the moment, there are only two committers,
and help is highly appreciated! GEF3D provides a website with a
wiki, questions are answered at the newsgroup or developer mailing
list [2].

Installation of GEF3D

The official release of GEF3D is not yet available. In order to
use GEF3D, Eclipse (and GEF) 3.5 are required. GEF3D has to be
imported into the workspace from its subversion repository (

http://dev.eclipse.org/svnroot/technology/org.eclipse.gef3d/trunk).

The following projects are to be imported:

  • org.eclipse.draw3d
  • org.eclipse.draw3d.geometry
  • org.eclipse.draw3d.graphics3d
  • org.eclipse.draw3d.lwjgl
  • org.eclipse.draw3d.ui
  • org.eclipse.gef3d
  • org.eclipse.gef3d.ext

All examples are found under
org.eclipse.gef3d.examples.*, for 3D-fying GMF based
editors, some helper classes are provided in
org.eclipse.gef3d.gmf.

Besides the GEF3D (and Draw3D) code, a rendering library is
required in order to actually draw the 3D scene. For creating 3D
output on the screen, OpenGL is used. Since OpenGL only provides a
C API, a Java wrapper library such as LWJGL [10] is required. At
the LWJGL update site (http://lwjgl.org/update), an Eclipse plugin can be
downloaded, which was created in the context of GEF3D.

An up-to-date installation instruction can be found in the
GEF3D-wiki (http://wiki.eclipse.org/GEF3D)

Links & Literature

[1] Voormann, H.: Quo vadis Eclipse? Eclipse Magazin 4 (2008),
April, S. 20–27

[2] Eclipse GEF3D: www.eclipse.org/gef3d

[3] Jacobson, I.: Object-Oriented Software Engineering: A Use
Case Driven Approach. Addison-Wesley Professional, 1992 (acm
press)

[4] Eclipse GEF: www.eclipse.org/gef

[5] Eclipse GMF: www.eclipse.org/gmf

[6] Eclipse Modeling Framework Technology (EMFT): www.eclipse.org/modeling/emft

[7] Eclipse Model Development Tools (MDT): www.eclipse.org/modeling/mdt

[8] Eclipse EMF: www.eclipse.org/modeling/emf

[9] Bokowski, B. ; Gerhardt, F.: GEF: Maßgeschneiderte grafische
Editoren selbst erstellen. In: Eclipse Magazin 2 (2005), Februar,
S. 85–90

[10] Lighweight Java Game Library (LWJGL): www.lwjgl.org

[11] Java Bindings for OpenGL API (JOGL): jogl.dev.java.net

Author
Jens von Pilgrim
Jens von Pilgrim is a research assistant at the Software Engineering Group at FernUniversit
Comments
comments powered by Disqus