My Indigo Year: A Review
Holger Voorman gives an overview of 2011’s Simultaneous Release.
In my ‘The Helios Year in Retrospect’ article for last year’s Java Tech Journal (Issue #1 on Eclipse Helios), I concluded that the new Java owner, Oracle, should create a reliable release schedule like the one commonly used at Eclipse. But things turned out differently: according to current plans, Java 7 will be completed just after the Indigo release, in late July. While NetBeans 7.0 and IntelliJ IDEA 10.5 have already achieved Java 7 support, the most popular Java IDE hasn’t shipped with Java 7.0 support, with the current version 3.7. Was sticking to the set release date the right thing to do, or would it have been wiser to make an exception, and pay attention to the saying “it’s done when it’s done”?
Many companies and open source projects act according to this motto. The best known example is the first-person shooter “Duke Nukem Forever”: announced in 1997, the release was moved back countless times. The game has now been available since the 10th of June, twelve days before Indigo was released. What would have happened if Indigo had not been delivered until all of the originally planned functions were complete, instead of trying to develop the maximum number of features in the time available, and sticking to the fixed delivery date in mid/late June, like in previous years? While Eclipse could advertise itself as having “Full Java 7 support,” all the other new features of the Java IDE and many other projects would have had to wait.
Together with the Eclipse platform, Java Development Tools (JDT) is still Eclipse’s flagship project, but it’s just one ship of many. This move would have been especially unwelcome for those who won’t use Java 7 in the near future. It also would have been a disadvantage for Eclipse-based products and projects, as their schedules would have become confused. Luckily, Indigo resisted the temptation and their eighth Siimultaneous Release (if Eclipse 3.0 and 3.1 are taken into account) was shipped on schedule, on 22nd June.
On the Indigo website 62 projects, or project teams, are listed as being involved in the release. In comparison, last year´s Helios release contained 39. Take a closer look and you´ll see that the way of counting has changed. Some sub-projects are now counted that have previously not been counted individually, but were considered as part of their parent project. Irrespective of the method of counting, there are two more projects in Indigo than in Helios: eleven projects joined, and nine left. This is, in terms of percentages, is the lowest growth of all the Simultaneous Releases (Fig. 1).
Whether this means that Indigo has already reached the saturation point, or whether this is just a temporary effect of the financial crisis and the subsequent reduction of the open source commitment of many companies, only time will tell. 408 of approximately 1,000 Eclipse committers (that’s 24 less than the Helios committers), who are the only ones allowed to change the source code, created Indigo. Indirectly, some 50 companies were involved, as these were where the 408 committers were recruited from. This does not include the numerous contributions that were submitted through the bug tracking system, which found their way through the committers, and into the code.
Unfortunately, the Test and Performance Tools Platform Project (TPTP) is no longer on board. TPTP was, together with C/C++ Development Tooling (CDT) and the Java IDE, one of the three very first projects in the Eclipse Simultaneous Release in 28th June, 2004. In the absence of a name, the second release was named after the Eclipse IDE version. That was 3.0, the first version, based on OSGi. Since February 2010, TPTP is now history; the development was stopped and the project has been archived. The Runtime Analysis Tools (RAT) is now set to be its successor. In March, the existing Eclipse project was made possible thanks to a donation from Google, who had come to possess the Java Profiler CodePro through the acquisition of the U.S. company Instantiations. RAT, which helps to locate where most time is wasted, is complemented by MAT, the Memory Analyzer, which is in its third year, and evaluates the Java heap to track down memory leaks and to optimize memory consumption. Another tool that fits would be JaCoCo, a project unfortunately not (yet) part of Eclipse, from the makers of the test coverage report plug-ins EclEmma. The new, developed-from-scratch EclEmm replacement is based on the technology recording run-time data; the so-called instrumentalization of Java classes. The other projects that are absent in Indigo weren’t a flash in the pan either, and had been with Eclipse for at least two years, since Galileo. The top-level project Device Software Development Platform (DSDP) was dissolved: the project’s target management can be found partly in CDT; the rest was formed into an independent project, just like Sequoyah, which is aimed at mobile devices with the Tools for Mobile Linux and Mobile Tools for Java. The other seven projects are still in their original forms; they just don´t take part in the Simultaneous Release. Their health can be judged, among other things, by looking at the number of code changes in the last few months. EMF Teneo (a connection of the Eclipse Modeling Framework to relational databases) and GMF tooling (the only one of four sub-projects of the Graphical Modeling Framework project, which is not at Indigo) look quite healthy. In the Java Emitter Templates (JET) project, which generates code and other text from an EMF model, there are only a few changes. But, because JET is mature and there are a number of alternatives, you have nothing to fear. Also vital, but increasingly exposed to internal competition, is Buckminster. In particular, Tycho which is based on Maven and which has been an Eclipse project since August 2010 is becoming more popular in plug-in building than it. The two SOA projects Swordfish (an extensible SOA Framework) and BPMN Modeler (an editor for so-called business-process modeling notation diagrams,) as well as Mint (JDT extension for model-driven software development) are not looking good, and it’s likely they will be archived soon.
Considering the size and range of projects, the growth of Indigo is bigger than the addition of two new projects suggests. Also encouraging is that some projects that have not been with Eclipse very long, have found their way into Indigo. One of the more major projects which migrated is Object Teams, a language extension for Java. Similar to aspect-oriented programming, Object Teams adds object orientation in order to get to grips with complexity. Refreshingly different and closer to the real world, various aspects are implemented through roles and teams on a higher level of abstraction. Object Teams is based on JDT, and it will provide the necessary tools for editing source code and generating executable bytecode directly. Object Teams was created in collaboration with the Fraunhofer Institute for Computer Architecture and Software Technology, and the University of Berlin. Since January 2010, it has been an Eclipse project, and because during incubation the version number may not be larger than 1.0, the version number 0.7 in Eclipse followed an external release of version 1.4. A few milestones of version 0.8 followed, and finally came the great leap in Indigo, to version 2.0. WindowBuilder which can be used to graphically edit surfaces is similar to CodePro: it’s also a donation by Google, which goes back to the purchase of Instantiations (Fig. 2). And there’s another parallel: just as CodePro (alias Runtime Analysis tools) now fills the gap left by TPTP, the better equipped WindowBuilder suppresses the Visual Editor. For the Visual Editor, the last Simultaneous Release came after several partially successful attempts to revive it in June of this year (Visual Editor was one of seven projects to take part in the second Simultaneous Release in 2005, and in 2006 it was one of the ten Callisto projects). In contrast to NetBeans’ Swing GUI Builder, also known as Matisse, WindowBuilder and Visual Editor operate directly on the source code: changes can be performed in the code, and existing code can later be edited graphically, if the code is not too complicated. Its commercial past explains why this project is supported in addition to SWT Swing. Support for the Google Web Toolkit (GWT) – which is probably one reason for the takeover – will be continued outside of Eclipse by Google itself, as GWT Designer.
Also fairly new to Eclipse and anything but a lightweight is Jubula, a project for creating and running automated tests of graphical user interfaces. Jubula was introduced to Eclipse by the German BREDEX company, and earlier sold as GUIdancer. Therefore, it is not surprising that, just like WindowBuilder, not only is SWT supported, but Swing and Web applications are also supported. Jubula is aimed not at software developers but at test engineers, unlike SWTBot, a Java framework for surface testing. The new download package “Eclipse for Testers” is also aimed at them.
With four new projects in the main project Eclipse Modeling Framework (EMF) it is difficult to keep track. The Agent Modeling Platform (AMP) provides tools primarily for scientific research, for example swarm behaviour or epidemics. The behaviour of an individual agent is modelled as a funcTion of adjacent agents, in order to simulate a variety of such agents in a system and display the results, both graphically and in other formats (Fig. 3). AMP and the Eclipse project generation Factories (EGF) have the 22th of April 2009 in common: this is the day on which the project proposals were approved. While AMP only uses EMF for its own needs, EGF extends EMF to improve the tooling for orchestrating the generation of large, complex and customizable models. EMF Facet starts working after the generation. Facet extends existing models without changing them, similar to aspect-oriented programming, which extends object-oriented programming. This is to ensure that independently developed extensions add more facets to a model, without interfering with each other.
The fourth EMF Project, and Indigo newcomer, is Graphiti, which is located in the new sub-project Graphical Modeling Project (GMP). Using Graphiti, one can create graphical editors, not only for EMF models without much knowledge of Draw2d and the Graphical Editing Framework (GEF). Even a version for Flash is being considered.
Scout has nothing at all to do with EMF. Scout was developed by the Swiss company, BSI AG, and in Eclipse it now co-exists with Riena as a platform for business applications. According to the company, Scout is used by more than 30 customers. The client exists as a SWT and Swing version, and a Web interface is planned. The development of Gyrex is driven solely by the German company AGETO, by Gunnar Wagenknecht, who is not unknown at Eclipse. Gyrex was originally Cloud-free, but was renamed in early 2009 to avoid associations with cloud computing. Gyrex is a server-side Equinox, to connect multiple servers to form a cluster, not to be confused with Virgo, the Enterprise Java Web Application Server that is not a part of Indigo. Unfortunately, M2E : (“m” stands for Maven, “2” for “to”, because it brings the build system for “e” Eclipse: it integrates Maven into Eclipse) is only a one-company project. It contains an editor for pom.xml files and the integration of Maven repositories. The company is none other than the inventor of Maven Sonatype, from the USA.
The growth of sub-projects by WTP and Mylyn, together with projects such as ObjectTeams, WindowBuilder, Jubula and Scout, which were already mature when they joined Eclipse and Indigo, all explains the enormous increase in the amount of code. But this amount of code needs to be maintained. I especially like the fact that Indigo gained support in the core competencies of Eclipse, the support for Java development – not only for SWT, but increasingly also for Swing and the web.
In addition to the eleven new projects, the other 28 – or 51, depending on how you count the Indigo projects – have been improved as well. For example, the Marketplace Client, where the installation of plug-ins has been made even easier. If you’ve found something on the web and the plug-in vendor offers a special text link or a specially linked image, you can just drag and drop it onto the title bar of the currently running Eclipse application, whereupon the Marketplace dialog will open, and the installation only needs to be approved, to install the plug-in (Fig. 5).
Since it is not working with the update site link, now only the plug-in vendors can add this special link to their websites. Since the Marketplace is established as a catalog of plug-ins and services, and since most installations are done and recorded by it (on 20th May, the 500 000th installation was counted) Marketplace provides us with a lot of exciting numbers. For example, the ranking of the number of installations in the version control systems shows “SVN ahead of Git, ahead of Mercurial” in the code analysis tools “FindBugs ahead of EclEmma, ahead of Checkstyle”.
Even if SVN is still ahead, distributed version control systems, and in particular Git, play an increasingly important role. The switch to Git does require some rethinking on the part of long-term CVS/SVN users, but rewards them with high speed and with merging that is simpler than they ever thought possible. As those with an eagle eye will have observed, EGit is certainly a highlight of Indigo. The Eclipse Git support has matured to version 1.0 in Indigo, and is ready for production use (Fig. 6). Also, for some time now NetBeans 7.0 and IntelliJ IDEA have offered Git support. IDEA, however, requires the installation of the command line version of Git for the specific operating system; and NetBeans, who advertise it in their 7.0 video, employ shamelessly, but legally, JGit, the Java implementation of Git. These, and other use options, are allowed by the strict separation in JGit, without depending on Eclipse and EGit, the Eclipse integration.
Xtext has made the jump from 1.0 to 2.0 and brings the first refactoring feature in the form of the renaming of identifiers. The version of CDT  jumped from 7.0 to 8.0, and with Codan it offers improved static code analysis. The version number of the PHP Development Tools (PDT) reached 3.0, just like the version number of the Dynamic Language Toolkit (DLTK), which is based on PDT. Unfortunately, there is no longer a download package available from Eclipse for PHP developers. Due to incompatible licenses, it was decided to offer packages with the debugger as an external download.
Often, it’s the little things that make life pleasant. For example, in the JDT debugger a dialog now prevents the accidental deletion of conditional breakpoints. That’s already happened to me several times, because I like to use the conditional breakpoints for debugging or tracing purposes, by using System.out.println () in the condition. Another new feature is the ability to reuse a condition of a previous input, making it easy to move or restore. The help system suits my bumbling, now that a set Scope is displayed directly above the search results. Previously, the Scope was easy to miss, and you wondered about too few hits. For those who are interested in even more highlights of Indigo, the Indigo series of articles on JAXenter and the traditional top-ten countdown by Ian Bull is recommended.
In addition to Indigo
Only about a quarter of all Eclipse projects made the effort to meet the requirements of the Simultaneous Release. The total number of Eclipse projects has increased by nine projects, which corresponds to the proportional growth of the Helios / Indigo projects. There are 23 new projects, and 14 terminated projects, some of which have been mentioned already. To list all the new and terminated projects would spoil this paper. If you wish to know more, you should keep an eye on the new project proposals, and the creation or termination reviews at the project overview page and on the side of the recent reviews.
Code Recommenders, one of these 23 new projects has, in my opinion, the potential to be the next Mylyn. Like Mylyn, Code Recommenders helps the developer to focus on the essentials, by highlighting certain things and hiding others. This information is gathered by Mylyn, from observing which files are opened and which areas are edited. So, what’s new in Indigo? When a new task is started, the information from a possibly existing Java stack trace can be obtained. Code Recommenders analyses how a given framework is used, for example, whether methods are often, rarely or never used. The most commonly used methods appear in a list above for code completion (Fig. 7). I once had a button that could be operated with the mouse, but not with the keyboard. As it turned out, the author had registered, by mistake or ignorance, a wrong listener. Using Code Recommenders that would not have happened.
Eclipse Labs, which launched in May 2010 and was largely advocated for the not-yet-Eclipse projects, has hardly been heard of. The five most active Labs projects are listed at the very bottom right-hand corner on Eclipse.org, but that’s all. Eclipse is quite keen on Git, and since Eclipse Labs does not support Git, it is not in the picture. But, perhaps it has only has been waiting for JGit 1.0. Eclipse has also been silent about e4, the next Eclipse platform; even though Eclipse Platform 4.1 brings many new and exciting technologies, such as Dependency Injection. The nice thing is that it is not really a new platform, but a collection of several independent technologies that can be used separately and in combination with Eclipse 3.x.
What else happened
The Indigo year got off to a bumpy start. Shortly after the release of Helios, in the Java 6 update 21, Oracle replaced the manufacturer name “Sun” with its own. This small, inconspicuous change caused Eclipse in Windows to sporadically crash, because the launcher using the manufacturer’s declaration no longer recognized the Hotspot VM, and did not increase the PermGen memory space, as is necessary for the stable operation of Eclipse. Three weeks later, Oracle updated the undo, and solved the problem. In August, Oracle dropped a bombshell and sued Google for alleged patent infringement by Android. Whether justified, or a move made out of greed, it was also hotly debated in the Eclipse community.
Nevertheless, more than 90 percent of the developers of Hudson, a popular continuous integration server, voted in favour of a move to GitHub under the new name Jenkins. The story took a bizarre turn when a little later, in February, the original Hudson project also moved to GitHub, and then, in May, Oracle transferred Hudson to Eclipse. Many of the Eclipse community sympathized with Jenkins and wanted the proposal to be rejected. But, even if Hudson is still not a true Eclipse project, there are only formal hurdles in the way now. In an oddly parallel move, Oracle donated its open source office package, OpenOffice.org to the Apache Foundation. Oracle’s open source support is commendable, even if it was reduced in certain areas, but an improved communication with the community would be desirable – especially before making decisions.
Not only outside, but also within the Foundation, sometimes debates flare up. Following discussions on the use of the Friends of Eclipse program to attract donations in May, the Hudson project proposal once again sparked a fundamental debate over the interplay between industry and individuals, through licenses in general and the (supposed) requirements made by the Eclipse development process. A key element of the Eclipse development process is to ensure Intellectual Property (IP), thus the Foundation requires that all code conforms to the Eclipse license, and that no intellectual property is violated. Eclipse Bugzilla is currently serving as a control gate, through which each code contributed from outside – and therefore not written by Eclipse committers themselves – must travel. A contribution will be added as a patch to a Bugzilla entry, which is only possible after prior authentication, whereby the contributor agrees to the use of their contributions under the Eclipse license. Distributed systems such as Git version offer more convenient options than using a patch, when it comes to accepting proposed changes – especially if you wish to change a code base that has been further developed. This is one of the challenges that affects the Eclipse process, and is one of the reasons for ongoing adjustments at Eclipse.
Slimming did not just take place at the process level, but also with regard to staff. For four and a half years, Lynn Gayowski was the friendly face of the Foundation. Lynn was also behind a number of jokes, for example imitating the dance style of the Eclipse Europa Representative Ralph Müller in a YouTube Video in a response to a video recording of the Eclipse Summit Europe. The gap that resulted from her move to Klocwork in February, was unfortunately not filled, probably because, due to the financial crisis, funding at Eclipse is in short supply (in 2011, the revenue is expected to be under four million dollars; the lowest of the last five years). Therefore, there are only two admins that must keep the increasingly complex computing infrastructure alive. In the weeks before the release, they underwent hard test cases by failure of devices.
As an extension to the Eclipse development process, there is an initiative, which has previously received little attention: planned long- term support which should ensure care and maintenance beyond the two service releases. I personally would rather see these resources be invested in development, rather than in the preservation of the obsolete. If you do not use any internal interfaces in a framework, then it does not rely on an outdated version – and it can easily and safely be replaced by the current version. Even after eight years since Eclipse 3.0 was developed and programmed, plug-ins still run, for example, on 3.7. If you have concerns about the continued existence of a project, you can avoid this through active participation. More information regarding the long- term support is unlikely to be announced before the appropriate workshop on 11th July.
It’s quite possible that, with the first of the two Indigo service releases due this September and in February of next year, beside bug fixes, Java 7 support will also be included. The only thing that will change is that the annoying download and compiling of the existing, but not yet released JDT plug-in, will be eliminated. Actually, not many developers will really be using Java 7 straight away. Experience shows that adoption takes quite a while. A year passes quickly, and Juno will be released in June 2012. Unlike Java, Eclipse users move to current versions much more quickly. In combination with the annual, frequent releases, the wait for an innovation to arrive at the customer – the so-called Time to Market – is much shorter than in Java. Not just for Java, but for all the applications and frameworks I use, I would like to see a dependable release schedule, following the example of Eclipse. The Java one should additionally be coordinated with the Simultaneous Release of Eclipse. But I’ve already requested that last time and then things turned out differently.