Creating the IDE 2.0
Eclipse Juno: Introduction to Eclipse Code Recommenders 1.0
Code Recommenders is an extension to Eclipse’s Java Development Tools that analyzes code of existing applications, extracts common patterns of how other developers used and extended certain APIs before, and re-integrates this knowledge back into your IDE.
Developers and CTOs know: frequently changing teams and the “liberal” use of the latest and greatest technologies make your burndown charts and project cost estimations burst just within a few weeks. To compensate the brain-drain and to lower the entrance barrier, team leads plan a fair amount of their team’s time for documenting software, doing reviews, and working together in pairs in front of one screen to evenly spread the knowledge about how to use APIs in the entire team – and newcomers in particular.
If done right, this is the best a team lead can do to get the maximum performance out of his team in the long run. But Code Recommenders thinks there is more a team lead can do...
Leveraging the hidden gems in your source code
API designers typically have some expectations as to how developers should use their APIs, i.e. they expect their clients to call certain methods at certain points in time and in a particular order etc. The challenge for every newcomer is to learn about these implicit expectations and API usage rules fast enough to be of help to the team they joined.
Sure, the API documentation may contain the necessary information. Somewhere. But between ourselves:how often do you read the API documentation of, say, JButton with its 381 methods to figure out how to use it? Don’t you rather use Google to find a code snippet that does what you need? Or if Google can’t help because you are programming against an in-house library:don’t you prefer looking at your existing code base to see how your colleagues successfully used that API before? And sometimes you just scroll through the code completion proposal pop-up to see which proposals may sound best for what you are trying to achieve, right?
There is nothing bad about it. It’s just horribly ineffective and costly. And apparently API documentation as it is today is only of limited use for developers.
This is where Code Recommenders comes in. Code Recommenders is an extension to Eclipse’s Java Development Tools that analyzes code of existing applications, extracts common patterns of how other developers used and extended certain APIs before, and re-integrates this knowledge back into your IDE in form of (i) intelligent code completion, (ii) extended API documentation, (iii) sophisticated example code search, and even (iv) bug detection tools – all powered by the implicit knowledge of the programming masses. If you like, you can think of Code Recommenders as bringing the idea of Web 2.0 into your IDE – or as we call it sometimes: Code Recommenders is about creating The IDE 2.0.
The remainder of this article will give a short overview over Code Recommender’s completion engines coming up in Eclipse Juno – as part of the Eclipse Java Developer Package, the Eclipse RCP packaging, or – if you start with any other package – can be installed from the Juno Release Train update site.
Recommenders' Code Completion Engines
Code Recommenders 1.0 adds five new code completion engines to Eclipse:
1. Intelligent Call Completion,
2. Intelligent Code Snippet Completion,
3. Intelligent Overrides Completion,
4. Call Chain Completion,
5. Subwords Completion.
Intelligent Call Completion
The intelligent call completion engine probably illustrates the idea of Code Recommenders best. When dealing with Framework APIs, developers frequently have to deal with complex APIs. For illustration, consider the public API of javax.swing.JButton which consists of 381 (!) public methods (Figure 1). A huge API of which a developer typically only has to know a small subset. The remaining, say, 360 methods unnecessarily bloat the API (from a API user’s viewpoint) and thus increase the complexity and burden of learning and using this API.
Fig. 1: All potential completions for JButton
This is where Code Recommenders’ Intelligent Call Completion comes in. It assists the developer by recommending only those methods that are actually relevant for the task at hand. For instance, given that a developer just created a text widget, Code Recommenders makes it obvious which methods a developer should want to use next – even if the developer doesn’t know it themselves (Figure 2).
Fig. 2: Recommenders’ Intelligent Completion on SWT Text after calling new Text()
At the time of writing this article (1.0.0.rc2), Code Recommenders partially supports the Java Standard Library, namely the main packages under java.* and some packages under javax.*. As the recommendation models are generated from the Eclipse Juno Release Train code base only, packages like java.awt or javax.swing are not supported, as no data was available at generation time. For a detailed list of which libraries or packages are supported, check the reports section on the Code Recommenders homepage.
Intelligent Code Templates (single-object, no-order)
Code templates are helpful when code is needed to iterate over an array of objects or when creating a getter for a property of a class. But code templates really shine when developers have to use APIs they are not familiar with. Code templates then serve as additional documentation that quickly shows how to use an API, and thus can save developers a lot of time that would otherwise be needed for reading API documentation.
Eclipse maintains more than 70 of such Java code templates ranging from simple loops up to complex API usage patterns like creating an SWT Button or Composite. Unfortunately, developing templates for API usage patterns is an extremely costly and tedious job and consequently only few templates on how to use complex APIs of, say, JFace, Eclipse UI or even the Java Standard Library exist.
This is where Code Recommenders comes in again. In the previous section we showed how to recommend single methods to invoke on an object. Code Recommenders’ Templates Completion takes this to the next level by recommending not only single methods but complete sets of methods (Figure 3).
Fig. 3: Intelligent Template Completion on JDT ASTParser
As you probably have noticed in the example above, Recommenders' templates completion can be applied on existing variables to complete existing usages but also works on type names as the example in Figure 4 shows. The final result after applying the template then looks as displayed in Figure 5.
Fig. 4: Intelligent Template Completion on JFace TableViewer
Fig. 5: Resulting code snippet for JFace TableViewer
The generated template proposals do not reflect method execution ordering constraints, i.e. the order of the proposed method may have to be changed manually after insertion.
Intelligent Overrides Completion
Similar to recommending method calls, one can also recommend which methods a developer should typically override. This is what Recommenders’ Intelligent Overrides Completion (Figure 6) does. There is much more to say about how classes can be extended, but we’ll save this for another article about Code Recommenders’ Extended Documentation Platform – a platform that extracts valuable (extension) patterns in code and enriches existing API documentation with these patterns.
Fig. 6: Recommenders’ Intelligent Overrides Completion on JFace Dialogue
Another noteworthy extension engine is Code Recommenders Subwords Completion.
As an experienced Eclipse user you probably know JDT’s CamelCaseCompletion. This engine is nice but requires you to remember the exact uppercase letters of the completion proposal you want to insert.
Subwords makes this more convenient. The idea is simple enough: You do not have to type a name from the beginning to find a match in the content assist pop-up. It helps if a developer does not know if one has to “find” – or – “get” an element.
Fig. 7: Recommenders’ Subwords Completion on JDT's CompilationUnit
And it’s even sophisticated enough to understand a rough shorthand e. g. dclr for declaration or combinations of words such as 'ty + dclr' , which finds all proposals containing the words 'type' + 'declarations' (fig. 8 and 9).
Fig. 8: Recommenders’ Subwords Completion Completion on JDT's AST
Fig. 9: Recommenders’ Subwords Completion Completion on JDT's AST
Note: Subwords does not fall into the group of intelligent completion engines, i.e. it does not need any training data and thus works out of the box with any framework or API.
The last engine I’d like to introduce is Recommenders’ Chain Completion.
Sometimes you need to access objects that can be reached by invoking several method calls in a row – so-called call chains. Usually you have to find these call chains yourself by traversing the API call graph manually, and evaluating whether each potential chain may return an instance of the required type.
Code Recommenders’ call chain completion automates this for you. It quickly traverses the whole API call graph and finds all possible paths through the API that may return an appropriate object (Fig. 10).
Fig. 10: Recommenders’ Chain Completion on IStatusLineManager in a ViewPart
What’s coming next?
The Code Recommenders completion engines we introduced in this article are just a teaser. There are many more exciting features coming up, e. g. personalized code search engines, template completion engines that find complex multi-object usage patterns in code, stacktrace search engines, and many more tools . Keep an eye on this project which has exciting things in stock. Promised.
Marcel Bruch is project lead of the Eclipse Code Recommenders project, Eclipse trainer at vogella.com, and researcher at Darmstadt University of Technology. He aims to change the way how software is developed by leveraging the Big (Software Development) Data to make IDEs a better place.
This article appeared in Java Tech Journal: Eclipse Juno. Check out more of that issue and others here.