The Java puzzle

Java 9 delays: Why Jigsaw isn’t ready yet

Uwe Schindler
Delay image via Shutterstock

The Java 9 release postponement may come as little surprise to the Java community given the JDK’s track record of tardiness. But what exactly is causing the hold-up? And does this mean more time for more language features?

On Tuesday, December 1st, 2015, in the OpenJDK mailing list, Mark Reinhold confirmed what many had expected: the release of Java 9 was postponed by six months. It’s not official yet – but the schedule will be adopted if the community doesn’t raise objections by next week.

According to Mark Reinhold, this decision was made due because the key Java 9 feature, the module system Jigsaw, is simply not ready for release yet. But why so? Work on this system has been going on for years now – it was even supposed to be released with Java 8!

Jigsaw’s status quo

Let’s take a look at the current status quo. The feature freeze milestone for Java 9 is set to be on December 10th, which will be the last day for adding new features. However, Jigsaw modularization has not been added to the main code branch yet. To test the new module system, you can’t just download the official Java 9 preview builds, you have to choose a separate “Jigsaw” variety. The author wasn’t able to find the reason for this to date.

One of the reasons for separating the download locations might be that many popular open-source projects don’t run with the Jigsaw version of Java 9. Until recently, that included the popular JVM language Groovy. The reasons for this are similar to those involved in the disputed sun.misc.Unsafe: an additional – and very welcome – security layer will be added to the JDK by the module system.

It’s been fairly easy to make hidden classes and methods visible in JDK via reflection (if the appropriate rights are granted by the Java Security Manager, as is the case most of the time for server-side applications). Using the reflective method setAccessible() to make private methods and classes visible will be penalized by a new InaccessibleObjectException if the code in question is not whitelisted (which includes the disputed sun.misc.Unsafe). So this workaround doesn’t cut it anymore. Java 9 prohibits this to this date – with and without security manager. The reason for this is those signatures often work with private interfaces instead of public ones, which means that in the module system the interface would have to be public when making it accessible. When you look for the typical programming patterns about this on GitHub, like the well known setAccessible(), you can already tell which code won’t run in Java 9. Unfortunately, this affects a huge amount of code, even renowned libraries.

Lucene, Solr, Elasticsearch

For Apache Lucene, Apache Solr and Elasticsearch, developers already found remedy by simply prohibiting all use of setAccessible in their source codes. But looking further into the problem, we quickly see this isn’t really necessary most of the time. The only accessible thing left in Apache Lucene is sun.misc.Cleaner in MMapDirectory. But this is whitelisted by Java 9 anyway, since it’s a known issue.

In any case, this might be one of the issues to be fixed until the release of Java 9, if more time is granted. So we can assume that the Jigsaw team also needs more time to solve puzzles like these and to at least make the most commonly used components and libraries.

Module file format

Going further into detail, you’ll come across several other unsolved issues around Jigsaw, meaning the module system isn’t ready to be tested by the public yet. One of the current issues is the file format for modules. There’s no decision here yet, but plenty of ongoing discussion. To complicate things even further, two formats are currently in existence, JIMAGE and JMOD.

The first is a truly new format, offering fast access to the classes inside it without having to execute the usual slow search through ZIP Entry Listing. This new format is used within the JDK to speed up the loading of classes. But there’s still the other new Jigsaw-JMOD format which is also based on ZIP (as is the JAR-Format), but offers to include platform-specific libraries like DLL or SO files in addition to the module-specific metadata. Jigsaw builds are shipping all modules in both formats, while regular Java 9 builds just support the faster JIMAGE. What exactly the final layout of JDK-installations will look like is not yet sure. The new JMOD-Format is supposed to replace the JAR format, so you can deliver whole Java applications as a module containing all metadata on dependencies and exported APIs. This means it will also be possible to deliver a slim version of JRE with an application that only contains the necessary modules.

SEE ALSO: The features Jigsaw brings to Java 9 

Due to the delay in development, postponing the release-date seems to be the sensible thing to do for another reason. By the time of the release, the new Java 9 module system needs to be usable in existing development environments like Eclipse. And because things are still changing on that side as well, the developers of those tools should also be granted enough time to prepare. There are many debates on the OpenJDK Jigsaw mailing list about how developers can build new modules with Apache Maven themselves. The community is already busy providing suitable Maven Mojos for JMOD packaging. The latest Gradle release (2.9) already comes with the infrastructure for Java 9 modules too, even if it can’t be used yet.

More time for Java 9 – more time for new features

Overall, we can already see many projects preparing their codebase for Java 9. Apache Lucene is expected to feature full support of Jigsaw by version 5.4 and will be released within the next weeks. And Elasticsearch is already running smoothly with the module system. The runtime permissions required by Apache Lucene are minimal, so there are no issues to be expected on Java 9.

As a member of the Apache Lucene Team I’m really looking forward to the new tools and features enabled by the prolonged development period. Personally, I’d like to see a feature around the MethodHandles from Java 8: Direct Converting of method references (as String::length) in source code to MethodHandles by the Java Compiler (including the related static type testing).

Uwe Schindler
Uwe Schindler is a member of the Project Management Committee at the Apache Lucene Project. He and his Consulting firm SD DataSolutions GmbH are based in Bremen. He regularly blogs at The Generic Policeman's Blog.

comments powered by Disqus