Java 7 Causes Headaches for Lucene and Solr Users
Java 7 has arrived but not without its share of controversy….
Java 7 is finally here; but this release hasn’t been without controversy! Shortly after the release, Apache Lucene committer Uwe Schindler posted a warning that Java 7 contains hotspot compiler optimisations, which miscompile some loops. He warned that this bug could affect the code of “several” Apache projects – but that currently, it is known to affect all released versions of Lucene Core and Apache Solr. The bug isn’t just restricted to Java 7 users: Java 6 users are also warned that the problem can impact them, if they use one of the JVM options that are not enabled by default:
These bugs were discovered five days before Java 7 was published, and there has been some debate as to whether Oracle had the time to fix them. At his blog, Uwe Schindler questions the lack of updates that found their way into the code, between the preview release and the final release of Java 7. He states that users can verify that the final JDK release is the same as the June 27th preview, by checking the timestamp on the signature of the Windows Installer EXE file.
“So Oracle ignored all bugs (not only ours) in the preview release and simply released a one month old package! So what was the sense of the preview release? They could have released it one month before! It was for sure not intended for public review and bug hunting!” he says.
However, others have argued that the problem was discovered far too close to the release date, for Oracle to address the issue:
“These problems were detected only 5 days before the official Java 7 release, so Oracle had no time to fix those bugs,” says Markus Eisele, Oracle ACE Director, who also argues that “the source code is open so anyone stubborn enough to dig into it can make a fix.”
The severity of the problem has also been cause for debate. Cay Horstmann, author of Core Java and Enterprise Java for Elvis, posted that although he is unworried about Hotspot problems which could cause crashes, he is worried about Java 7 silently producing the wrong result. “I try not to be a scaredy squirrel about these things, but I must say that “rarely happening” bugs in a widely used platform bother me,” he says. “A rare crash is manageable, but silently getting the wrong result is not. I would have expected this to be a high priority bug.”
‘How serious is the Java 7 Solr/ Lucene bug?’ is also currently being debated on stackoverflow, leading Lucene committer Robert Muir to post his thoughts on the matter:
“I’d say this bug is much sneakier than previous loop optimizer bugs we have hit hotspot (eg sign-flip stuff, https://issues.apache.org/jira/browse/LUCENE-2975 ). In that case we got wacky negative document deltas, which make it easy to catch. So we only had to manually unroll a single method to dodge it. On the other hand, the only “test” we had initially for that was a huge 10GB index of http://www.pangaea.de/ , so it was painful to narrow it down to this bug.
In this case, I spent a good amount of time (eg every night last week) trying to manually unroll / inline various things, not trying to create some workaround so we could dodge the bug and have the possibility of corrupt indexes being created. I could dodge some cases, but there were many more cases I couldn’t….. and I’m sure if we can trigger this stuff in our tests there are more cases out there…”
Meanwhile, Markus Eisele points out that the problems are not about Java 7 in general, but about the HotSpot JVM. He also assures developers that the problem is confined to software that makes “big use” of the optimisation methods which are broken, and “for the average use cases this will not affect you.”
An Oracle Bug has been filed, which has now been upgraded from a ‘low’ to a ‘high’ priority, and it is thought that the issue will be corrected in Update Pack 1.