Apache Storm v2.0: Porting from Clojure to pure Java, performance upgrades, & more
The latest major release from Apache Storm introduces some new changes and improvements. Apache Storm v2.0.0 rearchitects from Clojure to pure Java and improves user performance with a new high-performance core. See what’s new and migrate to the latest version.
Apache Storm underwent quite a journey since its inception. The real-time computation system became open source after Twitter acquired its developer team BackType. In 2013, it underwent incubation and then became an Apache Top-Level Project in 2014.
Now, it crests the horizon with a new major release. Apache Storm 2.0.0 arrived on May 30, 2019 and is available to download. It includes performance improvements, several new features, bug fixes, and more.
New Apache Storm features
What was added in the latest update?
- Removal of storm-kafka and storm-kafka-client Subscription interface: replaced with TopicFilter and ManualPartitioner interfaces
- KafkaBolt custom callback
- Windowing API improvements: including stateful windowing and window checkpointing
- Nimbus admin groups: enables group level permissions
- Authority through delegation tokens for workers
- Optional impersonation
- Added Lambda support
- Blacklist Scheduler: adds unstable nodes to blacklist
- Abstract class ConfigurableTopology
- Introduce ClusterMetricsConsumer
- “progress” method added to OutputCollector
- symlinks alternate
- Maven artifacts reorganization: split into client and server-facing parts
- Admin commands: new commands for debugging cluster state
View the full in-depth release notes for v2.0.0.
This update comes with several notable changes and performance enhancements. Firstly, Storm changes its architecture from Clojure over to pure Java. Why make this change? One of the reasons is in order to offer a more user-friendly developer experience. Storm v2.0 requires Java 8.
From the Storm website:
In previous releases a large part of Storm’s core functionality was implemented in Clojure. Storm 2.0.0 has been rearchitected with it’s core functionality implemented in pure Java. The new Java-based implementation has improved performance significantly, and made Storm’s internal APIs more maintainable and extensible. While Storm’s Clojure implementation served it well for many years, it was often cited as a barrier for entry to new contributors. Storm’s codebase is now more accessible to developers who don’t want to learn Clojure in order to contribute.
According to a post from the Yahoo! Developer Network, new benchmarks and testing show that its performance improvements now make Storm 2.0 one of the best-performing platforms. Version 2.0 offers less overhead and processes more data with a new leaner performance core. This new core also supports backwards compatibility.