Apache Kafka 1.0 is here
© Shutterstock / Artist_R
Apache Kafka (originally developed by LinkedIn) was open sourced in early 2011 and graduated from the Apache Incubator in October 2012. It took a while but now version 1.0.0 is here. What does it mean for Apache Kafka? Let’s find out.
Apache Kafka 1.0 is finally here. Yes, they took their time getting there but, as Neha Narkhede, co-founder and CTO at Confluent and co-creator of Apache Kafka explained in a recent blog post, “for Apache Kafka, the wait for 1.0 was less about stability and more about completeness of the vision that we and the community set to build towards back when we first created Kafka.”
Apache Kafka 1.0 overview
Version 1.0.0 includes performance improvements with exactly-once semantics, significantly faster TLS and CRC32C implementations with Java 9 support, significantly faster controlled shutdown, and better JBOD support, among other general improvements and bug fixes, according to the official announcement.
Neha Narkhede summarizes the most important enhancements:
- Since its introduction in version 0.10, the Streams API was introduced in version 0.10 and has become hugely popular among Kafka users — just ask Pinterest, Rabobank, Zalando, and The New York Times. So what’s up with version 1.0? The builder API has been improved (KIP-120) and a new API has been added to expose the state of active tasks at runtime (KIP-130).
- Improvements to metrics: Connect metrics have been improved (KIP-196), a litany of new health check metrics are now exposed (KIP-188), and theree’s a global topic and partition count (KIP-168).
- Apache Kafka 1.0 supports Java 9, which leads to faster TLS and CRC32C implementations. “Over-the-wire encryption will be faster now, which will keep Kafka fast and compute costs low when encryption is enabled,” she added.
- Security: KIP-152 cleans up the error handling on Simple Authentication Security Layer (SASL) authentication attempts.
- Kafka can now tolerate disk failures better. KIP-112 helps it handle disk failure better — which means that a single disk failure in a JBOD broker will not bring the entire broker down. Instead, the broker will continue serving any log files that remain on functioning disks.
SEE ALSO: Big Data becomes Fast Data
Apache Kafka 1.0 — More than just a release
The wait for version 1.0 was long but there’s a reason for that. According to Neha, “there isn’t a set of guidelines that measure when a project is ready to be 1.0, and every project has its own unique story.”
For Apache Kafka, the wait for 1.0 was less about stability and more about completeness of the vision that we and the community set to build towards back when we first created Kafka. After all, Kafka has been in production at thousands of companies for several years. In fact, the very reason for Kafka’s success and its broad adoption in the marketplace has been about its ease of use and stability in production; the “it just works” experience of using Kafka.
With 1.0, Kafka has become “a stable, mature, and enterprise-ready Streaming Platform poised to continue serving users faithfully and evolving as stream processing plays a first-class role in enabling a central nervous system for all companies worldwide.”
If you want to know more about Kafka’s transformation from a Messaging System to a Streaming Platform, you need to read Neha’s blog post.