Apache Flink 1.10.0 adds native Kubernetes integration as beta version
Apache Flink has been released in version 1.10.0. More than 200 contributors worked on this release, and they added several new features including updates for PyFlink, a beta version of native Kubernetes integration, managed memory extension, and RocksDB updates.
The latest version of Apache Flink, v1.10.0, has arrived. Apache Flink is a framework and distributed processing engine for stateful computations. It supports unbounded streams with a defined start, as well as bounded streams with a defined start and end.
In Apache Flink 1.10, over 1,200 issues were implemented by more than 200 contributors. Let’s see what this means for Flink users.
Native Kubernetes integration
Version 1.10 has the first phase of active Kubernetes integration on board, which adds initial support for session clusters. In order to allocate new pods on-demand, Flink’s ResourceManager K8sResMngr natively communicates with Kubernetes. This is similar to how Flink’s integration with Yarn and Mesos works.
PyFlink was first introduced as preview in Apache Flink 1.9, and its goal is to add full Python support to Flink. In v1.10, it has received additional features. For one thing, it is now possible to install PyFlink, still in “a very initial version,” via pip:
pip install apache-flink.
But that’s not all: Python User-Defined Functions (UDF) in the TableAPI/SQL are now supported, and support for UDTF/UDAF is planned.
Since Python 2 reached its end of life last month, it is no longer supported. Python 3.5 is now the minimum requirement.
There are several other updates in Apache Flink 1.10 as well. For example, managed memory can now account for memory usage of RocksDBStateBackend, the configuration of RocksDB has been simplified, and v1.10 can be compiled and run on Java 11.