Scale By the Bay 2018

Deploying Kafka Streams Applications with Docker and Kubernetes
Abstract: Kafka Streams, Apache Kafka’s stream processing library,
allows developers to build sophisticated stateful stream processing
applications which you can deploy in an environment of your choice.
Kafka Streams is not only scalable, but fully-elastic allowing for
dynamic scale-in and scale-out as the library handles state migration
transparently in the background. By running Kafka Streams applications
on Kubernetes, you will be able to use Kubernetes powerful control
plane to standardize and simplify the application management – from
deployment to dynamic scaling.

In this technical deep dive, we’ll explains the internals of dynamic
scaling and state migration in Kafka Streams. We’ll then show, with a
live demo, how a Kafka Streams application can run in a Docker
container on Kubernetes and the dynamic scaling of an application
running in Kubernetes.