Running Kafka in Kubernetes: a practical guide

Cloud, Containers & Infrastructure
Voting no longer possible
Voting enabled when talk has started

The rise of Apache Kafka as the de-facto standard for event streaming has coincided with the rise of Kubernetes for cloud-native applications. While Kubernetes is a great choice for any distributed system, that doesn't mean it is easy to deploy and maintain a Kafka cluster running on it.

At IBM we have hands-on experience with running Kafka in Kubernetes and in this session I will share our top tips for a smooth ride. I will show an example deployment of Kafka on Kubernetes and step through the system to explain the common pitfalls and how to avoid them. This will include the Kubernetes objects to use, resource considerations and connecting applications to the cluster.

I will also discuss useful Kafka metrics to include in Kubernetes liveness and readiness probes. Finally I will introduce some of the tools available to help automate the management of your Kafka deployment.

Katherine Stanley


Katherine Stanley is a Software Engineer in the IBM Event Streams team based in the UK. Through her work on IBM Event Streams she has gained experience running Apache Kafka on Kubernetes and running enterprise Kafka applications. In her previous role she specialised in cloud native Java applications and microservices architectures. Katherine has co-authored an IBM Redbook on Java microservices and has contributed to the open source microservice project Game On. She enjoys sharing her experiences and has presented at conferences around the world, including the Kafka Summits in New York and London, JavaLand in Germany and JFokus in Sweden.

Talks by tracksTalks by session typesList of SpeakersSchedule