skinny.

Latest / AWS Big Data Blog

Optimize consumer rebalancing on Amazon MSK with next generation protocol

If you run large consumer groups on Apache Kafka and Amazon Managed Streaming for Apache Kafka (Amazon MSK), you’ve likely experienced the pain of slow rebalances: processing stalls across all consumers, “rebalance storms” triggered by routine scaling events, and prolonged recovery times that impact downstream applications. With the classic rebalance protocol, even a single consumer joining or leaving the group forces a global synchronization barrier, pausing every consumer regardless of whether its partition assignments changed.

The KIP-848 consumer protocol, introduced in Apache Kafka 4.0…

The skinny

The skinny isn't ready yet — notes appear once the transcript is processed.