Kafka hard
How does the 'Cooperative Sticky' rebalancing protocol improve upon the legacy 'Eager' protocol in large clusters?
- A It allows the broker to skip the ZooKeeper metadata check entirely for every third rebalance.
- B Incremental Revocation: Instead of a 'Stop-the-World' pause where all members drop all partitions, it allows members to keep their current assignments and only revokes the minimal set of partitions that actually need to move, maintaining higher throughput.
- C It forces the Producer to pause all writes until the Consumer group is perfectly stable again.
- D It encrypts the group coordinator's state using a shared secret between all members.
Answer: B. Incremental Revocation: Instead of a 'Stop-the-World' pause where all members drop all partitions, it allows members to keep their current assignments and only revokes the minimal set of partitions that actually need to move, maintaining higher throughput.
Introduced in KIP-429, this protocol is vital for large groups (e.g., 500 consumers). Eager rebalances cause massive spikes in lag because processing stops cluster-wide. Cooperative rebalancing allows the group to 'settle' into a new state without stopping the world.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- When implementing 'Exactly-Once Semantics' (EOS) in a Kafka stream, what is the role of the `isolation.level=read_committed` setting on the Consumer?
- Historically, which external service was required by Kafka to manage cluster hierarchy, metadata, and leader election status?
- Once a message is successfully appended to a Kafka partition log, what is its mutability status?
- In a Kafka cluster, what is the role of a 'Broker'?
- What is the primary indicator of a bottleneck in a Kafka consumer's processing logic?
- What is the primary architectural role of Apache Kafka in a distributed system?