Kafka hard
A Consumer application is heartbeating successfully, but the logs show frequent 'Rebalancing' events. The application team suspects a 'Livelock'. Which parameter should they investigate?
- A `retry.backoff.ms`: the time between individual network retries.
- B `max.poll.interval.ms`: if the business logic for processing a single batch of records exceeds this limit, the broker assumes the consumer is hung and evicts it, even if heartbeats are healthy.
- C `message.max.bytes`: the physical size limit of a single event in the partition.
- D `num.network.threads`: the number of background threads used for TCP handling.
Answer: B. `max.poll.interval.ms`: if the business logic for processing a single batch of records exceeds this limit, the broker assumes the consumer is hung and evicts it, even if heartbeats are healthy.
Heartbeats are handled by a background thread, while `poll()` is calling by the application thread. If your code is stuck in an infinite loop or a slow DB call, the heartbeat thread keeps saying 'I'm alive', but the application isn't actually doing work. `max.poll.interval.ms` fixes this 'Zombie' state.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- When implementing 'Exactly-Once Semantics' (EOS) in a Kafka stream, what is the role of the `isolation.level=read_committed` setting on the Consumer?
- Historically, which external service was required by Kafka to manage cluster hierarchy, metadata, and leader election status?
- Once a message is successfully appended to a Kafka partition log, what is its mutability status?
- In a Kafka cluster, what is the role of a 'Broker'?
- What is the primary indicator of a bottleneck in a Kafka consumer's processing logic?
- What is the primary architectural role of Apache Kafka in a distributed system?