Kafka medium
When `enable.idempotence=true` is enabled, how does a Kafka broker prevent duplicate writes when a Producer retries a failed network request?
- A By performing a full string comparison of the message body against the last 1MB of log data.
- B By checking the 'Producer ID' (PID) and a 'Sequence Number' attached to every batch. If the broker has already successfully committed that specific sequence number, it discards the retry.
- C By asking the ZooKeeper ensemble to verify if a similar write was attempted.
- D It requires the Producer to manually 'rollback' the previous failed write first.
Answer: B. By checking the 'Producer ID' (PID) and a 'Sequence Number' attached to every batch. If the broker has already successfully committed that specific sequence number, it discards the retry.
Network ACKs can fail even if the data was written. Without idempotence, retrying would cause 'Zombie' duplicates. With it, Kafka can detect that 'Sequence 10' already exists and safely return success for the retry without writing the data again.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- When implementing 'Exactly-Once Semantics' (EOS) in a Kafka stream, what is the role of the `isolation.level=read_committed` setting on the Consumer?
- Historically, which external service was required by Kafka to manage cluster hierarchy, metadata, and leader election status?
- Once a message is successfully appended to a Kafka partition log, what is its mutability status?
- In a Kafka cluster, what is the role of a 'Broker'?
- What is the primary indicator of a bottleneck in a Kafka consumer's processing logic?
- What is the primary architectural role of Apache Kafka in a distributed system?