Kafka hard
In a compacted topic, after writing a 'Tombstone' (null value) for a specific key, what mechanism prevents the record from occupying disk space forever?
- A The record is deleted as soon as the first consumer confirms it has seen the null value.
- B The 'Log Cleaner' thread eventually removes the tombstone after the `delete.retention.ms` period, ensuring all consumers have had time to see the deletion signal.
- C Kafka performs a 'Force Flush' that overwrites the sector on the physical disk.
- D The ZooKeeper coordinator manually prunes the segment indices during low-traffic periods.
Answer: B. The 'Log Cleaner' thread eventually removes the tombstone after the `delete.retention.ms` period, ensuring all consumers have had time to see the deletion signal.
A tombstone must stay in the log for a minimum duration to ensure that any consumer currently 'catching up' from a backup or a crash sees that the key should be deleted from its local state. If deleted too early, a consumer might miss the deletion forever.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- When implementing 'Exactly-Once Semantics' (EOS) in a Kafka stream, what is the role of the `isolation.level=read_committed` setting on the Consumer?
- Historically, which external service was required by Kafka to manage cluster hierarchy, metadata, and leader election status?
- Once a message is successfully appended to a Kafka partition log, what is its mutability status?
- In a Kafka cluster, what is the role of a 'Broker'?
- What is the primary indicator of a bottleneck in a Kafka consumer's processing logic?
- What is the primary architectural role of Apache Kafka in a distributed system?