Kafka hard
The 'High Watermark' in a Kafka partition represents the largest offset successfully replicated to all In-Sync Replicas. What is the consequence of allowing a Consumer to read beyond this watermark?
- A Dirty Read / Durability Violation: If the consumer reads Offset 100 which hasn't been replicated, and the Leader then crashes, Offset 100 may never exist on the new Leader, leading to inconsistent data views.
- B It would cause the Consumer's Kafka-client library to throw an 'OutOfMemory' exception due to uncompressed headers.
- C It would triple the bandwidth cost as the consumer has to fetch from all replicas simultaneously.
- D The Topic would be automatically converted to 'Compact' mode to prevent further offset drift.
Answer: A. Dirty Read / Durability Violation: If the consumer reads Offset 100 which hasn't been replicated, and the Leader then crashes, Offset 100 may never exist on the new Leader, leading to inconsistent data views.
By enforcing that consumers only see data up to the HW, Kafka guarantees that even if a Leader fails, a consumer will never have to 'rewind' its state because it read data that vanished. This ensures 'Read-Committed' behavior across the cluster.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- When implementing 'Exactly-Once Semantics' (EOS) in a Kafka stream, what is the role of the `isolation.level=read_committed` setting on the Consumer?
- Historically, which external service was required by Kafka to manage cluster hierarchy, metadata, and leader election status?
- Once a message is successfully appended to a Kafka partition log, what is its mutability status?
- In a Kafka cluster, what is the role of a 'Broker'?
- What is the primary indicator of a bottleneck in a Kafka consumer's processing logic?
- What is the primary architectural role of Apache Kafka in a distributed system?