Kafka hard
Why would a Kafka broker roll a new log segment (file) even if the current segment is only 10% full (based on `segment.bytes`)?
- A Because the broker detected a 'fragmentation' level higher than 50% in the underlying XFS filesystem.
- B Retention Granularity: The `segment.ms` (time limit) was reached. Rolling ensures that the timestamp of the segment is kept fresh, allowing the background retention thread to accurately delete or compact old data based on its age.
- C To reset the sequence numbers for the internal Idempotent Producer protocol.
- D Because a Consumer successfully completed a `commitSync()` for every message in the current segment.
Answer: B. Retention Granularity: The `segment.ms` (time limit) was reached. Rolling ensures that the timestamp of the segment is kept fresh, allowing the background retention thread to accurately delete or compact old data based on its age.
Retention in Kafka is per-file. You cannot delete half of a file. If you have a 1GB limit but zero traffic for a year, a huge file would stay forever. Time-based rolling ensures that even low-traffic topics are regularly 'chunked' so they can be rotated out of the system according to policy.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- When implementing 'Exactly-Once Semantics' (EOS) in a Kafka stream, what is the role of the `isolation.level=read_committed` setting on the Consumer?
- Historically, which external service was required by Kafka to manage cluster hierarchy, metadata, and leader election status?
- Once a message is successfully appended to a Kafka partition log, what is its mutability status?
- In a Kafka cluster, what is the role of a 'Broker'?
- What is the primary indicator of a bottleneck in a Kafka consumer's processing logic?
- What is the primary architectural role of Apache Kafka in a distributed system?