Kafka hard
How does the 'Fetch Session' (KIP-227) optimization reduce the metadata overhead for consumers reading from topics with thousands of partitions?
- A It switches the communication protocol from TCP to UDP for faster packet throughput.
- B Incremental Fetch Metadata: The client and broker maintain a stateful session where the client only sends the *changes* in its partition list (delta) rather than re-sending the full list of 1,000+ partition IDs in every single fetch request.
- C It eliminates the need for the consumer to provide an offset.
- D It allows consumers to read data directly from the OS Page Cache of a neighboring broker.
Answer: B. Incremental Fetch Metadata: The client and broker maintain a stateful session where the client only sends the *changes* in its partition list (delta) rather than re-sending the full list of 1,000+ partition IDs in every single fetch request.
In large-scale deployments, the 'request' headers for a fetch could be larger than the actual 'data' being returned. Fetch Sessions make the 'Steady State' extremely lightweight, significantly reducing network and CPU overhead for both the client and the broker.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- When implementing 'Exactly-Once Semantics' (EOS) in a Kafka stream, what is the role of the `isolation.level=read_committed` setting on the Consumer?
- Historically, which external service was required by Kafka to manage cluster hierarchy, metadata, and leader election status?
- Once a message is successfully appended to a Kafka partition log, what is its mutability status?
- In a Kafka cluster, what is the role of a 'Broker'?
- What is the primary indicator of a bottleneck in a Kafka consumer's processing logic?
- What is the primary architectural role of Apache Kafka in a distributed system?