Kafka medium
Why would a data engineering team implement the Confluent Schema Registry (or equivalent) alongside their Kafka cluster?
- A To serve as a secondary backup for the raw partition log files in PDF format.
- B To enforce 'Data Contracts' (e.g., Avro/Protobuf). It prevents Producers from sending incompatible data that would cause downstream Consumer applications to crash during deserialization.
- C To provide a GUI for manually editing and updating messages already written to disk.
- D To increase the maximum number of partitions permitted for a single topic.
Answer: B. To enforce 'Data Contracts' (e.g., Avro/Protobuf). It prevents Producers from sending incompatible data that would cause downstream Consumer applications to crash during deserialization.
Kafka itself is 'Schema Blind'—it only sees bytes. The Registry ensures that schemas evolve safely (e.g., you can't delete a field that consumers still expect), protecting the reliability of the entire data pipeline.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- When implementing 'Exactly-Once Semantics' (EOS) in a Kafka stream, what is the role of the `isolation.level=read_committed` setting on the Consumer?
- Historically, which external service was required by Kafka to manage cluster hierarchy, metadata, and leader election status?
- Once a message is successfully appended to a Kafka partition log, what is its mutability status?
- In a Kafka cluster, what is the role of a 'Broker'?
- What is the primary indicator of a bottleneck in a Kafka consumer's processing logic?
- What is the primary architectural role of Apache Kafka in a distributed system?