Sharding hard
When implementing a 'Saga' or distributed transaction across shards, why is an 'Idempotency Key' absolutely mandatory for every request?
- A To provide a cryptographic hash of the user's password for secure multi-node authentication.
- B To prevent 'Dreaded Double-Spend': if a network timeout occurs, the client will retry the request. The Idempotency Key allows the server to recognize the retry and return the previous success instead of processing the transaction twice.
- C To ensure that the request is routed to the shard with the most available CPU capacity at that specific microsecond.
- D To allow the database to automatically convert the transaction into a single-node request to save on network bandwidth.
Answer: B. To prevent 'Dreaded Double-Spend': if a network timeout occurs, the client will retry the request. The Idempotency Key allows the server to recognize the retry and return the previous success instead of processing the transaction twice.
In a distributed world, 'Failures' are a feature, not a bug. If Shard 2 is slightly slow, a client might timeout and send the request again. Without an Idempotency Key, the system would see two separate requests and debit the user twice. An Idempotency Key (usually a UUID generated by the client) acts as a unique 'fingerprint'. The server stores this key after a successful operation; if it ever sees that fingerprint again, it knows the work is already done and safely returns the cached result, ensuring 'Exactly-Once' semantics in an 'At-Least-Once' network.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- In an E-Commerce platform, why might you choose to shard the `Orders` table by `user_id` rather than the more obvious `order_id`?
- In the context of database scaling, what is the fundamental difference between 'Vertical Partitioning' and 'Horizontal Partitioning' (Sharding)?
- If Shard A fails in a large cluster, and its incoming traffic is automatically re-routed to Shard B, what is the risk of a 'Cascading Failure'?
- In a standard 'Modulo Sharding' setup (`key % N`), what is the primary consequence of adding a new server to the cluster (increasing N from 10 to 11)?
- From an engineering perspective, when should you choose 'Vertical Scaling' (a bigger server) over 'Horizontal Scaling' (Sharding)?
- Which sharding strategy is mathematically optimized for efficient 'Range Scans' (e.g., retrieving all records created between January and March)?