Sharding hard
In a professionally managed sharded system, why is it considered vital to decouple 'Logical Shards' (buckets) from 'Physical Shards' (servers)?
- A It allows the system to support multiple different operating systems within the same database cluster.
- B It ensures that the database remains compatible with older SQL versions while the physical hardware is upgraded.
- C It provides 'Dynamic Resizability'. By having 10,000 logical buckets sharded across 10 servers, you can easily grow to 20 servers just by moving 5,000 buckets, without rehashing the entire database.
- D It is a requirement for meeting SOC2 and GDPR compliance regarding data isolation.
Answer: C. It provides 'Dynamic Resizability'. By having 10,000 logical buckets sharded across 10 servers, you can easily grow to 20 servers just by moving 5,000 buckets, without rehashing the entire database.
Think of it as 'Mapping'. If you map a User straight to a Physical Server, you are stuck. If you map a User to 'Bucket #402', and then you have a map that says 'Bucket #402 lives on Server A', you gain incredible flexibility. When Server A gets too full, you only move Bucket #402 to Server B. This 'Abstraction Layer' is how modern systems like CockroachDB (using 'Ranges') and MongoDB (using 'Chunks') handle petabytes of data without ever having to 'Shut everything down' to add a new server.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- In an E-Commerce platform, why might you choose to shard the `Orders` table by `user_id` rather than the more obvious `order_id`?
- In the context of database scaling, what is the fundamental difference between 'Vertical Partitioning' and 'Horizontal Partitioning' (Sharding)?
- If Shard A fails in a large cluster, and its incoming traffic is automatically re-routed to Shard B, what is the risk of a 'Cascading Failure'?
- In a standard 'Modulo Sharding' setup (`key % N`), what is the primary consequence of adding a new server to the cluster (increasing N from 10 to 11)?
- From an engineering perspective, when should you choose 'Vertical Scaling' (a bigger server) over 'Horizontal Scaling' (Sharding)?
- Which sharding strategy is mathematically optimized for efficient 'Range Scans' (e.g., retrieving all records created between January and March)?