Sharding easy
Why is 'Resharding' (adding or removing servers from a sharded cluster) considered a difficult operational task?
- A Purchasing and racking new physical hardware is the primary source of complexity.
- B It requires assigning new, unique IP addresses to every single server in the network.
- C It involves a 'mass migration' of existing data from old servers to the new ones to balance the load, often while the system is under heavy live traffic.
- D It automatically invalidates all existing data encryption keys in the cluster.
Answer: C. It involves a 'mass migration' of existing data from old servers to the new ones to balance the load, often while the system is under heavy live traffic.
Resharding is a high-risk operation. If you increase your shard count from 10 to 12, a significant portion of your data must move to different locations to remain consistent with your routing algorithm. Moving Terabytes of data across the network consumes massive amounts of CPU and I/O bandwidth, potentially slowing down your real user requests. Modern 'distributed' databases aim to solve this by automating the background migration, but for legacy systems, it is often a manual, high-stress event.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- In an E-Commerce platform, why might you choose to shard the `Orders` table by `user_id` rather than the more obvious `order_id`?
- In the context of database scaling, what is the fundamental difference between 'Vertical Partitioning' and 'Horizontal Partitioning' (Sharding)?
- If Shard A fails in a large cluster, and its incoming traffic is automatically re-routed to Shard B, what is the risk of a 'Cascading Failure'?
- In a standard 'Modulo Sharding' setup (`key % N`), what is the primary consequence of adding a new server to the cluster (increasing N from 10 to 11)?
- From an engineering perspective, when should you choose 'Vertical Scaling' (a bigger server) over 'Horizontal Scaling' (Sharding)?
- Which sharding strategy is mathematically optimized for efficient 'Range Scans' (e.g., retrieving all records created between January and March)?