Consistent Hashing easy
Can Consistent Hashing alone solve the 'Celebrity Problem' (one key getting 1M req/sec)?
- A Yes, standard consistent hashing algorithms automatically detect hot keys and spread them across all available cluster nodes.
- B Yes, the use of Virtual Nodes ensures that any individual key is physically replicated and load-balanced across the entire ring.
- C No, it maps a key to exactly one set of nodes; if a single specific key receives extreme traffic, it can still overwhelm those specific servers.
- D Yes, by simply increasing the replication factor until every node in the cluster holds a copy of the high-traffic 'celebrity' record.
Answer: C. No, it maps a key to exactly one set of nodes; if a single specific key receives extreme traffic, it can still overwhelm those specific servers.
Consistent Hashing solves the problem of 'Data Skew' (ensuring nodes have a similar number of keys), but it cannot natively solve 'Request Skew' (one key being much more popular than others). If a specific key like 'TrendingTopic' hashes to Node 5, Node 5 must handle all that traffic. Solving this requires additional strategies like client-side caching or dividing high-traffic keys into sub-shards.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- In a 'Smart Client' architecture (e.g., modern Cassandra drivers), how does the client find the right node?
- When a new node (Node X) joins the ring, where does it acquire its initial data from?
- Google's Maglev hashing algorithm (used in their Network Load Balancers) differs from standard Consistent Hashing rings. How does it generate the lookup table?
- Virtual Nodes improve balance but increase metadata size (Memory). How does 'Multi-Probe Consistent Hashing' achieve good balance without VNodes?
- In the Chord DHT protocol (a P2P consistent hashing implementation), naive lookups are O(N). How does Chord optimize this to O(log N)?
- Which of these systems is a famous user of Consistent Hashing?