Consistent Hashing hard
Why is Jump Consistent Hash NOT suitable for general distributed caching systems like Memcached/Redis?
- A Because it relies on expensive floating-point math that makes it significantly slower than ring-based lookups in high-traffic environments.
- B Because it requires a massive amount of RAM to store its internal probability jump tables, making it unsuitable for commodity cache nodes.
- C Because it is a patented algorithm that requires a restrictive and expensive commercial licensing fee for use in production environments.
- D Because it strictly requires numeric indices (0...N-1); removing a random node (e.g., Node 4) would require reshuffling everything beyond it.
Answer: D. Because it strictly requires numeric indices (0...N-1); removing a random node (e.g., Node 4) would require reshuffling everything beyond it.
Jump Hash is brilliant but limited. It maps keys to a fixed range. Dropping a cluster from 10 nodes to 9 works if you remove Node 9. But if Node 4 crashes, Jump Hash can't 'remove' index 4 without shifting 5 to 4, 6 to 5, etc., which triggers a massive reshuffle. Ring-based hashing handles random physical node failures much better because nodes are placed independently on the continuum.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- In a 'Smart Client' architecture (e.g., modern Cassandra drivers), how does the client find the right node?
- When a new node (Node X) joins the ring, where does it acquire its initial data from?
- Google's Maglev hashing algorithm (used in their Network Load Balancers) differs from standard Consistent Hashing rings. How does it generate the lookup table?
- Virtual Nodes improve balance but increase metadata size (Memory). How does 'Multi-Probe Consistent Hashing' achieve good balance without VNodes?
- In the Chord DHT protocol (a P2P consistent hashing implementation), naive lookups are O(N). How does Chord optimize this to O(log N)?
- Which of these systems is a famous user of Consistent Hashing?