Consistent Hashing medium
Why is MurmurHash often preferred over MD5/SHA for Consistent Hashing partitioning?
- A MurmurHash provides significantly better cryptographic security against external attackers attempting to brute-force keys.
- B MD5 generates a 128-bit hash which is considered too short for modern large-scale distributed clusters with billions of keys.
- C MurmurHash is extremely CPU-efficient and provides excellent distribution properties for internal routing and partitioning.
- D MurmurHash supports non-integer input keys while MD5 is strictly limited to processing binary data and complex byte arrays.
Answer: C. MurmurHash is extremely CPU-efficient and provides excellent distribution properties for internal routing and partitioning.
In consistent hashing, the hash function is executed on every single client request. Therefore, performance and the 'Avalanche Effect' (where a small change in input leads to a large change in output) are much more important than cryptographic collision resistance. MurmurHash is a non-cryptographic function designed specifically for fast table lookups, making it the industry standard for sharding.
One system design question a day, with spaced repetition so the ones you miss come back.
Get the appRelated questions
- In a 'Smart Client' architecture (e.g., modern Cassandra drivers), how does the client find the right node?
- When a new node (Node X) joins the ring, where does it acquire its initial data from?
- Google's Maglev hashing algorithm (used in their Network Load Balancers) differs from standard Consistent Hashing rings. How does it generate the lookup table?
- Virtual Nodes improve balance but increase metadata size (Memory). How does 'Multi-Probe Consistent Hashing' achieve good balance without VNodes?
- In the Chord DHT protocol (a P2P consistent hashing implementation), naive lookups are O(N). How does Chord optimize this to O(log N)?
- Which of these systems is a famous user of Consistent Hashing?