Hash-Based Partitioning
What you'll learn: How hash functions distribute data evenly across shards to prevent performance bottlenecks.
In distributed systems, you've already seen how range-based partitioning divides data by value ranges. But what if certain ranges get far more traffic than others? That's where hash-based partitioning shines.
The Core Idea
Hash-based partitioning applies a hash function to the partition key, then uses the hash value to determine which shard stores the data. Instead of storing all customers with names A-M on one shard, you hash each customer's ID and assign it based on the hash result.
The goal: Distribute data uniformly across all shards, avoiding "hotspots" where one shard becomes overwhelmed.
How It Works
- Choose a partition key (e.g.,
user_id) - Apply a hash function:
hash(user_id) → numeric_value - Map the hash to a shard:
shard_number = hash_value % total_shards
For example, with 4 shards:
hash(user_123) = 857 → 857 % 4 = 1(goes to shard 1)hash(user_456) = 1042 → 1042 % 4 = 2(goes to shard 2)
Even if user IDs are sequential, the hash function scrambles them, spreading load evenly.
Why This Matters
Avoiding hotspots: If you stored all recent orders on one shard (like in range partitioning by date), that shard would handle all write traffic. Hash-based partitioning distributes recent and old data across all shards.
Trade-offs: You lose efficient range queries. Finding "all users with IDs 100-200" requires querying every shard since hashing scatters them. But you gain balanced load distribution.
Analogy
Think of dealing cards: range-based partitioning gives all hearts to one player (unbalanced). Hash-based partitioning deals cards round-robin (balanced), but now finding "all hearts" means checking everyone's hand.
Key Takeaway: Hash-based partitioning uses hash functions to uniformly distribute data across shards, preventing hotspots at the cost of efficient range queries.