Course contentsShow
← Computer Science
Lesson 2687 of 2,87260. Distributed Data and CommunicationFree lesson

Hash-Based Partitioning

Using hash functions to distribute data uniformly across shards, avoiding hotspots.

Hash-Based Partitioning

What you'll learn: How hash functions distribute data evenly across shards to prevent performance bottlenecks.


In distributed systems, you've already seen how range-based partitioning divides data by value ranges. But what if certain ranges get far more traffic than others? That's where hash-based partitioning shines.

The Core Idea

Hash-based partitioning applies a hash function to the partition key, then uses the hash value to determine which shard stores the data. Instead of storing all customers with names A-M on one shard, you hash each customer's ID and assign it based on the hash result.

The goal: Distribute data uniformly across all shards, avoiding "hotspots" where one shard becomes overwhelmed.

How It Works

  1. Choose a partition key (e.g., user_id)
  2. Apply a hash function: hash(user_id) → numeric_value
  3. Map the hash to a shard: shard_number = hash_value % total_shards

For example, with 4 shards:

  • hash(user_123) = 857 → 857 % 4 = 1 (goes to shard 1)
  • hash(user_456) = 1042 → 1042 % 4 = 2 (goes to shard 2)

Even if user IDs are sequential, the hash function scrambles them, spreading load evenly.

Why This Matters

Avoiding hotspots: If you stored all recent orders on one shard (like in range partitioning by date), that shard would handle all write traffic. Hash-based partitioning distributes recent and old data across all shards.

Trade-offs: You lose efficient range queries. Finding "all users with IDs 100-200" requires querying every shard since hashing scatters them. But you gain balanced load distribution.

Analogy

Think of dealing cards: range-based partitioning gives all hearts to one player (unbalanced). Hash-based partitioning deals cards round-robin (balanced), but now finding "all hearts" means checking everyone's hand.

Key Takeaway: Hash-based partitioning uses hash functions to uniformly distribute data across shards, preventing hotspots at the cost of efficient range queries.