Course contentsShow
← System Design
Lesson 260 of 1,9197. Database Scaling: ShardingFree lesson

Consistent Hashing for Minimal Resharding

Understand how consistent hashing reduces data movement when the shard count changes.

Consistent Hashing for Minimal Resharding

You'll learn: How consistent hashing dramatically reduces the amount of data that needs to move when you add or remove shards from your database.

The Problem with Traditional Hashing

Imagine you have 5 shards and use simple hash-based sharding: shard = hash(key) % 5. This works fine until you need to scale to 6 shards. Now the formula becomes hash(key) % 6, and suddenly almost every key maps to a different shard. You'd need to move roughly 80% of your data!

How Consistent Hashing Solves This

Consistent hashing arranges shards on a virtual "ring" (imagine a clock face from 0 to 360 degrees). Each key is also hashed to a position on this ring, and it belongs to the first shard encountered moving clockwise.

The magic happens during resharding: When you add a new shard to the ring, it only "claims" keys from its immediate neighbors. When you remove a shard, only its keys need to move to the next shard on the ring.

The Math Behind the Win

With traditional hashing and N shards, adding one shard requires moving approximately (N/(N+1)) of all data. With 5→6 shards, that's 83% of data.

With consistent hashing, adding a shard only requires moving approximately 1/(N+1) of data. That's just 17% — a massive improvement!

This makes resharding operations feasible in production without lengthy maintenance windows or massive data transfer operations.

Key Takeaway: Consistent hashing minimizes data movement during resharding by localizing the impact of shard changes to only the affected portions of the hash ring, rather than remapping everything.