Course contentsShow
← System Design
Lesson 246 of 1,9197. Database Scaling: ShardingFree lesson

Consistent Hashing for Sharding

Apply consistent hashing to minimize data movement when adding or removing shards dynamically.

Consistent Hashing for Sharding

What you'll learn: How consistent hashing minimizes chaos when your sharded database needs to grow or shrink.

The Resharding Problem

When you use simple hash-based sharding (like hash(user_id) % num_shards), adding or removing a shard is painful. Change the number of shards, and almost every key maps to a different shard—triggering massive data migration. If you go from 4 to 5 shards, roughly 80% of your data needs to move!

How Consistent Hashing Helps

Consistent hashing places both shards and data keys on a circular hash ring (imagine a clock face). Each key hashes to a position on the ring and gets assigned to the next shard encountered clockwise.

The magic: When you add or remove a shard, only data near that position on the ring needs to move. Instead of affecting 80% of keys, consistent hashing typically redistributes only 1/n of the data (where n is the number of shards).

The Ring Analogy

Think of a circular running track with water stations (shards). Runners (data) start at their hash position and run clockwise to their nearest water station. Add a new station? Only runners between the new station and the previous one switch—everyone else keeps their assignment.

Virtual Nodes Enhancement

To avoid uneven distribution, each physical shard gets multiple positions on the ring (called "virtual nodes"). This smooths out data distribution and makes adding/removing shards more gradual.

Practical Impact

Consistent hashing makes dynamic sharding feasible. You can scale your database horizontally without planning massive downtime windows for data migration—critical for growing systems that can't afford service interruptions.

Key Takeaway: Consistent hashing transforms sharding from a one-time architectural decision into a flexible scaling strategy by limiting data movement to only the affected portion of the ring when shards change.