Course contentsShow
← System Design
Lesson 361 of 1,91910. NoSQL: Key-Value StoresFree lesson

Partitioning in Distributed Key-Value Stores

How data is divided across nodes using hash-based partitioning to enable horizontal scaling.

Partitioning in Distributed Key-Value Stores

What You'll Learn

You'll learn how distributed key-value stores split data across multiple nodes using hash-based partitioning to handle massive scale horizontally.

The Core Challenge

When a single Redis or key-value server can't hold all your data or handle all requests, you need to distribute the data across multiple machines. But how do you decide which key goes on which server?

Hash-Based Partitioning Explained

Think of partitioning like sorting mail into postal boxes. Instead of one giant box, you have many smaller boxes, each responsible for a subset of addresses.

Hash-based partitioning works by:

  1. Taking your key (e.g., "user:12345")
  2. Running it through a hash function to generate a number
  3. Using that number to determine which node stores the key

For example, with 3 nodes:

  • hash("user:12345") % 3 = 2 → goes to Node 2
  • hash("session:abc") % 3 = 0 → goes to Node 0

Why This Enables Horizontal Scaling

Instead of buying a bigger server (vertical scaling), you add more nodes (horizontal scaling). Each node handles approximately 1/N of the total data and traffic, where N is your node count. Need more capacity? Add nodes and rebalance.

The hash function distributes keys evenly and deterministically — the same key always maps to the same node, and all nodes get roughly equal data.

The Distribution Guarantee

Because hash functions produce uniform output, you naturally get load balancing. No single node becomes a "hot spot" (assuming your key distribution is random). This is why partitioning is foundational for distributed key-value stores at scale.

Key Takeaway: Hash-based partitioning divides data across nodes by hashing keys to determine placement, enabling horizontal scaling by distributing both storage and request load evenly across a growing cluster.