Course contentsShow
← System Design
Lesson 1491 of 1,91936. Data Partitioning Strategies in DepthFree lesson

Data Skew and Cardinality Issues

How uneven data distribution emerges from partition key cardinality and value frequency patterns.

Data Skew and Cardinality Issues

You'll learn how choosing the wrong partition key characteristics leads to unbalanced data distribution that undermines your partitioning strategy.

The Core Problem

Data skew occurs when your partitions hold vastly different amounts of data, creating "heavy" and "light" partitions. This happens primarily due to two partition key properties:

Cardinality refers to the number of distinct values your partition key can take. Low cardinality (few unique values) means many records share the same key, forcing them into the same partition.

Value frequency describes how often each key value appears. Even with high cardinality, if certain values dominate your dataset, those partitions become overloaded.

Real-World Example

Imagine partitioning user data by country_code. If 70% of your users are from the US, that partition holds 70% of your data—a classic hot partition. The cardinality is decent (~200 countries), but the frequency distribution is terrible.

Compare this to partitioning by user_id (hash-based). With millions of users and relatively uniform distribution, neither cardinality nor frequency causes problems.

Why It Matters

Skewed partitions create:

  • Performance degradation: One server does most of the work
  • Capacity limits: The heaviest partition determines your scaling ceiling
  • Hotspot cascades: Overloaded partitions trigger the problems you learned earlier

The solution requires choosing keys with both high cardinality AND uniform value distribution, or applying techniques like composite keys, salting, or the hotspot mitigation strategies from previous lessons.

Key Takeaway:
Data skew emerges when partition keys have low cardinality or non-uniform value frequency—choose keys where values are both numerous and evenly distributed across your dataset.