Course contentsShow
← Machine Learning and Deep Learning
Lesson 421 of 3,53810. Feature EngineeringFree lesson

Handling High-Cardinality Categories

Strategies for encoding variables with many unique values that would create excessive one-hot columns.

Handling High-Cardinality Categories

What You'll Learn

You'll learn practical strategies for encoding categorical variables with hundreds or thousands of unique values without creating unmanageable one-hot encoded datasets.

The High-Cardinality Problem

Imagine you have a dataset of customer purchases where one feature is "City" with 500 unique cities. If you applied one-hot encoding (lesson 420), you'd create 500 new binary columns! This explosion of features:

  • Makes your dataset enormous and slow to process
  • Introduces extreme sparsity (mostly zeros)
  • Can cause overfitting since each city appears in few rows
  • Wastes memory and computational resources

High-cardinality means a categorical variable has many unique values, making standard one-hot encoding impractical.

Strategic Approaches

1. Frequency/Count Encoding

Replace each category with how often it appears. City "New York" (appears 1,000 times) becomes 1000, rare cities get smaller numbers. Simple and keeps just one column.

2. Target Encoding (Mean Encoding)

Replace each category with the average target value for that category. If customers from "Boston" spend $150 on average, encode Boston as 150. Works well but requires careful validation splitting to avoid data leakage.

3. Grouping Rare Categories

Combine infrequent categories into an "Other" bucket. If 450 cities each appear fewer than 10 times, group them together, leaving you with ~50 meaningful categories plus "Other."

4. Embedding Approaches

For deep learning, treat categories like words and learn dense vector representations (covered in advanced lessons).

When to Use Each

Use frequency encoding when category popularity matters. Use target encoding when categories have clear relationships with your target. Use grouping when many categories are truly rare and interchangeable.

Key Takeaway: High-cardinality categories require smarter encoding strategies than one-hot encoding—frequency encoding, target encoding, or grouping rare values can dramatically reduce dimensionality while preserving predictive information.