Handling High-Cardinality Categoricals
You'll learn smart strategies for dealing with categorical variables that have too many unique categories to dummy-encode safely.
When a categorical variable has dozens or hundreds of unique values (like city names, product IDs, or user IDs), creating a separate dummy variable for each category becomes impractical. This is called high cardinality. With 50 categories, you'd add 49 dummy columns to your model—quickly overwhelming your data, causing overfitting, and making interpretation impossible.
Why High Cardinality Is a Problem
Remember the dummy variable trap? Now imagine having 100+ dummies. Your model becomes unstable, slow to train, and extremely sensitive to noise. You also risk running out of observations relative to predictors.
Strategy 1: Grouping Categories
Combine similar or rare categories into broader groups. For example, if you have 50 U.S. states, you might group them into 4-5 regions (Northeast, South, Midwest, West). This preserves geographic information while drastically reducing dimensions.
Strategy 2: Frequency Encoding
Replace each category with how often it appears in your data. A city that appears 500 times gets encoded as 500, while a rare city gets a small number. This single numeric column captures "popularity" without creating multiple dummies.
When to Avoid Dummies Altogether
If cardinality exceeds ~10-15 categories and you can't meaningfully group them, consider alternatives: frequency encoding, target encoding (covered later in more advanced techniques), or simply dropping the variable if it's not essential.
Key Takeaway: High-cardinality categoricals require smart reduction strategies—group related categories, use frequency encoding, or reconsider whether the variable is worth including rather than blindly creating hundreds of dummy columns.