Choosing the Right Encoding Strategy
What you'll learn: How to systematically select the best encoding method for your categorical variables based on their characteristics and your model requirements.
The Decision Framework
You've learned many encoding techniques—but which one should you actually use? The answer depends on three key factors: cardinality (how many unique categories), ordinality (whether order matters), and model type (tree-based vs linear).
Decision Criteria
1. Check for Ordinality First
If your categories have a natural order (like "low," "medium," "high"), use label encoding. This preserves the meaningful relationship without inflating dimensions.
2. Consider Cardinality
- Low cardinality (< 10-15 categories): One-hot encoding works well for most models
- Medium cardinality (15-50 categories): Use target encoding or frequency encoding to avoid dimension explosion
- High cardinality (50+ categories): Consider embedding layers (deep learning) or binary encoding to manage memory
3. Match Your Model Type
- Tree-based models (Random Forest, XGBoost): Can handle label encoding even for nominal variables—they split on any numeric value
- Linear models (Logistic Regression, Neural Networks): Need one-hot encoding or embeddings to capture non-ordinal relationships properly
4. Watch for Special Cases
- Target leakage risk: Add proper cross-validation to target encoding
- Unseen categories at test time: Choose methods with fallback strategies
- Memory constraints: Prefer binary encoding or frequency encoding over one-hot
Key Takeaway: Choose encoding strategies by first checking ordinality, then evaluating cardinality and model compatibility—ordinal variables get label encoding, low-cardinality nominal variables get one-hot encoding, and high-cardinality variables need specialized approaches like target encoding or embeddings based on your model type.