Target Encoding and Mean Encoding
What you'll learn: How to encode high-cardinality categorical features by replacing each category with the average target value seen in training data.
The Core Idea
When you have a categorical feature with many unique values (like city names, product IDs, or user IDs), one-hot encoding creates an explosion of columns. Target encoding offers an elegant alternative: replace each category with the mean of the target variable for that category.
For example, imagine you're predicting house prices and have a "Neighborhood" feature with 200 unique values. Instead of creating 200 binary columns, target encoding calculates the average house price for each neighborhood and replaces the neighborhood name with that number.
How It Works
- Group your training data by the categorical feature
- Calculate the mean target value for each category
- Replace category labels with these means
- Apply the same mapping to your test data
The Analogy
Think of it like using restaurant ratings. Instead of listing every restaurant's name (one-hot encoding), you replace each name with its average customer rating. "Mario's Pizza" becomes 4.5, "Joe's Diner" becomes 3.2. The category itself becomes a meaningful number.
Important Considerations
Target encoding creates a direct relationship between the feature and target, which can lead to overfitting, especially with rare categories. It also risks data leakage if not done carefully—you must fit the encoding on training data only and never let test information influence the mapping.
Key Takeaway: Target encoding transforms high-cardinality categories into numerical features by replacing each category with its mean target value, dramatically reducing dimensionality while preserving predictive information.