Categorical Feature Encoding in Production
You'll learn how to handle categorical variables reliably at inference time, ensuring your model never breaks when encountering new or unexpected categories.
The Challenge: When "Paris" Meets "Mars"
In production, your model will see categories it never encountered during training. If you trained on cities like "New York," "London," and "Tokyo," what happens when a user inputs "Reykjavik"? Your encoding scheme must handle this gracefully—crashing or silently corrupting features is unacceptable.
Managing Vocabularies
Think of your category vocabulary as a fixed dictionary created during training. You must:
- Save the vocabulary alongside your model (e.g., with
joblibor in your feature store) - Freeze it — no new categories get added at inference time
- Version it — tie each vocabulary to a specific model version
Handling Unseen Categories
You have several strategies:
- Default/UNK token: Map unknowns to a special
<UNKNOWN>category - Drop to most frequent: Replace with the most common training category
- Zero vector: For one-hot encoding, use all zeros
- Hash collision: With hash encoding, unknowns naturally collide with existing buckets
Production-Ready Encoding Patterns
Your encoding pipeline must be deterministic and reproducible. Store encoders (like LabelEncoder or OneHotEncoder fitted objects) in your feature store or alongside the model. At inference:
- Load the same encoder used during training
- Apply
handle_unknown='ignore'or equivalent settings - Log when unknowns occur for monitoring
Avoid computing encodings on-the-fly from the inference dataset—always use the training-time vocabulary.
Key Takeaway: Production categorical encoding means freezing vocabularies at training time, explicitly handling unseen categories with fallback strategies, and serializing encoders to guarantee training-serving consistency.