Course contentsShow
← AI Engineering
Lesson 1627 of 1,88639. Classical ML Deployment (Non-LLM)Free lesson

Categorical Feature Encoding in Production

Managing vocabularies, handling unseen categories, and encoding strategies that work reliably at inference time.

Categorical Feature Encoding in Production

You'll learn how to handle categorical variables reliably at inference time, ensuring your model never breaks when encountering new or unexpected categories.

The Challenge: When "Paris" Meets "Mars"

In production, your model will see categories it never encountered during training. If you trained on cities like "New York," "London," and "Tokyo," what happens when a user inputs "Reykjavik"? Your encoding scheme must handle this gracefully—crashing or silently corrupting features is unacceptable.

Managing Vocabularies

Think of your category vocabulary as a fixed dictionary created during training. You must:

  1. Save the vocabulary alongside your model (e.g., with joblib or in your feature store)
  2. Freeze it — no new categories get added at inference time
  3. Version it — tie each vocabulary to a specific model version

Handling Unseen Categories

You have several strategies:

  • Default/UNK token: Map unknowns to a special <UNKNOWN> category
  • Drop to most frequent: Replace with the most common training category
  • Zero vector: For one-hot encoding, use all zeros
  • Hash collision: With hash encoding, unknowns naturally collide with existing buckets

Production-Ready Encoding Patterns

Your encoding pipeline must be deterministic and reproducible. Store encoders (like LabelEncoder or OneHotEncoder fitted objects) in your feature store or alongside the model. At inference:

  1. Load the same encoder used during training
  2. Apply handle_unknown='ignore' or equivalent settings
  3. Log when unknowns occur for monitoring

Avoid computing encodings on-the-fly from the inference dataset—always use the training-time vocabulary.

Key Takeaway: Production categorical encoding means freezing vocabularies at training time, explicitly handling unseen categories with fallback strategies, and serializing encoders to guarantee training-serving consistency.