Course contentsShow
← Data Science
Lesson 638 of 2,14517. Categorical Predictors and InteractionsFree lesson

One-Hot Encoding Overview

Creating a dummy variable for every category, including when this approach is appropriate.

One-Hot Encoding Overview

What you'll learn: You'll discover when and how to create a dummy variable for every category—and when this approach makes sense versus using a reference category.

The Core Idea

You already know about dummy variable encoding where you create k-1 dummy variables for k categories, leaving one as the reference. One-hot encoding takes a different approach: it creates k dummy variables for k categories—one for every level, with no reference category left out.

For example, if you have a color variable with values "Red," "Green," and "Blue," one-hot encoding creates three separate binary columns: color_Red, color_Green, and color_Blue. Each observation gets a 1 in exactly one column and 0s elsewhere.

When Is One-Hot Encoding Appropriate?

One-hot encoding is useful in these scenarios:

  • Tree-based models (decision trees, random forests): These algorithms don't use the same linear framework as regression and can handle all k variables without issues
  • Neural networks: Many deep learning frameworks expect one-hot encoded inputs
  • When interpretability requires seeing all categories: Sometimes you want explicit coefficients for every group
  • Non-regression contexts: Classification tasks, clustering, or similarity calculations

The Critical Caveat

In linear regression, one-hot encoding creates perfect multicollinearity. If you include all k dummy variables plus an intercept, the model cannot uniquely estimate coefficients (the columns sum to a constant). This is why standard regression uses k-1 dummy variables with a reference category.

Key Takeaway: One-hot encoding creates a dummy variable for every category level; use it for tree-based models and neural networks, but stick with k-1 dummy encoding (with a reference category) for linear regression to avoid multicollinearity.