Course contentsShow
← Machine Learning and Deep Learning
Lesson 448 of 3,53810. Feature EngineeringFree lesson

Recursive Feature Elimination

Recursively train models and remove the least important features to find the optimal feature set.

Recursive Feature Elimination

What you'll learn: How to systematically eliminate weak features by repeatedly training models and removing the least important ones until you find the optimal feature set.

The Core Idea

Imagine you're assembling a sports team and need to figure out which players contribute most to winning. You could start with everyone, play a season, then cut the weakest player. Repeat this process until performance starts dropping—that's when you know you've found your core team.

Recursive Feature Elimination (RFE) works exactly this way with your dataset's features. It's a wrapper method that:

  1. Trains a model on all features
  2. Ranks features by importance (using the model's built-in scoring)
  3. Removes the least important feature(s)
  4. Repeats steps 1-3 until reaching a desired number of features

Why This Matters

RFE automates the backward selection process you learned earlier, but with a smarter approach. Instead of manually testing each removal, RFE leverages model-specific importance scores (like coefficients in linear models or feature importances in trees) to guide elimination.

The "recursive" part means it continuously refines its understanding—after removing weak features, the model retrains and may discover that other features become more or less important in the new context.

The Process in Practice

Start with 20 features → train model → eliminate 2 weakest → train on 18 features → eliminate 2 more → continue until you have your target number (say, 10 features) or until performance degrades.

Key Takeaway: Recursive Feature Elimination automates feature selection by iteratively training models, ranking features by importance, and pruning the weakest ones—giving you a data-driven path to an optimal, compact feature set.