Course contentsShow
← Machine Learning and Deep Learning
Lesson 3200 of 3,53870. Interpretability: Feature-Based MethodsFree lesson

Train vs Test Set Permutation

Why permutation importance should be computed on test/validation data to avoid misleading importance scores from overfitting.

Train vs Test Set Permutation

What you'll learn: Why permutation importance must be computed on test or validation data—not training data—to get honest feature importance scores.

The Core Problem

When you compute permutation importance, you shuffle a feature's values and measure how much model performance drops. But where you compute this matters enormously.

If you permute features on your training set, you're measuring how much the model relied on those features to fit the training data—including all the noise and overfitting. An overfit model might show high importance for features that just happened to correlate with training labels by chance.

If you permute on the test/validation set, you measure how much the feature truly matters for generalization to unseen data. This is what you actually care about.

Why Training Set Permutation Misleads

Think of it like grading a student's study habits. If you only observe them during practice tests they've already seen, you can't tell which study methods actually help versus which just helped them memorize specific questions. You need to test them on new material.

A feature might look crucial on training data simply because your model memorized patterns involving it. When permuted on test data, that same feature might show little importance—revealing the model learned spurious correlations that don't generalize.

Best Practice

Always compute permutation importance on a held-out test or validation set. The performance drop you observe will reflect the feature's true contribution to generalization, not its role in overfitting.

Key Takeaway: Compute permutation importance on test/validation data to measure true predictive value; training set permutation conflates importance with overfitting and gives misleading scores.