Correlated Features and Permutation Importance
What you'll learn: How feature correlations create unrealistic data distributions when permuted, leading to potential misinterpretation of feature importance.
The Core Problem
When features are correlated in your dataset, permutation importance can produce misleading results. Here's why: permutation importance works by shuffling one feature's values while keeping all others fixed. But when two features naturally move together (like "house square footage" and "number of rooms"), randomly shuffling one breaks their real-world relationship.
Why This Matters
Imagine height and weight in a medical dataset. In reality, taller people tend to weigh more. When you permute height values while keeping weights fixed, you create impossible combinations: 5-foot patients weighing 250 lbs, 6-foot patients weighing 90 lbs. Your model sees data it would never encounter naturally, and the resulting performance drop may not reflect the feature's true importance.
The double-counting trap: If two features are highly correlated and both useful, permuting either one might show low importance because the other can compensate. The importance appears "split" between them, even though each individually would be critical if the other weren't present.
The Unrealistic Distribution Effect
When you permute a correlated feature, you're testing your model on a data distribution that violates the learned covariance structure. The performance drop measures both the feature's true contribution AND the model's confusion from seeing impossible feature combinations. You can't separate these effects.
Key Takeaway: Permutation importance can underestimate or misattribute importance when features are correlated, because shuffling breaks natural relationships and creates unrealistic data distributions the model never learned to handle.