Feature Importance Normalization
You'll learn how to normalize feature importance scores so they're comparable across different features, models, and evaluation runs.
Why Normalize Feature Importances?
Raw importance scores from different methods often live on completely different scales. Tree-based MDI might give you values in the thousands, permutation importance could range from -0.1 to 0.5, and linear coefficients might be tiny decimals. Without normalization, you can't meaningfully compare which feature is "twice as important" or judge relative contributions.
Think of it like converting currencies: €100 and $100 represent different purchasing power. Similarly, an MDI score of 1000 and a permutation score of 0.5 need a common scale to be compared.
Common Normalization Techniques
Sum-to-One Normalization
Divide each importance score by the sum of all importance scores. This makes importances interpretable as percentage contributions:
normalized_importance[i] = raw_importance[i] / sum(raw_importance)
Min-Max Scaling
Scale all scores to [0, 1] range, where the most important feature gets 1.0:
normalized = (score - min_score) / (max_score - min_score)
Standardization (Z-score)
Center around zero with unit variance — useful when comparing across different model runs or datasets.
When to Use Each Method
- Sum-to-one: When you want relative percentage contributions
- Min-max: When you want easy visual comparison (all features on same 0-1 axis)
- Standardization: When comparing importance patterns across different models or datasets
Key Takeaway: Normalizing feature importances transforms raw scores into comparable units, enabling meaningful cross-feature and cross-model interpretation — choose your normalization strategy based on whether you need relative contributions, fixed ranges, or distributional comparisons.