Course contentsShow
← Machine Learning and Deep Learning
Lesson 3187 of 3,53870. Interpretability: Feature-Based MethodsFree lesson

Linear Model Coefficients as Importance

Use coefficients in linear regression and logistic regression to interpret feature importance, accounting for feature scaling.

Linear Model Coefficients as Importance

What you'll learn: How to extract and interpret feature importance directly from linear and logistic regression coefficients, while accounting for feature scaling effects.

The Core Idea

In linear models (linear regression, logistic regression), each feature has a coefficient (weight) that quantifies its contribution to the prediction. A coefficient tells you: "If this feature increases by 1 unit, the prediction changes by this amount."

At first glance, the magnitude of a coefficient seems like a natural measure of feature importance—bigger coefficient means bigger impact, right? Almost. The critical catch: coefficients are scale-dependent. A feature measured in millimeters will have a much larger coefficient than the same feature measured in kilometers, even though they represent identical information.

Making Coefficients Comparable

To use coefficients as importance scores, you must ensure all features are on the same scale. Common approaches:

  • Standardization (z-score normalization): Transform features to have mean=0 and standard deviation=1
  • Min-max scaling: Scale features to a fixed range like [0, 1]

After standardization, coefficient magnitudes become directly comparable. A coefficient of 2.5 now genuinely means "more important" than a coefficient of 0.8, because both operate on standardized units.

Interpretation Notes

  • Sign matters: In linear regression, positive coefficients increase predictions; negative decrease them
  • Logistic regression: Coefficients affect log-odds, not probabilities directly
  • Absolute value: Use |coefficient| to rank importance when direction doesn't matter
  • Linear assumptions: This only works because linear models explicitly encode each feature's marginal effect

Key Takeaway: Linear model coefficients provide direct, interpretable feature importance scores, but only when features are properly scaled to comparable units—otherwise you're comparing apples measured in inches to oranges measured in miles.