Influence: Impact on Fitted Model
You'll learn how influence differs from leverage by measuring actual change to your regression model.
Understanding Influence vs Leverage
You've already learned that leverage measures how far a data point sits from the average in the predictor space (X-space). But leverage alone doesn't tell you whether that point actually changes your regression results.
Influence is the real impact: it measures how much the fitted model—the coefficients, predictions, or fitted line itself—would change if you removed that observation and re-fit the model.
Think of it this way: a person might have a high-leverage position (standing far from the crowd) but still agree with everyone's opinion (low influence). Conversely, someone standing in the crowd might be a loud dissenter who changes group consensus (high influence despite low leverage).
The Key Distinction
- High leverage = potential to be influential (unusual X-value)
- High influence = actually changes the fitted model (unusual X and unusual Y given that X)
A point with high leverage becomes influential when its residual is also large—when it doesn't follow the pattern suggested by the other points. It "pulls" the regression line toward itself.
Why This Matters
Influential points can:
- Change coefficient estimates dramatically
- Alter predictions across the entire range
- Mislead your conclusions about relationships
Identifying influence helps you decide whether a point represents:
- A legitimate but unusual case (keep it)
- A data error (investigate/correct)
- An outlier requiring special modeling
Key Takeaway: Influence measures the actual change to your fitted model when removing an observation—it combines leverage (position in X-space) with an unusual Y-value to quantify real impact on coefficients and predictions.