Course contentsShow
← Data Science
Lesson 574 of 2,14515. Regression DiagnosticsFree lesson

Influence: Impact on Fitted Model

Distinguishing influence from leverage—how removing an observation changes coefficient estimates or predictions.

Influence: Impact on Fitted Model

You'll learn how influence differs from leverage by measuring actual change to your regression model.

Understanding Influence vs Leverage

You've already learned that leverage measures how far a data point sits from the average in the predictor space (X-space). But leverage alone doesn't tell you whether that point actually changes your regression results.

Influence is the real impact: it measures how much the fitted model—the coefficients, predictions, or fitted line itself—would change if you removed that observation and re-fit the model.

Think of it this way: a person might have a high-leverage position (standing far from the crowd) but still agree with everyone's opinion (low influence). Conversely, someone standing in the crowd might be a loud dissenter who changes group consensus (high influence despite low leverage).

The Key Distinction

  • High leverage = potential to be influential (unusual X-value)
  • High influence = actually changes the fitted model (unusual X and unusual Y given that X)

A point with high leverage becomes influential when its residual is also large—when it doesn't follow the pattern suggested by the other points. It "pulls" the regression line toward itself.

Why This Matters

Influential points can:

  • Change coefficient estimates dramatically
  • Alter predictions across the entire range
  • Mislead your conclusions about relationships

Identifying influence helps you decide whether a point represents:

  • A legitimate but unusual case (keep it)
  • A data error (investigate/correct)
  • An outlier requiring special modeling

Key Takeaway: Influence measures the actual change to your fitted model when removing an observation—it combines leverage (position in X-space) with an unusual Y-value to quantify real impact on coefficients and predictions.