Course contentsShow
← Data Science
Lesson 540 of 2,14514. Simple Linear RegressionFree lesson

The Residual Formula

Express residuals mathematically as e_i = y_i - ŷ_i; understand positive vs negative residuals.

The Residual Formula

What you'll learn: How to mathematically express the difference between observed and predicted values using the residual formula.

The Mathematical Expression

You've already learned that residuals represent the distance between actual observations and the fitted line. Now let's formalize this with the residual formula:

e_i = y_i - ŷ_i

Where:

  • e_i is the residual for the i-th observation
  • y_i is the actual observed value
  • ŷ_i (pronounced "y-hat") is the predicted/fitted value from your regression line

This simple subtraction tells you exactly how far off your prediction was.

Understanding Positive vs Negative Residuals

Positive residuals (e_i > 0) occur when the actual value is above the fitted line. Your model underpredicted — the reality was higher than expected.

Negative residuals (e_i < 0) occur when the actual value is below the fitted line. Your model overpredicted — the reality was lower than expected.

Zero residuals (e_i = 0) mean your prediction was exactly correct (rare in practice!).

Example

If your model predicts a house will sell for 300,000(y^=300,000)butitactuallysellsfor300,000 (ŷ = 300,000) but it actually sells for 320,000 (y = 320,000):

e = 320,000 - 300,000 = +20,000

The positive residual shows your model underestimated by $20,000.

Why This Matters

The residual formula is the foundation for evaluating model performance. Each residual quantifies one specific prediction error, and analyzing the pattern of all residuals reveals how well your linear model fits the data.

Key Takeaway: The residual formula e_i = y_i - ŷ_i expresses prediction error as observed minus predicted; positive residuals indicate underprediction, negative residuals indicate overprediction.