Calculating the Regression Coefficients
You'll learn how to derive and compute the exact formulas for the slope and intercept of the least squares regression line.
The Core Idea
In the previous lesson, you learned that the least squares criterion minimizes the sum of squared vertical distances between data points and the regression line. Now we'll actually calculate the coefficients that make this happen.
For a simple linear regression model ŷ = b₀ + b₁x, we need two values:
- b₁ (the slope): how much y changes for each unit increase in x
- b₀ (the intercept): where the line crosses the y-axis
The Formulas
Using calculus (taking partial derivatives of the sum of squared errors and setting them to zero), we derive:
Slope:
b₁ = Σ[(xᵢ - x̄)(yᵢ - ȳ)] / Σ[(xᵢ - x̄)²]
Or equivalently:
b₁ = r · (sᵧ / sₓ)
where r is the correlation coefficient, sᵧ is the standard deviation of y, and sₓ is the standard deviation of x.
Intercept:
b₀ = ȳ - b₁x̄
What This Means
The slope formula weights how x and y vary together (covariance) against how much x varies by itself. Think of it as "shared movement divided by x's movement."
The intercept formula ensures the regression line always passes through the point (x̄, ȳ) — the center of your data cloud.
Key Takeaway: The least squares regression coefficients b₁ and b₀ are computed using the means, standard deviations, and correlation of your data, ensuring the line minimizes total squared prediction error.