Course contentsShow
← Data Science
Lesson 598 of 2,14516. Multiple Linear RegressionFree lesson

Estimating Coefficients with Least Squares

Apply ordinary least squares to estimate β-hat = (X'X)⁻¹X'Y in multiple dimensions.

Estimating Coefficients with Least Squares

What You'll Learn

You'll learn how to apply the ordinary least squares (OLS) formula to estimate all coefficients simultaneously in multiple linear regression using matrix algebra.

The Core Idea

In simple regression, you calculated slope and intercept separately using formulas. In multiple regression with many predictors, that approach becomes unwieldy. Instead, we use one elegant matrix formula that solves for all coefficients at once:

β-hat = (X'X)⁻¹X'Y

Here's what each piece means:

  • β-hat: The vector of estimated coefficients (including the intercept)
  • X: Your design matrix (observations × predictors, with a column of 1s for the intercept)
  • X': The transpose of X (flip rows and columns)
  • Y: Your response vector (actual outcomes)
  • (X'X)⁻¹: The inverse of X'X (think "dividing" in matrix world)

This formula minimizes the sum of squared residuals across all dimensions simultaneously—the same least squares criterion you used before, now extended to multiple predictors.

Why This Matters

Think of it like solving a system of equations. With one predictor, you could solve by hand. With ten predictors, you'd have ten interconnected equations. The matrix formula handles all of them at once, accounting for how predictors relate to each other while finding the best-fit coefficients.

The computation (X'X)⁻¹X'Y essentially finds which combination of predictor weights produces predictions closest to your actual Y values.

Key Takeaway: The ordinary least squares estimator β-hat = (X'X)⁻¹X'Y provides a single, closed-form solution for all regression coefficients in multiple dimensions, extending the least squares principle from simple to multiple regression.