Course contentsShow
← Mathematics
Lesson 3073 of 3,12160. Statistics — ModelingFree lesson

The Matrix Formulation of Multiple Regression

Express the multiple regression model using matrices: Y = Xβ + ε; understand the design matrix and parameter vector.

The Matrix Formulation of Multiple Regression

You'll learn how to express multiple linear regression compactly using matrix notation, making both theory and computation much cleaner.

Why Matrices?

When you have multiple predictors (independent variables), writing out each equation by hand becomes tedious. For example, with three predictors:

y₁ = β₀ + β₁x₁₁ + β₂x₁₂ + β₃x₁₃ + ε₁
y₂ = β₀ + β₁x₂₁ + β₂x₂₂ + β₃x₂₃ + ε₂
...

This gets unwieldy fast! Matrix notation lets us condense everything into one elegant equation.

The Matrix Equation: Y = Xβ + ε

Instead of many individual equations, we represent the entire regression model as:

Y = Xβ + ε

Breaking Down Each Component

Y (the response vector): An n×1 column vector containing all observed response values [y₁, y₂, ..., yₙ]ᵀ

X (the design matrix): An n×(p+1) matrix where:

  • Each row represents one observation
  • The first column is all 1's (for the intercept β₀)
  • Columns 2 through (p+1) contain the predictor values
  • If you have 100 observations and 3 predictors, X is 100×4

β (the parameter vector): A (p+1)×1 column vector containing the coefficients [β₀, β₁, ..., βₚ]ᵀ we want to estimate

ε (the error vector): An n×1 column vector of random errors [ε₁, ε₂, ..., εₙ]ᵀ

The Big Picture

Think of the design matrix X as your "data table"—rows are observations, columns are predictors (plus that intercept column of 1's). When you multiply X by β, you're generating predictions for all observations at once. The error vector ε captures the difference between predictions and reality.

Key Takeaway: The matrix formulation Y = Xβ + ε expresses the entire multiple regression model compactly, with X (design matrix) containing predictors, β (parameter vector) containing coefficients, and ε representing errors.