Course contentsShow
← Data Science
Lesson 597 of 2,14516. Multiple Linear RegressionFree lesson

Matrix Representation of Multiple Regression

Express multiple regression using matrix notation: Y = Xβ + ε for computational efficiency.

Matrix Representation of Multiple Regression

What You'll Learn

You'll learn how to express multiple regression compactly using matrices, making computation faster and notation cleaner.

Why Use Matrices?

When you have multiple predictors, writing out each term gets cumbersome. Instead of writing:

y₁ = β₀ + β₁x₁₁ + β₂x₁₂ + ... + βₚx₁ₚ + ε₁
y₂ = β₀ + β₁x₂₁ + β₂x₂₂ + ... + βₚx₂ₚ + ε₂
...

...for every observation, we package everything into matrices.

The Matrix Form: Y = Xβ + ε

Y (response vector): An n×1 column vector of all outcome values
X (design matrix): An n×(p+1) matrix where each row is one observation, and columns are predictors (first column is all 1s for the intercept)
β (coefficient vector): A (p+1)×1 vector of parameters [β₀, β₁, ..., βₚ]
ε (error vector): An n×1 vector of residuals

Think of it like an assembly line: X holds your raw materials (data), β contains your recipe (coefficients), and Y is your final product (predictions plus some noise ε).

Why This Matters

Matrix notation lets computers solve regression in one operation instead of looping through equations. The least squares solution becomes simply:

β̂ = (XᵀX)⁻¹XᵀY

This single formula works whether you have 2 predictors or 200. It's the computational backbone of modern regression software.

Example Structure

For 3 observations with 2 predictors:

  • Y: [y₁, y₂, y₃]ᵀ
  • X: [[1, x₁₁, x₁₂], [1, x₂₁, x₂₂], [1, x₃₁, x₃₂]]
  • β: [β₀, β₁, β₂]ᵀ

Key Takeaway: Matrix notation Y = Xβ + ε compresses multiple regression into a single equation, enabling efficient computation and elegant mathematical derivation of solutions.