Matrix Representation of Multiple Regression
What You'll Learn
You'll learn how to express multiple regression compactly using matrices, making computation faster and notation cleaner.
Why Use Matrices?
When you have multiple predictors, writing out each term gets cumbersome. Instead of writing:
y₁ = β₀ + β₁x₁₁ + β₂x₁₂ + ... + βₚx₁ₚ + ε₁
y₂ = β₀ + β₁x₂₁ + β₂x₂₂ + ... + βₚx₂ₚ + ε₂
...
...for every observation, we package everything into matrices.
The Matrix Form: Y = Xβ + ε
Y (response vector): An n×1 column vector of all outcome values
X (design matrix): An n×(p+1) matrix where each row is one observation, and columns are predictors (first column is all 1s for the intercept)
β (coefficient vector): A (p+1)×1 vector of parameters [β₀, β₁, ..., βₚ]
ε (error vector): An n×1 vector of residuals
Think of it like an assembly line: X holds your raw materials (data), β contains your recipe (coefficients), and Y is your final product (predictions plus some noise ε).
Why This Matters
Matrix notation lets computers solve regression in one operation instead of looping through equations. The least squares solution becomes simply:
β̂ = (XᵀX)⁻¹XᵀY
This single formula works whether you have 2 predictors or 200. It's the computational backbone of modern regression software.
Example Structure
For 3 observations with 2 predictors:
- Y: [y₁, y₂, y₃]ᵀ
- X: [[1, x₁₁, x₁₂], [1, x₂₁, x₂₂], [1, x₃₁, x₃₂]]
- β: [β₀, β₁, β₂]ᵀ
Key Takeaway: Matrix notation Y = Xβ + ε compresses multiple regression into a single equation, enabling efficient computation and elegant mathematical derivation of solutions.