Course contentsShow
← Machine Learning and Deep Learning
Lesson 200 of 3,5383. Linear RegressionFree lesson

Matrix Formulation of Multiple Linear Regression

Expressing multiple regression as y = Xβ + ε and leveraging linear algebra for compact representation.

Matrix Formulation of Multiple Linear Regression

What you'll learn: How to express multiple linear regression compactly using matrix notation, making computation and understanding much cleaner.

The Power of Matrix Notation

When you have multiple input features (like predicting house price from square footage, number of bedrooms, and age), writing out equations for each data point becomes messy fast. Matrix formulation lets you express the entire regression problem in one elegant line.

The Compact Representation

Instead of writing separate equations for each observation, we stack everything into matrices:

y = Xβ + ε

Here's what each piece means:

  • y: A column vector of all your target values (outputs you're trying to predict)
  • X: A matrix where each row is one data point, each column is one feature (plus a column of 1s for the intercept)
  • β (beta): A column vector of parameters (weights for each feature, plus the intercept)
  • ε (epsilon): A column vector of errors/residuals for each prediction

Why This Matters

Think of X as a spreadsheet: rows are examples, columns are features. The matrix β holds all your coefficients. When you multiply Xβ, linear algebra automatically computes predictions for all data points at once—no loops needed!

This formulation lets you use the Normal Equation you learned earlier, now generalized: β = (XᵀX)⁻¹Xᵀy

It's the same least squares principle, just expressed in the language of linear algebra for multiple features.

Key Takeaway: Matrix formulation y = Xβ + ε turns multiple linear regression from messy multi-variable equations into clean linear algebra operations, enabling efficient computation for any number of features.