Applications: Linear Regression
What You'll Learn
You'll learn how to use the least squares method to find the best-fitting straight line through a scatter of data points.
The Core Problem
Imagine you've collected real-world data—say, hours studied vs. exam scores for 20 students. When you plot these points, they don't fall perfectly on a line. But you want to find the line that best represents the trend. How do you define "best"?
Linear regression answers this by finding the line that minimizes the total squared vertical distance from each data point to the line. This is exactly the least squares problem you've been studying!
From Data to Linear Model
Suppose you have data points . You want to fit a line .
For each data point, there's a residual (error): the difference between the actual and the predicted value .
We want to minimize:
This is a least squares problem! In matrix form:
where has rows , , and .
Solving It
Use the normal equations:
Solving this gives you the optimal slope and intercept . The projection of onto the column space of gives the "closest possible" predictions.
Why This Matters
Linear regression appears everywhere: predicting sales, analyzing trends, fitting scientific models. The least squares framework guarantees you get the line that minimizes prediction error in the squared-distance sense.
Key Takeaway: Linear regression applies least squares to find the best-fit line through data by minimizing the sum of squared residuals—transforming a practical curve-fitting problem into a solvable linear algebra equation.