Applications: Data Fitting and Approximation
You'll learn how to use least squares projection to find the best-fit polynomial or function for noisy data, turning impossible-to-solve overdetermined systems into practical approximations.
The Core Idea
Imagine you have 100 temperature measurements throughout the day, but you want a simple polynomial curve that captures the overall trend. You have far more data points than polynomial coefficients—this is an overdetermined system with no exact solution. Instead of giving up, we use least squares projection to find the coefficients that minimize the total error.
The magic: by projecting your data vector onto the column space of your design matrix (the matrix built from your basis functions), you find the approximation that's as close as possible to the real data.
How It Works
When fitting data points to a polynomial :
- Build your design matrix where each row contains powers of an value:
- Set up the system where contains your data values
- Solve the normal equations to get optimal coefficients
- The resulting polynomial minimizes the sum of squared vertical distances to your data points
This same framework works for approximating functions using orthogonal polynomials, trigonometric series, or any set of basis functions—you're always projecting onto the span of your chosen basis.
Key Takeaway: Least squares transforms impossible overdetermined problems into optimization problems, finding the "best approximate solution" by orthogonally projecting data onto the space of candidate functions—enabling practical curve fitting, data smoothing, and function approximation throughout science and engineering.