Course contentsShow
← Mathematics
Lesson 2239 of 3,12146. Linear Algebra — Inner Products and OrthogonalityFree lesson

Applications: Data Fitting and Approximation

Use least squares projection to approximate functions, fit polynomials, and solve overdetermined systems.

Applications: Data Fitting and Approximation

You'll learn how to use least squares projection to find the best-fit polynomial or function for noisy data, turning impossible-to-solve overdetermined systems into practical approximations.

The Core Idea

Imagine you have 100 temperature measurements throughout the day, but you want a simple polynomial curve that captures the overall trend. You have far more data points than polynomial coefficients—this is an overdetermined system with no exact solution. Instead of giving up, we use least squares projection to find the coefficients that minimize the total error.

The magic: by projecting your data vector onto the column space of your design matrix (the matrix built from your basis functions), you find the approximation that's as close as possible to the real data.

How It Works

When fitting data points (x1,y1),(x2,y2),…,(xm,ym)(x_1, y_1), (x_2, y_2), \ldots, (x_m, y_m) to a polynomial p(x)=c0+c1x+c2x2+⋯+cnxnp(x) = c_0 + c_1 x + c_2 x^2 + \cdots + c_n x^n:

  1. Build your design matrix AA where each row contains powers of an xix_i value: [1,xi,xi2,…,xin][1, x_i, x_i^2, \ldots, x_i^n]
  2. Set up the system Ac≈yA\mathbf{c} \approx \mathbf{y} where y\mathbf{y} contains your data values
  3. Solve the normal equations ATAc=ATyA^T A \mathbf{c} = A^T \mathbf{y} to get optimal coefficients
  4. The resulting polynomial minimizes the sum of squared vertical distances to your data points

This same framework works for approximating functions using orthogonal polynomials, trigonometric series, or any set of basis functions—you're always projecting onto the span of your chosen basis.

Key Takeaway: Least squares transforms impossible overdetermined problems into optimization problems, finding the "best approximate solution" by orthogonally projecting data onto the space of candidate functions—enabling practical curve fitting, data smoothing, and function approximation throughout science and engineering.