Least Squares with QR Decomposition
You'll learn how to solve least squares problems more efficiently and accurately using QR factorization instead of the normal equations.
Why QR is Better Than Normal Equations
You already know that least squares problems can be solved using the normal equations: A^T A x̂ = A^T b. While this works in theory, it has a serious practical flaw: forming A^T A squares the condition number of your matrix, amplifying numerical errors and potentially causing computational instability.
The QR decomposition provides an elegant alternative. Recall that any matrix A can be factored as A = QR, where Q has orthonormal columns and R is upper triangular.
How It Works
When we write the least squares problem ||Ax - b||² and substitute A = QR, we get:
||QRx - b||²
Since Q has orthonormal columns, multiplying by Q^T preserves norms (think of Q as a rigid rotation—it doesn't stretch or compress). The least squares solution becomes:
Rx̂ = Q^T b
This is just an upper triangular system! You can solve it efficiently with back-substitution—no matrix inversion needed, and no squaring of condition numbers.
The Advantage
Think of it like this: the normal equations are like photocopying a photocopy—quality degrades. QR decomposition works with the original matrix directly, preserving numerical accuracy. You avoid the dangerous A^T A computation entirely.
Key Takeaway: Use QR factorization to solve least squares problems: factor A = QR, compute Q^T b, then solve the triangular system Rx̂ = Q^T b. This method is numerically stable and avoids the conditioning problems of normal equations.