The Pseudoinverse via SVD
You'll learn how to construct the Moore-Penrose pseudoinverse using Singular Value Decomposition to solve systems that don't have ordinary solutions.
What's the Problem?
When you encounter a linear system Ax = b that's inconsistent (no exact solution) or has infinitely many solutions, the standard matrix inverse doesn't help—it may not even exist. You need something more flexible: the pseudoinverse A⁺.
The SVD Construction
Remember that any matrix A has a Singular Value Decomposition:
A = UΣV^T
where U and V are orthogonal matrices containing left and right singular vectors, and Σ is diagonal with singular values.
The Moore-Penrose pseudoinverse is elegantly constructed by:
A⁺ = VΣ⁺U^T
Notice we've "reversed" the order and used Σ⁺ instead of Σ⁻¹.
What is Σ⁺?
Here's the key: Σ⁺ is formed by:
- Taking the reciprocal of each non-zero singular value
- Transposing the resulting matrix
For example, if Σ has singular values [5, 3, 0], then Σ⁺ replaces them with [1/5, 1/3, 0] in the transposed position. Zero singular values stay zero—this handles rank deficiency gracefully.
Why This Solves Inconsistent Systems
When you compute x = A⁺b, you get the least-squares solution—the vector that makes Ax as close to b as possible. If multiple solutions exist, it gives you the one with minimum norm (shortest length).
This works because SVD reveals the true "structure" of your matrix, and the pseudoinverse inverts what can be inverted while safely handling the parts that can't be.
Key Takeaway: The pseudoinverse A⁺ = VΣ⁺U^T extends matrix inversion to all matrices, providing best-fit solutions to inconsistent or underdetermined systems by inverting only the non-zero singular values.