Computing SVD via Eigenvalue Decomposition
You'll learn how to find the singular values and singular vectors of any matrix by computing eigenvalues of related symmetric matrices.
The Core Idea
Remember that the Singular Value Decomposition writes a matrix A as A = UΣV^T, where Σ contains singular values and U, V contain left and right singular vectors. Instead of computing these directly, we leverage what we already know about eigenvalue decomposition of symmetric matrices.
The trick is beautifully simple: A^T A and AA^T are both symmetric matrices whose eigenvalues and eigenvectors directly reveal the SVD components.
The Recipe
Step 1: Find Right Singular Vectors (V)
Compute A^T A. This symmetric matrix has:
- Eigenvalues equal to the squares of the singular values (σ²)
- Eigenvectors that are exactly the right singular vectors (columns of V)
Use the spectral theorem to orthogonally diagonalize A^T A.
Step 2: Extract Singular Values
Take the square roots of the eigenvalues from A^T A:
- If λ is an eigenvalue of A^T A, then σ = √λ is a singular value
- Arrange them in decreasing order in Σ
Step 3: Find Left Singular Vectors (U)
Compute AA^T. This symmetric matrix has:
- The same nonzero eigenvalues as A^T A
- Eigenvectors that are the left singular vectors (columns of U)
Alternatively, once you have V and Σ, you can compute U via u_i = (1/σ_i)Av_i for each pair.
Why This Works
The symmetry of A^T A and AA^T guarantees orthogonal eigenvectors (spectral theorem!), which is exactly what we need for the orthogonal matrices U and V.
Key Takeaway: The SVD of A can be computed by finding eigendecompositions of the symmetric matrices A^T A (for V and singular values) and AA^T (for U).