Reduced and Full SVD
What you'll learn: You'll understand the difference between full and reduced SVD forms and when each is most appropriate.
When you compute the Singular Value Decomposition of a matrix A (size m×n), you actually have two choices for how to express it: full SVD and reduced (economical) SVD.
Full SVD
The full SVD represents A as:
A = UΣV^T
where:
- U is m×m (full orthogonal matrix)
- Σ is m×n (with singular values and extra zero rows/columns)
- V^T is n×n (full orthogonal matrix)
This form gives you complete orthonormal bases for all relevant spaces. U spans the entire m-dimensional space, and V spans the entire n-dimensional space—even the parts that don't contribute to A's action.
Reduced (Economical) SVD
The reduced SVD keeps only the meaningful parts:
A = U_r Σ_r V_r^T
where r = min(m,n):
- U_r is m×r (only the first r columns of U)
- Σ_r is r×r (square diagonal matrix with singular values)
- V_r^T is r×n (only the first r rows of V^T)
Think of it like packing for a trip: full SVD is bringing your entire closet "just in case," while reduced SVD packs only what you'll actually wear.
When to Use Each
Use reduced SVD when:
- Working with tall/skinny or short/wide matrices
- Memory and computation efficiency matter
- You only need the active transformation part
Use full SVD when:
- You need complete orthonormal bases for theoretical work
- Analyzing all four fundamental subspaces
- The matrix is close to square anyway
Key Takeaway: Reduced SVD is computationally cheaper and strips away the zeros, keeping only what contributes to the transformation; full SVD provides complete orthonormal bases but includes extra dimensions that map to zero.