Course contentsShow
← Mathematics
Lesson 2288 of 3,12147. Linear Algebra — Applications and Advanced TopicsFree lesson

Low-Rank Approximation and Data Compression

Truncate small singular values to obtain the best low-rank approximation in the Frobenius norm.

Low-Rank Approximation and Data Compression

You'll learn how to create the best possible low-rank approximation of a matrix by strategically discarding small singular values from the SVD.

The Core Idea

Once you have the Singular Value Decomposition (SVD) of a matrix A = UΣVᵀ, you can create a simpler, compressed version by keeping only the k largest singular values and setting the rest to zero. This gives you the best rank-k approximation of A in the Frobenius norm—no other rank-k matrix is closer to A.

Think of it like a photograph: the largest singular values capture the "main features" (major shapes, contrasts), while smaller singular values represent fine details or noise. By truncating small singular values, you keep the essence while discarding less important information, dramatically reducing storage requirements.

How It Works

Given A = UΣVᵀ where the singular values are ordered σ₁ ≥ σ₂ ≥ ... ≥ σᵣ > 0:

  1. Choose k < r (the desired rank)
  2. Keep only the first k singular values in Σ, setting all others to zero
  3. Form Aₖ = UₖΣₖVₖᵀ, using only the first k columns of U and V

The matrix Aₖ is the closest rank-k matrix to A. The approximation error is exactly σₖ₊₁ (the first discarded singular value).

Why This Matters

This technique powers:

  • Image compression: Store large images with fewer numbers
  • Data analysis: Extract dominant patterns from noisy datasets
  • Recommendation systems: Compress user-rating matrices
  • Machine learning: Reduce dimensionality while preserving information

Key Takeaway: Truncating the SVD by keeping only the k largest singular values produces the optimal rank-k approximation of your matrix, enabling efficient data compression while minimizing information loss.