Course contentsShow
← Machine Learning and Deep Learning
Lesson 23 of 3,5381. Mathematical Foundations for Machine LearningFree lesson

Computing and Interpreting SVD

Learn the mechanics of SVD computation and how singular values represent importance of directions.

Computing and Interpreting SVD

What you'll learn: How to break down matrices using SVD and understand what those singular values actually mean.

The Mechanics of SVD

You already know that Singular Value Decomposition (SVD) factorizes any matrix into three components. Now let's see how it works in practice.

For any matrix A (m×n), SVD gives you: A = UΣV^T

Where:

  • U (m×m): Orthogonal matrix whose columns are left singular vectors (directions in output space)
  • Σ (m×n): Diagonal matrix containing singular values (σ₁ ≥ σ₂ ≥ ... ≥ 0)
  • V^T (n×n): Orthogonal matrix whose rows are right singular vectors (directions in input space)

What Singular Values Tell You

Think of singular values as importance scores for different directions in your data. The largest singular value (σ₁) corresponds to the direction of maximum variance—the most important pattern. The second largest (σ₂) captures the next most important pattern, orthogonal to the first, and so on.

Analogy: Imagine compressing a photo. The largest singular values capture the essential structure (edges, main shapes), while tiny singular values represent fine details or noise. Drop the smallest values, and you still recognize the image—that's dimensionality reduction in action!

Interpreting the Decomposition

  • Large singular values → Important directions that capture significant variation
  • Small singular values → Less important directions, possibly noise
  • Zero singular values → Dimensions that contribute nothing (related to rank)

The rank of matrix A equals the number of non-zero singular values—connecting back to what you learned about matrix rank and column space.

Key Takeaway: SVD computation reveals hidden structure by ordering directions from most to least important, with singular values quantifying each direction's significance for representing your data.