Course contentsShow
← Machine Learning and Deep Learning
Lesson 2356 of 3,53851. Recommender SystemsFree lesson

Singular Value Decomposition for Recommendations

Applying SVD to approximate the rating matrix and handle missing values in collaborative filtering.

Singular Value Decomposition for Recommendations

What you'll learn: How SVD decomposes rating matrices into latent factors to predict missing ratings and power collaborative filtering systems.

The Core Idea

Singular Value Decomposition (SVD) takes your sparse, incomplete user-item rating matrix and breaks it down into three smaller matrices that capture hidden patterns. Instead of storing every individual rating, SVD discovers latent features—underlying factors like "how much does this user like action movies" or "how sci-fi is this movie"—without being told what those factors are.

The magic formula: R ≈ U × Σ × V^T

  • R: Your original ratings matrix (users × items)
  • U: User-to-latent-features matrix (users × k features)
  • Σ: Diagonal matrix of "importance" weights (k × k)
  • V^T: Latent-features-to-items matrix (k × items)

Here, k is much smaller than your original dimensions—you might reduce 10,000 movies and 1 million users down to just 50 latent factors.

Why This Matters for Recommendations

Your rating matrix is mostly empty—most users haven't rated most items. SVD fills in the blanks by learning patterns from existing ratings. If User A and User B both loved sci-fi movies, and User A loved Blade Runner (which User B hasn't seen), the latent factors will capture this shared preference and predict User B would also enjoy it.

Think of it like compressing a photo: you lose some detail but keep the essential structure, and now you can "reconstruct" pixels (ratings) that were never there.

The Approximation Advantage

By keeping only the top k singular values (the largest entries in Σ), you filter out noise and focus on the strongest patterns. This regularizes your model and prevents overfitting to the sparse data you have.

Key Takeaway: SVD decomposes sparse rating matrices into compact latent factor representations that capture user preferences and item characteristics, enabling accurate prediction of missing ratings through low-rank approximation.