Course contentsShow
← Machine Learning and Deep Learning
Lesson 2357 of 3,53851. Recommender SystemsFree lesson

Alternating Least Squares

Iteratively optimizing user and item factors by alternating between fixing one while solving for the other.

Alternating Least Squares

What you'll learn: How to efficiently solve matrix factorization by alternately optimizing user and item factors while keeping the other fixed.

The Core Idea

In Matrix Factorization, we decompose our rating matrix into user factors and item factors. But optimizing both simultaneously is complex and non-convex. Alternating Least Squares (ALS) cleverly sidesteps this by breaking the problem into two simpler steps:

  1. Fix item factors, solve for user factors (this becomes a linear least squares problem)
  2. Fix user factors, solve for item factors (again, linear least squares)
  3. Repeat until convergence

Think of it like two friends organizing a party: one person finalizes the guest list while the other picks the venue based on that list, then they switch—the venue-picker holds their choice constant while the list-maker adjusts based on the venue. They alternate until both are satisfied.

Why This Works

When you fix one set of factors, the optimization problem transforms from non-convex to convex—specifically, a least squares problem with a closed-form solution. This makes each step computationally efficient and guaranteed to improve (or maintain) the objective.

The algorithm alternates between these two easy problems rather than tackling one hard problem. Each iteration reduces the overall reconstruction error until the factors converge to a good representation of user preferences and item characteristics.

The Mathematics

For each user, solve: minimize ||ratings - user_factor × item_factors||²

For each item, solve: minimize ||ratings - user_factors × item_factor||²

Both have analytical solutions, making ALS particularly fast for large-scale systems.

Key Takeaway: Alternating Least Squares makes matrix factorization tractable by alternately fixing one factor matrix and solving a simple least squares problem for the other, turning a hard non-convex optimization into a series of easy convex ones.