Course contentsShow
← Machine Learning and Deep Learning
Lesson 2370 of 3,53851. Recommender SystemsFree lesson

Self-Attention for Recommendation (SASRec)

Use self-attention mechanisms to model user behavior sequences and long-range dependencies effectively.

Self-Attention for Recommendation (SASRec)

What You'll Learn

You'll understand how self-attention mechanisms capture long-range dependencies in user behavior sequences, enabling more powerful sequential recommendations than RNNs alone.

The Problem with Sequential RNNs

In the previous lesson, you learned about using RNNs for sequential recommendations. However, RNNs process items one-by-one in order, which makes them struggle with long-range dependencies. If a user bought hiking boots 50 items ago and now you're recommending camping gear, an RNN might have "forgotten" that crucial early signal.

Enter Self-Attention

SASRec (Self-Attentive Sequential Recommendation) applies the self-attention mechanism—the core of Transformer models—to user behavior sequences. Instead of processing items sequentially, self-attention lets each item in the sequence "look at" all other items simultaneously and decide which ones are most relevant.

Think of it like this: rather than reading a user's purchase history word-by-word like RNNs, self-attention reads the entire history at once, highlighting the most relevant past purchases for predicting the next item—even if they happened far apart.

How It Works

For a sequence of items a user has interacted with, SASRec:

  1. Embeds each item into a vector representation
  2. Applies positional encoding so the model knows the order
  3. Uses self-attention layers where each item computes attention weights over all previous items
  4. Stacks multiple attention layers to capture complex patterns
  5. Predicts the next item the user will interact with

The attention mechanism specifically learns which past items matter most for predicting future behavior, regardless of how far back they occurred in the sequence.

Key Takeaway: SASRec uses self-attention to model user sequences, allowing the model to capture long-range dependencies and item relationships more effectively than RNNs by attending to all past interactions simultaneously rather than sequentially.