Similarity Metrics for Collaborative Filtering
What you'll learn: How to measure similarity between users or items in collaborative filtering using three fundamental metrics: cosine similarity, Pearson correlation, and Jaccard index.
Why Similarity Matters
In collaborative filtering, you need to find users with similar tastes or items with similar rating patterns. Think of it like finding friends who enjoy the same movies you do—the more aligned your preferences, the better their recommendations will be for you.
Three Core Similarity Metrics
Cosine Similarity
Measures the angle between rating vectors, focusing on direction rather than magnitude. Two users who rate items proportionally similarly (e.g., one always rates 2 points higher) still have high cosine similarity. Values range from -1 to 1, where 1 means identical preferences.
Use when: Rating scales differ between users, or you care about preference patterns more than absolute values.
Pearson Correlation
Captures linear relationships between rating patterns after centering them around each user's mean. This accounts for users who are "harsh critics" (low average ratings) versus "generous raters" (high averages). Values also range from -1 to 1.
Use when: Users have different rating tendencies, and you want to normalize for individual biases.
Jaccard Index
Measures overlap in binary preferences—what percentage of items rated by both users/items are shared? Ignores rating values entirely, only considering presence/absence. Values range from 0 to 1.
Use when: You only know if someone liked/purchased something, not how much (implicit feedback scenarios).
Choosing Your Metric
Cosine similarity works well for sparse rating matrices. Pearson correlation handles rating bias better but requires more shared ratings. Jaccard index suits binary interaction data like clicks or purchases.
Key Takeaway: The right similarity metric depends on your data characteristics—use cosine for directional patterns, Pearson for normalized linear relationships, and Jaccard for binary overlap in collaborative filtering systems.