Coverage and Diversity Metrics
What you'll learn: How to measure whether your recommender system explores the full catalog rather than just pushing popular items to everyone.
Why These Metrics Matter
Imagine a music streaming service that only recommends the top 10 most-streamed songs to every user. Your accuracy metrics (like NDCG) might look decent—users do click on popular songs—but you've created a boring, homogeneous experience that ignores niche interests and long-tail content.
Coverage and diversity metrics help you detect and prevent this concentration problem.
Coverage: Catalog Reach
Coverage measures what percentage of your item catalog actually gets recommended to users over time.
- Catalog coverage = (Number of unique items recommended) / (Total items in catalog)
- A coverage of 100% means every item gets recommended at least once
- Low coverage suggests your system is stuck recommending the same popular items repeatedly
Think of it like a library: if patrons only ever check out 5% of the books, most of your collection is wasted.
Diversity: Avoiding Echo Chambers
Diversity measures how varied the recommendations are within a single user's recommendation list or across different users.
- Intra-list diversity: How different items are within one user's top-K recommendations
- Inter-user diversity: How different recommendation lists are between users
You can measure diversity using the average pairwise distance between recommended items (using content features or embedding similarity).
High diversity means users see varied options; low diversity means repetitive suggestions.
The Trade-off
These metrics often trade off with accuracy: pushing users toward less popular but relevant items may lower immediate click rates but improve long-term satisfaction and catalog utilization.
Key Takeaway: Coverage and diversity metrics ensure your recommender system doesn't over-concentrate on popular items, helping you balance accuracy with catalog exploration and user experience variety.