Course contentsShow
← AI Engineering
Lesson 273 of 1,8867. Vector Databases - FundamentalsFree lesson

Diversity and MMR in Search Results

Using Maximal Marginal Relevance to balance relevance with result diversity.

Diversity and MMR in Search Results

What You'll Learn

You'll learn how to use Maximal Marginal Relevance (MMR) to prevent your search results from being too repetitive, balancing similarity with variety.

The Problem with Pure Similarity Search

When you retrieve the top-K most similar vectors, you often get results that are too similar to each other. Imagine searching for "python programming" and getting back 10 tutorial articles that all say nearly the same thing—technically relevant, but not very useful.

Think of it like this: If you ask for restaurant recommendations and get five Italian restaurants all on the same street, you're missing out on diversity even though they're all "relevant" to food.

What is MMR?

Maximal Marginal Relevance is a re-ranking technique that balances two competing goals:

  1. Relevance: How similar is this result to your query?
  2. Diversity: How different is this result from what you've already selected?

MMR picks results iteratively. First, it chooses the most relevant item. Then, for each subsequent choice, it looks for items that are:

  • Still reasonably relevant to your query
  • Different from what's already been selected

The Trade-off Parameter

MMR uses a parameter (commonly λ, lambda) between 0 and 1:

  • λ = 1: Pure relevance (ignore diversity)
  • λ = 0: Maximum diversity (ignore relevance)
  • λ = 0.5: Balanced approach

This lets you tune how much you value variety versus strict similarity.

Why This Matters

In RAG systems and search applications, diverse results give users broader context and reduce redundancy. Instead of five nearly-identical documents, you get five perspectives on the same topic.

Key Takeaway: MMR helps you avoid the "echo chamber" effect in vector search by iteratively selecting results that are relevant to the query but dissimilar to each other, controlled by a single diversity parameter.