Course contentsShow
← Machine Learning and Deep Learning
Lesson 2378 of 3,53851. Recommender SystemsFree lesson

Hit Rate and Mean Reciprocal Rank (MRR)

Learn binary success metrics: whether any relevant item appears and at what position.

Hit Rate and Mean Reciprocal Rank (MRR)

What you'll learn: How to measure whether your recommender system found any relevant item and how quickly it surfaced the first good result.

The Binary Success Question

While metrics like NDCG@K consider the quality of the entire ranked list, sometimes you just need to answer simpler questions: "Did we show the user anything relevant?" and "How far did they have to scroll to find it?"

Hit Rate (also called Hit Ratio) is the simplest binary metric: for each user, did at least one relevant item appear in your top-K recommendations? If yes, that's a "hit" (score = 1); if no, it's a miss (score = 0). Average across all users to get your overall hit rate. Think of it like a pass/fail test for your recommender.

Hit Rate@K = (Number of users with ≥1 relevant item in top-K) / (Total users)

Position Matters: Mean Reciprocal Rank

Mean Reciprocal Rank (MRR) refines this by caring about where the first relevant item appears. If the first good recommendation is at position 1, you get a score of 1/1 = 1.0. At position 2? Score = 1/2 = 0.5. Position 5? Score = 1/5 = 0.2. If there's no relevant item at all, score = 0.

RR = 1 / (rank of first relevant item)
MRR = Average of RR across all users

Analogy: Hit Rate asks "Did you pass the treasure somewhere in these 10 map locations?" MRR asks "Was the treasure at location #1, #2, or #10?" — rewarding systems that put gold at the top.

MRR is especially useful for search and recommendation scenarios where users typically only engage with the first relevant result (like web search queries).

Key Takeaway: Hit Rate measures whether you succeeded at all in the top-K; MRR measures how quickly you succeeded by rewarding higher positions for the first relevant item — both are simple, interpretable binary success metrics.