Score Normalization Techniques
What you'll learn: How to combine retrieval scores from different search methods (like vector and keyword search) by normalizing them to a common scale.
Why Normalization Matters
When you run hybrid search (combining vector similarity scores with BM25 keyword scores), you face a problem: the scores are on completely different scales. A cosine similarity might range from 0.8 to 0.95, while BM25 scores could be 5 to 50. You can't simply add or average them — it's like adding apples measured in pounds to oranges measured in kilograms!
The Three Core Techniques
Min-Max Normalization
Rescales all scores to a fixed range, typically [0, 1]:
normalized_score = (score - min_score) / (max_score - min_score)
Analogy: Converting test scores from different classes (one out of 50 points, another out of 100) to percentages so they're comparable.
Z-Score Normalization
Centers scores around their mean and scales by standard deviation:
z_score = (score - mean) / std_dev
This handles outliers better and shows how many "standard deviations" above/below average each result is.
Reciprocal Rank Normalization
Based on ranking position rather than raw scores:
rr_score = 1 / rank
The top result gets 1.0, second gets 0.5, third gets 0.33, etc. This is rank-based rather than score-based, making it robust when score distributions are unreliable.
Combining After Normalization
Once normalized, you can safely combine scores:
- Simple weighted average:
0.7 * vector_score + 0.3 * bm25_score - Maximum score
- Reciprocal Rank Fusion (averages reciprocal ranks)
Key Takeaway: Score normalization translates different retrieval methods onto a common scale, enabling fair combination of vector similarity and keyword search results in hybrid RAG systems.