Testing Retrieval Logic
What you'll learn: How to test vector search, similarity calculations, and result filtering independently of a full vector database.
The Core Problem
Retrieval logic is the brain of your RAG system—it decides which documents are relevant based on similarity scores, applies filters, and ranks results. But spinning up a full vector database for every test is slow, expensive, and adds unnecessary complexity. You need to test the logic without the infrastructure.
Testing Vector Search Logic
Think of vector search like finding the closest color match in a paint store. You don't need the entire warehouse to test if your "closest match" algorithm works—you just need a few sample colors and their RGB values.
def test_cosine_similarity_calculation():
query_vector = [0.5, 0.5, 0.5]
doc_vectors = {
"doc1": [0.6, 0.4, 0.5],
"doc2": [0.1, 0.9, 0.2],
}
scores = calculate_similarity(query_vector, doc_vectors)
assert scores["doc1"] > scores["doc2"] # doc1 is closer
assert 0.0 <= scores["doc1"] <= 1.0 # valid range
Testing Filtering and Ranking
Your retrieval logic likely filters by metadata (date ranges, categories) and applies thresholds. Test these with controlled, synthetic data:
def test_retrieval_with_threshold():
results = [
{"id": "1", "score": 0.9, "category": "tech"},
{"id": "2", "score": 0.6, "category": "sports"},
{"id": "3", "score": 0.4, "category": "tech"},
]
filtered = filter_results(results, min_score=0.5, category="tech")
assert len(filtered) == 1
assert filtered[0]["id"] == "1"
Key Takeaway: Test retrieval logic with in-memory vectors and mock data—verify similarity calculations, score thresholds, and filtering rules work correctly before introducing database complexity.