Search Result Caching and Invalidation
What You'll Learn
You'll learn how to speed up repeated vector searches by caching results and when to refresh or discard cached data to keep searches accurate.
Why Cache Search Results?
When users run the same or very similar queries repeatedly, your vector database performs the same expensive similarity calculations over and over. Caching stores these results temporarily so you can return them instantly without re-computing.
Think of it like a restaurant memorizing popular orders. If ten customers order "the usual breakfast," the kitchen doesn't need to read the full menu each time—they already know what to prepare.
Core Caching Strategies
Query-Based Caching
Store results keyed by the exact query vector or text. When an identical query arrives, return the cached results immediately.
Challenge: Queries must match exactly. Even tiny differences in embeddings mean a cache miss.
Semantic Similarity Caching
Cache results for query "regions" rather than exact matches. If a new query is very similar to a cached one (e.g., cosine similarity > 0.98), reuse those results.
Cache Invalidation: Keeping Results Fresh
Cached results become stale when:
- New documents are added to your vector database
- Existing documents are updated or deleted
- Your data changes frequently
Invalidation Approaches
- Time-based (TTL): Expire cache entries after X minutes/hours
- Event-based: Clear cache when documents change
- Partial invalidation: Remove only entries affected by updates
- Lazy invalidation: Keep cache until someone queries, then check freshness
Key Takeaway: Caching dramatically speeds up repeated searches, but you must balance speed gains against data freshness—invalidate too rarely and users see outdated results; too often and you lose performance benefits.