Course contentsShow
← AI Engineering
Lesson 959 of 1,88623. LLM Application Architecture PatternsFree lesson

Cache Invalidation Strategies

Managing cache freshness through TTLs, versioning, and invalidation triggers when models or data change.

Cache Invalidation Strategies

What you'll learn: How to keep your LLM application cache fresh and accurate when models, prompts, or underlying data change.

The Challenge of Stale Caches

You've built a powerful caching layer for your LLM application—whether it's semantic, prefix-based, or exact matching. But what happens when you update your prompt template, switch to GPT-5, or your knowledge base gets new documents? Your cache suddenly holds wrong answers. Cache invalidation is the art of keeping cached responses trustworthy while maximizing hit rates.

Three Core Strategies

Time-To-Live (TTL)

Set expiration timestamps on cached entries. After 24 hours (or whatever duration makes sense), the cache entry automatically expires and must be regenerated. Simple but blunt—you might invalidate perfectly good responses or keep bad ones too long.

# Redis example (conceptual)
cache.set(key, response, ttl=86400)  # 24 hours

Version-Based Invalidation

Tag cache entries with version identifiers. When your prompt template changes from v1 to v2, or you switch from gpt-4 to gpt-4-turbo, include the version in your cache key. Old versions naturally become unreachable.

cache_key = f"prompt_v2:model_gpt4turbo:{semantic_hash}"

Event-Driven Invalidation

Actively delete or mark cache entries when triggering events occur. Document updated in your vector database? Flush semantic cache entries that might reference it. Model switched? Clear all responses generated with the old model.

Choosing Your Strategy

Use TTL for general freshness, versioning for controlled deployments, and event-driven invalidation for data-dependent responses. Often you'll combine all three: version tags for model changes, short TTLs for dynamic data, and manual invalidation for urgent corrections.

Key Takeaway: Effective cache invalidation balances performance (keeping useful entries) with correctness (removing stale data) through TTLs, versioning, and trigger-based purging when your AI system evolves.