Feature Caching Strategies
What you'll learn: How to cache expensive feature computations to slash inference latency without serving stale or incorrect predictions.
Why Cache Features?
Real-time feature computation (lesson 1624) can be costly—think aggregating user behavior over millions of events or running complex transformations. Caching pre-computed features lets you serve predictions in milliseconds instead of seconds. But caching introduces two challenges: freshness (how old can data be?) and invalidation (when do you discard or update cached values?).
Core Caching Patterns
Time-Based Expiration (TTL)
Set a time-to-live on cached features. A user's 30-day purchase history might be cached for 1 hour. After expiration, recompute. Simple but may serve slightly stale data.
Analogy: Like a carton of milk with an expiration date—good enough until then, toss it afterward.
Event-Driven Invalidation
Invalidate cache entries when upstream data changes. If a user makes a purchase, immediately recompute their cached aggregates. Keeps data fresh but requires event plumbing.
Lazy vs. Eager Refresh
- Lazy: Recompute only when a request arrives and cache is stale (may add latency on first miss)
- Eager: Proactively refresh before expiration (background jobs keep cache warm)
Hierarchical Caching
Cache at multiple layers: in-memory (Redis), application-level, or even within the feature store. Balance speed, cost, and consistency.
Managing Staleness Trade-offs
Not all features demand real-time accuracy. Static features (user demographics) can cache for hours; volatile features (click-in-last-minute) may need sub-second TTLs or skip caching entirely. Define SLAs per feature based on business impact.
Key Takeaway: Feature caching dramatically reduces latency by storing expensive computations, but requires careful TTL tuning and invalidation logic to balance speed with prediction accuracy.