Course contentsShow
← AI Engineering
Lesson 1625 of 1,88639. Classical ML Deployment (Non-LLM)Free lesson

Feature Caching Strategies

Caching expensive-to-compute features to reduce latency while managing freshness and invalidation.

Feature Caching Strategies

What you'll learn: How to cache expensive feature computations to slash inference latency without serving stale or incorrect predictions.

Why Cache Features?

Real-time feature computation (lesson 1624) can be costly—think aggregating user behavior over millions of events or running complex transformations. Caching pre-computed features lets you serve predictions in milliseconds instead of seconds. But caching introduces two challenges: freshness (how old can data be?) and invalidation (when do you discard or update cached values?).

Core Caching Patterns

Time-Based Expiration (TTL)

Set a time-to-live on cached features. A user's 30-day purchase history might be cached for 1 hour. After expiration, recompute. Simple but may serve slightly stale data.

Analogy: Like a carton of milk with an expiration date—good enough until then, toss it afterward.

Event-Driven Invalidation

Invalidate cache entries when upstream data changes. If a user makes a purchase, immediately recompute their cached aggregates. Keeps data fresh but requires event plumbing.

Lazy vs. Eager Refresh

  • Lazy: Recompute only when a request arrives and cache is stale (may add latency on first miss)
  • Eager: Proactively refresh before expiration (background jobs keep cache warm)

Hierarchical Caching

Cache at multiple layers: in-memory (Redis), application-level, or even within the feature store. Balance speed, cost, and consistency.

Managing Staleness Trade-offs

Not all features demand real-time accuracy. Static features (user demographics) can cache for hours; volatile features (click-in-last-minute) may need sub-second TTLs or skip caching entirely. Define SLAs per feature based on business impact.

Key Takeaway: Feature caching dramatically reduces latency by storing expensive computations, but requires careful TTL tuning and invalidation logic to balance speed with prediction accuracy.