Online vs. Offline Feature Computation
What you'll learn: When to compute features in real-time at inference time versus pre-computing and storing them for fast lookup.
The Core Trade-off
In classical ML deployment, you often need features derived from raw data — aggregations, transformations, embeddings, etc. You have two fundamental options:
Offline (batch) computation means calculating features ahead of time — often on a schedule — and storing them in a feature store for lookup at inference. Think of it like meal-prepping on Sunday: when dinner time comes, you just grab what's ready.
Online (real-time) computation means calculating features on-demand during the inference request itself. Like cooking fresh when someone orders — slower, but always up-to-date.
When to Use Each
Use offline features when:
- Computation is expensive (e.g., 90-day rolling averages over terabytes)
- Freshness isn't critical (user demographics, historical aggregates)
- You need consistent features for training and serving
- Latency budgets are tight (<100ms)
Use online features when:
- You need the absolute latest data (fraud detection on current transaction)
- Features are lightweight to compute (simple lookups, recent clicks)
- Staleness would harm model quality
- You can't predict which features you'll need in advance
Hybrid Patterns
Most production systems use both: offline for heavy historical aggregates, online for last-minute signals. Your feature store acts as the pre-computed cache, while your serving layer computes fresh features and merges them before inference.
Key Takeaway: Offline feature computation trades freshness for speed and cost efficiency; online computation trades latency for up-to-the-second accuracy. Choose based on your model's sensitivity to staleness and your serving latency budget.