Course contentsShow
← AI Engineering
Lesson 1621 of 1,88639. Classical ML Deployment (Non-LLM)Free lesson

Online vs. Offline Feature Computation

When to compute features in real-time during inference versus pre-computing and storing them for lookup.

Online vs. Offline Feature Computation

What you'll learn: When to compute features in real-time at inference time versus pre-computing and storing them for fast lookup.

The Core Trade-off

In classical ML deployment, you often need features derived from raw data — aggregations, transformations, embeddings, etc. You have two fundamental options:

Offline (batch) computation means calculating features ahead of time — often on a schedule — and storing them in a feature store for lookup at inference. Think of it like meal-prepping on Sunday: when dinner time comes, you just grab what's ready.

Online (real-time) computation means calculating features on-demand during the inference request itself. Like cooking fresh when someone orders — slower, but always up-to-date.

When to Use Each

Use offline features when:

  • Computation is expensive (e.g., 90-day rolling averages over terabytes)
  • Freshness isn't critical (user demographics, historical aggregates)
  • You need consistent features for training and serving
  • Latency budgets are tight (<100ms)

Use online features when:

  • You need the absolute latest data (fraud detection on current transaction)
  • Features are lightweight to compute (simple lookups, recent clicks)
  • Staleness would harm model quality
  • You can't predict which features you'll need in advance

Hybrid Patterns

Most production systems use both: offline for heavy historical aggregates, online for last-minute signals. Your feature store acts as the pre-computed cache, while your serving layer computes fresh features and merges them before inference.

Key Takeaway: Offline feature computation trades freshness for speed and cost efficiency; online computation trades latency for up-to-the-second accuracy. Choose based on your model's sensitivity to staleness and your serving latency budget.