Course contentsShow
← Machine Learning and Deep Learning
Lesson 2889 of 3,53862. ML Pipelines and OrchestrationFree lesson

Online Feature Serving Patterns

Low-latency retrieval of feature vectors from key-value stores during real-time inference.

Online Feature Serving Patterns

What You'll Learn

You'll learn how to architect low-latency retrieval of feature vectors from key-value stores during real-time inference, ensuring your models can make predictions in milliseconds.

The Real-Time Challenge

When a model serves predictions in production—like recommending a product or detecting fraud—it needs features right now. You can't afford the seconds or minutes a batch query might take. Online feature serving solves this by pre-computing and storing features in ultra-fast key-value stores optimized for sub-10ms lookups.

Core Serving Patterns

1. Direct Key-Value Lookup

The simplest pattern: the feature store maintains a key-value mapping (e.g., user_id → feature_vector). At inference time, you query by entity key and get back the latest materialized features. Think of it like a cache: you've pre-loaded everything into Redis or DynamoDB for instant retrieval.

2. Batch Retrieval

Instead of one lookup per prediction, fetch features for multiple entities in a single network call. This reduces overhead when scoring batches (e.g., ranking 100 products for one user).

3. Feature Join Service

For complex models requiring features from multiple entities (user + product + context), a dedicated service orchestrates parallel lookups across different key spaces and combines them into one feature vector—all within your latency budget.

4. Push vs Pull

  • Pull: Model service requests features synchronously at prediction time
  • Push: Features stream to the model service or edge cache proactively (e.g., via Kafka), reducing lookup latency to near-zero

Why It Matters

Real-time ML is bottlenecked by feature retrieval. A 500ms feature fetch makes sub-second predictions impossible. These patterns, paired with in-memory stores and smart caching, ensure your offline-to-online consistency doesn't sacrifice speed.

Key Takeaway:
Online feature serving uses pre-materialized features stored in low-latency key-value databases, with patterns like batch retrieval and push-based caching ensuring real-time models meet strict inference SLAs.