Online Feature Serving Patterns
What You'll Learn
You'll learn how to architect low-latency retrieval of feature vectors from key-value stores during real-time inference, ensuring your models can make predictions in milliseconds.
The Real-Time Challenge
When a model serves predictions in production—like recommending a product or detecting fraud—it needs features right now. You can't afford the seconds or minutes a batch query might take. Online feature serving solves this by pre-computing and storing features in ultra-fast key-value stores optimized for sub-10ms lookups.
Core Serving Patterns
1. Direct Key-Value Lookup
The simplest pattern: the feature store maintains a key-value mapping (e.g., user_id → feature_vector). At inference time, you query by entity key and get back the latest materialized features. Think of it like a cache: you've pre-loaded everything into Redis or DynamoDB for instant retrieval.
2. Batch Retrieval
Instead of one lookup per prediction, fetch features for multiple entities in a single network call. This reduces overhead when scoring batches (e.g., ranking 100 products for one user).
3. Feature Join Service
For complex models requiring features from multiple entities (user + product + context), a dedicated service orchestrates parallel lookups across different key spaces and combines them into one feature vector—all within your latency budget.
4. Push vs Pull
- Pull: Model service requests features synchronously at prediction time
- Push: Features stream to the model service or edge cache proactively (e.g., via Kafka), reducing lookup latency to near-zero
Why It Matters
Real-time ML is bottlenecked by feature retrieval. A 500ms feature fetch makes sub-second predictions impossible. These patterns, paired with in-memory stores and smart caching, ensure your offline-to-online consistency doesn't sacrifice speed.
Key Takeaway:
Online feature serving uses pre-materialized features stored in low-latency key-value databases, with patterns like batch retrieval and push-based caching ensuring real-time models meet strict inference SLAs.