In this lesson, you will learn how to design high-throughput async message consumers that scale horizontally while preserving strict message ordering where business logic demands it.
The Balancing Act: Throughput vs. Order
When consuming messages from a queue, the simplest approach is spawning a new task for every incoming message. While this maximizes parallelism, it often violates ordering guarantees (e.g., if message B arrives after message A but completes processing first). To handle this, we move from "unbounded spawning" to a Sharded Consumer Pattern.
By hashing the message key (like a User ID or Account ID), we ensure that all messages for a specific entity are routed to the same logical task or "lane." This allows us to maintain strict sequence within a partition while scaling across all available CPU cores.
Implementation Pattern
Using tokio::sync::mpsc channels as the backbone, we can implement a router that dispatches messages to fixed-capacity worker tasks:
async fn start_consumer(receiver: mpsc::Receiver<Message>) {
let mut workers: Vec<mpsc::Sender<Message>> = (0..NUM_SHARDS)
.map(|_| spawn_worker()) // Each worker maintains its own order
.collect();
while let Some(msg) = receiver.recv().await {
let shard = calculate_shard(&msg.key, NUM_SHARDS);
// Direct routing ensures ordering per key
let _ = workers[shard].send(msg).await;
}
}
Ensuring Consistency
When a worker processes a message, it is crucial to handle the "wait-free" requirement of the consumer. If a worker task blocks on an external dependency (like a database or an external API), the entire shard stalls. To keep the pipeline saturated, utilize tokio::spawn to handle the I/O, but use a Semaphore to limit the number of concurrent in-flight requests per shard. This prevents memory exhaustion while ensuring that even if one operation is slow, the next message in that specific shard doesn't jump ahead of the queue.
By decoupling the ingestion (the router) from the execution (the shard workers), you create a system that is both highly parallelized for the overall throughput and strictly ordered for the individual data points that require it.
Key Takeaway: Use sharding based on message keys to pin related operations to specific tasks, ensuring local ordering remains intact while scaling total processing capacity across the entire thread pool.