Measuring Fanout Performance
You'll learn: How to track the health of your fanout system using key metrics that reveal bottlenecks and ensure timely feed delivery.
Why Measure Fanout Performance?
Fanout—the process of distributing a post to followers' feeds—is the heartbeat of your social media system. If it's slow, broken, or backlogged, users see stale content or miss posts entirely. Without measurements, you're flying blind.
Core Metrics to Track
Fanout Completion Time measures how long it takes from post creation to the last follower receiving it in their feed. Track this per-post and as percentile distributions (p50, p95, p99). A spike here signals worker overload or database bottlenecks.
Queue Depth tells you how many fanout jobs are waiting to be processed. A growing queue means your workers can't keep up with incoming posts. Think of it like a restaurant kitchen—if orders pile up faster than chefs can cook, diners wait longer.
Follower Reach Rate tracks what percentage of a user's followers successfully received the post within your SLA (e.g., 5 seconds). This catches partial failures where some workers crash or databases timeout.
Additional Health Signals
- Worker utilization: Are your fanout workers idle or maxed out?
- Retry rate: How often do fanout operations need retries?
- Celebrity post latency: Track high-follower accounts separately—they're outliers
- Per-shard metrics: Identify if specific database shards are slower
Monitoring in Practice
Set up dashboards showing real-time queue depths and alert thresholds. Log completion times per post for later analysis. Use distributed tracing to follow a single post's fanout journey across workers.
Key Takeaway: Fanout performance metrics—completion time, queue depth, and reach rate—are essential health indicators that help you detect issues before users notice degraded feed freshness.