Celebrity Follower Caching
You'll learn: How to optimize celebrity post distribution by caching their posts separately and merging them into user feeds at read time, avoiding the expensive fanout problem.
The Core Problem
When a celebrity with millions of followers publishes a post, performing fanout-on-write means writing that post to millions of individual user feeds—an operation that's slow, resource-intensive, and often unnecessary since most followers won't read their feed immediately. This is the celebrity problem we identified earlier.
The Solution: Separate Caching
Instead of fanning out celebrity posts to millions of timelines, we cache celebrity content separately:
- Celebrity Post Cache: When a celebrity posts, store it in a dedicated, shared cache keyed by the celebrity's ID
- At Read Time: When a user requests their feed, fetch both:
- Their pre-computed timeline (from regular follows via fanout-on-write)
- Posts from celebrities they follow (from the celebrity cache)
- Merge: Combine and rank both sources in real-time before returning the feed
Why This Works
Think of it like a news website: breaking news (celebrity posts) is stored once and read by millions, rather than copied into millions of individual mailboxes. This single shared cache is far more efficient than duplicating the same post millions of times.
The trade-off: slightly more complex read logic, but dramatically reduced write costs and storage.
Key Takeaway:
Celebrity follower caching stores celebrity posts in a shared cache and merges them into feeds at read time, eliminating the need to fanout celebrity content to millions of individual timelines and solving the celebrity problem efficiently.