Course contentsShow
← System Design
Lesson 1611 of 1,91939. Real-World System: Image and Video HostingFree lesson

Multi-Tier Caching Architecture

Designing edge, shield, and origin cache layers to optimize hit rates and origin protection.

Multi-Tier Caching Architecture

What you'll learn: How to design layered cache tiers—edge, shield, and origin—to maximize cache hit rates and protect your origin servers from overwhelming traffic.


The Core Idea

When serving images and videos globally, a single CDN layer isn't always enough. Multi-tier caching introduces multiple levels of cache between users and your origin storage, each with a specific purpose:

  • Edge caches sit closest to users (geographically distributed)
  • Shield caches act as a middle layer between edge and origin
  • Origin caches protect your object storage from direct requests

Think of it like a company hierarchy: employees (users) ask their manager (edge), who asks a director (shield), who finally asks the CEO (origin) only when absolutely necessary.


Why Multiple Tiers?

Without shielding, every edge location missing content would hit your origin directly. With thousands of edge servers globally, that's potentially thousands of simultaneous requests for the same newly-uploaded video—overwhelming your origin.

Edge tier serves the majority of requests with low latency. When an edge misses, instead of hitting origin, it queries the shield tier—a smaller pool of regional caches. The shield consolidates requests: if 50 edge servers miss the same file, the shield fetches it once from origin and serves all 50.

Origin cache (in-memory or SSD layer before object storage) handles the final backstop, reducing actual disk/object-store reads.


Hit Rate Optimization

Each tier increases the probability of cache hits:

  • Edge hit rate: 85-95% (frequent content)
  • Shield hit rate: 70-85% of edge misses
  • Only 1-5% of total requests reach origin

Lower tiers also enable cache warming—preloading hot content into shield/edge before user requests arrive.


Key Takeaway:
Multi-tier caching with edge, shield, and origin layers maximizes cache hit rates, reduces origin load exponentially, and improves global delivery performance by consolidating cache-miss requests at intermediate tiers.