Course contentsShow
← System Design
Lesson 1557 of 1,91938. Real-World System: PastebinFree lesson

Hot vs Cold Storage Tiering

Move older or less-accessed pastes to cheaper cold storage like S3 Glacier after retention period.

Hot vs Cold Storage Tiering

What you'll learn: How to optimize Pastebin storage costs by automatically moving older, rarely-accessed pastes to cheaper archival storage.


The Storage Cost Problem

In a system like Pastebin, not all pastes are accessed equally. New pastes often get heavy traffic in their first hours or days, but most eventually become dormant. Storing millions of cold pastes on expensive, high-performance storage (like SSD-backed databases or premium object storage) wastes money.

Hot vs Cold Storage

Think of hot storage as your kitchen counter—expensive space for items you use daily. Cold storage is your basement freezer—cheap, spacious, but slower to access.

Hot storage (S3 Standard, fast SSD):

  • Fast retrieval (milliseconds)
  • Higher cost per GB
  • For recent or frequently accessed pastes

Cold storage (S3 Glacier, tape backup):

  • Slower retrieval (minutes to hours)
  • Much cheaper per GB (often 10× less)
  • For old or rarely accessed pastes

Implementing Tiering

After a paste's retention period (say, 30 days of inactivity or 6 months since creation), automatically migrate it:

  1. Scheduled job scans the metadata database for old pastes
  2. Copy paste content from S3 Standard → S3 Glacier
  3. Update metadata to track new storage location and tier
  4. Delete from hot storage to free space

When a cold paste is requested:

  • Initiate retrieval from Glacier (may take minutes)
  • Optionally promote back to hot storage if accessed frequently
  • Cache the result to avoid repeated slow fetches

Cost Savings

A paste system storing 10 TB might keep only 1 TB hot and 9 TB cold, reducing storage costs by 80%+ while maintaining excellent performance for active content.

Key Takeaway:
Hot vs cold storage tiering dramatically reduces costs by automatically moving inactive pastes to cheaper archival storage like S3 Glacier, accepting slower retrieval times for rarely accessed content.