Storage Tier Migration
You'll learn how Pastebin can automatically move aging content to cheaper storage while keeping it accessible to users.
The Business Problem
Storing millions of pastes in high-performance SSD-backed storage forever becomes prohibitively expensive. However, not all pastes need millisecond access—most are never viewed again after the first few days.
How Storage Tiering Works
Think of storage tiers like your closet organization: frequently worn clothes up front (hot storage), seasonal items on high shelves (warm storage), and archived items in the attic (cold storage). Each tier trades performance for cost.
Hot Storage (Day 1-30): SSD-backed databases or object storage for instant access. Most expensive.
Warm Storage (Day 31-180): Standard object storage with slightly higher latency but 50% cheaper.
Cold Storage (180+ days): Glacier-like archival storage with minutes-to-hours retrieval time, 90% cheaper than hot storage.
Migration Architecture
A background migration service runs periodically:
- Query the metadata database for pastes exceeding age thresholds (
created_atorlast_accessed_at) - Copy paste content from hot storage to the target tier (S3 → S3 Glacier, for example)
- Update the metadata record's
storage_tierfield andstorage_locationreference - Delete from the original tier after confirmation
- Track migration status to handle failures gracefully
The read path checks storage_tier in metadata—if cold, warn users about retrieval delays or pre-fetch asynchronously.
Cost Savings
For a system with 100 million pastes where 80% are never accessed after 30 days, tiering can reduce storage costs by 60-70% while maintaining a good user experience for active content.
Key Takeaway: Storage tier migration balances cost efficiency with performance by automatically moving older or rarely-accessed pastes to cheaper storage, using metadata to track location and tier for intelligent retrieval.