Course contentsShow
← System Design
Lesson 1562 of 1,91938. Real-World System: PastebinFree lesson

Content Compression and Encoding

Compress paste text with gzip or zstd before storage to reduce storage costs and transfer bandwidth.

Content Compression and Encoding

What you'll learn: How compressing paste text before storage can dramatically reduce storage costs and bandwidth usage in a Pastebin system.

Why Compress Paste Content?

Text is highly compressible. A 10 KB paste might shrink to 2-3 KB with good compression—an instant 70-80% storage savings. When you're storing millions of pastes, this translates directly to lower cloud storage bills and faster transfer times.

Think of compression like vacuum-sealing clothes before storing them: the same content takes up far less space, and you can fit more into the same container.

How It Works in Pastebin

When a user creates a paste, the write path compresses the text before sending it to object storage:

  1. Client submits paste text
  2. Application server applies gzip or zstd compression
  3. Compressed blob stored in object storage
  4. Metadata table stores original size + compression flag

On the read path, the process reverses:

  1. Retrieve compressed blob from storage
  2. Decompress on application server or edge cache
  3. Serve plain text to user

Choosing the Right Algorithm

gzip is the universal standard—widely supported, fast decompression, good compression ratios (typically 60-70% reduction for text).

zstd (Zstandard) offers better compression ratios and faster decompression than gzip, making it ideal for modern high-throughput systems. It's become the industry favorite for scenarios where you control both compression and decompression.

Additional Benefits

Beyond storage savings, compression reduces bandwidth costs when transferring pastes between regions or to CDN edge locations. Smaller payloads also mean faster cache warming and lower latency for end users on slow connections.

Key Takeaway: Compressing paste content with gzip or zstd before storage can reduce storage costs by 70%+ and decrease transfer bandwidth—essential optimizations for any text-heavy system like Pastebin.