Course contentsShow
← System Design
Lesson 1586 of 1,91939. Real-World System: Image and Video HostingFree lesson

Multipart Upload for Large Files

Break large videos into chunks for parallel upload, enabling resume capability and improved reliability at scale.

Multipart Upload for Large Files

You'll learn how to split large video or image files into smaller parts for parallel, resumable uploads that dramatically improve reliability and speed at scale.

Why Uploading Large Files Is Hard

When a user uploads a 2 GB video file in one continuous HTTP request, a single network hiccup means starting over from scratch. Multiply this by thousands of concurrent uploads, and you've created a system prone to failures, timeouts, and frustrated users.

The Multipart Upload Strategy

Break the file into chunks (typically 5-100 MB each) on the client side. Each chunk uploads independently and in parallel. The server tracks which parts have arrived, then reassembles them once all pieces are present.

Think of it like moving to a new apartment: instead of carrying one enormous box, you pack several smaller boxes. If you drop one, you only repack that single box—not everything. Friends can also carry multiple boxes simultaneously, speeding up the move.

Core Components

  1. Initiation: Client requests an upload ID from the server
  2. Chunking: Client splits the file locally (e.g., 20 MB parts)
  3. Parallel Upload: Each part uploads with its part number and upload ID
  4. Progress Tracking: Server records completed parts (often in metadata storage or database)
  5. Completion: Client signals all parts are uploaded; server validates and merges them

Resume Capability

If part 7 of 50 fails, the client simply retries only part 7. The upload ID and part numbers ensure the server knows exactly what's missing. This is critical for mobile users on unstable networks or large enterprise uploads over long durations.

Scale Benefits

Parallel uploads saturate available bandwidth and distribute load across multiple storage nodes. A single 1 GB file might upload in 10 minutes sequentially but complete in 2 minutes with 10 parallel parts.

Key Takeaway: Multipart upload transforms large file ingestion from a fragile, sequential bottleneck into a resilient, parallelized workflow that gracefully handles failures and maximizes throughput.