Processing State and Progress Tracking
What you'll learn: How to track and report the progress of long-running media processing jobs so users know their upload status in real-time.
When you upload a large video to YouTube or a high-resolution image for processing, you don't sit in darkness wondering if anything's happening. You see progress bars, status updates, and notifications. This is processing state and progress tracking — the system's way of maintaining visibility into asynchronous jobs.
Core Concepts
Processing States
Your processing jobs move through distinct states:
PENDING: Job queued but not yet startedIN_PROGRESS: Worker actively processingCOMPLETED: Successfully finishedFAILED: Encountered unrecoverable errorPARTIALLY_FAILED: Some variants succeeded, others failed
These states are stored in your metadata database alongside the upload record, typically in a processing_status column or dedicated job_status table.
Progress Updates
Workers periodically update progress metrics:
- Percentage complete (e.g., "transcoding 45% done")
- Current stage (e.g., "extracting audio track")
- Time remaining estimates
Workers write these updates to a fast datastore (Redis, database) that your API can query.
Partial Failure Handling
In video transcoding, if generating the 1080p version fails but 720p and 480p succeed, you mark specific variants as failed while allowing the upload to remain usable. This requires granular state tracking per processing task, not just per upload.
Think of it like tracking a pizza delivery: pending (in kitchen), in progress (out for delivery with live GPS updates), delivered (complete), or failed (wrong address). You want real-time visibility, not silence until it appears at your door.
Key Takeaway:
Processing state and progress tracking stores job status in your database, enables workers to report incremental progress, and handles partial failures gracefully so users stay informed throughout long-running media operations.