Pushgateway for Short-Lived Jobs
What you'll learn: How to collect metrics from batch jobs and ephemeral processes that finish before Prometheus can scrape them.
The Challenge with Short-Lived Jobs
Prometheus scrapes metrics from long-running services at regular intervals (typically every 15–60 seconds). But what happens when your job finishes in 10 seconds? By the time Prometheus comes around to scrape, your batch job, cron task, or CI/CD pipeline step has already exited. The metrics vanish into the void.
What Is the Pushgateway?
The Pushgateway is a special Prometheus component that acts as a temporary metrics repository. Instead of waiting to be scraped, short-lived jobs push their metrics to the Pushgateway before they exit. Prometheus then scrapes the Pushgateway on its regular schedule, capturing those metrics even though the original job is long gone.
Think of it like a "drop box" for metrics: your delivery driver (batch job) drops off a package (metrics) at your doorstep (Pushgateway), then leaves. When you get home (Prometheus scrapes), the package is waiting for you.
When to Use It
Use the Pushgateway for:
- Batch jobs and cron tasks
- CI/CD pipeline steps
- Serverless functions with metrics
- Any process that completes quickly
Don't use it for regular services—those should expose metrics endpoints and be scraped normally.
Basic Flow
- Your batch job completes its work
- Before exiting, it pushes metrics to the Pushgateway (
/metrics/job/<job_name>) - Prometheus scrapes the Pushgateway periodically
- Metrics include labels like
jobandinstanceto identify the source
Key Takeaway: The Pushgateway bridges the gap between Prometheus's pull model and ephemeral workloads that can't wait around to be scraped, enabling observability for batch jobs and short-lived processes.