Course contentsShow
← Cloud and Infrastructure
Lesson 1979 of 2,98942. Observability - MetricsFree lesson

Prometheus Federation Basics

Using federation to create hierarchical Prometheus architectures and aggregate metrics across multiple instances.

Prometheus Federation Basics

What You'll Learn

You'll learn how Prometheus federation creates hierarchical monitoring architectures to aggregate metrics from multiple Prometheus instances into a centralized view.

The Core Idea

Federation lets one Prometheus server scrape selected metrics from other Prometheus servers. Think of it like a corporate reporting structure: regional offices (edge Prometheus instances) collect detailed local metrics, while headquarters (central Prometheus) pulls summary reports from each region without needing every single detail.

Federation solves a key scaling challenge: a single Prometheus can't monitor thousands of services across multiple clusters efficiently. Instead, you deploy multiple Prometheus instances close to their targets, then use federation to pull aggregated metrics upward.

How Federation Works

The federated Prometheus scrapes the /federate endpoint of downstream Prometheus servers, retrieving only the metrics you specify via label selectors. This creates a hierarchical architecture:

  • Bottom tier: Individual Prometheus instances scrape application metrics
  • Middle tier (optional): Regional Prometheus instances aggregate metrics from their zone
  • Top tier: Global Prometheus federates key metrics from all lower tiers

Typical Configuration Pattern

In your central Prometheus prometheus.yml:

scrape_configs:
  - job_name: 'federate'
    honor_labels: true
    metrics_path: '/federate'
    params:
      'match[]':
        - '{job="kubernetes-pods"}'
        - '{__name__=~"job:.*"}'  # Recording rules
    static_configs:
      - targets:
        - 'prometheus-eu:9090'
        - 'prometheus-us:9090'

The honor_labels: true setting preserves original labels from downstream instances, preventing conflicts.

When to Use Federation

Federation excels when monitoring multiple data centers, Kubernetes clusters, or geographic regions where network latency or isolation makes centralized scraping impractical. It's especially powerful combined with recording rules (covered earlier) to pre-aggregate data at lower tiers.

Key Takeaway: Prometheus federation creates hierarchical monitoring by letting upstream Prometheus instances selectively scrape aggregated metrics from downstream instances, enabling scalable multi-cluster observability without overwhelming a single server.