Course contentsShow
← Cloud and Infrastructure
Lesson 1981 of 2,98942. Observability - MetricsFree lesson

Remote Write and Remote Read

Configuring remote storage endpoints to offload long-term data and enable distributed query patterns.

Remote Write and Remote Read

What you'll learn: How to configure Prometheus to push metrics to external storage systems and query them back for long-term retention and distributed patterns.

The Core Problem

Prometheus stores time-series data locally with limited retention (typically days or weeks). At scale, you need long-term storage without exploding local disk usage, and you may want to centralize queries across multiple Prometheus instances.

What Are Remote Write and Remote Read?

Remote Write lets Prometheus continuously stream scraped metrics to external storage backends (like Thanos, Cortex, or cloud-native solutions). Think of it as a "live backup pipe"—every sample gets forwarded in near real-time.

Remote Read allows Prometheus to fetch older data from those external systems when executing queries. If your local Prometheus doesn't have historical data, it transparently reads from the remote endpoint.

Configuration Basics

In your prometheus.yml:

remote_write:
  - url: "https://remote-storage.example.com/api/v1/write"
    queue_config:
      capacity: 10000
      max_shards: 5

remote_read:
  - url: "https://remote-storage.example.com/api/v1/read"
    read_recent: true
  • remote_write defines where to send data; queue settings control buffering and parallelism
  • remote_read defines where to retrieve historical data; read_recent: true queries both local and remote

Why This Matters

This pattern decouples storage from collection, enabling you to retain years of metrics without overwhelming individual Prometheus servers. It also supports multi-cluster federation—centralize dashboards and alerts across geographically distributed Prometheus instances.

Key Takeaway:
Remote Write streams metrics to durable external storage for long-term retention, while Remote Read enables querying that historical data transparently from Prometheus.