Monitoring Replication Lag

In this lesson, you will learn how to instrument your PostgreSQL environment to monitor replication lag trends, ensuring your read replicas remain synchronized with the primary.

Understanding Replication Lag

Replication lag occurs when the secondary server (replica) is unable to apply changes from the primary's write-ahead log (WAL) as fast as they are being generated. This creates a delay, meaning your application might perform a "write" on the primary and immediately attempt a "read" on the replica, only to receive stale data.

Instrumenting the Primary

To monitor this, we look at the WAL locations. On your primary node, you can query pg_stat_replication to see how far each connected standby has progressed.

SELECT 
    client_addr, 
    state, 
    sent_lsn, 
    write_lsn, 
    flush_lsn, 
    replay_lsn,
    pg_wal_lsn_diff(sent_lsn, replay_lsn) AS lag_bytes
FROM pg_stat_replication;

The replay_lsn indicates the exact position in the transaction log that the replica has finished executing. By calculating the difference (pg_wal_lsn_diff) between what the primary has sent and what the replica has replayed, we get the lag in bytes.

Why Trends Matter

Monitoring a single snapshot is rarely enough. A temporary spike in lag is often acceptable during a heavy batch job, but a steady upward trend—or an "ever-increasing" lag—indicates a performance bottleneck on the replica (e.g., hardware contention, heavy vacuuming, or high read-load interference).

You should instrument your monitoring tool (like Prometheus or Datadog) to alert when:

1. The lag byte count exceeds a predefined threshold (e.g., 100MB).
2. The rate of growth of the lag is accelerating, suggesting the replica is losing ground rather than catching up.

By setting up alerts based on these lsn (Log Sequence Number) calculations, you move from reactive troubleshooting to proactive capacity planning. If you notice persistent lag, it may be time to revisit your Vacuum settings or consider if the replica needs stronger I/O throughput to keep pace with the primary's write volume.

Key Takeaway: Monitoring replication lag requires tracking the byte difference between the primary’s sent LSN and the replica’s replayed LSN to identify performance trends before they impact data consistency.