Course contentsShow
← Cloud and Infrastructure
Lesson 1958 of 2,98942. Observability - MetricsFree lesson

Alerting Rules

Defining alerting rules that evaluate PromQL expressions and trigger notifications when conditions are met.

Alerting Rules

You'll learn how to define alerting rules in Prometheus that evaluate PromQL expressions and automatically trigger notifications when specific conditions are met.

What Are Alerting Rules?

Alerting rules are YAML-based configurations that tell Prometheus to continuously evaluate metrics against defined conditions. When a condition becomes true (and stays true for a specified duration), Prometheus fires an alert. Think of them as automated watchers that never sleep—constantly checking if something needs your attention.

Unlike recording rules (which precompute and store query results), alerting rules don't create new metrics. Instead, they create alerts that get sent to Alertmanager for notification routing.

Anatomy of an Alerting Rule

Alerting rules live in rule files and follow this structure:

groups:
  - name: example-alerts
    interval: 30s
    rules:
      - alert: HighCPUUsage
        expr: rate(cpu_usage_seconds_total[5m]) > 0.8
        for: 5m
        labels:
          severity: warning
          team: platform
        annotations:
          summary: "High CPU on {{ $labels.instance }}"
          description: "CPU usage is {{ $value | humanizePercentage }}"

Key components:

  • alert: Name of the alert
  • expr: PromQL expression to evaluate (uses metric types like Counter or Gauge)
  • for: How long condition must be true before firing
  • labels: Metadata for routing and grouping (remember labels from dimensionality?)
  • annotations: Human-readable context (supports templating)

The for clause prevents flapping—brief spikes won't wake you at 3 AM.

How It Fits Together

Prometheus evaluates alerting rules at the group's interval, queries your time-series data, and when thresholds breach, creates pending alerts. After the for duration, alerts become firing and route to Alertmanager.

Key Takeaway: Alerting rules transform metric queries into actionable notifications by continuously evaluating PromQL expressions and firing alerts when conditions persist beyond defined thresholds.