Course contentsShow
← AI Engineering
Lesson 1254 of 1,88630. Monitoring and ObservabilityFree lesson

Threshold-Based Alerting

Set up alerts based on static and dynamic thresholds for latency, error rates, token usage, and cost metrics.

Threshold-Based Alerting

What you'll learn: How to configure alerts that fire when specific metrics cross predefined static or dynamic boundaries in your LLM system.


The Core Idea

Threshold-based alerting triggers notifications when monitored metrics exceed acceptable limits. You define boundaries for critical metrics—like latency, error rates, token usage, or cost—and your monitoring system alerts you when they're breached.

Think of it like a smoke detector: you set a threshold (smoke concentration), and when that level is crossed, the alarm sounds. You're not constantly watching; the system watches for you.


Static vs. Dynamic Thresholds

Static thresholds are fixed values you set based on requirements or experience:

  • Latency > 3 seconds
  • Error rate > 2%
  • Hourly token usage > 1 million
  • Daily cost > $500

Dynamic thresholds adapt based on historical patterns and context:

  • Latency > 2× the 7-day average
  • Error rate > mean + 3 standard deviations
  • Token usage > 150% of yesterday's peak hour
  • Cost growth > 20% week-over-week

Static thresholds work well for hard business requirements. Dynamic thresholds catch anomalies in systems with natural variation (weekday vs. weekend traffic, seasonal patterns).


Common Metrics to Alert On

Based on your instrumentation (from lessons 1232-1241), you should alert on:

  • Latency: P95/P99 response times crossing SLA bounds
  • Error rates: Failed requests, timeouts, rate limit hits
  • Token usage: Sudden spikes indicating inefficient prompts or abuse
  • Cost metrics: Budget overruns or unexpected expense acceleration
  • Quality degradation: Falling confidence scores or increased retries

Implementation Approach

Combine your metric collection with alert policies. Set multiple severity levels (warning vs. critical) and appropriate notification channels (email, Slack, PagerDuty) based on impact.

Key Takeaway: Threshold-based alerting transforms passive monitoring into active protection by automatically notifying you when metrics indicate problems, using either fixed limits for known requirements or adaptive boundaries for detecting unusual patterns.