Monitoring and Observability for SRE
You'll learn how SRE approaches instrumentation by focusing on user-experienced symptoms rather than just internal system metrics.
The SRE Instrumentation Philosophy
Traditional monitoring often obsesses over internal metrics: CPU usage, memory consumption, disk I/O. While these matter, SRE flips the script: start with what the user experiences, then instrument backward to support diagnosis.
Think of it like being a doctor. A poor approach asks "What's the patient's white blood cell count?" before asking "What symptoms are they experiencing?" SRE asks first: Is the user experiencing slow responses? Errors? Can they complete their tasks?
This aligns perfectly with concepts you've already learned:
- Four Golden Signals (latency, traffic, errors, saturation) describe user-facing behavior
- SLIs measure reliability from the user's perspective
- Symptoms vs Causes alerting fires on what users feel, not what machines think
Why This Matters
When you monitor symptoms first:
- Alerts become actionable — "Users can't log in" is clearer than "Database CPU at 87%"
- You avoid false positives — High CPU might be fine if users are happy
- You focus on what matters — A server crash that doesn't affect users isn't urgent
Internal metrics still matter, but as diagnostic tools during incident response, not primary alert sources. Your dashboard should show user-facing health first, then drill down to internal causes second.
The Observability Connection
This symptom-first approach requires observability: the ability to ask arbitrary questions about your system's behavior after an incident occurs. Metrics tell you what users experience; traces and logs help you understand why.
Key Takeaway: SRE instrumentation prioritizes measuring what users experience (symptoms) over internal system health (causes), ensuring monitoring reflects actual reliability and guides meaningful action.