System Health and Availability Metrics
What you'll learn: How to monitor system-level reliability indicators that reveal whether your LLM application is actually available and healthy for users.
While performance and quality metrics tell you how well your system works, health and availability metrics tell you whether it works at all. These are the vital signs that alert you when something breaks before users start complaining.
Core Health Indicators
Uptime measures the percentage of time your service is operational. Track this as a ratio: successful health checks divided by total checks. Most production systems aim for "three nines" (99.9%) or better.
Service availability goes deeper than simple uptime—it measures whether your service can actually fulfill requests. A server might be "up" but unable to call the LLM API, making it effectively unavailable.
Dependency health monitors the status of external services you rely on: LLM provider APIs, vector databases, caching layers, and authentication services. If OpenAI's API is down, your system is impacted even if your code is perfect.
Operational Signals
Queue depths show how many requests are waiting to be processed. Growing queues indicate capacity problems or downstream bottlenecks before they become full outages.
Circuit breaker states reveal when your system has automatically stopped calling failing dependencies. If your circuit breaker to the embedding service is "open," you know why RAG retrieval isn't working.
Think of these metrics like a car's dashboard: oil pressure, temperature gauges, and warning lights tell you about problems before the engine fails completely.
Key Takeaway: System health metrics—uptime, availability, dependency status, queue depths, and circuit breaker states—provide early warning signals that keep your LLM application reliable and prevent cascading failures.