Course contentsShow
← Machine Learning and Deep Learning
Lesson 3091 of 3,53867. A/B Testing and Deployment StrategiesFree lesson

Health Checks and Readiness Probes

Configure endpoint monitoring to ensure new model instances are fully operational before receiving production traffic.

Health Checks and Readiness Probes

What You'll Learn

You'll learn how to configure endpoint monitoring to ensure new model instances are fully operational before they receive production traffic, preventing failed predictions during deployment.

The Core Problem

When you deploy a new model version (canary, blue-green, or rolling), the container or service may start before the model is actually ready to serve predictions. Loading large models into memory, initializing GPUs, warming up inference pipelines, and compiling optimizations can take seconds or even minutes. If traffic hits the endpoint during this startup phase, requests will fail or time out.

Health checks and readiness probes solve this by continuously testing whether your model service is alive and ready to handle requests.

Two Types of Probes

Health/Liveness Probe

Checks if the service is alive. If it fails repeatedly, the orchestration system (Kubernetes, ECS, etc.) restarts the container. Think of this as "Is the process running?"

Readiness Probe

Checks if the service is ready to accept traffic. Until this passes, the load balancer won't route requests to this instance. Think of this as "Has the model loaded and is it ready to predict?"

Implementation Pattern

Your model service should expose dedicated endpoints:

  • /health – Returns 200 if the process is alive
  • /ready – Returns 200 only after the model is loaded, GPU initialized, and a test inference succeeds

The readiness probe often runs a lightweight dummy prediction to verify the full inference pipeline works before declaring the instance ready.

Key Takeaway: Health checks confirm your service is alive, while readiness probes ensure the model is fully operational before receiving production traffic—critical for safe deployments and preventing user-facing errors during instance startup.