Course contentsShow
← AI Engineering
Lesson 1098 of 1,88627. Cloud Deployment and ScalingFree lesson

Health Checks and Readiness Probes

Implement container health checks and readiness endpoints to enable orchestrators to manage AI service lifecycle properly.

Health Checks and Readiness Probes

You'll learn: How to implement health checks and readiness endpoints so container orchestrators like Kubernetes can automatically manage your AI service's lifecycle and recover from failures.

The Problem: When Is Your AI Service Actually Ready?

Your containerized AI model might be running, but is it ready to serve requests? Loading a multi-gigabyte model into GPU memory can take minutes. If traffic arrives before your model is loaded, requests fail catastrophically. Container orchestrators need a way to ask: "Are you truly ready to handle traffic?"

Two Critical Endpoints

Health checks (or liveness probes) answer: "Are you still alive?" If your service becomes deadlocked or the process crashes silently, the orchestrator needs to know to restart the container.

Readiness probes answer: "Are you ready to accept requests?" During startup, model loading, or when temporarily overloaded, your service should signal "not ready yet" so traffic gets routed elsewhere.

Implementation Pattern

In your FastAPI or Flask app, expose simple endpoints:

from fastapi import FastAPI

app = FastAPI()
model_loaded = False

@app.get("/health")
async def health():
    return {"status": "alive"}

@app.get("/ready")
async def ready():
    if model_loaded:
        return {"status": "ready"}
    return Response(status_code=503)

In your Dockerfile, add:

HEALTHCHECK --interval=30s --timeout=3s \
  CMD curl -f http://localhost:8000/health || exit 1

Kubernetes uses these probes to avoid sending traffic to containers still loading models, and restarts containers that fail health checks repeatedly.

Key Takeaway: Health checks detect dead containers (trigger restarts), while readiness probes prevent traffic from reaching containers that aren't yet able to serve requests—essential for AI workloads with long initialization times.