Course contentsShow
← Cloud and Infrastructure
Lesson 1430 of 2,98929. Kubernetes ObservabilityFree lesson

Debugging Failed Probes

Use kubectl commands and events to troubleshoot and resolve probe failures.

Debugging Failed Probes

What you'll learn: How to diagnose and resolve probe failures using kubectl commands and Kubernetes events.

The Challenge

When probes fail, your Pods may be stuck in a crash loop, marked unready, or never start at all. Rather than guessing, Kubernetes provides diagnostic tools to pinpoint exactly why health checks are failing.

Using kubectl to Investigate

Start with kubectl describe pod <pod-name>. This command reveals probe failure events in the Events section at the bottom. Look for messages like "Liveness probe failed: Get http://10.0.0.5:8080/health: dial tcp 10.0.0.5:8080: connect: connection refused."

Check current pod status with kubectl get pods — you'll see restarts incrementing if liveness probes are failing, or pods stuck in "Not Ready" if readiness probes fail.

Reading the Clues

Events tell you what failed. Common patterns:

  • Connection refused: Port mismatch or app not listening
  • Timeout: initialDelaySeconds too short or app too slow
  • HTTP 500 errors: Application logic issue in health endpoint
  • Command exit code 1: Exec probe script failing

Use kubectl logs <pod-name> to see application logs around the time of failures. The timestamps in events correlate with log entries, revealing if your app was mid-startup or crashed.

Testing the Fix

If you suspect timing issues, check your probe parameters with kubectl get pod <pod-name> -o yaml. Verify initialDelaySeconds, periodSeconds, and timeoutSeconds match your application's actual startup and response times.

For exec probes, test commands directly: kubectl exec <pod-name> -- /path/to/healthcheck.sh to verify the script works as expected.

Key Takeaway: Failed probes leave a clear trail in events and logs — use kubectl describe, kubectl logs, and direct testing with kubectl exec to systematically identify whether the issue is timing, configuration, or application-level failure.