Course contentsShow
← Docker, Kubernetes & CI/CD
Lesson 443 of 8026. Kubernetes Traffic and ScalingFree lesson

Automating Rollback Procedures for Failed Deployments

You will define automated health checks that trigger an immediate reversion to the previous stable version upon deployment failure.

In this lesson, you will learn how to configure automated health checks that trigger an immediate reversion to the previous stable version when a new deployment fails.

In a production environment, relying on manual intervention to roll back a broken deployment introduces unnecessary downtime. By leveraging Kubernetes' native readinessProbes alongside RollingUpdate strategies, we can automate this process. The system monitors the health of new pods; if they fail to pass their readiness checks within a defined timeframe, the Deployment controller stops the rollout and preserves the last known stable state.

Defining the Safety Net

To automate this, you define a readinessProbe that tests your application's critical functionality. If the new pods cannot handle traffic, the deployment will stall, preventing the "bad" code from reaching your users.

spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  template:
    spec:
      containers:
      - name: web-app
        image: my-app:v2
        readinessProbe:
          httpGet:
            path: /health
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 10
          failureThreshold: 3

How the Rollback Triggers

When you update your image to v2, Kubernetes begins replacing v1 pods. If your application code has a bug that causes the /health endpoint to fail or time out, the failureThreshold is reached. Because Kubernetes observes that these new pods are not "Ready," it refuses to terminate the remaining v1 pods.

While the system is effectively "stuck" in a failed state, you can invoke the automated rollback command to revert the deployment to its previous revision:

kubectl rollout undo deployment/web-app

In more advanced CI/CD setups, this command is triggered automatically by your pipeline tool upon receiving a "Deployment Failed" notification from the Kubernetes API. By combining aggressive health checks with automated rollout undo commands, you transform a potentially disastrous release into a controlled, self-healing operation.

Key Takeaway: Automated rollbacks rely on strict readiness probes to catch failures early, allowing you to trigger an instant reversion to the last stable configuration before your traffic is fully impacted.