Course contentsShow
← Cloud and Infrastructure
Lesson 1663 of 2,98934. CI/CD FoundationsFree lesson

Rollback and Recovery Patterns

Design pipelines with automatic rollback capabilities when deployments fail health checks or validation tests.

Rollback and Recovery Patterns

What you'll learn: How to design CI/CD pipelines that automatically undo failed deployments and restore service when health checks or validation tests fail.

The Core Idea

Rollback and Recovery Patterns ensure your pipeline doesn't just push forward—it knows how to step backward safely when something goes wrong. Think of it like an automatic parachute that deploys when your deployment "falls" below acceptable performance thresholds.

When you deploy new code, things can fail after the initial deployment succeeds. A health check might fail, validation tests might catch runtime issues, or monitoring might detect performance degradation. Instead of leaving broken code in production, your pipeline should automatically revert to the last known good version.

How It Works

The pattern integrates validation gates after deployment:

  1. Deploy the new version
  2. Run health checks (HTTP endpoints, database connectivity)
  3. Execute smoke tests or validation tests
  4. Monitor key metrics (error rates, latency)
  5. If any check fails → trigger automatic rollback
  6. If all checks pass → mark deployment as successful

This differs from Manual Approval Gates (which pause before deploying) by adding automated safety checks after deployment.

Common Rollback Mechanisms

  • Blue-Green Deployments: Switch traffic back to the previous environment
  • Rolling Updates: Stop rollout and redeploy previous version
  • Artifact Versioning: Redeploy the last stable artifact from your artifact repository
  • Database Migrations: Run reverse migrations if schema changes fail

The key is defining clear failure criteria: What metrics indicate failure? How long should you wait before rolling back? Should rollback be automatic or require confirmation?

Key Takeaway: Design pipelines with automatic rollback triggered by post-deployment health checks and validation—don't assume deployments that "succeed" technically are actually working correctly in production.