Fallback and Retry Logic
What you'll learn: How to ensure notification delivery even when primary channels fail by implementing intelligent fallback and retry mechanisms.
The Core Problem
When sending millions of notifications, failures happen constantly: SMS providers go down, email servers reject messages, push notification services time out. Without fallback and retry logic, these transient failures result in lost notifications and poor user experience.
Retry Logic for Transient Errors
Not all failures are permanent. A provider might be temporarily overloaded or experiencing a brief outage. Implement exponential backoff retry:
- First retry: Wait 1 second
- Second retry: Wait 2 seconds
- Third retry: Wait 4 seconds
- Max retries: Typically 3-5 attempts
Use jitter (random delay variation) to prevent thundering herd problems when many notifications retry simultaneously.
retry_delay = base_delay * (2 ^ attempt) + random(0, 1000ms)
Some errors shouldn't trigger retries: invalid phone numbers, malformed email addresses, or revoked device tokens are permanent failures. Classify errors appropriately.
Channel Fallback Strategy
When the primary channel exhausts retries, automatically fall back to secondary channels based on your Channel Selection Strategy (lesson 1688):
- Try push notification (instant, free)
- Fall back to SMS (reliable, costs money)
- Final fallback to email (slowest but most reliable)
Your Channel Provider Abstraction (lesson 1690) makes this seamless—the system just calls notificationService.send() with the next channel priority.
Track fallback metrics: if you're constantly falling back from push to SMS, investigate the push provider or your Delivery Receipt Tracking (lesson 1693).
Key Takeaway: Implement exponential backoff with jitter for transient failures and automatic channel fallback for permanent failures—this ensures maximum delivery reliability while controlling costs and respecting rate limits.