In this lesson, you will learn how to maintain data integrity in distributed systems by implementing undo logic for failed long-running processes.
The Saga Challenge
When a business process spans multiple services—such as OrderService, InventoryService, and PaymentService—you cannot rely on standard database transactions (ACID) to ensure atomicity. If the PaymentService fails after the InventoryService has already deducted stock, you are left with a data inconsistency. Because we cannot "roll back" a committed database change in another service, we must perform a Compensating Transaction: an explicit operation that reverses the side effects of a previous, successful step.
Implementing Compensation
Think of a Saga as a sequence of forward-moving operations. For every action that changes state, you must define a corresponding "undo" action.
public async Task Handle(OrderPlacedEvent message)
{
try
{
await _inventoryService.ReserveItem(message.OrderId);
await _paymentService.ProcessPayment(message.OrderId);
}
catch (PaymentException)
{
// Compensation: Undo the inventory reservation
await _inventoryService.ReleaseItem(message.OrderId);
throw; // Escalate failure
}
}
Designing Robust Undo Logic
To ensure your compensating transactions actually work, follow these rules:
- Idempotency is Mandatory: Since your messaging infrastructure (like an Outbox or retries) might deliver a compensation message more than once, ensure the
ReleaseItemmethod checks if the stock was already released before trying to update the database. - Semantic Undo: Compensation isn't always a "delete." If you created a record, you might update its status to
Cancelledrather than deleting it, preserving the audit trail. - Ensure Reliability: What if the
ReleaseItemcall fails? The compensating transaction must be retried until it succeeds. Often, this involves placing the compensation command into yourOutboxor a message queue to ensure it is eventually processed.
If the compensation itself fails repeatedly, you must transition the saga to a "manual intervention" state, as the system can no longer guarantee consistency automatically.
Key Takeaway: Compensating transactions are the "undo" mechanism for distributed sagas; they must be idempotent and designed to execute reliably even when initial operations fail.