Outbox Pattern
What You'll Learn
You'll learn how to reliably publish events as part of database transactions without distributed locks, ensuring that data changes and event notifications never get out of sync.
The Core Problem
When your service updates a database and needs to publish an event (like "Order Created"), you face a dilemma: if you save to the database first then the event publish fails, external systems never know about the change. If you publish first then the database write fails, you've notified everyone about something that didn't happen. Using distributed transactions is complex, slow, and brittle.
How the Outbox Pattern Works
Instead of publishing events directly, write them to an outbox table in the same database transaction as your business data. This guarantees atomicity—both the data change and the event record succeed or fail together.
A separate process (often called a "message relay" or "publisher") polls or listens to the outbox table, reads unpublished events, publishes them to your message queue or event stream, then marks them as published. This component can retry safely since events remain in the outbox until confirmed published.
Practical Flow
- Begin database transaction
- Insert/update business data (e.g.,
orderstable) - Insert event record into
outboxtable - Commit transaction—both changes are atomic
- Background publisher reads from outbox
- Publishes event to queue/stream
- Marks outbox record as published or deletes it
Think of it like a restaurant where the waiter writes your order on a ticket (database + outbox) before shouting it to the kitchen (publishing). The ticket ensures nothing gets forgotten.
Key Takeaway: The Outbox Pattern solves dual-write problems by leveraging database transaction guarantees, storing events alongside data changes and publishing them asynchronously through a reliable relay process.