In this lesson, you will learn how to synchronize database operations with job enqueueing to prevent the common pitfall of "orphaned" background tasks.
The Problem: The Dual-Write Trap
When building background processing systems, developers often perform a database update (e.g., updating an order status) and then immediately push a job to a queue (e.g., sending an email). A subtle but devastating failure occurs if the process crashes or loses network connectivity between these two actions.
If your database update succeeds but the queue.add() call fails or never executes, you end up with an "orphaned" state: the system thinks the work is done, but the background task that was supposed to trigger the side effect never runs. Conversely, if you enqueue first and the database update fails, you trigger side effects for a task that shouldn't exist.
Achieving Atomicity via Transactions
To guarantee that the database update and the job submission are treated as a single unit of work, you must link them through a Transactional Outbox Pattern or a database-backed queue.
Instead of calling your queue provider directly during your business logic, write the job details into a dedicated outbox table within the same database transaction used for your primary data.
// A pattern to ensure atomicity
await db.transaction(async (trx) => {
// 1. Update your business data
await trx('orders').where({ id }).update({ status: 'processed' });
// 2. Insert the task into an internal outbox table in the SAME transaction
await trx('job_outbox').insert({
task_type: 'send_email',
payload: JSON.stringify({ orderId: id }),
created_at: new Date()
});
});
By keeping the job description inside the database transaction, you guarantee that the job is only recorded if the business data changes successfully. A separate, reliable process then polls this job_outbox table and pushes those items into your actual task queue (like Redis or RabbitMQ). If the polling process fails, it simply retries, knowing that the record in the outbox table remains until it is successfully offloaded.
Key Takeaway: Never trigger an external queue operation outside of your database transaction; instead, write the intent to an outbox table within the same transaction to guarantee consistency.