Designing Idempotent Async Tasks

In this lesson, you will learn how to design asynchronous tasks that can safely fail and restart without causing duplicate side effects in your database.

The Problem: The "Ghost" Write

In asynchronous Node.js systems, network failures or process crashes often force us to retry tasks. Imagine a function that charges a customer’s credit card and updates your database. If the network drops after the bank processes the payment but before your database records it, your retry logic will attempt to charge the customer a second time. This is the opposite of idempotency: an idempotent operation is one that can be applied multiple times without changing the result beyond the initial application.

Solving with Idempotency Keys

To prevent these "ghost" side effects, we use idempotency keys. A key is simply a unique identifier (like a UUID) attached to a specific task. Before executing a side effect, your task checks if that specific key has already been processed in your database.

Consider this pattern:

async function processPayment(orderId, idempotencyKey) {
  // 1. Check if this specific task has already finished
  const existingRecord = await db.payments.findOne({ idempotencyKey });
  if (existingRecord) {
    return existingRecord; // Skip logic, return original result
  }

  // 2. Perform the side effect
  const paymentResult = await bankProvider.charge(orderId);

  // 3. Save the result and the key together
  await db.payments.insert({
    orderId,
    idempotencyKey,
    status: 'completed'
  });

  return paymentResult;
}

By wrapping the check and the save in a consistent flow, you ensure that even if processPayment is called five times due to network retries, the bankProvider.charge logic only executes on the first attempt. The subsequent calls will identify the existing idempotencyKey in the database and exit early, effectively ignoring the redundant triggers.

This technique turns a "dangerous" retry into a safe, deterministic operation, shielding your database from corrupted states caused by asynchronous interruptions.

Key Takeaway: Always pair your asynchronous side effects with a unique idempotency key checked against your database to ensure that retried tasks produce the same result as the initial execution.