Implementing Idempotent Message Consumers

In this lesson, you will learn how to design background message consumers that safely process the same message multiple times without corrupting your system state.

The Idempotency Challenge

In distributed systems, "at-least-once" delivery is the standard. This means network glitches or process restarts can cause your BackgroundService to receive the same message twice. If your consumer performs an action like "increment account balance," receiving a duplicate message would result in incorrect data. An idempotent consumer ensures that performing the same operation multiple times has the same effect as performing it exactly once.

Strategy: The Idempotency Key

The most robust approach is to track processed MessageIds. Before executing any business logic, your consumer should verify if the ID has already been persisted to a database.

protected override async Task ExecuteAsync(CancellationToken stoppingToken)
{
    await foreach (var message in _messageQueue.Reader.ReadAllAsync(stoppingToken))
    {
        using var scope = _serviceScopeFactory.CreateScope();
        var db = scope.ServiceProvider.GetRequiredService<MyDbContext>();

        // 1. Check if processed
        if (await db.ProcessedMessages.AnyAsync(m => m.Id == message.Id))
        {
            continue; // Already handled, skip safely
        }

        // 2. Perform business logic in a transaction
        using var transaction = await db.Database.BeginTransactionAsync();
        try 
        {
            await ProcessBusinessLogic(message);
            
            // 3. Record the Idempotency Key
            db.ProcessedMessages.Add(new ProcessedMessage { Id = message.Id });
            await db.SaveChangesAsync();
            
            await transaction.CommitAsync();
        }
        catch 
        {
            await transaction.RollbackAsync();
            throw; // Allow the broker to retry later
        }
    }
}

Why this works

By wrapping the business logic and the persistence of the MessageId in a single database transaction, you create an atomic operation. If the process crashes after the business logic but before recording the MessageId, the database transaction rolls back. When the message is redelivered, the system treats it as a fresh request, ensuring consistency.

Always favor operations that are natively idempotent, such as UPDATE balance = 100 rather than UPDATE balance = balance + 10. When native idempotency isn't possible, the MessageId tracking pattern acts as your safety net.

Key Takeaway: Design your consumers to track processed message IDs within an atomic transaction to ensure that duplicate deliveries result in exactly one execution of business logic.