Course contentsShow
← System Design
Lesson 663 of 1,91918. Message Queues and BrokersFree lesson

Hybrid Patterns: Topic + Queue

Combining pub-sub topics with queues per subscriber for at-least-once delivery with competing consumers.

Hybrid Patterns: Topic + Queue

You'll learn how combining pub-sub topics with individual queues gives you both broadcast flexibility and reliable, scalable message processing.


The Core Idea

Some systems need to broadcast events to multiple services (pub-sub's strength) while also ensuring at-least-once delivery and supporting competing consumers within each service (queue's strength). A hybrid pattern solves this by placing a queue behind each topic subscriber.

Here's how it works:

  1. A topic receives messages and broadcasts them to all subscribers
  2. Each subscriber has its own queue that receives a copy
  3. Multiple consumer instances can read from each queue as competing consumers
  4. Each queue independently tracks acknowledgments and retries

Real-World Analogy

Imagine a news agency (topic) that broadcasts breaking news. Each newspaper (subscriber) has its own inbox queue. Within each newspaper, multiple journalists (competing consumers) can pull stories from their shared inbox to write articles. The news agency doesn't care how each newspaper processes stories internally—it just broadcasts once.


Why This Matters

  • Fan-out broadcasting: One message reaches all interested services
  • Independent scaling: Each subscriber service scales its consumers based on its own load
  • Reliability per service: If Service A is down, its queue holds messages until it recovers—Service B is unaffected
  • No duplicate cross-service work: Each service processes the event exactly once (though retries within a service are possible)

This pattern appears in systems like AWS SNS → SQS, Google Pub/Sub with subscriptions, and Kafka consumer groups.

Key Takeaway:
Hybrid topic-queue patterns let you broadcast events widely while giving each subscriber team independent, reliable, scalable processing with competing consumers—combining the best of both messaging models.