Course contentsShow
← System Design
Lesson 702 of 1,91919. Event Streaming PlatformsFree lesson

Producers and Message Publishing

Understand how producers write messages to Kafka topics and how they select target partitions using keys or custom logic.

Producers and Message Publishing

You'll learn: How Kafka producers send messages to topics and control which partition they land in.

How Producers Work

A producer is any application that publishes messages (events) to Kafka topics. Think of producers as writers adding entries to a distributed notebook—each page of the notebook is a partition, and the producer decides which page to write on.

When a producer sends a message, it performs three key steps:

  1. Serializes the message (converts your data to bytes)
  2. Determines the target partition within the topic
  3. Sends the message to the broker managing that partition

Partition Selection Strategies

Producers use different strategies to decide where messages go:

Key-Based Partitioning

If you provide a message key (like a user ID or order ID), Kafka hashes it to consistently route all messages with the same key to the same partition. This guarantees ordering for related events.

Example: All events for user_123 always go to partition 2, ensuring you process that user's events in order.

Round-Robin Distribution

If you send messages without a key, Kafka distributes them evenly across partitions in round-robin fashion—great for maximizing throughput when order doesn't matter.

Custom Partitioner

You can write custom logic to control partitioning based on your specific needs (geographic routing, load balancing, etc.).

Why This Matters

Partition selection directly impacts ordering guarantees and parallelism. Keys group related events together, while keyless publishing maximizes distribution and throughput.

Key Takeaway: Producers write messages to Kafka topics and control partition placement through message keys or custom logic—this determines both ordering guarantees and how work gets distributed across consumers.