Producers and Message Publishing
You'll learn: How Kafka producers send messages to topics and control which partition they land in.
How Producers Work
A producer is any application that publishes messages (events) to Kafka topics. Think of producers as writers adding entries to a distributed notebook—each page of the notebook is a partition, and the producer decides which page to write on.
When a producer sends a message, it performs three key steps:
- Serializes the message (converts your data to bytes)
- Determines the target partition within the topic
- Sends the message to the broker managing that partition
Partition Selection Strategies
Producers use different strategies to decide where messages go:
Key-Based Partitioning
If you provide a message key (like a user ID or order ID), Kafka hashes it to consistently route all messages with the same key to the same partition. This guarantees ordering for related events.
Example: All events for user_123 always go to partition 2, ensuring you process that user's events in order.
Round-Robin Distribution
If you send messages without a key, Kafka distributes them evenly across partitions in round-robin fashion—great for maximizing throughput when order doesn't matter.
Custom Partitioner
You can write custom logic to control partitioning based on your specific needs (geographic routing, load balancing, etc.).
Why This Matters
Partition selection directly impacts ordering guarantees and parallelism. Keys group related events together, while keyless publishing maximizes distribution and throughput.
Key Takeaway: Producers write messages to Kafka topics and control partition placement through message keys or custom logic—this determines both ordering guarantees and how work gets distributed across consumers.