Topics and Partitions
You'll learn how Kafka organizes event streams into topics and uses partitions to enable parallel processing and horizontal scaling.
What Are Topics?
A topic is a named stream of related events in Kafka. Think of it like a category or channel where producers write events and consumers read them. For example, you might have a user-signups topic for registration events or an order-placed topic for purchase transactions.
Topics provide logical organization—they group similar events together so consumers can subscribe to exactly what they need without sorting through unrelated data.
What Are Partitions?
Each topic is divided into one or more partitions. A partition is an ordered, immutable sequence of events—essentially a subset of the topic's total event stream. Each event within a partition gets a sequential ID called an offset.
Why Partitions Matter
Partitions solve two critical problems:
-
Parallelism: Multiple consumers can read from different partitions simultaneously, dramatically increasing throughput. One consumer reads partition 0, another reads partition 1, and so on.
-
Horizontal Scaling: Instead of one massive log file, your data spreads across many partitions, which can live on different brokers (servers). Need more capacity? Add more partitions and brokers.
Analogy: A topic is like a highway, and partitions are the individual lanes. More lanes mean more cars (events) can travel simultaneously without bottlenecks.
Key Properties
- Events within a single partition maintain strict ordering
- Events across partitions have no guaranteed order
- Each partition can be replicated across brokers for fault tolerance
- Producers choose which partition receives an event (often by key-based routing)
Key Takeaway: Topics organize event streams logically, while partitions enable Kafka to scale horizontally and process events in parallel—the foundation of high-throughput event streaming.