Course contentsShow
← System Design
Lesson 726 of 1,91919. Event Streaming PlatformsFree lesson

Multi-Datacenter Replication

Cross-cluster replication using MirrorMaker 2 for disaster recovery, geographic distribution, and data locality.

Multi-Datacenter Replication

What You'll Learn

You'll learn how to replicate Kafka data across geographically distributed datacenters using MirrorMaker 2 for disaster recovery, data locality, and global distribution.

The Challenge of Geographic Distribution

When your Kafka deployment spans multiple regions or datacenters, you need a way to replicate event streams across these boundaries. Unlike the built-in replication we covered earlier (which keeps copies within a single cluster), multi-datacenter replication moves data between entirely separate Kafka clusters.

Why Replicate Across Datacenters?

Think of it like having backup offices in different cities:

  • Disaster Recovery: If one datacenter fails, another has a complete copy of your data
  • Geographic Distribution: Serve users from nearby datacenters for lower latency
  • Data Locality: Comply with regulations requiring data processing in specific regions
  • Data Aggregation: Collect streams from edge locations into a central analytics cluster

MirrorMaker 2: Cross-Cluster Replication

MirrorMaker 2 (MM2) is Kafka's built-in tool for replicating topics between clusters. It's essentially a special Kafka Connect application that:

  1. Consumes from topics in the source cluster
  2. Produces those events to the destination cluster
  3. Preserves partition counts and key-based ordering
  4. Handles offset translation so consumers can fail over seamlessly
  5. Replicates consumer group offsets and ACLs

Unlike message brokers that push data around, MM2 follows Kafka's pull model—the destination cluster actively pulls data from the source.

Replication Patterns

  • Active-Passive: One primary cluster, replicas for failover only
  • Active-Active: Multiple clusters all accept writes (requires careful design)
  • Hub-and-Spoke: Regional clusters replicate to a central aggregation cluster

Key Takeaway: MirrorMaker 2 enables multi-datacenter Kafka deployments by replicating entire topics between independent clusters, supporting disaster recovery strategies and global event distribution patterns.