Course contentsShow
← Cloud and Infrastructure
Lesson 2485 of 2,98953. High Availability PatternsFree lesson

Split-Brain Prevention

Understanding split-brain scenarios, quorum-based decisions, and fencing mechanisms to prevent data corruption.

Split-Brain Prevention

You'll learn: How to prevent the catastrophic scenario where two nodes believe they're both the primary, leading to data corruption and inconsistency.

What is Split-Brain?

A split-brain occurs when a network partition separates nodes in your high-availability cluster, causing multiple nodes to simultaneously believe they're the primary (active) node. Imagine two captains on a ship, each giving contradictory orders—chaos ensues.

When this happens, both nodes may accept writes, causing divergent data states. When the network heals, you're left with conflicting versions of reality that are impossible to automatically reconcile.

Prevention Mechanisms

Quorum-Based Decisions

A quorum requires that a majority of nodes agree before any node can become primary. With 3 nodes, you need at least 2 votes. If network splits create two groups (2 nodes vs 1 node), only the majority group can operate.

This is why odd numbers of nodes (3, 5, 7) are preferred—there's always a clear majority on one side of any partition. The minority side enters read-only or standby mode.

Fencing Mechanisms

Fencing forcibly prevents a node from accessing shared resources when it's suspected of being split-brained. Think of it as revoking credentials.

Common fencing techniques:

  • STONITH ("Shoot The Other Node In The Head"): Physically power off or reboot the suspect node
  • Network fencing: Revoke network access to storage or databases
  • Disk fencing: Disable a node's ability to write to shared storage

Fencing is aggressive but necessary—better to lose one node temporarily than corrupt your entire dataset.

Why It Matters

Without split-brain prevention, your failover process that's designed to increase availability can actually destroy data integrity during network failures.

Key Takeaway: Split-brain prevention uses quorum-based voting to ensure only one primary exists, and fencing mechanisms to forcibly isolate nodes that can't reach consensus, protecting data integrity during network partitions.