In this lesson, you will learn how to use lightweight arbitrator nodes to maintain quorum and prevent split-brain scenarios in small Galera clusters.
The Problem: Even-Numbered Clusters
In a Galera Cluster, nodes use a voting mechanism to reach a consensus on transaction ordering. For a cluster to remain operational, it requires a quorum—a majority of the nodes (more than 50%) must be online and communicating.
If you have a two-node cluster, the system is fragile. If one node fails, the remaining node cannot be certain if its partner crashed or if the network has simply split the cluster in two (a "split-brain" scenario). Because it no longer holds a majority, the surviving node will stop accepting writes to prevent data inconsistency.
The Solution: The Arbitrator Node
An garbd (Galera Arbitrator) is a lightweight process that acts as a voting member of the cluster without storing any data. It participates in the consensus process to help your cluster reach a majority, but it does not participate in the actual replication of database tables.
By adding an garbd instance to a two-node cluster, you effectively turn it into a "three-node" cluster for voting purposes. Now, if one database node fails, the remaining database node and the Arbitrator still form a majority (2 out of 3), allowing your primary node to continue processing transactions.
Deployment Example
You typically run the arbitrator on a separate, small machine or within a different network zone. You start it using the command line, pointing it to your existing cluster nodes:
garbd -a gcomm://192.168.1.10:4567,192.168.1.11:4567 -g my_cluster_name
In this command:
-aspecifies the address of the existing nodes.-gspecifies the group name of your cluster.
The Arbitrator connects to the cluster, receives the write-set broadcasts, and performs the necessary voting to maintain quorum, but it never persists the data to disk. This is a cost-effective way to provide high availability without the overhead of maintaining an additional full database instance.
Key Takeaway: Arbitrator nodes provide a low-resource way to establish a majority vote in small clusters, ensuring your database remains write-accessible even when one data node goes offline.