Designing for Resilience to Network Partitions

In this lesson, you will learn how transaction managers determine whether to continue processing requests or stop entirely when a node becomes isolated from the rest of the cluster during a network partition.

The Quorum Necessity

When a network partition occurs, a database cluster essentially splits into two or more "islands" that cannot communicate. A transaction manager must decide: "Do I have enough consensus to safely commit data?"

To prevent two parts of the cluster from accepting conflicting writes (which would destroy transactional integrity), most modern systems rely on a quorum. A quorum is the minimum number of nodes (usually a majority) required to agree on a state change. If a transaction manager finds itself on a "minority" partition, it is effectively isolated from the quorum.

Transaction Manager Behavior

When a node loses its connection to the majority, the transaction manager typically shifts into a "read-only" or "halt" state. It does this to prevent split-brain scenarios, where different nodes independently confirm transactions that are logically incompatible.

Consider a cluster of 5 nodes. A quorum requires 3 nodes. If 3 nodes are on one side of a partition and 2 are on the other:

• The Majority Side (3 nodes): Can continue to progress transactions because it still holds the majority quorum.
• The Minority Side (2 nodes): The transaction manager detects the lack of heartbeat from the majority. It will stop accepting new COMMIT commands for any distributed transaction.

-- Conceptual logic inside the transaction manager:
IF (active_nodes_reachable < quorum_threshold) {
    SET database_mode = READ_ONLY;
    RETURN "Error: Insufficient nodes to guarantee durability.";
} ELSE {
    PROCEED_WITH_TRANSACTION();
}

If the transaction manager on the minority side allowed writes to proceed, you would end up with two different versions of the truth. Once the network partition heals, merging those conflicting logs would be mathematically impossible without violating the consistency rules we’ve discussed in previous lessons. Therefore, the transaction manager's primary responsibility during a partition is to sacrifice availability to protect consistency. By choosing to halt, the system ensures that once the network is restored, the database remains in a valid, serializable state.

Key Takeaway: During a network partition, an isolated transaction manager must cease accepting writes if it cannot reach a majority quorum, choosing to stay offline rather than risk violating data consistency.