Implementing Pod Anti-Affinity for High Availability

In this lesson, you will learn how to enforce strict scheduling rules to ensure your stateful replicas are spread across different physical nodes to prevent single points of failure.

The Problem: Node Dependency

When you deploy a StatefulSet with multiple replicas, Kubernetes' default scheduler tries its best to balance the load. However, it does not inherently guarantee that those pods won't land on the same physical host. If a node hosting three of your database replicas crashes, your entire application becomes unavailable, even though you have multiple replicas running. To build a truly resilient system, we must force Kubernetes to distribute pods across different nodes.

Introducing Pod Anti-Affinity

Pod Anti-Affinity is a rule that tells the Kubernetes scheduler: "Do not place this new pod on a node if a pod matching this label already exists there." By using requiredDuringSchedulingIgnoredDuringExecution, we create a hard constraint that the scheduler must follow to ensure high availability.

Implementation Example

We define this in the spec section of our StatefulSet pod template. We look for a label that identifies our application and specify the topologyKey, which is typically kubernetes.io/hostname to ensure separation at the node level:

spec:
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - database
        topologyKey: "kubernetes.io/hostname"

In this configuration, if the scheduler tries to place a second "database" pod onto a node that already hosts one, it will refuse to do so. This guarantees that your replicas are physically separated across your cluster infrastructure. If a specific node fails, only one replica is impacted, allowing the rest of your cluster to remain operational and continue serving traffic. This is a critical step in transitioning from a simple deployment to a production-grade distributed system.

Key Takeaway: Pod Anti-Affinity forces your replicas onto separate physical nodes, turning a cluster-wide risk into a single-node failure that your system can survive.