StatefulSet and DaemonSet Troubleshooting
You'll learn the most common issues that arise with StatefulSets and DaemonSets and how to identify them.
StatefulSets and DaemonSets introduce unique complexities compared to standard Deployments. Because StatefulSets maintain persistent identities and storage, and DaemonSets ensure pods run on specific nodes, they each face distinct troubleshooting challenges.
Common StatefulSet Issues
PVC Retention Problems
StatefulSets create PersistentVolumeClaims (PVCs) automatically via VolumeClaimTemplates. Critical issue: when you delete or scale down a StatefulSet, the PVCs are not automatically deleted. This is by design to protect data, but it can lead to:
- Orphaned storage consuming cloud resources and costs
- Confusion when scaling back up (old PVCs reattach with stale data)
- Storage quota exhaustion
Symptom: Pods stuck in Pending because new PVCs can't be created due to quota limits, while old PVCs sit unused.
Ordered Deployment Failures
StatefulSets deploy pods sequentially (app-0, then app-1, etc.). If app-0 fails to start, the entire rollout halts. Look for initialization errors, image pull failures, or configuration mistakes in the first pod.
Common DaemonSet Issues
Node Scheduling Problems
DaemonSets should run one pod per node, but several factors can prevent this:
- Node taints: If a node has taints that the DaemonSet doesn't tolerate, no pod will schedule there
- Node selectors: Misconfigured nodeSelector or affinity rules can exclude nodes unintentionally
- Resource constraints: If a node lacks CPU/memory, the DaemonSet pod won't fit
Symptom: kubectl get pods -o wide shows missing pods on certain nodes, or pods stuck in Pending.
Key Takeaway: StatefulSet troubleshooting focuses on PVC lifecycle and ordered deployment, while DaemonSet issues typically involve node selection, taints, and resource availability across the cluster.