Securing etcd and Secrets Management
What you'll learn: How to protect Kubernetes' critical datastore and handle sensitive information securely.
Why etcd Security Matters
etcd is Kubernetes' distributed key-value store—think of it as the cluster's brain and memory combined. It stores everything: cluster configuration, active workloads, network policies, and most critically, Secrets. If an attacker gains access to etcd, they can read passwords, tokens, certificates, and manipulate the entire cluster state.
By default, Kubernetes Secrets are only base64-encoded (not encrypted) in etcd—essentially stored in plain text.
Encryption at Rest
Enable encryption at rest for etcd to ensure Secrets are encrypted on disk:
- Create an
EncryptionConfigurationfile specifying encryption providers (likeaescbcor external KMS) - Pass this configuration to the API server via
--encryption-provider-config - Existing Secrets must be re-written to apply encryption
This protects data if someone gains filesystem access to etcd nodes.
Access Controls
Restrict who can reach etcd:
- Network segmentation: Isolate etcd on a private network accessible only to API server nodes
- Mutual TLS (mTLS): Require client certificates for all etcd connections
- RBAC boundaries: Limit API server permissions; overly permissive RBAC can allow privilege escalation to read Secrets directly
Secure Secret Handling Practices
- Never commit Secrets to source control (enforce via supply chain scanning)
- Use external secret management systems (HashiCorp Vault, AWS Secrets Manager) integrated via operators
- Implement Secret rotation policies
- Apply principle of least privilege: Grant Pods access only to the Secrets they need through RBAC
- Monitor Secret access patterns for anomalies
Key Takeaway: Securing etcd through encryption at rest, strict access controls, and proper Secret management is essential—it's the single source of truth for your entire Kubernetes cluster and the primary target for attackers seeking credentials.