ELK/EFK on Kubernetes
You'll learn: How to deploy and manage the ELK/EFK stack on Kubernetes using appropriate workload types and operators.
Core Concept
Running ELK/EFK in Kubernetes requires matching each component's characteristics to the right Kubernetes workload primitive. Elasticsearch nodes need stable network identities and persistent storage—perfect for StatefulSets. Fluentd/Fluent Bit collectors must run on every node to capture logs—ideal for DaemonSets. Kibana and Logstash can use standard Deployments. Modern operators automate lifecycle management, scaling, and configuration.
Workload Mapping
StatefulSets for Elasticsearch:
- Provides stable pod names (
es-0,es-1,es-2) - Guarantees ordered deployment and scaling
- Associates each pod with its own PersistentVolumeClaim
- Enables predictable cluster formation
DaemonSets for Collectors:
- Deploys Fluentd or Fluent Bit on every worker node automatically
- Ensures log collection even when nodes join the cluster
- Uses hostPath or volume mounts to access container logs at
/var/log/containers
Deployments for Kibana:
- Stateless visualization layer scales independently
- Multiple replicas provide high availability
- Simple rolling updates without data concerns
Operators Simplify Management
The Elastic Cloud on Kubernetes (ECK) operator and similar tools handle:
- Automated certificate generation and TLS setup (remember ELK/EFK Stack Security)
- Dynamic configuration changes
- Rolling upgrades with zero downtime
- Backup and snapshot orchestration
- Resource scaling based on cluster health
Think of operators as specialized administrators that understand Elasticsearch clustering, shard allocation, and recovery—translating declarative YAML into operational reality.
Key Takeaway: Deploy Elasticsearch with StatefulSets for data persistence, use DaemonSets for node-level log collection, and leverage operators to automate complex lifecycle management tasks—creating a production-grade logging stack that scales with your Kubernetes infrastructure.