Course contentsShow
← Cloud and Infrastructure
Lesson 2033 of 2,98943. Observability - LoggingFree lesson

ELK/EFK on Kubernetes

Deploying and managing ELK/EFK stack components using StatefulSets, DaemonSets, and operators.

ELK/EFK on Kubernetes

You'll learn: How to deploy and manage the ELK/EFK stack on Kubernetes using appropriate workload types and operators.

Core Concept

Running ELK/EFK in Kubernetes requires matching each component's characteristics to the right Kubernetes workload primitive. Elasticsearch nodes need stable network identities and persistent storage—perfect for StatefulSets. Fluentd/Fluent Bit collectors must run on every node to capture logs—ideal for DaemonSets. Kibana and Logstash can use standard Deployments. Modern operators automate lifecycle management, scaling, and configuration.

Workload Mapping

StatefulSets for Elasticsearch:

  • Provides stable pod names (es-0, es-1, es-2)
  • Guarantees ordered deployment and scaling
  • Associates each pod with its own PersistentVolumeClaim
  • Enables predictable cluster formation

DaemonSets for Collectors:

  • Deploys Fluentd or Fluent Bit on every worker node automatically
  • Ensures log collection even when nodes join the cluster
  • Uses hostPath or volume mounts to access container logs at /var/log/containers

Deployments for Kibana:

  • Stateless visualization layer scales independently
  • Multiple replicas provide high availability
  • Simple rolling updates without data concerns

Operators Simplify Management

The Elastic Cloud on Kubernetes (ECK) operator and similar tools handle:

  • Automated certificate generation and TLS setup (remember ELK/EFK Stack Security)
  • Dynamic configuration changes
  • Rolling upgrades with zero downtime
  • Backup and snapshot orchestration
  • Resource scaling based on cluster health

Think of operators as specialized administrators that understand Elasticsearch clustering, shard allocation, and recovery—translating declarative YAML into operational reality.

Key Takeaway: Deploy Elasticsearch with StatefulSets for data persistence, use DaemonSets for node-level log collection, and leverage operators to automate complex lifecycle management tasks—creating a production-grade logging stack that scales with your Kubernetes infrastructure.