Node-Level Logging Agents
What you'll learn: How logging agents deployed as DaemonSets automatically collect logs from every pod running on each node in your cluster.
The Core Idea
In Kubernetes, your pods generate logs that you need to collect, store, and analyze. Instead of configuring each pod individually to ship its logs somewhere, you can deploy a logging agent on every node that automatically collects logs from all containers running there.
This is accomplished using a DaemonSet — a Kubernetes resource that ensures exactly one copy of a pod runs on each node. Think of it like having a mail carrier assigned to every neighborhood: each carrier automatically picks up all the mail (logs) from every house (container) in their assigned area (node).
How It Works
When you deploy a logging agent as a DaemonSet:
- One agent pod per node: The DaemonSet controller ensures a logging agent runs on every node
- Automatic log access: The agent reads container logs from the node's filesystem (typically
/var/log/podsand/var/log/containers) - Centralized shipping: Each agent forwards collected logs to your chosen destination (Elasticsearch, Splunk, CloudWatch, etc.)
- Self-healing: If a node is added or an agent crashes, Kubernetes automatically starts the appropriate agent pod
Common Logging Agents
Popular choices include Fluentd, Fluent Bit, and Filebeat — all designed to run efficiently as DaemonSets with minimal resource overhead.
The beauty of this architecture is its simplicity: your application pods don't need to know anything about log collection. They write to stdout and stderr (as covered in Container Logging Fundamentals), and the node-level agent handles everything else.
Key Takeaway: Node-level logging agents run as DaemonSets to provide automatic, cluster-wide log collection without requiring per-pod configuration, creating a scalable and maintainable observability foundation.