Course contentsShow
← Cloud and Infrastructure
Lesson 1434 of 2,98929. Kubernetes ObservabilityFree lesson

Node-Level Logging Agents

How logging agents run as DaemonSets to collect logs from all pods on each node.

Node-Level Logging Agents

What you'll learn: How logging agents deployed as DaemonSets automatically collect logs from every pod running on each node in your cluster.

The Core Idea

In Kubernetes, your pods generate logs that you need to collect, store, and analyze. Instead of configuring each pod individually to ship its logs somewhere, you can deploy a logging agent on every node that automatically collects logs from all containers running there.

This is accomplished using a DaemonSet — a Kubernetes resource that ensures exactly one copy of a pod runs on each node. Think of it like having a mail carrier assigned to every neighborhood: each carrier automatically picks up all the mail (logs) from every house (container) in their assigned area (node).

How It Works

When you deploy a logging agent as a DaemonSet:

  1. One agent pod per node: The DaemonSet controller ensures a logging agent runs on every node
  2. Automatic log access: The agent reads container logs from the node's filesystem (typically /var/log/pods and /var/log/containers)
  3. Centralized shipping: Each agent forwards collected logs to your chosen destination (Elasticsearch, Splunk, CloudWatch, etc.)
  4. Self-healing: If a node is added or an agent crashes, Kubernetes automatically starts the appropriate agent pod

Common Logging Agents

Popular choices include Fluentd, Fluent Bit, and Filebeat — all designed to run efficiently as DaemonSets with minimal resource overhead.

The beauty of this architecture is its simplicity: your application pods don't need to know anything about log collection. They write to stdout and stderr (as covered in Container Logging Fundamentals), and the node-level agent handles everything else.

Key Takeaway: Node-level logging agents run as DaemonSets to provide automatic, cluster-wide log collection without requiring per-pod configuration, creating a scalable and maintainable observability foundation.