Fluentd and Fluent Bit
What you'll learn: How to deploy Fluentd or Fluent Bit as logging agents to collect and forward Kubernetes logs to centralized systems.
What Are Fluentd and Fluent Bit?
Fluentd and Fluent Bit are open-source log collection and forwarding tools that act as node-level logging agents in Kubernetes. Think of them as postal workers who gather all the mail (logs) from houses (containers) on their route (node) and deliver them to a central post office (logging backend like Elasticsearch, Splunk, or CloudWatch).
Fluentd is feature-rich and supports extensive plugins, while Fluent Bit is lightweight and optimized for resource-constrained environments—ideal for Kubernetes nodes where efficiency matters.
Deploying as DaemonSets
Both tools typically run as DaemonSets, ensuring one logging agent pod per node. They mount the node's container log directory (usually /var/log/containers/) to read log files directly:
volumeMounts:
- name: varlog
mountPath: /var/log
- name: varlibdockercontainers
mountPath: /var/lib/docker/containers
readOnly: true
How They Work
- Collection: The agent tails container log files on the host filesystem
- Parsing: Logs are parsed (often from JSON format) and enriched with Kubernetes metadata (pod name, namespace, labels)
- Filtering: You can transform, filter, or route logs based on rules
- Forwarding: Processed logs are shipped to your chosen backend(s)
This architecture separates log collection from your application containers, implementing the node-level logging pattern you learned earlier.
Key Takeaway: Fluentd and Fluent Bit are lightweight DaemonSet-based logging agents that collect container logs from nodes and forward them to centralized logging systems, enabling cluster-wide observability without modifying your applications.