Log Shipping Methods
You'll learn: How different architectures move logs from applications to centralized storage, and why choosing the right model matters for reliability and performance.
Core Idea
Log shipping is the mechanism that transports log data from where it's generated (your applications) to where it's stored and analyzed. There are three primary patterns: push models, pull models, and specialized deployment patterns like sidecars and daemons.
Push vs Pull Models
Push models have applications or local agents actively send logs to a central collector. Think of it like mailing a letter—you put it in the mailbox and it gets delivered. Examples include apps writing directly to a remote endpoint or a local agent forwarding batches.
Pull models have a central service periodically fetch logs from nodes or storage. This is like a mail carrier collecting letters from dropboxes on a schedule. The collecting service controls the pace and timing.
Sidecar Pattern
A sidecar is a separate container running alongside your application container in the same pod (Kubernetes context). It reads the app's log files or stdout and ships them independently. This decouples log handling from application logic—your app just writes logs normally, the sidecar handles the rest.
Daemon-Based Collection
A daemon runs as one agent per host/node, collecting logs from all containers or processes on that machine. It's more resource-efficient than sidecars (one agent instead of many), but requires host-level access and coordination. Think of it as a neighborhood mail truck rather than individual couriers per house.
Each approach trades off isolation, resource usage, and operational complexity.
Key Takeaway: Push models give apps control, pull models centralize it; sidecars isolate per-workload, daemons share infrastructure—choose based on your scale, isolation needs, and network reliability.