Course contentsShow
← Cloud and Infrastructure
Lesson 659 of 2,98912. Monitoring FoundationsFree lesson

Log Streaming vs Log Shipping

Comparing real-time log streaming with batched log shipping approaches.

Log Streaming vs Log Shipping

You'll learn: The difference between real-time log streaming and batched log shipping, and when to use each approach.

Two Ways to Move Logs

Once you've set up log collection agents (our previous lesson), you need to decide how those logs travel to your centralized logging system. There are two fundamental approaches: streaming and shipping.

Log Streaming: The Live Feed

Log streaming sends log data in real-time, as events occur. Think of it like a live news broadcast—the moment something happens, it's transmitted immediately to your centralized logging system.

  • Events flow continuously
  • Near-instant visibility into issues
  • Higher network bandwidth usage
  • More complex infrastructure
  • Best for critical systems requiring immediate alerts

Log Shipping: The Batch Delivery

Log shipping collects logs locally and sends them in batches at regular intervals (every 5 minutes, hourly, etc.). It's like the postal service—collecting mail throughout the day and delivering it in scheduled rounds.

  • Logs buffered locally before transmission
  • Reduced network overhead
  • Slight delay in visibility (acceptable for many use cases)
  • Simpler to implement
  • Better for high-volume, non-critical logs

Choosing Your Approach

Use streaming when you need immediate detection of errors, security incidents, or performance degradation. Use shipping for audit logs, debug logs, or systems where a few minutes' delay won't impact operations.

Many organizations use a hybrid: streaming for application logs at ERROR level, shipping for INFO and DEBUG levels.

Key Takeaway: Log streaming provides real-time visibility with higher costs and complexity, while log shipping batches data for efficiency—choose based on how quickly you need to see and react to events.