Centralized Logging Architecture
What you'll learn: How distributed systems collect logs from many services into one place for analysis and troubleshooting.
The Problem with Scattered Logs
In a distributed system with dozens or hundreds of services running across multiple machines, each service writes its own logs locally. Imagine trying to debug a user request that touched five different services — you'd need to SSH into five servers, grep through five different log files, and mentally stitch together the timeline. This is slow, error-prone, and doesn't scale.
The Centralized Solution
Centralized logging means all services send their logs to a single, unified system where you can search, filter, and analyze them together. Think of it like a library: instead of books scattered across everyone's homes, they're all in one building with a catalog system.
Basic Architecture Components
The typical log aggregation pipeline has three layers:
- Log Shippers/Agents: Lightweight processes running on each service host that collect logs and forward them (e.g., Fluentd, Filebeat, Vector)
- Transport Layer: Message queues or buffers that handle log volume spikes without data loss (e.g., Kafka, Redis)
- Storage and Indexing: A centralized database optimized for log search and retention (e.g., Elasticsearch, Loki, S3 + Athena)
Services write structured logs (remember JSON format from previous lessons) which agents tail and ship. The transport layer decouples producers from consumers, preventing log loss during traffic spikes. The storage layer indexes logs by time, service, correlation ID, and other fields for fast querying.
Key Takeaway: Centralized logging architecture solves the distributed debugging problem by funneling all service logs through agents → transport → storage, enabling unified search and correlation across your entire system.