Course contentsShow
← Cloud and Infrastructure
Lesson 671 of 2,98912. Monitoring FoundationsFree lesson

CloudWatch Logs Structure

Understanding log groups, log streams, retention policies, and how AWS services publish logs to CloudWatch.

CloudWatch Logs Structure

You'll learn how CloudWatch organizes log data using log groups, log streams, and retention policies to centrally manage application and service logs.

The Organizational Hierarchy

CloudWatch Logs uses a three-level hierarchy to organize massive amounts of log data from your AWS services and applications:

Log Groups

Think of a log group as a folder or container for related logs. You typically create one log group per application, service, or purpose. For example, you might have /aws/lambda/my-function for a Lambda function or /ecs/my-api for a container application. Log groups are where you set retention policies that apply to all logs inside them.

Log Streams

Within each log group, individual log streams represent a specific source generating logs. A stream is like a single file within the folder. For a Lambda function, each concurrent execution might create its own stream. For an EC2 instance, you might have separate streams for different log files (access logs, error logs, system logs).

Log Events

The actual log entries—timestamped messages with your data—live within streams. These are ordered chronologically and immutable once written.

How AWS Services Publish Logs

AWS services automatically publish logs to CloudWatch when configured. Lambda functions send logs directly to /aws/lambda/<function-name>. VPC Flow Logs, CloudTrail, and RDS can all be configured to stream to specific log groups. You control this through service-specific settings or IAM permissions that allow writing to CloudWatch Logs.

Retention Policies

You set retention at the log group level, choosing periods from 1 day to 10 years, or indefinite retention. After the retention period expires, CloudWatch automatically deletes old log data, helping control storage costs.

Key Takeaway: CloudWatch Logs organizes data hierarchically—log groups contain log streams, which contain log events—with retention policies defined at the group level to manage cost and compliance.