Log Retention and Lifecycle Policies
What you'll learn: How to configure retention periods, archival strategies, and compliance-driven log storage policies to balance security needs with cost and regulatory requirements.
Why Retention Matters
Logs are evidence. They prove who accessed what, when anomalies occurred, and how incidents unfolded. But storing everything forever is expensive and sometimes legally problematic. Log retention policies define how long you keep logs in different storage tiers before archival or deletion.
Think of it like a filing cabinet: recent cases stay on your desk (hot storage), older cases move to the filing room (warm storage), and ancient records go to the warehouse (cold/archive storage) or get shredded (deletion).
Configuring Retention Periods
Retention requirements come from multiple sources:
- Compliance regulations: HIPAA requires 6 years, PCI-DSS mandates 1 year of audit logs, GDPR limits unnecessary retention
- Incident response needs: Security teams typically want 90+ days of hot logs for investigation
- Legal hold requirements: Litigation may freeze deletion policies temporarily
- Cost optimization: Moving logs to cheaper storage tiers after 30-90 days
Archival Strategies
A typical lifecycle policy might look like:
- Days 0-30: CloudTrail logs in standard storage, instantly queryable
- Days 31-365: Moved to infrequent access storage, slower retrieval
- 1+ years: Archived to Glacier-equivalent cold storage, hours to retrieve
- After compliance period: Automatic deletion
Different log types warrant different policies. VPC Flow Logs for troubleshooting might only need 30 days, while financial transaction logs require years of retention.
Balancing Act
The key is documenting your retention rationale. Why 90 days for application logs? Because your mean-time-to-detect is 45 days, so you need double that window. This documentation satisfies auditors and provides justification when questioned.
Key Takeaway: Effective log retention policies balance compliance mandates, security investigation windows, and storage costs through automated lifecycle transitions—keeping critical evidence accessible while archiving or deleting data appropriately.