Log Rotation and Retention
What you'll learn: How to implement log rotation policies that balance disk space management with security retention requirements.
Why Logs Need Rotation
Logs grow continuously. Without management, a busy system can generate gigabytes of logs daily, eventually filling your disk and crashing services. Yet security investigations, compliance audits, and forensics require historical logs—sometimes weeks or months old. Log rotation solves this tension by archiving old logs and managing their lifecycle.
Core Rotation Mechanics
Log rotation moves active log files to timestamped archives at regular intervals (daily, weekly, or when reaching size thresholds). Think of it like replacing a full notebook: you file the completed one away and start fresh, but you don't throw away the old notebooks immediately.
Common rotation triggers:
- Time-based: Rotate daily at midnight, weekly on Sunday
- Size-based: Rotate when a log reaches 100MB
- Combined: Rotate daily or at 500MB, whichever comes first
Retention Policy Design
Retention determines how long archived logs survive before deletion. Your policy must satisfy:
- Compliance requirements: Many regulations mandate 90-day, 1-year, or longer retention
- Investigation needs: Security incidents may not be detected for weeks
- Storage constraints: Balance retention duration against available disk space
- Compression: Older archives can be compressed (
.gz,.xz) to save 70-90% space
Example policy: Keep 7 days uncompressed for quick access, compress days 8-90, delete after 365 days.
Implementation Considerations
Monitor disk usage regularly. Failed rotation or retention misconfiguration can lead to disk exhaustion—a availability issue and potential security blind spot. Test your rotation configuration in non-production environments first.
Key Takeaway: Log rotation and retention policies prevent disk space exhaustion while ensuring logs remain available long enough for security investigations and compliance requirements—balancing operational stability with forensic readiness.