Course contentsShow
← Python
Lesson 403 of 1,0606. Error Handling and File I/OFree lesson

Rotating Log Files to Prevent Disk Exhaustion

Learners will integrate file rotation strategies to automatically archive and purge historical log data based on size or time constraints.

In this lesson, you will learn how to prevent your server’s hard drive from filling up by automatically cycling your log files.

The Problem: Endless Logs

When you write logs to a file, the file grows every time your program runs. If left unchecked, your application will eventually consume every byte of available disk space, causing the system to crash or behave unpredictably. To solve this, we use log rotation. Instead of one infinite file, we maintain a small set of files that "rotate" as they reach a certain size or age.

Implementing Rotation

Python’s logging module provides a powerful tool called RotatingFileHandler. It allows you to specify a maximum file size and a number of "backup" files to keep. Once the current file reaches your limit, Python renames the old file and starts a fresh one. When you hit the maximum number of backups, the oldest file is deleted, ensuring your storage usage remains constant.

Here is how you set this up:

import logging
from logging.handlers import RotatingFileHandler

# Create a logger
logger = logging.getLogger("AppLogger")
logger.setLevel(logging.INFO)

# maxBytes=1024 (1KB), backupCount=3 (keep 3 old files)
handler = RotatingFileHandler(
    "app.log", maxBytes=1024, backupCount=3
)

logger.addHandler(handler)

# Simulate log growth
for i in range(100):
    logger.info(f"Logging message number {i}")

How it Works

In the example above, the RotatingFileHandler monitors the size of app.log. Once the file exceeds 1,024 bytes, Python automatically renames the current app.log to app.log.1. If that already exists, it renames it to app.log.2, and so on. Once you reach 3 backups, the oldest file is purged. This creates a "sliding window" of history, preserving recent events while automatically reclaiming space from older data.

This approach is essential for production applications, as it allows your code to run indefinitely without manual maintenance of the filesystem. By defining these boundaries upfront, you treat disk space as a finite, managed resource rather than an infinite bucket.

Key Takeaway: Use the RotatingFileHandler to cap individual log file sizes and limit the number of archived files, ensuring your application stays within its storage footprint.