Building Efficient Log Rotators

In this lesson, you will learn how to automate the rotation of application log files to prevent storage overflow and maintain system performance.

Why Rotate Logs?

When a Node.js application writes logs to a single file, that file can grow indefinitely, eventually consuming all available disk space or becoming too large to open in a text editor for debugging. A log rotator solves this by "cutting" the log file once it reaches a certain size or age, renaming the current file, and starting a fresh one.

The Logic of Rotation

To build an efficient rotator, we rely on the fs module to monitor file stats and perform atomic renaming. The process follows a simple lifecycle:

1. Monitor: Check the current log file size using fs.stat().
2. Rotate: If the size exceeds your threshold (e.g., 10MB), rename the active file (e.g., app.log becomes app.1.log).
3. Archive/Cleanup: Optionally delete files older than a certain duration or keep only a fixed number of backups.

Implementing a Size-Based Trigger

We can perform this check right before writing a new log entry to ensure we never exceed our limit:

const fs = require('fs');

const MAX_SIZE = 10 * 1024 * 1024; // 10MB
const logFile = 'app.log';

function rotateLog() {
  const timestamp = Date.now();
  fs.renameSync(logFile, `app.${timestamp}.log`);
}

function writeLog(message) {
  const stats = fs.existsSync(logFile) ? fs.statSync(logFile) : { size: 0 };
  
  if (stats.size > MAX_SIZE) {
    rotateLog();
  }
  
  fs.appendFileSync(logFile, message + '\n');
}

Performance Considerations

In a high-concurrency environment, fs.statSync can become a bottleneck. To optimize, you might cache the file size in memory and update it only after every N writes, rather than checking it for every single log entry. Additionally, always use fs.rename (which is an atomic operation at the OS level) to ensure that logs are never lost during the transition between the old file and the new one. By decoupling the rotation check from the write operation, you maintain high throughput without risking data integrity.

Key Takeaway: Log rotation protects your system by enforcing a lifecycle on file growth, using atomic renaming to transition between active and archived files without data loss.