Advanced Log Rotation Management

In this lesson, you will learn how to synchronize systemd timers with log rotation processes to ensure that active log files are never truncated or moved while a background process is mid-write.

The Conflict: Rotation vs. Execution

In high-throughput environments, log rotation often relies on logrotate, which is typically triggered by a daily cron or a systemd timer. However, if your custom automation services are writing to these same logs during the rotation window, you risk "file handle hijacking." If a process holds an open file descriptor to a log file that is moved or renamed, the process may continue writing into the "void" of the old file, leading to silent data loss when that file is eventually purged.

Orchestrating with Systemd Timers

To prevent this, you should avoid relying on the default system-wide rotation schedule for mission-critical logs. Instead, use systemd to create an atomic dependency chain. By defining a custom timer unit that triggers a oneshot service, you can execute a pre-rotate hook that pauses your data-writing services before the rotation occurs.

Consider this snippet for a service unit (log-rotate-job.service) that coordinates the rotation:

[Unit]
Description=Atomic Log Rotation
# Stop the data writer before rotating
Conflicts=data-writer.service

[Service]
Type=oneshot
ExecStart=/usr/sbin/logrotate /etc/logrotate.d/myapp
# Restart the data writer after rotation finishes
ExecStartPost=/usr/bin/systemctl start data-writer.service

By adding Conflicts=data-writer.service and managing the state transition explicitly within the unit, you ensure that the data-writer is not actively holding a file handle when the rotation command is issued.

Implementation Strategy

1. Decouple: Remove the application's logs from the global /etc/logrotate.conf to prevent uncontrolled rotation.
2. Explicit Dependency: Create a dedicated timer unit that triggers your custom log-rotate-job.service.
3. Graceful Handoff: Use ExecStartPre to signal your application (e.g., via SIGHUP or a systemctl stop) to flush buffers and close descriptors before the rename occurs.

This approach transforms log rotation from a "blind" background task into a deterministic, orchestrated event that respects your application’s current state.

Key Takeaway: To prevent data loss during log rotation, use systemd unit dependencies to ensure data-writing services are paused or have flushed their file descriptors before the rotation process takes action.