Course contentsShow
← Linux
Lesson 1297 of 1,81911. Automation and System SchedulingFree lesson

Automation with Journald Log Patterns

Trigger scripts based on specific error signatures appearing in system logs.

In this lesson, you will learn how to trigger automated system tasks the moment specific error patterns appear in your journald logs.

While timers are great for scheduled maintenance, some issues require immediate intervention. Instead of polling the log file with a script, we can leverage systemd-journal-gatewayd or a helper tool like systemd-journal-upload to watch the journal stream. However, the most direct, "system-native" way to bridge logs to automation is by using a small service that "tails" the journal and executes a unit when a match occurs.

The "Log-Triggered" Pattern

The core concept is to create a service that executes a "watcher" script. This script monitors the journal via journalctl -f (follow mode). When a specific regex pattern is matched—for example, a database connection timeout or a security violation—the script triggers another systemd service unit using systemctl start.

Consider this simplified watcher script (/usr/local/bin/log-watcher.sh):

#!/bin/bash
# Monitor the journal for a specific error signature
journalctl -f -n 0 --output=cat | while read line; do
    if [[ "$line" =~ "CRITICAL: Database connection lost" ]]; then
        systemctl start db-reconnect.service
    fi
done

Why Decouple the Watcher?

By separating the Watcher (which just monitors the stream) from the Action (the db-reconnect.service), you maintain the principle of single responsibility. The watcher doesn't need to know how to fix the database; it only needs to know that the error occurred.

You should wrap the watcher script itself in a systemd service unit file so that it starts at boot and stays alive. Ensure that this watcher service is configured with Restart=always so that if the log stream is interrupted or the process crashes, it automatically resumes monitoring.

By combining this with the dependency management you learned in previous lessons, you can ensure that the "Action" service is correctly ordered and constrained by system resources, preventing your automation from spiraling out of control if a log error loops repeatedly.

Key Takeaway: Use a persistent "watcher" service to stream journalctl output and trigger specific systemd units when matching error patterns are detected in real-time.