Debugging PAM Authentication Failures with Journald

In this lesson, you will learn how to leverage journald to isolate the specific PAM module responsible for rejecting a failed login attempt.

When a user fails to authenticate, PAM (Pluggable Authentication Modules) functions as a silent, layered stack. It executes multiple modules in sequence—like pam_unix.so for password verification or pam_faillock.so for account lockout—making it difficult to know which one triggered the failure just by looking at the login prompt.

Isolating the Failure with journalctl

Because PAM logs authentication events to systemd-journald, you can trace the stack execution by filtering for the sshd or login service tags. To pinpoint the exact failure point, use the following command immediately after a failed attempt:

journalctl -u sshd --since "1 minute ago" -p info

This output displays the PAM stack transition. Look for lines that mention pam_ and the status authentication failure. A typical failure log looks like this:

sshd[1234]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=192.168.1.5
sshd[1234]: pam_faillock(sshd:auth): Authentication attempt failed for user 'dev_user'

Analyzing the Stack Sequence

In this example, the logs reveal two critical things:

1. The Culprit: The module pam_faillock followed the failure logged by pam_unix, indicating that while the password might have been incorrect, the account was also being tracked for multiple failed attempts.
2. The Context: By observing the stack order in the journal, you can determine if a module is returning a required failure (which halts the stack) or if an optional module is simply logging a warning without blocking the user.

If the logs are too sparse, you can temporarily increase verbosity by editing /etc/pam.d/common-auth and adding the debug flag to the relevant module line (e.g., auth required pam_unix.so debug). Remember to remove it once you identify the bottleneck, as debug logs can grow rapidly.

Key Takeaway: Use journalctl -u [service] to trace the specific PAM module sequence, allowing you to identify if a failure is caused by an incorrect password, an account lockout, or a misconfigured security policy.