Course contentsShow
← Security
Lesson 1494 of 3,05230. Operating System Security - MonitoringFree lesson

System Call Auditing

Monitoring system calls with auditd, filtering by syscall number, tracking privilege escalation attempts, and performance considerations.

System Call Auditing

What You'll Learn

You'll learn how to monitor system calls using auditd, filter by specific syscalls, detect privilege escalation attempts, and manage the performance impact of syscall auditing.

Why Monitor System Calls?

System calls are the fundamental interface between user programs and the kernel. Monitoring them is like having a security camera pointed at every doorway between user applications and the operating system's core — you can see exactly what programs are requesting from the system.

Auditing Specific System Calls

While file watches track access to specific paths, syscall rules monitor how programs interact with the kernel regardless of target:

# Monitor all execve calls (program execution)
auditctl -a always,exit -F arch=b64 -S execve -k program_execution

# Watch for privilege escalation syscalls
auditctl -a always,exit -F arch=b64 -S setuid -S setgid -k privilege_change

# Monitor file deletion attempts
auditctl -a always,exit -F arch=b64 -S unlink -S unlinkat -k file_deletion

The -S flag specifies syscall numbers or names. You can find syscall numbers in /usr/include/asm/unistd_64.h.

Tracking Privilege Escalation

Critical syscalls for detecting attacks include:

# Monitor capability changes and credential modifications
auditctl -a always,exit -F arch=b64 -S capset -S setuid -S setreuid \
  -S setresuid -k privilege_escalation

This catches programs trying to gain root privileges or modify security capabilities.

Performance Considerations

Syscall auditing is expensive — the kernel must log every matching call. Best practices:

  • Be selective: audit only critical syscalls
  • Use filters (-F uid!=0) to exclude trusted users
  • Monitor ausearch performance with auditctl -s
  • Avoid auditing high-frequency syscalls like read or write in production

Key Takeaway: System call auditing provides deep kernel-level visibility for detecting privilege escalation and malicious behavior, but requires careful filtering to avoid overwhelming performance costs.