Configuring Audit Log Backends and Persistence

In this lesson, you will learn how to configure the Kubernetes API server to reliably stream audit logs to persistent local storage or remote webhooks for long-term security auditing.

When the Kubernetes API server performs an operation, it generates audit logs containing the "who, what, and when" of every cluster request. By default, these logs are ephemeral. To ensure they are available for compliance or forensic analysis, we must configure Audit Backends.

Configuring Local File Persistence

To persist logs locally, you must update your API server configuration file (usually found at /etc/kubernetes/manifests/kube-apiserver.yaml). You define the --audit-log-path and other parameters to specify where the host writes these logs:

spec:
  containers:
  - command:
    - kube-apiserver
    - --audit-policy-file=/etc/kubernetes/audit-policy.yaml
    - --audit-log-path=/var/log/kubernetes/audit.log
    - --audit-log-maxsize=100
    - --audit-log-maxbackup=5
    volumeMounts:
    - mountPath: /var/log/kubernetes
      name: audit-log
  volumes:
  - hostPath:
      path: /var/log/kubernetes
    name: audit-log

Using hostPath is critical here; it ensures that even if the API server pod restarts, the logs remain on the underlying node disk. You must also implement log rotation using maxsize and maxbackup to prevent the disk from filling up and crashing the control plane.

Configuring Remote Webhook Backends

For enterprise environments, storing logs on individual nodes is often insufficient. Instead, you can stream logs to a remote collector (like an OpenTelemetry collector or a central logging service) using the webhook backend. By defining an audit-webhook-config.yaml file, you instruct the API server to send events over HTTPS:

apiVersion: audit.k8s.io/v1
kind: Policy
# ... policy rules ...
# Inside kube-apiserver command flags:
--audit-webhook-config-file=/etc/kubernetes/webhook-config.yaml

This ensures that audit data is moved off the cluster immediately, protecting the logs even if a node is compromised. Whether you choose local files with log rotation or remote webhooks for centralized aggregation, the goal is to decouple the audit trail from the immediate lifecycle of the API server process.

Key Takeaway: Always pair your audit policy with a durable backend—using hostPath for local file persistence or a webhook configuration for remote streaming—to ensure your security audit trail survives pod restarts and infrastructure failures.