In this lesson, you will learn how to bypass the Linux kernel’s page cache to achieve deterministic performance for database workloads running inside containers.
The Problem with Double Buffering
When a database inside a container performs a read or write operation, the Linux kernel typically routes that data through the Host Page Cache. This acts as an intermediary buffer, storing frequently accessed disk blocks in RAM. While this is beneficial for general applications, it creates "double buffering" for databases. Databases like PostgreSQL or MySQL implement their own sophisticated caching and buffer management systems. Having both the containerized database and the host kernel manage the same data in memory causes redundant CPU cycles, increased memory pressure, and non-deterministic I/O latency.
Bypassing the Cache with O_DIRECT
To solve this, we use the O_DIRECT flag when opening files. When a containerized process accesses data with O_DIRECT, it instructs the kernel to bypass the page cache entirely. Instead, data is transferred directly between the storage device and the database's user-space memory buffers.
In a containerized environment, this ensures that the database has total control over its memory usage, preventing the host kernel from inadvertently evicting database pages or causing "jitter" during memory reclamation.
Implementation Concept
While the application code (the database) must explicitly request O_DIRECT, as an infrastructure engineer, you must ensure the underlying storage supports it. If you are using bind mounts to point a container to a host directory, ensure the filesystem (like XFS or ext4) is properly aligned.
// Simplified logic for a database opening a data file
int fd = open("/data/db_table", O_RDWR | O_DIRECT | O_SYNC);
// The kernel now bypasses the page cache for all reads/writes on 'fd'
When deploying databases in containers, O_DIRECT allows you to align the memory consumption of your container more accurately with the database's own configuration. By removing the kernel’s caching layer from the equation, you gain predictable I/O latency, which is critical for high-throughput transactional workloads.
Key Takeaway: By using O_DIRECT within containerized databases, you eliminate redundant page caching, reducing memory overhead and ensuring that I/O latency is managed exclusively by the database engine rather than the host kernel.