In this lesson, you will learn how to identify when system memory pressure forces the kernel into a cycle of frequent page evictions and re-reads, triggering performance-killing I/O storms.
The Mechanics of Thrashing
The Linux kernel uses the Page Cache to speed up disk access by keeping frequently used file data in RAM. Under ideal conditions, the "working set" of your application fits comfortably in memory. However, when system memory is overcommitted or constrained by memory limits (like cgroups), the kernel’s page reclamation algorithm (the LRU list) begins aggressively evicting pages to make room.
If your application’s working set exceeds available RAM, the kernel evicts a page, only for the application to immediately request it again milliseconds later. This forces the system to perform constant synchronous disk reads, leading to Page Cache Thrashing. Unlike simple disk I/O, thrashing is characterized by high pgpgin and pgpgout activity occurring simultaneously, often accompanied by high iowait CPU states.
Detecting the Storm with BPF
While vmstat shows broad page-in/out metrics, it cannot tell you which processes are causing the thrash. We can use BPF to hook into the kernel’s page reclaim path to measure how often the system is evicting pages that are immediately required again.
Specifically, we trace mark_page_accessed and the eviction path in __remove_mapping. By tracking timestamps of when a page is evicted versus when the same file offset is accessed, we can calculate the "Time to Re-access". If this delta is consistently low (e.g., under a few seconds), your system is actively thrashing.
// Simplified logic for a BPF probe on page eviction
SEC("kprobe/evict_page")
int BPF_KPROBE(evict_page, struct page *page) {
// Record the timestamp and inode/offset of the evicted page
u64 ts = bpf_ktime_get_ns();
update_eviction_map(page, ts);
return 0;
}
When you see a high frequency of "re-accesses" occurring shortly after an "eviction" event for the same file inodes, you have confirmed a thrashing scenario. The fix is rarely a kernel tuning parameter; instead, it usually requires reducing the resident memory footprint of the offending process or increasing physical memory capacity.
Key Takeaway: Page Cache Thrashing occurs when the working set exceeds available RAM, causing a performance-degrading loop of page eviction and immediate disk re-fetch that can be diagnosed by tracing the temporal gap between page evictions and re-accesses.