In this lesson, you will learn how to leverage the perf tool to identify CPU-intensive code paths in your embedded Linux system by accessing hardware performance monitoring units (PMUs).
When your embedded application or kernel module misses a real-time deadline, the bottleneck is often buried deep in the instruction pipeline. While strace or ltrace help identify system call or library overhead, perf operates at a lower level by sampling the CPU’s hardware counters. These counters track low-level architectural events, such as cache misses, branch mispredictions, or clock cycles, directly from the processor's silicon.
Capturing Architectural Events
To see which functions are consuming the most CPU cycles, you can use the perf record command. For instance, to sample your application's performance for 10 seconds:
perf record -g -- ./my_embedded_app
The -g flag captures the call graph, allowing you to see not just which function is slow, but the entire hierarchy of calls leading to that bottleneck. Once the data is collected, you can inspect the results with perf report. This opens an interactive interface that maps CPU samples to specific lines of code, pinpointing exactly where your logic triggers high-latency events.
Analyzing Hardware Counters
In embedded systems, memory latency is a common performance killer. You can specifically monitor cache performance to diagnose inefficient data structure traversal:
perf stat -e cache-references,cache-misses ./my_embedded_app
This command provides a summary of total cache hits versus misses. If your cache-misses count is disproportionately high, it suggests your application is suffering from poor cache locality, likely due to large, non-contiguous data buffers or frequent context switching.
By correlating these hardware events with your kernel modules or user-space logic, you can move beyond guessing and optimize your code based on actual hardware execution patterns. Remember that because perf interacts with the CPU's PMU, ensure your kernel was compiled with CONFIG_PERF_EVENTS enabled during your kernel build process.
Key Takeaway: Use perf to bridge the gap between high-level application logic and low-level hardware behavior by sampling PMU events to pinpoint cycle-intensive code paths.