eBPF-based Network Telemetry

In this lesson, you will learn how to bridge the gap between kernel-level packet processing and external observability platforms to visualize real-time network flow statistics.

From Kernel Maps to Observability Dashboards

Until now, we have used BPF_MAP_TYPE_HASH or BPF_MAP_TYPE_ARRAY to store network data like packet counts or latency metrics within the kernel. While these maps provide efficient storage, they are "silos" trapped inside the kernel's memory. To build robust network telemetry, we need to export this data to external platforms like Prometheus, Grafana, or specialized time-series databases.

The standard pattern for telemetry involves a two-part architecture:

1. The eBPF Program: Operates at a hook point (like XDP or TC), aggregates per-flow data into a BPF_MAP_TYPE_PERCPU_HASH to minimize lock contention, and periodically updates metrics like bytes_sent and packets_received.
2. The User-Space Agent: A long-running process that periodically performs a bpf_map_lookup_elem to pull the aggregated statistics from the kernel map. Once retrieved, the agent translates this binary data into a format (like JSON or OpenTelemetry protocol) and pushes it to an external backend.

Streamline Your Data

Because the kernel can process millions of packets per second, you must avoid saturating the communication channel between user-space and kernel-space. Instead of reading the map for every single packet, your user-space agent should perform "sampling" or "batch reads."

For example, your BPF code might look like this:

struct flow_stats {
    __u64 rx_packets;
    __u64 rx_bytes;
};

// Map storing flow telemetry
struct {
    __uint(type, BPF_MAP_TYPE_HASH);
    __type(key, struct flow_key);
    __type(value, struct flow_stats);
    __uint(max_entries, 10240);
} flow_map SEC(".maps");

Your user-space agent iterates over the keys in flow_map every few seconds, resets the counters if necessary, and pushes the snapshot to your observability dashboard. By decoupling the collection (in-kernel) from the reporting (user-space), you ensure that your monitoring overhead does not degrade the performance of the network path you are observing.

Key Takeaway: Network telemetry is achieved by using eBPF to aggregate granular packet data into kernel maps, which a user-space agent then polls and exports to external observability systems.