Tracing syscalls with eBPF without losing your mind

I have been writing eBPF programs on and off for about three years, and the single most underrated thing about the toolchain is how much time you spend negotiating with the verifier. The verifier is not your enemy. It is, however, a deeply suspicious colleague who needs every loop bound documented in triplicate.

What we’re building

A small program that traces every openat syscall on the host, captures the filename, PID, and UID, and emits records over a ring buffer to userspace. About a hundred lines of C plus a Go loader.

If you only need this for an afternoon, bpftrace will get you there in three lines. If you need it in production for more than a week, you want a real program.

The probe

#include <vmlinux.h>
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>

struct event {
    u32 pid;
    u32 uid;
    char comm[16];
    char filename[256];
};

struct {
    __uint(type, BPF_MAP_TYPE_RINGBUF);
    __uint(max_entries, 1 << 24);
} events SEC(".maps");

SEC("tracepoint/syscalls/sys_enter_openat")
int trace_openat(struct trace_event_raw_sys_enter *ctx) {
    struct event *e = bpf_ringbuf_reserve(&events, sizeof(*e), 0);
    if (!e) return 0;

    u64 id = bpf_get_current_pid_tgid();
    e->pid = id >> 32;
    e->uid = bpf_get_current_uid_gid() & 0xffffffff;
    bpf_get_current_comm(&e->comm, sizeof(e->comm));

    const char *fname = (const char *)ctx->args[1];
    bpf_probe_read_user_str(&e->filename, sizeof(e->filename), fname);

    bpf_ringbuf_submit(e, 0);
    return 0;
}

char LICENSE[] SEC("license") = "GPL";

Two things to flag. First, bpf_probe_read_user_str not bpf_probe_read_user — the former null-terminates and returns a length, which the verifier will quietly thank you for. Second, that __uint(max_entries, 1 << 24) is the ring buffer size in bytes. 16 MiB is generous. Tune it down if you are running on small instances.

Catches I hit

• On 6.1 and below the tracepoint argument layout for openat is subtly different. Use BPF_KPROBE if you need to support older kernels.
• Ring buffer reservations that fail silently are usually a sign your map is too small for the burst rate.
• If you fork a process that opens 50,000 files at startup (looking at you, node_modules), you will get back-pressure. Either filter early in the probe or accept the drops.

That’s the loop. Probe, ring buffer, loader. Everything else is filtering, aggregation, and convincing your security team that yes, this is fine.