Learn Seccomp - Seccomp User Notification (NOTIFY)

#Introduction

In episode 5, you learned about argument filters and architecture handling — making decisions more precisely inside BPF: checking the syscall number along with its arguments, and handling architecture differences. But all those decisions share one property: they're locked in when the filter is loaded. Once the filter is in the kernel, its behavior is static — it can't ask anyone, it can't adapt to runtime conditions.

Episode 6 introduces the big exception: Seccomp User Notification (NOTIFY). With this mechanism, a syscall is no longer decided by a frozen rule — it can be suspended and asked to a living supervisor process. This is what opens the door to userspace sandboxing with supervisor-in-the-loop intervention.

Imagine the difference like this. A nightclub with a static guest list: the bouncer decides on his own, can't ask anyone. That's ordinary seccomp. Seccomp NOTIFY is a club whose bouncer has a radio: a guest not on the list isn't immediately rejected or admitted — the bouncer contacts the control room, and the decision comes from there. That control room is the supervisor.

#Decisions Deferred to Runtime

The SECCOMP_RET_USER_NOTIF return action fundamentally changes a syscall's journey. When a filter produces this action, the kernel does three things: suspends the thread that made the syscall, creates a notification event, and wakes up the supervisor via a listener fd.

The supervisor then reads the event, inspects the syscall context, and sends a reply. There are two reply forms in struct seccomp_notif_resp:

• error != 0 — the syscall is not executed at all; the target immediately receives the errno in the error value.
• error == 0 — the syscall runs normally, and the val value becomes its return value.

Note the key point: if the supervisor decides to allow, that's not an "unconditional go-ahead" — the kernel still executes the syscall. The supervisor is only the judge, not the executor. And in the action priority chain, USER_NOTIF sits above ALLOW (below TRAP), so a syscall matching a notify rule will always be asked first.

#Dissecting the Data the Kernel Sends

When the supervisor receives a notification via SECCOMP_IOCTL_NOTIF_RECV, the kernel fills in a struct seccomp_notif. The two parts most often read:

struct seccomp_notif (key parts)

struct seccomp_notif {
    __u64 id;                 /* unique id of this notification */
    __u32 pid;                /* target pid (in the root userns) */
    __u32 flags;              /* unused, read as 0 */
    struct seccomp_data data; /* nr + args + arch of the syscall */
};

The data part is exactly the struct seccomp_data snapshot you already know from episodes 2 and 5: arch, syscall_nr, args[6], and instruction_pointer. The supervisor can read the syscall arguments without guessing — the kernel copies them before the thread is suspended.

For the reply, the structure is simpler:

struct seccomp_notif_resp

struct seccomp_notif_resp {
    __u64 id;   /* must match the request's id */
    __s64 val;  /* return value if error == 0 */
    __s32 error;/* 0 = execute the syscall, otherwise an errno */
    __u32 flags;/* for SECCOMP_USER_NOTIF_FLAG_* */
};

The rules of the game: id must be copied from the request, and only one of error/val is meaningful — they can't be used together.

#How It Works End-to-End

In short, the flow looks like this:

1. The target installs a filter with the SECCOMP_FILTER_FLAG_NEW_LISTENER flag via the seccomp(2) syscall (kernel 5.0 and up). The kernel creates a listener fd and returns it.
2. The fd is passed to the supervisor — via fork, a unix socket, or another IPC mechanism. Target and supervisor are two different processes; one watches, one is watched.
3. The target invokes a matching syscall — the thread suspends, the kernel sends a notification.
4. The supervisor receives the event with SECCOMP_IOCTL_NOTIF_RECV on the listener fd.
5. The supervisor validates the id with SECCOMP_IOCTL_NOTIF_ID_VALID to make sure the notification is still valid — this guards against TOCTOU races.
6. The supervisor sends its decision with SECCOMP_IOCTL_NOTIF_SEND.
7. The target resumes — the syscall is executed or fails according to the reply.

There's one bonus ioctl: SECCOMP_IOCTL_NOTIF_ADDFD (kernel 5.9) lets the supervisor inject an fd into the target — useful for patterns where the target needs a file descriptor only the supervisor holds, like opening a secret file for a specific target only.

#The Supervisor: Reference Code

Let's see the real thing. First, the target side — a filter that routes read to user notification while also creating the listener fd:

#include <linux/filter.h>
#include <linux/seccomp.h>
#include <stddef.h>
#include <sys/syscall.h>
#include <unistd.h>
 
static int install_filter(void) {
    struct sock_filter insn[] = {
        BPF_STMT(BPF_LD | BPF_W | BPF_ABS,
                 offsetof(struct seccomp_data, nr)),
        BPF_JUMP(BPF_JMP | BPF_JEQ | BPF_K, __NR_read, 0, 1),
        BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_USER_NOTIF),
        BPF_STMT(BPF_RET | BPF_K, SECCOMP_RET_ALLOW),
    };
    struct sock_fprog prog = {
        .len = sizeof(insn) / sizeof(insn[0]),
        .filter = insn,
    };
 
    return syscall(SYS_seccomp, SECCOMP_SET_MODE_FILTER,
                   SECCOMP_FILTER_FLAG_NEW_LISTENER, &prog);
}

The filter above reads the syscall number; if it's read, the result is SECCOMP_RET_USER_NOTIF — everything else is ALLOW. The return value of this function is the listener fd, which is then passed to the supervisor.

Now the supervisor side — the main loop that waits, evaluates, and decides:

#include <errno.h>
#include <linux/seccomp.h>
#include <stdio.h>
#include <sys/ioctl.h>
#include <sys/syscall.h>
 
static void supervise(int listener) {
    for (;;) {
        struct seccomp_notif req = { 0 };
        struct seccomp_notif_resp resp = { 0 };
 
        if (ioctl(listener, SECCOMP_IOCTL_NOTIF_RECV, &req) < 0) {
            perror("RECV");
            return;
        }
 
        resp.id = req.id;
        if (req.data.nr == __NR_read && req.data.args[0] == 0) {
            resp.error = -EACCES;   /* reading from stdin is blocked */
        } else {
            resp.error = 0;         /* allow, the syscall is executed */
            resp.val = 0;
        }
        ioctl(listener, SECCOMP_IOCTL_NOTIF_SEND, &resp);
    }
}

Read these lines slowly: RECV blocks until a notification arrives, the decision is made from the contents of req.data, and SEND releases the target thread. Mechanically simple, but this is where all the runtime policy lives — the supervisor can read config, consult policy, even ask for human approval.

#Handling Edge Cases

This mechanism has three behaviors you must understand so you don't write a fragile supervisor:

• TOCTOU and SECCOMP_IOCTL_NOTIF_ID_VALID. Between receiving a notification and sending a reply, the target could be killed or change identity. Before taking external action (like opening an fd via ADDFD), validate the id first. Don't send SEND for an id that has already expired.
• SECCOMP_USER_NOTIF_FLAG_CONTINUE (kernel 5.5). If the supervisor decides the target can proceed without any modification, it can send a reply with the CONTINUE flag instead of error == 0. The difference is subtle but important: with CONTINUE, the kernel runs the syscall exactly as if no filter existed — without overriding the return value via val. Use this when your reply changes nothing.
• Multiple stacked filters. If the target installs several filters, only the filter returning USER_NOTIF is active for that syscall. USER_NOTIF also requires no_new_privs or privileges — make sure the target qualifies before calling seccomp(2).

#When to Use NOTIFY

This mechanism is expensive, so use it with a clear purpose:

• Compatibility and shims — syscalls whose results need to be faked or adjusted per-context, for example hiding a certain syscall from the target.
• Context-based intervention — decisions that depend on runtime state: who the target is, what file, or argument combinations too complex to express in pure BPF.
• Sandboxing with a supervisor — the target runs under a strict filter, and the supervisor decides which syscalls are genuinely needed. This supervisor-binary interaction pattern is the foundation of several real-world sandboxes.
• Observation — observing the syscalls a target makes by mediating rather than merely blocking.

Don't use NOTIFY for syscalls called thousands of times per second — the cost is a two-way context switch per syscall. For pure denial, ERRNO remains the champion. Full details are in man 2 seccomp, the SECCOMP_USER_NOTIF section.

#Conclusion

In episode 6 you understood that seccomp doesn't have to be static: SECCOMP_RET_USER_NOTIF moves the decision to a supervisor process via a listener fd and three core ioctls — RECV, ID_VALID, and SEND. You also saw the concrete shape of target and supervisor in C, understood the roles of error versus val in replies, and know when this mechanism is worth using.

The keys to take home:

• NOTIFY suspends a syscall; the supervisor's speed is its cost.
• error != 0 cancels a syscall; error == 0 executes it with val.
• The supervisor is a single point of control — keep it alive.

The more complex the filters you build, the greater the need to see what's actually happening rather than guess. In episode 7, we move into Debugging & Testing Filters: disassembling BPF with seccomp-tools, observing syscalls with strace, using SECCOMP_RET_LOG for auditing, and building a test suite for deny behavior with a fallback strategy when a filter turns out to be wrong.