Learn Seccomp - Advanced Filter Design

#Introduction

In episode 16 you understood the kernel interface: filters are additive, inherited by child processes, and cannot be removed. Those constraints deliver one clear message: a filter must be correct from the moment it's installed, because fixing it means replacing the process that uses it. Filter design, in other words, isn't the work of typing rules — it's an engineering discipline.

This is the peak episode of the entire technical discussion of this series: advanced filter design. You already know the gate (seccomp), the material (BPF), the constraints (kernel), and how to observe it (audit). Now it's time to assemble everything into a methodology: how to write a filter that's precise without being oversized, how to measure the syscalls a workload truly needs, and how to compose filters into layers that reinforce each other.

The mindset we'll carry throughout this episode: a good filter isn't the one with the most rules, but the one that best fits the workload — as small as possible, as precise as possible. Let's begin.

#Main Discussion

#BPF Is Evaluated Linearly: Size Is Cost

The first foundation of filter design: a BPF program is executed linearly every time a syscall is called. The kernel starts from the first instruction, walks through each rule one by one, until it reaches a return action. Every rule you add means one extra step on the kernel's hot path — for every syscall, every process, every time.

Imagine a queue at airport security. One officer with a short list processes passengers in seconds; twenty officers with a thousand-name list make the line snake around. This is why episode 18 will take the cost of seccomp seriously — but from now on you must hold the principle: a lean filter is faster, easier to review, and more secure (recalling the oversized-filter CVE in episode 14).

#Precise Argument Rules with SCMP_CMP64

Not every syscall needs to be blocked or allowed blindly. For syscalls that are only dangerous under certain conditions, you can add argument checks. libseccomp provides the SCMP_CMP family for this — and SCMP_CMP64 to compare full 64-bit arguments.

Classic example: kill is a legitimate syscall, but an application doesn't need to kill other processes at will. Restrict kill so it's only allowed to target its own PID:

Precise argument rule: kill only for your own PID

#include <seccomp.h>
#include <unistd.h>
#include <stdint.h>
 
scmp_filter_ctx ctx;
 
ctx = seccomp_init(SCMP_ACT_ERRNO);
seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(read), 0);
seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(write), 0);
seccomp_rule_add(ctx, SCMP_ACT_ALLOW, SCMP_SYS(kill),
                 1,
                 SCMP_CMP64(0, SCMP_CMP_EQ, (int64_t) getpid()));
seccomp_load(ctx);

Let's dissect the line SCMP_CMP64(0, SCMP_CMP_EQ, (int64_t) getpid()): the number 0 refers to the first argument of kill (that is, the target PID), SCMP_CMP_EQ means an equality comparison, and the value compared is the process's own PID. The result: kill to another process is rejected, kill to itself is allowed.

#Avoiding Oversized Filters

The kernel limits the size of BPF programs — and the experience in episode 14 showed that giant filters also trigger bugs in libseccomp. A few guidelines to keep filters small:

• Prefer allow-lists. A default-deny allow-list with a few dozen syscalls is always shorter than a deny-list that tries to anticipate hundreds of syscalls.
• Avoid excessive argument rules. Every SCMP_CMP adds instructions and merge complexity. Add them only when the value is real.
• Split the filter into layers. The additive nature in episode 16 enables composition: instead of one giant filter, use several small filters that tighten each other.
• Remove what isn't used. Periodic review (episode 14) discards rules that are no longer relevant.

#Filter Composition: Base and Per-Service Layers

The additive nature of filters opens an elegant composition pattern. Instead of one profile trying to do everything, split it into layers, each with its own responsibility:

Base layer — applies to all workloads: block universally dangerous syscalls (mount, ptrace, kexec_load, and friends from episode 13).

Per-service layer — specific to one application: the allow-list of syscalls the workload actually uses.

{
  "defaultAction": "SCMP_ACT_ALLOW",
  "architectures": ["SCMP_ARCH_X86_64"],
  "syscalls": [
    {
      "names": ["mount", "ptrace", "kexec_load", "userfaultfd"],
      "action": "SCMP_ACT_ERRNO",
      "errnoRet": 1
    }
  ]
}

{
  "defaultAction": "SCMP_ACT_ERRNO",
  "architectures": ["SCMP_ARCH_X86_64"],
  "syscalls": [
    {
      "names": [
        "read", "write", "openat", "close", "mmap",
        "mprotect", "futex", "epoll_wait", "exit_group"
      ],
      "action": "SCMP_ACT_ALLOW"
    }
  ]
}

At the process level (C applications, systemd), both layers are installed as two separate filters — the deny-list at the base, the allow-list on top. Because filters are additive, the final result is the intersection of both: syscalls that are in the allow-list and not in the deny-list. At the container level, Docker only supports one profile per container — there you simply merge both lists into a single JSON.

#Branching Between Syscalls and Return Action Strategy

A filter doesn't have to behave the same way toward all syscalls — it's a branching decision tree. libseccomp arranges your rules into branches: the kernel tests the architecture, checks the syscall number, then (if needed) the arguments, and heads to the appropriate return action. This branching structure is generated automatically from the order and type of your rules, but you can steer the outcome with three design decisions:

• The default action determines the main branch. Choose SCMP_ACT_ALLOW for a deny-list (nearly all syscalls fall into the allow branch) or SCMP_ACT_ERRNO for an allow-list (every other syscall falls into the deny branch).
• Specific rules open new branches. Each more specific rule — a particular syscall, or a syscall plus arguments — adds a branch in front of the default. The more specific the rule, the longer the path the kernel must trace.
• Every syscall can be given a different action. In a single filter, read can be ALLOW, mount can be ERRNO, and io_uring_setup can be NOTIFY — each with its own branch.

The third point deserves emphasis. NOTIFY doesn't have to be used globally. You can let the deny-list run as usual and only set aside one or two syscalls that genuinely need dynamic decisions into NOTIFY mode — for example io_uring_setup or bpf — while other syscalls remain evaluated in-kernel. The result: supervisor precision only where it's truly needed, and hot-path syscalls aren't dragged into the NOTIFY cost you learned about in episode 6. This composition principle will be revisited from the cost side in episode 18.

#Design Pattern: Minimal Syscall Set per Workload

Now the most practical and most often neglected pattern: measure first, then write the filter. Don't guess which syscalls the application needs — measure them. The steps:

1. Profile the workload in staging. Run the workload under realistic conditions (test traffic, production scenarios) while recording syscalls with strace -c:

strace -c -f -p 1234

2. Read the result as the list of syscalls actually used.

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 44.17    0.021309        2131        10           epoll_wait
 28.11    0.013556        1356        10           write
 14.90    0.007187         719        10           read
  6.62    0.003194         319        10           futex
------ ----------- ----------- --------- --------- ----------------
100.00    0.048233                40           total

3. Build the allow-list from that list. Only the syscalls that appeared — plus a small allowance for initialization and error-handling paths that weren't recorded.

4. Test with LOG mode. Run in staging with SCMP_ACT_LOG (episode 15), check for syscalls accidentally blocked, and fix them before switching to SCMP_ACT_ERRNO.

5. Repeat whenever the application changes. Any release that changes I/O or networking behavior can change the syscall profile — measure again.

#Safe Rule Granularity

When to block a syscall, when to restrict arguments, and when to let it through? The simple guideline:

• Block outright syscalls that are never used and intrinsically dangerous — mount, ptrace, kexec_load. This is a cheap and clear decision.
• Restrict arguments only for syscalls that are legitimate but could be abused under certain conditions — the kill example above. Use SCMP_CMP sparingly.
• Allow fully syscalls needed on the hot path that pose no meaningful risk — read, write, futex. Adding argument checks on the hot path pays an unnecessary cost.
• Treat the deny-list as an additional layer, not the primary policy. The primary policy is the measured allow-list.

#Common Mistakes

1. Guessing without measuring. Writing a filter from intuition produces two mistakes at once: an important syscall is blocked and the application breaks, or a dangerous syscall slips through because you didn't think of it. Always measure.

2. Deny-list as the only policy. A deny-list is never complete — it only blocks what you know. Add an allow-list as the primary layer.

3. Over-engineering argument rules. Every SCMP_CMP adds complexity, overhead, and the risk of merge bugs. If an argument restriction doesn't prevent a realistic scenario, don't install it.

4. Giant filters. Violating BPF size limits, triggering library CVEs, and being hard to review. Split them into layers, prioritize the allow-list.

5. Forgetting to re-measure after application updates. The syscall profile changes across application versions. A filter designed for v1 can block v2's needs.

#Conclusion

In this episode 17 you assembled the entire series material into a filter design methodology: understanding the cost of linear BPF evaluation, composing precise argument rules with SCMP_CMP64, avoiding oversized filters, composing layered filters (base and per-service), applying the minimal syscall set pattern measured with strace and bpftrace, and arranging branches that combine different return actions — including inserting NOTIFY only where it's needed.

Key points to take with you:

• Filters are evaluated linearly — every rule is a cost; keep filters lean.
• Argument rules (SCMP_CMP64) provide precision, but use them sparingly and keep testing.
• Layered filter composition exploits seccomp's additive nature.
• A single filter can combine different return actions — ALLOW, ERRNO, NOTIFY — through per-syscall branches.
• Measure the syscall profile before writing the allow-list, and re-measure when the application changes.
• Deny-list for what's clearly dangerous, allow-list for what's actually used, arguments only at sensitive points.

Good design produces a filter that's correct from the start — and since filters can't be removed, that's a necessity, not a choice. The question that remains now is about the price: how much do all this precision actually cost? In the next episode 18 we'll discuss performance & overhead — measuring filter cost per syscall, kernel optimizations like BPF JIT and filter caching, and balancing security and performance at the scale of thousands of containers. See you then!