Learn Seccomp - Syscall Audit & Monitoring

#Introduction

In episode 14 you learned that an apparently strong filter can be weak — and that regular updates and reviews are the only cure. But there's an itching assumption running through that discussion: that you can see what happens to a filter. In reality, for most teams, seccomp is a black box. The filter is installed, denials happen, but nobody knows when, why, and which syscall is being attacked.

This is the episode that opens the box. Syscall audit & monitoring turns seccomp from a passive feature into an observable defense layer — like installing cameras in an already-locked building. Two immediate benefits: you can prove the filter works, and you can detect attack attempts before they succeed.

In this episode we'll cover three things: the Linux audit framework for recording syscalls, seccomp's logging mechanism via SECCOMP_RET_LOG and journald, and telemetry — denial rates, alerting, and workload syscall profiling with strace -c and bpftrace. Let's begin.

#Main Discussion

#Three Reasons to Observe Syscalls

Before discussing tools, let's set the observation goals. There are three different things to observe, and each answers a different question:

1. Verification — does the installed filter enforce what's written? Observing denials gives evidence that a specific syscall is truly blocked.
2. Detection — is anyone trying to call a blocked syscall? A denial burst from a process is the earliest alarm of an attack attempt.
3. Profiling — which syscalls does the workload actually call? This data is the main ingredient for designing the right filter (episode 17).

These three goals need different tools — the audit framework for the first two, strace/bpftrace for the third.

#The Linux Audit Framework

Linux has a built-in audit framework: the auditd daemon records system events, and auditctl configures its rules. Syscalls can be audited with syscall-name-based rules — for example, recording every open and execve:

sudo auditctl -a always,exit -F arch=b64 -S open,execve
sudo auditctl -l

Note the command parts: -a always,exit means always record when the syscall finishes. -F arch=b64 restricts to the 64-bit architecture so the rule isn't executed twice, and -S open,execve selects the syscalls to monitor. The auditctl -l output shows the list of active rules.

Read the results with ausearch to filter and aureport for summaries:

sudo ausearch -sc execve -ts today
sudo aureport --syscall --summary

ausearch -sc execve displays all execve records since today — useful for seeing which programs were run. aureport --syscall --summary counts each syscall's frequency, giving a one-screen view of the system's syscall profile.

#SECCOMP_RET_LOG: Recording Without Rejecting

Seccomp has a return action designed specifically for observation: SCMP_ACT_LOG. Unlike SCMP_ACT_ERRNO which rejects a syscall, LOG records the syscall to the kernel log while still allowing it to run. It's the perfect test mode: you can see what would be blocked before actually blocking it.

{
  "defaultAction": "SCMP_ACT_LOG",
  "architectures": ["SCMP_ARCH_X86_64"],
  "syscalls": [
    {
      "names": ["mount", "ptrace", "kexec_load"],
      "action": "SCMP_ACT_ERRNO",
      "errnoRet": 1
    }
  ]
}

The pattern above is interesting: the default is LOG — all syscalls are recorded; the truly dangerous syscalls stay ERRNO — blocked immediately. That way, when you move the filter to production, the whole deny-list has already been tested during the LOG phase.

The system also provides a global control: /proc/sys/kernel/seccomp/actions_logged determines which actions produce kernel log records. Adding errno to the list makes every rejection recorded:

echo "kill-process,kill-thread,errno" > /proc/sys/kernel/seccomp/actions_logged
journalctl -k | grep -i seccomp

After that, every deny appears in journald — journalctl -k | grep -i seccomp — complete with the context of the process that tried to call the forbidden syscall.

#Detecting Rejections: Reading the Attack Signs

Denials are the most interesting signal for a security team. When a filter works normally, denials are rare. When denials suddenly spike — or appear from a normally clean process — that's a sign: an application changing behavior, or an attacker probing the filter.

sudo ausearch -m SECCOMP --start today -i

Every denial produces a SECCOMP-type audit record carrying the process, syscall, and returned action. A pattern worth suspecting: repeated denials of the same syscall from the same process in a short time — exactly the "rattling doors looking for a gap" pattern.

#Telemetry: Denial Rates, Metrics, and Alerting

Per-incident observation isn't enough at production scale. You need telemetry: denial rate per process, denial count per syscall, and alerting when numbers spike past a baseline. A simple foundation can be built by parsing the audit log:

sudo ausearch -m SECCOMP --start today -i \
  | grep -o 'syscall=[a-z0-9_]*' \
  | sort | uniq -c | sort -rn

This pipeline turns thousands of log lines into one line per syscall with its denial count — data you can export as metrics (for example via a node exporter textfile) and alert on when it rises sharply. The alerting principle is the same as any metric: set a baseline first when the system is healthy, then alert when the denial rate deviates far from baseline — not when the denial rate is zero, because an alarm that never sounds is a dead alarm.

#Profiling Workloads: strace -c and bpftrace

The third observation goal is understanding a workload's syscall profile — the raw material for filter design in episode 17. Two main tools:

strace -c counts the number and time per syscall while a workload runs. It uses ptrace, so it suits debugging sessions and staging:

strace -c -f /usr/sbin/nginx -g 'daemon off;'

Its output is a syscall table with calls (call counts), errors, and time columns — a direct view of which syscalls dominate. This data is what will become the allow-list.

bpftrace produces the same results without modifying the application at all — it works at the kernel tracepoint level, so it's safe for production use:

bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @[comm, syscall] = count(); }'

The command bpftrace -e 'tracepoint:raw_syscalls:sys_enter { @[comm, syscall] = count(); }' collects syscall counts per process name. Pressing Ctrl-C prints a histogram showing which syscalls the workload calls in the real environment.

#Common Mistakes

1. Audit rules that are too broad. Auditing all syscalls without filtering balloons the logs and slows the system. Scope the rules to the syscalls you want to observe.

2. Blocking without monitoring. A filter without observation is blind trust — the first denial you see could be after an attack has succeeded. Install monitoring together with the filter.

3. Enabling LOG everywhere. SCMP_ACT_LOG on a workload calling millions of syscalls per second floods the logs. Limit LOG to syscalls genuinely under suspicion, and turn it off once the testing phase is done.

4. Alerting without a baseline. Alerts triggered by normal denial rates numb the team. Build a baseline in a healthy state, then alert on deviations.

#Conclusion

In episode 15 you turned seccomp from a black box into an observable layer. The audit framework records syscalls via auditctl and reads them with ausearch; SECCOMP_RET_LOG enables observation without blocking, and /proc/sys/kernel/seccomp/actions_logged makes denials land in journald; denial-rate telemetry gives you an alarm signal; and strace -c plus bpftrace open up a workload's syscall profile.

Key points to take with you:

• Seccomp observation serves three purposes: verification, detection, and profiling.
• SECCOMP_RET_LOG is the safe bridge between testing and production.
• A denial burst is an attack-attempt alarm — but you need a baseline to read it.
• Syscall profiling is the raw material for good filter design.

So far we've always said a filter is "installed" without ever discussing how exactly a filter reaches the kernel. In the next episode 16, we'll dive into the kernel interface & prctl — the difference between prctl and the seccomp syscall, the no_new_privs prerequisite, the additive filter nature inherited through fork and exec, and its interaction with user namespaces. See you then!