Learn AppArmor - Debugging Denials & Logs

#Introduction

In episode 6, you tightened profiles with network and capability rules — and you most likely also harvested denials in the logs right away. That's not failure; that's a signal at work. The problem is that many engineers respond to a denial by removing rules until the error goes away, without ever knowing what was actually blocked.

Episode 7 changes how you face denials: not as an enemy to silence, but as information to read. AppArmor records nearly every refusal with detail, and reading those records correctly is the skill that separates operators who guess from operators who understand.

#Where Denial Traces Are Written

By default, AppArmor denials are written to the kernel log, which then flows into the system logs. On Ubuntu and Debian with rsyslog, the traces live in a single file:

grep apparmor /var/log/syslog

On modern systems with journald, the faster way is through journalctl — especially for kernel messages:

journalctl -k --grep=apparmor

sudo journalctl -k --grep=apparmor -n 50

sudo journalctl -k --grep=apparmor -f

The -n 50 flag limits to the last 50 lines, while -f follows the log like tail -f — very useful while you reproduce a problem.

#Reading the apparmor=DENIED Format

Here's a typical denial:

audit: type=1400 audit(1710000000.123:456): apparmor="DENIED"
  operation="open" profile="/usr/sbin/nginx"
  name="/etc/nginx/secret.conf" pid=1234 comm="nginx"
  requested_mask="r" denied_mask="r" fsuid=0

Five elements you must read:

• apparmor="DENIED" — the marker that this is an AppArmor refusal, not an application error.
• operation — the blocked operation: open, exec, connect, mkdir, and so on.
• profile — the profile doing the refusing; here, the Nginx profile.
• name — the target object, usually a file path or socket address.
• requested_mask and denied_mask — the rights requested and the rights denied. r means read, w write, x execute, m memory map, k lock, a append.

A network connect denial looks a bit different:

audit: type=1400 audit(1710000000.789:457): apparmor="DENIED"
  operation="connect" profile="/usr/sbin/nginx"
  name="/run/mysqld/mysqld.sock" pid=1234 comm="nginx"
  requested_mask="rw" denied_mask="rw" fsuid=0

Notice name here is a Unix socket path, not an IP address. This is a direct consequence of episode 6: the Nginx profile has no network unix stream rule, so the connection to MySQL over the socket is denied.

#The Right Debug Loop

Debugging an AppArmor denial follows a four-step cycle. Skip a single step and you'll be fixing the wrong thing:

1. Observe — find the denial in the logs and read its five elements. Should this operation actually be allowed?
2. Adjust — edit the profile. Add the narrowest possible rule, or fix the wrong path. Use aa-logprof if you want step-by-step guidance.
3. Reload — reload the profile into the kernel without rebooting the system:

sudo apparmor_parser -r /etc/apparmor.d/usr.sbin.nginx

The -r (replace) flag updates an already-loaded profile. If you created a brand-new profile file that was never loaded, use -a (add), or let aa-enforce/aa-complain handle it.

4. Verify — repeat the operation that failed, then check the logs again. Denial gone? Done. Still showing with a different rule? Back to step 1.

#aa-notify: Denials Onto Your Screen

Running the manual loop is fine for debugging, but for continuous monitoring there's a more convenient tool: aa-notify. On Ubuntu it's available through the apparmor-notify package. For a summary of the last few days of denials:

aa-notify -s 1

-s takes the number of days — replace 1 with 7 for a week. The output looks roughly like this (simplified):

Profiles: 3    Denials: 12    Since: 1 day ago
  /usr/sbin/nginx          8
  /usr/sbin/mysqld         3
  /usr/bin/python3.12      1

For real-time desktop notifications, use poll mode:

aa-notify -p -u arman --display $DISPLAY

This mode keeps reading the logs and pops up a notification every time there's a new denial — so changes in application behavior are visible without opening a terminal. On servers without a GUI, the main scenario is scheduling aa-notify -s 1 via cron and checking the summary periodically. Global configuration (including who may run it and notification filtering) lives in /etc/apparmor/notify.conf.

#Conclusion

In episode 7 you now hold the key to AppArmor debugging: knowing where denials are written (syslog, journald, and audit.log), being able to read the apparmor="DENIED" format down to the operation and mask level, running the observe-adjust-reload-verify cycle with apparmor_parser -r, and using aa-notify for automated notifications.

Keys to take home:

• Logs are the source of truth; read before you change a rule.
• Reload with apparmor_parser -r, don't reboot.
• A denial that doesn't change after a reload means the rule still isn't matching.

Up to this point, all your profiles are written with hardcoded paths — and hardcoded paths make profiles non-portable across systems. In episode 8, we'll meet Variables, Tunables & Includes: @{HOME}, @{PROC}, the tunables directory, and abstractions that let a single profile run on many machines without being reworked.