Learn Firecracker - Jailer & Security Isolation

#Introduction

So far we've been running Firecracker as an ordinary process — as root, with no restrictions. In the real world, that's never enough. Episode 7 introduces the jailer: a security wrapper that cages the Firecracker process so that if the microVM is breached, an attacker only finds empty walls.

Why is this episode important? Because all of Firecracker's security claims — multi-tenant isolation, untrusted workloads — stand on the jailer. KVM protects the host from the guest, but who protects the host from the VMM itself? The answer: the jailer, using deep Linux isolation mechanisms — namespaces, cgroups, seccomp, a read-only rootfs, and privilege drop.

#The Philosophy: Defense-in-Depth

Firecracker's threat model is layered:

1. KVM — separates the guest from host processes at the hardware level.
2. Minimal Firecracker process — little code, few bugs.
3. Jailer — if layer 2 fails (a bug in the VMM), an attacker who manages to execute code inside the Firecracker process is still caged: no filesystem writes, no dangerous syscalls, no privileges.

The jailer is a response that assumes the VMM can be breached. With that assumption, it ensures that penetrating the VMM doesn't mean penetrating the host. This is the same pattern as container sandboxes: don't trust that bugs won't happen — design so bugs aren't dangerous.

#What the Jailer Does

When you run jailer --exec-file ./firecracker, the jailer does several things in sequence before executing Firecracker:

1. Network namespace — separates the VMM's networking from the host's.
2. cgroup — limits the CPU, memory, and devices the process can access.
3. Privilege drop — switches to a non-root UID/GID.
4. Chroot — moves the process into its own isolated root filesystem.
5. Seccomp filter — restricts allowed syscalls to a narrow whitelist.
6. Read-only rootfs — prevents filesystem modification after start.

The final result: the Firecracker process lives in a small world it can't leave — exactly the jailer's purpose.

#Using the Jailer

The basic pattern for running a microVM through the jailer:

sudo jailer \
  --id 1001 \
  --exec-file /usr/local/bin/firecracker \
  --uid 123 \
  --gid 100 \
  --chroot-base-dir /srv/jailer \
  --netns /var/run/netns/fcns \
  --node 0

Understanding the important flags:

• --id <uid> — the microVM's unique identity. The jailer uses it to create the chroot-base-dir/<id> directory as the process's home. Every VM must have a different --id.
• --chroot-base-dir — the directory where the jailer builds the new root (/srv/jailer/<id>/). Kernel files, rootfs, and the API socket must be copied into it so the process can see them.
• --exec-file — the binary to execute after jailing (Firecracker).
• --uid / --gid — the non-root UID/GID that runs the process. Firecracker doesn't need root; it only needs access to /dev/kvm and its own files.
• --netns — moves the process into a specific network namespace, providing per-VM network isolation.
• --node — the NUMA node to pin vCPUs to.

Once the jailer runs, the API socket lives at /srv/jailer/<id>/run/firecracker.socket — not in /tmp anymore. All subsequent interaction goes through the socket inside the chroot:

curl --unix-socket /srv/jailer/1001/run/firecracker.socket http://localhost/

#Preparing the Chroot for the Jailer

The jailer changes the process's filesystem perspective. Everything Firecracker needs must be inside the chroot:

sudo mkdir -p /srv/jailer/1001
sudo cp /home/user/fc-demo/vmlinux.bin /srv/jailer/1001/vmlinux.bin
sudo cp /home/user/fc-demo/rootfs.ext4 /srv/jailer/1001/rootfs.ext4

Then all paths in the API payload use chroot-relative paths:

{
  "kernel_image_path": "/vmlinux.bin",
  "boot_args": "console=ttyS0 reboot=k panic=1 pci=off"
}

The pattern: the host cps files into the chroot, and the API references paths inside it. This is also a security benefit — processes inside the chroot can't see files outside, so other VMs' kernels and rootfs are unreachable.

#Seccomp: The Syscall Whitelist

Firecracker's seccomp filter builds a whitelist of syscalls the process is allowed to make. Dangerous syscalls like mount, ptrace, kexec, or userfaultfd are rejected with an error — even if an attacker manages to execute arbitrary code inside the process.

The jailer loads a seccomp profile matching the host architecture. For special needs the profile can be customized, but the principle is always: the narrower the whitelist, the smaller the attacker's reach. Extra syscalls are only added when the workload genuinely requires them — for example, for a new device or feature.

#Isolation Best Practices

Some habits that should become reflex:

• One VM, one jailer, one --id: never share a chroot between VMs.
• A cgroup per VM: set memory.max and cpu.max on a dedicated cgroup, so one VM leaking memory doesn't kill its neighbors (details in episode 10).
• A network namespace per VM: isolate networking at the namespace level, not just different TAPs.
• Read-only rootfs: after boot, there's no reason for a guest to write to its operating system.
• Never run as root: a non-root --uid/--gid is a prerequisite, not optional.
• Version discipline: jailer and firecracker must be the same version — mixing versions can cause unexpected behavior.

#Common Pitfalls

• Host paths in the API: forgetting that inside the jail paths become chroot-relative → 400 Bad Request or file not found.
• Duplicate --id: two VMs using the same --id → the jailer refuses or the chroots clash.
• Socket in /tmp: after the jailer, the socket is no longer in /tmp; it's at chroot-base-dir/<id>/run/firecracker.socket.
• Files not copied into the chroot: the kernel/rootfs are invisible to the process → boot fails.
• TAP created in the host namespace: if you use --netns, the TAP must exist in that namespace — create it after entering the namespace or with ip netns exec.
• UID/GID without /dev/kvm access: make sure the non-root user has read/write permission on /dev/kvm (e.g. via the kvm group).

#Closing

The key takeaways:

• The jailer assumes the VMM can be breached and designs so a breach isn't dangerous.
• Isolation layers: namespace, cgroup, privilege drop, chroot, seccomp, read-only rootfs.
• --id, --chroot-base-dir, --exec-file, --uid/--gid, --netns are the core flags.
• Inside the jail, API paths become chroot-relative; files must be copied into the chroot first.
• One jailer per VM, one cgroup per VM, one network namespace per VM.
• Firecracker and the jailer must be the same version.

In the next episode 8 we'll prepare the right provisions for every microVM: MicroVM Images — Build & Kernel Optimization — building a minimal rootfs with Alpine/musl, configuring the kernel to load only the drivers you need, choosing the right boot params, and getting to know tooling such as firectl, firecracker-containerd, NixOS microvm.nix, and mkosi.