Learn Firecracker - MicroVM Images: Build & Kernel Optimization

#Introduction

In episode 7 we locked down the microVM with the jailer. Now the question is: what's inside? Episode 8 covers every microVM's provisions — the image: a rootfs and kernel built specifically to boot fast, use little memory, and contain only what's needed.

Why is this episode important? Firecracker promises ~125 ms boot and < 5 MiB overhead — but that promise is only kept if the image cooperates. A bloated rootfs, a kernel loading hundreds of unused drivers, and a slow init will make a microVM boot in seconds and consume many times more memory. The image is half the performance.

#MicroVM Image Philosophy

Kernels and rootfs in the microVM world must follow different principles from a normal server distro:

• Boot once, live briefly: nobody updates packages mid-flight; the image is an immutable template.
• Small is fast: the less that gets loaded at boot, the faster the cold start.
• All state lives outside: persistent data never stays in the rootfs — it lives on a data drive, in snapshots, or in external storage.
• Read-only is a friend: a read-only rootfs enables page cache sharing and eliminates a whole class of system-modification bugs.

With these principles, a microVM image becomes an artifact that can be built, tested, and reproduced deterministically — not a filesystem growing like a weed.

#Building a Minimal Rootfs with Alpine

Alpine Linux is a popular choice for microVMs: it's small, uses musl libc, and boots from almost nothing. The plan: create an ext4 filesystem, bootstrap Alpine with apk, and install only the packages needed.

dd if=/dev/zero of=rootfs.ext4 bs=1M count=256
mkfs.ext4 rootfs.ext4
mkdir -p /mnt/fc-rootfs
sudo mount rootfs.ext4 /mnt/fc-rootfs
sudo mkdir -p /mnt/fc-rootfs/etc/apk

Add the Alpine repository and bootstrap:

sudo apk add --root /mnt/fc-rootfs --initdb --arch x86_64 alpine-base

Install only what you need — for example, a shell and basic networking:

sudo apk add --root /mnt/fc-rootfs --no-cache \
  alpine-baselayout busybox musl

Don't install linux-firmware, don't install a whole desktop — every package adds boot time and memory. In the final stage, configure /etc/inittab so init runs with console=ttyS0, then unmount:

sudo umount /mnt/fc-rootfs

#A MicroVM-Friendly Kernel

The microVM kernel is built with a configuration that cuts everything unneeded. The main principles:

• Only the drivers you use: virtio-net, virtio-blk, virtio-fs, virtio-rng, serial 8250. No sound, USB, GPU, or exotic filesystems.
• Static builds for the critical parts: storage and network drivers are built into the kernel (not as modules) — without modules, there's no module loading time at boot.
• No initramfs: the ext4 rootfs is recognized directly by the kernel; without an initramfs, boot is faster and the image smaller.
• Security features on: seccomp and standard mitigation features stay enabled — optimization doesn't mean stripping security.

An example of the configuration direction (via make menuconfig or a config fragment):

CONFIG_VIRTIO=y
CONFIG_VIRTIO_MMIO=y
CONFIG_VIRTIO_BLK=y
CONFIG_VIRTIO_NET=y
CONFIG_VIRTIO_FS=y
CONFIG_VIRTIO_RNG=y
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_EXT4_FS=y
CONFIG_NO_HZ_IDLE=y

Then build:

make defconfig
make -j$(nproc) vmlinux

The result is a vmlinux (uncompressed ELF) ready for Firecracker. The final size is usually under 10 MB.

#The Right Boot Params

Boot args determine how the kernel greets the virtual hardware:

{
  "kernel_image_path": "/vmlinux.bin",
  "boot_args": "console=ttyS0 reboot=k panic=1 pci=off iommu=off"
}

• console=ttyS0 — output to serial, readable by operators.
• reboot=k — reboot via KVM (instant, no power cycle).
• panic=1 — reboot 1 second after a kernel panic; an ephemeral microVM shouldn't wait for a human.
• pci=off — disable PCI scanning; speeds up boot when all devices are used via MMIO.
• iommu=off — for hosts without an IOMMU, avoids IOMMU initialization time.

Boot params are a tuning instrument: without pci=off, the kernel wastes time scanning the PCI bus; with console=ttyS0, operators can read crashes. Choose according to your workload — don't copy blindly.

#Image Build Tooling

Building images by hand over and over isn't productive. The ecosystem provides tooling:

• firectl — a CLI to launch Firecracker from an OCI container image; build once, run many (episode 12).
• firecracker-containerd — a containerd runtime that wraps container images into microVMs (episode 11).
• NixOS microvm.nix — declares an entire microVM image (kernel, rootfs, init) as a reproducible, testable Nix configuration.
• mkosi — builds complete OS images (systemd-based) from a declarative spec, including kernel and rootfs for microVMs.

The common pattern of this tooling: declarative spec → image artifact. The kernel, rootfs, and init are defined once in configuration; the result is an immutable, hashed, distributed image. This is the foundation of the "build an image from a Dockerfile" flow we see in AWS Lambda MicroVMs (episode 18).

#Common Pitfalls

• Compressed kernel: vmlinuz can't be loaded; always use an ELF vmlinux.
• Drivers as modules: virtio devices aren't detected because their modules aren't loaded; build critical drivers into the kernel.
• Rootfs without init: the kernel boots but finds no first process → panic. Make sure /sbin/init exists and /etc/inittab is correct.
• console= not set: boot output disappears; debugging goes blind.
• Bloated image: loading unused packages slows boot and grows memory — audit image contents regularly.
• Unnecessary initramfs: the rootfs is recognized directly; without an initramfs, boot is faster.

#Closing

The key takeaways:

• MicroVM image: small, immutable, read-only, with state outside.
• Minimal rootfs with Alpine/musl; every package adds boot cost.
• Kernel built with only the static drivers you need, without an initramfs.
• Boot params like console=ttyS0, panic=1, pci=off are real tuning instruments.
• Use declarative tooling: firectl, firecracker-containerd, microvm.nix, mkosi.

In the next episode 9 we'll unlock the capability that makes Firecracker so beloved by serverless: Snapshot & Restore — freezing a running microVM with PUT /snapshot/create, bringing it back to life with PUT /snapshot/load, understanding the difference between memory and diff snapshots, and how this pattern delivers instant cold starts and suspend/resume.