Learn Firecracker - Scaling: Many MicroVMs per Host

#Introduction

In episode 18 we saw the products standing on Firecracker — they all boil down to one economic question: how many microVMs can live on a single host? Episode 19 covers scaling: the techniques that make densities of thousands of microVMs per host possible, and the operational discipline that keeps them healthy.

Why is this episode important? Firecracker exists to make that high density possible — a 125 ms boot and < 5 MiB overhead only mean something if hundreds or even thousands of VMs can share one host. This is the episode where those numbers translate into real capacity, with honest trade-offs.

#The Math of Density

A modern server host (e.g. 192 vCPUs, 512 GB RAM) can hold how many microVMs? Let's calculate with the data we have:

• Small microVMs (1 vCPU, 128 MiB): ~512 MiB overhead + guest memory per VM. With healthy overcommit, a host can hold hundreds to thousands of VMs like this.
• Large microVMs (16 vCPU, 32 GB — the Lambda MicroVMs class): far fewer per host, but still dense compared to traditional VMs.

The limiting factors aren't just RAM: there are vCPUs, page cache, I/O, and kernel limits (number of fds, threads, inotify). Realistic density is determined by load testing (episode 16), not paper calculations.

#Jailer per VM: Overhead Worth Paying

Every microVM runs through the jailer with a unique --id (episode 7). The per-jailer overhead is small — a few processes and a cgroup — but its value is large:

• Isolation: one VM failing doesn't touch the others.
• Cleanup: deleting the <id> chroot is a clean way to release all resources.
• Auditing: every VM has a clear trail in the filesystem and cgroups.

At the thousands scale, good habits become mandatory: never skip the jailer "for speed." Code managing thousands of VMs without a jailer is an accident waiting to happen.

#cgroup v2: The Basis of Collective Limits

At high density, a single leaking VM can exhaust the host. cgroup v2 gives structured control:

mkdir -p /sys/fs/cgroup/fc/vm-001
echo 536870912 > /sys/fs/cgroup/fc/vm-001/memory.max
echo 500000 100000 > /sys/fs/cgroup/fc/vm-001/cpu.max
echo <pid> > /sys/fs/cgroup/fc/vm-001/cgroup.procs

In cgroup v2, there's one bonus important for density: more precise memory control, including swap limits and page cache accounting. With per-VM limits, a greedy VM doesn't take down the host — it hits its own limit.

#Page Cache Sharing: Read-Only Rootfs Turns to Gold

This is one of the highest-impact density techniques. When many VMs use the same rootfs, the host kernel stores the same file pages once in the page cache — and all VMs share them.

cat /proc/meminfo | grep -i cached

For maximum sharing:

• A read-only rootfs (episode 6) — if the rootfs can change, the page cache can't be safely shared.
• The same image per VM — the more VMs use an identical image, the higher the sharing ratio.
• Page cache overcommit: shared pages aren't fully counted per VM, so total "used" memory drops drastically.

This is where episode 8's image decisions meet host economics: the same, read-only, lightweight image = more VMs per host.

#Memory Overcommit via Balloon

The balloon (episode 10) adds another dimension: promised memory isn't always used. A smart orchestrator uses balloon statistics to overcommit:

1. VMs are created with mem_size_mib larger than typical needs.
2. The balloon pulls when a VM is idle — memory returns to the host.
3. When a VM needs it, the balloon deflates (with deflate_on_oom as a safety net).

Overcommit increases the number of VMs you can promise — with a risk: if all VMs need peak memory at once, the host can run out. The discipline:

• Moderate overcommit (e.g. 1.2–1.5x), not extreme.
• deflate_on_oom: true on all VMs.
• Monitor balloon statistics and host memory pressure in real time.

#Operational Discipline at the Thousands Scale

The techniques above are worthless without disciplined operations. Three pillars:

#A Daemon per MicroVM

At large scale, VMs aren't managed one by one through a terminal. One daemon or agent per microVM (or per host) manages the lifecycle: launch, health check, restart, cleanup. This is Flintlock's role (episode 12) or a custom agent. The pattern: centralized control, per-VM execution, auditable state.

#Health Checks

Every VM needs a definition of "healthy." Useful indicators:

• The Firecracker process is alive (not a zombie).
• The guest responds (via vsock or HTTP inside the guest).
• A snapshot can be restored within the time limit.

Per-VM health checks at the thousands scale must be lightweight — don't create a load bigger than the VMs being monitored.

#Snapshot Garbage Collection

Snapshots (episode 9) pile up fast: every suspend/resume leaves state and memory files behind. Without cleanup, the host disk fills and the restore pipeline slows. A healthy discipline:

find /snapshots -name '*.mem' -mtime +7 -delete
find /snapshots -name 'base' -mtime +7 -delete

A rule of thumb: snapshots not accessed within a certain period (per your SLA) are deleted, and cleanup runs as a scheduled job with audit.

#Density vs Latency: An Honest Trade-off

Density isn't a free target. The denser you get:

• Boot time tends to rise (CPU competition during init).
• Snapshot restore can slow down (fragmented page cache, busy disk I/O).
• Cross-tenant disturbance risk rises (noisy neighbors).

The graphs you should build in the lab (episode 16): boot time vs number of VMs, restore time vs number of VMs. The point where the curve starts climbing is your healthy density — not the maximum number that can boot simultaneously.

#Common Pitfalls

• No jailer per VM: one leaking VM contaminates others; isolation is lost.
• Read-write rootfs: page cache sharing becomes ineffective; host memory is wasted.
• Overcommit without monitoring: mass OOM at peak load.
• Accumulated snapshots: disk full, slow restore — scheduled GC is mandatory.
• Expensive health checks: heavy per-VM polling kills the density benefit.
• Density measured on paper, not in the lab: real numbers come from load testing.

#Closing

The key takeaways:

• Density = jailer per VM + cgroup v2 + page cache sharing + balloon overcommit.
• The same read-only rootfs lets file pages be shared between VMs.
• Overcommit via balloon needs strict memory pressure monitoring.
• A daemon per VM, lightweight health checks, and snapshot GC are mandatory discipline.
• Healthy density is determined by the boot/restore curve vs number of VMs in the lab.

In the next episode 20 we'll keep the system visible: Observability & Troubleshooting — reading Firecracker logs with levels and ratelimit, pulling metrics via the /metrics API, tracing, debugging through the serial console, handling KVM errors and snapshot restore failures, and using firecracker --help and gdb when all else fails.