Learn Cloud Hypervisor - Nested Virtualization & EC2

#Introduction

So far we've assumed your host is bare-metal you fully control. But what if you want to run Cloud Hypervisor in the public cloud — where the "host" itself is already a VM? The answer is nested virtualization: running a hypervisor (KVM) inside a VM provided by the cloud provider.

This changes the game. Previously, microVM CI/CD or a Kata Containers cluster needed bare-metal — expensive and inflexible. With nested virtualization on AWS EC2 (and other providers), you can build a microVM platform on regular instances that scales with demand.

#Nested Virtualization on AWS EC2

#Official Support (February 2026)

AWS officially supports nested virtualization on certain instance classes — including C8i, M8i, and R8i (and similar classes) since February 2026. This means these instances expose the virtualization flags (vmx/svm) to the guest, so inside the instance you can:

1. Load KVM (/dev/kvm available).
2. Run any VMM — including Cloud Hypervisor.
3. Build VMs inside VMs (guest-of-guest).

Verify that your instance supports nested virtualization:

grep -cE "(vmx|svm)" /proc/cpuinfo
ls -l /dev/kvm

If /dev/kvm isn't there, load its module:

sudo modprobe kvm_intel   # atau kvm_amd untuk AMD
ls -l /dev/kvm

#Building MicroVM CI/CD

The most interesting scenario: microVM CI/CD runners. Instead of runners that share the host kernel, each job runs in its own Cloud Hypervisor VM — full isolation, with no VM "stickiness" between jobs.

cloud-hypervisor \
  --kernel kernel-vmlinux \
  --disk path=job.raw \
  --cpus boot=2 \
  --memory size=1G \
  --net tap=ci0,ip=172.16.0.1,mac=a8:21:95:80:00:99 \
  --cmdline "console=ttyS0 root=/dev/vda1 rw"

With this pattern, a single C8i instance can run a dozen parallel CI microVMs — replacing the need for bare-metal in such workloads. Cloud Hypervisor's boot speed (episode 16) is the key: jobs start in seconds.

#Calculating Capacity

Nested virtualization has a cost: layered virtualization adds overhead. Measure with benchmarks (episode 16) how much performance you can reach, and watch metrics like CPU steal time:

top -b -n1 | grep -E "steal|wa"

High steal indicates the physical host above you is busy too — consider a larger instance or a different class.

#Bare-metal and On-prem

#When Bare-metal Is Still the Choice

Nested virtualization isn't free and has limits. For workloads that need deterministic performance — HPC, heavy I/O workloads, or VFIO passthrough that needs direct hardware (episode 8) — bare-metal remains the primary choice:

• AWS .metal classes: Nitro bare-metal instances with full /dev/kvm and no extra virtualization layer.
• On-prem/colo: physical servers in your own data center with full control over BIOS and firmware.

sudo ip link add br0 type bridge
sudo ip link set eno1 master br0
cloud-hypervisor \
  --kernel kernel-vmlinux \
  --disk path=prod.raw \
  --net tap=prod0,ip=10.0.0.1,mac=a8:21:95:80:00:11 \
  --cpus boot=8 \
  --memory size=16G

#Edge Nodes

At the edge — devices with limited resources and unstable networks — Cloud Hypervisor offers two valuable things:

• Small footprint: a static binary and low memory overhead suit devices with limited RAM.
• Strong isolation: running third-party workloads on edge devices without worrying they'll take over the device.

A common pattern: an edge node runs Cloud Hypervisor with a small image and immutable workloads, managed centrally:

cloud-hypervisor \
  --kernel kernel-vmlinux \
  --disk path=edge.raw,readonly=on \
  --cpus boot=1 \
  --memory size=512M \
  --cmdline "console=ttyS0 root=/dev/vda1 ro quiet"

readonly=on and ro make the edge image immutable — every reboot returns to its original state, reducing drift and persistent-compromise risk.

#Recommended Deployment Architecture

For production, combine these patterns:

• MicroVM CI/CD: EC2 nested virtualization instances (C8i/M8i/R8i) → a Cloud Hypervisor microVM per job.
• Multi-tenant: on-prem bare-metal → a VM cluster with per-VM TAPs + firewall (episode 13).
• Edge: small devices → immutable read-only images, centralized control.

Cloud publik (EC2 nested)   → CI/CD microVM, Kata cluster
Bare-metal / on-prem        → workload deterministik, VFIO, HPC
Edge nodes                  → microVM immutable, footprint kecil

#Common Pitfalls

• /dev/kvm missing: check the instance class, or load the kvm module.
• Odd performance in nested: high steal time or slow boot — measure and compare with bare-metal.
• Passthrough in nested: VFIO needs direct hardware — it doesn't work in guest-of-guest.
• Edge with small storage: make sure the disk image fits; use qcow2/backing files to save space.
• Edge networking: a TAP on a device with one NIC — set up correct routing and DNS.

#Conclusion

Key takeaways:

• Nested virtualization enables KVM and Cloud Hypervisor on regular cloud instances.
• AWS EC2 C8i/M8i/R8i classes support nested virtualization (Feb 2026).
• MicroVM CI/CD: each job in an isolated VM, without needing bare-metal.
• Bare-metal is still required for VFIO, HPC, and deterministic performance.
• Edge nodes suit small footprint + immutable read-only images.
• Choose your deployment by need: cloud for elasticity, bare-metal for determinism.

In the next episode, episode 20, we'll refine performance & observability — hugepages, CPU pinning/affinity, NUMA awareness, then VMM logs, metrics, tracing, and debugging with gdb. From "a running VM" toward "a measured and optimized VM".