Learn Cloud Hypervisor - CPU, Memory & Machine Config

#Introduction

Now that your first VM booted successfully in episode 4, it's time to control its resources precisely. A VM without correct CPU and memory settings is like an engine forced to run without knowing its capacity — it might run, but not optimally, and it risks running out of resources at a critical moment.

In episode 5 we cover three things: how --cpus defines vCPU topology and features, how --memory manages RAM size, hotplug, and physical allocation (hugepages), and how to write all of this in a versionable JSON config file.

#Configuring CPUs

#Basics: --cpus boot=N

The default vCPU count is 1. For real workloads, you'll usually need more:

cloud-hypervisor --cpus boot=8 --kernel kernel-vmlinux

boot=8 means the VM has 8 vCPUs right from startup. Because vCPUs are threads on the host, don't give a VM more vCPUs than the available physical logical CPUs — otherwise you get oversubscription, which actually slows down workloads.

#Topology

For NUMA-sensitive guests or applications that infer CPU topology, set the topology explicitly:

cloud-hypervisor \
  --cpus boot=8,topology="1 socket,8 cores,1 thread" \
  --kernel kernel-vmlinux

topology="1 socket,8 cores,1 thread" tells the guest the virtual CPU's physical structure. Applications like databases often size their thread pools based on this topology — a wrong setting can cause unpredictable performance.

#CPU Features and PMU

By default, Cloud Hypervisor exposes the CPU features supported by the host. You can adjust the exposed features with features=:

cloud-hypervisor --cpus boot=4,features="amx,pmu" --kernel kernel-vmlinux

pmu enables the virtual Performance Monitoring Unit so the guest can use perf. amx enables Advanced Matrix Extensions — important for AI/ML workloads. These features must be supported by the physical host; otherwise the VM will fail to start with a clear error.

#CPU Hotplug

Additional vCPUs can be added while the VM is running (details in episode 8), but the total capacity is defined up front with max=:

cloud-hypervisor --cpus boot=2,max=16 --kernel kernel-vmlinux

max=16 allows adding vCPUs from 2 up to 16 without restarting — a typical strategy for handling load spikes.

#Configuring Memory

#Size, Hotplug, and Max

cloud-hypervisor \
  --memory size=4G,hotplug_size=12G,hotplugged_size=0 \
  --cpus boot=4,max=16 \
  --kernel kernel-vmlinux

• size=4G: 4 GB of initial RAM.
• hotplug_size=12G: the maximum additional memory that can be hotplugged.
• hotplugged_size=0: no memory hotplugged at startup.

This pattern gives room to grow without paying for physical memory up front — ideal for workloads with unpredictable peak loads.

#Hugepages

Normal page allocation (4 KB) forces the TLB to work harder. With hugepages (e.g., 2 MB or 1 GB), the number of TLB entries drops dramatically, making memory access much faster — especially for memory-intensive workloads.

echo 2048 | sudo tee /proc/sys/vm/nr_hugepages

Then point guest memory at the hugepage file:

cloud-hypervisor --memory size=4G,hugepages=on,shared=on --kernel kernel-vmlinux

hugepages=on allocates guest RAM from the host's hugepage pool; shared=on opens the memory mapping so other processes can access it (required for snapshot and live migration). We optimize this further in episode 20.

#Balloon and Shared Memory

The balloon driver lets the host reclaim unused guest memory. Cloud Hypervisor supports virtio-balloon: the host "deflates" the balloon to pressure the guest into returning memory pages, or "inflates" it to give them back. This is a common memory oversubscription mechanism in the cloud — we cover its operation during hotplug in episode 8.

Shared memory (shared=on) maps guest RAM as a file that other processes can mmap — a prerequisite for vhost-user devices that read ring buffers directly from guest memory.

#CLI Flags vs. JSON Config

The more options there are, the longer the command line gets. For versionable configuration, Cloud Hypervisor supports a JSON file via --config. Here's an example file equivalent to the episode 4 boot command:

{
  "kernel": {
    "path": "kernel-vmlinux"
  },
  "disk": [
    {
      "path": "ubuntu.raw"
    }
  ],
  "cpus": {
    "boot": 4
  },
  "memory": {
    "size": 4294967296
  },
  "net": [
    {
      "tap": "ch0",
      "ip": "192.168.100.1",
      "mac": "a8:21:95:80:35:e6"
    }
  ],
  "cmdline": {
    "args": "console=ttyS0 root=/dev/vda1 rw quiet"
  }
}

Note: size is written in bytes (4294967296 = 4 GB), not as a string. Run it with:

cloud-hypervisor --config guest.json

JSON configuration is the source of truth that can be stored in Git, reviewed, and versioned — a practice we push throughout this series. To make things easier, there's the --api-socket option that exposes runtime control (episode 8), and the config format can be converted between the two with cloud-hypervisor --config-file in newer versions.

#Pitfalls and Best Practices

• Excessive oversubscription: virtual vCPUs + memory exceeding physical capacity causes thrashing. Start conservative, monitor, then increase.
• Hugepages unavailable: if nr_hugepages is zero and the pool is full, the VM fails to start. Size the pool to match --memory size.
• Always set max: leave room for hotplug from the start; adding capacity after startup is very limited.
• Version your config: keep the JSON config in a repo, not in shell history.

#Conclusion

Key takeaways:

• --cpus boot=N sets the initial vCPUs; topology, features, and max provide advanced control.
• --memory size= for initial RAM, hotplug_size for room to grow, hugepages for performance.
• Balloon lets the host reclaim idle guest memory.
• A JSON file via --config is the right way to get reproducible, reviewable configuration.
• Memory sizes in JSON use bytes; in the CLI they use notation like 4G.

In the next episode, episode 6, we'll configure storage: block, pmem, and virtio-fs — presenting virtio-blk disks from various formats (raw/qcow2), persistent memory, sharing the filesystem with the host via virtio-fs, and converting images with qemu-img. This is where your data is really stored and shared.