Learn Kata Containers - Runtime Configuration (kata-configuration.toml)

#Introduction

Your first Kata pod ran in episode 4. Now it's time to understand why it ran the way it did. Behind every microVM there's one configuration file: /etc/kata-containers/configuration.toml. This file determines which hypervisor is used, which guest kernel is booted, how many resources are allocated, and how the agent behaves.

Episode 5 dissects this file section by section. The ability to read and modify configuration.toml is what separates an operator who merely runs Kata from an operator who can tune Kata for specific workloads — and it will be an important foundation when we discuss performance tuning in episode 20.

#General Structure of configuration.toml

A Kata installation provides several configuration files, one per backend:

• /etc/kata-containers/configuration.toml — default (QEMU).
• /etc/kata-containers/configuration-clh.toml — for Cloud Hypervisor.
• /etc/kata-containers/configuration-fc.toml — for Firecracker.

All files follow the same structure: one [hypervisor.*] section for VMM configuration, one [agent.kata] section for the Kata agent configuration, and [runtime] sections for runtime behavior. Let's dissect the sections you'll touch most often:

/etc/kata-containers/configuration.toml

# Hypervisor configuration
[hypervisor.qemu]
path = "/usr/bin/qemu-system-x86_64"
kernel = "/usr/share/kata-containers/vmlinux.container"
image = "/usr/share/kata-containers/kata-containers.img"
 
machine_type = "pc"
 
# Default memory and CPU for the VM
default_memory = 2048
default_vcpus = 1
 
# Kernel parameters passed to the guest kernel
kernel_params = ""
 
# Agent configuration
[agent.kata]
kernel_modules = []
 
# Runtime configuration
[runtime]
enable_debug = false
sandbox_cgroup_only = false

path points to the hypervisor binary, kernel to the guest kernel, and image to the guest image (initrd or guest root filesystem). These three values are what bring the microVM to life.

#Hypervisor and Guest Kernel

#Choosing the Backend

The [hypervisor.qemu] section controls the VMM. The section name determines which hypervisor is used — [hypervisor.qemu], [hypervisor.clh], [hypervisor.firecracker]. The kata-clh RuntimeClass (episode 4) points to the configuration-clh.toml file whose hypervisor section is [hypervisor.clh].

The hypervisor choice has a big impact on features and speed — a full topic in episode 6. In this file, you can adjust additional parameters:

Additional VMM parameters

[hypervisor.qemu]
firmware = ""
enable_vhost_user_store = false
valid_hypervisor_paths = [
    "/usr/bin/qemu-system-x86_64",
    "/usr/libexec/qemu-kvm"
]
enable_iothreads = true

enable_iothreads = true allows QEMU to use separate threads for I/O — reducing blocking between devices. valid_hypervisor_paths provides a list of alternative paths so the runtime can find the binary across different distros.

#Guest Kernel

The guest kernel is a separate Linux kernel booted inside the microVM. The kernel field points to this kernel file. Kata provides variants: vmlinux (ELF uncompressed, faster boot) and vmlinuz (compressed). For confidential computing (episode 11), a kernel with TDX/SEV support is used.

#Memory, CPU, and Resource Limits

#Default Configuration in the File

When a pod doesn't specify limits, Kata uses the default values from the file:

Default resource configuration

default_memory = 2048
default_memoryslots = 2
default_vcpus = 1
 
# Enable/disable memory hotplug
enable_memory_hotplug = true

default_memory = 2048 means 2 GiB of RAM is allocated to the microVM, and default_vcpus = 1 means one vCPU. When a pod specifies resources.limits, the pod's values override these defaults.

#How Pod Limits Are Applied

A Kata pod translates Kubernetes request/limit directly into microVM allocation. If a pod specifies:

spec:
  containers:
    - name: app
      resources:
        requests:
          memory: 512Mi
          cpu: 500m
        limits:
          memory: 1Gi
          cpu: "1"

The resulting microVM will be allocated about 1 GiB of memory and 1 vCPU — reflecting the limits, not the requests. This is what episode 10 discusses as "resource limits applied at the VM": pods no longer compete for cgroups on the host, but instead get a clear portion of virtual hardware.

#Block Device Driver: virtio-blk vs virtio-scsi

One of the parameters with the biggest impact on I/O is the block device driver choice. Two main options:

• virtio-blk: simple, one device per disk, low overhead. Suitable for general workloads.
• virtio-scsi: more complex, allows many disks per device, and supports SCSI operations such as persistent reservation — needed for cluster filesystems and some database workloads.

The setting in the configuration file:

Block device driver choice

[hypervisor.qemu]
# Support block device driver: virtio-blk or virtio-scsi
block_device_driver = "virtio-blk"
 
# Use virtio-fs for shared filesystems (episode 8)
shared_fs = "virtio-fs"
 
# Whether to use SCSI controller
enable_virtio_mem = false

block_device_driver = "virtio-blk" selects virtio-blk as the default. For workloads that need virtio-scsi, change the value to "virtio-scsi" and make sure the guest kernel supports it.

This decision is usually rarely changed after production is running, because it affects how the guest sees disks. Test it in a lab environment before applying it to a cluster.

#kernel_params: Tuning the Guest Kernel

The kernel_params field passes parameters to the guest kernel at boot. This is the entry point for tuning the guest without building a new kernel:

Adding guest kernel parameters

kernel_params = "console=ttyS0 quiet net.ifnames=0"
 
# Example: enable swap inside the guest
# kernel_params = "swapaccount=1"

kernel_params = "console=ttyS0 quiet net.ifnames=0" sets up the serial console for debugging and disables predictable interface naming. Other useful parameters: page_poison=1 for memory security, or nokaslr for debugging with stable symbol resolution.

Note that changing kernel_params affects all pods on the node using this configuration — the value is applied to every booted microVM. For per-pod needs, use the pod annotations Kata supports.

#Agent Configuration

The [agent.kata] section controls the Kata agent's behavior inside the guest:

Kata agent configuration

[agent.kata]
# When enabled, kernel modules can be loaded inside the guest
kernel_modules = []
 
# Enable debug logging inside the guest
debug_console_enabled = false
 
# Whether the agent can use vsock for communication
use_vsock = true

use_vsock = true enables shim-agent communication through vsock — the mechanism we discussed in episode 2. debug_console_enabled = true will give you console access into the guest, very useful when troubleshooting in episode 12.

#Verifying the Configuration

To see the configuration currently in effect for the runtime:

kata-runtime kata-env
kata-runtime version

kata-runtime kata-env displays the effective runtime configuration: hypervisor path, kernel, image, and default values — in JSON format. This is the fastest way to make sure the configuration file you changed is actually read by the runtime.

#Conclusion

What you should take away:

• configuration.toml controls the hypervisor, kernel, resources, and agent per backend.
• configuration-clh.toml and configuration-fc.toml for the Cloud Hypervisor and Firecracker backends.
• default_memory and default_vcpus are used when a pod doesn't specify limits.
• Kubernetes pod limits are translated directly into microVM allocation.
• block_device_driver selects virtio-blk or virtio-scsi.
• kernel_params forwards arguments to the guest kernel.
• kata-runtime kata-env verifies the effective configuration.

In the next episode, episode 6, we'll compare hypervisor backends — QEMU as the default with broad compatibility, Cloud Hypervisor which is modern and fast, Firecracker which is minimalist, and Dragonball which is built into the Rust runtime. You'll learn when to choose each one and the feature-versus-speed trade-offs.