Learn Proxmox VE - GPU Passthrough (PCIe Passthrough) & Nested Virtualization

#Introduction

Until now, VMs only saw virtual devices — virtual CPUs, virtual disks, virtual NICs. But there are workloads that demand physical devices: machine learning training, video transcoding, or gaming. Episode 16 opens the door to this world through PCIe passthrough.

We'll understand the passthrough concept, enable IOMMU in the BIOS and Proxmox, bypass the host GPU driver, and then pass the GPU to a VM. At the end of the episode, we'll also cover nested virtualization — running virtualization inside virtualization.

#The PCIe Passthrough Concept

#Direct Access to Physical Devices

PCIe passthrough gives a VM direct access to a physical device on the PCIe bus — a GPU, NIC, or storage controller. The VM no longer sees an emulated virtual device, but the actual hardware. The effect: performance almost identical to bare-metal, because there's no emulation layer.

Emulation   : VM -> virtual driver -> host -> physical device
Passthrough : VM -> native driver -> physical device (direct)

The downside: a passed-through device can't be used by other VMs, and live migration isn't supported because the hardware is tied to the node.

#Identifying the Device

Before configuring, identify the device and its PCI address:

lspci -nnk | grep -A3 -i vga

The lspci -nnk command shows all PCI devices along with the drivers currently using them. Note down the address like 01:00.0 and the vendor ID for your GPU.

#GPU Passthrough Configuration

#Enabling IOMMU

The first step is IOMMU — the technology that allows devices to be shared safely. Enable it in the BIOS: Intel VT-d for Intel, AMD-Vi for AMD. Then tell the kernel:

nano /etc/default/grub

Add the parameters to the GRUB_CMDLINE_LINUX_DEFAULT line. For Intel: intel_iommu=on iommu=pt. For AMD: amd_iommu=on iommu=pt. Save, then:

update-grub
reboot

After rebooting, verify that IOMMU is active:

dmesg | grep -e DMAR -e IOMMU

#Blacklisting the Host GPU Driver

A GPU can't be passed through while the host driver is still using it. Blacklist the drivers — nouveau and nvidia for NVIDIA, amdgpu for AMD:

echo "blacklist nouveau" > /etc/modprobe.d/blacklist-gpu.conf
update-initramfs -u

Also add the vfio-pci module to the list of modules loaded at boot so the GPU is immediately held by VFIO.

#Passing the GPU to a VM

With IOMMU active and the host driver bypassed, add the GPU as a device on the VM. From the web UI, open VM -> Hardware -> Add -> PCI Device, select the GPU by address, check options like Primary GPU if the VM uses the GPU as its main display, and ROM-Bar as needed.

Raw Device: 01:00.0 (GPU) -> connected directly to the VM

Install the driver inside the guest (NVIDIA or AMD depending on the GPU), and the GPU will be detected as real hardware.

#GPU Use Cases

GPU passthrough opens up many scenarios:

• Machine learning: training and inference using CUDA in a VM.
• Video transcoding: NVIDIA NVENC for video encoding.
• Gaming VMs: a full GPU for games inside a VM, with SPICE or USB passthrough.

For gaming, also add a USB controller passthrough or use Looking Glass for low-latency display.

#Nested Virtualization

#Virtualization Inside a VM

Nested virtualization allows a VM to run another hypervisor inside it — Docker, Kubernetes, even ESXi or Proxmox inside a Proxmox VM. Enable it with a single setting:

qm set 100 --args "-cpu host,+vmx"

The qm set 100 --args "-cpu host,+vmx" command adds the +vmx flag (or +svm for AMD) to the VM's CPU, so the guest sees virtualization support and can run its own hypervisor. This is a common pattern for laboratories, testing, and efficiently running container engines.

#Closing

Episode 16 opened up full hardware access: understanding PCIe passthrough, enabling IOMMU, bypassing the host GPU driver, passing GPUs to VMs for ML, transcoding, and gaming, and enabling nested virtualization for hypervisors inside VMs.

The key takeaways:

• Passthrough gives VMs direct access to physical devices.
• Enable IOMMU in the BIOS and kernel: VT-d for Intel, AMD-Vi for AMD.
• Blacklist the host GPU driver so it doesn't compete with VFIO.
• Add the GPU as a PCI device in the VM configuration.
• GPUs are useful for ML, transcoding, and gaming VMs.
• Nested virtualization is enabled with the +vmx or +svm flags.

In the next episode, episode 17, we will cover infrastructure automation — provisioning VMs with Terraform and OpenTofu via the bpg/proxmoxve provider, managing the lifecycle with Ansible, and end-to-end automated flows from cloud-init templates to a configured software stack. Manual provisioning is ready to be left behind; welcome to the era of Infrastructure as Code!