Learn FFmpeg - Hardware Acceleration

#Introduction

In episode 14 you saw one FFmpeg command produce three HLS variants at once. Cool, but there's a cost: three running libx264 encoders will squeeze the CPU. At 1080p 30 fps that might still be fine, but try 4K, or 60 fps, or twelve variants — the CPU becomes the bottleneck and you start counting how long each transcode takes.

This is where hardware acceleration comes in. GPUs aren't just for gaming; modern video chips have dedicated circuits for encoding and decoding video that are much faster and more power-efficient than the CPU. Episode 15 dissects the various acceleration interfaces in FFmpeg — NVENC, VAAPI, QSV, AMF, VideoToolbox, and D3D12 — plus the newest 2026 trend using Vulkan. The goal isn't memorizing every flag, but understanding which path is available on your machine and when it's worth using.

#Two Kinds of GPU Work

Before choosing an encoder, it's important to understand that GPUs handle video through two different mechanisms:

Fixed-function block. Since around 2012, GPUs ship with dedicated hardwired circuits for H.264/H.265/AV1 encoding and decoding. These circuits aren't flexible — they can only do specific codecs — but they're very fast and very power-efficient. This is what NVENC, QSV, AMF, and other block encoders use.

Compute shader. GPUs can also be generally programmed via parallel computation — an array of cores doing thousands of operations at once. This mechanism is used to process codecs without a fixed-function block (like ProRes or DPX), and it's the material for the 2026 trend we'll discuss at the end of the episode.

The practical consequence: GPU speed is almost independent of resolution or frame rate the way CPU is. Encoding 4K with NVENC is often not much slower than 1080p — its capacity is determined by the number of encode engines, not core strength.

#Checking Support on Your Machine

Never assume an encoder is available. Every FFmpeg is built with a different configuration, and every GPU has different capabilities. These two commands are the first habit you should have:

ffmpeg -hwaccels
ffmpeg -encoders | grep -i nvenc
ffmpeg -encoders | grep -i qsv

-hwaccels shows the available hwaccels (for decoding), and `-encoders | grep -i nvenc{:bash}` filters hardware encoders by name. If an encoder doesn't appear, don't guess — your FFmpeg simply wasn't built with that support, or the driver isn't detected.

#Acceleration Paths per Platform

#NVENC: NVIDIA

NVENC is the hardwired encode engine on NVIDIA GPUs, and one of the easiest to use in FFmpeg:

ffmpeg -i input.mp4 -c:v h264_nvenc -preset p6 -rc vbr -cq 23 -b:v 0 output.mp4

• -preset p6 picks the quality-versus-speed preset (p1 fastest, p7 slowest and best).
• -rc vbr uses variable bitrate mode; -cq 23 targets quality (smaller is better), and -b:v 0 disables the bitrate target so pure CQ decides.
• NVENC also has hevc_nvenc and av1_nvenc for newer codecs.

For a fully-GPU pipeline, add -hwaccel cuda so decoding also runs on the GPU and frames don't bounce back and forth through system memory. If there's scaling, use scale_cuda instead of plain scale — CUDA-based filters keep frames on the GPU.

#VAAPI: Linux for Intel and AMD

VAAPI (Video Acceleration API) is the video acceleration standard on Linux supporting both Intel and AMD GPUs. Its entry point is the render device node, usually /dev/dri/renderD128:

ffmpeg -vaapi_device /dev/dri/renderD128 -i input.mp4 -vf format=nv12,hwupload -c:v h264_vaapi -b:v 4M output.mp4

Notice the format=nv12,hwupload pattern: frames decoded on the CPU are converted to a hardware-supported format, then uploaded to the GPU with the hwupload filter. This is a mandatory pattern on the VAAPI path — forgetting hwupload is mistake number one when using VAAPI.

#QSV: Intel Quick Sync Video

QSV is Intel's acceleration technology. On Windows it uses the entrenched Intel driver, while on Linux it runs on top of VAAPI — so it needs the same device:

ffmpeg -init_hw_device qsv=hw:/dev/dri/renderD128 -i input.mp4 -vf hwupload=extra_hw_frames=64,format=qsv -c:v h264_qsv -b:v 4M output.mp4

-init_hw_device qsv=hw:/dev/dri/renderD128 creates a QSV device wrapping the VAAPI device. Allocating extra_hw_frames=64 gives the encoder frame queue space in VRAM — this value prevents some QSV encoders from crashing under high load.

#AMF: AMD

AMF (Advanced Media Framework) is AMD's acceleration API. In FFmpeg, its encoders are h264_amf and hevc_amf. On Windows, usage is direct:

ffmpeg -i input.mp4 -c:v h264_amf -b:v 4M -usage lowlatency output.mp4

On Linux, AMF support is more limited and depends on the GPU generation and driver; for AMD GPUs on Linux, the VAAPI path is usually still the most reliable choice.

#VideoToolbox: macOS

macOS provides VideoToolbox, the system acceleration API wrapped by the h264_videotoolbox and hevc_videotoolbox encoders:

ffmpeg -i input.mp4 -c:v hevc_videotoolbox -b:v 5M -allow_sw 1 output.mp4

There's no device setup as on Linux — VideoToolbox automatically uses the active GPU. -allow_sw 1 permits a software fallback if the codec isn't hardware-supported.

#D3D12: Modern Windows

Windows has two generations of graphics APIs: D3D11 (old) and D3D12 (modern). Since FFmpeg 8.1, D3D12-based encoders are available for H.264, HEVC, and AV1:

ffmpeg -i input.mp4 -c:v av1_d3d12va -b:v 5M output.mp4

D3D12 encoders use the modern Windows 10/11 API and fit pipelines that also use D3D12-based filters like scale_d3d12. This is the most relevant acceleration path in the new era, alongside the Vulkan trend below.

#Hardware vs Software Encoding: When to Use Which

The most frequent question: if the GPU is so much faster, why does anyone still use libx264? The answer is in one word: quality. Software encoders like libx264 and libx265 spend hundreds of CPU cycles per frame and produce better quality at the same bitrate — or smaller size at the same quality. Hardware encoders hit their speed targets with "good enough" quality.

A healthy rule of thumb: for a master that will be processed again or archived, invest CPU time. For delivery that must be fast and in volume, use the GPU. Many production teams do it twice: slow encode for the master, fast encode for distribution.

#The 2026 Trend: Vulkan and Compute Shader

Since the release of FFmpeg 8.1 in early 2026, the direction of hardware acceleration has shifted toward something more open and flexible: Vulkan. Vulkan is a modern graphics API supported by nearly all GPUs from all vendors, so a single FFmpeg implementation can run across many brands at once.

Fixed-function Vulkan encoders. In FFmpeg master, H.264 and H.265 encoders based on Vulkan Video are already present (h264_vulkan, hevc_vulkan), heading toward the next stable release. This brings encode acceleration to GPUs without an NVENC/QSV/AMF path — including some iGPUs and SoCs.

Stable D3D12 encoders. In FFmpeg 8.1, D3D12-based H.264 and AV1 encoders are stable for Windows, opening a more modern acceleration path than D3D11/DXVA2.

Vulkan compute for codecs without hardware. The most interesting innovation is on the compute shader side. FFmpeg 8.1 brings ProRes encoding and decoding, ProRes RAW decoding, DPX decoding, and FFv1 encode-decode executed via Vulkan compute shaders. Why does it matter? ProRes and DPX are post-production industry standards that never had a fixed-function block on any GPU. With compute shaders, any Vulkan-capable GPU can accelerate these professional codecs without special hardware. The encoders are named prores_ks_vulkan and ffv1_vulkan.

That means: processing 8K ProRes used to require a server-class workstation with dozens of cores; now, similar work can run on widespread consumer GPUs. The limit of "how much can be accelerated" is no longer set by aging hardwired circuits, but by the quality of the ever-improving shader implementations.

#Conclusion

In episode 15 you've explored the hardware acceleration map in FFmpeg: distinguishing fixed-function blocks and compute shaders, checking support with -encoders, using NVENC on NVIDIA, VAAPI for Intel/AMD on Linux, QSV for Intel, AMF for AMD, VideoToolbox on macOS, and D3D12 on modern Windows. You also understood the quality trade-off between hardware and software encoders, and the 2026 trend where Vulkan opens acceleration for professional codecs like ProRes via compute shaders.

Key takeaways:

• Always check -encoders before using a hardware encoder; never assume.
• The VAAPI/QSV path requires hwupload to get frames onto the GPU.
• GPU = speed and throughput; CPU = maximum quality per bit.
• Hardware encoders for delivery, software encoders for masters.
• FFmpeg 8.1 brings D3D12 H.264/AV1 and Vulkan compute for ProRes and DPX.

With encoding and packaging mastered, only one quality dimension remains that's often underestimated yet determines the whole look of video: color.

In the next episode 16 we'll cover HDR & Color Handling — HDR10 and HLG, the zscale and tonemap filters, HDR-to-SDR conversion, and the color metadata that determines how your video truly looks. Keep your enthusiasm up!