Learn FFmpeg - HDR & Color Handling

#Introduction

In episode 15 you learned to speed up encoding with a GPU. Now we talk about something that often goes unnoticed until the result looks wrong: color. You may have experienced video that looks pale and grayish, or too dark and oversaturated. In many cases, the video isn't broken — its color metadata is wrong, or there's a mismatch between how the video is meant to be displayed and how the device displays it.

Episode 16 covers color handling in FFmpeg comprehensively: what HDR10 and HLG are, how the zscale and tonemap filters turn HDR video into SDR, and the role of color metadata like -color_primaries, -color_trc, and -color_space. After this episode, you'll no longer ask "why did this video turn gray?" — you'll immediately know the answer.

#The Language of Video Color: Primaries, Transfer, and Matrix

Before diving into HDR, you must know three keywords that govern how color bits are interpreted into visible color. Together they're called color metadata, and all three can be set in FFmpeg:

• Color primaries determine the set of base colors (red, green, blue) used to represent the picture. The old standard uses BT.709 (for HDTV); HDR uses BT.2020 with a wider color gamut.
• Transfer function (often called gamma) determines how brightness values are stored non-linearly. SDR uses BT.709 or sRGB; HDR10 uses SMPTE ST 2084 (PQ); HLG uses ARIB STD-B67.
• Color matrix determines how RGB colors are transformed into luminance-chrominance signals (like YUV). BT.601 for SD, BT.709 for HD, BT.2020 for UHD.

Notice that the three are separate and can be combined arbitrarily — and that's where chaos often happens. A file could store 4K resolution (BT.2020 is natural), but its transfer function is written as BT.709. FFmpeg shows this metadata in the ffprobe output:

ffprobe -v error -select_streams v:0 -show_entries stream=color_primaries,color_trc,color_space,input.mp4

The result is lines like color_primaries=bt2020, color_trc=smpte2084, and color_space=bt2020nc — a typical combination for HDR10 video. Reading this metadata is the first step of all color work.

#HDR: More Than Just Brightness

High Dynamic Range (HDR) is the ability to represent a much wider range of brightness and color than SDR: pixels brighter than SDR white, deeper shadows, and more saturated colors. But HDR isn't a single standard — there are two main families you'll encounter:

HDR10. Based on the PQ transfer function (SMPTE ST 2084), colored with BT.2020 primaries. It uses static metadata: the same peak brightness value for the entire video. HDR10 is the most common HDR standard in the world, and because its metadata is static, its implementation is simple and cheap.

HLG (Hybrid Log-Gamma). Designed to be compatible with both SDR and HDR devices at once — the same signal can be displayed on SDR and HDR screens without full conversion. HLG is widely used for live broadcasts and television for this compatibility reason, and it's also why HLG sometimes looks "good enough" on SDR screens without tonemapping.

When writing metadata for HDR, you set all three at once. For HDR10 with libx265:

ffmpeg -i input.mp4 -c:v libx265 -crf 18 \
  -x265-params colorprim=bt2020:transfer=smpte2084:colormatrix=bt2020nc:master-display=G(8500,39850)B(6550,2300)R(35400,14600)WP(15635,16450)L(10000000,100):max-cll=1000,400 \
  -color_primaries bt2020 -color_trc smpte2084 -color_space bt2020nc \
  output-hevc-hdr10.mkv

Notice the two layers of settings: the -x265-params parameter writes HDR metadata into the codec stream (including master-display and max-cll, which state the target display's peak brightness), while the -color_primaries, -color_trc, and -color_space options write metadata at the container level. Modern players read both; incompleteness at either layer can make the video interpreted as plain SDR.

#HDR to SDR: The zscale and tonemap Filters

Now the most-needed part: converting HDR video to SDR so it can be watched on screens and platforms that don't support HDR. The process is called tonemapping: mapping the wide brightness range of HDR to the narrow SDR range, while trying to preserve shadow and highlight detail.

FFmpeg does it with two filters working together. The zscale filter (based on the zimg library) does color space and precision conversion, and the tonemap filter maps brightness. The working order is like sorting an archive: first move all data to a high-precision working format, do the mapping, then convert back to the target format:

ffmpeg -i input-hdr10.mkv -vf "zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=hable,zscale=t=bt709:m=bt709:r=tv,format=yuv420p" \
  -c:v libx264 -crf 18 output-sdr.mp4

Let's break down this filter chain, one of the most famous in the FFmpeg world:

• zscale=t=linear changes the PQ (HDR) transfer function to linear — color needs to be computed in a linear space for brightness mapping to work correctly.
• npl=100 states a nominal peak luminance of 100 nits, the brightness scale used as the tonemapping reference.
• format=gbrpf32le converts to 32-bit float RGB precision. This is mandatory: tonemapping done on 8-bit integers would discard detail before it ever gets mapped.
• tonemap=hable applies the tonemapping curve (a variant of an industry film curve). There are alternatives: mobius, which preserves contrast better for less extreme content, and clip, the simplest.
• zscale=t=bt709:m=bt709:r=tv returns to the BT.709 transfer with BT.709 matrix and video (limited) range.
• format=yuv420p converts to the standard SDR storage format for compatibility with almost every player.

The result: HDR video that looks normal on SDR screens, with highlights that keep their texture instead of clipping to plain white.

#Setting Color Metadata Manually

Not all color work needs tonemapping. Often the need is simply to set metadata correctly — e.g., when you receive a file whose metadata is empty or wrong. FFmpeg provides output options for this:

ffmpeg -i input.mp4 -c copy -color_primaries bt709 -color_trc bt709 -color_space bt709 out.mp4

Notice -c copy: color metadata is a stream attribute that can be rewritten without re-encoding, because it doesn't touch the video bits themselves. The three options used:

• -color_primaries — pick bt709 for HDTV, bt2020 for UHD/HDR.
• -color_trc — pick bt709 for SDR, smpte2084 for HDR10, arib-std-b67 for HLG.
• -color_space — matrix: bt601 (SD), bt709 (HD), bt2020nc (UHD non-constant luminance).

A rule you must remember: wrong metadata is more dangerous than empty metadata. If empty, devices use common default assumptions that are generally reasonable. If wrong — e.g., written BT.709 when the video is actually BT.2020 — devices will display clearly wrong colors without telling anyone. Always make sure the values you write match the video's actual data, not just "what looks cool".

#Conclusion

In episode 16 you've opened the Pandora's box of color in FFmpeg: understanding the three color metadata components (primaries, transfer function, and color matrix), distinguishing PQ-based HDR10 from hybrid HLG, writing complete HDR10 metadata for libx265, doing HDR-to-SDR tonemapping with the zscale and tonemap chain, and setting color metadata manually with -color_primaries, -color_trc, and -color_space.

Key takeaways:

• Color is determined by primaries + transfer + matrix; all three must be consistent.
• HDR10 = PQ + BT.2020; HLG = SDR/HDR compatible for broadcast.
• Tonemap HDR to SDR through 32-bit float space: linear first, map, then back.
• Wrong metadata is worse than empty metadata.
• -c copy can rewrite color metadata without re-encoding.

Color is the last "invisible" thing in a typical video pipeline. Now you're ready to build something genuinely complex: combining video, stacking text, cutting, and mixing many inputs at once in a single graph.

In the next episode 17 we'll cover Complex Filtergraphs — -filter_complex with labels, the split, overlay, xfade, and amix filters, and producing many outputs from one graph. Keep your enthusiasm up!