Learn FFmpeg - API & Programmatic Use

#Introduction

For 19 episodes you've typed FFmpeg in the terminal. But behind the ffmpeg binary there are dozens of libraries — libavcodec, libavformat, libavfilter, libavutil, libswscale, libswresample — the exact same ones used by VLC, OBS, and Chrome. These libraries are why FFmpeg is called the "multimedia Swiss Army knife".

This episode covers the programmatic side: (1) calling the libav* libraries directly from C code with the official API, (2) using bindings in other languages like Go and Python, and (3) automating the ffmpeg binary via CLI wrappers like ffmpeg-python in Python and fluent-ffmpeg in Node. You'll see that understanding the architecture from episode 1 actually has real practical value. Before starting, check which libraries are compiled into your binary with ffmpeg -buildconf.

#Architecture: Small Binary, Big Libraries

One thing that often confuses beginners: ffmpeg, ffprobe, and ffplay are just small programs that combine the libraries underneath them. That's why a single binary can do so much — it doesn't implement anything itself, it orchestrates mature libraries.

The practical consequence: your application doesn't need to call the ffmpeg binary if it wants maximum speed and full control — just link against libavformat and libavcodec and call their APIs.

#Official API from C

C is the native language of this API, and all other bindings are just wrappers on top of it. The basic flow is almost always the same: open input, find stream, open decoder, read packet, decode frame.

#include <libavformat/avformat.h>
#include <libavcodec/avcodec.h>
#include <stdio.h>
 
int main(int argc, char **argv) {
    if (argc < 2) return 1;
 
    AVFormatContext *fmt_ctx = NULL;
    if (avformat_open_input(&fmt_ctx, argv[1], NULL, NULL) < 0) {
        fprintf(stderr, "tidak bisa membuka input\n");
        return 1;
    }
    if (avformat_find_stream_info(fmt_ctx, NULL) < 0) {
        fprintf(stderr, "tidak bisa membaca stream info\n");
        return 1;
    }
 
    int stream_index = av_find_best_stream(fmt_ctx, AVMEDIA_TYPE_VIDEO, -1, -1, NULL, 0);
    if (stream_index < 0) {
        fprintf(stderr, "tidak ada stream video\n");
        return 1;
    }
    av_dump_format(fmt_ctx, 0, argv[1], 0);
    printf("stream video ada di index %d\n", stream_index);
 
    avformat_close_input(&fmt_ctx);
    return 0;
}

Notice the pattern — this is what you'll see in every libav tutorial:

1. Allocate and open with avformat_open_input — FFmpeg handles the memory allocation, then avformat_find_stream_info reads the stream list.
2. Always check return codes — every function returns a negative value for errors. This is the first and most important pattern of the C API.
3. Close resources with avformat_close_input — memory leaks in C are severe bugs.

Compile the example above with pkg-config pointing to the installed FFmpeg libraries:

cc probe.c $(pkg-config --cflags --libs libavformat libavcodec) -o probe

This "open → find → read → close" pattern appears again at every stage of decoding, encoding, filtering, and muxing. Master the pattern once, and you can read any API in the libav* family.

#Bindings in Go and Python

Writing pure C for every project isn't always practical. The community provides bindings so the FFmpeg libraries can be used from safer, more productive languages.

#Python: PyAV

PyAV is a fairly mature Python binding — it wraps libav* and retains FFmpeg's API style. You work with Container, Stream, and Frame objects, not command strings.

import av
 
container = av.open("input.mp4")
stream = container.streams.video[0]
for frame in container.decode(stream):
    print(f"frame {frame.index}: {frame.width}x{frame.height} @ {frame.pts}")
    img = frame.to_image()
    img.save("thumbnail.png")
    break

Why does PyAV excel in cases like this? Because frames become Python objects directly usable (e.g., as a PIL.Image) — without parsing stderr and IPC as when calling the binary.

#Go

In Go, projects like goav wrap the C-style bindings. Since Go calls C through cgo, the basic pattern is identical to the C API — the difference is that AVFormatContext is handled with finalizers and safer pointers:

package main
 
import (
    "fmt"
    "os"
 
    goav "github.com/giorgisio/goav/avformat"
)
 
func main() {
    if len(os.Args) < 2 {
        fmt.Println("usage: probe <file>")
        os.Exit(1)
    }
    ctx := goav.AvformatAllocContext()
    if err := ctx.AvformatOpenInput(os.Args[1], nil, nil); err != nil {
        fmt.Println("gagal buka input:", err)
        os.Exit(1)
    }
    defer ctx.AvformatCloseInput()
    fmt.Printf("durasi: %d detik\n", ctx.Duration()/1000000)
}

Go bindings use defer to close the context — that's where memory safety is more comfortable than C. The API structure stays the same, just wrapped in Go idioms.

#CLI Automation: ffmpeg-python

Not every need requires linking the library. If your application just runs a certain transcode, automating the ffmpeg binary via a wrapper is the simplest and most release-proof approach — because you use whatever binary is installed on the server. ffmpeg-python builds commands in a builder style: you compose a graph, then execute it.

import ffmpeg
 
input_ = ffmpeg.input("input.mp4")
video = input_.video.filter("scale", 1280, 720)
stream = ffmpeg.output(
    video, input_.audio,
    "output.mp4",
    vcodec="libx264", crf=23, preset="fast",
    acodec="aac",
)
stdout, stderr = stream.run(capture_stdout=True, capture_stderr=True)

The code above is equivalent to the transcode command you've learned since episode 4, but written as objects that can be tested and parameterized from program logic.

#The Correct Error Handling Pattern

This is where many wrapper users trip up. By default stream.run() throws an ffmpeg.Error exception when the process fails — and that exception object carries the stderr holding the real reason:

import ffmpeg
 
try:
    ffmpeg.input("missing.mp4").output("out.mp4").run(
        capture_stdout=True, capture_stderr=True
    )
except ffmpeg.Error as e:
    print("stderr:", e.stderr.decode())

Practical error handling rules across all wrappers (ffmpeg-python, fluent-ffmpeg, Node child_process):

1. Always catch process failures — don't let exceptions circulate without context.
2. Read stderr, not just the exit code. FFmpeg errors are almost always explicit: No such file, Invalid data, Permission denied. Also log the failing command so it's easy to reproduce.
3. Don't parse output for important data — for metadata and duration, use ffprobe with JSON output, not text parsing.

ffprobe -v error -print_format json -show_format input.mp4

JSON output is a stable contract for programming. Parsing stderr text is a source of unnecessary bugs — never build a parser on top of output designed for humans to read.

#Integration in Node Applications

In the Node ecosystem, fluent-ffmpeg is the most popular wrapper. It runs on top of the ffmpeg binary and offers a chained API with events for progress and errors.

const ffmpeg = require("fluent-ffmpeg");
 
ffmpeg("input.mp4")
  .output("output.mkv")
  .videoCodec("libsvtav1")
  .size("1280x720")
  .audioCodec("copy")
  .on("start", (cmd) => console.log("perintah:", cmd))
  .on("progress", (p) => {
    if (p.percent) console.log("proses:", p.percent.toFixed(1) + "%");
  })
  .on("end", () => console.log("selesai"))
  .on("error", (err) => {
    console.error("gagal:", err.message);
    if (err.stderr) console.error(err.stderr);
  })
  .run();

The same pattern appears again: call run() as the final execution, handle errors via events, and read stderr for diagnosis. The progress event can even be used to build a progress bar — something far harder when calling the binary raw.

#Conclusion

Episode 20 opened the door to integration:

• FFmpeg's architecture is split into libav* libraries and the thin binaries on top.
• The C API follows a consistent pattern: open, find stream, decode/encode, close — and always check return codes.
• Python (PyAV) and Go bindings give access to the library with safer language idioms.
• CLI wrappers like ffmpeg-python and fluent-ffmpeg automate the binary for transcode needs.
• True error handling starts with reading stderr and using ffprobe JSON for structured data.

You can now program FFmpeg — but where is it heading? In episode 21 we step back briefly to review modern features and the roadmap: the 8.0 "Huffman" and 8.1 "Hoare" releases, 9.0 "Lei", and the future direction of VVC, AV1, and GPU utilization. See you in episode 21.