Learn Borg Backup - Compression: zstd, lz4, zlib

#Introduction

Dedup discards identical data; compression shrinks the different data. In episode 4 you saw --stats display a "Compression ratio" — now we master the variables behind it. Compression is a balancing game: the higher the ratio, the heavier the CPU load and the slower the backup. This episode helps you pick the balance point for your own workloads.

#Borg's Compression Algorithms

#lz4: Fast, Almost Free

lz4 is Borg's default and the most CPU-efficient — ideal for data that does not compress well (media, encrypted files) or hosts with busy CPUs. Its compression ratio is modest, but its speed is outstanding.

Backup with lz4 (default)

borg create --compression lz4 --stats /backup/borg::"{hostname}-{now}" /home

#zstd: The Sweet Spot

zstd (Zstandard) is the modern "sweet spot" choice: high ratio with still-excellent speed. Levels 1-19; zstd,10 is commonly used in production for its comfortable balance.

Backup with zstd level 10

borg create --compression zstd,10 --stats /backup/borg::"{hostname}-{now}" /home

#zlib and lzma: High Ratio, Heavy CPU

zlib (levels 0-9): good ratio, moderate speed.
lzma (levels 0-9): the best ratio in Borg, but slow and CPU-hungry. Only for very large data that rarely changes.

#Comparing at a Glance

Compression algorithm trade-offs

+--------+------------+----------------+------------------+-------------+
| Algo   | Speed      | Ratio          | CPU load         | Example     |
+--------+------------+----------------+------------------+-------------+
| lz4    | very fast  | low            | very low         | default     |
| zstd,3 | fast       | good           | low              | production  |
| zstd,10| moderate   | very good      | moderate         | storage     |
| zlib,6 | moderate   | good           | moderate-high    | compatibility|
| lzma,6 | slow       | best           | high             | big archives|
+--------+------------+----------------+------------------+-------------+

Tip

Zstd beats zlib in almost every aspect: better ratio at higher speed. Choose zlib only if you need compatibility with non-Borg tools that read chunks — which is actually rare. zstd,10 is a sensible production default.

#Choosing the Right Compression

#Workload Considerations

Already-compressed data (JPEG/PNG images, video, .zip archives, encrypted files): compression barely helps. Use lz4 so you do not waste CPU.
Text & log data: compression pays off greatly. zstd,10 (or higher) is worth using.
Busy hosts during the day: night backups can use high zstd; daytime backups, consider lz4.
Expensive storage (cloud): zstd,19 or lzma if time is not a concern.

#Try and Measure

Do not guess — measure. Run trial backups with different compression and compare --stats:

Benchmark two compression modes

time borg create --compression lz4 --stats /backup/borg::bench-lz4 /var/lib/data
time borg create --compression zstd,10 --stats /backup/borg::bench-zstd /var/lib/data

The two time lines give you the duration; --stats gives you the sizes. Pick the point that makes the most sense for your priorities (speed vs space).

#Verification: Reading the Compression Ratio

borg info is the referee for all these decisions:

View the compression ratio

borg info /backup/borg::bench-zstd

The relevant part of the output:

Example borg info output

Original size: 12.6 GB
Deduplicated size: 4.2 GB
Compression ratio: 3.00x

Original size: the total source size.
Deduplicated size: the actual size in the repository after dedup + compression.
Compression ratio: the combined effect of dedup and compression (computed over the data "about to be compressed"). The higher, the more efficient.

Note

The "Compression ratio" in borg info includes the effect of deduplication. If you want to know the pure compression contribution, compare the --stats of two archives containing identical data created with different compression — the difference shows the algorithm's contribution.

#Common Pitfalls

Compressing already-encrypted data: encrypt first, then compress? Do not — encrypted data is nearly incompressible. Borg handles the internal ordering correctly (compress then encrypt), so do not reverse it.
Level too high: zstd,19 or lzma on large daily backups can stretch them out for hours. Measure first, and raise the level only if it makes a real difference in space.
Switching compression inconsistently: compression is a per-create property, so you can mix. But mixing makes --stats comparisons confusing — pick one default and stick to it.

#Closing

lz4 is fast and CPU-efficient; zstd,10 is a good production balance; zlib/lzma for the highest ratio at a CPU cost.
Already-compressed data does not need re-compression — choose lz4.
Measure with time + --stats before deciding on a level.
borg info shows the original size, the post-dedup size, and the compression ratio.
Dedup and compression work in sequence: dedup first, compression follows.

In episode 7 we lock up the data: encryption & key management — the difference between repokey, keyfile, and authenticated modes, where the key is stored, how to back up and import keys, and strategies for storing the passphrase so your repository is never locked forever.