Learn Rsync - Bandwidth & Resource Control

#Introduction

Rsync can be an "impolite neighbor": during a large backup, it can monopolize network bandwidth or flood the disk with I/O — slowing down your production applications. Episode 12 teaches resource etiquette: how to limit, prioritize, and measure rsync so it cooperates with other workloads instead of fighting them.

#--bwlimit: Limiting Bandwidth

--bwlimit=RATE caps the transfer speed in KiB/s:

Limit to 1 MiB/s

rsync -avh --bwlimit=1024 /home/data/ root@backup:/backup/

Meaning: the transfer runs at a maximum of 1 MiB/second — enough for a nightly backup without killing other applications' connections. This is very useful when a backup runs alongside production traffic (the migration in episode 11, for example). Reasonable values: --bwlimit=1024 up to --bwlimit=5120 (5 MiB/s) for a 100 Mbps connection.

Note

--bwlimit in rsync 3.x refers to KiB/s (1 KiB = 1024 bytes), not decimal KB. If your ISP says "100 Mbps", that transfer bandwidth is about 12,500 KiB/s — set --bwlimit well below that if you want to leave room for other applications.

#--max-size and --min-size

Filtering by file size — useful for focused backups:

Only files from 1 KB to 100 MB

rsync -avh --min-size=1K --max-size=100M src/ dest/

Flag	Function	Example
`--min-size=1K`	Skip files smaller than 1 KiB	Avoid empty log files
`--max-size=100M`	Skip files larger than 100 MiB	Avoid ISO/image files that don't need backing up

Supported suffixes: K (KiB), M (MiB), G (GiB). Combining both produces a precise size filter — handy for the "back up everything except giant videos" case.

#nice and ionice: CPU and Disk Priority

Besides the network, rsync also consumes CPU (compression, checksums) and disk (reading/writing). Lower its priority with nice and ionice:

Run rsync at low priority

nice -n 19 ionice -c3 rsync -avh /home/data/ /backup/

nice -n 19 — lowest CPU priority; other processes are scheduled first.
ionice -c3 — idle class; disk I/O is only served when nothing else needs the disk.

This combination lets large backups run on a production server without being noticed. It's a recommended habit for cron backups (episode 10).

Tip

For production workloads, a rule of thumb: limit the two most constrained resources. Narrow network → --bwlimit. Busy disk → ionice -c3. Busy CPU → nice -n 19. Often one or two is enough — you don't need all of them.

#-z: Compression vs Throughput

-z compresses data — but compression is a trade-off between CPU and bandwidth:

Slow networks (WAN, < 10 Mbps): compression wins big. Data shrinks, fewer bytes get sent, and completion is faster despite the extra CPU.
Fast networks (LAN, SSD-to-SSD): compression often slows things down. There's enough bandwidth for raw data, and CPU/disk becomes an unnecessary new bottleneck.
Already-compressed files (.zip, .jpg, .mp4, Docker images): -z barely shrinks anything — pointless.

Test the effect of -z

rsync -avh --stats src/ dest-nocompress/
rsync -avhz --stats src/ dest-compress/

Compare the sent and total size lines from both --stats outputs — you'll see how many bytes are saved vs how much time is added.

#Socket Buffer Tuning

Socket buffers affect throughput on links with a large bandwidth-delay product (cross-continent connections). The TCP buffer size is set by the system (sysctl net.core.rmem_max), and rsync can request it via --sockopts:

Larger socket buffers

rsync -avh --sockopts=SO_SNDBUF=262144,SO_RCVBUF=262144 \
  /home/data/ root@remote:/backup/

On long-distance connections, small buffers make the sender "wait for ACKs" so bandwidth isn't fully used. Enlarging the buffer helps — but only if the network path and kernel support it (check sysctl net.ipv4.tcp_rmem). For connections inside the same data center, this tuning is rarely needed.

#Benchmark Before Tuning

Don't tune based on guesses. Measure the physical limits first:

Basic bandwidth benchmark

iperf3 -c remote-host
rsync -avh --stats /large-file.bin remote:/tmp/

iperf3 tells you the maximum TCP throughput; the sent line in --stats gives rsync's actual throughput. If rsync is far below iperf3, there's room to tune — if it's already close, the problem isn't rsync but the network path.

#Closing

In this episode you've mastered resource control.

Key takeaways:

--bwlimit=RATE caps KiB/s; mandatory when backups run alongside production.
--min-size/--max-size filter by file size.
nice -n 19 + ionice -c3 makes rsync "invisible" on busy servers.
-z pays off on WAN, hurts on LAN and compressed files; measure with --stats.
Socket buffers (--sockopts) help on cross-continent links; benchmark with iperf3 first.

In episode 13 we turn to the dark side: security & CVE 3.4.x — the 33 vulnerabilities rsync 3.4.0 closed including 2 RCEs (CVE-2024-12084/12085), the heap overflow, the --safe-links fix, the 3.4.3/3.4.4 patches, and best practices for safe usage. See you in episode 13!