Learn Rsync - Server Migration & Directory Sync

#Introduction

All the material so far leads to a moment every sysadmin has experienced: moving servers. Whether it's a hardware upgrade, cloud migration, or data center consolidation — rsync is the primary tool for moving data without long downtime.

Episode 11 ties it all together: how to migrate important system directories (home, var/www, database dumps) with high fidelity via two often-forgotten flags — --numeric-ids and -H — and then an honest look at what rsync can't do: two-way synchronization.

#Server Migration: The Scenario

A common scenario: migrating from an old server (old.example.com) to a new one (new.example.com) while minimizing downtime. A two-phase strategy:

1. Initial phase — copy the main data while services are still running (can take hours, speed doesn't matter, use the throttle from episode 12).
2. Final phase — stop the services, run rsync once more (only picks up the delta since the initial phase, a matter of minutes), then cut over DNS/traffic.

rsync -avHAX --numeric-ids -e ssh \
  /home/ /var/www/ root@new.example.com:/migrate-in/

That short final phase is why rsync is perfect for migration: delta-transfer makes the "final copy" take minutes, not hours.

#--numeric-ids: Numeric IDs

By default, rsync maps owners/groups based on names. On different machines, a username can point to a different UID — the result is files migrating with the wrong owner.

--numeric-ids disables name mapping and stores the numeric UID/GID as-is:

rsync -av --numeric-ids /home/ root@new.example.com:/home/

#-H: Hardlinks

-H (--hard-links) preserves the hardlink relationships between files — something -a doesn't do:

rsync -avH /var/www/ root@new.example.com:/var/www/

Without -H, two files that share the same inode at the source get copied as two separate copies at the destination — duplicates that bloat storage. With -H, the relationship is preserved. For directories that use hardlinks (like --link-dest-based backups, alternative Git setups, or media libraries), this flag is mandatory. The cost: rsync must track all files to match inodes, so it's a bit slower and uses more memory.

#A Complete Migration Example

Migrating a database dump (e.g. PostgreSQL) — dump first, then rsync:

pg_dump -Fc mydb > /var/backups/mydb.dump
rsync -avhP --numeric-ids /var/backups/ root@new.example.com:/var/backups/

The same pattern applies to all system data:

For directories that change constantly (live databases), don't rsync the data directory directly while the server is running — it can be inconsistent. A dump is the safe route; a consistent alternative for DBs is discussed in episode 18.

#Post-Migration

After the data moves, verify before cut-over:

du -s /home/ /var/www/
ssh root@new.example.com "du -s /home/ /var/www/"

Comparing du -s gives a quick signal: wildly different sizes mean something was left behind (or hardlinks weren't preserved). For strict per-file verification, run -n -i rsync (episode 16) from the new server against the old source — if it comes up clean, the migration is done.

#Two-Way Sync: Rsync's Limitation

This is a boundary you need to understand: rsync is one-way. It always copies from SRC to DEST, and changes at DEST never flow back to SRC. If you run two opposite rsyncs alternately, the result: files changed on both sides overwrite each other — the newest version wins and conflicts silently vanish.

#Unison: The Two-Way Solution

Unison is a two-way synchronization tool based on rsync-style delta that tracks changes on both sides and handles conflicts explicitly:

unison /home/data ssh://user@otherhost//home/data

If a file is changed on both sides, unison doesn't guess — it shows the conflict and asks for a decision. Unison stores profiles in ~/.unison/ and can run non-interactively for non-conflicting cases. That's the right choice when you genuinely need two-way sync (for example, a laptop and a server both actively used).

#When to Use Which

#Closing

In this episode you've mastered server migration and understood rsync's limits.

Key takeaways:

• Two-phase migration: an initial copy while services run + a short delta copy before cut-over.
• --numeric-ids preserves UID/GID; create identical users at the destination first.
• -H preserves hardlink relationships — mandatory for hardlink-heavy directories.
• Dump the database and rsync the dump; don't rsync a live DB's data directory.
• Rsync is one-way; for two-way use unison (conflict detection), not two rsyncs overwriting each other.

In episode 12 we control resources: bandwidth & resource control — --bwlimit=RATE, --max-size/--min-size, nice/ionice, the effect of -z compression on throughput, and socket buffer tuning. See you in episode 12!