Learn Rsync - Include/Exclude & Filter Rules

#Introduction

The flags in episode 5 control how rsync copies; episode 6 controls what gets copied. Filters are the difference between a smart backup and one that hoards junk: carrying a 2 GB node_modules to the backup server every night is a waste of bandwidth, time, and storage.

Episode 6 covers --exclude, --include, --exclude-from, and the nuances of path patterns — then we assemble a real filter for a development project.

#Why Filters Matter

Imagine backing up a project directory that contains:

.git/ — repo history, can be re-cloned, no need to back up.
node_modules/ — can be reinstalled with npm install.
.cache/, *.log, *.tmp — temporary data that keeps changing.

Without filters, your backup is full of data that can be rebuilt. With filters, the backup becomes small, fast, and focused on data that truly can't be recreated: source code, configuration, and assets.

#--exclude: Excluding Files

The simplest way to exclude a pattern:

Exclude basic patterns

rsync -avh --exclude='*.tmp' --exclude='*.log' src/ dest/

--exclude='*.tmp' rejects every .tmp file; *.log rejects log files. Patterns are matched against the file name and the relative path — the more specific the pattern, the narrower its reach.

Note

Always wrap patterns in quotes ('*.tmp'). Without quotes, the shell may expand * before rsync sees it — and the result won't be what you expect.

#--include: Exclude Everything, Then Select Some

--include is rarely used alone. The classic pattern: exclude everything first, then include what you want to keep:

Only back up .git and config files

rsync -avh --include='*/' --include='.git/**' --include='*.conf' --exclude='*' src/ dest/

Read it in order from the top: include all directories (*/), include everything inside .git, include *.conf files, then reject everything else (*). Because rsync evaluates rules in order, and the first matching rule wins, the include-then-exclude order must be maintained.

#Rule Evaluation Order

Rsync filter rules are evaluated one by one in written order. Here are the consequences:

--include must be written before --exclude='*' to work.
If --exclude='*' appears first, all files are already rejected before --include gets a chance.

Common pattern: include first to "rescue", exclude last to "reject the rest".

#--exclude-from: Filters in a File

When the filter list gets long, store it in a file:

Filter file

cat > ~/.rsync-excludes <<'EOF'
.git/
node_modules/
.cache/
*.log
*.tmp
*.swp
EOF

Use the filter file

rsync -avh --exclude-from=~/.rsync-excludes src/ dest/

A filter file is tidier for long lists, can be versioned in a repo, and reused across scripts. --include-from is also available and works the same way.

#Patterns: /dir/ vs dir/

The most confusing nuance of path patterns — note the difference:

Pattern	Meaning
`dir/`	Matches a directory named `dir` anywhere in the hierarchy
`/dir/`	Matches `dir` only at the root of the synchronized hierarchy
`dir`	Matches a file OR directory named `dir` anywhere
`/dir`	Matches the `dir` entry only at the root
`**`	Matches any number of path segments (zero or more)
`*`	Matches a single segment only (doesn't cross `/`)

Real example: a project has public/uploads and src/static/uploads directories. You want to exclude both → use uploads/. You only want to exclude public/uploads → use /public/uploads.

Tip

Safe habit: to exclude "any folder with this name across the whole tree", use name/. To exclude one specific location, start the pattern with /. One leading slash in a pattern changes its reach dramatically.

#Real Case: Backing Up a Development Project

Assemble a complete filter for a Node.js project:

Back up a Node.js project without junk

rsync -avh \
  --exclude='.git/' \
  --exclude='node_modules/' \
  --exclude='.cache/' \
  --exclude='dist/' \
  --exclude='.env.local' \
  --exclude='*.log' \
  --exclude='*.tmp' \
  ~/project/ backup/project/

Each pattern has a reason: .git and node_modules can be rebuilt; dist is a build output; .env.local contains secrets (never back it up somewhere it shouldn't go); logs and tmp files have no historical value. The result: a backup containing only source and assets — far smaller and faster.

Warning

Note the .env.local above. Secret files are often forgotten in exclusions. Before pushing a backup, run rsync -n -i (episode 16) and check whether any .env file is being sent. Spreading secrets through backups is as dangerous as committing them to a public repo.

#Syncing Excluded Directories

A common combination: mirror with delete, but keep some destination directories that don't exist at the source. Use --delete-excluded carefully — this flag deletes excluded files at the destination. If you don't want that, leave the default: excluded files at the destination are kept.

#Closing

In this episode you've mastered rsync filters.

Key takeaways:

--exclude='pattern' rejects files; --include rescues files from a global exclude.
Sequential evaluation: include first, exclude last — the first matching rule wins.
--exclude-from=file for tidy, versionable filter lists.
/dir/ = only at the root; dir/ = anywhere; always quote your patterns.
Filter .git, node_modules, cache, and secrets before production backups.

In episode 7 we cover batch mode & partial transfer: --write-batch/--read-batch for mass transfers to many hosts, --partial/--partial-dir for resuming, and --timeout for unstable networks. See you in episode 7!