Learn pgBackRest - WAL Archiving & Archive Command

#Introduction

The full, differential, and incremental backups in episode 5 give you "snapshots." But a snapshot is only useful up to a point: what happens to data that changes after the last snapshot? The answer is in WAL archiving — the mechanism that lets pgBackRest recover the database to any point in time, not just to a backup point.

This is the difference between a "snapshot-only" backup and a "point-in-time" backup. In episode 6 we dissect how archive_mode, archive_command, and its counterpart archive-get work — plus the async option to reduce the impact of latency on heavy write workloads.

#WAL: The Log That Holds the Key to Recovery

Before discussing archiving, understand what's being archived. PostgreSQL writes WAL (Write-Ahead Log) for every transaction before writing to the data files. WAL is rotated in segments (16 MB by default) — and it's these segments that get archived. A complete WAL chain lets PostgreSQL "replay" the change history to reach a state at a specific point in time.

#Enabling WAL Archiving

#Parameters in postgresql.conf

Two parameters must be enabled for archiving to run:

archive_mode = on
archive_command = 'pgbackrest --stanza=main archive-push %p'
archive_timeout = 60

• archive_mode = on — enables archiving (read at startup; needs a restart).
• archive_command — the command run every time a WAL segment is rotated.
• archive_timeout = 60 — forces WAL rotation every 60 seconds when the server is idle, so the repository keeps receiving WAL periodically (reducing data loss on crash).

Notice %p: this is the full path of the WAL being archived. pgBackRest copies it into the repository, compresses it, and records metadata.

#What Happens on Restart

After changing postgresql.conf, restart PostgreSQL:

sudo systemctl restart postgresql

Check that archiving is active:

SELECT * FROM pg_stat_archiver;

A growing archived_count column means WAL is being archived to the repository successfully. This is a very useful quick check in episode 15 when WAL gets stuck.

#archive-push and archive-get: The Two WAL Directions

#archive-push: WAL In

When PostgreSQL calls archive-push %p, pgBackRest stores the WAL into the stanza repository. If it fails (for example the repository is full), archive_command returns a failure status and PostgreSQL retries until it succeeds — this is why a full repository is a serious problem that must be monitored (episode 14).

#archive-get: WAL Out during Restore

When restore is run, pgBackRest needs to "replay" WAL. For that, PostgreSQL needs a restore_command — and here's a big advantage: pgBackRest injects the restore_command automatically during restore. You don't have to write it manually. This WAL direction is called archive-get:

pgbackrest --stanza=main archive-get %f "%p"

%f is the name of the requested WAL segment, %p is where it's written when replayed. Because it's configured automatically, you only need to focus on archive-push on the primary side.

#Validating with pgbackrest check

pgbackrest check tests both directions at once: it archives a test file (archive-push) then retrieves it back (archive-get). If either direction fails, the check output shows exactly where the chain breaks:

P00   INFO: check command end: completed successfully

If archive-push fails because WAL isn't active yet, check shows a clear error — usually pointing you back to the archive_mode or archive_command above.

#Async archive-push: Reducing Latency

#The Problem with Synchronous Mode

Every time a WAL segment is rotated, PostgreSQL waits for archive_command to finish before marking the WAL "archived." On high-write workloads, network/compression latency to the repository can slow down WAL rotation and hurt performance.

#The Solution: Async

Async mode separates the archiving process: archive_command only writes the WAL to a local spool (fast), while a background pgBackRest process sends it to the repository later. The result: WAL archiving is no longer a bottleneck.

[global]
repo1-path = /var/lib/pgbackrest
async-archiving = y
spool-path = /var/spool/pgbackrest

A spool directory owned by postgres is needed:

sudo mkdir -p /var/spool/pgbackrest
sudo chown -R postgres:postgres /var/spool/pgbackrest

#Monitoring WAL in the Repository

Check how much WAL has been archived per stanza:

sudo -u postgres pgbackrest info

The wal archive min/max line shows the range of stored WAL segments. A steadily advancing range means archiving is running normally. If the range stops, something is wrong — this is the first alarm you should set up in episode 14.

#Conclusion

Key takeaways:

• archive_mode=on + archive_command='pgbackrest --stanza=main archive-push %p' is the bridge of WAL into the repository.
• archive-get (restore_command) is injected automatically during restore — the WAL-out direction.
• pgbackrest check validates both archive-push and archive-get at once.
• WAL that fails to archive is retried by PostgreSQL until it succeeds — monitor so it doesn't pile up.
• Async archiving (spool-path + async-archiving) reduces latency for high-write workloads.

In the next episode we'll speed everything up: parallelism and compression — how process-max works, its impact on backup/restore with many CPUs, choosing compress-type=zst (zstd default) with compression levels, up to delta backup that only copies the changed parts of files. Your first backups will feel much faster!