Learn Proxmox Backup Server - Troubleshooting & Recovery

#Introduction

Every previous episode built a healthy PBS — but in the real world, problems will come. A backup fails at 3 AM, a datastore fills up, or a PVE connection is suddenly refused. Episode 16 trains you to think like an operator: read the symptoms, find the root cause in the logs, and recover. Calm during an incident is the result of practice, not luck.

Imagine it like a mechanic reading an engine diagnostic: symptoms are only clues; the task log is the instrument panel you must read before disassembling anything. The right diagnosis saves time, data, and sanity.

#Debug: Reading Symptoms from Logs

#Detailed Task Logs

Every PBS operation is recorded as a task. Always start here:

List recent tasks

proxmox-backup-manager task list --since '6 hours ago'

Open the detailed log of a task

proxmox-backup-manager task log <UPID>

task log shows the sequential output of the operation — this is where the real error (e.g. "no space left on device" or "connection refused") is visible. The second diagnosis source is the service log:

proxmox-backup systemd log

journalctl -u proxmox-backup -n 100 --no-pager

#`proxmox-backup-manager` Commands for Diagnosis

A quick set of commands to check health:

Check basic health

proxmox-backup-manager versions
proxmox-backup-manager datastore list
df -h
pvesm status        # on the PVE side

proxmox-backup-manager versions confirms all component versions are aligned — version inconsistencies between PVE and PBS are often a source of integration problems.

#Common Cases and Their Solutions

#Full Disk

Symptoms: backup fails with "no space left on device", or the datastore status passes the watermark. Handle it in stages:

Check df -h and the datastore status (proxmox-backup-manager datastore list).
Run prune to reduce snapshots (episode 8).
Run GC to free orphaned chunks (episode 6).
If still full, add capacity: extend the LV/ZFS pool, or add a disk.

A full disk is a process problem, not an event: every "full" incident means capacity was not monitored or retention was too loose. Install monitoring (episode 20) and tighten retention.

#Garbage Collection Race

Symptoms: GC runs at the same time as a backup, a newly created snapshot is considered "missing", or GC fails/takes an extremely long time. Root cause: backup and GC competing for the datastore. PBS designs GC to consider active snapshots, but running both simultaneously still hurts performance and can raise warnings.

Solution: schedule GC in a backup-free window (episode 6), check for conflicting job schedules, and if forced, abort the running GC (proxmox-backup-manager task list → abort task) before a large backup.

#Fingerprint Mismatch

Symptoms: PVE/client refuses the connection with a "fingerprint mismatch" error after previously working. The most common cause: the PBS certificate changed (reinstall, restore from an old image, or PKI renewal).

The solution is not to "ignore" it — update the fingerprint:

Get the new fingerprint (on PBS)

proxmox-backup-manager cert info

Update the fingerprint on PVE storage

pvesm set pbs1 --fingerprint AA:BB:CC:DD:...

A fingerprint that changed without a known reason is a danger sign — verify that the PBS server really is the same server (episode 13). Never disable verification.

Warning

If the fingerprint changes and you never changed the certificate, suspect unauthorized intervention on the PBS server. Verify the server's identity before updating the fingerprint — in a backup context, "fixing it fast" without suspicion is exactly the gap attackers exploit.

#Disaster: Recovering a Datastore to a New Server

#Scenario: Primary PBS Totally Destroyed

A destroyed PBS (dead disk, lost server) must be replaced. There are two main paths:

1. Sync from a remote PBS (if off-site is running): install a new PBS at the replacement location, then create a sync job from the remote to the new datastore. All snapshots come back without touching the destroyed PVE.

2. Import/resync from old data: if the datastore disk is still alive (server failed, disk survived), attach the disk to the new PBS server and re-register the datastore path (proxmox-backup-manager datastore create store1 /path). PBS recognizes the stored chunk structure.

#Recovery Testing

A datastore recovery is not complete until it is tested:

Register the new PBS as storage in PVE (episode 4).
Restore one VM/CT to local storage (episode 5).
Start the restored result and verify services.
Verify several old snapshots (not just the newest) to ensure history is intact.

Verify synced snapshots

proxmox-backup-client snapshot list \
  --repository backup@pbs@10.0.3.30:store1

Tip

Recovery testing is a scheduled drill, not a one-time event. Create a quarterly agenda: simulate "primary PBS lost" and measure how long it takes to be fully operational again. That number is the real RTO of your backup architecture.

#Closing

Key takeaways:

Diagnosis starts from the task log (proxmox-backup-manager task list/log) and journalctl.
proxmox-backup-manager versions, datastore list, and df -h are the basic health tools.
Full disk: prune → GC → add capacity; monitor so it does not recur.
GC race: do not run GC at the same time as backups; check conflicting schedules.
Fingerprint mismatch: get the new fingerprint and update it in PVE — but investigate the cause first.
Datastore recovery: sync from remote, or re-register the old disk; always end with recovery testing.

In the next episode, episode 17, we will look at the present: PBS 4.2 & the latest features — the Debian 13.4 Trixie base, Proxmox kernel 7.0, ZFS 2.4, and S3 object storage support, plus integration with PVE 8.x/9.x and the PBS release journey from 3.x to 4.2. Upgrades and new features await!