Learn Velero - Performance & Data Management

#Introduction

Throughout this series our focus has been: is the backup correct? Episode 20 adds a second, equally important question: how fast is the backup? A backup taking 10 hours when the RPO demands 1 hour is not a backup that keeps its promise. And a slow restore while the cluster is on fire is the definition of DR failure.

Think of the backup path as a water pipe: pipe capacity (parallelism), pressure (resource limits), and connection points (node-agent) all determine the flow rate. This episode teaches you to read and enlarge all three.

#Backup Parallelism

#Item Operation Parallelism

--item-operation-parallelism (at install) controls how many resources are processed at once within a single backup. The default is fairly conservative; for large clusters, raise it:

Install with higher parallelism

velero install \
  --provider aws \
  --plugins velero/velero-plugin-for-aws:v1.14.0 \
  --bucket velero \
  --secret-file ./credentials-velero \
  --backup-location-config region=us-east-1 \
  --item-operation-parallelism 10

"Conservative" here isn't an excuse; it's a balance: high parallelism speeds up backups but loads the API server and object storage.

#Parallel Files Upload

For kopia file-level backups, two parameters determine throughput when reading data from volumes:

--parallel-files-upload — the number of files uploaded concurrently.
--parallel-files-download — for restore.

Raise file upload parallelism

velero backup create perf-test --include-namespaces app \
  --default-volumes-to-fs-backup --parallel-files-upload 16

Tip

Special note for Velero 1.18: Velero uses Golang 1.25, which respects pod CPU limits (container-aware GOMAXPROCS). If you set a small CPU limit on node-agent pods, throughput drops unexpectedly. Raise the CPU limit, or adjust parallel-files-upload to the existing limit — the 1.18 docs explicitly mention this trade-off.

#Velero Pod Resource Limits

#Server Pod

The velero pod must be given enough resources to process backups of many resources:

Do the sizing: start from requests of 512m CPU / 512Mi, and limits of 2 CPU / 2Gi — then monitor. A pod running out of memory restarts mid-backup (impact: backup fails).

#Node-Agent Scaling

The node-agent DaemonSet runs on every node with one pod per node. File-level backup load spreads naturally — but one pod can handle many volumes. The bottleneck usually isn't the number of pods, but resources per pod:

Node-agent resources

resources:
  requests:
    cpu: 500m
    memory: 512Mi
  limits:
    cpu: 2
    memory: 2Gi

Scaling "vertically" (raising resources) is more common than horizontal — node-agent is already one per node. Monitor per-node usage to know which one needs a bump.

#Analyzing Backup Sizes

#Reading Sizes from Object Storage

Use your object storage client (mc/awscli) to view the size per backup:

Backup sizes in MinIO

mc du --recursive local/velero/backups/

Large data almost always comes from the kopia/restic volume repository. Watching size growth tells you when to trim scope or extend retention.

#Monitoring Duration and Status

Combine size with backup duration (from velero backup describe or the CRD metadata) to calculate actual throughput: size ÷ duration. This number is your baseline for setting RTO expectations.

#Exporting Backups to Other Formats

#Why Export

Common reasons: audit (backup evidence for auditors), migration to another system, or offline archiving. Velero backups in object storage aren't exactly human-readable files — but their manifests can be extracted.

#Accessing Manifests

Each backup's manifests are stored as JSON in the bucket (<backup>/velero-backup.json and the resources/ folder):

Download a backup manifest

aws s3 cp s3://velero/backups/my-backup/velero-backup.json ./
aws s3 sync s3://velero/backups/my-backup/resources/ ./resources/

The JSON manifests can be converted to YAML for review or re-import. For full backup replication between buckets (e.g. to an archive bucket in another region), use object storage synchronization or cross-region replication — more efficient than restore-and-rebackup.

Note

File-level backups (kopia) can't be "converted" to another format directly — their repository is proprietary. What's portable is the restore result. If you need data out of Velero, restore to a staging cluster/namespace first, then export from there.

#Closing

Key takeaways:

--item-operation-parallelism speeds up resource processing; raise it gradually and monitor the load.
Kopia has --parallel-files-upload/--parallel-files-download for file-level throughput.
Golang 1.25 in Velero 1.18 makes pod CPU limits affect throughput — adjust both.
Node-agent scales vertically (resources per pod), not horizontally.
Analyze size + duration for a throughput baseline; export manifests via object storage for audit.

In episode 21 next, we look ahead and into the community: Roadmap & Community — the kopia-default focus, CSI maturity, multi-version K8s support, and the community channels: GitHub velero-io/velero, Slack #velero, and the velero.io docs.