Learning Restic - Core Concepts & Key Architecture

#Introduction

In episode 1 you understood why restic exists. Now it is time to break down how it works. This episode is the foundation for the entire series: once you understand repository, snapshot, and chunking, all the commands in episodes 3–22 are just variations of these three concepts.

#The Three-Pillar Architecture

#Repository

A repository is a collection of encrypted data that holds all snapshots. It can live in a local directory, on an SFTP server, in an S3 bucket, or on a rest-server. Inside it, data is split into small encrypted blobs — nothing can be read without the key.

#Snapshot

A snapshot is the state of a directory (or set of directories) at a specific point in time. Think of it like a full-resolution photo: it records the file contents, metadata, and the relationships between files at the moment it was taken. A snapshot does not store physical copies of files — it stores references to the blobs that make up each file's contents. This is what makes restic snapshots efficient and immutable.

#Content-Defined Chunking

For deduplication to work, files are split into chunks using a content-defined chunking algorithm: chunk boundaries are determined by the data content (via a rolling hash), not by byte position. As a result:

• Identical files always produce identical chunks → stored once.
• The same file with a single byte prepended produces only one new chunk → the rest stays the same.
• Moved files are still deduplicated because their chunks are identical.

Each chunk is then compressed, encrypted, and stored as a blob in the repository.

#Repository Structure

Let's peek inside an initialized repository:

/backup/restic/
├── config
├── data/
├── index/
├── keys/
└── snapshots/

• config — repository version and cryptographic parameters (for example: version: 2).
• data/ — encrypted blobs, split across many sub-directories.
• index/ — the blob catalog: which file contains which blob.
• keys/ — encrypted keys (one per repository password).
• snapshots/ — snapshot metadata stored as small files.

#System Components

#The restic Binary

A single program (single binary) with no runtime dependencies — one of the reasons restic is easy to install, copy between servers, and containerize.

#Backend

An abstract storage layer: local, SFTP, S3-compatible, B2, Azure, GCS, and REST server. The same commands work on every backend — only the repository URL (-r) changes. Episode 7 covers this thoroughly.

#Repository Key (Password)

The repository password is used to decrypt the key that holds the master key. Without this password the repository is impossible to access — which is why storing it safely matters (episodes 3 and 13).

#Core Command Set

restic init      # create a new repository
restic backup    # save a new snapshot
restic restore   # restore files from a snapshot
restic check     # verify repository integrity
restic forget    # remove snapshots according to policy
restic prune     # remove unused blobs

#Backup Data Flow

When restic backup /data is run:

1. Scan: restic reads the directory and builds a file tree along with metadata.
2. Chunking: each file is split into content-based chunks.
3. Deduplication: chunks already present in the repository are skipped.
4. Compression & Encryption: new chunks are compressed (zstd) then encrypted (AES-256).
5. Indexing: new blobs are recorded in the index.
6. Snapshot: the file-tree metadata plus the blob list is stored as a snapshot.

Note point 3: this is why the second backup onward feels much faster — only the delta is uploaded.

#Conclusion

• Repository = a collection of encrypted blobs; snapshot = the state of a directory at one point in time.
• Content-defined chunking enables content-based dedup: only the delta is stored.
• Repository structure: config, data/, index/, keys/, snapshots/.
• System components: binary, backend, password/key, and the core command set.
• Backup flow: scan → chunking → dedup → compression & encryption → index → snapshot.

In the next episode, episode 3, we perform the first action on a real repository: repository initialization — restic init for local and S3 backends, managing the password securely, and the mistakes you should avoid from the very beginning.