Learn Vault - Core Architecture & Security Concepts

#Introduction

In episode 2, we mapped the entire secret management ecosystem and concluded that Vault is the right choice when the needs are complex: multi-cloud, dynamic secrets, encryption-as-a-service, PKI, and strict audit. Now, in episode 3, we'll open the hood and dissect Vault's core architecture and security concepts — from the storage backend, Vault Core, request flow, to the initialization and unsealing process.

Why is this episode important? Because Vault differs from other tools in one fundamental way: Vault locks itself down to protect its data. Every time the server restarts, Vault refuses to serve requests until it's "unsealed" — and understanding why this is designed that way will change how you view system security as a whole. In the real world, failure to understand this architecture is the root of nearly every operational mistake: operators who lose unseal keys, root tokens left in production, or incorrectly configured storage backends.

In this episode we'll cover: first, the core architecture — storage backend vs Vault Core (Barrier) and the request flow; second, initialization & unsealing with Shamir's Secret Sharing, including manual unseal and an introduction to auto-unseal; third, the root token risk and production best practices. Get your Vault CLI ready, because this episode is full of hands-on practice.

#Vault's Core Architecture

Before looking at the request flow, we need to understand Vault's two main layers: the storage backend and the Vault Core (Barrier). The two are often mistakenly treated as the same thing — yet their difference defines Vault's security model.

#Storage Backend vs Vault Core (Barrier)

The storage backend is Vault's raw data storage location. It can be a file on disk, Consul, or Raft integrated storage. Think of it as a warehouse — the place where all items (encrypted data) are stored. The crucial point: the storage backend never stores data in plaintext. All data written to the storage backend has already been encrypted by Vault Core.

Vault Core is Vault's brain — specifically the part called the Barrier. The Barrier is the encryption layer that protects all data entering and leaving the storage backend. Every write is encrypted with the data encryption key, and every read is decrypted before being passed on to the secrets engine. Think of the Barrier as a warehouse guard who padlocks every item coming in and unlocks every item going out — without the key (the unsealed master key), this guard refuses to operate at all.

┌────────────────────────────────────────────────┐
│                   VAULT CORE                    │
│  Auth Methods → Policies → Secrets Engines     │
│  ─────────────────────────────                 │
│  BARRIER (encrypt/decrypt all data)             │
└──────────────────────┬─────────────────────────┘
                       │ encrypted data
┌──────────────────────▼─────────────────────────┐
│              STORAGE BACKEND                    │
│  File / Consul / Raft (encrypted data, not      │
│  plaintext)                                     │
└─────────────────────────────────────────────────┘

#The Request Flow: Client → Storage

Now let's trace the journey of a request — for example vault kv get secret/api/database — from the client and back. The flow involves several security layers that work in sequence:

Client
  │  (1) send token + request
  ▼
Auth Method          ← (2) "Who are you?" — validates identity
  │
  ▼
Policy Check         ← (3) "What can you do?" — evaluates HCL policy
  │
  ▼
Secrets Engine       ← (4) "What data are you asking for?" — processes & returns data
  │
  ▼
Storage Backend      ← (5) "Store/read encrypted data" (through the Barrier)

Let's break down these five steps:

1. The Client sends the request complete with the authentication token. All traffic should go over TLS.
2. The Auth Method answers the question "who are you?" — which entity does this token belong to, and what method was used (token, userpass, AppRole, Kubernetes, etc.). In episode 10 we'll dissect auth methods thoroughly.
3. The Policy Check answers "what can you do?" — the HCL policy determines which paths the client may access and with what capability. This is the firewall before secrets are seen. Episode 9 covers policies in full.
4. The Secrets Engine is the module that handles the type of data: KV engine, database engine, transit, PKI, and others. It's the part that knows how to generate, store, or process secrets according to their type.
5. The Storage Backend stores the final result — always in a form encrypted by the Barrier.

#Initialization & Unsealing

Now we enter the most distinctive part of Vault — and the one that surprises beginners the most: Vault is born in a locked state. Let's understand the process from the start.

#Initialization: Vault's Birth

When the Vault server first runs (for example, the production mode we set up in episode 0), it's in an uninitialized state — no keys yet, nothing yet. We must initialize Vault to create its first "cryptographic identity":

vault operator init
Unseal Key 1: 4fBvZ0QvK9hHlVY1Aq2mC3xE5rT6uI7oL8pM9nB0cV
Unseal Key 2: dE5rT6uI7oL8pM9nB0cV4fBvZ0QvK9hHlVY1Aq2mC3x
Unseal Key 3: mC3xE5rT6uI7oL8pM9nB0cV4fBvZ0QvK9hHlVY1Aq2mC
Unseal Key 4: pM9nB0cV4fBvZ0QvK9hHlVY1Aq2mC3xE5rT6uI7oL8
Unseal Key 5: t6uI7oL8pM9nB0cV4fBvZ0QvK9hHlVY1Aq2mC3xE5rT
 
Initial Root Token: hvs.Ga4y9JkL2mQ8zXcV1bN5wK3pR7dF6sH8jT0uY4aZ
 
Success! Vault is initialized

The output above is the most decisive moment in a Vault's life. There are two kinds of secret material here:

1. Unseal Keys (5 keys) — the result of splitting the master key. Vault does NOT store its master key. It splits the master key into parts (shares) and gives each part to a different person.
2. Root Token (hvs.Ga4y...) — a token with total access to Vault. The only token that can do anything, including disabling secrets engines and creating other tokens.

#Shamir's Secret Sharing: The Split Key

Behind the output above is an algorithm called Shamir's Secret Sharing (SSS), invented by Adi Shamir in 1979. Its principle is elegant: a secret (the master key) is split into N parts (shares), and reconstructing the secret requires a minimum of K parts (threshold).

Master Key
  │  split by Shamir's Secret Sharing
  ├──► Share 1 (Unseal Key 1)
  ├──► Share 2 (Unseal Key 2)
  ├──► Share 3 (Unseal Key 3)
  ├──► Share 4 (Unseal Key 4)
  └──► Share 5 (Unseal Key 5)
 
Threshold = 3  →  a MINIMUM of 3 shares is needed to unseal Vault

Why is this design so clever?

• Anti single point of failure. No single person holds full access. Stealing 1 or 2 shares is meaningless — mathematically there isn't enough information to reconstruct the master key.
• Anti insider threat. You can distribute 5 shares to 5 different people. To unseal Vault, at least 3 of the 5 must cooperate. No single individual can open Vault alone.
• Resilience. If one share is lost (someone resigns, a laptop breaks), Vault can still be opened as long as K shares remain.

#Manual Unseal

Every time Vault is restarted, it returns to the sealed state: the Barrier is locked, and all requests are rejected. The operator's job is to collect K shares in sequence to reconstruct the master key. This is called manual unseal:

vault operator unseal
Unseal Key (will be hidden): 4fBvZ0QvK9hHlVY1Aq2mC3xE5rT6uI7oL8pM9nB0cV
Key                Value
---                -----
Seal Type          shamir
Initialized        true
Sealed             true
Total Shares       5
Threshold          3
Unseal Progress    1/3
Unseal Nonce       a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d

vault operator unseal
Unseal Key (will be hidden): dE5rT6uI7oL8pM9nB0cV4fBvZ0QvK9hHlVY1Aq2mC3x
Key                Value
---                -----
Seal Type          shamir
Initialized        true
Sealed             true
Total Shares       5
Threshold          3
Unseal Progress    2/3
Unseal Nonce       a1b2c3d4-e5f6-4a7b-8c9d-0e1f2a3b4c5d

vault operator unseal
Unseal Key (will be hidden): mC3xE5rT6uI7oL8pM9nB0cV4fBvZ0QvK9hHlVY1Aq2mC
Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    5
Threshold       3
Version         1.18.3
Storage Type    file
Cluster Name    vault-cluster-6e4b2c9a
Cluster ID      3f4a5b6c-7d8e-4f90-a1b2
HA Enabled      false

Notice the transition: Unseal Progress 1/3 → 2/3 → when the third share is entered, Sealed changes to false and Vault starts accepting requests. This process must be repeated every time Vault restarts — and this is why manual unseal becomes a real operational challenge in production (imagine 3 servers in a data center, a power reboot, and the entire team having to gather and type in shares).

vault status
Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          true
Total Shares    5
Threshold       3
Version         1.18.3

vault status
Key             Value
---             -----
Seal Type       shamir
Initialized     true
Sealed          false
Total Shares    5
Threshold       3
Version         1.18.3

#Introduction: Auto-Unseal

The operational challenge of manual unseal gave birth to a modern solution: auto-unseal. Instead of humans collecting shares, Vault entrusts opening its Barrier to the cloud KMS services we already know from episode 2 — AWS KMS, GCP Cloud KMS, or Azure Key Vault.

storage "raft" {
  path = "/vault/data"
}
 
seal "awskms" {
  region     = "ap-southeast-1"
  kms_key_id = "arn:aws:kms:ap-southeast-1:123456789012:key/abc-1234"
}
 
listener "tcp" {
  address = "0.0.0.0:8200"
}

How it works: on boot, Vault calls the cloud KMS to decrypt the master key stored in encrypted form. No human intervention — server restarts, Vault automatically unseals, requests are served immediately. We'll cover auto-unseal in depth in episode 21.

#Root Token: Risks & Best Practices

The second piece of secret material from the init process is the root token (hvs.Ga4y... at the start). It's a token with absolute access — it can do anything in Vault without being blocked by any policy. Let's discuss its risks and best practices.

#Why Is the Root Token So Dangerous?

• Unlimited access. The root token bypasses all policies. This single token can read every secret, delete every engine, even change Vault's system configuration.
• No constraints leave a trace. With the root token, the principle of least privilege collapses entirely — whoever holds it is equivalent to a "universal admin".
• Hard to partially revoke. You can't revoke the root token's access to one specific path; revoking it means deleting the token entirely.

# A root token can be regenerated by operators as long as valid
# unseal keys still exist — this is why access to unseal keys
# is also highly sensitive
vault operator generate-root -init
Nonce: a1b2c3d4-...
Started: true

#Root Token Best Practices

1. Never use the root token for daily work. Even administrative work should use an authenticated token with a minimal policy.
2. Revoke the root token in production. Vault supports revoking the root token after the setup process is complete. It can be regenerated at any time via vault operator generate-root as long as unseal keys still exist — so revoking it doesn't mean "losing access forever," it means "closing the wide-open door temporarily."

export VAULT_TOKEN='hvs.Ga4y9JkL2mQ8zXcV1bN5wK3pR7dF6sH8jT0uY4aZ'
vault token revoke hvs.Ga4y9JkL2mQ8zXcV1bN5wK3pR7dF6sH8jT0uY4aZ
Success! Revoked token (if it existed)

3. Store the root token in an isolated place (e.g. an enterprise password manager or HSM), away from routine use, and only for emergency recovery.
4. Enable audit logging before performing root operations, so every activity is recorded.

#Common Mistakes Around Architecture & Unseal

As a technical closer, these are the mistakes that most often haunt beginner Vault operators:

#Conclusion

In episode 3, we dissected Vault's internals: the difference between the storage backend (warehouse) and the Vault Core / Barrier (padlocking guard), the five-layer request flow from client to storage, the initialization process that produces unseal keys and a root token, the Shamir's Secret Sharing mechanism (N shares, K threshold), manual unseal vs auto-unseal, and the risks and best practices for managing the root token.

Key takeaways to remember:

• The storage backend never stores plaintext — all data is encrypted by the Barrier.
• Request flow: Auth → Policy → Engine → Storage — no shortcuts.
• Shamir's Secret Sharing splits the master key into N shares with a K threshold — anti single point of failure.
• Manual unseal requires operator coordination; auto-unseal moves that to the cloud KMS.
• The root token must be revoked after setup; never use it for daily work.
• Never store unseal keys and Vault data in the same place.

Now you understand why Vault is secure. In episode 4, we start touching the most concrete part: the KV Secrets Engine — comparing KV v1 vs KV v2 with versioning, enabling a secrets engine, and practicing all the vault kv operations (put, get, list, rollback, destroy, undelete). This is the episode where we truly start "storing secrets" in Vault. Make sure your dev-mode Vault is still running and keep your enthusiasm up, because the real technical journey starts here!