Learn Secret Management - Complete Production-Grade OpenBao Architecture

#Introduction

Episode 19 closed the troubleshooting phase, and now the moment you have been waiting for has arrived: episode 20 — the final episode of this series. Across 19 episodes you learned piece by piece: from initialization and unseal, secrets engines, policies and auth, application integration, Kubernetes, CI/CD, to high availability and hardening. This last episode brings all those pieces together into one complete design: a production-grade architecture for unified, enterprise-scale secret management that is 100% open source.

We will build a company case study with real needs: an OpenBao cluster that is always available, Kubernetes applications that get secrets without touching code, a database using short-lived dynamic credentials, PII data such as national ID and credit card numbers encrypted at rest, human access authenticated via SSO, and a recorded audit trail with automated backups.

#The Integrated Architecture Flow

Before going into detail, here is the big map of the architecture we will build:

All these layers connect to one shared OpenBao cluster. Let's build them one by one.

#Infrastructure Layer: HA Cluster with Auto-Unseal

The architecture's foundation is a 3-node OpenBao HA cluster using Raft integrated storage and AWS KMS auto-unseal. Three nodes mean the cluster survives if one node dies, while auto-unseal ensures every node opens automatically after a restart — without a human carrying keys.

ui = true
 
storage "raft" {
  path      = "/etc/openbao/data"
  node_id   = "openbao-1"
  retry_join {
    leader_api_addr = "https://openbao-1.example.com:8200"
  }
}
 
seal "awskms" {
  region     = "ap-southeast-1"
  kms_key_id = "arn:aws:kms:ap-southeast-1:123456789012:key/abc123"
}
 
listener "tcp" {
  address       = "0.0.0.0:8200"
  tls_disable   = "false"
  tls_cert_file = "/etc/openbao/tls/server.crt"
  tls_key_file  = "/etc/openbao/tls/server.key"
}
 
disable_mlock = false

The second and third nodes differ only in node_id and the retry_join address. Once all nodes join via bao operator raft join, the cluster elects one leader. All the principles from episodes 15 and 16 — quorum, failover, and auto-unseal — are now a reality in your infrastructure.

#Kubernetes Integration: Agent Sidecar Injector

Applications on EKS or GKE need secrets but must not store them themselves. With the OpenBao Agent Sidecar Injector, every annotated pod automatically receives a sidecar that fetches secrets and renders them to a file on a shared volume. Authentication uses the Kubernetes Auth Engine — the pod proves its identity with its ServiceAccount JWT.

Pod annotations for sidecar injection

metadata:
  annotations:
    openbao.org/agent-inject: "true"
    openbao.org/role: "web-app"
    openbao.org/agent-inject-secret-config: "secret/data/myapp"
    openbao.org/agent-inject-file-config: "config.env"

Before that, enable and configure kubernetes auth on the OpenBao side:

bao auth enable kubernetes
bao write auth/kubernetes/config kubernetes_host="https://kubernetes.default.svc"

The application now reads the sidecar-rendered /bao/secrets/config, updated automatically according to the lease lifecycle. This is exactly the pattern you learned in episode 12 — clean application code, always-fresh secrets.

#Database Security: Dynamic PostgreSQL Credentials

The part most often used as a benchmark of success: dynamic PostgreSQL credentials with a short TTL. Every application that needs a database connection requests credentials from OpenBao, and OpenBao instantly creates a user with a temporary password — living for 1 hour, then automatically removed.

bao secrets enable database
bao write database/config/postgres \
  plugin_name="postgresql-database-plugin" \
  connection_url="postgresql://{{username}}:{{password}}@db.internal:5432/app" \
  allowed_roles="web-role"
bao write database/roles/web-role \
  db_name="postgres" \
  creation_statements="CREATE USER {{name}} WITH PASSWORD '{{password}}' VALID UNTIL '{{expiration}}'" \
  default_ttl="1h" max_ttl="24h"

When a service calls bao read database/creds/web-role, OpenBao immediately creates a temporary user. With a 1-hour TTL, even if credentials leak, their lifetime is very short — and because each instance uses different credentials, one leak does not open the door for all. The principle from episode 4 now runs fully in production.

#Application Security: Transit for PII Data

Sensitive data such as national ID and credit card numbers must not be stored as plain text in application databases. With the Transit Secrets Engine, OpenBao becomes the centralized encryption service: applications send plain text, receive ciphertext, and OpenBao never stores the data — only the keys.

bao write -f transit/keys/pii-key
bao write transit/encrypt/pii-key plaintext=$(base64 <<< "NIK-3171xxxx")
bao write transit/decrypt/pii-key ciphertext="vault:v1:..."

The application backend calls the transit endpoint when storing and reading PII data. Key rotation, versions, and lifecycle are managed in one place — OpenBao — not scattered across each service's code. You can add convergent or built-in keyrings as needed. This is the real-world form of the EaaS introduced in episode 5.

#Human Access and IaC: OIDC, OpenTofu, Loki, and Snapshots

Finally, the layer for humans and infrastructure-as-code. Humans no longer use static tokens: they log in via OIDC to the company SSO, and OpenTofu uses the Vault provider (fully compatible with OpenBao) to fetch secrets during provisioning. All activity is recorded to Grafana Loki, and a raft snapshot is taken daily.

bao auth enable oidc
bao write auth/oidc/config oidc_discovery_url="https://sso.example.com" \
  oidc_client_id="openbao" oidc_client_secret="***"
bao audit enable syslog
0 2 * * * bao operator raft snapshot save /backups/daily.snap

Meanwhile, on the IaC side, the OpenTofu provider reads dynamic credentials for provisioning and issues internal PKI certificates as needed. The audit device you enabled — whether to a file or syslog forwarded to Loki — records every one of these steps, giving the security team a complete trail.

#Production Readiness and Security Audit Checklist

A great architecture means nothing without verification. Use the following checklist as the final security audit material before and after go-live:

#Conclusion

Congratulations, you have completed the entire Learn Secret Management with OpenBao series journey — from episode 0 that prepared your environment and covered the fork history and the Linux Foundation manifesto, through architecture and initialization, KV, dynamic database secrets, transit, PKI, policies, authentication methods, lease management, application integration, agent and auto-auth, Kubernetes, CI/CD, OpenTofu and Ansible, high availability clusters, auto-unseal, audit logging and hardening, migration from HashiCorp Vault, troubleshooting, to the production-grade architecture you just designed.

This final episode summarized everything into a single blueprint: a 3-node HA cluster with Raft storage and AWS KMS auto-unseal, the Agent Sidecar Injector with kubernetes auth on EKS or GKE, dynamic PostgreSQL credentials with a 1-hour TTL, Transit for encrypting national ID and credit card numbers, human access via OIDC, provisioning with the OpenTofu provider, audit logs to Grafana Loki, and daily snapshots — all with a verified production readiness and security audit checklist.

Key takeaways:

• All layers center on one cluster maintained with discipline: quorum, unseal, and backup.
• Dynamic credentials and Transit turn secret management from storage into an active security service.
• Automation is a production requirement — auto-unseal, injectors, snapshots, and audit must run without human intervention.
• Verification matters more than construction — the audit checklist and restore tests determine true readiness.

But remember: mastering OpenBao is not the end, it is the entrance. The principles you learned — least privilege, dynamic secrets, observability, and defense in depth — apply equally across the rest of the security field. Keep pursuing the next security topics on this platform, because that is where you will refine the skills you built over these 21 episodes. See you in the next series!