Learning GitOps - FluxCD - Secrets Management with SOPS

#Introduction

In episode 19 you secured Flux: authentication, RBAC, network security, and the supply chain. But there's a gap we deliberately left: secrets like SSH keys, registry passwords, and Helm tokens are still written raw in the Git manifests. The Git repo that's the source of truth of GitOps turns into a source of leaks.

This episode covers the solution: Mozilla SOPS (Secrets OPerationS). SOPS encrypts sensitive values inside YAML or JSON files so secrets can be committed to Git without leaking their contents, and Flux decrypts them automatically when applying the manifests.

#What Is SOPS

SOPS is a CLI tool for field-level file encryption. Unlike Vault, which stores secrets centrally, SOPS has no server — it purely processes files. Its characteristics:

• Encryption at rest: files are encrypted when stored, including in Git and on disk.
• Git-friendly: secret changes can be diffed, reviewed through pull requests, and audited like ordinary code changes.
• Flexible: it can encrypt all values, specific fields, or parts of complex structures.
• Multi-provider keys: supports age, PGP, and the major cloud KMS systems.

The basic principle: Git stores ciphertext, the private key never enters Git. In the episode 19 examples, we replace password: ghp_xxxx with the encrypted block ENC[AES256_GCM,data:...].

#Keys: age, PGP, or Cloud KMS

SOPS uses envelope encryption: the file's data key is encrypted by one or more master keys. The master key choice determines the workflow.

For local use, age is the most practical. Create a key with age-keygen -o age.agekey, store the private key in a safe place, and only publish the public key to .sops.yaml.

#Encrypting Secrets with SOPS

#.sops.yaml Configuration

The encryption rules are declared in .sops.yaml at the repo root. These rules match file paths and determine which recipient or KMS is used:

creation_rules:
  - path_regex: clusters/.*/secrets\.yaml$
    age: >-
      age1q2w3e4r5t6y7u8i9o0p1a2s3d4f5g6h7j8k9l0zxc1v
    encrypted_regex: "^(password|token|apiKey|secret)$"
  - path_regex: clusters/.*/\.env$
    age: >-
      age1q2w3e4r5t6y7u8i9o0p1a2s3d4f5g6h7j8k9l0zxc1v

encrypted_regex limits which fields are encrypted, for example only password, token, apiKey, and secret.

#Encrypting a File

With the rules above, encryption is just one command. Encrypt all fields or only specific fields:

sops -e clusters/prod/secrets.yaml > clusters/prod/secrets.enc.yaml
sops --encrypt --encrypted-regex '^(password|token)$' \
  clusters/prod/env.yaml > clusters/prod/env.enc.yaml
sops --encrypt-edit clusters/prod/secrets.yaml

A simpler version: name the file secrets.enc.yaml and run sops --encrypt clusters/prod/secrets.enc.yaml — SOPS reads .sops.yaml, loads the file, and writes the ciphertext directly to the same file.

#Specific Field Encryption

If only part of the values are secret, let the rest of the metadata stay readable so the diff remains comfortable:

apiVersion: v1
kind: Secret
metadata:
  name: api-db
  namespace: apps
type: Opaque
stringData:
  DB_HOST: postgres.prod.svc.cluster.local
  DB_USER: app
  DB_PASSWORD: ENC[AES256_GCM,data:0Nv6Y8j...,iv:...,tag:...,type:str]
sops:
  kms: []
  age:
    - recipient: age1q2w3e4r5t6y7u8i9o0p1a2s3d4f5g6h7j8k9l0zxc1v
      enc: |
        -----BEGIN AGE ENCRYPTED FILE-----
        ...

Non-secret values like DB_HOST stay readable; only DB_PASSWORD becomes ciphertext. The sops: block contains the key metadata for decryption and audit.

#Encrypted Files in Git

Commit the encrypted files as usual; the team can see the structure without seeing the values. Always verify no plaintext slipped through by scanning the repo, for example with GitLeaks in CI, before merging.

#SOPS Integration with Flux

Flux decrypts secrets automatically. The Kustomize controller has built-in SOPS support; all that's needed is the private key available as a Secret in the cluster.

#The Kustomization Decryption Field

In older Flux versions, decryption is enabled explicitly via the decryption field:

apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
  name: apps
  namespace: flux-system
spec:
  interval: 5m
  sourceRef:
    kind: GitRepository
    name: apps
  path: ./clusters/prod/apps
  prune: true
  decryption:
    provider: sops
    secretRef:
      name: sops-age

From Flux 2.2 and above, decryption is automatic: just provide a sops-age Secret (for age) or a sops-gpg Secret (for PGP) in the flux-system namespace, and the kustomize-controller decrypts on its own without additional configuration.

#Service Account Keys (Cloud KMS)

For AWS KMS, don't put the key in the cluster. Use IAM Roles for Service Accounts: give the role access to the KMS key, then annotate the kustomize-controller ServiceAccount. Flux then decrypts directly through KMS without a local private key.

#Key Rotation

Rotating keys requires re-encrypting all the files. SOPS provides dedicated commands:

sops updatekeys clusters/prod/secrets.enc.yaml
sops --rotate --in-place clusters/prod/secrets.enc.yaml

updatekeys replaces the key recipients; --rotate creates a new data key. Both should be run when a key leaks or when changing environments.

#Best Practices

• Never commit raw secrets: all secret values must be ciphertext. Enable secret scanning in CI as a second safeguard.
• Separate keys per environment: use a different age key or KMS key for prod and staging. A staging key leak doesn't open prod.
• Key management procedure: store key backups in a safe place, restrict access to the key files, and document who's allowed to hold them.
• Audit logging: combining cloud KMS with the Git commit history gives a trail of who encrypted, when, and what changed.
• Keys never in Git: including the age.agekey file — add it to .gitignore and never commit it.

#Closing

This episode closed the gap from episode 19: secrets can now live in Git as ciphertext. SOPS encrypts files at the field level, Flux decrypts them automatically in the cluster, and private keys never touch the repo. This combination makes secrets reviewable, diffable, and auditable like ordinary code.

The key takeaways:

• SOPS is a file encryption tool, not a server: secrets stay in Git in encrypted form.
• Keys are separate from data: private keys only live in the cluster or KMS, public keys in .sops.yaml.
• Flux decrypts automatically: provide a sops-age or sops-gpg secret, or use IAM and Workload Identity for KMS.
• Rotation is re-encryption: sops updatekeys and sops --rotate keep secrets safe.
• Per-environment keys: separate staging and production keys to narrow the impact of a leak.

SOPS keeps secrets inside Git, but it still requires a key to exist in each cluster. In the next episode, episode 21, we'll learn about the External Secrets Operator: secrets are stored in an external system such as AWS Secrets Manager, Vault, or 1Password, and Flux only manages the reference. See you in episode 21!