How to Encrypt Kubernetes Secrets in Git with SOPS and age (2026)
This guide is for engineers who want Kubernetes Secrets in the same Git repository as everything else, encrypted, and decrypted only by the cluster. By the end you will have an age key pair, a .sops.yaml that encrypts the right fields automatically, an encrypted Secret committed to Git, Flux or Argo CD decrypting it at deploy time, and a key rotation procedure you can actually run. Commands are written against current releases as of 31 October 2026: SOPS v3.13.3, age v1.3.2, Flux v2.10.0 (kustomize-controller v1.10.1), Argo CD v3.5.6 and KSOPS v4.5.1. You need kubectl, a cluster running Flux or Argo CD, and a Git repository you can push to. If you are still choosing between SOPS, Sealed Secrets and External Secrets Operator, read the comparison post first.
Step 1: Install sops and age with pinned versions
Package managers are fine on a laptop, but in CI and on shared bastions pin the exact binary. SOPS publishes a cosign-signed checksums file with every release, so verify it.
# Linux amd64. Swap linux.amd64 for darwin.arm64 etc. as needed.
SOPS_VERSION=v3.13.3
AGE_VERSION=v1.3.2
curl -LO "https://github.com/getsops/sops/releases/download/${SOPS_VERSION}/sops-${SOPS_VERSION}.linux.amd64"
curl -LO "https://github.com/getsops/sops/releases/download/${SOPS_VERSION}/sops-${SOPS_VERSION}.checksums.txt"
curl -LO "https://github.com/getsops/sops/releases/download/${SOPS_VERSION}/sops-${SOPS_VERSION}.checksums.sigstore.json"
# Verify the checksums file was signed by the getsops release workflow
cosign verify-blob "sops-${SOPS_VERSION}.checksums.txt" \
--bundle "sops-${SOPS_VERSION}.checksums.sigstore.json" \
--certificate-identity-regexp=https://github.com/getsops \
--certificate-oidc-issuer=https://token.actions.githubusercontent.com
sha256sum -c "sops-${SOPS_VERSION}.checksums.txt" --ignore-missing
sudo install -m 0755 "sops-${SOPS_VERSION}.linux.amd64" /usr/local/bin/sops
# The age tarball holds age, age-keygen, age-inspect and (since 1.3.2) the compatibility plugins
curl -L "https://github.com/FiloSottile/age/releases/download/${AGE_VERSION}/age-${AGE_VERSION}-linux-amd64.tar.gz" | tar -xz
sudo install -m 0755 age/age age/age-keygen /usr/local/bin/
sops --version
# sops 3.13.3 (latest)
age --version
# v1.3.2
Pin 3.13.3 specifically. The SOPS changelog records that 3.13.2 computed a wrong MAC during decryption for YAML lists that contain a comment, fixed in 3.13.3.
Step 2: Generate age keys for the cluster and for break-glass
Create at least two key pairs. One is the cluster key, which goes into Kubernetes. The other is a break-glass key that lives offline (a password manager or a hardware-backed vault), so you can still decrypt the repo if the cluster is destroyed.
age-keygen -o cluster-prod.agekey
# Public key: age1qe5lxzzeppw5k79vxn3872272sgy224g2nzqlzy3uljs84say3yqgvd0sw
age-keygen -o breakglass.agekey
# Public key: age1s3cqcks5genc6ru8chl0hkkd04zmxvczsvdxq99ekffe4gmvjpzsedk23c
# The file contains a comment with the public key, then the private key
cat cluster-prod.agekey
# # created: 2026-10-31T09:12:44Z
# # public key: age1qe5l...
# AGE-SECRET-KEY-1...
# Never commit these. Belt and braces:
echo '*.agekey' >> .gitignore
The public keys above are examples from the SOPS docs; use your own. Public keys are safe to commit and share. Private keys are not.
To decrypt locally, SOPS looks for keys.txt in $XDG_CONFIG_HOME/sops/age/, falling back to ~/.config/sops/age/keys.txt on Linux and ~/Library/Application Support/sops/age/keys.txt on macOS, per the SOPS age docs. You can override that with SOPS_AGE_KEY_FILE, SOPS_AGE_KEY or SOPS_AGE_KEY_CMD.
Post-quantum keys: age v1.3.0 added hybrid ML-KEM-768 recipients via age-keygen -pq (public keys start age1pq1, private keys AGE-SECRET-KEY-PQ-1), and SOPS gained support in 3.12.0. Both decrypting sides covered here are new enough on paper: kustomize-controller v1.10.1's go.mod pins SOPS 3.13.3 and age 1.3.2, and KSOPS v4.5.1 pins SOPS 3.12.2. I would still test with a throwaway Secret before switching a production repo, because PQ recipients are larger and fairly new.
Step 3: Write .sops.yaml creation rules
.sops.yaml at the repo root tells SOPS which recipients and which fields to use for each path, so nobody has to remember flags. Rules are evaluated in order and the first match wins. path_regex is matched against the file path relative to the .sops.yaml, but SOPS searches for the config starting from your current working directory, so run commands from the repo root. The file must be named .sops.yaml; .sops.yml is not auto-discovered (config file docs).
# .sops.yaml (repo root)
creation_rules:
# Production: prod cluster key + offline break-glass key
- path_regex: clusters/prod/.*\.sops\.yaml$
encrypted_regex: ^(data|stringData)$
age: >-
age1qe5lxzzeppw5k79vxn3872272sgy224g2nzqlzy3uljs84say3yqgvd0sw,
age1s3cqcks5genc6ru8chl0hkkd04zmxvczsvdxq99ekffe4gmvjpzsedk23c
# Staging: its own cluster key + the same break-glass key
- path_regex: clusters/staging/.*\.sops\.yaml$
encrypted_regex: ^(data|stringData)$
age: >-
age1yt3tfqlfrwdwx0z0ynwplcr6qxcxfaqycuprpmy89nr83ltx74tqdpszlw,
age1s3cqcks5genc6ru8chl0hkkd04zmxvczsvdxq99ekffe4gmvjpzsedk23c
encrypted_regex: ^(data|stringData)$ matters. Flux's Kustomization spec requires metadata, kind and apiVersion to stay in plain text, and keeping them readable also means reviewers can see which Secret a pull request touches. The .sops.yaml suffix convention on filenames keeps the rules from catching ordinary manifests.
Step 4: Encrypt, decrypt and edit a Secret
Generate the Secret with kubectl so the shape is right, then encrypt it in place. Use stringData so you never hand-encode base64.
mkdir -p clusters/prod/apps/billing
cat > clusters/prod/apps/billing/db-creds.sops.yaml <<'EOF'
apiVersion: v1
kind: Secret
metadata:
name: db-creds
namespace: billing
type: Opaque
stringData:
username: billing
password: change-me-now
EOF
sops encrypt --in-place clusters/prod/apps/billing/db-creds.sops.yaml
cat clusters/prod/apps/billing/db-creds.sops.yaml
The result keeps the structure and replaces only the values:
apiVersion: v1
kind: Secret
metadata:
name: db-creds
namespace: billing
type: Opaque
stringData:
username: ENC[AES256_GCM,data:...,iv:...,tag:...,type:str]
password: ENC[AES256_GCM,data:...,iv:...,tag:...,type:str]
sops:
age:
- recipient: age1qe5lxzzeppw5k79vxn3872272sgy224g2nzqlzy3uljs84say3yqgvd0sw
enc: |
-----BEGIN AGE ENCRYPTED FILE-----
...
- recipient: age1s3cqcks5genc6ru8chl0hkkd04zmxvczsvdxq99ekffe4gmvjpzsedk23c
enc: |
...
lastmodified: "2026-10-31T09:20:11Z"
mac: ENC[AES256_GCM,data:...,type:str]
encrypted_regex: ^(data|stringData)$
version: 3.13.3
Day-to-day commands:
# Print plaintext to stdout (needs a matching private key locally)
sops decrypt clusters/prod/apps/billing/db-creds.sops.yaml
# Pull out one value
sops decrypt --extract '["stringData"]["password"]' clusters/prod/apps/billing/db-creds.sops.yaml
# Edit in $EDITOR; SOPS decrypts to a temp file and re-encrypts on save
sops edit clusters/prod/apps/billing/db-creds.sops.yaml
Always change encrypted files through sops edit or sops set. Editing the YAML by hand breaks the MAC, and the file will refuse to decrypt. Add a CI check that fails if any *.sops.yaml file lacks a sops: block, and run a secret scanner as well; my Gitleaks pre-commit guide covers that.
Do not kubectl apply these files. The encrypted values would land in the cluster as literal ENC[...] strings. The controller does the decryption.
Step 5: Decrypt in-cluster with Flux kustomize-controller
Flux needs the private cluster key in a Secret in the same namespace as the Kustomization (here flux-system), with the data key ending in .agekey. That suffix is how kustomize-controller recognises an age identity, per the Flux SOPS guide.
kubectl create secret generic sops-age \
--namespace=flux-system \
--from-file=identity.agekey=cluster-prod.agekey
# Then remove the private key from your laptop, keeping only the offline copy
shred -u cluster-prod.agekey
Point the Kustomization at it:
# clusters/prod/flux-system/billing-secrets.yaml
apiVersion: kustomize.toolkit.fluxcd.io/v1
kind: Kustomization
metadata:
name: billing-secrets
namespace: flux-system
spec:
interval: 10m
path: ./clusters/prod/apps/billing
prune: true
sourceRef:
kind: GitRepository
name: flux-system
decryption:
provider: sops # the only supported provider
secretRef:
name: sops-age # Secret with *.agekey entries
Commit, push, then check:
flux reconcile kustomization billing-secrets --with-source
flux get kustomizations billing-secrets
# NAME REVISION SUSPENDED READY MESSAGE
# billing-secrets main@sha1:3f2a... False True Applied revision: main@sha1:3f2a...
kubectl -n billing get secret db-creds -o jsonpath='{.data.password}' | base64 -d
# change-me-now
Two optional refinements from the spec: add .sops.yaml to the GitRepository's spec.ignore so the config file is not applied, and use the --sops-age-secret controller flag if you want one global age Secret for every Kustomization instead of a secretRef on each.
Step 6: Decrypt with Argo CD using KSOPS (or helm-secrets)
Argo CD has no built-in SOPS support. Know what you are accepting first: the Argo CD secret management docs caution against generation-time decryption, because rendered manifests, secrets included, sit in plaintext in Argo CD's Redis cache and are reachable through the repo-server API. If you go ahead, apply NetworkPolicies to Redis and the repo-server, as those docs suggest.
Create the key Secret in the argocd namespace, then install KSOPS through the Argo CD Helm chart values, adapted from the KSOPS README:
kubectl create secret generic sops-age \
--namespace=argocd \
--from-file=keys.txt=cluster-prod.agekey
# argocd-values.yaml for the argo-cd Helm chart
configs:
cm:
# Required: let Kustomize run exec KRM plugins
kustomize.buildOptions: "--enable-alpha-plugins --enable-exec"
repoServer:
env:
- name: SOPS_AGE_KEY_FILE
value: /home/argocd/.config/sops/age/keys.txt
volumes:
- name: custom-tools
emptyDir: {}
- name: sops-age
secret:
secretName: sops-age
initContainers:
- name: install-ksops
image: viaductoss/ksops:v4.5.1
# Since v4.5.1 this copies only ksops; add --with-kustomize to override Argo's kustomize too
command: ["/usr/local/bin/ksops", "install", "/custom-tools"]
volumeMounts:
- mountPath: /custom-tools
name: custom-tools
volumeMounts:
- mountPath: /usr/local/bin/ksops
name: custom-tools
subPath: ksops
- mountPath: /home/argocd/.config/sops/age
name: sops-age
readOnly: true
In the app directory, replace the plain Secret reference with a KSOPS generator:
# clusters/prod/apps/billing/secret-generator.yaml
apiVersion: viaduct.ai/v1
kind: ksops
metadata:
name: billing-secrets
annotations:
config.kubernetes.io/function: |
exec:
path: ksops
files:
- ./db-creds.sops.yaml
---
# clusters/prod/apps/billing/kustomization.yaml
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
generators:
- ./secret-generator.yaml
Test locally before Argo CD does (requires ksops on your PATH and a local key):
kustomize build --enable-alpha-plugins --enable-exec clusters/prod/apps/billing
If your apps are Helm charts, helm-secrets (v4.7.10) is the alternative: it registers as a Helm downloader plugin, you list secrets:// among helm.valuesFileSchemes in argocd-cm, and reference values files as secrets://secrets.yaml in the Application. Its Argo CD integration doc has the repo-server image and init-container options.
Step 7: Rotate age keys without locking yourself out
There are two kinds of rotation. Changing recipients (adding a cluster, removing a person) uses sops updatekeys. Changing the per-file data key uses sops rotate. The SOPS key management docs give the order for a compromised key: remove it from .sops.yaml, run updatekeys, then rotate, so the old key loses access before the data key changes. After that I change the real credentials too.
A planned cluster key rollover looks like this:
# 1. Generate the new cluster key and ADD it to the cluster Secret alongside the old one.
# Flux tries every *.agekey entry; KSOPS tries every line in keys.txt.
age-keygen -o cluster-prod-2026q4.agekey
# Recover the current key from the cluster (or use your offline copy)
kubectl -n flux-system get secret sops-age \
-o jsonpath='{.data.identity\.agekey}' | base64 -d > old-cluster-prod.agekey
kubectl -n flux-system create secret generic sops-age \
--from-file=identity.agekey=old-cluster-prod.agekey \
--from-file=identity-2026q4.agekey=cluster-prod-2026q4.agekey \
--dry-run=client -o yaml | kubectl apply -f -
# 2. In .sops.yaml, replace the old prod recipient with the new public key.
# 3. Re-wrap every prod file for the new recipients, then rotate its data key.
find clusters/prod -name '*.sops.yaml' -print0 |
while IFS= read -r -d '' f; do
sops updatekeys -y "$f"
sops rotate --in-place "$f"
done
# 4. Commit, push, confirm Flux or Argo CD reconciles cleanly.
# 5. Remove the old key from the cluster Secret and destroy remaining copies.
If the old key leaked, Git history still holds files encrypted to it. Rotating the key protects future commits only, so change the actual passwords and API keys as the last step.
Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
config file not found, or has no creation rules, and no keys provided through command line options |
Run from a directory without .sops.yaml above it, or the file is named .sops.yml |
Run from the repo root, rename to .sops.yaml, or pass --config |
| File encrypts with the wrong recipients | An earlier path_regex matched first |
Order rules from most to least specific; test with a new file |
Failed to get the data key required to decrypt the SOPS file with no identity matched any of the recipients |
Local or in-cluster private key does not match any recipient in the file |
Check SOPS_AGE_KEY_FILE or keys.txt; compare the public key comment with the file's sops.age list |
MAC mismatch on decrypt |
File edited by hand, or bad merge conflict resolution | Restore from Git and redo the change with sops edit; upgrade off 3.13.2 if YAML lists have comments |
Secret in cluster contains literal ENC[AES256_GCM,...] values |
Applied with kubectl, or Kustomization has no decryption block |
Remove manual applies; add spec.decryption with provider: sops |
| Flux reconciliation fails on an encrypted Secret | metadata, kind or apiVersion were encrypted |
Re-create the file using the encrypted_regex rule from Step 3 |
| Flux ignores the key Secret | Data key does not end in .agekey, or Secret is in another namespace |
Recreate as identity.agekey in the Kustomization's namespace |
| Argo CD: kustomize says external plugins are disabled | kustomize.buildOptions not set in argocd-cm |
Set --enable-alpha-plugins --enable-exec and restart the repo-server |
Argo CD: ksops executable not found |
Init container or subPath mount missing |
Check repo-server pod spec and init container logs |
sops updatekeys reports no changes |
The file path does not match any path_regex |
Check the regex against the path relative to .sops.yaml |
FAQ
Is age better than PGP for SOPS?
For most teams, yes. Both the SOPS docs and the Flux guide recommend age over PGP where possible. Keys are one line each, there is no keyring or agent to manage, and age 1.3 adds post-quantum hybrid recipients.
Can I commit the .sops.yaml file and public keys?
Yes. .sops.yaml contains only public recipients and path rules, and teammates need it to encrypt new files correctly. Never commit any *.agekey or keys.txt file.
Should I use KSOPS or helm-secrets with Argo CD?
Use KSOPS if your apps are Kustomize-based and helm-secrets if they are Helm charts with encrypted values files. Both decrypt in the repo-server, so both carry the Redis caching risk described in Argo CD's docs. If that is unacceptable, Sealed Secrets or External Secrets Operator move decryption to the cluster.
What happens if I lose the age private key?
Every file encrypted only to that key is unrecoverable. That is why every rule in this guide includes an offline break-glass recipient. Test the break-glass key once a quarter by decrypting a file with it.
In production, the first thing I check is the recipients list: does every encrypted file include a key that exists outside the cluster, and does sops decrypt with only the break-glass key still work? The second is that nobody has a cluster private key on a laptop. After that, rotation is a calendar entry, not an incident. The authoritative references are the SOPS documentation, the Flux SOPS guide, the Argo CD secret management page and the age README.
Related: How to Use External Secrets Operator with Vault on Kubernetes (2026), How to Set Up Argo CD for GitOps: Step-by-Step Guide (2026) and Argo CD vs Flux in 2026: GitOps After Argo CD 3.5 and Flux 2.9.
I'm Ajay Kumar — I build and operate PandaStack, an open-source Firecracker microVM cloud for AI agents, and write about the infrastructure and tooling around it.
Need this kind of infrastructure work? See what I do or email hello@ajayk.sh.
Related
How to Fix Kubernetes RBAC Forbidden: User Cannot List Resource (2026)
Fix the Kubernetes error 'Error from server (Forbidden): User cannot list resource' step by step: parse the message, check identity with kubectl auth whoami and can-i --as, write a least-privilege Role and RoleBinding, fix service account tokens, EKS access entries, GKE IAM and AKS.
13 minOct 31, 2026How to Sign and Verify Container Images with Cosign Keyless (2026)
Step-by-step: sign container images with cosign v3.1.4 keyless in GitHub Actions, verify with certificate identity flags, attach SBOM and SLSA attestations, enforce signatures in Kubernetes with Kyverno 1.19 ImageValidatingPolicy or Sigstore policy-controller, and fix common errors.
11 minOct 20, 2026How to Set Up cert-manager with Let's Encrypt and Fix Errors (2026)
Install cert-manager v1.21 with Helm, create Let's Encrypt ClusterIssuers for HTTP-01 (Gateway API or Ingress) and DNS-01, then debug a Certificate stuck not ready by walking CertificateRequest, Order and Challenge, with rate limits, ARI and alerts.
14 min