Secrets Management

// 1) The hierarchy of secret storage (best -> worst)
//
//   • Hardware security module (HSM) — keys never leave the device
//   • Cloud KMS (AWS KMS, GCP KMS, Azure Key Vault)
//   • Secrets manager (HashiCorp Vault, AWS Secrets Manager, GCP Secret Manager, Doppler)
//   • CI/CD secret store (GitHub Actions secrets, GitLab CI variables)
//   • Application config service (Spring Config Server with encryption)
//   • .env file (DEV ONLY; never commit)
//   • Hard-coded in source — NEVER

// 2) Local development
// .env.local (gitignored)
DATABASE_URL=postgresql://postgres:dev@localhost/app_dev
JWT_SECRET=dev-only-secret
STRIPE_KEY=sk_test_...

// .env.example (committed) — placeholders for new devs
DATABASE_URL=postgresql://USER:PASS@HOST/DB
JWT_SECRET=set-via-vault
STRIPE_KEY=sk_test_set-via-vault

// .gitignore
.env
.env.*
!.env.example

// Load in Node:
//   import 'dotenv/config';
// In Python: dotenv.load_dotenv(); pydantic-settings BaseSettings.

// 3) Production — pull secrets at startup or per-request

// AWS Secrets Manager
import { SecretsManagerClient, GetSecretValueCommand } from '@aws-sdk/client-secrets-manager';
const sm = new SecretsManagerClient({ region: 'ap-southeast-2' });
const { SecretString } = await sm.send(new GetSecretValueCommand({ SecretId: 'app/prod/db' }));
const secret = JSON.parse(SecretString);
const dbUrl = `postgresql://${secret.user}:${secret.password}@${secret.host}/${secret.dbname}`;

// HashiCorp Vault — short-lived dynamic credentials
import vault from 'node-vault';
const client = vault({ endpoint: 'https://vault.example.com', token: process.env.VAULT_TOKEN });
const { data } = await client.read('database/creds/app-readonly');
// data.username, data.password — valid for 30 min; rotated automatically

// 4) Cloud-native secret injection
// • AWS ECS — task definition references Secrets Manager ARN
// • Kubernetes — External Secrets Operator syncs Vault/AWS into K8s Secrets
// • Cloud Run / Lambda — env vars resolved from Secret Manager at start
// • Application code reads from process.env — never touches the vault directly

# K8s ExternalSecret example
apiVersion: external-secrets.io/v1beta1
kind: ExternalSecret
metadata: { name: app-db, namespace: prod }
spec:
    refreshInterval: 1h
    secretStoreRef: { name: aws-sm, kind: ClusterSecretStore }
    target: { name: app-db-secret }
    data:
        - secretKey: DATABASE_URL
            remoteRef: { key: app/prod/db, property: url }

// 5) Rotation strategies
// • DB users — rotate via vault dynamic creds (Vault, AWS RDS proxy + IAM)
// • Static API keys — rotate quarterly minimum; automate with CI
// • Encryption keys (KMS) — rotate yearly, keep old versions for decrypt
// • Service-to-service tokens — short-lived (OIDC, IRSA, workload identity)
//
// Automated rotation runbook:
//   1. Generate new secret
//   2. Deploy app config pointing at new
//   3. Wait for deploy to finish
//   4. Verify new secret works (synthetic check)
//   5. Revoke old secret
//
// Manual rotation = humans forget. Automated rotation = uptime + safety.

// 6) Detection — prevent secrets from being committed
//
// Pre-commit hook (gitleaks)
// .pre-commit-config.yaml
repos:
    - repo: https://github.com/gitleaks/gitleaks
      rev: v8.18.0
      hooks:
          - id: gitleaks

// CI scan
name: secrets-scan
on: [push, pull_request]
jobs:
    scan:
        runs-on: ubuntu-latest
        steps:
            - uses: actions/checkout@v4
              with: { fetch-depth: 0 }     # full history
            - uses: gitleaks/gitleaks-action@v2
            - uses: trufflesecurity/trufflehog@main
              with: { extra_args: --only-verified }

// Also: GitHub Secret Scanning + push protection (automatic for public repos)

// 7) If a secret is leaked — REVOKE FIRST
// 1. Rotate / revoke at the source IMMEDIATELY (don't tell anyone first)
// 2. Audit logs — what did the leaked secret access?
// 3. Notify (security team, customer if applicable)
// 4. Remove from git history — git-filter-repo or BFG (rewriting history is a separate decision)
// 5. Post-mortem — how did it leak? Add a control to prevent recurrence
//
// Do NOT remove from history before rotating. Once public, assume the secret is captured forever.

// 8) Secret-related logs
// • Log AUTH events (token issued, token used), not the secret value
// • Mask secrets in error stacks — many frameworks have built-in scrubbers
// • Centralised logger config strips known secret-bearing field names (password, token, key)

function scrubSecrets(obj) {
    const SECRETS = /^(?:password|token|secret|api[_-]?key|authorization)$/i;
    return JSON.parse(JSON.stringify(obj, (k, v) => SECRETS.test(k) ? '<redacted>' : v));
}

// 9) Database connection secrets — special handling
// • Prefer IAM auth (AWS RDS IAM) or workload identity for short-lived creds
// • If you must use static — store in Secrets Manager + rotate quarterly
// • Network: only the app's security group can reach the DB; no public access
// • Audit: query log + DMS / Datadog DB Monitoring shows unusual access

// 10) Service-to-service auth — prefer identity over secrets
// • AWS IAM Roles for Service Accounts (IRSA) — workload's K8s SA maps to a role
// • OIDC tokens — GitHub Actions to AWS without long-lived keys
// • SPIFFE / SPIRE — universal workload identity
// • mTLS — cert-based authentication; certificates rotated by the platform
//
// Long-lived API keys for internal services are a smell.

// 11) Encryption in code — when you absolutely need to store a secret in a database
// • Use libsodium / Web Crypto / cloud KMS — never roll your own
// • Per-tenant DEK (data encryption key) wrapped by a KEK (key encryption key) in KMS
// • Authenticated encryption (AES-GCM, ChaCha20-Poly1305) — not just encryption
// • Compress + encrypt + sign in that order

// 12) Sealed Secrets / SOPS for GitOps
// • Encrypt the secret with a cluster-owned key; commit the encrypted file
// • The cluster decrypts at runtime; humans can review the ciphertext
// • sops + age + git is a popular workflow

// 13) Audit + governance
// • Catalog every secret (name, owner, last-rotated, environment)
// • Quarterly review — anything not used? rotate or delete
// • Compliance: SOC 2, ISO 27001 — secret management is a control they audit
// • Track 'secret age' as a metric — encourages rotation

// 14) UI / config self-service
// • Provide a CLI for engineers to read secrets they OWN — no shoulder-surfing engineers asking SREs
// • SSO + JIT access — get the secret only when needed, time-boxed
// • Audit log every access for compliance

// 15) Common bugs / mistakes
// • Hard-coded secret in tests / examples — even 'fake' secrets are flagged by scanners
// • Same secret across dev / staging / prod — one breach affects all
// • Storing JWT secret in code — instant compromise on leak
// • Long-lived static API keys — rotate or replace with workload identity
// • Logging full request bodies in dev → secrets in logs → propagation to log aggregators
// • Skipping rotation when an employee leaves — revoke + rotate
// • .env.example with REAL values — defeats the purpose
// • Committing secrets then 'git revert' — they're still in history; rotate + clean
// • Not scanning private repos — gitleaks doesn't care if it's public or private

// Never commit secrets. Inject at deploy time from a vault.
// Tools: AWS/GCP Secrets Manager, HashiCorp Vault, Doppler, 1Password CLI.
// Detect leaks with gitleaks / trufflehog in CI.