Secrets Management in Infrastructure: Storage, Distribution and Rotation

Secrets – passwords, API keys, certificates, connection strings – are sensitive assets. Storing them in code, config files in the repo, or unprotected environment variables exposes them to leakage (including in git history), confusion between environments, and unauthorized access. Proper secrets management separates secure storage from controlled distribution and application use. This article reviews principles and tools.

Principles: secrets do not go into source code or git. Not in comments, default values or sample config. Access to secrets requires authentication (identity) and authorization (what may be read). Least Privilege: every service and user gets only the secrets required. Documentation: where each secret is stored, who may access, and when rotation was performed.

Central secrets management systems: HashiCorp Vault, AWS Secrets Manager, Azure Key Vault, GCP Secret Manager – all provide encrypted storage, API access with authentication (IAM, tokens), and rotation capability. The application receives a secret at runtime (e.g. at startup or before DB call) and does not store the raw value in a file. Define policy that limits which roles can access which paths.

Development environment: dummy values or access to dev vault/secret store – not real production secrets. Local .env files that do not go into git (listed in .gitignore) are acceptable for local development, with a warning: do not upload .env.example with real values.

Production environment: secrets are injected via secrets management system or orchestrator. In Kubernetes: Secrets resource (prefer encryption at rest and RBAC); or integration with External Secrets Operator / Vault Agent that sync from Vault to K8s. Do not pass secrets in command line or environment in plain text in YAML in git.

Rotation: replacing secrets periodically or after an event (employee departure, suspected leakage). Automatic rotation – when the system supports it (e.g. RDS with Secrets Manager) – reduces burden and risk. Plan for the application to reconnect or load an updated secret without downtime (e.g. periodic load or receiving a signal).

CI/CD: the pipeline needs access to registries, cloud accounts and test systems. Using OIDC or short-lived credentials is preferred over fixed keys. Pipeline secrets are stored in the CI secrets management system (GitHub Secrets, GitLab CI variables, Vault) and injected as env without printing in logs.

Audit and compliance: logging access to secrets (who read, when) enables post-incident investigation and compliance. Limiting access by need-to-know and expiration policy reduces risk.

Migration from keys in code: if secrets already exist in code or config – migrate them to a management system gradually, update the code for dynamic loading, and rotate the old secrets to invalidate leaked values.

In summary: secrets management requires not putting secrets in code or git, using a secure central system, controlled injection into run environments (including K8s and CI), and rotation and documentation. A systematic approach reduces leakage and enables fast response in security events.