Using SOPS to manage secrets
When do that ?
When developing or deploying a new app/service, you should probably create a new secret or update an existing one.
What is SOPS ?
For k8saas, we managed the secret with sops.
Motivation to use SOPS
Automating the distribution of secrets and credentials to components of an infrastructure is a hard problem. We know how to encrypt secrets and share them between humans, but extending that trust to systems is difficult. Particularly when these systems follow devops principles and are created and destroyed without human intervention. The issue boils down to establishing the initial trust of a system that just joined the infrastructure, and providing it access to the secrets it needs to configure itself.
Operational requirements
Secrets must be stored in YAML or JSON files for easy integration into hiera Secrets must be stored in GIT, and when a new CloudFormation stack is built, the current HEAD is pinned to the stack. (This allows secrets to be changed in GIT without impacting the current stack that may autoscale). Entries must be encrypted separately. Encrypting entire files as blobs makes git conflict resolution almost impossible. Encrypting each entry separately is much easier to manage. Secrets must always be encrypted on disk (admin laptop, upstream git repo, jenkins and S3) and only be decrypted on the target systems
What to do ?
0) Install sops
brew install sops
1) First, create a JSON file (secrets.json) with your secret
We recommend using the following JSON format (to not impact too much the current deployments)
{
"locals": {
"key1": "value1",
"key2": "value2"
}
}
Note that key1 and key2 are names of variables that will be accessed from the variable in step #4
2) Second, get the key.id
e.g.
# Read the key id:
$ az keyvault key show --name sops-key --vault-name k8saasTestKeyvault --query key.kid
The previous command should gives:
https://k8saastestkeyvault.vault.azure.net/keys/sops-key/some-string
If you're facing an issue, probably the access policy is incorrect. To fix it, access to the Azure Portal, go to the keyvault resource and update the access policy.
3) Third, encrypt your secret, replace the ked.id by the value of the previous step
sops --encrypt --azure-kv <https://k8saastestkeyvault.vault.azure.net/keys/sops-key/some-string> secrets.json > secrets.enc.json
4) adapt your deployment
Here's an exemple we use for our terragrunt deployment.
for instance.hcl:
secrets_vars = jsondecode(sops_decrypt_file("${get_parent_terragrunt_dir()}/${path_relative_to_include()}/secrets.enc.json"))