Secrets with SOPS & Age

Photo Credits: Unsplash and SBTS2018 on FlatIcon

Definitions

If you're not familiar with the concept of Secrets Management, here's a quick definition from Imperva:

Secret management is a practice that allows developers to securely store sensitive data such as passwords, keys, and tokens, in a secure environment with strict access controls.
For small software projects, secret management can be simple to achieve. But as teams and software codebases grow, there are additional secrets spread across an application ecosystem, making them more difficult to manage. Microservices, development tools, containers, orchestrators, and API connections all require secrets to perform their functions, and these must be stored and delivered in a secure manner.
It is very common to have secrets hard-coded into scripts, configurations, or source code, making them easily accessible to attackers. Secret management solutions can ensure that sensitive information is never embedded into any artifact in plaintext, saving secrets separately from code, and providing an audit trail by enforcing privilege-based sessions for all access attempts.

Introduction

Most developers - and homelab admins - would agree that storing credentials in code is less than ideal.

However, many - or, at least, me - have been skeptical about implementing secrets management not because of its usefulness but because of its value proposition: the freely available solutions are, for the most part, massively more complex than what I need. HashiCorp Vault is a fantastic tool, but I don't want to run it in my homelab for casual and infrequent use - especially if the code I'm working on is actually the provisioning material for the homelab itself (see: "chicken and egg").

Fortunately, a friend has recently introduced me to a lightweight, convenient, and open source solution: SOPS + Age.

About The Tools

SOPS: Secrets OPerationS

SOPS is "an editor of encrypted files that supports YAML, JSON, ENV, INI and BINARY formats and encrypts with AWS KMS, GCP KMS, Azure Key Vault, age, and PGP."

I use SOPS for two reasons:

Convenience. Once configured, you can use SOPS to create or edit encrypted files without running a single encrypt/decrypt command. Just sops filename.
Automation. There's a fantastic Terraform provider for SOPS. (If the value of that isn't immediately clear, stay tuned for a future article on how I integrate the two.)

DEK/KEK Encryption

SOPS uses a DEK/KEK model for encryption. When it creates or encrypts a file, it does so with a randomly-generated 256 bit data encryption key (DEK), then encrypts that data key with the configured or specified key encryption key(s) (KEK).

Multiple KEKs can be used, allowing multiple private keys to decrypt the secrets. This is accomplished by storing multiple encrypted copies of the DEK in the encrypted data file, each associated with its own public key.

FiloSottile/age

age is "a simple, modern and secure file encryption tool, format, and Go library."

SOPS can use several different key/encryption providers, including age. Most of the rest are cloud-based, e.g. AWS KMS. age is local, lightweight, and extremely easy to set up.

Testing it Out

Start a Dev Environment

To proactively keep my host system 'clean,' I do all of my testing in containers or VMs. Feel free to substitute your own approach.

root@host:/> docker run -it debian:latest /bin/bash

Install Tools

root@2b92d1f7368b:/:> apt update && apt install curl vim -y

# Output Removed here, and elsewhere when unnecessary

root@2b92d1f7368b:/:> cd /opt

# Download the binary - check for the latest one, of course!
root@2b92d1f7368b:/opt:> curl -LO https://github.com/getsops/sops/releases/download/v3.8.0-rc.1/sops-v3.8.0-rc.1.linux.amd64

# Download the checksums file, certificate and signature
root@2b92d1f7368b:/opt:> curl -LO https://github.com/getsops/sops/releases/download/v3.8.0-rc.1/sops-v3.8.0-rc.1.checksums.txt

root@2b92d1f7368b:/opt:> curl -LO https://github.com/getsops/sops/releases/download/v3.8.0-rc.1/sops-v3.8.0-rc.1.checksums.pem

root@2b92d1f7368b:/opt:> curl -LO https://github.com/getsops/sops/releases/download/v3.8.0-rc.1/sops-v3.8.0-rc.1.checksums.sig

root@2b92d1f7368b:/opt:> sha256sum -c sops-v3.8.0-rc.1.checksums.txt --ignore-missing
sops-v3.8.0-rc.1.linux.amd64: OK

# Download cosign for even more verification
root@2b92d1f7368b:/opt:> LATEST_VERSION=$(curl https://api.github.com/repos/sigstore/cosign/releases/latest \
| grep tag_name | cut -d : -f2 | tr -d "v\", ")

root@2b92d1f7368b:/opt:> curl -O -L "https://github.com/sigstore/cosign/releases/latest/download/cosign_${LATEST_VERSION}_amd64.deb"

root@2b92d1f7368b:/opt:> dpkg -i cosign_${LATEST_VERSION}\_amd64.deb

root@2b92d1f7368b:/opt:> cosign verify-blob sops-v3.8.0-rc.1.checksums.txt --certificate sops-v3.8.0-rc.1.checksums.pem \
--signature sops-v3.8.0-rc.1.checksums.sig --certificate-identity-regexp=https://github.com/getsops \
--certificate-oidc-issuer=https://token.actions.githubusercontent.com
Verified OK

# Move the binary in to your PATH
root@2b92d1f7368b:/opt:> mv sops-v3.8.0-rc.1.linux.amd64 /usr/local/bin/sops

# Make the binary executable
root@2b92d1f7368b:/opt:> chmod +x /usr/local/bin/sops

root@2b92d1f7368b:/opt:> apt install age -y

root@2b92d1f7368b:/opt:> mkdir -p ~/.config/sops/age

root@2b92d1f7368b:/opt:> age-keygen -o ~/.config/sops/age/keys.txt
Public key: age1rwrumcadz6jk99j2gxjw3nsywdcyuwyc9858gqg5k3wvq8w6gqvqn8qj47

root@2b92d1f7368b:/opt:> cat ~/.config/sops/age/keys.txt
# created: 2023-08-26T00:16:25Z
# public key: age1rwrumcadz6jk99j2gxjw3nsywdcyuwyc9858gqg5k3wvq8w6gqvqn8qj47
AGE-SECRET-KEY-REDACTEDKEYHERE

Using SOPS+AGE

# Create a config file so SOPS knows what key to use
root@2b92d1f7368b:/opt:> cat \<\< EOF \> .sops.yaml
> creation_rules:
>   - age: "age1rwrumcadz6jk99j2gxjw3nsywdcyuwyc9858gqg5k3wvq8w6gqvqn8qj47"
> EOF
# obviously - use your own public key

root@2b92d1f7368b:/opt:> sops test.yaml

SOPS will then drop you into vim, with the following yaml template:

hello: Welcome to SOPS! Edit this file as you please!
example_key: example_value
# Example comment
example_array:
  - example_value1
  - example_value2
example_number: 1234.56789
example_booleans:
  - true
  - false

Note: It's possible to create and edit "raw" (i.e. unstructured) files with SOPS. However, with specific, structured filetypes (i.e., yaml) it's capable of "targeted" encryption. Essentially, it will only encrypt the values of our yaml file and not the keys. From the README:

sops uses the file extension to decide which encryption method to use on the file content. YAML, JSON, ENV, and INI files are treated as trees of data, and key/values are extracted from the files to only encrypt the leaf values. The tree structure is also used to check the integrity of the file.

After saving the file, you can view the encrypted version with a cat test.yaml:

hello: ENC[AES256_GCM,data:fNNoqMbZLW2n5vz4nd9KyBPpkGbdJTE7YZZeH4sJ6t3/GRlIF/QzEnD7PndClQ==,iv:2li3yVay4KBGK/EgFoDD4vXtAffnffruGJj8huUMFZo=,tag:f60yzaNqSL2KYJHpSX82og==,type:str]
example_key: ENC[AES256_GCM,data:Wt03RCvFzBHfapwFIw==,iv:e9o8thLfXbsc76C0FPLyQNBaDyChBcbtby/WnPYApIg=,tag:RM5NEUYM2UK+hNXcuOVyzA==,type:str]
#ENC[AES256_GCM,data:kqFjXMan8Dn1qRYQ2k/hCw==,iv:N/2QmyeoGqzuw7w35FWiC53jYoRJAFnho9C0s91DCZM=,tag:KWOtfFmraQBfCoUK2mnsjw==,type:comment]
example_array:
  - ENC[AES256_GCM,data:55u6LOQSVag/7uxm8L0=,iv:tDlUjrMSNaNCtbT6wrL8zEWLiVO/VDc8N08cLbv033o=,tag:1W001s6bNZDemofAVDYIzw==,type:str]
  - ENC[AES256_GCM,data:GU8Ur24YxlA0EQtcIy4=,iv:lNYb9YKlDTfZ2g7+qFrYwaQnO5eWn2ulu+/UItkfr14=,tag:qZy8qqKNbD5O5c2i6lwY3Q==,type:str]
example_number: ENC[AES256_GCM,data:WewtHzjhHj5FBQ==,iv:dfmOOYAOPIV0cVnc6E5S4VaEjb+MU+DZhaP7Ldi9DVI=,tag:t0H09QMUieIjKXe0MZALpw==,type:float]
example_booleans:
  - ENC[AES256_GCM,data:A+6Flg==,iv:/5qpoHX70v3X4hywCBvvzXVQQLYNKFactbkwite+EHs=,tag:CKgbO5lNXIo9GYLoMrOPhQ==,type:bool]
  - ENC[AES256_GCM,data:KDYY+i8=,iv:G/hEXLZ8G9b6CHjajWkHNm0XIO7vRrxsE+bOS8joPQU=,tag:z0Bp71+2+4gxyrJkqLLHbg==,type:bool]
sops:
  kms: []
  gcp_kms: []
  azure_kv: []
  hc_vault: []
  age:
    - recipient: age1rwrumcadz6jk99j2gxjw3nsywdcyuwyc9858gqg5k3wvq8w6gqvqn8qj47
      enc: |
        -----BEGIN AGE ENCRYPTED FILE-----
        YWdlLWVuY3J5cHRpb24ub3JnL3YxCi0+IFgyNTUxOSBWMUtBcStYdCt4TGkrYlVL
        MkhIWlIwVzBNRkFqb3pZd2dNaE9ib2JQYm5VCnZCUURvdmJFS01CSE9qbTcvVnhw
        Nlk4QjQ0SXdKVVo4RjF4MkFPckFIaTgKLS0tIFFSeVhxZEZ4cGpLeGsrWnpQMEg2
        eDR0Z1hCZjdmUFdoaHlyS1ZsdTV6Mm8KxskZTV+o2yJ506pijDfXYHPeNuLkeSsD
        qJnq1M5A5/WnfKBn29AuthMTP45oo4zj3rFB3JQnqipaken0YijhTQ==
        -----END AGE ENCRYPTED FILE-----
  lastmodified: '2023-08-29T00:28:56Z'
  mac: ENC[AES256_GCM,data:ykzKTV5NPo547Uywc+3NhrrOhuyc3FoxmZH0DdAV2fpSSYYFGilCTWwvsS1QcRFaI4XCUtzVNu9pnhvNNkbn9soHcjQNCW4lB08rNVjBm2fCYh+dkULWzeG52NZT+szbynmAvxYcgitnu+pBB4h/oZ+BvJLLb9LrOiXQXu6TH4E=,iv:jzfxc0N13DDLiAUV+cWj4E+86OsUSijDUdGVE3XDcZc=,tag:PfAdq92sSmXEQz8ESQHjxg==,type:str]
  pgp: []
  unencrypted_suffix: _unencrypted
  version: 3.8.0-rc.1

You can always edit the file again with sops test.yaml, or decrypt it to stdout with sops -d test.yaml.

Conclusion

And thus, you can securely store your encrypted secrets within your repositories (assuming you keep that age key where it belongs!).

SOPS + age will become even more useful when used with tools like Terraform that support integrations. In the case of Terraform - once the provider is configured - a simple terraform apply will call sops in the background to fetch specified data. But... more on that later.

Stay tuned!

Author

Tags