diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a33b37e77..885e14376 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -3,7 +3,7 @@ The SOPS project welcomes contributions from everyone. Here are a few guidelines and instructions if you are thinking of helping with the development of SOPS. -# Getting started +## Getting started - Make sure you have Go 1.19 or greater installed. You can find information on how to install Go [here](https://go.dev/doc/install) @@ -11,21 +11,22 @@ and instructions if you are thinking of helping with the development of SOPS. - Run the tests with `make test`. They should all pass. - Fork the project on GitHub. - Add your fork to Git's remotes: - + If you use SSH authentication: `git remote add git@github.com:/sops.git`. - + Otherwise: `git remote add https://github.com//sops.git`. + - If you use SSH authentication: + `git remote add git@github.com:/sops.git`. + - Otherwise: `git remote add https://github.com//sops.git`. - Make any changes you want to SOPS, commit them, and push them to your fork. - **Create a pull request against `main`**, and a maintainer will come by and review your code. They may ask for some changes, and hopefully your contribution will be merged! -# Guidelines +## Guidelines - Unless it's particularly hard, changes that fix a bug should have a regression test to make sure that the bug is not introduced again. - New features and changes to existing features should be documented, and, if possible, tested. -# Communication +## Communication If you need any help contributing to SOPS, several maintainers are on the [`#sops-dev` channel](https://cloud-native.slack.com/archives/C059800AJBT) on diff --git a/README.rst b/README.rst index 7f4b9dba1..c6a40453b 100644 --- a/README.rst +++ b/README.rst @@ -1,7 +1,7 @@ SOPS: Secrets OPerationS ======================== -**sops** is an editor of encrypted files that supports YAML, JSON, ENV, INI and BINARY +**SOPS** is an editor of encrypted files that supports YAML, JSON, ENV, INI and BINARY formats and encrypts with AWS KMS, GCP KMS, Azure Key Vault, age, and PGP. (`demo `_) @@ -10,7 +10,7 @@ formats and encrypts with AWS KMS, GCP KMS, Azure Key Vault, age, and PGP. ------------ .. image:: https://pkg.go.dev/badge/github.com/getsops/sops/v3.svg - :target: https://pkg.go.dev/github.com/getsops/sops/v3 + :target: https://pkg.go.dev/github.com/getsops/sops/v3 Download -------- @@ -25,10 +25,10 @@ For the adventurous, unstable features are available in the `main` branch, which .. code:: bash - $ mkdir -p $GOPATH/src/github.com/getsops/sops/ - $ git clone https://github.com/getsops/sops.git $GOPATH/src/github.com/getsops/sops/ - $ cd $GOPATH/src/github.com/getsops/sops/ - $ make install + $ mkdir -p $GOPATH/src/github.com/getsops/sops/ + $ git clone https://github.com/getsops/sops.git $GOPATH/src/github.com/getsops/sops/ + $ cd $GOPATH/src/github.com/getsops/sops/ + $ make install (requires Go >= 1.19) @@ -36,14 +36,14 @@ If you don't have Go installed, set it up with: .. code:: bash - $ {apt,yum,brew} install golang - $ echo 'export GOPATH=~/go' >> ~/.bashrc - $ source ~/.bashrc - $ mkdir $GOPATH + $ {apt,yum,brew} install golang + $ echo 'export GOPATH=~/go' >> ~/.bashrc + $ source ~/.bashrc + $ mkdir $GOPATH Or whatever variation of the above fits your system and shell. -To use **sops** as a library, take a look at the `decrypt package `_. +To use **SOPS** as a library, take a look at the `decrypt package `_. .. sectnum:: .. contents:: Table of Contents @@ -51,7 +51,7 @@ To use **sops** as a library, take a look at the `decrypt package `_ to communicate with AWS KMS. It will automatically read the credentials from the ``~/.aws/credentials`` file which can be created with the ``aws configure`` command. An example of the ``~/.aws/credentials`` file is shown below: -.. code:: +.. code:: sh - $ cat ~/.aws/credentials - [default] - aws_access_key_id = AKI..... - aws_secret_access_key = mw...... + $ cat ~/.aws/credentials + [default] + aws_access_key_id = AKI..... + aws_secret_access_key = mw...... In addition to the ``~/.aws/credentials`` file, you can also use the ``AWS_ACCESS_KEY_ID`` and ``AWS_SECRET_ACCESS_KEY`` environment variables to specify your credentials: .. code:: bash - export AWS_ACCESS_KEY_ID="AKI......" - export AWS_SECRET_ACCESS_KEY="mw......" + export AWS_ACCESS_KEY_ID="AKI......" + export AWS_SECRET_ACCESS_KEY="mw......" For more information and additional environment variables, see `specifying credentials `_. @@ -92,22 +92,22 @@ separated, in the **SOPS_PGP_FP** env variable. .. code:: bash - export SOPS_PGP_FP="85D77543B3D624B63CEA9E6DBC17301B491B3F21,E60892BB9BD89A69F759A1A0A3D652173B763E8F" + export SOPS_PGP_FP="85D77543B3D624B63CEA9E6DBC17301B491B3F21,E60892BB9BD89A69F759A1A0A3D652173B763E8F" Note: you can use both PGP and KMS simultaneously. Then simply call ``sops`` with a file path as argument. It will handle the encryption/decryption transparently and open the cleartext file in an editor -.. code:: shell +.. code:: sh - $ sops mynewtestfile.yaml - mynewtestfile.yaml doesn't exist, creating it. - please wait while an encryption key is being generated and stored in a secure fashion - file written to mynewtestfile.yaml + $ sops mynewtestfile.yaml + mynewtestfile.yaml doesn't exist, creating it. + please wait while an encryption key is being generated and stored in a secure fashion + file written to mynewtestfile.yaml Editing will happen in whatever ``$EDITOR`` is set to, or, if it's not set, in vim. -Keep in mind that sops will wait for the editor to exit, and then try to reencrypt +Keep in mind that SOPS will wait for the editor to exit, and then try to reencrypt the file. Some GUI editors (atom, sublime) spawn a child process and then exit immediately. They usually have an option to wait for the main editor window to be closed before exiting. See `#127 `_ for @@ -126,25 +126,25 @@ The resulting encrypted file looks like this: key: |- ENC[AES256_GCM,data:Ea3kL5O5U8=,iv:DM=,aad:FKA=,tag:EA==] an_array: - - ENC[AES256_GCM,data:v8jQ=,iv:HBE=,aad:21c=,tag:gA==] - - ENC[AES256_GCM,data:X10=,iv:o8=,aad:CQ=,tag:Hw==] - - ENC[AES256_GCM,data:KN=,iv:160=,aad:fI4=,tag:tNw==] + - ENC[AES256_GCM,data:v8jQ=,iv:HBE=,aad:21c=,tag:gA==] + - ENC[AES256_GCM,data:X10=,iv:o8=,aad:CQ=,tag:Hw==] + - ENC[AES256_GCM,data:KN=,iv:160=,aad:fI4=,tag:tNw==] sops: kms: - - created_at: 1441570389.775376 - enc: CiC....Pm1Hm - arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e - - created_at: 1441570391.925734 - enc: Ci...awNx - arn: arn:aws:kms:ap-southeast-1:656532927350:key/9006a8aa-0fa6-4c14-930e-a2dfb916de1d + - created_at: 1441570389.775376 + enc: CiC....Pm1Hm + arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e + - created_at: 1441570391.925734 + enc: Ci...awNx + arn: arn:aws:kms:ap-southeast-1:656532927350:key/9006a8aa-0fa6-4c14-930e-a2dfb916de1d pgp: - - fp: 85D77543B3D624B63CEA9E6DBC17301B491B3F21 - created_at: 1441570391.930042 - enc: | - -----BEGIN PGP MESSAGE----- - hQIMA0t4uZHfl9qgAQ//UvGAwGePyHuf2/zayWcloGaDs0MzI+zw6CmXvMRNPUsA - ...=oJgS - -----END PGP MESSAGE----- + - fp: 85D77543B3D624B63CEA9E6DBC17301B491B3F21 + created_at: 1441570391.930042 + enc: | + -----BEGIN PGP MESSAGE----- + hQIMA0t4uZHfl9qgAQ//UvGAwGePyHuf2/zayWcloGaDs0MzI+zw6CmXvMRNPUsA + ...=oJgS + -----END PGP MESSAGE----- A copy of the encryption/decryption key is stored securely in each KMS and PGP block. As long as one of the KMS or PGP method is still usable, you will be able @@ -152,19 +152,19 @@ to access your data. To decrypt a file in a ``cat`` fashion, use the ``-d`` flag: -.. code:: bash +.. code:: sh - $ sops -d mynewtestfile.yaml + $ sops -d mynewtestfile.yaml -``sops`` encrypted files contain the necessary information to decrypt their content. -All a user of ``sops`` needs is valid AWS credentials and the necessary +SOPS encrypted files contain the necessary information to decrypt their content. +All a user of SOPS needs is valid AWS credentials and the necessary permissions on KMS keys. -Given that, the only command a ``sops`` user needs is: +Given that, the only command a SOPS user needs is: -.. code:: bash +.. code:: sh - $ sops + $ sops `` will be opened, decrypted, passed to a text editor (vim by default), encrypted if modified, and saved back to its original location. All of these @@ -173,13 +173,13 @@ steps, apart from the actual editing, are transparent to the user. Test with the dev PGP key ~~~~~~~~~~~~~~~~~~~~~~~~~ -If you want to test **sops** without having to do a bunch of setup, you can use +If you want to test **SOPS** without having to do a bunch of setup, you can use the example files and pgp key provided with the repository:: - $ git clone https://github.com/getsops/sops.git - $ cd sops - $ gpg --import pgp/sops_functional_tests_key.asc - $ sops example.yaml + $ git clone https://github.com/getsops/sops.git + $ cd sops + $ gpg --import pgp/sops_functional_tests_key.asc + $ sops example.yaml This last step will decrypt ``example.yaml`` using the test private key. @@ -193,11 +193,11 @@ encrypting files. It's recommended to use age over PGP, if possible. You can encrypt a file for one or more age recipients (comma separated) using the ``--age`` option or the **SOPS_AGE_RECIPIENTS** environment variable: -.. code:: bash +.. code:: sh - $ sops --encrypt --age age1yt3tfqlfrwdwx0z0ynwplcr6qxcxfaqycuprpmy89nr83ltx74tqdpszlw test.yaml > test.enc.yaml + $ sops --encrypt --age age1yt3tfqlfrwdwx0z0ynwplcr6qxcxfaqycuprpmy89nr83ltx74tqdpszlw test.yaml > test.enc.yaml -When decrypting a file with the corresponding identity, sops will look for a +When decrypting a file with the corresponding identity, SOPS will look for a text file name ``keys.txt`` located in a ``sops`` subdirectory of your user configuration directory. On Linux, this would be ``$XDG_CONFIG_HOME/sops/age/keys.txt``. On macOS, this would be ``$HOME/Library/Application Support/sops/age/keys.txt``. On @@ -210,7 +210,7 @@ The contents of this key file should be a list of age X25519 identities, one per line. Lines beginning with ``#`` are considered comments and ignored. Each identity will be tried in sequence until one is able to decrypt the data. -Encrypting with SSH keys via age is not yet supported by sops. +Encrypting with SSH keys via age is not yet supported by SOPS. Encrypting using GCP KMS @@ -219,35 +219,37 @@ GCP KMS uses `Application Default Credentials `_. If you already logged in using -.. code:: bash +.. code:: sh + + $ gcloud auth login - $ gcloud auth login +you can enable application default credentials using the sdk: -you can enable application default credentials using the sdk:: +.. code:: sh - $ gcloud auth application-default login + $ gcloud auth application-default login Encrypting/decrypting with GCP KMS requires a KMS ResourceID. You can use the cloud console the get the ResourceID or you can create one using the gcloud sdk: -.. code:: bash +.. code:: sh - $ gcloud kms keyrings create sops --location global - $ gcloud kms keys create sops-key --location global --keyring sops --purpose encryption - $ gcloud kms keys list --location global --keyring sops + $ gcloud kms keyrings create sops --location global + $ gcloud kms keys create sops-key --location global --keyring sops --purpose encryption + $ gcloud kms keys list --location global --keyring sops - # you should see - NAME PURPOSE PRIMARY_STATE - projects/my-project/locations/global/keyRings/sops/cryptoKeys/sops-key ENCRYPT_DECRYPT ENABLED + # you should see + NAME PURPOSE PRIMARY_STATE + projects/my-project/locations/global/keyRings/sops/cryptoKeys/sops-key ENCRYPT_DECRYPT ENABLED Now you can encrypt a file using:: - $ sops --encrypt --gcp-kms projects/my-project/locations/global/keyRings/sops/cryptoKeys/sops-key test.yaml > test.enc.yaml + $ sops --encrypt --gcp-kms projects/my-project/locations/global/keyRings/sops/cryptoKeys/sops-key test.yaml > test.enc.yaml And decrypt it using:: - $ sops --decrypt test.enc.yaml + $ sops --decrypt test.enc.yaml Encrypting using Azure Key Vault ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -271,57 +273,57 @@ For example, you can use a Service Principal with the following environment vari .. code:: bash - AZURE_TENANT_ID - AZURE_CLIENT_ID - AZURE_CLIENT_SECRET + AZURE_TENANT_ID + AZURE_CLIENT_ID + AZURE_CLIENT_SECRET You can create a Service Principal using the CLI like this: -.. code:: bash +.. code:: sh - $ az ad sp create-for-rbac -n my-keyvault-sp + $ az ad sp create-for-rbac -n my-keyvault-sp - { - "appId": "", - "displayName": "my-keyvault-sp", - "name": "http://my-keyvault-sp", - "password": "", - "tenant": "" - } + { + "appId": "", + "displayName": "my-keyvault-sp", + "name": "http://my-keyvault-sp", + "password": "", + "tenant": "" + } The `appId` is the client ID, and the `password` is the client secret. Encrypting/decrypting with Azure Key Vault requires the resource identifier for a key. This has the following form:: - https://${VAULT_URL}/keys/${KEY_NAME}/${KEY_VERSION} + https://${VAULT_URL}/keys/${KEY_NAME}/${KEY_VERSION} To create a Key Vault and assign your service principal permissions on it from the commandline: -.. code:: bash +.. code:: sh - # Create a resource group if you do not have one: - $ az group create --name sops-rg --location westeurope - # Key Vault names are globally unique, so generate one: - $ keyvault_name=sops-$(uuidgen | tr -d - | head -c 16) - # Create a Vault, a key, and give the service principal access: - $ az keyvault create --name $keyvault_name --resource-group sops-rg --location westeurope - $ az keyvault key create --name sops-key --vault-name $keyvault_name --protection software --ops encrypt decrypt - $ az keyvault set-policy --name $keyvault_name --resource-group sops-rg --spn $AZURE_CLIENT_ID \ - --key-permissions encrypt decrypt - # Read the key id: - $ az keyvault key show --name sops-key --vault-name $keyvault_name --query key.kid + # Create a resource group if you do not have one: + $ az group create --name sops-rg --location westeurope + # Key Vault names are globally unique, so generate one: + $ keyvault_name=sops-$(uuidgen | tr -d - | head -c 16) + # Create a Vault, a key, and give the service principal access: + $ az keyvault create --name $keyvault_name --resource-group sops-rg --location westeurope + $ az keyvault key create --name sops-key --vault-name $keyvault_name --protection software --ops encrypt decrypt + $ az keyvault set-policy --name $keyvault_name --resource-group sops-rg --spn $AZURE_CLIENT_ID \ + --key-permissions encrypt decrypt + # Read the key id: + $ az keyvault key show --name sops-key --vault-name $keyvault_name --query key.kid - https://sops.vault.azure.net/keys/sops-key/some-string + https://sops.vault.azure.net/keys/sops-key/some-string Now you can encrypt a file using:: - $ sops --encrypt --azure-kv https://sops.vault.azure.net/keys/sops-key/some-string test.yaml > test.enc.yaml + $ sops --encrypt --azure-kv https://sops.vault.azure.net/keys/sops-key/some-string test.yaml > test.enc.yaml And decrypt it using:: - $ sops --decrypt test.enc.yaml + $ sops --decrypt test.enc.yaml Encrypting using Hashicorp Vault @@ -331,58 +333,58 @@ We assume you have an instance (or more) of Vault running and you have privilege To easily deploy Vault locally: (DO NOT DO THIS FOR PRODUCTION!!!) -.. code:: bash +.. code:: sh - $ docker run -d -p8200:8200 vault:1.2.0 server -dev -dev-root-token-id=toor + $ docker run -d -p8200:8200 vault:1.2.0 server -dev -dev-root-token-id=toor -.. code:: bash +.. code:: sh + + $ # Substitute this with the address Vault is running on + $ export VAULT_ADDR=http://127.0.0.1:8200 + + $ # this may not be necessary in case you previously used `vault login` for production use + $ export VAULT_TOKEN=toor + + $ # to check if Vault started and is configured correctly + $ vault status + Key Value + --- ----- + Seal Type shamir + Initialized true + Sealed false + Total Shares 1 + Threshold 1 + Version 1.2.0 + Cluster Name vault-cluster-618cc902 + Cluster ID e532e461-e8f0-1352-8a41-fc7c11096908 + HA Enabled false + + $ # It is required to enable a transit engine if not already done (It is suggested to create a transit engine specifically for SOPS, in which it is possible to have multiple keys with various permission levels) + $ vault secrets enable -path=sops transit + Success! Enabled the transit secrets engine at: sops/ + + $ # Then create one or more keys + $ vault write sops/keys/firstkey type=rsa-4096 + Success! Data written to: sops/keys/firstkey + + $ vault write sops/keys/secondkey type=rsa-2048 + Success! Data written to: sops/keys/secondkey - $ # Substitute this with the address Vault is running on - $ export VAULT_ADDR=http://127.0.0.1:8200 - - $ # this may not be necessary in case you previously used `vault login` for production use - $ export VAULT_TOKEN=toor - - $ # to check if Vault started and is configured correctly - $ vault status - Key Value - --- ----- - Seal Type shamir - Initialized true - Sealed false - Total Shares 1 - Threshold 1 - Version 1.2.0 - Cluster Name vault-cluster-618cc902 - Cluster ID e532e461-e8f0-1352-8a41-fc7c11096908 - HA Enabled false - - $ # It is required to enable a transit engine if not already done (It is suggested to create a transit engine specifically for sops, in which it is possible to have multiple keys with various permission levels) - $ vault secrets enable -path=sops transit - Success! Enabled the transit secrets engine at: sops/ - - $ # Then create one or more keys - $ vault write sops/keys/firstkey type=rsa-4096 - Success! Data written to: sops/keys/firstkey - - $ vault write sops/keys/secondkey type=rsa-2048 - Success! Data written to: sops/keys/secondkey - - $ vault write sops/keys/thirdkey type=chacha20-poly1305 - Success! Data written to: sops/keys/thirdkey - - $ sops --encrypt --hc-vault-transit $VAULT_ADDR/v1/sops/keys/firstkey vault_example.yml - - $ cat < .sops.yaml - creation_rules: - - path_regex: \.dev\.yaml$ - hc_vault_transit_uri: "$VAULT_ADDR/v1/sops/keys/secondkey" - - path_regex: \.prod\.yaml$ - hc_vault_transit_uri: "$VAULT_ADDR/v1/sops/keys/thirdkey" - EOF - - $ sops --verbose -e prod/raw.yaml > prod/encrypted.yaml + $ vault write sops/keys/thirdkey type=chacha20-poly1305 + Success! Data written to: sops/keys/thirdkey + + $ sops --encrypt --hc-vault-transit $VAULT_ADDR/v1/sops/keys/firstkey vault_example.yml + + $ cat < .sops.yaml + creation_rules: + - path_regex: \.dev\.yaml$ + hc_vault_transit_uri: "$VAULT_ADDR/v1/sops/keys/secondkey" + - path_regex: \.prod\.yaml$ + hc_vault_transit_uri: "$VAULT_ADDR/v1/sops/keys/thirdkey" + EOF + + $ sops --verbose -e prod/raw.yaml > prod/encrypted.yaml Adding and removing keys ~~~~~~~~~~~~~~~~~~~~~~~~ @@ -395,15 +397,15 @@ the environment variables ``SOPS_KMS_ARN``, ``SOPS_PGP_FP``, ``SOPS_GCP_KMS_IDS` parameters again. Master PGP and KMS keys can be added and removed from a ``sops`` file in one of -three ways:: +three ways: -1. By using a .sops.yaml file and the ``updatekeys`` command. +1. By using a ``.sops.yaml`` file and the ``updatekeys`` command. 2. By using command line flags. 3. By editing the file directly. -The sops team recommends the ``updatekeys`` approach. +The SOPS team recommends the ``updatekeys`` approach. ``updatekeys`` command @@ -422,11 +424,11 @@ separated list. 85D77543B3D624B63CEA9E6DBC17301B491B3F21, FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4 -.. code:: bash +.. code:: sh - $ sops updatekeys test.enc.yaml + $ sops updatekeys test.enc.yaml -Sops will prompt you with the changes to be made. This interactivity can be +SOPS will prompt you with the changes to be made. This interactivity can be disabled by supplying the ``-y`` flag. Command Line @@ -442,21 +444,21 @@ Note that ``-r`` or ``--rotate`` is mandatory in this mode. Not specifying rotate will ignore the ``--add-*`` options. Use ``updatekeys`` if you want to add a key without rotating the data key. -.. code:: bash +.. code:: sh - # add a new pgp key to the file and rotate the data key - $ sops -r -i --add-pgp 85D77543B3D624B63CEA9E6DBC17301B491B3F21 example.yaml + # add a new pgp key to the file and rotate the data key + $ sops -r -i --add-pgp 85D77543B3D624B63CEA9E6DBC17301B491B3F21 example.yaml - # remove a pgp key from the file and rotate the data key - $ sops -r -i --rm-pgp 85D77543B3D624B63CEA9E6DBC17301B491B3F21 example.yaml + # remove a pgp key from the file and rotate the data key + $ sops -r -i --rm-pgp 85D77543B3D624B63CEA9E6DBC17301B491B3F21 example.yaml Direct Editing ************** Alternatively, invoking ``sops`` with the flag **-s** will display the master keys -while editing. This method can be used to add or remove kms or pgp keys under the -sops section. Invoking ``sops`` with the **-i** flag will perform an in-place edit +while editing. This method can be used to add or remove ``kms`` or ``pgp`` keys under the +``sops`` section. Invoking ``sops`` with the **-i** flag will perform an in-place edit instead of redirecting output to ``stdout``. For example, to add a KMS master key to a file, add the following entry while @@ -464,19 +466,19 @@ editing: .. code:: yaml - sops: - kms: - - arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e + sops: + kms: + - arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e And, similarly, to add a PGP master key, we add its fingerprint: .. code:: yaml - sops: - pgp: - - fp: 85D77543B3D624B63CEA9E6DBC17301B491B3F21 + sops: + pgp: + - fp: 85D77543B3D624B63CEA9E6DBC17301B491B3F21 -When the file is saved, ``sops`` will update its metadata and encrypt the data key +When the file is saved, SOPS will update its metadata and encrypt the data key with the freshly added master keys. The removed entries are simply deleted from the file. @@ -491,10 +493,10 @@ If you want to use a specific profile, you can do so with `aws_profile`: .. code:: yaml - sops: - kms: - - arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e - aws_profile: foo + sops: + kms: + - arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e + aws_profile: foo If no AWS profile is set, default credentials will be used. @@ -518,44 +520,44 @@ the user is allowed to assume in each account. The `IAM roles `_ documentation has full details on how this needs to be configured on AWS's side. -From the point of view of ``sops``, you only need to specify the role a KMS key +From the point of view of SOPS, you only need to specify the role a KMS key must assume alongside its ARN, as follows: .. code:: yaml - sops: - kms: - - arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e - role: arn:aws:iam::927034868273:role/sops-dev-xyz + sops: + kms: + - arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e + role: arn:aws:iam::927034868273:role/sops-dev-xyz The role must have permission to call Encrypt and Decrypt using KMS. An example policy is shown below. .. code:: json - { - "Sid": "Allow use of the key", - "Effect": "Allow", - "Action": [ - "kms:Encrypt", - "kms:Decrypt", - "kms:ReEncrypt*", - "kms:GenerateDataKey*", - "kms:DescribeKey" - ], - "Resource": "*", - "Principal": { - "AWS": [ - "arn:aws:iam::927034868273:role/sops-dev-xyz" - ] - } - } + { + "Sid": "Allow use of the key", + "Effect": "Allow", + "Action": [ + "kms:Encrypt", + "kms:Decrypt", + "kms:ReEncrypt*", + "kms:GenerateDataKey*", + "kms:DescribeKey" + ], + "Resource": "*", + "Principal": { + "AWS": [ + "arn:aws:iam::927034868273:role/sops-dev-xyz" + ] + } + } You can specify a role in the ``--kms`` flag and ``SOPS_KMS_ARN`` variable by appending it to the ARN of the master key, separated by a **+** sign:: - + - arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500+arn:aws:iam::927034868273:role/sops-dev-xyz + + + arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500+arn:aws:iam::927034868273:role/sops-dev-xyz AWS KMS Encryption Context ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -567,9 +569,9 @@ to refine the access control of a given KMS master key. When creating a new file, you can specify encryption context in the ``--encryption-context`` flag by comma separated list of key-value pairs: -.. code:: bash +.. code:: sh - $ sops --encryption-context Environment:production,Role:web-server test.dev.yaml + $ sops --encryption-context Environment:production,Role:web-server test.dev.yaml The format of the Encrypt Context string is ``:,:,...`` @@ -600,16 +602,16 @@ Key Rotation ~~~~~~~~~~~~ It is recommended to renew the data key on a regular basis. ``sops`` supports key -rotation via the ``-r`` flag. Invoking it on an existing file causes sops to +rotation via the ``-r`` flag. Invoking it on an existing file causes ``sops`` to reencrypt the file with a new data key, which is then encrypted with the various KMS and PGP master keys defined in the file. -.. code:: bash +.. code:: sh - sops -r example.yaml + $ sops -r example.yaml Using .sops.yaml conf to select KMS, PGP and age for new files -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ It is often tedious to specify the ``--kms`` ``--gcp-kms`` ``--pgp`` and ``--age`` parameters for creation of all new files. If your secrets are stored under a specific directory, like a @@ -628,41 +630,41 @@ can manage the three sets of configurations for the three types of files: .. code:: yaml - # creation rules are evaluated sequentially, the first match wins - creation_rules: - # upon creation of a file that matches the pattern *.dev.yaml, - # KMS set A as well as PGP and age is used - - path_regex: \.dev\.yaml$ - kms: 'arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500,arn:aws:kms:us-west-2:361527076523:key/5052f06a-5d3f-489e-b86c-57201e06f31e+arn:aws:iam::361527076523:role/hiera-sops-prod' - pgp: 'FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4' - age: 'age129h70qwx39k7h5x6l9hg566nwm53527zvamre8vep9e3plsm44uqgy8gla' - - # prod files use KMS set B in the PROD IAM, PGP and age - - path_regex: \.prod\.yaml$ - kms: 'arn:aws:kms:us-west-2:361527076523:key/5052f06a-5d3f-489e-b86c-57201e06f31e+arn:aws:iam::361527076523:role/hiera-sops-prod,arn:aws:kms:eu-central-1:361527076523:key/cb1fab90-8d17-42a1-a9d8-334968904f94+arn:aws:iam::361527076523:role/hiera-sops-prod' - pgp: 'FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4' - age: 'age129h70qwx39k7h5x6l9hg566nwm53527zvamre8vep9e3plsm44uqgy8gla' - hc_vault_uris: "http://localhost:8200/v1/sops/keys/thirdkey" - - # gcp files using GCP KMS - - path_regex: \.gcp\.yaml$ - gcp_kms: projects/mygcproject/locations/global/keyRings/mykeyring/cryptoKeys/thekey - - # Finally, if the rules above have not matched, this one is a - # catchall that will encrypt the file using KMS set C as well as PGP - # The absence of a path_regex means it will match everything - - kms: 'arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500,arn:aws:kms:us-west-2:142069644989:key/846cfb17-373d-49b9-8baf-f36b04512e47,arn:aws:kms:us-west-2:361527076523:key/5052f06a-5d3f-489e-b86c-57201e06f31e' - pgp: 'FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4' + # creation rules are evaluated sequentially, the first match wins + creation_rules: + # upon creation of a file that matches the pattern *.dev.yaml, + # KMS set A as well as PGP and age is used + - path_regex: \.dev\.yaml$ + kms: 'arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500,arn:aws:kms:us-west-2:361527076523:key/5052f06a-5d3f-489e-b86c-57201e06f31e+arn:aws:iam::361527076523:role/hiera-sops-prod' + pgp: 'FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4' + age: 'age129h70qwx39k7h5x6l9hg566nwm53527zvamre8vep9e3plsm44uqgy8gla' + + # prod files use KMS set B in the PROD IAM, PGP and age + - path_regex: \.prod\.yaml$ + kms: 'arn:aws:kms:us-west-2:361527076523:key/5052f06a-5d3f-489e-b86c-57201e06f31e+arn:aws:iam::361527076523:role/hiera-sops-prod,arn:aws:kms:eu-central-1:361527076523:key/cb1fab90-8d17-42a1-a9d8-334968904f94+arn:aws:iam::361527076523:role/hiera-sops-prod' + pgp: 'FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4' + age: 'age129h70qwx39k7h5x6l9hg566nwm53527zvamre8vep9e3plsm44uqgy8gla' + hc_vault_uris: "http://localhost:8200/v1/sops/keys/thirdkey" + + # gcp files using GCP KMS + - path_regex: \.gcp\.yaml$ + gcp_kms: projects/mygcproject/locations/global/keyRings/mykeyring/cryptoKeys/thekey + + # Finally, if the rules above have not matched, this one is a + # catchall that will encrypt the file using KMS set C as well as PGP + # The absence of a path_regex means it will match everything + - kms: 'arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500,arn:aws:kms:us-west-2:142069644989:key/846cfb17-373d-49b9-8baf-f36b04512e47,arn:aws:kms:us-west-2:361527076523:key/5052f06a-5d3f-489e-b86c-57201e06f31e' + pgp: 'FBC7B9E2A4F9289AC0C1D4843D16CEE4A27381B4' When creating any file under **mysecretrepo**, whether at the root or under -a subdirectory, sops will recursively look for a ``.sops.yaml`` file. If one is +a subdirectory, SOPS will recursively look for a ``.sops.yaml`` file. If one is found, the filename of the file being created is compared with the filename regexes of the configuration file. The first regex that matches is selected, and its KMS and PGP keys are used to encrypt the file. It should be noted that the looking up of ``.sops.yaml`` is from the working directory (CWD) instead of the directory of the encrypting file (see `Issue 242 `_). -The path_regex checks the path of the encrypting file relative to the .sops.yaml config file. Here is another example: +The ``path_regex`` checks the path of the encrypting file relative to the ``.sops.yaml`` config file. Here is another example: * files located under directory **development** should use one set of KMS A * files located under directory **production** should use another set of KMS B @@ -688,17 +690,17 @@ The path_regex checks the path of the encrypting file relative to the .sops.yaml Creating a new file with the right keys is now as simple as -.. code:: bash +.. code:: sh - $ sops .prod.yaml + $ sops .prod.yaml Note that the configuration file is ignored when KMS or PGP parameters are -passed on the sops command line or in environment variables. +passed on the SOPS command line or in environment variables. Specify a different GPG executable ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -``sops`` checks for the ``SOPS_GPG_EXEC`` environment variable. If specified, +SOPS checks for the ``SOPS_GPG_EXEC`` environment variable. If specified, it will attempt to use the executable set there instead of the default of ``gpg``. @@ -706,13 +708,13 @@ Example: place the following in your ``~/.bashrc`` .. code:: bash - SOPS_GPG_EXEC = 'your_gpg_client_wrapper' + SOPS_GPG_EXEC = 'your_gpg_client_wrapper' Specify a different GPG key server ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -By default, ``sops`` uses the key server ``keys.openpgp.org`` to retrieve the GPG +By default, SOPS uses the key server ``keys.openpgp.org`` to retrieve the GPG keys that are not present in the local keyring. This is no longer configurable. You can learn more about why from this write-up: `SKS Keyserver Network Under Attack `_. @@ -720,19 +722,19 @@ This is no longer configurable. You can learn more about why from this write-up: Key groups ~~~~~~~~~~ -By default, ``sops`` encrypts the data key for a file with each of the master keys, +By default, SOPS encrypts the data key for a file with each of the master keys, such that if any of the master keys is available, the file can be decrypted. However, it is sometimes desirable to require access to multiple master keys in order to decrypt files. This can be achieved with key groups. -When using key groups in sops, data keys are split into parts such that keys from -multiple groups are required to decrypt a file. ``sops`` uses Shamir's Secret Sharing +When using key groups in SOPS, data keys are split into parts such that keys from +multiple groups are required to decrypt a file. SOPS uses Shamir's Secret Sharing to split the data key such that each key group has a fragment, each key in the key group can decrypt that fragment, and a configurable number of fragments (threshold) are needed to decrypt and piece together the complete data key. When decrypting a -file using multiple key groups, ``sops`` goes through key groups in order, and in +file using multiple key groups, SOPS goes through key groups in order, and in each group, tries to recover the fragment of the data key using a master key from -that group. Once the fragment is recovered, ``sops`` moves on to the next group, +that group. Once the fragment is recovered, SOPS moves on to the next group, until enough fragments have been recovered to obtain the complete data key. By default, the threshold is set to the number of key groups. For example, if @@ -745,14 +747,14 @@ Management of key groups is done with the ``sops groups`` command. For example, you can add a new key group with 3 PGP keys and 3 KMS keys to the file ``my_file.yaml``: -.. code:: bash +.. code:: sh $ sops groups add --file my_file.yaml --pgp fingerprint1 --pgp fingerprint2 --pgp fingerprint3 --kms arn1 --kms arn2 --kms arn3 Or you can delete the 1st group (group number 0, as groups are zero-indexed) from ``my_file.yaml``: -.. code:: bash +.. code:: sh $ sops groups delete --file my_file.yaml 0 @@ -764,36 +766,36 @@ like so: creation_rules: - path_regex: .*keygroups.* key_groups: - # First key group - - pgp: - - fingerprint1 - - fingerprint2 - kms: - - arn: arn1 - role: role1 - context: - foo: bar - - arn: arn2 - # Second key group - - pgp: - - fingerprint3 - - fingerprint4 - kms: - - arn: arn3 - - arn: arn4 - # Third key group - - pgp: - - fingerprint5 + # First key group + - pgp: + - fingerprint1 + - fingerprint2 + kms: + - arn: arn1 + role: role1 + context: + foo: bar + - arn: arn2 + # Second key group + - pgp: + - fingerprint3 + - fingerprint4 + kms: + - arn: arn3 + - arn: arn4 + # Third key group + - pgp: + - fingerprint5 Given this configuration, we can create a new encrypted file like we normally would, and optionally provide the ``--shamir-secret-sharing-threshold`` command line -flag if we want to override the default threshold. ``sops`` will then split the data +flag if we want to override the default threshold. SOPS will then split the data key into three parts (from the number of key groups) and encrypt each fragment with the master keys found in each group. For example: -.. code:: bash +.. code:: sh $ sops --shamir-secret-sharing-threshold 2 example.json @@ -806,26 +808,26 @@ with ``shamir_threshold``: - path_regex: .*keygroups.* shamir_threshold: 2 key_groups: - # First key group - - pgp: - - fingerprint1 - - fingerprint2 - kms: - - arn: arn1 - role: role1 - context: - foo: bar - - arn: arn2 - # Second key group - - pgp: - - fingerprint3 - - fingerprint4 - kms: - - arn: arn3 - - arn: arn4 - # Third key group - - pgp: - - fingerprint5 + # First key group + - pgp: + - fingerprint1 + - fingerprint2 + kms: + - arn: arn1 + role: role1 + context: + foo: bar + - arn: arn2 + # Second key group + - pgp: + - fingerprint3 + - fingerprint4 + kms: + - arn: arn3 + - arn: arn4 + # Third key group + - pgp: + - fingerprint5 And then run ``sops example.json``. @@ -833,16 +835,16 @@ The threshold (``shamir_threshold``) is set to 2, so this configuration will req master keys from two of the three different key groups in order to decrypt the file. You can then decrypt the file the same way as with any other SOPS file: -.. code:: bash +.. code:: sh $ sops -d example.json Key service ~~~~~~~~~~~ -There are situations where you might want to run ``sops`` on a machine that +There are situations where you might want to run SOPS on a machine that doesn't have direct access to encryption keys such as PGP keys. The ``sops`` key -service allows you to forward a socket so that ``sops`` can access encryption +service allows you to forward a socket so that SOPS can access encryption keys stored on a remote machine. This is similar to GPG Agent, but more portable. @@ -873,14 +875,14 @@ services. The local key service can be disabled with For example, to decrypt a file using both the local key service and the key service exposed on the unix socket located in ``/tmp/sops.sock``, you can run: -.. code:: bash +.. code:: sh $ sops --keyservice unix:///tmp/sops.sock -d file.yaml` And if you only want to use the key service exposed on the unix socket located in ``/tmp/sops.sock`` and not the local key service, you can run: -.. code:: bash +.. code:: sh $ sops --enable-local-keyservice=false --keyservice unix:///tmp/sops.sock -d file.yaml @@ -932,13 +934,13 @@ provide more than one backend, and SOPS will log to all of them: Saving Output to a File ~~~~~~~~~~~~~~~~~~~~~~~ -By default ``sops`` just dumps all the output to the standard output. We can use the +By default SOPS just dumps all the output to the standard output. We can use the ``--output`` flag followed by a filename to save the output to the file specified. Beware using both ``--in-place`` and ``--output`` flags will result in an error. Passing Secrets to Other Processes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -In addition to writing secrets to standard output and to files on disk, ``sops`` +In addition to writing secrets to standard output and to files on disk, SOPS has two commands for passing decrypted secrets to a new process: ``exec-env`` and ``exec-file``. These commands will place all output into the environment of a child process and into a temporary file, respectively. For example, if a @@ -946,71 +948,71 @@ program looks for credentials in its environment, ``exec-env`` can be used to ensure that the decrypted contents are available only to this process and never written to disk. -.. code:: bash +.. code:: sh - # print secrets to stdout to confirm values - $ sops -d out.json - { - "database_password": "jf48t9wfw094gf4nhdf023r", - "AWS_ACCESS_KEY_ID": "AKIAIOSFODNN7EXAMPLE", - "AWS_SECRET_KEY": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" - } + # print secrets to stdout to confirm values + $ sops -d out.json + { + "database_password": "jf48t9wfw094gf4nhdf023r", + "AWS_ACCESS_KEY_ID": "AKIAIOSFODNN7EXAMPLE", + "AWS_SECRET_KEY": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" + } - # decrypt out.json and run a command - # the command prints the environment variable and runs a script that uses it - $ sops exec-env out.json 'echo secret: $database_password; ./database-import' - secret: jf48t9wfw094gf4nhdf023r + # decrypt out.json and run a command + # the command prints the environment variable and runs a script that uses it + $ sops exec-env out.json 'echo secret: $database_password; ./database-import' + secret: jf48t9wfw094gf4nhdf023r - # launch a shell with the secrets available in its environment - $ sops exec-env out.json 'sh' - sh-3.2# echo $database_password - jf48t9wfw094gf4nhdf023r + # launch a shell with the secrets available in its environment + $ sops exec-env out.json 'sh' + sh-3.2# echo $database_password + jf48t9wfw094gf4nhdf023r - # the secret is not accessible anywhere else - sh-3.2$ exit - $ echo your password: $database_password - your password: + # the secret is not accessible anywhere else + sh-3.2$ exit + $ echo your password: $database_password + your password: If the command you want to run only operates on files, you can use ``exec-file`` -instead. By default ``sops`` will use a FIFO to pass the contents of the +instead. By default SOPS will use a FIFO to pass the contents of the decrypted file to the new program. Using a FIFO, secrets are only passed in memory which has two benefits: the plaintext secrets never touch the disk, and the child process can only read the secrets once. In contexts where this won't work, eg platforms like Windows where FIFOs unavailable or secret files that need to be available to the child process longer term, the ``--no-fifo`` flag can be -used to instruct ``sops`` to use a traditional temporary file that will get cleaned +used to instruct SOPS to use a traditional temporary file that will get cleaned up once the process is finished executing. ``exec-file`` behaves similar to ``find(1)`` in that ``{}`` is used as a placeholder in the command which will be substituted with the temporary file path (whether a FIFO or an actual file). -.. code:: bash +.. code:: sh + + # operating on the same file as before, but as a file this time + $ sops exec-file out.json 'echo your temporary file: {}; cat {}' + your temporary file: /tmp/.sops894650499/tmp-file + { + "database_password": "jf48t9wfw094gf4nhdf023r", + "AWS_ACCESS_KEY_ID": "AKIAIOSFODNN7EXAMPLE", + "AWS_SECRET_KEY": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" + } + + # launch a shell with a variable TMPFILE pointing to the temporary file + $ sops exec-file --no-fifo out.json 'TMPFILE={} sh' + sh-3.2$ echo $TMPFILE + /tmp/.sops506055069/tmp-file291138648 + sh-3.2$ cat $TMPFILE + { + "database_password": "jf48t9wfw094gf4nhdf023r", + "AWS_ACCESS_KEY_ID": "AKIAIOSFODNN7EXAMPLE", + "AWS_SECRET_KEY": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" + } + sh-3.2$ ./program --config $TMPFILE + sh-3.2$ exit - # operating on the same file as before, but as a file this time - $ sops exec-file out.json 'echo your temporary file: {}; cat {}' - your temporary file: /tmp/.sops894650499/tmp-file - { - "database_password": "jf48t9wfw094gf4nhdf023r", - "AWS_ACCESS_KEY_ID": "AKIAIOSFODNN7EXAMPLE", - "AWS_SECRET_KEY": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" - } - - # launch a shell with a variable TMPFILE pointing to the temporary file - $ sops exec-file --no-fifo out.json 'TMPFILE={} sh' - sh-3.2$ echo $TMPFILE - /tmp/.sops506055069/tmp-file291138648 - sh-3.2$ cat $TMPFILE - { - "database_password": "jf48t9wfw094gf4nhdf023r", - "AWS_ACCESS_KEY_ID": "AKIAIOSFODNN7EXAMPLE", - "AWS_SECRET_KEY": "wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY" - } - sh-3.2$ ./program --config $TMPFILE - sh-3.2$ exit - - # try to open the temporary file from earlier - $ cat /tmp/.sops506055069/tmp-file291138648 - cat: /tmp/.sops506055069/tmp-file291138648: No such file or directory + # try to open the temporary file from earlier + $ cat /tmp/.sops506055069/tmp-file291138648 + cat: /tmp/.sops506055069/tmp-file291138648: No such file or directory Additionally, on unix-like platforms, both ``exec-env`` and ``exec-file`` support dropping privileges before executing the new program via the @@ -1022,46 +1024,46 @@ for added security. To overwrite the default file name (``tmp-file``) in ``exec-file`` use the ``--filename `` parameter. -.. code:: bash +.. code:: sh - # the encrypted file can't be read by the current user - $ cat out.json - cat: out.json: Permission denied + # the encrypted file can't be read by the current user + $ cat out.json + cat: out.json: Permission denied - # execute sops as root, decrypt secrets, then drop privileges - $ sudo sops exec-env --user nobody out.json 'sh' - sh-3.2$ echo $database_password - jf48t9wfw094gf4nhdf023r + # execute sops as root, decrypt secrets, then drop privileges + $ sudo sops exec-env --user nobody out.json 'sh' + sh-3.2$ echo $database_password + jf48t9wfw094gf4nhdf023r - # dropped privileges, still can't load the original file - sh-3.2$ id - uid=4294967294(nobody) gid=4294967294(nobody) groups=4294967294(nobody) - sh-3.2$ cat out.json - cat: out.json: Permission denied + # dropped privileges, still can't load the original file + sh-3.2$ id + uid=4294967294(nobody) gid=4294967294(nobody) groups=4294967294(nobody) + sh-3.2$ cat out.json + cat: out.json: Permission denied Using the publish command ~~~~~~~~~~~~~~~~~~~~~~~~~ -``sops publish $file`` publishes a file to a pre-configured destination (this lives in the sops +``sops publish $file`` publishes a file to a pre-configured destination (this lives in the SOPS config file). Additionally, support re-encryption rules that work just like the creation rules. This command requires a ``.sops.yaml`` configuration file. Below is an example: .. code:: yaml - destination_rules: - - s3_bucket: "sops-secrets" - path_regex: s3/* - recreation_rule: - pgp: F69E4901EDBAD2D1753F8C67A64535C4163FB307 - - gcs_bucket: "sops-secrets" - path_regex: gcs/* - recreation_rule: - pgp: F69E4901EDBAD2D1753F8C67A64535C4163FB307 - - vault_path: "sops/" - vault_kv_mount_name: "secret/" # default - vault_kv_version: 2 # default - path_regex: vault/* - omit_extensions: true + destination_rules: + - s3_bucket: "sops-secrets" + path_regex: s3/* + recreation_rule: + pgp: F69E4901EDBAD2D1753F8C67A64535C4163FB307 + - gcs_bucket: "sops-secrets" + path_regex: gcs/* + recreation_rule: + pgp: F69E4901EDBAD2D1753F8C67A64535C4163FB307 + - vault_path: "sops/" + vault_kv_mount_name: "secret/" # default + vault_kv_version: 2 # default + path_regex: vault/* + omit_extensions: true The above configuration will place all files under ``s3/*`` into the S3 bucket ``sops-secrets``, all files under ``gcs/*`` into the GCS bucket ``sops-secrets``, and the contents of all files under @@ -1083,7 +1085,7 @@ There are a few settings for Vault that you can place in your destination rules. is ``vault_path``, which is required. The others are optional, and they are ``vault_address``, ``vault_kv_mount_name``, ``vault_kv_version``. -``sops`` uses the official Vault API provided by Hashicorp, which makes use of `environment +SOPS uses the official Vault API provided by Hashicorp, which makes use of `environment variables `_ for configuring the client. @@ -1095,32 +1097,32 @@ will be skipped. Below is an example of publishing to Vault (using token auth with a local dev instance of Vault). -.. code:: bash - - $ export VAULT_TOKEN=... - $ export VAULT_ADDR='http://127.0.0.1:8200' - $ sops -d vault/test.yaml - example_string: bar - example_number: 42 - example_map: - key: value - $ sops publish vault/test.yaml - uploading /home/user/sops_directory/vault/test.yaml to http://127.0.0.1:8200/v1/secret/data/sops/test.yaml ? (y/n): y - $ vault kv get secret/sops/test.yaml - ====== Metadata ====== - Key Value - --- ----- - created_time 2019-07-11T03:32:17.074792017Z - deletion_time n/a - destroyed false - version 3 - - ========= Data ========= - Key Value - --- ----- - example_map map[key:value] - example_number 42 - example_string bar +.. code:: sh + + $ export VAULT_TOKEN=... + $ export VAULT_ADDR='http://127.0.0.1:8200' + $ sops -d vault/test.yaml + example_string: bar + example_number: 42 + example_map: + key: value + $ sops publish vault/test.yaml + uploading /home/user/sops_directory/vault/test.yaml to http://127.0.0.1:8200/v1/secret/data/sops/test.yaml ? (y/n): y + $ vault kv get secret/sops/test.yaml + ====== Metadata ====== + Key Value + --- ----- + created_time 2019-07-11T03:32:17.074792017Z + deletion_time n/a + destroyed false + version 3 + + ========= Data ========= + Key Value + --- ----- + example_map map[key:value] + example_number 42 + example_string bar Important information on types @@ -1129,7 +1131,7 @@ Important information on types YAML and JSON type extensions ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -``sops`` uses the file extension to decide which encryption method to use on the file +SOPS uses the file extension to decide which encryption method to use on the file content. ``YAML``, ``JSON``, ``ENV``, and ``INI`` files are treated as trees of data, and key/values are extracted from the files to only encrypt the leaf values. The tree structure is also used to check the integrity of the file. @@ -1138,46 +1140,46 @@ Therefore, if a file is encrypted using a specific format, it need to be decrypt in the same format. The easiest way to achieve this is to conserve the original file extension after encrypting a file. For example: -.. code:: bash +.. code:: sh - $ sops -e -i myfile.json - $ sops -d myfile.json + $ sops -e -i myfile.json + $ sops -d myfile.json If you want to change the extension of the file once encrypted, you need to provide -sops with the ``--input-type`` flag upon decryption. For example: +``sops`` with the ``--input-type`` flag upon decryption. For example: -.. code:: bash +.. code:: sh - $ sops -e myfile.json > myfile.json.enc + $ sops -e myfile.json > myfile.json.enc - $ sops -d --input-type json myfile.json.enc + $ sops -d --input-type json myfile.json.enc When operating on stdin, use the ``--input-type`` and ``--output-type`` flags as follows: -.. code:: bash +.. code:: sh $ cat myfile.json | sops --input-type json --output-type json -d /dev/stdin YAML anchors ~~~~~~~~~~~~ -``sops`` only supports a subset of ``YAML``'s many types. Encrypting YAML files that +SOPS only supports a subset of ``YAML``'s many types. Encrypting YAML files that contain strings, numbers and booleans will work fine, but files that contain anchors will not work, because the anchors redefine the structure of the file at load time. -This file will not work in ``sops``: +This file will not work in SOPS: .. code:: yaml - bill-to: &id001 - street: | - 123 Tornado Alley - Suite 16 - city: East Centerville - state: KS + bill-to: &id001 + street: | + 123 Tornado Alley + Suite 16 + city: East Centerville + state: KS - ship-to: *id001 + ship-to: *id001 -``sops`` uses the path to a value as additional data in the AEAD encryption, and thus +SOPS uses the path to a value as additional data in the AEAD encryption, and thus dynamic paths generated by anchors break the authentication step. JSON and TEXT file types do not support anchors and thus have no such limitation. @@ -1186,119 +1188,119 @@ YAML Streams ~~~~~~~~~~~~ ``YAML`` supports having more than one "document" in a single file, while -formats like ``JSON`` do not. ``sops`` is able to handle both. This means the +formats like ``JSON`` do not. SOPS is able to handle both. This means the following multi-document will be encrypted as expected: .. code:: yaml - --- - data: foo - --- - data: bar + --- + data: foo + --- + data: bar Note that the ``sops`` metadata, i.e. the hash, etc, is computed for the physical file rather than each internal "document". Top-level arrays ~~~~~~~~~~~~~~~~ -``YAML`` and ``JSON`` top-level arrays are not supported, because ``sops`` +``YAML`` and ``JSON`` top-level arrays are not supported, because SOPS needs a top-level ``sops`` key to store its metadata. -This file will not work in sops: +This file will not work in SOPS: .. code:: yaml - --- - - some - - array - - elements + --- + - some + - array + - elements But this one will work because the ``sops`` key can be added at the same level as the ``data`` key. .. code:: yaml - data: - - some - - array - - elements + data: + - some + - array + - elements Similarly, with ``JSON`` arrays, this document will not work: .. code:: json - [ - "some", - "array", - "elements" - ] + [ + "some", + "array", + "elements" + ] But this one will work just fine: .. code:: json - { - "data": [ - "some", - "array", - "elements" - ] - } + { + "data": [ + "some", + "array", + "elements" + ] + } Examples -------- -Take a look into the `examples `_ folder for detailed use cases of sops in a CI environment. The section below describes specific tips for common use cases. +Take a look into the `examples folder `_ for detailed use cases of SOPS in a CI environment. The section below describes specific tips for common use cases. Creating a new file ~~~~~~~~~~~~~~~~~~~ The command below creates a new file with a data key encrypted by KMS and PGP. -.. code:: bash +.. code:: sh - $ sops --kms "arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500" --pgp C9CAB0AF1165060DB58D6D6B2653B624D620786D /path/to/new/file.yaml + $ sops --kms "arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500" --pgp C9CAB0AF1165060DB58D6D6B2653B624D620786D /path/to/new/file.yaml Encrypting an existing file ~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Similar to the previous command, we tell sops to use one KMS and one PGP key. -The path points to an existing cleartext file, so we give sops flag ``-e`` to +Similar to the previous command, we tell SOPS to use one KMS and one PGP key. +The path points to an existing cleartext file, so we give ``sops`` the flag ``-e`` to encrypt the file, and redirect the output to a destination file. -.. code:: bash +.. code:: sh - $ export SOPS_KMS_ARN="arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500" - $ export SOPS_PGP_FP="C9CAB0AF1165060DB58D6D6B2653B624D620786D" - $ sops -e /path/to/existing/file.yaml > /path/to/new/encrypted/file.yaml + $ export SOPS_KMS_ARN="arn:aws:kms:us-west-2:927034868273:key/fe86dd69-4132-404c-ab86-4269956b4500" + $ export SOPS_PGP_FP="C9CAB0AF1165060DB58D6D6B2653B624D620786D" + $ sops -e /path/to/existing/file.yaml > /path/to/new/encrypted/file.yaml Decrypt the file with ``-d``. -.. code:: bash +.. code:: sh - $ sops -d /path/to/new/encrypted/file.yaml + $ sops -d /path/to/new/encrypted/file.yaml Encrypt or decrypt a file in place ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -Rather than redirecting the output of ``-e`` or ``-d``, sops can replace the +Rather than redirecting the output of ``-e`` or ``-d``, ``sops`` can replace the original file after encrypting or decrypting it. -.. code:: bash +.. code:: sh - # file.yaml is in cleartext - $ sops -e -i /path/to/existing/file.yaml - # file.yaml is now encrypted - $ sops -d -i /path/to/existing/file.yaml - # file.yaml is back in cleartext + # file.yaml is in cleartext + $ sops -e -i /path/to/existing/file.yaml + # file.yaml is now encrypted + $ sops -d -i /path/to/existing/file.yaml + # file.yaml is back in cleartext Encrypting binary files ~~~~~~~~~~~~~~~~~~~~~~~ -``sops`` primary use case is encrypting YAML and JSON configuration files, but it -also has the ability to manage binary files. When encrypting a binary, sops will +SOPS primary use case is encrypting YAML and JSON configuration files, but it +also has the ability to manage binary files. When encrypting a binary, SOPS will read the data as bytes, encrypt it, store the encrypted base64 under ``tree['data']`` and write the result as JSON. @@ -1307,103 +1309,103 @@ file larger than the cleartext one. In-place encryption/decryption also works on binary files. -.. code:: +.. code:: sh - $ dd if=/dev/urandom of=/tmp/somerandom bs=1024 - count=512 - 512+0 records in - 512+0 records out - 524288 bytes (524 kB) copied, 0.0466158 s, 11.2 MB/s + $ dd if=/dev/urandom of=/tmp/somerandom bs=1024 + count=512 + 512+0 records in + 512+0 records out + 524288 bytes (524 kB) copied, 0.0466158 s, 11.2 MB/s - $ sha512sum /tmp/somerandom - 9589bb20280e9d381f7a192000498c994e921b3cdb11d2ef5a986578dc2239a340b25ef30691bac72bdb14028270828dad7e8bd31e274af9828c40d216e60cbe /tmp/somerandom + $ sha512sum /tmp/somerandom + 9589bb20280e9d381f7a192000498c994e921b3cdb11d2ef5a986578dc2239a340b25ef30691bac72bdb14028270828dad7e8bd31e274af9828c40d216e60cbe /tmp/somerandom - $ sops -e -i /tmp/somerandom - please wait while a data encryption key is being generated and stored securely + $ sops -e -i /tmp/somerandom + please wait while a data encryption key is being generated and stored securely - $ sops -d -i /tmp/somerandom + $ sops -d -i /tmp/somerandom - $ sha512sum /tmp/somerandom - 9589bb20280e9d381f7a192000498c994e921b3cdb11d2ef5a986578dc2239a340b25ef30691bac72bdb14028270828dad7e8bd31e274af9828c40d216e60cbe /tmp/somerandom + $ sha512sum /tmp/somerandom + 9589bb20280e9d381f7a192000498c994e921b3cdb11d2ef5a986578dc2239a340b25ef30691bac72bdb14028270828dad7e8bd31e274af9828c40d216e60cbe /tmp/somerandom Extract a sub-part of a document tree ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -``sops`` can extract a specific part of a YAML or JSON document, by provided the +SOPS can extract a specific part of a YAML or JSON document, by provided the path in the ``--extract`` command line flag. This is useful to extract specific values, like keys, without needing an extra parser. -.. code:: bash +.. code:: sh - $ sops -d --extract '["app2"]["key"]' ~/git/svc/sops/example.yaml - -----BEGIN RSA PRIVATE KEY----- - MIIBPAIBAAJBAPTMNIyHuZtpLYc7VsHQtwOkWYobkUblmHWRmbXzlAX6K8tMf3Wf - ImcbNkqAKnELzFAPSBeEMhrBN0PyOC9lYlMCAwEAAQJBALXD4sjuBn1E7Y9aGiMz - bJEBuZJ4wbhYxomVoQKfaCu+kH80uLFZKoSz85/ySauWE8LgZcMLIBoiXNhDKfQL - vHECIQD6tCG9NMFWor69kgbX8vK5Y+QL+kRq+9HK6yZ9a+hsLQIhAPn4Ie6HGTjw - fHSTXWZpGSan7NwTkIu4U5q2SlLjcZh/AiEA78NYRRBwGwAYNUqzutGBqyXKUl4u - Erb0xAEyVV7e8J0CIQC8VBY8f8yg+Y7Kxbw4zDYGyb3KkXL10YorpeuZR4LuQQIg - bKGPkMM4w5blyE1tqGN0T7sJwEx+EUOgacRNqM2ljVA= - -----END RSA PRIVATE KEY----- + $ sops -d --extract '["app2"]["key"]' ~/git/svc/sops/example.yaml + -----BEGIN RSA PRIVATE KEY----- + MIIBPAIBAAJBAPTMNIyHuZtpLYc7VsHQtwOkWYobkUblmHWRmbXzlAX6K8tMf3Wf + ImcbNkqAKnELzFAPSBeEMhrBN0PyOC9lYlMCAwEAAQJBALXD4sjuBn1E7Y9aGiMz + bJEBuZJ4wbhYxomVoQKfaCu+kH80uLFZKoSz85/ySauWE8LgZcMLIBoiXNhDKfQL + vHECIQD6tCG9NMFWor69kgbX8vK5Y+QL+kRq+9HK6yZ9a+hsLQIhAPn4Ie6HGTjw + fHSTXWZpGSan7NwTkIu4U5q2SlLjcZh/AiEA78NYRRBwGwAYNUqzutGBqyXKUl4u + Erb0xAEyVV7e8J0CIQC8VBY8f8yg+Y7Kxbw4zDYGyb3KkXL10YorpeuZR4LuQQIg + bKGPkMM4w5blyE1tqGN0T7sJwEx+EUOgacRNqM2ljVA= + -----END RSA PRIVATE KEY----- The tree path syntax uses regular python dictionary syntax, without the variable name. Extract keys by naming them, and array elements by numbering them. -.. code:: bash +.. code:: sh - $ sops -d --extract '["an_array"][1]' ~/git/svc/sops/example.yaml - secretuser2 + $ sops -d --extract '["an_array"][1]' ~/git/svc/sops/example.yaml + secretuser2 Set a sub-part in a document tree ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ -``sops`` can set a specific part of a YAML or JSON document, by providing +SOPS can set a specific part of a YAML or JSON document, by providing the path and value in the ``--set`` command line flag. This is useful to set specific values, like keys, without needing an editor. -.. code:: bash +.. code:: sh - $ sops --set '["app2"]["key"] "app2keystringvalue"' ~/git/svc/sops/example.yaml + $ sops --set '["app2"]["key"] "app2keystringvalue"' ~/git/svc/sops/example.yaml The tree path syntax uses regular python dictionary syntax, without the variable name. Set to keys by naming them, and array elements by numbering them. -.. code:: bash +.. code:: sh - $ sops --set '["an_array"][1] "secretuser2"' ~/git/svc/sops/example.yaml + $ sops --set '["an_array"][1] "secretuser2"' ~/git/svc/sops/example.yaml The value must be formatted as json. -.. code:: bash +.. code:: sh - $ sops --set '["an_array"][1] {"uid1":null,"uid2":1000,"uid3":["bob"]}' ~/git/svc/sops/example.yaml + $ sops --set '["an_array"][1] {"uid1":null,"uid2":1000,"uid3":["bob"]}' ~/git/svc/sops/example.yaml Showing diffs in cleartext in git ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ You most likely want to store encrypted files in a version controlled repository. -Sops can be used with git to decrypt files when showing diffs between versions. +SOPS can be used with git to decrypt files when showing diffs between versions. This is very handy for reviewing changes or visualizing history. -To configure sops to decrypt files during diff, create a ``.gitattributes`` file +To configure SOPS to decrypt files during diff, create a ``.gitattributes`` file at the root of your repository that contains a filter and a command. -.. code:: +.. code:: text - *.yaml diff=sopsdiffer + *.yaml diff=sopsdiffer Here we only care about YAML files. ``sopsdiffer`` is an arbitrary name that we map -to a sops command in the git configuration file of the repository. +to a SOPS command in the git configuration file of the repository. -.. code:: bash +.. code:: sh - $ git config diff.sopsdiffer.textconv "sops -d" + $ git config diff.sopsdiffer.textconv "sops -d" - $ grep -A 1 sopsdiffer .git/config - [diff "sopsdiffer"] - textconv = "sops -d" + $ grep -A 1 sopsdiffer .git/config + [diff "sopsdiffer"] + textconv = "sops -d" With this in place, calls to ``git diff`` will decrypt both previous and current versions of the target file prior to displaying the diff. And it even works with @@ -1414,14 +1416,14 @@ Encrypting only parts of a file Note: this only works on YAML and JSON files, not on BINARY files. -By default, ``sops`` encrypts all the values of a YAML or JSON file and leaves the +By default, SOPS encrypts all the values of a YAML or JSON file and leaves the keys in cleartext. In some instances, you may want to exclude some values from being encrypted. This can be accomplished by adding the suffix **_unencrypted** to any key of a file. When set, all values underneath the key that set the **_unencrypted** suffix will be left in cleartext. Note that, while in cleartext, unencrypted content is still added to the -checksum of the file, and thus cannot be modified outside of sops without +checksum of the file, and thus cannot be modified outside of SOPS without breaking the file integrity check. The unencrypted suffix can be set to a different value using the @@ -1433,9 +1435,9 @@ by adding a chosen suffix to those keys and passing it to the ``--encrypted-suff A third method is to use the ``--encrypted-regex`` which will only encrypt values under keys that match the supplied regular expression. For example, this command: -.. code:: bash +.. code:: sh - $ sops --encrypt --encrypted-regex '^(data|stringData)$' k8s-secrets.yaml + $ sops --encrypt --encrypted-regex '^(data|stringData)$' k8s-secrets.yaml will encrypt the values under the ``data`` and ``stringData`` keys in a YAML file containing kubernetes secrets. It will not encrypt other values that help you to @@ -1445,9 +1447,9 @@ Conversely, you can opt in to only left certain keys without encrypting by using ``--unencrypted-regex`` option, which will leave the values unencrypted of those keys that match the supplied regular expression. For example, this command: -.. code:: bash +.. code:: sh - $ sops --encrypt --unencrypted-regex '^(description|metadata)$' k8s-secrets.yaml + $ sops --encrypt --unencrypted-regex '^(description|metadata)$' k8s-secrets.yaml will not encrypt the values under the ``description`` and ``metadata`` keys in a YAML file containing kubernetes secrets, while encrypting everything else. @@ -1460,7 +1462,7 @@ mutually exclusive and cannot all be used in the same file. Encryption Protocol ------------------- -When sops creates a file, it generates a random 256 bit data key and asks each +When SOPS creates a file, it generates a random 256 bit data key and asks each KMS and PGP master key to encrypt the data key. The encrypted version of the data key is stored in the ``sops`` metadata under ``sops.kms`` and ``sops.pgp``. @@ -1470,9 +1472,9 @@ For KMS: sops: kms: - - enc: CiC6yCOtzsnFhkfdIslYZ0bAf//gYLYCmIu87B3sy/5yYxKnAQEBAQB4usgjrc7JxYZH3SLJWGdGwH//4GC2ApiLvOwd7Mv+cmMAAAB+MHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAyGdRODuYMHbA8Ozj8CARCAO7opMolPJUmBXd39Zlp0L2H9fzMKidHm1vvaF6nNFq0ClRY7FlIZmTm4JfnOebPseffiXFn9tG8cq7oi - enc_ts: 1439568549.245995 - arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e + - enc: CiC6yCOtzsnFhkfdIslYZ0bAf//gYLYCmIu87B3sy/5yYxKnAQEBAQB4usgjrc7JxYZH3SLJWGdGwH//4GC2ApiLvOwd7Mv+cmMAAAB+MHwGCSqGSIb3DQEHBqBvMG0CAQAwaAYJKoZIhvcNAQcBMB4GCWCGSAFlAwQBLjARBAyGdRODuYMHbA8Ozj8CARCAO7opMolPJUmBXd39Zlp0L2H9fzMKidHm1vvaF6nNFq0ClRY7FlIZmTm4JfnOebPseffiXFn9tG8cq7oi + enc_ts: 1439568549.245995 + arn: arn:aws:kms:us-east-1:656532927350:key/920aff2e-c5f1-4040-943a-047fa387b27e For PGP: @@ -1480,32 +1482,32 @@ For PGP: sops: pgp: - - fp: 85D77543B3D624B63CEA9E6DBC17301B491B3F21 - created_at: 1441570391.930042 - enc: | - -----BEGIN PGP MESSAGE----- - Version: GnuPG v1 - - hQIMA0t4uZHfl9qgAQ//UvGAwGePyHuf2/zayWcloGaDs0MzI+zw6CmXvMRNPUsA - pAgRKczJmDu4+XzN+cxX5Iq9xEWIbny9B5rOjwTXT3qcUYZ4Gkzbq4MWkjuPp/Iv - qO4MJaYzoH5YxC4YORQ2LvzhA2YGsCzYnljmatGEUNg01yJ6r5mwFwDxl4Nc80Cn - RwnHuGExK8j1jYJZu/juK1qRbuBOAuruIPPWVdFB845PA7waacG1IdUW3ZtBkOy3 - O0BIfG2ekRg0Nik6sTOhDUA+l2bewCcECI8FYCEjwHm9Sg5cxmP2V5m1mby+uKAm - kewaoOyjbmV1Mh3iI1b/AQMr+/6ZE9MT2KnsoWosYamFyjxV5r1ZZM7cWKnOT+tu - KOvGhTV1TeOfVpajNTNwtV/Oyh3mMLQ0F0HgCTqomQVqw5+sj7OWAASuD3CU/dyo - pcmY5Qe0TNL1JsMNEH8LJDqSh+E0hsUxdY1ouVsg3ysf6mdM8ciWb3WRGxih1Vmf - unfLy8Ly3V7ZIC8EHV8aLJqh32jIZV4i2zXIoO4ZBKrudKcECY1C2+zb/TziVAL8 - qyPe47q8gi1rIyEv5uirLZjgpP+JkDUgoMnzlX334FZ9pWtQMYW4Y67urAI4xUq6 - /q1zBAeHoeeeQK+YKDB7Ak/Y22YsiqQbNp2n4CKSKAE4erZLWVtDvSp+49SWmS/S - XgGi+13MaXIp0ecPKyNTBjF+NOw/I3muyKr8EbDHrd2XgIT06QXqjYLsCb1TZ0zm - xgXsOTY3b+ONQ2zjhcovanDp7/k77B+gFitLYKg4BLZsl7gJB12T8MQnpfSmRT4= - =oJgS - -----END PGP MESSAGE----- - -``sops`` then opens a text editor on the newly created file. The user adds data to the + - fp: 85D77543B3D624B63CEA9E6DBC17301B491B3F21 + created_at: 1441570391.930042 + enc: | + -----BEGIN PGP MESSAGE----- + Version: GnuPG v1 + + hQIMA0t4uZHfl9qgAQ//UvGAwGePyHuf2/zayWcloGaDs0MzI+zw6CmXvMRNPUsA + pAgRKczJmDu4+XzN+cxX5Iq9xEWIbny9B5rOjwTXT3qcUYZ4Gkzbq4MWkjuPp/Iv + qO4MJaYzoH5YxC4YORQ2LvzhA2YGsCzYnljmatGEUNg01yJ6r5mwFwDxl4Nc80Cn + RwnHuGExK8j1jYJZu/juK1qRbuBOAuruIPPWVdFB845PA7waacG1IdUW3ZtBkOy3 + O0BIfG2ekRg0Nik6sTOhDUA+l2bewCcECI8FYCEjwHm9Sg5cxmP2V5m1mby+uKAm + kewaoOyjbmV1Mh3iI1b/AQMr+/6ZE9MT2KnsoWosYamFyjxV5r1ZZM7cWKnOT+tu + KOvGhTV1TeOfVpajNTNwtV/Oyh3mMLQ0F0HgCTqomQVqw5+sj7OWAASuD3CU/dyo + pcmY5Qe0TNL1JsMNEH8LJDqSh+E0hsUxdY1ouVsg3ysf6mdM8ciWb3WRGxih1Vmf + unfLy8Ly3V7ZIC8EHV8aLJqh32jIZV4i2zXIoO4ZBKrudKcECY1C2+zb/TziVAL8 + qyPe47q8gi1rIyEv5uirLZjgpP+JkDUgoMnzlX334FZ9pWtQMYW4Y67urAI4xUq6 + /q1zBAeHoeeeQK+YKDB7Ak/Y22YsiqQbNp2n4CKSKAE4erZLWVtDvSp+49SWmS/S + XgGi+13MaXIp0ecPKyNTBjF+NOw/I3muyKr8EbDHrd2XgIT06QXqjYLsCb1TZ0zm + xgXsOTY3b+ONQ2zjhcovanDp7/k77B+gFitLYKg4BLZsl7gJB12T8MQnpfSmRT4= + =oJgS + -----END PGP MESSAGE----- + +SOPS then opens a text editor on the newly created file. The user adds data to the file and saves it when done. -Upon save, sops browses the entire file as a key/value tree. Every time sops +Upon save, SOPS browses the entire file as a key/value tree. Every time SOPS encounters a leaf value (a value that does not have children), it encrypts the value with AES256_GCM using the data key and a 256 bit random initialization vector. @@ -1523,7 +1525,7 @@ Any valid KMS or PGP master key can later decrypt the data key and access the data. Multiple master keys allow for sharing encrypted files without sharing master -keys, and provide a disaster recovery solution. The recommended way to use sops +keys, and provide a disaster recovery solution. The recommended way to use SOPS is to have two KMS master keys in different regions and one PGP public key with the private key stored offline. If, by any chance, both KMS master keys are lost, you can always recover the encrypted data using the PGP private key. @@ -1532,9 +1534,9 @@ Message Authentication Code ~~~~~~~~~~~~~~~~~~~~~~~~~~~ In addition to authenticating branches of the tree using keys as additional -data, sops computes a MAC on all the values to ensure that no value has been +data, SOPS computes a MAC on all the values to ensure that no value has been added or removed fraudulently. The MAC is stored encrypted with AES_GCM and -the data key under tree->`sops`->`mac`. +the data key under tree -> ``sops`` -> ``mac``. Motivation ---------- @@ -1642,17 +1644,17 @@ all our KMS master keys. SOPS can be used without KMS entirely, the same way you would use an encrypted PGP file: by referencing the pubkeys of each individual who has access to the file. -It can easily be done by providing sops with a comma-separated list of public keys +It can easily be done by providing SOPS with a comma-separated list of public keys when creating a new file: -.. code:: bash +.. code:: sh - $ sops --pgp "E60892BB9BD89A69F759A1A0A3D652173B763E8F,84050F1D61AF7C230A12217687DF65059EF093D3,85D77543B3D624B63CEA9E6DBC17301B491B3F21" mynewfile.yaml + $ sops --pgp "E60892BB9BD89A69F759A1A0A3D652173B763E8F,84050F1D61AF7C230A12217687DF65059EF093D3,85D77543B3D624B63CEA9E6DBC17301B491B3F21" mynewfile.yaml Threat Model ------------ -The security of the data stored using sops is as strong as the weakest +The security of the data stored using SOPS is as strong as the weakest cryptographic mechanism. Values are encrypted using AES256_GCM which is the strongest symmetric encryption algorithm known today. Data keys are encrypted in either KMS, which also uses AES256_GCM, or PGP which uses either RSA or @@ -1664,7 +1666,7 @@ Compromised AWS credentials grant access to KMS master key ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ An attacker with access to an AWS console can grant itself access to one of -the KMS master keys used to encrypt a sops data key. This threat should be +the KMS master keys used to encrypt a ``sops`` data key. This threat should be mitigated by protecting AWS accesses with strong controls, such as multi-factor authentication, and also by performing regular audits of permissions granted to AWS users. @@ -1674,29 +1676,29 @@ Compromised PGP key PGP keys are routinely mishandled, either because owners copy them from machine to machine, or because the key is left forgotten on an unused machine -an attacker gains access to. When using PGP encryption, sops users should take +an attacker gains access to. When using PGP encryption, SOPS users should take special care of PGP private keys, and store them on smart cards or offline as often as possible. Factorized RSA key ~~~~~~~~~~~~~~~~~~ -sops doesn't apply any restriction on the size or type of PGP keys. A weak PGP +SOPS doesn't apply any restriction on the size or type of PGP keys. A weak PGP keys, for example 512 bits RSA, could be factorized by an attacker to gain -access to the private key and decrypt the data key. Users of sops should rely +access to the private key and decrypt the data key. Users of SOPS should rely on strong keys, such as 2048+ bits RSA keys, or 256+ bits ECDSA keys. Weak AES cryptography ~~~~~~~~~~~~~~~~~~~~~ A vulnerability in AES256_GCM could potentially leak the data key or the KMS -master key used by a sops encrypted file. While no such vulnerability exists +master key used by a SOPS encrypted file. While no such vulnerability exists today, we recommend that users keep their encrypted files reasonably private. Backward compatibility ---------------------- -``sops`` will remain backward compatible on the major version, meaning that all +SOPS will remain backward compatible on the major version, meaning that all improvements brought to the 1.X and 2.X branches (current) will maintain the file format introduced in **1.0**. @@ -1727,8 +1729,8 @@ and had not been possible without the contributions of numerous `contributors `_, -`credstash `_ , +SOPS was inspired by `hiera-eyaml `_, +`credstash `_, `sneaker `_, `password store `_ and too many years managing PGP encrypted files by hand... diff --git a/docs/release.md b/docs/release.md index 42207b35c..7485b136a 100644 --- a/docs/release.md +++ b/docs/release.md @@ -54,6 +54,7 @@ This configuration is quite sophisticated, and ensures at least the following: git checkout main git pull ``` + - [ ] Create a **signed tag** for the release, using the following command: ```sh diff --git a/shamir/README.md b/shamir/README.md index d92f3b557..256077de7 100644 --- a/shamir/README.md +++ b/shamir/README.md @@ -1,3 +1,5 @@ +# Shamir's secret sharing + Forked from [Vault](https://github.com/hashicorp/vault/tree/master/shamir) ## How it works @@ -102,7 +104,6 @@ L(x) = So the polynomial we were looking for is `y = x^2`. - ## Splitting a secret So we have the ability of splitting a function into parts, but in the context