Rotate Linux builder PAT for HRA2 and upgrade runner software

[ThetaSinner]

• Replace PAT which had expired
• Update runner software
• Accept changes to my own SSH keys

[ThetaSinner]

Was this needed? We can always make a new token if we need to.

Don't want to have to set up GPG on my system if I can avoid it

[steveej]

probably stating the obvious: it allows decrypting and viewing the secret with the given public key.

did you get an error at some point? if you use the sops from within nix develop the required binaries to encrypt the keys should be there. i'd prefer having the keys readable on my machine so it'd be good to have this back with the new secret as well.

[ThetaSinner]

I can't enter a dev shell and there's no docs to tell me how to fix that:

> nix develop
++ REPO_SECRETS_DIR=/home/thetasinner/.holochain-infra-secrets
++ mkdir -p /home/thetasinner/.holochain-infra-secrets
++ chmod 700 /home/thetasinner/.holochain-infra-secrets
++ export NOMAD_CLIENT_KEY=/home/thetasinner/.holochain-infra-secrets/global-cli-nomad-key
++ NOMAD_CLIENT_KEY=/home/thetasinner/.holochain-infra-secrets/global-cli-nomad-key
++ sops -d secrets/nomad/cli/keys.yaml
++ yq .global-cli-nomad-key
Failed to get the data key required to decrypt the SOPS file.

Group 0: FAILED
  6F7069FE6B96E894E60EC45C6EEFA706CB17E89B: FAILED
    - | could not decrypt data key with PGP key:
      | github.com/ProtonMail/go-crypto/openpgp error: could not
      | load secring: open /home/thetasinner/.gnupg/pubring.gpg: no
      | such file or directory; GnuPG binary error: failed to
      | decrypt sops data key with pgp: gpg: encrypted with RSA key,
      | ID 44871BFCC5DF6DBB
      | gpg: decryption failed: No secret key
  
  age1ygzy9clj0xavlmau0ham7j5nw8yy4z0q8hvkfpdgwc4fcr8nufpqrdxgvx: FAILED
    - | failed to load age identities: failed to open file: open
      | /home/thetasinner/.config/sops/age/keys.txt: no such file or
      | directory
  
  age194xfar0gfdauu2dcxwqk9lh9d0vjfrzzs2ke0ppanpwv9eqxzs2qp7q7cn: FAILED
    - | failed to load age identities: failed to open file: open
      | /home/thetasinner/.config/sops/age/keys.txt: no such file or
      | directory

Recovery failed because no master key was able to decrypt the file. In
order for SOPS to recover the file, at least one key has to be successful,
but none were.
+ command rm -f /tmp/nix-shell.j9KGXu

I don't see that the Nix shell sets up GPG anyway though? I would still need to import your public key to be able encrypt with it. It's not like age where the public key is just a string. The GPG identifiers are just fingerprints so I won't be able to do anything with that.

I get that being able to decrypt is useful for debugging but we really shouldn't need to have access to these tokens beyond doing the job of transferring them GitHub to the builder. That's why I'm questioning the need to retain this?

[steveej]

I can't enter a dev shell and there's no docs to tell me how to fix that:

gotcha, that error is noisy however it won't actually prevent the shell from spawning. despite that it's not a good long-term solution anyway to have that secret decrypted to the filesystem permanently. i'll clean that up in the near-term, i created an issue for tracking that.

I don't see that the Nix shell sets up GPG anyway though? I would still need to import your public key to be able encrypt with it.

sops transparently retrieves publicly available keys automatically so it should've just worked to encrypt the secret towards my public key. in the case of my key lookup by fingerprint works. would you mind giving this a try?

I get that being able to decrypt is useful for debugging but we really shouldn't need to have access to these tokens beyond doing the job of transferring them GitHub to the builder. That's why I'm questioning the need to retain this?

the sops updatekeys command requires temporarily decrypting the secrets. maybe in a perfect set up we'd never update and generate new secrets instead and re-deploy all relevant machines. it seems like a leap from here and i'd like to keep the convenience of re-encrypting existing keys until we have the better solution. WDYT?

[steveej]

@ThetaSinner PTAL at my comments above. given your changes are already deployed, i'd like to resolve this one way or another to get your PR merged to develop.

[ThetaSinner]

Having to update keys would mean that the SSH key of the Linux builder has changed. Given that these tokens expire anyway, I don't see the problem with needing to issue a new PAT if the machine's identity changes. It doesn't seem like a lift to me, it just works and is something to be aware of if the SSH key for the Linux builder changes.

[steveej]

Having to update keys would mean that the SSH key of the Linux builder has changed.

also that we've added another machine. agreed that it works to create a new PAT. it does make (disaster recovery) deployment slightly slower as i've had issues with new PATs in the past with missing permissions. i think i'm getting clarity that the quick and reliably generation of a minimally scoped PAT is the crux here.

[steveej]

#90

[ThetaSinner]

I relied on the GH UI to tell me what permissions the new token needed - it shows you what was available to the previous token even though you can't get at the token value. E.g. one of my tokens