Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add jumpbox clone command to dev tool #2054

Merged
merged 5 commits into from
Nov 17, 2023
Merged

Add jumpbox clone command to dev tool #2054

merged 5 commits into from
Nov 17, 2023

Conversation

macrael
Copy link
Contributor

@macrael macrael commented Nov 14, 2023

Summary

This PR adds a subcommand to ./dev to allow cloning down a db from dev/val/prod to your local machine

Related issues

https://qmacbis.atlassian.net/browse/MCR-2633


if (jumpboxStartState !== 80) {
console.info('Jumpbox is not stopped yet. waiting to start it')
// wait for it to be running
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we're waiting for it to be stopped here? Probably should just fix that comment.

return startedInstance
}

async function ensureWhitelistIP(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not to be that sjw, but I think people like to use "allowlist"/"blocklist" more recently.

await ssh.connect({
host: jumpboxIP,
username: 'ubuntu',
privateKeyPath: '/Users/macrae/.ssh/wml_jumpbox',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, we're going to need to use each individual's private key here. I looked at the lib and it seems to require a path. There's another lib that can read from ssh-agent, but we should probably think about what to do here.

@@ -181,7 +184,11 @@ resources:
UserData:
Fn::Base64: !Sub |
#!/bin/bash
apt update && apt install unzip postgresql postgresql-contrib -y
# get apt data for postgres-14
sudo sh -c 'echo "deb http://apt.postgresql.org/pub/repos/apt $(lsb_release -cs)-pgdg main" > /etc/apt/sources.list.d/pgdg.list'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't really need a sudo here as everything runs as root.

One thing though is that the user data scripts only get run once and it's at time of creation, so we should make sure we cycle the machines so this gets applied on dev/val/prod VMs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've already run this on the existing jump boxes, should we be cycling these automatically?

@haworku
Copy link
Contributor

haworku commented Nov 14, 2023

YAY! A few comments and TODOs from running through things

  • Add docs for this dev command and note when it should be used. Also any prerequisites? Does the developer need to add their IP to the ssh allowlist for the env for example?
  • Address Error: Region is missing AWS config error on initial run. You said this might be something to add to the script itself.
  • Address failed to connect to jumpbox over ssh AssertionError [ERR_ASSERTION]: config.privateKeyPath does not exist at given fs path. You said script could possibly just use the generic ssh key location.
  • Have the script print out where the file was copied to locally which path and what filename? I have a few versions of the file so having the filename would be important.
  • Have the script put the files that are created into a specific folder that is .gitignored for this repo. Don't want things accidentally committed when this is run in our repo root.
  • After merging check that we addressed error fetching jumpbox Error: Did not find one and only one instance when trying to run on VAL and above

@macrael macrael merged commit 6ea94cb into main Nov 17, 2023
27 checks passed
@macrael macrael deleted the wml-clone-db-script branch November 17, 2023 04:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants