Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Migrate to same setup as Oracle Relayer #18

Merged
merged 3 commits into from
Aug 29, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .trunk/trunk.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
# To learn more about the format of this file, see https://docs.trunk.io/reference/trunk-yaml
version: 0.1
cli:
version: 1.22.2
version: 1.22.3
# Trunk provides extensibility via plugins. (https://docs.trunk.io/plugins)
plugins:
sources:
- id: trunk
ref: v1.6.1
ref: v1.6.2
uri: https://github.com/trunk-io/plugins
# Many linters and tools depend on runtimes - configure them here. (https://docs.trunk.io/runtimes)
runtimes:
Expand All @@ -23,25 +23,25 @@ lint:
- markdown-table-prettify
enabled:
- actionlint@1.7.1
- checkov@3.2.198
- checkov@3.2.236
- dotenv-linter@3.3.0
- dustilock@1.2.0
- eslint@9.7.0
- eslint@9.9.1
- git-diff-check
- gitleaks@8.18.4
- markdown-link-check@3.12.2
- markdownlint@0.41.0
- osv-scanner@1.8.2
- osv-scanner@1.8.4
- oxipng@9.1.2
- pre-commit-hooks@4.6.0
- prettier@3.3.3
- shellcheck@0.10.0
- shfmt@3.6.0
- sort-package-json@2.10.0
- sort-package-json@2.10.1
- terraform@1.1.0
- terrascan@1.19.1
- tflint@0.52.0
- trufflehog@3.80.1
- tflint@0.53.0
- trufflehog@3.81.9
- yamllint@1.35.1
actions:
disabled:
Expand Down
162 changes: 162 additions & 0 deletions DEPLOY_FROM_SCRATCH.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,162 @@
# Deployment from Scratch

How to deploy the entire governance watchdog infrastructure from scratch.

- [Infra Deployment via Terraform](#infra-deployment-via-terraform)
- [Terraform State Management](#terraform-state-management)
- [Google Cloud Permission Requirements](#google-cloud-permission-requirements)
- [Using Service Account Impersonation (recommended)](#using-service-account-impersonation-recommended)
- [Using Your Own Gcloud User Account (not recommended)](#using-your-own-gcloud-user-account-not-recommended)
- [Deployment](#deployment)
- [Debugging Problems](#debugging-problems)
- [View Logs](#view-logs)
- [Teardown](#teardown)

## Infra Deployment via Terraform

### Terraform State Management

- The Terraform State for this project lives in our shared Terraform Seed Project with the ID `mento-terraform-seed-ffac`
- Deploying the project for the first time should automatically create a subfolder in the [google storage bucket used for terraform state management in the seed project](https://console.cloud.google.com/storage/browser/mento-terraform-tfstate-6ed6;tab=objects?forceOnBucketsSortingFiltering=true&project=mento-terraform-seed-ffac&prefix=&forceOnObjectsSortingFiltering=false)

### Google Cloud Permission Requirements

#### Using Service Account Impersonation (recommended)

The project is preconfigured to impersonate our shared terraform service account (see `./infra/versions.tf`).
The only permission you will need on your own gcloud user account is `roles/iam.serviceAccountTokenCreator` to allow you to impersonate our shared terraform service account.

#### Using Your Own Gcloud User Account (not recommended)

If for whatever reason service account impersonation doesn't work, you'll need at least the following permissions on your personal gcloud account to deploy this project with terraform:

- `roles/resourcemanager.folderViewer` on the folder that you want to create the project in
- `roles/resourcemanager.organizationViewer` on the organization
- `roles/resourcemanager.projectCreator` on the organization
- `roles/billing.user` on the organization
- `roles/storage.admin` to allow creation of new storage buckets

### Deployment

<!-- markdown-link-check-disable -->

1. Run `./bin/set-up-terraform.sh` to check required permissions and provision all required terraform providers and modules

1. Create a `./infra/terraform.tfvars` file. This is like `.env` for Terraform:

```sh
touch ./infra/terraform.tfvars
# This file is `.gitignore`d to avoid accidentally leaking sensitive data
```

1. Add Google Cloud Org ID and Billing Account to your local `terraform.tfvars`

```hcl
# Required for creating new GCP projects
# Get it via `gcloud organizations list`
org_id = "<our-org-id>"

# Required for creating new GCP projects
# Get it via `gcloud billing accounts list` (pick the GmbH account)
billing_account = "<our-billing-account-id>"
```

1. [Create a Discord Webhook URL](https://support.discord.com/hc/en-us/articles/228383668-Intro-to-Webhooks) for the channel you want to receive notifications in <!-- markdown-link-check-enable -->

1. Add the Discord Webhook URL to your local `terraform.tfvars`:

```sh
# This will be stored in Google Secret Manager upon deployment via Terraform
echo "discord_webhook_url = \"<discord-webhook-url>"" >> terraform.tfvars
```

1. Create a Telegram group and invite a new bot into it

- Open a new telegram chat with @BotFather
- Use the `/newbot` command to create a new bot
- Copy the API key printed out at the end of the prompt and store it in your `terraform.tfvars`

```hcl
telegram_bot_token = "<bot-api-key>"
```

- Get the Chat ID by inviting @MissRose_bot to the group and then using the `/id` command
- Add the Chat ID to your `terraform.tfvars`

```hcl
telegram_chat_id = "<group-chat-id>"
```

- Remove @MissRose_bot after you got the Chat ID

1. Get (or generate if non-existing) a QuickNode API key to enable Terraform to provision QuickAlerts

- Grab the API key from our QuickNode dashboard: <https://dashboard.quicknode.com/api-keys>
- Add it to `terraform.tfvars`

```hcl
quicknode_api_key = "<quicknode-api-key>"
```

1. Get a VictorOps webhook URL by copying the Service API Endpoint URL from the [VictorOps Stackdriver Integration](https://portal.victorops.com/dash/mento-labs-gmbh#/advanced/stackdriver). The routing key can be founder under the [`Settings`](https://portal.victorops.com/dash/mento-labs-gmbh#/routekeys) tab

```hcl
# Required to send on-call alerts to VictorOps
victorops_webhook_url = "<victorops-webhook-url>/<victorops-routing-key>"
```

1. Generate an auth key to allow us to test the deployed function from our local machines

- You can use your password manager to generate a long and secure (url-compatible) key
- Add it to `terraform.tfvars`

```hcl
x_auth_token = "<x-auth-token>"
```

1. **Deploy the entire project via `terraform apply`**

- You will see an overview of all resources to be created. Review them if you like and then type "Yes" to confirm.
- This command can take up to 10 minutes because it does a lot of work creating and configuring all defined Google Cloud Resources
- ❌ Given the complexity of setting up an entire Google Cloud Project incl. service accounts, permissions, etc., you might run
into deployment errors with some components.

**Often a simple retry of `terraform apply` helps**. Sometimes a dependency of a resource has simply not finished creating when terraform already tried to deploy the next one, so waiting a few minutes for things to settle can help.

1. Set your local `gcloud` project ID to our freshly created one and populate your local cache with frequently used project values:

```sh
npm run cache:clear
```

1. Check that everything worked as expected

```sh
# 1. Call the deployed function via:
npm run test:prod

# 2. Monitor the configured Discord channel for a message to appear
open https://discord.com/channels/966739027782955068/1262714272476037212

# 3. Monitor the configured Telegram channel for a message to appear

# 4. Check the function logs via:
npm run logs # prints logs into your local terminal (with a few seconds of latency)
# OR
npm run logs:url # prints a URL to the cloud console logs in the browser
```

## Debugging Problems

### View Logs

For most problems, you'll likely want to check the cloud function logs first.

- `npm run logs` will print the latest 50 staging log entries into your local terminal for quick and easy access
- `npm run logs:url` will print the URL to the staging function logs in the Google Cloud Console for full access

## Teardown

1. Run `npm run destroy` to delete the entire production environment from google cloud
- You might run into permission issues here, especially around deleting the associated billing account resources
- I didn't have time to figure out the minimum set of permissions required to delete this project so the easiest would be to let an organization owner (i.e. Bogdan) run this with full permissions if you face any issues
Loading
Loading