Infrastructure as Code for Digital Records Repository's (DR2) AWS environments, see nationalarchives/dr2-ingest for more information and documentation.
The project is divided into separate files corresponding to one part of the infrastructure. All of these files are run at once when terraform runs.
common.tf
Common modules like VPCs, shared security groups and system-wide secretscustodial_copy.tf
Shared resources for the custodial copy workflow.slack_notifications_lambda
Resources for the notifications lambda.deploy_preservica_config
A lambda, queue, topic and bucket for deploying XML config to Preservica.
To start a deployment, run the DR2 Terraform Environments Deploy job in GitHub actions by clicking 'Run Workflow' and selecting the environment you want to deploy to. All changes must be deployed first to integration, then staging, then production.
The deployment will pause when Terraform has determined which changes need to be applied. Review the Terraform plan output by clicking the link provided in the Slack notification. This will be a link to Cloudwatch in the management account so you will need to be logged in to the management AWS account to use this.
Check whether the changes look correct, then open the actions approval page and accept or reject them. To find the actions approval page, follow the link from the Slack notification.
Deployments can be approved by anyone in the digital-records-repository
GitHub team.
Each environment has one elastic IP per AZ created manually within the AWS console and then used within terraform using data "aws_eip"
.
This removes the risk of the EIP being accidentally deleted as this would change the IP address, which is allow-listed by the Preservation System service provider.
See: https://learn.hashicorp.com/terraform/getting-started/install.html
See: https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html
HCL Language Support: https://plugins.jetbrains.com/plugin/7808-hashicorp-terraform--hcl-language-support
NOTE: Running Terraform locally should only be used to check the Terraform plan. Updating the DR2 environments should only ever be done through GitHub actions
-
Clone DR2 Environments project to local machine: https://github.com/nationalarchives/dr2-terraform-environments and navigate to the directory
-
Switch to the Terraform workspace corresponding to the DR2 environment to be worked on:
[location of project] $ terraform workspace select intg
-
Set the following Terraform environment variables on the local environment:
- TF_VAR_account_number=[account number of the environment to update]
-
Initialize Terraform (if not done so previously):
[location of project] $ terraform init
-
To ensure the modules are up-to-date, run
[location of project] $ terraform get -update
-
Make your terraform changes
- Add/update a tf file to the root of this project (might be best to copy an existing tf file as a base)
- If you are creating a Lambda, add its arn to the
deploy_lambda_policy
in thedeploy_roles
file at the root of this project
- If you are creating a Lambda, add its arn to the
- Add/update an IAM policy, depending on the change you are making
- If what you've created is part of a step function (e.g. a lambda):
- add it to the step function's module in the
common.tf
- add it to the step function's policy module in the
common.tf
- add it to the step function's
json.tpl
file in theiam_policy
folder - add it to the step function's
json.tpl
file in thesfn
folder
- add it to the step function's module in the
- If item created needs a KMS key, add it to the
dr2_kms_key
module in thecommon.tf
file - If this is a lambda which needs to be added to the ingest dashboard, add the lambda name to
local.dashboard_lambdas
incommon.tf
- Add/update a tf file to the root of this project (might be best to copy an existing tf file as a base)
-
(Optional) To quickly validate the changes you made, run
[location of project] $ terraform validate
-
Run Terraform to view changes that will be made to the DR2 environment AWS resources
- Make sure your credentials are valid
- If you have the AWS CLI installed, run
aws sso login --profile [account name where credentials are] && export AWS_PROFILE=[account name where credentials are]
- If you have the AWS CLI installed, run
[location of project] $ terraform plan
- Make sure your credentials are valid
-
Run
terraform fmt --recursive
to properly format your Terraform changes before pushing to a branch.
- Terraform website: https://www.terraform.io/
- Terraform basic tutorial: https://learn.hashicorp.com/terraform/getting-started/build