nubis-dpaste

This is a Nubis deployment of the dpaste application. The purpose of this project is to demonstrate an example of a django / python deployment. This repository is an example of a "deployment repository", in other words a repository that does not contain any application code.

Repository Structure

The structure of the repository is quite simple. The application is installed as a git submodule. There is a directory called nubis which contains all of the bits necessary to deploy the application.

Deployment Process

Currently there are two steps necessary to deploy this project. While these steps are listed in order to build and deploy, it is typically not necessary to run the build steps. This means you can skip the Nubis Builder bits and jump straight to the Terraform section.

Puppet

We are using puppet in this example to bootstrap up our instance. Puppet installs and configures services such as Apache and MySQL. We are using the nubis-puppet project for our module collection. This is conveniently installed on the base image (built by nubis-base) that we are going to use as the starting image for our Packer build in the next step. This means that there is nothing for you to do here, however you should be aware of this for when you deploy your own applications.

Nubis Builder

Nubis Builder is the piece that will build our Amazon Machine Image (AMI). It is made up of a two pieces:

1. The main.json file which:
   • Contains one or more provisioners statements for calling any bootstrapping commands for the application
   • Invokes a puppet standalone run through nubis-builder
   • Invokes a builders statement describing where to build the AMI, also run through nubis-builder
2. A project.json file which contains settings for nubis-builder

To run nubis-builder, from the repository root you first need to ensure that you have installed the prerequisites and nubis-builder. After which you simply call nubis-builder like so:

nubis-builder build

This takes around 8m7.350s to complete.

Terraform

The next step is to take the shiny new AMI that nubis-builder built and deploy it. This is where Terraform comes into play. Terraform is our infrastructure deployment framework, and it's a fantastic tool for the job. All in all, it consists of only a few files:

0. terraform.tfvars-dist simply lists the inputs you will need to provide
1. main.tf is where the real heavy lifting takes place. This is where you describe your infrastructure. Things like worker pools, load balancers, DNS and so on.
2. outputs.tf is where you can define handy outputs from your deployment, like the final URL of the deployed application.
3. consul.tf is where you can define the infrastructure settings your app needs access to.
4. README.md contains some handy cut-and-paste cheat-sheet style commands for your future reference.

To get ready to execute Terraform, first change your current directory to the terraform/ directory

$> cd nubis/terraform

Then, to execute Terraform, you first need to create your terraform.tfvars file, for which there is a template provided (terraform.tfvars-dist). After which you will execute Terraform.

NOTE: You will likely need to change the service_name from dpaste- to something unique as each deployment requires a unique name.

Also, if you skipped the build step above you can use the pre-built AMIs to deploy with:

• us-east-1: ami-ebcab6fc
• us-west-2: ami-eea67b8e

Terraform Get

Terraform makes heavy uses of modules, and before we can successfully deploy an application with it, we need it to first download the modules it needs

$> terraform get -update=true
Get: git::https://github.com/nubisproject/nubis-terraform.git?ref=master (update)
Get: git::https://github.com/nubisproject/nubis-terraform.git?ref=master (update)
Get: git::https://github.com/nubisproject/nubis-terraform.git?ref=master (update)
Get: git::https://github.com/nubisproject/nubis-terraform.git?ref=master (update)
Get: git::https://github.com/nubisproject/nubis-terraform.git?ref=master (update)

Terraform Plan

Now, we are ready to plan our deployment. In terraform parlance, this means just previewing all the steps necessary to achieve the deployment we have specified, without actually making any changes to the infrastructure yet.

$> CONSUL_HTTP_SSL_VERIFY=0 aws-vault exec account-name-admin -- terraform plan
2016/09/20 11:58:04 Parsing config file /home/gozer/.aws/config
2016/09/20 11:58:04 Looking up keyring for nubis-lab
2016/09/20 11:58:04 Using session ****************XXXX, expires in 55m30.765625184s
2016/09/20 11:58:04 Assuming role arn:aws:iam::XXXXXXXXXX:role/nubis/admin/gozer
2016/09/20 11:58:04 Using role ****************YYYY, expires in 14m59.434980674s
2016/09/20 11:58:04 Parsing config file /home/gozer/.aws/config
Refreshing Terraform state prior to plan...

module.database.info.terraform_remote_state.info: Refreshing state... (ID: 2016-09-20 15:55:32.045480098 +0000 UTC)
module.dns.info.terraform_remote_state.info: Refreshing state... (ID: 2016-09-20 15:55:32.046647692 +0000 UTC)
module.worker.info.terraform_remote_state.info: Refreshing state... (ID: 2016-09-20 15:55:32.045973702 +0000 UTC)
module.load_balancer.info.terraform_remote_state.info: Refreshing state... (ID: 2016-09-20 15:55:32.048851647 +0000 UTC)
[...]
-/+ module.worker.aws_autoscaling_group.asg
[...]
-/+ module.worker.aws_autoscaling_policy.down
[...]
-/+ module.worker.aws_autoscaling_policy.up
[...]
~ module.worker.aws_cloudwatch_metric_alarm.down
[...]
~ module.worker.aws_cloudwatch_metric_alarm.up
[...]
-/+ module.worker.aws_launch_configuration.launch_config
[...]

Plan: 4 to add, 2 to change, 4 to destroy.Plan: 4 to add, 2 to change, 4 to destroy.

A few important things of note here, for completeness.

First, notice CONSUL_HTTP_SSL_VERIFY=0, this is an unfortunate side-effect of not yet being able to verify the SSL certificate of the Consul endpoint, and is especially annoying for developer launches, as it can't be set anywhere but in the environemnt at the moment. Save yourself some trouble in the future and stick it somewhere in your ~/.bash_profile

Second, for security, we've wrapped the invocation of Terraform with aws-vault, so we don't have to directly manage or manipulate AWS credentials.

Third, we could also have passed in variables with the -var argument, instead of in the variables file. Command-line arguments take precedence over the contents of the file. So to plan for a new AMI, we could have instead done:

$> terraform plan -var ami=ami-XYZ123

And finally, the output of Terraform's plan shows precisely what steps will be taken, and they are quite human parseable. In this example, for instance, we can see this deploy will change an autoscaling group, a launch configuration, and some associated autoscaling policies and cloudwatch alarms.

Nothing has hapenned yet.

Terraform Apply

Now that we have reviewed the proposed changes, we can apply them with:

$> CONSUL_HTTP_SSL_VERIFY=0 aws-vault exec account-name-admin -- terraform apply
2016/09/20 11:58:04 Parsing config file /home/gozer/.aws/config
2016/09/20 11:58:04 Looking up keyring for nubis-lab
2016/09/20 11:58:04 Using session ****************XXXX, expires in 55m30.765625184s
2016/09/20 11:58:04 Assuming role arn:aws:iam::XXXXXXXXXX:role/nubis/admin/gozer
2016/09/20 11:58:04 Using role ****************YYYY, expires in 14m59.434980674s
2016/09/20 11:58:04 Parsing config file /home/gozer/.aws/config
Refreshing Terraform state prior to plan...

module.database.info.terraform_remote_state.info: Refreshing state... (ID: 2016-09-20 15:55:32.045480098 +0000 UTC)
module.dns.info.terraform_remote_state.info: Refreshing state... (ID: 2016-09-20 15:55:32.046647692 +0000 UTC)
[...]
module.worker.aws_launch_configuration.launch_config: Creating...
module.worker.aws_launch_configuration.launch_config: Creation complete
module.worker.aws_autoscaling_group.asg: Creating...
module.worker.aws_autoscaling_group.asg: Still creating... (10s elapsed)
module.worker.aws_autoscaling_group.asg: Still creating... (20s elapsed)
module.worker.aws_autoscaling_group.asg: Creation complete
module.worker.aws_cloudwatch_metric_alarm.up: Modifying...
module.worker.aws_cloudwatch_metric_alarm.down: Modifying...
module.worker.aws_autoscaling_group.asg: Destroying...
[...]
Apply complete! Resources: 4 added, 2 changed, 4 destroyed.

Outputs:

  address = https://www.dpaste-<username>.<env>.<env>.<region>.<account>.allizom.org/

And we can see here that all is well, some resources got modified, while others got created and destroyed. In the end, we see the outputs provided by our deployment.

Quick Commands

git clone https://github.com/mozilla/nubis-dpaste.git

git submodule update --init --recursive

nubis-builder build

*Edit nubis/terraform/terraform.tfvars

cd nubis/terraform

* Download/update TF modules
CONSUL_HTTP_SSL_VERIFY=0 aws-vault exec account-name-admin -- terraform get

* Preview proposed changes
CONSUL_HTTP_SSL_VERIFY=0 aws-vault exec account-name-admin -- terraform plan

*Apply proposed changes
CONSUL_HTTP_SSL_VERIFY=0 aws-vault exec account-name-admin -- terraform apply