diff --git a/CHANGELOG.md b/CHANGELOG.md index e71e2e3e3..f7ad5bd31 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,23 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](http://keepachangelog.com/) and this project adheres to [Semantic Versioning](http://semver.org/). +## 4.0.0 - 2019-07-29 + +### Terraform 0.12 + +Module is available as Terraform 0.12 module, pin to version 4.x. Please submit pull-requests to the `develop` branch. + +Migration from 0.11 to 0.12 is tested for the `runner-default` example. To migrate the runner, execute the following steps. + +- Update to Terraform 0.12 +- Migrate your Terraform code via Terraform `terraform 0.12upgrade`. +- Update the module from 3.10.0 to 4.0.0, next run `terraform init` +- Run `terraform apply`. This should trigger only a re-creation of the the auto launch configuration and a minor change in the auto-scaling group. + +### Terraform 0.11 + +Module is available as Terraform 0.11 module, pin module to version 3.x. Please submit pull-requests to the `terraform011` branch. + ## 3.10.0 - 2019-07-29 - Chnaged - THe user data script for the EC2 runner agent instance is not logging anymore on trace level. To enable bash xtrace set `enable_runner_user_data_trace_log` to `true`. #49 @@ -177,7 +194,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/). - Update default AMI's to The latest Amazon Linux AMI 2017.09.1 - released on 2018-01-17. - Minor updates in the example -[Unreleased]: https://github.com/npalm/terraform-aws-gitlab-runner/compare/3.10.0...HEAD +[Unreleased]: https://github.com/npalm/terraform-aws-gitlab-runner/compare/4.0.0...HEAD +[4.0.0]: https://github.com/npalm/terraform-aws-gitlab-runner/compare/3.10.0...4.0.0 [3.10.0]: https://github.com/npalm/terraform-aws-gitlab-runner/compare/3.9.0...3.10.0 [3.9.0]: https://github.com/npalm/terraform-aws-gitlab-runner/compare/3.8.0...3.9.0 [3.8.0]: https://github.com/npalm/terraform-aws-gitlab-runner/compare/3.7.0...3.8.0 diff --git a/README.md b/README.md index ad79e64cf..6e4f280c7 100644 --- a/README.md +++ b/README.md @@ -129,7 +129,7 @@ Finally, the runner still supports the manual runner creation. No changes are re By default the module creates a a cache for the runner in S3. Old objects are automatically remove via a configurable life cycle policy on the bucket. -Creation of the bucket can be disabled and managed outside this module. A good use case is for sharing the cache cross multiple runners. For this purpose the cache is implemented as sub module. For more details see the [cache module](./cache). An example implementation of this use case can be find in the [runner-public](https://github.com/npalm/terraform-aws-gitlab-runner/tree/develop/examples/runner-public) example. +Creation of the bucket can be disabled and managed outside this module. A good use case is for sharing the cache cross multiple runners. For this purpose the cache is implemented as sub module. For more details see the [cache module](https://github.com/npalm/terraform-aws-gitlab-runner/tree/4.0.0/cache). An example implementation of this use case can be find in the [runner-public](https://github.com/npalm/terraform-aws-gitlab-runner/tree/__GIT_REF__/examples/runner-public) example. ## Usage @@ -147,7 +147,7 @@ The base image used to host the GitLab Runner agent is the latest available Amaz ### Usage module -Below a basic examples of usages of the module. The dependencies such as a VPC, and SSH keys have a look at the [default example](https://github.com/npalm/terraform-aws-gitlab-runner/tree/develop/examples/runner-default). +Below a basic examples of usages of the module. The dependencies such as a VPC, and SSH keys have a look at the [default example](https://github.com/npalm/terraform-aws-gitlab-runner/tree/4.0.0/examples/runner-default). ``` hcl module "runner" { @@ -179,7 +179,7 @@ module "runner" { ## Examples -A few [examples](examples) are provided. Use the following steps to deploy. Ensure your AWS and Terraform environment is set up correctly. All commands below should be run from the `terraform-aws-gitlab-runner/examples/` directory. +A few [examples](https://github.com/npalm/terraform-aws-gitlab-runner/tree/4.0.0/examples/) are provided. Use the following steps to deploy. Ensure your AWS and Terraform environment is set up correctly. All commands below should be run from the `terraform-aws-gitlab-runner/examples/` directory. ### SSH keys @@ -210,3 +210,88 @@ To destroy runner: ``` sh terraform destroy ``` + +## Inputs + +| Name | Description | Type | Default | Required | +|------|-------------|:----:|:-----:|:-----:| +| allow\_iam\_service\_linked\_role\_creation | Boolean used to control attaching the policy to a runner instance to create service linked roles. | bool | `"true"` | no | +| ami\_filter | List of maps used to create the AMI filter for the Gitlab runner agent AMI. Currently Amazon Linux 2 `amzn2-ami-hvm-2.0.????????-x86_64-ebs` looks to *not* be working for this configuration. | map(list(string)) | `` | no | +| ami\_owners | The list of owners used to select the AMI of Gitlab runner agent instances. | list(string) | `` | no | +| aws\_region | AWS region. | string | n/a | yes | +| aws\_zone | AWS availability zone (typically 'a', 'b', or 'c'). | string | `"a"` | no | +| cache\_bucket | Configuration to control the creation of th the cache bucket. By default the bucket will be crated and used as shared cache. To use the same cache cross multiple runners disable the cration of the cache and provice a policy and bucket name. See the public runner example for more details. | map | `` | no | +| cache\_bucket\_prefix | Prefix for s3 cache bucket name. | string | `""` | no | +| cache\_bucket\_versioning | Boolean used to enable versioning on the cache bucket, false by default. | bool | `"false"` | no | +| cache\_expiration\_days | Number of days before cache objects expires. | number | `"1"` | no | +| cache\_shared | Enables cache sharing between runners, false by default. | bool | `"false"` | no | +| create\_runners\_iam\_instance\_profile | Boolean to control the creation of the runners IAM instance profile | bool | `"true"` | no | +| docker\_machine\_instance\_type | Instance type used for the instances hosting docker-machine. | string | `"m5a.large"` | no | +| docker\_machine\_options | List of additional options for the docker machine config. Each element of this list must be a key=value pair. E.g. '["amazonec2-zone=a"]' | list(string) | `` | no | +| docker\_machine\_role\_json | Docker machine runner instance override policy, expected to be in JSON format. | string | `""` | no | +| docker\_machine\_spot\_price\_bid | Spot price bid. | string | `"0.06"` | no | +| docker\_machine\_ssh\_cidr\_blocks | List of CIDR blocks to allow SSH Access to the docker machine runner instance. | list(string) | `` | no | +| docker\_machine\_user | Username of the user used to create the spot instances that host docker-machine. | string | `"docker-machine"` | no | +| docker\_machine\_version | Version of docker-machine. | string | `"0.16.1"` | no | +| enable\_cloudwatch\_logging | Boolean used to enable or disable the CloudWatch logging. | bool | `"true"` | no | +| enable\_gitlab\_runner\_ssh\_access | Enables SSH Access to the gitlab runner instance. | bool | `"false"` | no | +| enable\_manage\_gitlab\_token | Boolean to enable the management of the GitLab token in SSM. If `true` the token will be stored in SSM, which means the SSM property is a terraform managed resource. If `false` the Gitlab token will be stored in the SSM by the user-data script during creation of the the instance. However the SSM parameter is not managed by terraform and will remain in SSM after a `terraform destroy`. | bool | `"true"` | no | +| enable\_runner\_user\_data\_trace\_log | Enable bash xtrace for the user data script that creates the EC2 instance for the runner agent. Be aware this could log sensitive data such as you GitLab runner token. | bool | `"false"` | no | +| environment | A name that identifies the environment, used as prefix and for tagging. | string | n/a | yes | +| gitlab\_runner\_registration\_config | Configuration used to register the runner. See the README for an example, or reference the examples in the examples directory of this repo. | map(string) | `` | no | +| gitlab\_runner\_ssh\_cidr\_blocks | List of CIDR blocks to allow SSH Access to the gitlab runner instance. | list(string) | `` | no | +| gitlab\_runner\_version | Version of the GitLab runner. | string | `"12.1.0"` | no | +| instance\_role\_json | Default runner instance override policy, expected to be in JSON format. | string | `""` | no | +| instance\_type | Instance type used for the GitLab runner. | string | `"t3.micro"` | no | +| overrides | This maps provides the possibility to override some defaults. The following attributes are supported: `name_sg` overwrite the `Name` tag for all security groups created by this module. `name_runner_agent_instance` override the `Name` tag for the ec2 instance defined in the auto launch configuration. `name_docker_machine_runners` ovverrid the `Name` tag spot instances created by the runner agent. | map(string) | `` | no | +| runner\_ami\_filter | List of maps used to create the AMI filter for the Gitlab runner docker-machine AMI. | map(list(string)) | `` | no | +| runner\_ami\_owners | The list of owners used to select the AMI of Gitlab runner docker-machine instances. | list(string) | `` | no | +| runner\_instance\_spot\_price | By setting a spot price bid price the runner agent will be created via a spot request. Be aware that spot instances can be stopped by AWS. | string | `""` | no | +| runners\_additional\_volumes | Additional volumes that will be used in the runner config.toml, e.g Docker socket | list | `` | no | +| runners\_concurrent | Concurrent value for the runners, will be used in the runner config.toml. | number | `"10"` | no | +| runners\_environment\_vars | Environment variables during build execution, e.g. KEY=Value, see runner-public example. Will be used in the runner config.toml | list(string) | `` | no | +| runners\_executor | The executor to use. Currently supports `docker+machine` or `docker`. | string | `"docker+machine"` | no | +| runners\_gitlab\_url | URL of the GitLab instance to connect to. | string | n/a | yes | +| runners\_iam\_instance\_profile\_name | IAM instance profile name of the runners, will be used in the runner config.toml | string | `""` | no | +| runners\_idle\_count | Idle count of the runners, will be used in the runner config.toml. | number | `"0"` | no | +| runners\_idle\_time | Idle time of the runners, will be used in the runner config.toml. | number | `"600"` | no | +| runners\_image | Image to run builds, will be used in the runner config.toml | string | `"docker:18.03.1-ce"` | no | +| runners\_limit | Limit for the runners, will be used in the runner config.toml. | number | `"0"` | no | +| runners\_monitoring | Enable detailed cloudwatch monitoring for spot instances. | bool | `"false"` | no | +| runners\_name | Name of the runner, will be used in the runner config.toml. | string | n/a | yes | +| runners\_off\_peak\_idle\_count | Off peak idle count of the runners, will be used in the runner config.toml. | number | `"0"` | no | +| runners\_off\_peak\_idle\_time | Off peak idle time of the runners, will be used in the runner config.toml. | number | `"0"` | no | +| runners\_off\_peak\_periods | Off peak periods of the runners, will be used in the runner config.toml. | string | `""` | no | +| runners\_off\_peak\_timezone | Off peak idle time zone of the runners, will be used in the runner config.toml. | string | `""` | no | +| runners\_output\_limit | Sets the maximum build log size in kilobytes, by default set to 4096 (4MB) | number | `"4096"` | no | +| runners\_post\_build\_script | Commands to be executed on the Runner just after executing the build, but before executing after_script. | string | `""` | no | +| runners\_pre\_build\_script | Script to execute in the pipeline just before the build, will be used in the runner config.toml | string | `""` | no | +| runners\_pre\_clone\_script | Commands to be executed on the Runner before cloning the Git repository. this can be used to adjust the Git client configuration first, for example. | string | `""` | no | +| runners\_privileged | Runners will run in privileged mode, will be used in the runner config.toml | bool | `"true"` | no | +| runners\_pull\_policy | pull_policy for the runners, will be used in the runner config.toml | string | `"always"` | no | +| runners\_request\_concurrency | Limit number of concurrent requests for new jobs from GitLab (default 1) | number | `"1"` | no | +| runners\_root\_size | Runner instance root size in GB. | number | `"16"` | no | +| runners\_shm\_size | shm_size for the runners, will be used in the runner config.toml | number | `"0"` | no | +| runners\_token | Token for the runner, will be used in the runner config.toml. | string | `"__REPLACED_BY_USER_DATA__"` | no | +| runners\_use\_private\_address | Restrict runners to the use of a private IP address | bool | `"true"` | no | +| secure\_parameter\_store\_runner\_token\_key | The key name used store the Gitlab runner token in Secure Parameter Store | string | `"runner-token"` | no | +| ssh\_key\_pair | Set this to use existing AWS key pair | string | `""` | no | +| ssh\_public\_key | Public SSH key used for the GitLab runner EC2 instance. | string | `""` | no | +| subnet\_id\_runners | List of subnets used for hosting the gitlab-runners. | string | n/a | yes | +| subnet\_ids\_gitlab\_runner | Subnet used for hosting the GitLab runner. | list(string) | n/a | yes | +| tags | Map of tags that will be added to created resources. By default resources will be tagged with name and environment. | map(string) | `` | no | +| userdata\_post\_install | User-data script snippet to insert after GitLab runner install | string | `""` | no | +| userdata\_pre\_install | User-data script snippet to insert before GitLab runner install | string | `""` | no | +| vpc\_id | The target VPC for the docker-machine and runner instances. | string | n/a | yes | + +## Outputs + +| Name | Description | +|------|-------------| +| runner\_agent\_role\_arn | ARN of the role used for the ec2 instance for the GitLab runner agent. | +| runner\_agent\_role\_name | Name of the role used for the ec2 instance for the GitLab runner agent. | +| runner\_as\_group\_name | Name of the autoscaling group for the gitlab-runner instance | +| runner\_cache\_bucket\_arn | ARN of the S3 for the build cache. | +| runner\_cache\_bucket\_name | Name of the S3 for the build cache. | +| runner\_role\_arn | ARN of the role used for the docker machine runners. | +| runner\_role\_name | Name of the role used for the docker machine runners. | diff --git a/_docs/TF_MODULE.md b/_docs/TF_MODULE.md index e69de29bb..df974665e 100644 --- a/_docs/TF_MODULE.md +++ b/_docs/TF_MODULE.md @@ -0,0 +1,85 @@ +## Inputs + +| Name | Description | Type | Default | Required | +|------|-------------|:----:|:-----:|:-----:| +| allow\_iam\_service\_linked\_role\_creation | Boolean used to control attaching the policy to a runner instance to create service linked roles. | bool | `"true"` | no | +| ami\_filter | List of maps used to create the AMI filter for the Gitlab runner agent AMI. Currently Amazon Linux 2 `amzn2-ami-hvm-2.0.????????-x86_64-ebs` looks to *not* be working for this configuration. | map(list(string)) | `` | no | +| ami\_owners | The list of owners used to select the AMI of Gitlab runner agent instances. | list(string) | `` | no | +| aws\_region | AWS region. | string | n/a | yes | +| aws\_zone | AWS availability zone (typically 'a', 'b', or 'c'). | string | `"a"` | no | +| cache\_bucket | Configuration to control the creation of th the cache bucket. By default the bucket will be crated and used as shared cache. To use the same cache cross multiple runners disable the cration of the cache and provice a policy and bucket name. See the public runner example for more details. | map | `` | no | +| cache\_bucket\_prefix | Prefix for s3 cache bucket name. | string | `""` | no | +| cache\_bucket\_versioning | Boolean used to enable versioning on the cache bucket, false by default. | bool | `"false"` | no | +| cache\_expiration\_days | Number of days before cache objects expires. | number | `"1"` | no | +| cache\_shared | Enables cache sharing between runners, false by default. | bool | `"false"` | no | +| create\_runners\_iam\_instance\_profile | Boolean to control the creation of the runners IAM instance profile | bool | `"true"` | no | +| docker\_machine\_instance\_type | Instance type used for the instances hosting docker-machine. | string | `"m5a.large"` | no | +| docker\_machine\_options | List of additional options for the docker machine config. Each element of this list must be a key=value pair. E.g. '["amazonec2-zone=a"]' | list(string) | `` | no | +| docker\_machine\_role\_json | Docker machine runner instance override policy, expected to be in JSON format. | string | `""` | no | +| docker\_machine\_spot\_price\_bid | Spot price bid. | string | `"0.06"` | no | +| docker\_machine\_ssh\_cidr\_blocks | List of CIDR blocks to allow SSH Access to the docker machine runner instance. | list(string) | `` | no | +| docker\_machine\_user | Username of the user used to create the spot instances that host docker-machine. | string | `"docker-machine"` | no | +| docker\_machine\_version | Version of docker-machine. | string | `"0.16.1"` | no | +| enable\_cloudwatch\_logging | Boolean used to enable or disable the CloudWatch logging. | bool | `"true"` | no | +| enable\_gitlab\_runner\_ssh\_access | Enables SSH Access to the gitlab runner instance. | bool | `"false"` | no | +| enable\_manage\_gitlab\_token | Boolean to enable the management of the GitLab token in SSM. If `true` the token will be stored in SSM, which means the SSM property is a terraform managed resource. If `false` the Gitlab token will be stored in the SSM by the user-data script during creation of the the instance. However the SSM parameter is not managed by terraform and will remain in SSM after a `terraform destroy`. | bool | `"true"` | no | +| enable\_runner\_user\_data\_trace\_log | Enable bash xtrace for the user data script that creates the EC2 instance for the runner agent. Be aware this could log sensitive data such as you GitLab runner token. | bool | `"false"` | no | +| environment | A name that identifies the environment, used as prefix and for tagging. | string | n/a | yes | +| gitlab\_runner\_registration\_config | Configuration used to register the runner. See the README for an example, or reference the examples in the examples directory of this repo. | map(string) | `` | no | +| gitlab\_runner\_ssh\_cidr\_blocks | List of CIDR blocks to allow SSH Access to the gitlab runner instance. | list(string) | `` | no | +| gitlab\_runner\_version | Version of the GitLab runner. | string | `"12.1.0"` | no | +| instance\_role\_json | Default runner instance override policy, expected to be in JSON format. | string | `""` | no | +| instance\_type | Instance type used for the GitLab runner. | string | `"t3.micro"` | no | +| overrides | This maps provides the possibility to override some defaults. The following attributes are supported: `name_sg` overwrite the `Name` tag for all security groups created by this module. `name_runner_agent_instance` override the `Name` tag for the ec2 instance defined in the auto launch configuration. `name_docker_machine_runners` ovverrid the `Name` tag spot instances created by the runner agent. | map(string) | `` | no | +| runner\_ami\_filter | List of maps used to create the AMI filter for the Gitlab runner docker-machine AMI. | map(list(string)) | `` | no | +| runner\_ami\_owners | The list of owners used to select the AMI of Gitlab runner docker-machine instances. | list(string) | `` | no | +| runner\_instance\_spot\_price | By setting a spot price bid price the runner agent will be created via a spot request. Be aware that spot instances can be stopped by AWS. | string | `""` | no | +| runners\_additional\_volumes | Additional volumes that will be used in the runner config.toml, e.g Docker socket | list | `` | no | +| runners\_concurrent | Concurrent value for the runners, will be used in the runner config.toml. | number | `"10"` | no | +| runners\_environment\_vars | Environment variables during build execution, e.g. KEY=Value, see runner-public example. Will be used in the runner config.toml | list(string) | `` | no | +| runners\_executor | The executor to use. Currently supports `docker+machine` or `docker`. | string | `"docker+machine"` | no | +| runners\_gitlab\_url | URL of the GitLab instance to connect to. | string | n/a | yes | +| runners\_iam\_instance\_profile\_name | IAM instance profile name of the runners, will be used in the runner config.toml | string | `""` | no | +| runners\_idle\_count | Idle count of the runners, will be used in the runner config.toml. | number | `"0"` | no | +| runners\_idle\_time | Idle time of the runners, will be used in the runner config.toml. | number | `"600"` | no | +| runners\_image | Image to run builds, will be used in the runner config.toml | string | `"docker:18.03.1-ce"` | no | +| runners\_limit | Limit for the runners, will be used in the runner config.toml. | number | `"0"` | no | +| runners\_monitoring | Enable detailed cloudwatch monitoring for spot instances. | bool | `"false"` | no | +| runners\_name | Name of the runner, will be used in the runner config.toml. | string | n/a | yes | +| runners\_off\_peak\_idle\_count | Off peak idle count of the runners, will be used in the runner config.toml. | number | `"0"` | no | +| runners\_off\_peak\_idle\_time | Off peak idle time of the runners, will be used in the runner config.toml. | number | `"0"` | no | +| runners\_off\_peak\_periods | Off peak periods of the runners, will be used in the runner config.toml. | string | `""` | no | +| runners\_off\_peak\_timezone | Off peak idle time zone of the runners, will be used in the runner config.toml. | string | `""` | no | +| runners\_output\_limit | Sets the maximum build log size in kilobytes, by default set to 4096 (4MB) | number | `"4096"` | no | +| runners\_post\_build\_script | Commands to be executed on the Runner just after executing the build, but before executing after_script. | string | `""` | no | +| runners\_pre\_build\_script | Script to execute in the pipeline just before the build, will be used in the runner config.toml | string | `""` | no | +| runners\_pre\_clone\_script | Commands to be executed on the Runner before cloning the Git repository. this can be used to adjust the Git client configuration first, for example. | string | `""` | no | +| runners\_privileged | Runners will run in privileged mode, will be used in the runner config.toml | bool | `"true"` | no | +| runners\_pull\_policy | pull_policy for the runners, will be used in the runner config.toml | string | `"always"` | no | +| runners\_request\_concurrency | Limit number of concurrent requests for new jobs from GitLab (default 1) | number | `"1"` | no | +| runners\_root\_size | Runner instance root size in GB. | number | `"16"` | no | +| runners\_shm\_size | shm_size for the runners, will be used in the runner config.toml | number | `"0"` | no | +| runners\_token | Token for the runner, will be used in the runner config.toml. | string | `"__REPLACED_BY_USER_DATA__"` | no | +| runners\_use\_private\_address | Restrict runners to the use of a private IP address | bool | `"true"` | no | +| secure\_parameter\_store\_runner\_token\_key | The key name used store the Gitlab runner token in Secure Parameter Store | string | `"runner-token"` | no | +| ssh\_key\_pair | Set this to use existing AWS key pair | string | `""` | no | +| ssh\_public\_key | Public SSH key used for the GitLab runner EC2 instance. | string | `""` | no | +| subnet\_id\_runners | List of subnets used for hosting the gitlab-runners. | string | n/a | yes | +| subnet\_ids\_gitlab\_runner | Subnet used for hosting the GitLab runner. | list(string) | n/a | yes | +| tags | Map of tags that will be added to created resources. By default resources will be tagged with name and environment. | map(string) | `` | no | +| userdata\_post\_install | User-data script snippet to insert after GitLab runner install | string | `""` | no | +| userdata\_pre\_install | User-data script snippet to insert before GitLab runner install | string | `""` | no | +| vpc\_id | The target VPC for the docker-machine and runner instances. | string | n/a | yes | + +## Outputs + +| Name | Description | +|------|-------------| +| runner\_agent\_role\_arn | ARN of the role used for the ec2 instance for the GitLab runner agent. | +| runner\_agent\_role\_name | Name of the role used for the ec2 instance for the GitLab runner agent. | +| runner\_as\_group\_name | Name of the autoscaling group for the gitlab-runner instance | +| runner\_cache\_bucket\_arn | ARN of the S3 for the build cache. | +| runner\_cache\_bucket\_name | Name of the S3 for the build cache. | +| runner\_role\_arn | ARN of the role used for the docker machine runners. | +| runner\_role\_name | Name of the role used for the docker machine runners. | + diff --git a/variables.tf b/variables.tf index 7512a6f65..2834485f8 100644 --- a/variables.tf +++ b/variables.tf @@ -135,14 +135,9 @@ variable "runners_additional_volumes" { default = [] } -variable "runners_volumes" { - description = "Specify additional volumes that should be mounted (same syntax as Docker’s -v flag)" - type = "list" - default = ["/cache"] -} - variable "runners_shm_size" { description = "shm_size for the runners, will be used in the runner config.toml" + type = number default = 0 }