Terraform module to create Amazon MWAA (Managed Workflows for Apache Airflow). Amazon Managed Workflows for Apache Airflow (MWAA) orchestrates your workflows using Directed Acyclic Graphs (DAGs) written in Python.
The MIT License (MIT)
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
Source: <https://opensource.org/licenses/MIT>
See LICENSE for full details.
| Name | Version |
|---|---|
| terraform | >= 0.14 |
| aws | >= 2.0.0 |
No modules.
| Name | Type |
|---|---|
| aws_iam_role.this | resource |
| aws_iam_role_policy.this | resource |
| aws_mwaa_environment.this | resource |
| aws_security_group.this | resource |
| aws_caller_identity.current | data source |
| aws_iam_policy_document.assume | data source |
| aws_iam_policy_document.base | data source |
| aws_iam_policy_document.this | data source |
| aws_partition.current | data source |
| aws_region.current | data source |
| Name | Description | Type | Default | Required |
|---|---|---|---|---|
| additional_associated_security_group_ids | Security group IDs of existing security groups that should be associated with the MWAA environment. Available only when create_security_group is set to true. |
list(string) |
[] |
no |
| additional_execution_role_policy_document_json | Additional permissions to attach to the base mwaa execution role. | string |
"{}" |
no |
| airflow_configuration_options | additional configuration to overwrite airflows standard config. | map(string) |
{} |
no |
| airflow_version | Airflow version of the MWAA environment, will be set by default to the latest version that MWAA supports. | string |
"" |
no |
| associated_security_group_ids | A list of IDs of Security Groups to associate the created resource with, in addition to the created security group. These security groups will not be modified and, if create_security_group is false, must have rules providing the desired access. |
list(string) |
[] |
no |
| create_security_group | Enabling or disabling the creation of a default Security Group for AWS MWAA | bool |
true |
no |
| dag_processing_logs_enabled | Enabling or disabling the collection of logs | bool |
true |
no |
| dag_processing_logs_level | Available log level: DEBUG, INFO, WARNING, ERROR, CRITICAL. |
string |
"WARNING" |
no |
| dag_s3_path | Relative path of the dags folder within the source bucket. | string |
"dags" |
no |
| environment_class | Environment class for the cluster. Possible options are mw1.small, mw1.medium, mw1.large. Will be set by default to mw1.small.Please check the AWS Pricing for more information about the environment classes. |
string |
"mw1.small" |
no |
| environment_name | Name of the MWAA environment. | string |
n/a | yes |
| kms_key_arn | KMS CMK ARN to use by MWAA for data encryption. MUST reference the same KMS key as used by S3 bucket specified by source_bucket_arn, if the bucket uses KMS. If not specified, the default AWS owned key for MWAA will be used for backward compatibility with version 1.0.1 of this module. |
string |
null |
no |
| max_workers | numeric string, min 1. | string |
"10" |
no |
| min_workers | numeric string, min 1. | string |
"1" |
no |
| plugins_s3_object_version | The plugins.zip file version you want to use. | string |
null |
no |
| plugins_s3_path | relative path of the plugins.zip within the source bucket. | string |
null |
no |
| requirements_s3_object_version | The requirements.txt file version you want to use. | string |
null |
no |
| requirements_s3_path | relative path of the requirements.txt (incl. filename) within the source bucket. | string |
null |
no |
| scheduler_logs_enabled | Enabling or disabling the collection of logs | bool |
true |
no |
| scheduler_logs_level | Available log level: DEBUG, INFO, WARNING, ERROR, CRITICAL. |
string |
"WARNING" |
no |
| security_group_description | The Security Group description. Using only when create_security_group is set to true. |
string |
"Security Group for AWS MWAA" |
no |
| source_bucket_arn | ARN of the bucket in which DAGs, Plugin and Requirements are put. | string |
n/a | yes |
| subnet_ids | Subnet Ids of the existing subnets that MWAA should be used. Must be at least 2 and subnets must be private. | list(string) |
n/a | yes |
| tags | Map of tags to assign to bucket. | map(string) |
{} |
no |
| task_logs_enabled | Enabling or disabling the collection of logs | bool |
true |
no |
| task_logs_level | Available log level: DEBUG, INFO, WARNING, ERROR, CRITICAL. |
string |
"INFO" |
no |
| vpc_id | VPC id of the VPC in which the environments resources are created. | string |
n/a | yes |
| webserver_access_mode | Specifies whether the webserver should be accessible over the internet or via your specified VPC. Possible options: PRIVATE_ONLY (default) and PUBLIC_ONLY. |
string |
"PRIVATE_ONLY" |
no |
| webserver_logs_enabled | Enabling or disabling the collection of logs | bool |
true |
no |
| webserver_logs_level | Available log level: DEBUG, INFO, WARNING, ERROR, CRITICAL. |
string |
"WARNING" |
no |
| weekly_maintenance_window_start | The day and time of the week in Coordinated Universal Time (UTC) 24-hour standard time to start weekly maintenance updates of your environment in the following format: DAY:HH:MM. For example: TUE:03:30. You can specify a start time in 30 minute increments only. |
string |
"MON:01:00" |
no |
| worker_logs_enabled | Enabling or disabling the collection of logs | bool |
true |
no |
| worker_logs_level | Available log level: DEBUG, INFO, WARNING, ERROR, CRITICAL. | string |
"WARNING" |
no |
| Name | Description |
|---|---|
| arn | The arn of the created MWAA environment. |
| webserver_url | The webserver URL of the MWAA Environment. |
module "app_prod_airflow_label" {
source = "cloudposse/label/null"
version = "v0.25.0"
name = var.name
namespace = "app"
stage = "prod"
delimiter = "-"
tags = {
"BusinessUnit" = "XYZ",
}
}
module "app_prod_airflow_bucket" {
source = "Infrastrukturait/s3-bucket/aws"
version = "0.4.0"
bucket_name = join(module.app_prod_airflow_label.delimiter, [module.app_prod_airflow_label.stage, module.app_prod_airflow_label.name])
bucket_acl = var.bucket_acl
versioning = true
tags = module.app_prod_airflow_label.tags
}
module "app_prod_airflow" {
source = "../../"
environment_name = join(module.app_prod_airflow_label.delimiter, [module.app_prod_airflow_label.stage, module.app_prod_airflow_label.name])
subnet_ids = var.subnet_ids
vpc_id = var.vpc_id
webserver_access_mode = var.webserver_access_mode
source_bucket_arn = module.app_prod_airflow_bucket.arn
tags = module.app_prod_airflow_label.tags
}