AWS Msk Kafka Cluster

Terraform module which creates Msk Kafka Cluster on AWS.

These types of resources are supported:

• Aws Msk Cluster
• Aws Msk Configuration

Features

This module create a fully featured Msk Kafka Cluster on Aws. You could configure monitoring, encryption, server options, etc...

Usage

module "kafka" {
  source = "angelabad/msk-cluster"

  cluster_name    = "kafka"
  instance_type   = "kafka.t3.small"
  number_of_nodes = 2
  client_subnets  = ["subnet-0ab97cbe1bd1406c2", "subnet-0d6cbf60360dbac64"]
  kafka_version   = "2.4.1"

  extra_security_groups = ["sg-019fc0f7d26f6c70f"]

  enhanced_monitoring = "PER_BROKER"

  s3_logs_bucket = aws_s3_bucket.logs.id
  s3_logs_prefix = "msklogs"

  prometheus_jmx_exporter  = true
  prometheus_node_exporter = true

  server_properties = {
    "auto.create.topics.enable"  = "true"
    "default.replication.factor" = "2"
  }

  encryption_in_transit_client_broker = "TLS"

  tags = {
    Owner       = "user"
    Environment = "dev"
  }
}

Providers

Name	Version
aws	>= 2.55
random	>= 2.1

Inputs

Name	Description	Type	Default	Required
client_subnets	A list of subnets to connect to in client VPC	list(string)	n/a	yes
cloudwatch_logs_group	Name of the Cloudwatch Log Group to deliver logs to.	string	""	no
cluster_name	Name of the MSK cluster.	string	n/a	yes
encryption_at_rest_kms_key_arn	You may specify a KMS key short ID or ARN (it will always output an ARN) to use for encrypting your data at rest. If no key is specified, an AWS managed KMS ('aws/msk' managed service) key will be used for encrypting the data at rest.	string	""	no
encryption_in_transit_client_broker	Encryption setting for data in transit between clients and brokers. Valid values: TLS, TLS_PLAINTEXT, and PLAINTEXT. Default value is TLS_PLAINTEXT.	string	"TLS_PLAINTEXT"	no
encryption_in_transit_in_cluster	Whether data communication among broker nodes is encrypted. Default value: true.	bool	true	no
enhanced_monitoring	Specify the desired enhanced MSK CloudWatch monitoring level to one of three monitoring levels: DEFAULT, PER_BROKER, or PER_TOPIC_PER_BROKER. See Monitoring Amazon MSK with Amazon CloudWatch.	string	"DEFAULT"	no
extra_security_groups	A list of extra security groups to associate with the elastic network interfaces to control who can communicate with the cluster.	list(string)	[]	no
firehose_logs_delivery_stream	Name of the Kinesis Data Firehose delivery stream to deliver logs to.	string	""	no
instance_type	Specify the instance type to use for the kafka brokers. e.g. kafka.m5.large.	string	n/a	yes
kafka_version	Specify the desired Kafka software version.	string	n/a	yes
number_of_nodes	The desired total number of broker nodes in the kafka cluster. It must be a multiple of the number of specified client subnets.	number	n/a	yes
prometheus_jmx_exporter	Indicates whether you want to enable or disable the JMX Exporter.	bool	false	no
prometheus_node_exporter	Indicates whether you want to enable or disable the Node Exporter.	bool	false	no
s3_logs_bucket	Name of the S3 bucket to deliver logs to.	string	""	no
s3_logs_prefix	Prefix to append to the folder name.	string	""	no
server_properties	A map of the contents of the server.properties file. Supported properties are documented in the MSK Developer Guide.	map(string)	{}	no
tags	A mapping of tags to assign to the resource.	map(string)	{}	no
volume_size	The size in GiB of the EBS volume for the data drive on each broker node.	number	1000	no

Outputs

Name	Description
arn	Amazon Resource Name (ARN) of the MSK cluster.
bootstrap_brokers	A comma separated list of one or more hostname:port pairs of kafka brokers suitable to boostrap connectivity to the kafka cluster. Only contains value if client_broker encryption in transit is set o PLAINTEXT or TLS_PLAINTEXT.
bootstrap_brokers_tls	A comma separated list of one or more DNS names (or IPs) and TLS port pairs kafka brokers suitable to boostrap connectivity to the kafka cluster. Only contains value if client_broker encryption in transit is set to TLS_PLAINTEXT or TLS.
current_version	Current version of the MSK Cluster used for updates, e.g. K13V1IB3VIYZZH
default_security_group	Msk cluster default security group id.
encryption_at_rest_kms_key_arn	The ARN of the KMS key used for encryption at rest of the broker data volumes.
zookeeper_connect_string	A comma separated list of one or more hostname:port pairs to use to connect to the Apache Zookeeper cluster.

NOTE: The API does not support deleting MSK configurations.

Authors

Module managed by Angel Abad

License

Apache 2 Licensed. See LICENSE for full details