Skip to content

Latest commit

 

History

History
211 lines (188 loc) · 11.2 KB

File metadata and controls

211 lines (188 loc) · 11.2 KB

terraform-module-cloud-aws-kubernetes-cluster

This terraform module is to help you quickly deploy a EKS cluster on Amazon Web Services (AWS). This is part of the opionated GlueOps Platform. If you came here directly then you should probably visit https://github.com/glueops/admiral as that is the start point.

Prerequisites to use this Terraform module

  • A Dedicated AWS Sub account
  • Service account with environment variable set
  • Service Quotas (Depending on Cluster Size)

For more details see: https://github.com/GlueOps/terraform-module-cloud-aws-kubernetes-cluster/wiki/

Example usage of module

module "captain" {
  iam_role_to_assume = "arn:aws:iam::1234567890:role/glueops-captain-role"
  source             = "git::https://github.com/GlueOps/terraform-module-cloud-aws-kubernetes-cluster.git"
  eks_version        = "1.30"
  csi_driver_version = "v1.37.0-eksbuild.1"
  coredns_version    = "v1.11.3-eksbuild.2"
  kube_proxy_version = "v1.30.6-eksbuild.3"
  vpc_cidr_block     = "10.65.0.0/26"
  region             = "us-west-2"
  availability_zones = ["us-west-2a", "us-west-2b"]
  node_pools = [
#    {
#      "kubernetes_version" : "1.30",
#      "ami_release_version" : "1.30.6-20241121",
#      "ami_type" : "AL2_x86_64",
#      "instance_type" : "t3a.large",
#      "name" : "glueops-platform-node-pool-1",
#      "node_count" : 4,
#      "spot" : false,
#      "disk_size_gb" : 20,
#      "max_pods" : 110,
#      "ssh_key_pair_names" : [],
#      "kubernetes_labels" : {
#        "glueops.dev/role" : "glueops-platform"
#      },
#      "kubernetes_taints" : [
#        {
#          key    = "glueops.dev/role"
#          value  = "glueops-platform"
#          effect = "NO_SCHEDULE"
#        }
#      ]
#    },
#    {
#      "kubernetes_version" : "1.30",
#      "ami_release_version" : "1.30.6-20241121",
#      "ami_type" : "AL2_x86_64",
#      "instance_type" : "t3a.small",
#      "name" : "glueops-platform-node-pool-argocd-app-controller-1",
#      "node_count" : 2,
#      "spot" : false,
#      "disk_size_gb" : 20,
#      "max_pods" : 110,
#      "ssh_key_pair_names" : [],
#      "kubernetes_labels" : {
#        "glueops.dev/role" : "glueops-platform-argocd-app-controller"
#      },
#      "kubernetes_taints" : [
#        {
#          key    = "glueops.dev/role"
#          value  = "glueops-platform-argocd-app-controller"
#          effect = "NO_SCHEDULE"
#        }
#      ]
#    },
#    {
#      "kubernetes_version" : "1.30",
#      "ami_release_version" : "1.30.6-20241121",
#      "ami_type" : "AL2_x86_64",
#      "instance_type" : "t3a.medium",
#      "name" : "clusterwide-node-pool-1",
#      "node_count" : 2,
#      "spot" : false,
#      "disk_size_gb" : 20,
#      "max_pods" : 110,
#      "ssh_key_pair_names" : [],
#      "kubernetes_labels" : {},
#      "kubernetes_taints" : []
#    }
  ]
  peering_configs = [
#    {
#    vpc_peering_connection_id = "pcx-0df92b5241651ba92"
#    destination_cidr_block = "10.69.0.0/26"
#    }
  ]
}

VPC Peering

This terraform module expects only to be an accepter VPC. This means a VPC peering request must come from the requesting account. As an accepter VPC you must provide the requester your VPC ID, your AWS Account ID (The subaccount being used for the cluster deployment), and the VPC CIDR you configured for the cluster deployment.

When providing them with the above, please ask them to enable DNS resolution of hosts within the requester VPC.

EFS/NFS Example Manifest

apiVersion: v1
kind: PersistentVolume
metadata:
  name: nfs-pv-test
spec:
  storageClassName: efs-fun-test
  capacity:
    storage: 1000Gi # Adjust based on your needs
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  mountOptions:
      - timeo=600
      - retrans=2
      - nfsvers=4.1
      - rsize=1048576
      - wsize=1048576
      - noresvport
      - hard
  nfs:
    path: /
    server: nfs.nonprod.antoniostacos.onglueops.com
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: my-pvc
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: efs-fun-test
  resources:
    requests:
      storage: 1Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: my-pod
spec:
  containers:
    - name: my-container
      image: nginx
      volumeMounts:
        - name: my-volume
          mountPath: /mnt/data  # Mount path within the container
          subPath: pod1-fun
  volumes:
    - name: my-volume
      persistentVolumeClaim:
        claimName: my-pvc  # Name of the PVC to be mounted

Requirements

No requirements.

Providers

Name Version
aws n/a

Modules

Name Source Version
kubernetes cloudposse/eks-cluster/aws 3.0.0
node_pool cloudposse/eks-node-group/aws 3.1.1
subnets cloudposse/dynamic-subnets/aws 2.4.2
vpc cloudposse/vpc/aws 2.2.0
vpc_peering_accepter_with_routes ./modules/vpc_peering_accepter_with_routes n/a

Resources

Name Type
aws_eks_addon.coredns resource
aws_eks_addon.ebs_csi resource
aws_eks_addon.kube_proxy resource
aws_iam_role.eks_addon_ebs_csi_role resource
aws_iam_role_policy_attachment.ebs_csi resource
aws_security_group.captain resource
aws_security_group_rule.allow_all_within_group resource
aws_security_group_rule.captain_egress_all_ipv4 resource
aws_iam_openid_connect_provider.provider data source
aws_iam_policy_document.eks_assume_addon_role data source

Inputs

Name Description Type Default Required
availability_zones The availability zones to deploy into list(string)
[
"us-west-2a",
"us-west-2b",
"us-west-2c"
]
no
coredns_version You should grab the appropriate version number from: https://docs.aws.amazon.com/eks/latest/userguide/managing-coredns.html string "v1.11.3-eksbuild.2" no
csi_driver_version You should grab the appropriate version number from: https://github.com/kubernetes-sigs/aws-ebs-csi-driver/blob/master/CHANGELOG.md string "v1.37.0-eksbuild.1" no
eks_version The version of EKS to deploy string "1.30" no
iam_role_to_assume The full ARN of the IAM role to assume string n/a yes
kube_proxy_version You should grab the appropriate version number from: https://docs.aws.amazon.com/eks/latest/userguide/managing-kube-proxy.html string "v1.30.6-eksbuild.3" no
node_pools node pool configurations:
- name (string): Name of the node pool. MUST BE UNIQUE! Recommended to use YYYYMMDD in the name
- node_count (number): number of nodes to create in the node pool.
- instance_type (string): Instance type to use for the nodes. ref: https://instances.vantage.sh/
- kubernetes_version (string): Generally this is the same version as the EKS cluster. But if doing a node pool upgrade this may be a different version.
- ami_release_version (string): AMI Release version to use for EKS worker nodes. ref: https://github.com/awslabs/amazon-eks-ami/releases
- ami_type (string): e.g. AMD64 or ARM
- spot (bool): Enable spot instances for the nodes. DO NOT ENABLE IN PROD!
- disk_size_gb (number): Disk size in GB for the nodes.
- max_pods (number): max pods that can be scheduled per node.
- ssh_key_pair_names (list(string)): List of SSH key pair names to associate with the nodes. ref: https://us-west-2.console.aws.amazon.com/ec2/home?region=us-west-2#KeyPairs:
- kubernetes_labels (map(string)): Map of labels to apply to the nodes. ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
- kubernetes_taints (list(object)): List of taints to apply to the nodes. ref: https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/
list(object({
name = string
node_count = number
instance_type = string
kubernetes_version = string
ami_release_version = string
ami_type = string
spot = bool
disk_size_gb = number
max_pods = number
ssh_key_pair_names = list(string)
kubernetes_labels = map(string)
kubernetes_taints = list(object({
key = string
value = string
effect = string
}))

}))
[
{
"ami_release_version": "1.30.6-20241121",
"ami_type": "AL2_x86_64",
"disk_size_gb": 20,
"instance_type": "t3a.large",
"kubernetes_labels": {},
"kubernetes_taints": [],
"kubernetes_version": "1.30",
"max_pods": 110,
"name": "default-pool",
"node_count": 1,
"spot": false,
"ssh_key_pair_names": []
}
]
no
peering_configs A list of maps containing VPC peering configuration details
list(object({
vpc_peering_connection_id = string
destination_cidr_block = string
}))
[] no
region The AWS region to deploy into string n/a yes
vpc_cidr_block The CIDR block for the VPC string "10.65.0.0/26" no

Outputs

No outputs.