Skip to content

Commit

Permalink
Tidy IRSA docs (#7273)
Browse files Browse the repository at this point in the history
  • Loading branch information
@chrisdoherty4
chrisdoherty4 authored Jan 9, 2024
1 parent ad2e139 commit ed36457
Showing 1 changed file with 107 additions and 103 deletions.
210 changes: 107 additions & 103 deletions docs/content/en/docs/getting-started/optional/irsa.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,15 @@ description: >

### IAM Role for Service Account on EKS Anywhere clusters with self-hosted signing keys

IAM Roles for Service Account (IRSA) enables applications running in clusters to authenticate with AWS services using IAM roles. The current solution for leveraging this in EKS Anywhere involves creating your own OIDC provider for the cluster, and hosting your cluster's public service account signing key. The public keys along with the OIDC discovery document should be hosted somewhere that AWS STS can discover it. The steps below assume the keys will be hosted on a publicly accessible S3 bucket. Refer [this](https://docs.aws.amazon.com/AmazonS3/latest/userguide/configuring-block-public-access-bucket.html) doc to ensure that the s3 bucket is publicly accessible.
IAM Roles for Service Account (IRSA) enables applications running in clusters to authenticate with AWS services using IAM roles. The current solution for leveraging this in EKS Anywhere involves creating your own OIDC provider for the cluster, and hosting your cluster's public service account signing key. The public keys along with the OIDC discovery document should be hosted somewhere that AWS STS can discover it.

The steps below are based on the [guide for configuring IRSA for DIY Kubernetes,](https://github.com/aws/amazon-eks-pod-identity-webhook/blob/master/SELF_HOSTED_SETUP.md) with modifications specific to EKS Anywhere's cluster provisioning workflow. The main modification is the process of generating the keys.json document. As per the original guide, the user has to create the service account signing keys, and then use that to create the keys.json document prior to cluster creation. This order is reversed for EKS Anywhere clusters, so you will create the cluster first, and then retrieve the service account signing key generated by the cluster, and use it to create the keys.json document. The sections below show how to do this in detail.

### Create an OIDC provider and make its discovery document publicly accessible

You must use a single OIDC provider per EKS Anywhere cluster, which is the best practice to prevent a token from one cluster being usable with the API server of another cluster. These steps describe the process of using a S3 bucket to host the OIDC `discovery.json` and `keys.json` documents. You do not have to use S3, as the primary requirement is that AWS STS can access the OIDC documents. If you do use S3, you can have a single bucket or multiple buckets to host the OIDC documents. If you use a single bucket, you can host the OIDC documents in separate paths within the bucket, which can then be used for the `ISSUER_HOSTPATH` in the discovery document for each OIDC provider.
You must use a single OIDC provider per EKS Anywhere cluster, which is the best practice to prevent a token from one cluster being used with another cluster. These steps describe the process of using a [public S3 bucket](https://docs.aws.amazon.com/AmazonS3/latest/userguide/configuring-block-public-access-bucket.html) to host the OIDC `discovery.json` and `keys.json` documents.

1. Create an S3 bucket to host the public signing keys and OIDC discovery document for your cluster as per [this section.](https://github.com/aws/amazon-eks-pod-identity-webhook/blob/master/SELF_HOSTED_SETUP.md#create-an-s3-bucket) Ensure you follow all the steps and save the `$HOSTNAME` and `$ISSUER_HOSTPATH`.
1. [Create an S3 bucket to host the public signing keys and OIDC discovery document for your cluster](https://github.com/aws/amazon-eks-pod-identity-webhook/blob/master/SELF_HOSTED_SETUP.md#create-an-s3-bucket). Make a note of the `$HOSTNAME` and `$ISSUER_HOSTPATH`.

1. Create the OIDC discovery document as follows:

Expand All @@ -45,140 +45,144 @@ You must use a single OIDC provider per EKS Anywhere cluster, which is the best
EOF
```
1. Upload it to the publicly accessible S3 bucket:
1. Upload the `discovery.json` file to the S3 bucket:
```bash
aws s3 cp --acl public-read ./discovery.json s3://$S3_BUCKET/.well-known/openid-configuration
```
1. Create an [OIDC provider](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html) for your cluster. Set the `Provider URL` to `https://$ISSUER_HOSTPATH`, and audience to `sts.amazonaws.com`.
1. Note down the `Provider` field of OIDC provider after it is created.
1. Assign an IAM role to this OIDC provider.
1. From the IAM console, select and click on the OIDC provider created from above, and click on **Assign role** at the top right.
1. Select **Create a new role**.
1. In the **Select type of trusted entity** section, choose Web identity.
1. In the Choose a web identity provider section:
* For Identity provider, choose the auto selected Identity Provider URL for your cluster.
* For Audience, choose sts.amazonaws.com.
1. Choose Next: Permissions.
1. In the **Attach Policy** section, select the IAM policy that has the permissions that you want your applications running in the pods to use.
1. Continue with the next sections of adding tags if desired and a suitable name for this role and create the role.
1. Below is a sample trust policy of IAM role for your pods. Remember to replace `Account ID` and `ISSUER_HOSTPATH` with required values.
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::111122223333:oidc-provider/ISSUER_HOSTPATH"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"__doc_comment": "scope the role to the service account (optional)",
"StringEquals": {
"ISSUER_HOSTPATH:sub": "system:serviceaccount:default:my-serviceaccount"
},
"__doc_comment": "OR scope the role to a namespace (optional)",
"StringLike": {
"ISSUER_HOSTPATH/CLUSTER_ID:sub": ["system:serviceaccount:default:*","system:serviceaccount:observability:*"]
}
}
}
]
}
1. Create an [OIDC provider](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_roles_providers_create_oidc.html) for your cluster. Set the _Provider URL_ to `https://$ISSUER_HOSTPATH` and _Audience_ to `sts.amazonaws.com`.
1. Make a note of the `Provider` field of OIDC provider after it is created.
1. Assign an IAM role to the OIDC provider.
1. Navigate to the AWS IAM Console.
1. Click on the OIDC provider.
1. Click _Assign role_.
1. Select _Create a new role_.
1. Select _Web identity_ as the trusted entity.
1. In the _Web identity_ section:
* If your _Identity provider_ is not auto selected, select it.
* Select `sts.amazonaws.com` as the _Audience_.
1. Click _Next_.
1. Configure your desired _Permissions poilicies_.
1. Below is a sample trust policy of IAM role for your pods. Replace `ACCOUNT_ID`, `ISSUER_HOSTPATH`, `NAMESPACE` and `SERVICE_ACCOUNT`.
_Example: Scoped to a service account_
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::ACCOUNT_ID:oidc-provider/ISSUER_HOSTPATH"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringEquals": {
"ISSUER_HOSTPATH:sub": "system:serviceaccount:NAMESPACE:SERVICE_ACCOUNT"
},
}
}
]
}
```
1. After the role is created, note down the name of this IAM Role as `OIDC_IAM_ROLE`.
1. Once the cluster is created, you can create service accounts and grant them this role by editing the trust relationship of this role. You can use `StringLike` condition to add required service accounts, as mentioned in the above sample. Please also refer to section [Configure the trust relationship for the OIDC provider's IAM Role](#configure-the-trust-relationship-for-the-oidc-providers-iam-role).
1. Create the IAM Role and make a note of the _Role name_.
1. After the cluster is created you can grant service accounts access to the role by modifying the trust relationship. See the [How to use trust policies with IAM Roles](https://aws.amazon.com/blogs/security/how-to-use-trust-policies-with-iam-roles/) for more information on trust policies. Refer to [Configure the trust relationship for the OIDC provider's IAM Role](#configure-the-trust-relationship-for-the-oidc-providers-iam-role) for a working example.
### Create the EKS Anywhere cluster
1. When creating the EKS Anywhere cluster, you need to configure the kube-apiserver's `service-account-issuer` flag so it can issue and mount projected service account tokens in pods. For this, use the value obtained in the first section for `$ISSUER_HOSTPATH` as the `service-account-issuer`. Configure the kube-apiserver by setting this value through the EKS Anywhere cluster spec as follows:
```yaml
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
name: my-cluster-name
spec:
podIamConfig:
When creating the EKS Anywhere cluster, you need to configure the kube-apiserver's `service-account-issuer` flag so it can issue and mount projected service account tokens in pods. For this, use the value obtained in the first section for `$ISSUER_HOSTPATH` as the `service-account-issuer`. Configure the kube-apiserver by setting this value through the EKS Anywhere cluster spec:
```yaml
apiVersion: anywhere.eks.amazonaws.com/v1alpha1
kind: Cluster
metadata:
name: my-cluster-name
spec:
podIamConfig:
serviceAccountIssuer: https://$ISSUER_HOSTPATH
```
```
Set the remaining fields in cluster spec as required and create the cluster using the `eksctl anywhere create cluster` command.
Set the remaining fields in cluster spec as required and create the cluster.
#### Generate keys.json and make it publicly accessible
1. The cluster provisioning workflow generates a pair of service account signing keys. Retrieve the public signing key generated and used by the cluster, and create a keys.json document containing the public signing key.
1. The cluster provisioning workflow generates a pair of service account signing keys. Retrieve the public signing key from the cluster and create a `keys.json` document with the content.
```bash
git clone https://github.com/aws/amazon-eks-pod-identity-webhook
cd amazon-eks-pod-identity-webhook
kubectl get secret ${CLUSTER_NAME}-sa -n eksa-system -o jsonpath={.data.tls\\.crt} | base64 --decode > ${CLUSTER_NAME}-sa.pub
kubectl get secret ${CLUSTER_NAME}-sa -n eksa-system -o jsonpath={.data.tls\\.crt} | base64 --decode > ${CLUSTER_NAME}-sa.pub
go run ./hack/self-hosted/main.go -key ${CLUSTER_NAME}-sa.pub | jq '.keys += [.keys[0]] | .keys[1].kid = ""' > keys.json
```
1. Upload the keys.json document to the s3 bucket.
1. Upload the `keys.json` document to the S3 bucket.
```bash
aws s3 cp --acl public-read ./keys.json s3://$S3_BUCKET/keys.json
```
### Deploy pod identity webhook
After hosting the service account public signing key and OIDC discovery documents, the applications running in pods can start accessing the desired AWS resources, as long as the pod is mounted with the right service account tokens. This part of configuring the pods with the right service account tokens and env vars is automated by the [Amazon Pod Identity Webhook](https://github.com/aws/amazon-eks-pod-identity-webhook). Once the webhook is deployed, it mutates any pods launched using service accounts annotated with `eks.amazonaws.com/role-arn`. Follow the steps below to deploy the Amazon EKS Pod Identity Webhook to your cluster:
The [Amazon Pod Identity Webhook](https://github.com/aws/amazon-eks-pod-identity-webhook) configures pods with the necessary environment variables and tokens (via file mounts) to interact with AWS services. The webhook will configure any pod associated with a service account that has an `eks-amazonaws.com/role-arn` annotation.
1. Clone [amazon-eks-pod-identity-webhook](https://github.com/aws/amazon-eks-pod-identity-webhook) if not done already.
1. Clone [amazon-eks-pod-identity-webhook](https://github.com/aws/amazon-eks-pod-identity-webhook).
1. Set the $KUBECONFIG env var to the path of the EKS Anywhere cluster.
1. Set the $KUBECONFIG environment variable to the path of the EKS Anywhere cluster.
1. Create `my-service-account.yaml` with `OIDC_IAM_ROLE` and other annotations as mentioned in sample below.
1. Apply the manifests for the `amazon-eks-pod-identity-webhook`. The image used here will be pulled from docker.io. Optionally, the image can be imported into (or proxied through) your private registry. Change the `IMAGE` argument here to your private registry if needed.
```bash
make cluster-up IMAGE=amazon/amazon-eks-pod-identity-webhook:latest
```
1. Create a service account with an `eks.amazonaws.com/role-arn` annotation set to the IAM Role created for the OIDC provider.
```yaml
apiVersion: v1
kind: ServiceAccount
metadata:
name: my-serviceaccount
namespace: default
annotations:
# set this with value of OIDC_IAM_ROLE
eks.amazonaws.com/role-arn: "arn:aws:iam::111122223333:role/s3-reader"
# set this with value of OIDC_IAM_ROLE
eks.amazonaws.com/role-arn: "arn:aws:iam::ACCOUNT_ID:role/s3-reader"
# optional: Defaults to "sts.amazonaws.com" if not set
eks.amazonaws.com/audience: "sts.amazonaws.com"
# optional: When set to "true", adds AWS_STS_REGIONAL_ENDPOINTS env var
# to containers
eks.amazonaws.com/sts-regional-endpoints: "true"
# optional: Defaults to 86400 for expirationSeconds if not set
# Note: This value can be overwritten if specified in the pod
# Note: This value can be overwritten if specified in the pod
# annotation as shown in the next step.
eks.amazonaws.com/token-expiration: "86400"
```
1. Run the following command to apply the manifests for the `amazon-eks-pod-identity-webhook`. The image used here will be pulled from docker.io. Optionally, the image can be imported into (or proxied through) your private registry. Change the `IMAGE` argument here to your private registry if needed.
```bash
make cluster-up IMAGE=amazon/amazon-eks-pod-identity-webhook:latest
```
1. Finally, apply the `my-service-account.yaml` file to create your service account.
```bash
kubectl apply -f my-service-account.yaml
```
1. You can validate IRSA by using test steps mentioned [here]({{< relref "../../packages/adot/adot_amp_amg/#irsa-set-up-test" >}}). Ensure awscli pod is deployed in same namespace of ServiceAccount `pod-identity-webhook`.
1. You can validate IRSA by following [IRSA setup and test]({{< relref "../../packages/adot/adot_amp_amg/#irsa-set-up-test" >}}). Ensure the awscli pod is deployed in the same namespace of ServiceAccount `pod-identity-webhook`.
### Configure the trust relationship for the OIDC provider's IAM Role
In order to grant certain service accounts access to the desired AWS resources, edit the trust relationship for the OIDC provider's IAM Role (`OIDC_IAM_ROLE`) created in the first section, and add in the desired service accounts.
1. Choose the role in the console to open it for editing.
1. Choose the **Trust relationships** tab, and then choose **Edit trust relationship**.
Expand All @@ -192,29 +196,29 @@ In order to grant certain service accounts access to the desired AWS resources,
```
"$ISSUER_HOSTPATH:sub": "system:serviceaccount:KUBERNETES_SERVICE_ACCOUNT_NAMESPACE:KUBERNETES_SERVICE_ACCOUNT_NAME"
```
Below are the one exmaple to allow list your `my-serviceaccount` service account in `default` namespace and all service accounts in `observability` namespace for `us-west-2` region. Remember to replace `Account_ID` and `S3_BUCKET` with required values.
```
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::$Account_ID:oidc-provider/s3.us-west-2.amazonaws.com/$S3_BUCKET"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringLike": {
"s3.us-west-2.amazonaws.com/$S3_BUCKET:sub": [
"system:serviceaccount:default:my-serviceaccount",
"system:serviceaccount:amazon-cloudwatch:*"
]
}
}
}
]
}
```
The allow list example below applies `my-serviceaccount` service account to the `default` namespace and all service accounts to the `observability` namespace for the `us-west-2` region. Remember to replace `Account_ID` and `S3_BUCKET` with the required values.
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Federated": "arn:aws:iam::$Account_ID:oidc-provider/s3.us-west-2.amazonaws.com/$S3_BUCKET"
},
"Action": "sts:AssumeRoleWithWebIdentity",
"Condition": {
"StringLike": {
"s3.us-west-2.amazonaws.com/$S3_BUCKET:sub": [
"system:serviceaccount:default:my-serviceaccount",
"system:serviceaccount:amazon-cloudwatch:*"
]
}
}
}
]
}
```
Expand Down

0 comments on commit ed36457

Please sign in to comment.