Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DRIVERS-2882 Update EKS documentation and test against AKS #527

Merged
merged 8 commits into from
Oct 25, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions .evergreen/auth_oidc/k8s/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ The scripts can be run locally as follows:
```bash
bash setup.sh local # needs to be done once to set up variables
bash setup-pod.sh aks # or gke, or eks
bash start-server.sh
bash start-server.sh # start a server on the pod
bash run-self-test.sh
```

Expand All @@ -24,7 +24,7 @@ Or if running tests for a specific driver:
```bash
bash setup.sh local # needs to be done once to set up variables
bash setup-pod.sh aks # or gke, or eks
bash start-server.sh
bash start-server.sh # start a server on the pod
pushd $PROJECT_HOME
export K8S_DRIVERS_TAR_FILE=/tmp/driver.tgz
git archive -o $K8S_DRIVERS_TAR_FILE HEAD
Expand All @@ -35,26 +35,30 @@ bash run-driver-test.sh

### Local EKS Testing

Local EKS testing requires assuming a role to interact with the EKS cluster.
Local EKS testing requires assuming the drivers test secrets role to interact with the EKS cluster.
See the [Wiki](https://wiki.corp.mongodb.com/display/DRIVERS/Using+AWS+Secrets+Manager+to+Store+Testing+Secrets) for details.


## EVG Usage

The test should use a task group to ensure the resources are cleaned up properly.

Note: The EKS test relies on assuming the drivers test secrets role, so ensure you are assuming that
role prior to the EKS test and that `duration_seconds` is set to account for the full duration of the tests.

```yaml
- name: test_oidc_k8s_task_group
setup_group_can_fail_task: true
setup_group_timeout_secs: 1800
teardown_task_can_fail_task: true
teardown_group_timeout_secs: 1800 # 30 minutes
teardown_group_timeout_secs: 180 # 3 minutes (max allowed time)
setup_group:
- func: fetch source
- func: prepare resources
- command: subprocess.exec
params:
binary: bash
include_expansions_in_env: ["AWS_ACCESS_KEY_ID", "AWS_SECRET_ACCESS_KEY", "AWS_SESSION_TOKEN"]
args:
- ${DRIVERS_TOOLS}/.evergreen/auth_oidc/k8s/setup.sh
teardown_group:
Expand All @@ -77,6 +81,10 @@ And should be run for all three variants:
- name: "test-oidc-k8s"
tags: ["latest", "oidc", "pr"]
commands:
- command: ec2.assume_role
params:
role_arn: ${drivers_test_secrets_role}
duration_seconds: 1800
- func: "run oidc k8s test"
vars:
VARIANT: eks
Expand All @@ -92,9 +100,6 @@ Where the test looks something like:

```yaml
"run oidc k8s test":
- command: ec2.assume_role
params:
role_arn: ${drivers_test_secrets_role}
- command: shell.exec
type: test
params:
Expand Down
7 changes: 4 additions & 3 deletions .evergreen/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -952,9 +952,9 @@ tasks:
- func: "run oidc k8s atlas test"
vars:
VARIANT: gke
# - func: "run oidc k8s atlas test"
# vars:
# VARIANT: aks
- func: "run oidc k8s atlas test"
vars:
VARIANT: aks

- name: "test-oidc-k8s-local"
tags: ["latest", "oidc", "pr"]
Expand All @@ -965,6 +965,7 @@ tasks:
- func: "run oidc k8s local test"
vars:
VARIANT: gke
# TODO: This still has to be implemented.
# - func: "run oidc k8s local test"
# vars:
# VARIANT: aks
Expand Down
8 changes: 4 additions & 4 deletions .evergreen/ensure-binary.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,10 @@ if [ -z "$DRIVERS_TOOLS" ]; then
return 1
fi

# if command -v $NAME &> /dev/null; then
# echo "$NAME found in PATH!"
# return 0
# fi
if command -v $NAME &> /dev/null; then
echo "$NAME found in PATH!"
return 0
fi

OS_NAME=$(uname -s | tr '[:upper:]' '[:lower:]')
MARCH=$(uname -m | tr '[:upper:]' '[:lower:]')
Expand Down
12 changes: 10 additions & 2 deletions .evergreen/k8s/eks/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,17 @@ Scripts to manage a drivers test cluster on AWS.
These steps must be done by an account with admin access (one time):

1. Run `setup-cluster.sh`
2. Set up an [access entry](https://docs.aws.amazon.com/eks/latest/userguide/access-entries.html) for the
drivers test secrets role.
2. Set up an Access entry for the drivers test secrets role.
- Go to the cluster on the AWS Console.
- Click "Access".
- Click "Create access entry".
- Use the drivers test secrets role.
- Give it admin access to the cluster.
3. Store the secrets in the AWS vault.
- When re-creating the cluster, you must update `K8S_OIDC_ISSUER` in the eks vault
with the new issuer which can be found in IAM > Identity Providers
(prepending the Provider with `https://`).
You must also update the issuer in Atlas cloud-dev.

## Usage

Expand Down
Loading