Use configure-aws-credentials workflow instead of passing secret_access_key

[vudiep411]

Summary

This PR fixes #1346 where we can get rid of the long term credentials by using OpenID Connect. OpenID Connect (OIDC) allows your GitHub Actions workflows to access resources in Amazon Web Services (AWS), without needing to store the AWS credentials as long-lived GitHub secrets.

Changes

We can remove these secrets that were passed in previously:

token: ${{ secrets.GITHUB_TOKEN }}
      bucket: ${{ secrets.AWS_S3_BUCKET }}
      access_key_id: ${{ secrets.AWS_S3_ACCESS_KEY_ID }}
      secret_access_key: ${{ secrets.AWS_S3_ACCESS_KEY }}

Instead we only need the role-to-assume arn. For more information OIDC.

Prerequisites

Before merging this PR, we need to make sure to set up the proper Identity providers on the production AWS account. Follow this guides.
Quick guide:

1. Create an Identity provider with OpenID Connect.
   Provider url: https://token.actions.githubusercontent.com
   
   Audience: sts.amazonaws.com

2. Assign the role into this new provider with this trust policy like below

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "arn:aws:iam::<aws_account_id>:oidc-provider/token.actions.githubusercontent.com"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringLike": {
                    "token.actions.githubusercontent.com:sub": [
                        "repo:valkey-io/valkey:ref:refs/heads/unstable",
                        "repo:valkey-io/valkey:ref:refs/tags/*"
                    ],
                    "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
                }
            }
        }
    ]
}

3. In Github secrets, Add these secrets:

Results

Github action run:

Files in S3 Dev env:

Note: the failed workflow can be found in this issue #1343 which will be in a separate PR.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 70.56%. Comparing base (9305b49) to head (cd1fe2d).
Report is 2 commits behind head on unstable.

Additional details and impacted files

@@             Coverage Diff              @@
##           unstable    #1363      +/-   ##
============================================
- Coverage     70.62%   70.56%   -0.06%     
============================================
  Files           117      117              
  Lines         63315    63315              
============================================
- Hits          44714    44679      -35     
- Misses        18601    18636      +35     

see 11 files with indirect coverage changes

[enjoy-binbin]

seems ok, @roshkhatri please take a look

[roshkhatri]

LGTM, just had a few questions.

[roshkhatri]

Why do we need the write permissions here?

[vudiep411]

This is to ensure that your workflow has the necessary permission to perform OIDC authentication with AWS.

[roshkhatri]

why do we need write permissions here?

[vudiep411]

Same as the comment above. Each job spawned in the matrix runs in a separate environment and each of this env needs permission to perform OIDC authentication with AWS.

[roshkhatri]

Is the aws cli installed by aws-actions/configure-aws-credentials@v4 ??

[vudiep411]

Yes, aws-actions/configure-aws-credentials@v4 will configured the runner environment with the proper AWS credentials and CLI

[roshkhatri]

We will also need to do the prerequisites steps for the main repo before we merge this.

[roshkhatri]

@vudiep411, will it be possible to add a test, where we uploads a test build binary to a test s3 bucket when ever changes are made to these workflows.

With this we can be sure that when we release stuff, it doesn't break on the main valkey repository

[vudiep411]

Yes it is possible, we can use github environment for that. So we can write something like this:

- name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v2
        with:
          role-to-assume: ${{ env.ROLE_TO_ASSUME }}
          role-session-name: ${{ env.ROLE_SESSION_NAME }}
          aws-region: ${{ env.AWS_REGION }}

We use env to dynamically assigned the role to assume and env. But this would required us to set up multiple OCID on different accounts not just in the workflow itself so it should be in a separate issue. For now we can just set this one up and after this looks good, I will open another issue for it if that's ok.
@roshkhatri

[roshkhatri]

I think we can also do it based on the github event trigger maybe?

1. We set two OCID for two buckets, releases and test.
2. When the changes are made to release workflow files, for a pull_request event we run the same workflow with secrets.AWS_TEST_ROLE_TO_ASSUME and secrets.AWS_TEST_S3_BUCKET and for a workflow_dispatch and release
   event trigger we run with the current setup.
3. We would do it at the head of unstable.
4. Once we upload the test binaries, we can confirm the upload, delete the delete the file and pass the test.

Also, I think we should add the test on the same PR so we can run the test on this PR and know that the tests also work. We would also have to do the pre-requisites only once for both the scenarios.

[vudiep411]

That's a really good suggestion. We can look into that. That way it would automated the testing of a PR with a test bucket. Definitely doable