❗NOTICE: Cross-account deployments are failing in v2.163.0 with error 'Need to perform AWS calls for account'

[cgatt]

Please add your +1 👍 to let us know you have encountered this

---

Status: RESOLVED

Overview:

Release v2.163.0 introduced an issues with cross-account deployments. The AWS CDK fails with the error Need to perform AWS calls for account XXXXX, but the current credentials are for YYYYY. The root cause is a new check for the bootstrap version added in v2.163.0. The check needs to make an API call to AWS CloudFormation to describe the bootstrap stack. This API call was using incorrect credentials.

Complete Error Message:

Need to perform AWS calls for account XXXXXXXXXXXX, but the current credentials are for YYYYYYYYYYYY

Workaround:

Downgrade the AWS CDK CLI to version 2.162.1

Solution:

Upgrade the AWS CDK CLI to version 2.163.1

Related Issues:

n/a

---

Original issue:

Describe the bug

Release v2.163.0 causes deployments using cross account role assumption to fail with the error Need to perform *** calls for account XXXXX, but the current credentials are for YYYYY. This is presumably caused by #31623.
Based on comments on this PR (as it has no description or linked issue) this was done to address a vulnerability when the assets bucket for an account is not longer in the target account, but this failure is occurring when the assets bucket is still in the target account with no change.
These deployments and buckets are in the ap-southeast-2 region.

Regression Issue

Confirmed Regression

Last Known Working CDK Version

2.162.1

Expected Behavior

Cross account deployments should succeed unless the asset bucket is no longer in the target account.

Current Behavior

Cross account deployments fail with the error Need to perform *** calls for account XXXXX, but the current credentials are for YYYYY
cdk diff succeeds as expected, and cloudtrail shows role assumption is successful.
Debug log excerpt just before the failure:

[04:15:56] datadog-lambda-apm-test-dev: check: Check s3://cdk-hnb659fds-assets-XXXXXXXXXXXX-ap-southeast-2/2819175352ad1ce0dae768e83fc328fb70fb5f10b4a8ff0ccbcb791f02b0716d.zip
[04:15:56] [*** s3 200 0.06s 0 retries] listObjectsV2({
  Bucket: 'cdk-hnb659fds-assets-XXXXXXXXXXXX-ap-southeast-2',
  Prefix: 'fdfffca2098cb4f12a1dd3f0f2f98b4606063436cdfc311da0589251fa61cbe0.json',
  MaxKeys: 1
})
[04:15:56] [*** s3 200 0.074s 0 retries] listObjectsV2({
  Bucket: 'cdk-hnb659fds-assets-XXXXXXXXXXXX-ap-southeast-2',
  Prefix: '7fbaff7dbc84f82c58c19512abc1c6e165bebea0038d20da2d2ee153808fee70.zip',
  MaxKeys: 1
})
[04:15:56] datadog-lambda-apm-test-dev: found: Found s3://cdk-hnb659fds-assets-XXXXXXXXXXXX-ap-southeast-2/7fbaff7dbc84f82c58c19512abc1c6e165bebea0038d20da2d2ee153808fee70.zip
[04:15:56] [*** s3 200 0.06s 0 retries] listObjectsV2({
  Bucket: 'cdk-hnb659fds-assets-XXXXXXXXXXXX-ap-southeast-2',
  Prefix: '2819175352ad1ce0dae768e83fc328fb70fb5f10b4a8ff0ccbcb791f02b0716d.zip',
  MaxKeys: 1
})
[04:15:56] datadog-lambda-apm-test-dev: found: Found s3://cdk-hnb659fds-assets-XXXXXXXXXXXX-ap-southeast-2/2819175352ad1ce0dae768e83fc328fb70fb5f10b4a8ff0ccbcb791f02b0716d.zip
[04:15:56] 3 total assets, 1 still need to be published
[04:15:56] [trace] SdkProvider#resolveEnvironment()
[04:15:56] [trace] SdkProvider#baseCredentialsPartition()
[04:15:56] [trace]   SdkProvider#resolveEnvironment()
[04:15:56] [trace]   SdkProvider#obtainBaseCredentials()
[04:15:56] [trace]     SdkProvider#defaultAccount()
[04:15:56] [trace]     SdkProvider#defaultCredentials()
[04:15:56] [trace]   SDK#currentAccount()
[04:15:56] [trace]     SDK#forceCredentialRetrieval()
[04:15:56] Retrieved account ID YYYYYYYYYYYY from disk cache
[04:15:56] [trace] SDK#ssm()
[04:15:56] [trace]   SDK#wrapServiceErrorHandling()
[04:15:56] [*** ssm 200 0.062s 0 retries] getParameter({ Name: '/cdk-bootstrap/hnb659fds/version' })
datadog-lambda-apm-test-dev: start: Building fdfffca2098cb4f12a1dd3f0f2f98b4606063436cdfc311da0589251fa61cbe0:XXXXXXXXXXXX-ap-southeast-2
datadog-lambda-apm-test-dev: success: Built fdfffca2098cb4f12a1dd3f0f2f98b4606063436cdfc311da0589251fa61cbe0:XXXXXXXXXXXX-ap-southeast-2
[04:15:56] [trace] SdkProvider#resolveEnvironment()
[04:15:56] [trace] SdkProvider#baseCredentialsPartition()
[04:15:56] [trace]   SdkProvider#resolveEnvironment()
[04:15:56] [trace]   SdkProvider#obtainBaseCredentials()
[04:15:56] [trace]     SdkProvider#defaultAccount()
[04:15:56] [trace]     SdkProvider#defaultCredentials()
[04:15:56] [trace]   SDK#currentAccount()
[04:15:56] [trace]     SDK#forceCredentialRetrieval()
[04:15:56] Retrieved account ID YYYYYYYYYYYY from disk cache
[04:15:56] [trace] SdkProvider#forEnvironment()
[04:15:56] [trace]   SdkProvider#resolveEnvironment()
[04:15:56] [trace]   SdkProvider#obtainBaseCredentials()
[04:15:56] [trace]     SdkProvider#defaultAccount()
[04:15:56] [trace]     SdkProvider#defaultCredentials()
[04:15:56] Notices refreshed
Need to perform *** calls for account XXXXXXXXXXXX, but the current credentials are for YYYYYYYYYYYY
[04:15:56] Error: Need to perform *** calls for account XXXXXXXXXXXX, but the current credentials are for YYYYYYYYYYYY
    at SdkProvider.forEnvironment (/home/runner/.npm/_npx/39b4457c6eead837/node_modules/aws-cdk/lib/api/aws-auth/sdk-provider.ts:195:60)
    at Deployments.cachedSdkForEnvironment (/home/runner/.npm/_npx/39b4457c6eead837/node_modules/aws-cdk/lib/api/deployments.ts:918:17)
    at Deployments.allowCrossAccountAssetPublishingForEnv (/home/runner/.npm/_npx/39b4457c6eead837/node_modules/aws-cdk/lib/api/deployments.ts:863:20)
    at Deployments.publishSingleAsset (/home/runner/.npm/_npx/39b4457c6eead837/node_modules/aws-cdk/lib/api/deployments.ts:855:62)
    at Object.publishAsset (/home/runner/.npm/_npx/39b4457c6eead837/node_modules/aws-cdk/lib/cdk-toolkit.ts:254:7)
    at /home/runner/.npm/_npx/39b4457c6eead837/node_modules/aws-cdk/lib/util/work-graph.ts:111:11

Reproduction Steps

1. Create a role in account A
2. Bootstrap account B with account A (or the role in account A) as the trusted principle
3. Assume role in account A
4. Run npx cdk deploy on a stack with account B as the env.account property. May need to also set region to ap-southeast-2, I have not been able to test this in us-east-1 yet.

Possible Solution

Roll back #31623 until full integration testing can be performed.

Additional Information/Context

This change has caused all deployments from our CICD to fail, as they make use of cross account role assumption and use the latest CDK V2 CLI unless otherwise specified (due to caret version matching to avoid breaking changes). I would like to request that this change be rolled back until it can be thoroughly integration tested. If this is not possible, the new expected requirements for cross account CDK deployments need to be clearly documented.

CDK CLI Version

2.163.0

Framework Version

No response

Node.js Version

v20.13.1

OS

Debian Linux

Language

TypeScript

Language Version

TypeScript (5.6.3)

Other information

No response

[yassine-ops]

We are facing the same issue after upgrading to release v2.163.0, we are encountering errors during deployments involving cross-account role assumptions:

Need to perform *** calls for account XXXXX, but the current credentials are for YYYYY.

Downgrading to v2.162.0 resolves the issue for now.

[cgatt]

Apologies in advance that I have not tested this as thoroughly as I normally would before raising an issue, but my ability to spin up completely fresh AWS accounts is limited in the org environment. If the reproduction steps as I've written them do work (i.e. dont reproduce the issue) I have the following additional variables that I've observed in our environment:

• Bootstrap stack name is different in account A vs account B
• Bootstrap stack version is v20
• Bootstrack trust principal is a role, rather than account

I will attempt to do further testing if I can get some clean accounts approved, but I'm hoping that you guys have better options in the "clean test accounts" field than I do.

[manumaki]

Having same problem here.

[kevin-journey]

Same issue here. Downgrading to cdk@2.162.1 resolves it for now.

[cgatt]

After further investigation, I can confirm this was caused by #31623, specifically this line:
https://github.com/aws/aws-cdk/pull/31623/files#diff-1c33e80dabddc5697544f07e314339fa3abe37c68183cb882ce5127b76f7deeeR863

const sdk = (await this.cachedSdkForEnvironment(env, Mode.ForReading)).sdk;

All other uses of cachedSdkForEnvironment in this file also specify the assume role details:

const stackSdk = await this.cachedSdkForEnvironment(resolvedEnvironment, Mode.ForReading, {
  assumeRoleArn: arns.lookupRoleArn,
  assumeRoleExternalId: stack.lookupRole?.assumeRoleExternalId,
  assumeRoleAdditionalOptions: stack.lookupRole?.assumeRoleAdditionalOptions,
});

In a cross-account deployment allowCrossAccountAssetPublishingForEnv is now requesting SDK credentials for an env that doesnt match the base credentials, but not providing an assume role ARN. This hits the exception handling here throwing the exception seen in the logs.

[niklaskallander]

We're having the same issue. Can confirm downgrading to cdk@2.162.1 works as suggested by @kevin-journey (Thanks!).

[github-actions]

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

[github-actions]

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

[mrgrain]

Release with fix is in progress.

[mrgrain]

v2.163.1 was released with a fix.

Can you please confirm if this resolves the issue for you?

[scottrmercer]

Same issue here that started with 2.163.0, except downgrading did not resolve anything (I tried going back to several different versions) - even after re-bootstrapoping both accounts. It broke deploys from githubactions as well as codepipeline. I've had a ticket open with AWS support for 4 days with absolutely no progress. The weirest part is that if I open up permissions on the deployment account, it will deploy the metadata to the correct target account, while all of the application assets get created in the deployment/pipeline account.

Fortunately, we can still deploy from local machines using a profile for the target account to bypass the assumerole step, otherwise this would be a very big problem.

[mrgrain]

Same issue here that started with 2.163.0, except downgrading did not resolve anything (I tried going back to several different versions) - even after re-bootstrapoping both accounts. It broke deploys from githubactions as well as codepipeline.

Hi @scottrmercer sorry you are having issues. Can you please provides some more details on the error you're hitting and the setup you're using? A full dump of the error output will do. What version did you upgrade from? What's the last known working version for you?

While I'm sure you are experiencing issues, this particular change was only introduced in v2.163.0. If reverting back to older versions does not resolve this for you, we are probably looking at a (slighty) different situation. Unfortunately the possible causes for this error message are quite varied and many of them are intentional.

[scottrmercer]

Of course! Thanks for the reply. Here is an abbreviated version of the log:

// AAAAAAAAAAA = deploment pipeline account
// BBBBBBBBBBB = target account

// When launching env:  {"account":"BBBBBBBBBBB","region":"us-west-2"}


// it assumes the correct role in BBBBBBBBBBB exactly once, then all subsequent assume role calls are against the AAAAAAAAAAA account

[21:48:51] Retrieved account ID 180239838014 from disk cache
[21:48:51] [trace]                 SdkProvider#forEnvironment()
[21:48:51] [trace]                   SdkProvider#resolveEnvironment()
[21:48:51] [trace]                   SdkProvider#obtainBaseCredentials()
[21:48:51] [trace]                     SdkProvider#defaultAccount()
[21:48:51] [trace]                     SdkProvider#defaultCredentials()
[21:48:51] [trace]                   SdkProvider#withAssumedRole()
[21:48:51] Assuming role 'arn:aws:iam::BBBBBBBBBBB:role/cdk-hnb659fds-deploy-role-BBBBBBBBBBB-us-west-2'.

// The deploy fails because the required permissions are not configured in the deploy account, since that is not our target

21:48:52] exampleservicesandboxexampleserviceExampleWebapisandbox1115F3F1: check: Check s3://cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2/e7d43f88a752fbb345e314be7c94924f7ec773f779fc8822c74fdd4fa71a7988.zip
[21:48:52] Assuming role failed: User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::AAAAAAAAAAA:role/cdk-hnb659fds-file-publishing-role-AAAAAAAAAAA-us-west-2
[21:48:52] Could not assume role in target account using current credentials User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: sts:AssumeRole on resource: arn:aws:iam::AAAAAAAAAAA:role/cdk-hnb659fds-file-publishing-role-AAAAAAAAAAA-us-west-2 . Please make sure that this role exists in the account. If it doesn't exist, (re)-bootstrap the environment with the right '--trust', using the latest version of the CDK CLI.
[21:48:52] current credentials could not be used to assume 'arn:aws:iam::AAAAAAAAAAA:role/cdk-hnb659fds-file-publishing-role-AAAAAAAAAAA-us-west-2', but are for the right account. Proceeding anyway.
[21:48:52] [trace] SDK#s3()
[21:48:52] [trace]   SDK#wrapServiceErrorHandling()
[21:48:52] exampleservicesandboxexampleserviceExampleWebapisandbox1115F3F1: check: Check s3://cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2/e8f803f71c760c26ea6e93c13029e9df0daa9000fe9c0ce7ff67e8af9b0be5a4.json
[21:48:52] [trace] SDK#makeDetailedException()
[21:48:52] Call failed: listObjectsV2({"Bucket":"cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2","Prefix":"bfff475abaadbe8543c8b90da0a4e6a0e59418f5e2e6faa867f0eaf93710d92e.zip","MaxKeys":1}) => User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: s3:ListBucket on resource: "arn:aws:s3:::cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2" because no identity-based policy allows the s3:ListBucket action (code=AccessDenied)
[21:48:52] exampleservicesandboxexampleserviceExampleWebapisandbox1115F3F1: debug: User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: s3:ListBucket on resource: "arn:aws:s3:::cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2" because no identity-based policy allows the s3:ListBucket action
[21:48:52] [trace] SDK#makeDetailedException()
[21:48:52] Call failed: listObjectsV2({"Bucket":"cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2","Prefix":"e7d43f88a752fbb345e314be7c94924f7ec773f779fc8822c74fdd4fa71a7988.zip","MaxKeys":1}) => User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: s3:ListBucket on resource: "arn:aws:s3:::cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2" because no identity-based policy allows the s3:ListBucket action (code=AccessDenied)
[21:48:52] exampleservicesandboxexampleserviceExampleWebapisandbox1115F3F1: debug: User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: s3:ListBucket on resource: "arn:aws:s3:::cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2" because no identity-based policy allows the s3:ListBucket action
[21:48:52] [trace] SDK#makeDetailedException()
[21:48:52] Call failed: listObjectsV2({"Bucket":"cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2","Prefix":"de51aa2658be27a8a82b34fb97810f073ca10764a9ade0840a9efb204489c530.zip","MaxKeys":1}) => User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: s3:ListBucket on resource: "arn:aws:s3:::cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2" because no identity-based policy allows the s3:ListBucket action (code=AccessDenied)
[21:48:52] exampleservicesandboxexampleserviceExampleWebapisandbox1115F3F1: debug: User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: s3:ListBucket on resource: "arn:aws:s3:::cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2" because no identity-based policy allows the s3:ListBucket action
[21:48:53] [trace] SDK#makeDetailedException()
[21:48:53] Call failed: listObjectsV2({"Bucket":"cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2","Prefix":"e8f803f71c760c26ea6e93c13029e9df0daa9000fe9c0ce7ff67e8af9b0be5a4.json","MaxKeys":1}) => User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: s3:ListBucket on resource: "arn:aws:s3:::cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2" because no identity-based policy allows the s3:ListBucket action (code=AccessDenied)
[21:48:53] exampleservicesandboxexampleserviceExampleWebapisandbox1115F3F1: debug: User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: s3:ListBucket on resource: "arn:aws:s3:::cdk-hnb659fds-assets-AAAAAAAAAAA-us-west-2" because no identity-based policy allows the s3:ListBucket action
[21:48:53] 5 total assets, 5 still need to be published
[21:48:53] [trace] SdkProvider#resolveEnvironment()
[21:48:53] [trace]   SdkProvider#defaultAccount()
[21:48:53] [trace] SdkProvider#baseCredentialsPartition()
[21:48:53] [trace]   SdkProvider#resolveEnvironment()
[21:48:53] [trace]   SdkProvider#obtainBaseCredentials()
[21:48:53] [trace]     SdkProvider#defaultAccount()
[21:48:53] [trace]     SdkProvider#defaultCredentials()
[21:48:53] [trace]   SDK#currentAccount()
[21:48:53] [trace]     SDK#forceCredentialRetrieval()
[21:48:53] Retrieved account ID AAAAAAAAAAA from disk cache
[21:48:53] [trace] SDK#ssm()
[21:48:53] [trace]   SDK#wrapServiceErrorHandling()
[21:48:53] [trace] SDK#makeDetailedException()
[21:48:53] Call failed: getParameter({"Name":"/cdk-bootstrap/hnb659fds/version"}) => User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: ssm:GetParameter on resource: arn:aws:ssm:us-west-2:AAAAAAAAAAA:parameter/cdk-bootstrap/hnb659fds/version because no identity-based policy allows the ssm:GetParameter action (code=AccessDeniedException)
[21:48:53] [trace] SDK#cloudFormation()
[21:48:53] [trace]   SDK#wrapServiceErrorHandling()
[21:48:53] Waiting for stack CDKToolkit to finish creating or updating...
[21:48:53] [trace] SDK#makeDetailedException()
[21:48:53] Call failed: describeStacks({"StackName":"CDKToolkit"}) => User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: cloudformation:DescribeStacks on resource: arn:aws:cloudformation:us-west-2:AAAAAAAAAAA:stack/CDKToolkit/5a42d3d0-3833-11ed-8a26-06f20dc37c49 because no identity-based policy allows the cloudformation:DescribeStacks action (code=AccessDenied)
[21:48:53] Notices refreshed
exampleservicesandboxexampleserviceExampleWebapisandbox1115F3F1: This CDK deployment requires bootstrap stack version '6', but during the confirmation via SSM parameter /cdk-bootstrap/hnb659fds/version the following error occurred: AccessDeniedException: User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: ssm:GetParameter on resource: arn:aws:ssm:us-west-2:AAAAAAAAAAA:parameter/cdk-bootstrap/hnb659fds/version because no identity-based policy allows the ssm:GetParameter action
[21:48:53] Error: exampleservicesandboxexampleserviceExampleWebapisandbox1115F3F1: This CDK deployment requires bootstrap stack version '6', but during the confirmation via SSM parameter /cdk-bootstrap/hnb659fds/version the following error occurred: AccessDeniedException: User: arn:aws:iam::AAAAAAAAAAA:user/github-actions-cdk-deployment is not authorized to perform: ssm:GetParameter on resource: arn:aws:ssm:us-west-2:AAAAAAAAAAA:parameter/cdk-bootstrap/hnb659fds/version because no identity-based policy allows the ssm:GetParameter action
    at Deployments.validateBootstrapStackVersion (/home/runner/work/example-service/example-service/cdk/node_modules/aws-cdk/lib/api/deployments.ts:898:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Deployments.buildSingleAsset (/home/runner/work/example-service/example-service/cdk/node_modules/aws-cdk/lib/api/deployments.ts:837:7)
    at Object.buildAsset (/home/runner/work/example-service/example-service/cdk/node_modules/aws-cdk/lib/cdk-toolkit.ts:254:7)
    at /home/runner/work/example-service/example-service/cdk/node_modules/aws-cdk/lib/util/work-graph.ts:108:11
Error: Process completed with exit code 1.

My CDK project is very vanilla:

bin/cdk.ts:

#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { CdkStack } from '../lib/cdk-stack';
import * as path from "path";
import {Config} from "@company/cdk-base-constructs";

const app = new cdk.App();

const environment = process.env.ENV || 'sandbox';

const config = new Config(path.join(__dirname, "../config.yml"));
const environmentConfig = config.getEnvironment(environment);

console.log('Deploying environment: ${environmentConfig.get('ENVIRONMENT')} -  ${JSON.stringify(environmentConfig.get("env"))}');

new CdkStack(app, 'example-service-' + environmentConfig.get('ENVIRONMENT'), {
    env: {account: "BBBBBBBBBBB", region: "us-west-2"},
    config: environmentConfig
});


------------------------------

lib/cdk-stack.js

import * as cdk from 'aws-cdk-lib';
import {Construct} from 'constructs';
import {StackProps} from "aws-cdk-lib";

import {Config} from "@everwise/cdk-base-constructs";
import {ExampleWebServiceApiStack} from "./example-web-service-api";

interface CdkStackProps extends StackProps {
    env: any;
    config: Config;
}

export class CdkStack extends cdk.Stack {
    constructor(scope: Construct, id: string, props: CdkStackProps) {
        super(scope, id, props);
        console.log('Building ExampleWebServiceApiStack env: ${JSON.stringify(props.config)}');
        new ExampleWebServiceApiStack(this, 'example-service-example-web-api-${props.config.get("ENVIRONMENT")}', {
            env: props.env,
            config: props.config,
        })

    }
}

lib/example-web-service-api.ts:

import {ApplicationSecrets, Config} from "@everwise/cdk-base-constructs";
import * as cdk from "aws-cdk-lib";
import {PolicyStatement} from "aws-cdk-lib/aws-iam";
import * as lambda from "aws-cdk-lib/aws-lambda";
import {Construct} from "constructs";
import * as path from "path";
import * as bedrock from "aws-cdk-lib/aws-bedrock";

export interface ExampleWebServiceStackProps extends cdk.StackProps {
    env: any;
    config: Config;
}

export class ExampleWebServiceApiStack extends cdk.Stack {
    constructor(scope: Construct, id: string, props: ExampleWebServiceStackProps) {
        super(scope, id);

        console.log('Building ExampleWebServiceApiStack env: ${JSON.stringify(props.env)}');

        cdk.Tags.of(this).add("Service", "example-api");
        cdk.Tags.of(this).add("Owner", "Falcon");

        // convert the above to a list of strings
        const piiEntities = [
            "ADDRESS",
            "AGE",
            "AWS_ACCESS_KEY",
            "AWS_SECRET_KEY",
            "CA_HEALTH_NUMBER",
            "CA_SOCIAL_INSURANCE_NUMBER",
            "CREDIT_DEBIT_CARD_CVV",
            "CREDIT_DEBIT_CARD_EXPIRY",
            "CREDIT_DEBIT_CARD_NUMBER",
            "DRIVER_ID",
            "EMAIL",
            "INTERNATIONAL_BANK_ACCOUNT_NUMBER",
            "IP_ADDRESS",
            "LICENSE_PLATE",
            "MAC_ADDRESS",
            "NAME",
            "PASSWORD",
            "PHONE",
            "PIN",
            "SWIFT_CODE",
            "UK_NATIONAL_HEALTH_SERVICE_NUMBER",
            "UK_NATIONAL_INSURANCE_NUMBER",
            "UK_UNIQUE_TAXPAYER_REFERENCE_NUMBER",
            "URL",
            "USERNAME",
            "US_BANK_ACCOUNT_NUMBER",
            "US_BANK_ROUTING_NUMBER",
            "US_INDIVIDUAL_TAX_IDENTIFICATION_NUMBER",
            "US_PASSPORT_NUMBER",
            "US_SOCIAL_SECURITY_NUMBER",
            "VEHICLE_IDENTIFICATION_NUMBER",
        ];

        const cfnGuardrail = new bedrock.CfnGuardrail(
            this,
            'ExampleWebServiceGuardrail-${props.config.get("ENVIRONMENT")}',
            {
                blockedInputMessaging: "blockedInputMessaging",
                blockedOutputsMessaging: "blockedOutputsMessaging",
                name: 'ExampleWebServiceGuardrail-${props.config.get("ENVIRONMENT")}',
                contextualGroundingPolicyConfig: {
                    filtersConfig: [
                        {
                            threshold: 0.7,
                            type: "GROUNDING",
                        },
                        {
                            threshold: 0.7,
                            type: "RELEVANCE",
                        },
                    ],
                },
                description: "PII filtering for API",
                sensitiveInformationPolicyConfig: {
                    piiEntitiesConfig: piiEntities.map((entity) => ({
                        action: "ANONYMIZE",
                        type: entity,
                    })),
                },
            }
        );

        const cfnGuardrailVersion = new bedrock.CfnGuardrailVersion(
            this,
            'ExampleWebServiceGuardrailVersion-${props.config.get("ENVIRONMENT")}',
            {
                guardrailIdentifier: cfnGuardrail.attrGuardrailId,
                description: "PII filtering - versioned",
            }
        );

        const apiSecrets = new ApplicationSecrets(
            this,
            'ExampleWebServiceApiSecrets-${props.config.get("ENVIRONMENT")}',
            {}
        );
        const apiWebhook = new lambda.Function(
            this,
            'ExampleWebServiceApiWebhook-${props.config.get("ENVIRONMENT")}',
            {
                runtime: lambda.Runtime.PYTHON_3_12,
                handler: "handler.handler",
                timeout: cdk.Duration.minutes(15),
                memorySize: 2048,
                environment: {
                    GUARDRAIL_ID: cfnGuardrail.attrGuardrailId,
                    GUARDRAIL_VERSION: cfnGuardrailVersion.attrVersion,
                    LOG_LEVEL: props.config.get("logLevel"),
                },
                code: lambda.Code.fromAsset(
                    path.join(__dirname, "../../src/lambdas/example-web-service"),
                    {
                        bundling: {
                            image: lambda.Runtime.PYTHON_3_12.bundlingImage,
                            command: [
                                "bash",
                                "-xc",
                                [
                                    "pip install -r requirements.txt -t /asset-output",
                                    "cp -a . /asset-output",
                                    "rm -rf /asset-output/{.vscode,requirements-dev.txt,run.py,README.md}",
                                ].join("&&"),
                            ],
                        },
                    }
                ),
            }
        );

        apiSecrets.wrapLambda(apiWebhook);

        apiWebhook.addToRolePolicy(
            new PolicyStatement({
                actions: ["bedrock:InvokeModel"],
                resources: ["*"],
            })
        );

        apiWebhook.addToRolePolicy(
            new PolicyStatement({
                actions: ["bedrock:ApplyGuardrail"],
                resources: [cfnGuardrail.attrGuardrailArn],
            })
        );

        const fnUrl = new lambda.FunctionUrl(
            this,
            "ExampleWebServiceApiWebhookUrl",
            {
                function: apiWebhook,
                authType: lambda.FunctionUrlAuthType.NONE,
                cors: {
                    allowedOrigins: ["*"],
                    allowedHeaders: [
                        "authorization",
                        "x-request-id",
                        "content-type",
                        "x-app-agent",
                    ],
                    allowCredentials: false,
                },
            }
        );

        new cdk.CfnOutput(this, "ExampleWebServiceApiWebhookUrlOutput", {
            value: fnUrl.url,
            exportName: 'ExampleWebServiceApiWebhookUrlOutput${props.config.get(
                "ENVIRONMENT"
            )}',
            description: "",
        });
    }
}```

[mrgrain]

Thanks @scottrmercer that's very helpful. One more thing: What version of the bootstrap template are you on? Could you manually check this please. Are you using custom bootstrapping at all?

[rix0rrr]

I can't reproduce this on my machine. I'm seeing some irregularity in the code of your app (or at least, it doesn't look like this is what you intended):

export class ExampleWebServiceApiStack extends cdk.Stack {
    constructor(scope: Construct, id: string, props: ExampleWebServiceStackProps) {
        super(scope, id);
//	               ^^^^^^ missing passing 'props' to super()

Even having said that, that shouldn't cause a failure, just deploy to the wrong environment.

EDIT: it will cause a failure if it's not possible to deploy to environment AAAAAAAAAA at all because it hasn't been bootstrapped; assuming the AAAA-deploy-role will fail, but it will proceed anyway with the current credentials because they are for the right account, and then fail because presumably your GitHub role doesn't have a lot of permissions.

[scottrmercer]

@rix0rrr - that was the issue, excellent catch.
@mrgrain - I appreciate your time.
Many thanks to you both, this is a big relief!

[cgatt]

v2.163.1 was released with a fix.

Can you please confirm if this resolves the issue for you?

I can confirm that 2.163.1 resolved the issue for us, thanks for the quick turnaround.