Does this solution work together with AWS Control Tower?

[lhenriks]

Hi,
I'm having trouble to get this solution working when using an existing Config Aggregator deployed/configured by AWS Control Tower StackSets.

Has anyone got this solution working together with AWS Control Tower with CrossAccountDiscovery=AWS_ORGANIZATIONS?

Br,
Lassi

[svozza]

What sort of errors are you seeing (you can find any errors in the discovery process by following the steps at the bottom of this page)? How many resources are in the aggregator? If the number is above 100k, you'll need to increase the size of the Neptune DB and OpenSearch instances. Also, you may need to increase the memory of the Fargate task that does resource discovery. These are all parameters in the CloudFormation template (you can see descriptions of them in step 5 of this page in the documentation).

[lhenriks]

Sorry for the late response, I was having a small break from this topic.

I did double the Neptune DB & OpenSearch instance sizes and defined 2vCPU/4096MB for the Fargate task.
Here's the logs:

/ecs/workload-discovery-task

{
    "message": "Beginning discovery of resources",
    "level": "info",
    "timestamp": "2023-10-17T12:16:41.159Z"
}
{
    "message": "Initialising discovery process",
    "level": "info",
    "timestamp": "2023-10-17T12:16:41.162Z"
}
{
    "level": "info",
    "durationMs": 242,
    "message": "Time to initialise",
    "timestamp": "2023-10-17T12:16:41.404Z"
}
{
    "message": "There were 0 errors when importing resource types unsupported by advanced query.",
    "level": "error",
    "timestamp": "2023-10-17T12:16:41.872Z"
}
{
    "level": "info",
    "durationMs": 1142,
    "message": "Time to download relationships from Neptune",
    "timestamp": "2023-10-17T12:16:42.559Z"
}
{
    "level": "info",
    "durationMs": 1158,
    "message": "Time to download resources from Neptune",
    "timestamp": "2023-10-17T12:16:42.577Z"
}
{
    "message": "Error in Discovery process.",
    "level": "error",
    "timestamp": "2023-10-17T12:16:43.653Z"
}
{
    "message": "Cannot read properties of undefined (reading 'AllAwsRegions')",
    "level": "error",
    "timestamp": "2023-10-17T12:16:43.654Z"
}

/aws/lambda/aws-wl-discovery-GremlinRes-GremlinAppSyncFunction

Each run will generate two separate log streams:

# 1st stream:

INIT_START Runtime Version: nodejs:18.v13	Runtime Version ARN: arn:aws:lambda:eu-west-1::runtime:0229ff5ced939264450549058d8f267110e92677c27063e6dcd781a280f2462b
START RequestId: 47671f07-04c2-42a8-928a-744d8aa36732 Version: $LATEST
2023-10-17T12:16:42.453Z	47671f07-04c2-42a8-928a-744d8aa36732	INFO	Initializing connection
2023-10-17T12:16:42.452Z	47671f07-04c2-42a8-928a-744d8aa36732	INFO	GraphQL arguments:	
{
    "awsRequestId": "47671f07-04c2-42a8-928a-744d8aa36732",
    "x-correlation-trace-id": "Root=1-652e7b29-0bc3c78b3bd7cd364ea0a7f7;Parent=4dee40717ea98284;Sampled=1;Lineage=de5b9c1d:0",
    "x-correlation-id": "47671f07-04c2-42a8-928a-744d8aa36732",
    "level": 30,
    "time": 1697545002452,
    "pid": 8,
    "hostname": "169.254.131.177",
    "arguments": {
        "pagination": {
            "start": 0,
            "end": 1000
        },
        "resourceTypes": null,
        "accounts": null
    },
    "msg": "GraphQL arguments:"
}

END RequestId: 47671f07-04c2-42a8-928a-744d8aa36732
REPORT RequestId: 47671f07-04c2-42a8-928a-744d8aa36732	Duration: 106.82 ms	Billed Duration: 107 ms	Memory Size: 1024 MB	Max Memory Used: 85 MB	Init Duration: 422.98 ms	
XRAY TraceId: 1-652e7b29-0bc3c78b3bd7cd364ea0a7f7	SegmentId: 7adda8a06e6aee1a	Sampled: true	

# 2nd stream:

INIT_START Runtime Version: nodejs:18.v13	Runtime Version ARN: arn:aws:lambda:eu-west-1::runtime:0229ff5ced939264450549058d8f267110e92677c27063e6dcd781a280f2462b
START RequestId: 0716d675-50ed-41bf-937a-79e8a7147ca1 Version: $LATEST
2023-10-17T12:16:42.437Z	0716d675-50ed-41bf-937a-79e8a7147ca1	INFO	GraphQL arguments:	
{
    "awsRequestId": "0716d675-50ed-41bf-937a-79e8a7147ca1",
    "x-correlation-trace-id": "Root=1-652e7b29-3245debc24dc3ecf09aad1be;Parent=ec032a0997a578b3;Sampled=1;Lineage=de5b9c1d:0",
    "x-correlation-id": "0716d675-50ed-41bf-937a-79e8a7147ca1",
    "level": 30,
    "time": 1697545002437,
    "pid": 8,
    "hostname": "169.254.40.177",
    "arguments": {
        "pagination": {
            "start": 0,
            "end": 2000
        }
    },
    "msg": "GraphQL arguments:"
}

2023-10-17T12:16:42.440Z	0716d675-50ed-41bf-937a-79e8a7147ca1	INFO	Initializing connection
END RequestId: 0716d675-50ed-41bf-937a-79e8a7147ca1
REPORT RequestId: 0716d675-50ed-41bf-937a-79e8a7147ca1	Duration: 99.83 ms	Billed Duration: 100 ms	Memory Size: 1024 MB	Max Memory Used: 87 MB	Init Duration: 416.54 ms	
XRAY TraceId: 1-652e7b29-3245debc24dc3ecf09aad1be	SegmentId: 7363296a1faf31a9	Sampled: true	

[svozza]

It appears as though there is a field missing in the response we get back from the describe call we make to discover the organisations Config aggregator. Could you run the following CLI command in the region where the aggregator and show me the description return of the Control Tower aggregator you are using:

aws configservice describe-configuration-aggregators

[lhenriks]

Here's the output:

{
    "ConfigurationAggregators": [
        {
            "ConfigurationAggregatorName": "aws-controltower-GuardrailsComplianceAggregator",
            "ConfigurationAggregatorArn": "arn:aws:config:eu-west-1:111111111111:config-aggregator/config-aggregator-1z0pack7",
            "AccountAggregationSources": [
                {
                    "AccountIds": [
                        "111111111111",
                        "222222222222",
                        "333333333333",
                        "444444444444",
                        "555555555555",
                        "666666666666",
                        "777777777777",
                        "888888888888",
                        "999999999999",
                        "101010101010"
                    ],
                    "AllAwsRegions": true
                }
            ],
            "CreationTime": "2021-10-29T16:36:30.862000+03:00",
            "LastUpdatedTime": "2023-08-30T17:13:31.420000+03:00"
        }
    ]
}

[svozza]

The issue is that the aggregator you are providing is not an Organization wide aggregator. If it was, the output would look something like this:

{
    "ConfigurationAggregators": [
        {
            "ConfigurationAggregatorName": "someAggregatorName",
            "ConfigurationAggregatorArn": "arn:aws:config:eu-west-1:111111111111:config-aggregator/config-aggregator-h06fp2ge",
            "OrganizationAggregationSource": {
                "RoleArn": "arn:aws:iam::111111111111:role/service-role/someRole",
                "AllAwsRegions": true
            },
            "CreationTime": "2023-09-19T08:57:39.276000+01:00",
            "LastUpdatedTime": "2023-09-19T08:57:39.276000+01:00"
        }
    ]
}

If you rerun the deployment template and and change the ConfigAggregatorName to be blank then the solution will create an organisation wide aggregator for you. I will also add an item to our backlog to throw a more descriptive error if the aggregator is not an org-wide aggregator.

[lhenriks]

Oh, indeed the account where I was trying to setup this solution had only "Individual accounts" type Config Aggregator, not "My organization". It was deployed by the AWS Control Tower, so I somehow assumed that it's already organization wide aggregator as it had all the accounts there, also the new ones.

Now it started discovering the resources, thank you for the assistance!

[svozza]

Yeah, I thought the Control Tower one would be an Org one too as well! Glad it's working now!

[lhenriks]

Actually the aggregator inside the Management account were org. wide (deployed by CT), but not the one which was sitting under Audit account.