Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ec2): flow logs from TransitGateway and TransitGatewayAttachment #28605

Merged
merged 30 commits into from
Jan 22, 2024

Conversation

badmintoncryer
Copy link
Contributor

@badmintoncryer badmintoncryer commented Jan 7, 2024

I have enabled the configuration of flow logs for TransitGateway and TransitGatewayAttachment.

Create flow logs from TransitGateway:

declare const tgw: ec2.CfnTransitGateway;

new ec2.FlowLog(this, 'TransitGatewayFlowLog', {
  resourceType: ec2.FlowLogResourceType.fromTransitGatewayId(tgw.ref)
})

Create flowlogs from TransitGatewayAttachment:

declare const tgwAttachment: ec2.CfnTransitGatewayAttachment;

new ec2.FlowLog(this, 'TransitGatewayAttachmentFlowLog', {
  resourceType: ec2.FlowLogResourceType.fromTransitGatewayAttachmentId(tgwAttachment.ref)
})

Since trafficType cannot be set for flow logs related to TransitGateway resources, I have also added error handling for this.

if (props.resourceType.resourceType === 'TransitGateway' || props.resourceType.resourceType === 'TransitGatewayAttachment') {
      if (props.trafficType) {
        throw new Error('trafficType is not supported for Transit Gateway and Transit Gateway Attachment');
      }
      ...
    }

Closes #27222.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@github-actions github-actions bot added effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2 labels Jan 7, 2024
@aws-cdk-automation aws-cdk-automation requested a review from a team January 7, 2024 07:00
@github-actions github-actions bot added the valued-contributor [Pilot] contributed between 6-12 PRs to the CDK label Jan 7, 2024
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@aws-cdk-automation aws-cdk-automation dismissed their stale review January 7, 2024 12:52

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

@badmintoncryer badmintoncryer marked this pull request as ready for review January 7, 2024 12:53
@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Jan 7, 2024
Copy link
Contributor

@go-to-k go-to-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Validation of maxAggregationInterval might be a good idea in this PR.
Also, it might be good to comment the description for the validation in the doc of the maxAggregationInterval parameter (and add unit tests for it).

--max-aggregation-interval has a default value of 60, and is the only accepted value for transit gateway resource types. An error is returned if you try to pass any other value. This limit applies only to transit gateway resource types.

https://docs.aws.amazon.com/vpc/latest/tgw/working-with-flow-logs.html

This parameter must be 60 seconds for transit gateway resource types.

https://docs.aws.amazon.com/ja_jp/AWSCloudFormation/latest/UserGuide/aws-resource-ec2-flowlog.html#cfn-ec2-flowlog-maxaggregationinterval

@aws-cdk-automation aws-cdk-automation removed the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Jan 10, 2024
@go-to-k
Copy link
Contributor

go-to-k commented Jan 10, 2024

And this PR title feat(ec2): flow logs from TransitGateway and TransitGatewayAttachment would be better.

Co-authored-by: k.goto <24818752+go-to-k@users.noreply.github.com>
@badmintoncryer badmintoncryer changed the title feat(ec2): Creation of flow logs from TransitGateway and TransitGatewayAttachment feat(ec2): flow logs from TransitGateway and TransitGatewayAttachment Jan 10, 2024
@badmintoncryer
Copy link
Contributor Author

@go-to-k
Thank you for your review! I have incorporated your suggested modifications.

I've implemented validation of maxAggregationInterval.
The default value for this parameter is 10 minutes, which should result in an error when creating a Transit Gateway flow log. However, in practice, CFN allows deployment when maxAggregationInterval is set to undefined. It's likely that, internally, the default value is adjusted to 1 minute for Transit Gateway targets.

Therefore, when the target is TransitGateway, we allow maxAggregationInterval to be either ONE_MINUTE or undefined, and only consider it an error if TEN_MINUTES is explicitly specified."

Copy link
Contributor

@go-to-k go-to-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I've made a few more adjustment comments.

packages/aws-cdk-lib/aws-ec2/lib/vpc-flow-logs.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ec2/lib/vpc-flow-logs.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ec2/lib/vpc-flow-logs.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ec2/test/vpc-flow-logs.test.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ec2/test/vpc-flow-logs.test.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ec2/test/vpc-flow-logs.test.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ec2/test/vpc-flow-logs.test.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ec2/test/vpc-flow-logs.test.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ec2/test/vpc-flow-logs.test.ts Outdated Show resolved Hide resolved
badmintoncryer and others added 3 commits January 11, 2024 23:52
Co-authored-by: k.goto <24818752+go-to-k@users.noreply.github.com>
Co-authored-by: k.goto <24818752+go-to-k@users.noreply.github.com>
Co-authored-by: k.goto <24818752+go-to-k@users.noreply.github.com>
badmintoncryer and others added 7 commits January 12, 2024 00:03
Co-authored-by: k.goto <24818752+go-to-k@users.noreply.github.com>
Co-authored-by: k.goto <24818752+go-to-k@users.noreply.github.com>
Co-authored-by: k.goto <24818752+go-to-k@users.noreply.github.com>
Co-authored-by: k.goto <24818752+go-to-k@users.noreply.github.com>
Co-authored-by: k.goto <24818752+go-to-k@users.noreply.github.com>
Co-authored-by: k.goto <24818752+go-to-k@users.noreply.github.com>
Copy link
Contributor

@go-to-k go-to-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jan 14, 2024
Copy link
Contributor

@paulhcsun paulhcsun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work @badmintoncryer, looks good overall, just a few minor comments to address.

packages/aws-cdk-lib/aws-ec2/lib/vpc-flow-logs.ts Outdated Show resolved Hide resolved
packages/aws-cdk-lib/aws-ec2/lib/vpc-flow-logs.ts Outdated Show resolved Hide resolved
@@ -800,6 +829,17 @@ export class FlowLog extends FlowLogBase {
}).join(' ');
}

let trafficType: FlowLogTrafficType | undefined = props.trafficType ?? FlowLogTrafficType.ALL;
if (props.resourceType.resourceType === 'TransitGateway' || props.resourceType.resourceType === 'TransitGatewayAttachment') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you create constants or enums for the resourceType?

Copy link
Contributor Author

@badmintoncryer badmintoncryer Jan 20, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@paulhcsun
I agree with creating an enum!
However, in the existing code, a resourceType defined as a string is already being used. Along with defining the enum, is it okay to also modify the FlowLogResourceType's ResourceType variable and the methods fromSubnet(), fromVpc(), and fromNetworkInterfaceId()?

export enum ResourceType {
  TRANSIT_GATEWAY = 'transitGateway',
  ...
}

export abstract class FlowLogResourceType {
  public static fromSubnet(subnet: ISubnet): FlowLogResourceType {
    return {
      resourceType: ResourceType.SUBNET,
      resourceId: subnet.subnetId,
    };
  }
...
  /**
   * The type of resource to attach a flow log to.
   */
  public abstract resourceType: ResourceType;
...
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah I see. I believe it should be safe to make that change for FlowLogResourceType's ResourceType as I see something similar being done with the FlowLogDestinationType enum in FlowLogDestination but let me double check that it won't create any backwards compatibility issues and get back to you.

I'm okay with approving this PR as other parts of the code are defining resourceType with a string. Updating resourceType to an enum can be addressed in a separate PR if you're interested, otherwise I can make that change once I've confirmed it's safe.

/**
 * The available destination types for Flow Logs
 */
export enum FlowLogDestinationType {
  /**
   * Send flow logs to CloudWatch Logs Group
   */
  CLOUD_WATCH_LOGS = 'cloud-watch-logs',
  ...
}


export abstract class FlowLogDestination {
  /**
   * Use CloudWatch logs as the destination
   */
  public static toCloudWatchLogs(logGroup?: logs.ILogGroup, iamRole?: iam.IRole): FlowLogDestination {
    return new CloudWatchLogsDestination({
      logDestinationType: FlowLogDestinationType.CLOUD_WATCH_LOGS,
      ...
    });
  }
...
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@paulhcsun I see! I may be create PR for that later.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@badmintoncryer I don't think doing so would be safe so I would hold off on that for now

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kaizencc Understood. Thank you for the additional information.

packages/aws-cdk-lib/aws-ec2/test/vpc-flow-logs.test.ts Outdated Show resolved Hide resolved
@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jan 19, 2024
Co-authored-by: paulhcsun <47882901+paulhcsun@users.noreply.github.com>
@mergify mergify bot dismissed paulhcsun’s stale review January 20, 2024 03:55

Pull request has been modified.

Co-authored-by: paulhcsun <47882901+paulhcsun@users.noreply.github.com>
@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jan 20, 2024
@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 3a14a7e
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

Copy link
Contributor

@paulhcsun paulhcsun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work @badmintoncryer!

And thanks for the review @go-to-k!

Copy link
Contributor

mergify bot commented Jan 22, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@mergify mergify bot merged commit a238590 into aws:main Jan 22, 2024
10 checks passed
@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jan 22, 2024
@badmintoncryer badmintoncryer deleted the 27222-flowLog branch January 23, 2024 09:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2 valued-contributor [Pilot] contributed between 6-12 PRs to the CDK
Projects
None yet
Development

Successfully merging this pull request may close these issues.

aws-ec2: flow logs for TransitGateway & TransitGatewayAttachment
5 participants