-
Notifications
You must be signed in to change notification settings - Fork 4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: implement code generated handler framework #28251
Changes from 65 commits
b983212
7145c7a
dd2ff42
f7b4593
2038bcf
d888a5d
c87613b
6de2be2
5873fe4
dccc985
688a69d
c920b1e
c78566a
0f8c8e5
00fc75a
ba9c570
3f450da
ba044b6
43e60d7
8ef2e15
7ef5139
0c45784
1647a82
7da55a3
2210613
9d950d9
e19f090
6712a4e
fd0b93f
bacd7b6
ca3be84
721c59f
f2504ae
98cbe11
aa3c176
c89f470
b0e9ba4
d6d5cbf
ca0ad0a
2ca1f33
e036649
b8da22d
13a0b48
1496b55
972836b
7b87231
c4a20ba
510bcaa
a7b74e1
228aafd
c3fdb61
187c7df
f857264
2db1c20
3c81f54
cc69345
034c7ce
7ba9f86
1db5e8c
c11b42a
3744d65
a64afe4
3a8757a
7a326ca
8582ad6
cb75044
812ac0a
7b3e7c1
cf435a0
4d25941
e8f5a3a
46aa221
ab259ed
11f9476
f77cacf
dedc574
2dffd0a
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,87 @@ | ||
# AWS CDK Custom Resource Framework | ||
|
||
The CDK custom resource framework is an internal framework developed to establish best practices, runtime enforcement, and consistency for custom resource providers and their associated source code. | ||
|
||
## Custom Resource Framework Concepts | ||
|
||
Custom resource providers can be created in one of three forms - `CdkFunction`, `CdkSingletonFunction`, or `CdkCustomResourceProvider`. These three custom resource provider formats are a code generated wrapper around the lambda `Function`, lambda `SingletonFunction`, and `CustomResourceProvider` constructs, respectively. These new CDK prefixed constructs will offer the same behavior and initialization options as their parent minus the ability to configure the following properties during construction: | ||
- `handler` - the name of the method within your code that the provider calls to execute your function | ||
- `code` - the source code of your provider | ||
- `codeDirectory` - a local file system directory with the provider's code | ||
- `runtime` - the runtime environment for the provider that you are uploading | ||
|
||
Instead, these properties will be automatically generated using the `ProviderProps` configured in the [config](./config.ts) file: | ||
- `type`: the custom resource provider type to generate | ||
- `sourceCode`: the source code that will be executed by the provider | ||
- `compatibleRuntimes`: runtimes that are compatible with the provider's source code | ||
- `handler`: the name of the method within your code that the provider calls to execute your function | ||
|
||
The [config](./config.ts) file is structured with the top most level mapping to a `aws-cdk-lib` module, i.e., `aws-s3`, `aws-dynamodb`, etc. Each service can contain one or more provider modules. Provider modules are containers for custom resource providers and will be rendered as a code generated file. Each provider module can contain one or more `ProviderProps` objects. Each `ProviderProps` object contains all the necessary information required to generate a single custom resource provider. The following example shows a more structural breakdown of how the [config](./config.ts) file is configured: | ||
|
||
```ts | ||
const config = { | ||
'aws-s3': { // the aws-cdk-lib module | ||
'replica-provider': [ // the custom resource provider module | ||
// custom resource provider defined as ProviderProps object | ||
{ | ||
// the custom resource provider type - `CdkFunction` | ||
type: ComponentType.CDK_FUNCTION, | ||
// the source code that the provider will execute | ||
sourceCode: path.resolve(__dirname, '..', 'aws-dynamodb', 'replica-handler', 'index.ts'), | ||
// runtimes that are compatible with the source code | ||
compatibleRuntimes: [Runtime.NODEJS_18_X], | ||
// the handler in the source code that the provider will execute | ||
handler: 'index.onEventHandler', | ||
}, | ||
], | ||
}, | ||
'aws-stepfunctions-tasks': { | ||
// contains multiple custom resource provider modules | ||
'eval-nodejs-provider': [ | ||
{ | ||
// the custom resource provider type - `CdkSingletonFunction` | ||
type: ProviderType.CDK_SINGLETON_FUNCTION, | ||
sourceCode: path.resolve(__dirname, '..', 'aws-stepfunctions-tasks', 'eval-nodejs-handler', 'index.ts'), | ||
compatibleRuntimes: [Runtime.NODEJS_18_X], | ||
}, | ||
], | ||
'role-policy-provider': [ | ||
{ | ||
type: ProviderType.CDK_SINGLETON_FUNCTION, | ||
sourceCode: path.resolve(__dirname, '..', 'aws-stepfunctions-tasks', 'role-policy-handler', 'index.py'), | ||
compatibleRuntimes: [Runtime.PYTHON_3_9], | ||
// prevent minify and bundle flag is set since the source code is a python file | ||
preventMinifyAndBundle: true, | ||
}, | ||
], | ||
}, | ||
}; | ||
``` | ||
|
||
Code generation for the provider modules is triggered when this package - `@aws-cdk/custom-resource-handlers` - is built. Importantly, this framework is also responsible for minifying and bundling the custom resource providers' source code and dependencies. A flag named `preventMinifyAndBundle` can be configured as part of the `ProviderProps` to prevent minifying and bundling the source code for a specific provider. This flag is `false` by default and is only needed for Python files or for JavaScript/TypeScript files containing require imports. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. nit: Swap this around to be the positive: |
||
|
||
Once built, all generated code and bundled source code will be written to `@aws-cdk/custom-resource-handlers/dist`. The top level field in the [config](./config.ts) file defining individual `aws-cdk-lib` modules will be used to create specific directories within `@aws-cdk/custom-resource-handlers/dist` and each provider module will be a separate code generated file within these directories named as `<provider-module>.generated.ts`. As an example, the sample [config](./config.ts) file above would create the following file structure: | ||
|
||
|--- @aws-cdk | ||
| |--- custom-resource-handlers | ||
| | |--- dist | ||
| | | |--- aws-s3 | ||
| | | | |--- replica-handler | ||
| | | | | |--- index.js | ||
| | | | |--- replica-provider.generated.ts | ||
| | | |--- aws-stepfunctions-tasks | ||
| | | | |--- eval-nodejs-handler | ||
| | | | | |--- index.js | ||
| | | | |--- role-policy-handler | ||
| | | | | |--- index.py | ||
| | | | |--- eval-nodejs-provider.generated.ts | ||
| | | | |--- role-policy-provider.generated.ts | ||
|
||
The code generated custom resource providers are usable in `aws-cdk-lib` once `aws-cdk-lib` is built. The custom resource providers will be consumable from `aws-cdk-lib/custom-resource-handlers/dist` and the file structure therein will match what was generated in `@aws-cdk/custom-resource-handlers/dist` except for providers defined in `core`. To prevent circular dependencies, all custom resource providers defined in `core`and any associated source code will be consumable from `aws-cdk-lib/core/dist/core`. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
|
||
## Creating a Custom Resource Provider | ||
|
||
Creating a new custom resource provider involves three steps: | ||
1. Add the custom resource provider's source code to `@aws-cdk/custom-resource-handlers/lib/<aws-cdk-lib-module>` | ||
2. Update the [config](./config.ts) file with the custom resource provider to generate by specifying all required `ProviderProps`. | ||
3. At this point you can directly build `@aws-cdk/custom-resource-handlers` with `yarn build` to view the generated provider in `@aws-cdk/custom-resource-handlers/dist`. Alternatively, you can build `aws-cdk-lib` with `npx lerna run build --scope=aws-cdk-lib --skip-nx-cache` to make the generated provider available for use within `aws-cdk-lib` |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This framing around
CdkFunction
,CdkSingletonFunction
etc is confusing me a little.As far as I can tell, those exact classes are gone (which is great!) so now these are just concepts. That's fine, but as a concept a
CdkFunction
doesn't really mean anything to me.Can we not give these names after what you would use them for:
CdkFunction
->BuiltinLambdaFunction
(because I think these are never used as custom resource providers?)SingletonFunction
-> actually confusing to me. Are these usually/always used for custom resource providers? Or no? Are they also built-in Lambda functions that are not actually used as custom resource providers? If true, the naming of this section is confusing, because you're heavily stating it's all about custom resources everywhere, but it might not actually be? Maybe it's just non-Custom Resource Lambdas we provision for you?CdkCustomResourceProvider
-> this should make a distinction between thecore.CustomResourceProvider
and thecustom-resources.CustomResourceProvider
. Maybe name them something likeCoreCustomResource
andLongStabilizationCustomResource
?Maybe the flavors of Lambdas we generate here are:
RuntimeLambda({ singleton: boolean })
CoreCustomResource
LongStabilizationCustomResource({ onEvent, isComplete })
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(I'm a little worried we're only adding more layers of indirection, and it looks like we're only adding complexity instead of taking it away...)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(If
CdkFunction
,CdkSingletonFunction
etc are still in from the previous commit, then I think we can take them out again, and we can just codegen their implementations)