CL-807 Adding support for Glue features.

[swhite-oreilly]

This PR adds support for Glue blueprints, ml transforms, sessions, and workflows.

The test script for creating these resources can be found here.

[danarbaugh]

Have you seen these errors during cleanup?

ERRO[0008] Listing AWS::Glue::SchemaVersionMetadata failed:
    InvalidRequestException: Missing Or Invalid ResourceModel property in AWS::Glue::SchemaVersionMetadata list handler request input. Required property: [SchemaVersionId] 
ERRO[0014] Listing AWS::Glue::SchemaVersion failed:
    InvalidRequestException: Missing Or Invalid ResourceModel property in AWS::Glue::SchemaVersion list handler request input. Required property: [SchemaDefinition, Schema/RegistryName, Schema/SchemaName] or [SchemaDefinition, Schema/SchemaArn] 

I see the following resources found in log output and all eventually succeed in their deletion.

us-east-1 - GlueSession - test-glue-session - [Id: "test-glue-session"] - triggered remove
us-east-1 - GlueBlueprint - testBlueprint - [Name: "testBlueprint"] - triggered remove
us-east-1 - GlueDatabase - my_database - triggered remove
us-east-1 - GlueCrawler - my_crawler - triggered remove
us-east-1 - GlueMLTransform - tfm-163757f7edea63c2c20dbc5b56baaf22b297b7d6 - [Id: "tfm-163757f7edea63c2c20dbc5b56baaf22b297b7d6"] - triggered remove
us-east-1 - GlueJob - notebook-job-cli-633745921f7d5f275250 - triggered remove
us-east-1 - GlueJob - python-job-cli - triggered remove
us-east-1 - GlueTrigger - testTrigger - triggered remove
us-east-1 - GlueConnection - my_kafka_connection - triggered remove
us-east-1 - GlueClassifier - my_json_classifier - triggered remove
us-east-1 - GlueWorkflow - test-workflow - [Name: "test-workflow"] - triggered remove
us-east-1 - AWS::Glue::Registry - arn:aws:glue:us-east-1:012854795394:registry/test-registry - [Arn: "arn:aws:glue:us-east-1:012854795394:registry/test-registry", Identifier: "arn:aws:glue:us-east-1:012854795394:registry/test-registry", Name: "test-registry"] - triggered remove
us-east-1 - AWS::Glue::Registry - arn:aws:glue:us-east-1:012854795394:registry/default-registry - [Arn: "arn:aws:glue:us-east-1:012854795394:registry/default-registry", Identifier: "arn:aws:glue:us-east-1:012854795394:registry/default-registry", Name: "default-registry"] - triggered remove
us-east-1 - AWS::Glue::Schema - arn:aws:glue:us-east-1:012854795394:schema/default-registry/testSchemaJson - [Arn: "arn:aws:glue:us-east-1:012854795394:schema/default-registry/testSchemaJson", Identifier: "arn:aws:glue:us-east-1:012854795394:schema/default-registry/testSchemaJson", Name: "testSchemaJson"] - triggered remove

[gsoria]

I saw in my tests similar errors to the ones Dan reported above.

[03/18/2024 06:27:00 PM] - [ERROR] - aws-nuke command ran with some errors.
 Account: 384736907310
 Errors: time="2024-03-18T18:25:00Z" level=error msg="Listing AWS::Glue::SchemaVersion failed:\n    InvalidRequestException: Missing Or Invalid ResourceModel property in AWS::Glue::SchemaVersion list handler request input. Required property: [SchemaDefinition, Schema/RegistryName, Schema/SchemaName] or [SchemaDefinition, Schema/SchemaArn]"
time="2024-03-18T18:25:13Z" level=error msg="Listing AWS::Glue::SchemaVersionMetadata failed:\n    InvalidRequestException: Missing Or Invalid ResourceModel property in AWS::Glue::SchemaVersionMetadata list handler request input. Required property: [SchemaVersionId]"
time="2024-03-18T18:25:31Z" level=error msg="Listing AWS::Glue::SchemaVersionMetadata failed:\n    InvalidRequestException: Missing Or Invalid ResourceModel property in AWS::Glue::SchemaVersionMetadata list handler request input. Required property: [SchemaVersionId]"
time="2024-03-18T18:25:41Z" level=error msg="Listing AWS::Glue::SchemaVersion failed:\n    InvalidRequestException: Missing Or Invalid ResourceModel property in AWS::Glue::SchemaVersion list handler request input. Required property: [SchemaDefinition, Schema/RegistryName, Schema/SchemaName] or [SchemaDefinition, Schema/SchemaArn]"

[swhite-oreilly]

Have you seen these errors during cleanup?

ERRO[0008] Listing AWS::Glue::SchemaVersionMetadata failed:
    InvalidRequestException: Missing Or Invalid ResourceModel property in AWS::Glue::SchemaVersionMetadata list handler request input. Required property: [SchemaVersionId] 
ERRO[0014] Listing AWS::Glue::SchemaVersion failed:
    InvalidRequestException: Missing Or Invalid ResourceModel property in AWS::Glue::SchemaVersion list handler request input. Required property: [SchemaDefinition, Schema/RegistryName, Schema/SchemaName] or [SchemaDefinition, Schema/SchemaArn] 

I see the following resources found in log output and all eventually succeed in their deletion.

us-east-1 - GlueSession - test-glue-session - [Id: "test-glue-session"] - triggered remove
us-east-1 - GlueBlueprint - testBlueprint - [Name: "testBlueprint"] - triggered remove
us-east-1 - GlueDatabase - my_database - triggered remove
us-east-1 - GlueCrawler - my_crawler - triggered remove
us-east-1 - GlueMLTransform - tfm-163757f7edea63c2c20dbc5b56baaf22b297b7d6 - [Id: "tfm-163757f7edea63c2c20dbc5b56baaf22b297b7d6"] - triggered remove
us-east-1 - GlueJob - notebook-job-cli-633745921f7d5f275250 - triggered remove
us-east-1 - GlueJob - python-job-cli - triggered remove
us-east-1 - GlueTrigger - testTrigger - triggered remove
us-east-1 - GlueConnection - my_kafka_connection - triggered remove
us-east-1 - GlueClassifier - my_json_classifier - triggered remove
us-east-1 - GlueWorkflow - test-workflow - [Name: "test-workflow"] - triggered remove
us-east-1 - AWS::Glue::Registry - arn:aws:glue:us-east-1:012854795394:registry/test-registry - [Arn: "arn:aws:glue:us-east-1:012854795394:registry/test-registry", Identifier: "arn:aws:glue:us-east-1:012854795394:registry/test-registry", Name: "test-registry"] - triggered remove
us-east-1 - AWS::Glue::Registry - arn:aws:glue:us-east-1:012854795394:registry/default-registry - [Arn: "arn:aws:glue:us-east-1:012854795394:registry/default-registry", Identifier: "arn:aws:glue:us-east-1:012854795394:registry/default-registry", Name: "default-registry"] - triggered remove
us-east-1 - AWS::Glue::Schema - arn:aws:glue:us-east-1:012854795394:schema/default-registry/testSchemaJson - [Arn: "arn:aws:glue:us-east-1:012854795394:schema/default-registry/testSchemaJson", Identifier: "arn:aws:glue:us-east-1:012854795394:schema/default-registry/testSchemaJson", Name: "testSchemaJson"] - triggered remove

@danarbaugh I didn't see this but am going back to see if I can reproduce.

[corybekk]

Hey Sherd, I was able to clean the resources with the aws-nuke additions you made, however I also received the same error's as Dan and Gaby.

[swhite-oreilly]

@danarbaugh, @gsoria, @corybekk I have removed the cloud control schema version references. They aren't needed due to the versions getting deleted when the schema is deleted.