@aws-cdk/aws-glue-alpha: pythonShell should not require glueVersion

[urkle]

Describe the bug

When creating a pythongshell job type the GlueVersion should not be required (as it is not used for this mode).

All examples of creating python shell jobs do not specify the GlueVersion at all, just the python version

Regression Issue

• Select this option if this issue appears to be a regression.

Last Known Working CDK Version

No response

Expected Behavior

I should be able to omit the glueVersion parameter to JobExecution.pythonShell.

Current Behavior

It throws an error stating that the glue_version parameter is required.

Reproduction Steps

glue.Job(
self, 'ShellJob',
job_name='myShellJob',
role=glue_role,
default_arguments={
"--additional-python-modules": "awswrangler,fastparquet,google-api-python-client,google-auth-oauthlib,google-auth-httplib2",
},
timeout=Duration.minutes(30),
max_concurrent_runs=1,
executable=glue.JobExecutable.python_shell(
python_version=glue.PythonVersion.THREE_NINE,
script=glue.Code.from_asset(glue_script_path('my-shell-job.py')),
)
)

Possible Solution

make it optional for PythonShellExecutableProps

Additional Information/Context

No response

CDK CLI Version

2.166.0

Framework Version

No response

Node.js Version

18.16.1

OS

macOS 12.7.6

Language

Python

Language Version

3.9

Other information

@aws-cdk/aws-glue-alpha = 2.166.0a0

[khushail]

Hi @urkle , thanks for reaching out. I see that in CDK documentation for Python shell jobs, it clearly mentions that Glue version is required.

Could you please share your insights on why this is not required and if this shared context of CDK Docs makes any sense.

Thanks

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[urkle]

@khushail whenever I create pyhon shell jobs via CF I have to NOT set the glue_version to create the resource, otherwise it refuses to create saying that isn't supported.

And when I look at my jobs in the glue job list, the shell jobs have NO glue version specified at all.

The only value I was able to get to behave when using CDK was to set the glue version to 3.0 (some default somewhere I assume). But as you can see in the screenshot there is no glue version associated with my python shell job.