Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(glue): timeout and worker type validation for Ray jobs #32119

Merged
merged 3 commits into from
Nov 20, 2024

Conversation

badmintoncryer
Copy link
Contributor

Issue # (if applicable)

Closes #29612.

Reason for this change

AWS Glue Ray job has some restriction.

  • must use Z.2X worker type
CREATE_FAILED [...] Worker type cannot be null and only [Z.2X] worker types are supported for glueray jobs
  • must not specify timeout
UPDATE_FAILED [...] Timeout not supported for Ray jobs

Description of changes

Add validation for above restriction.

    if (executable.type.name === JobType.RAY.name) {
      if (props.workerType !== WorkerType.Z_2X) {
        throw new Error(`WorkerType must be Z_2X for Ray jobs, got: ${props.workerType}`);
      }
      if (props.timeout !== undefined) {
        throw new Error('Timeout cannot be set for Ray jobs');
      }
    }

Description of how you validated changes

Add unit test.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation aws-cdk-automation requested a review from a team November 14, 2024 00:36
@github-actions github-actions bot added bug This issue is a bug. effort/medium Medium work item – several days of effort p2 distinguished-contributor [Pilot] contributed 50+ PRs to the CDK labels Nov 14, 2024
@badmintoncryer badmintoncryer marked this pull request as draft November 14, 2024 00:38
@badmintoncryer badmintoncryer changed the title chore(glue): add validation for ray job chore(glue): timeout and worker type validation for ray job Nov 14, 2024
Copy link

codecov bot commented Nov 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 77.17%. Comparing base (c768554) to head (52be7e4).
Report is 1 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main   #32119   +/-   ##
=======================================
  Coverage   77.17%   77.17%           
=======================================
  Files         105      105           
  Lines        7169     7169           
  Branches     1315     1315           
=======================================
  Hits         5533     5533           
  Misses       1455     1455           
  Partials      181      181           
Flag Coverage Δ
suite.unit 77.17% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
packages/aws-cdk 77.17% <ø> (ø)
---- 🚨 Try these New Features:

@badmintoncryer badmintoncryer marked this pull request as ready for review November 14, 2024 06:43
@badmintoncryer badmintoncryer changed the title chore(glue): timeout and worker type validation for ray job chore(glue): timeout and worker type validation for Ray jobs Nov 14, 2024
@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Nov 14, 2024
Copy link
Contributor

@mazyu36 mazyu36 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
However, this fix will not be reflected once #30833 is merged.
I'll leave the decision on how to proceed to the maintainers.

@aws-cdk-automation aws-cdk-automation added pr/needs-maintainer-review This PR needs a review from a Core Team Member and removed pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. labels Nov 16, 2024
@badmintoncryer
Copy link
Contributor Author

@mazyu36 Thanks for the information. I don't know this PR. I'll wait for the maintainer's opinion.

Copy link
Contributor

mergify bot commented Nov 20, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Nov 20, 2024
Copy link
Contributor

mergify bot commented Nov 20, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: 52be7e4
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit be33610 into aws:main Nov 20, 2024
17 checks passed
Copy link
Contributor

mergify bot commented Nov 20, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

Copy link

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Nov 20, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug This issue is a bug. distinguished-contributor [Pilot] contributed 50+ PRs to the CDK effort/medium Medium work item – several days of effort p2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

(glue): Timeout and workerType missing validation for Ray job types
4 participants