Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for customizing path for job mount file system #368

Merged
merged 5 commits into from
Oct 12, 2023

Conversation

lu-ohai
Copy link
Member

@lu-ohai lu-ohai commented Oct 9, 2023

Added support for customizing path for job mount file system

  • Previously all file systems will be mounted under /mnt and now we added support for customizing the destination path in the format: "dest" : "<destination_path>/<destination_directory_name>"
    ** dir - "fss" & path - "/opc" : mount happens under "/opc/fss"
    ** dir - "fss" & path - "/" : mount happens under "/fss"
    ** dir - "fss" & path - omitted : mount happens under "/mnt/fss" (for backward compatibility of current users)

User experience

infrastructure = (
    DataScienceJob()
    .with_log_group_id("<log_group_ocid>")
    .with_log_id("<log_ocid>")
    .with_storage_mount(
      {
        "src" : "<mount_target_ip_address>@<export_path>",
        "dest" : "/test_path/test_directory_name"
      }
    )
)

The file system will be mounted to the test_directory_name folder under /test_path.

infrastructure = (
    DataScienceJob()
    .with_log_group_id("<log_group_ocid>")
    .with_log_id("<log_ocid>")
    .with_storage_mount(
      {
        "src" : "<mount_target_ip_address>@<export_path>",
        "dest" : "/test_directory_name"
      }
    )
)

The file system will be mounted to the test_directory_name folder under /.

infrastructure = (
    DataScienceJob()
    .with_log_group_id("<log_group_ocid>")
    .with_log_id("<log_ocid>")
    .with_storage_mount(
      {
        "src" : "<mount_target_ip_address>@<export_path>",
        "dest" : "test_directory_name"
      }
    )
)

The file system will be mounted to the test_directory_name folder under /mnt.

@oracle-contributor-agreement oracle-contributor-agreement bot added the OCA Verified All contributors have signed the Oracle Contributor Agreement. label Oct 9, 2023
@github-actions
Copy link

github-actions bot commented Oct 9, 2023

📌 Cov diff with main:

No success to gather report. 😿

📌 Overall coverage:

No success to gather report. 😿

mrDzurb
mrDzurb previously approved these changes Oct 10, 2023
@@ -47,6 +48,25 @@ def update_from_dsc_model(cls, dsc_model) -> dict:
"""
pass

@staticmethod
def get_destination_path_and_name(dest: str) -> (str, str):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this method can be simplified a bit

def get_destination_path_and_name(dest: str) -> (str, str):
    return (os.path.dirname(dest.rstrip("/")), os.path.basename(dest.rstrip("/")))

image

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mrDzurb Thanks, this makes it easier! Just updated the code

@lu-ohai lu-ohai marked this pull request as ready for review October 10, 2023 17:32
@lu-ohai lu-ohai requested a review from qiuosier October 10, 2023 17:32
@github-actions
Copy link

📌 Cov diff with main:

No success to gather report. 😿

📌 Overall coverage:

No success to gather report. 😿

@github-actions
Copy link

📌 Cov diff with main:

No success to gather report. 😿

📌 Overall coverage:

No success to gather report. 😿

@github-actions
Copy link

📌 Cov diff with main:

No success to gather report. 😿

📌 Overall coverage:

No success to gather report. 😿

@lu-ohai
Copy link
Member Author

lu-ohai commented Oct 11, 2023

As the OCI job mount file systems hasn't GA yet so the tests will be skipped in github workflow. Below is the local unit test:
image

…ence into ODSC-47899/allow_customize_destination_folder_job_mount
@github-actions
Copy link

📌 Cov diff with main:

Coverage-30%

📌 Overall coverage:

Coverage-71.17%

@lu-ohai lu-ohai merged commit a4a3d9a into main Oct 12, 2023
13 of 14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
new feature OCA Verified All contributors have signed the Oracle Contributor Agreement.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants