[ODSC-65517] Support freeform and defined tags for resource creation in Aqua

[VipulMascarenhas]

Description

This PR supports defined and freeform tags as inputs when creating resources in AI Quick Actions. The following have been updated:

• Creating model from verified models
• Registering new models (verified or unverified)
• Creating deployment
• Creating evaluation model, job and job run
• Creating finetuning model, job and job run
• Updated model and deployment handlers to accept freeform_tags and defined_tags, finetuning and evaluation accepts a json dict, so no updated needed.
• Unit tests to support all the changes

Following changes will be a part of subsequent PR:

• Improve deployment parameters (use dataclass)
• Update model and deployment handler to accept dict instead of individual params

Note:
hpo tests were failing since scikit-learn has updated fit_params to params in sklearn.model_selection.cross_validate, and sktime has restricted the scikit-learn version to <1.6.0. We can relax the scikit-learn version once this restriction is removed.

Testing

1. Register Model

aqua model register --model <model_name> --os_path <oci_path> --download_from_hf True --compartment_id $PROJECT_COMPARTMENT_OCID --defined_tags '{"test": {"testType": "aquaRegisterTest"}}' --freeform_tags '{"testFreeform": "aquaRegisterTest"}' 

Model is created with the above freeform and defined tags.

2. Deploy Model

ads aqua deployment create --model_id ocid1.datasciencemodel.oc1.iad.<ocid> --instance_shape VM.GPU.A10.1 --display_name "MD with Tags" --defined_tags '{"test": {"testType": "aquaDeployTest"}}' --freeform_tags '{"testFreeform": "aquaDeployTest"}'

Deployment is created with the above freeform and defined tags. If same tags are present in the model, them they are copied to the deployment but overridden with new values.

3. FineTune Model

ads aqua fine_tuning create --ft_source_id ocid1.datasciencemodel.oc1.iad.<ocid> --ft_name "FT with Tags" --dataset_path <oci_path> --report_path <oci_path>  --ft_parameters '{"batch_size": 1, "sequence_len": 2048, "pad_to_sequence_len": true, "learning_rate": 0.0002, "lora_r": 32, "lora_alpha": 16, "lora_dropout": 0.05, "lora_target_linear": true, "epochs": 1}' --shape_name VM.GPU.A10.1 --replica 1 --validation_set_size 0.1 --log_group_id ocid1.loggroup.oc1.iad.<ocid> --log_id ocid1.log.oc1.iad.<ocid> --experiment_id ocid1.datasciencemodelversionset.oc1.iad.<ocid> --defined_tags '{"test": {"testType": "aquaFTTest"}}' --freeform_tags '{"testAnotherFreeform": "aquaFTTest"}'

MVS, Model, Job and Job Run are created with the above freeform and defined tags.

4. Evaluating Model

ads aqua evaluation create --evaluation_name "Eval with Tags" --evaluation_description "Eval with Tags" --dataset_path <oci_path> --report_path <oci_path> --model_parameters '{"max_tokens": 500, "top_p": 0.99, "top_k": 50, "temperature": 0.7, "frequency_penalty": 0, "presence_penalty": 0}' --shape_name "VM.Standard.E4.Flex" --memory_in_gbs 128 --ocpus 8 --block_storage_size 50 --experiment_name "Eval Tags" --log_group_id ocid1.loggroup.oc1.iad.<ocid> --log_id ocid1.log.oc1.iad.<ocid> --metrics '[{"name": "bertscore", "args": {}}, {"name": "rouge", "args": {}}, {"name": "bleu", "args": {}}]' --evaluation_source_id ocid1.datasciencemodeldeploymentint.oc1.iad.<ocid>  --defined_tags '{"test": {"testType": "aquaEvalTest"}}' --freeform_tags '{"testFreeform": "aquaEvalTest"}'

MVS, Model, Job and Job Run are created with the above freeform and defined tags.

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[mrDzurb]

We also need to add tags to handlers. However It can be done in a separate ticket.

[github-actions]

⚠️ This PR changed pyproject.toml file. ⚠️

• PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
• PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

[github-actions]

⚠️ This PR changed pyproject.toml file. ⚠️

• PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
• PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

[github-actions]

⚠️ This PR changed pyproject.toml file. ⚠️

• PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
• PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

[VipulMascarenhas]

We also need to add tags to handlers. However It can be done in a separate ticket.

@mrDzurb yes, missed that earlier. I've added it in this PR itself since we only have to add them for model/deployment handlers. Eval and FT already should be able to take in that input, we can do the same for model/deployment in subsequent updates.

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[github-actions]

⚠️ This PR changed pyproject.toml file. ⚠️

• PR Creator must update 📃 THIRD_PARTY_LICENSES.txt, if any 📚 library added/removed in pyproject.toml.
• PR Approver must confirm 📃 THIRD_PARTY_LICENSES.txt updated, if any 📚 library added/removed in pyproject.toml.

[github-actions]

📌 Cov diff with main:

📌 Overall coverage:

[darenr]

some comment questions only