Create a new example that will run full ADF package (not from notebook) #141

mnlevy1981 · 2024-10-10T21:35:46Z

This example can be generalized to other outside diagnostic packages as well (e.g. ILAMB). The goal is to have the JupyterBook-generated web site link to the ADF website from the side bar.

All Submissions:

Have you followed the guidelines in our Contributor's Guide (including the pre-commit check)?
Have you checked to ensure there aren't other open Pull Requests for the same update/change? Other pull requests that create the same changes do not exist; however, this is dependent upon generate ADF config file #142

New Feature Submissions:

Does your submission pass tests?
Have you lint your code locally prior to submission?

Changes to Core Features:

Have you added an explanation of what your changes do and why you'd like us to include them?
Have you successfully tested your changes locally?

This notebook should inject a link to the actual ADF output webpage into the jupyter book

cleaned up a couple files that were triggering pre-commit fails

hide input and output for imports, and input for cell that will eventually have plots as output

Notebook was linking to adf_root before updating the value from config.yml; config.yml should set the default to where we expect ADF to write webpage (which is in the same directory tree that CUPiD writes webpage)

TeaganKing · 2024-11-08T22:21:39Z

The last thing we have to do here is to add a flag if ADF exists in build.py (Done week of 11/11)

TeaganKing · 2024-11-12T18:09:04Z

cupid/build.py

@@ -45,6 +46,14 @@ def build(config_path):
    subprocess.run(
        ["jupyter-book", "build", f"{run_dir}/computed_notebooks/{sname}", "--all"],
    )
+    adf_run = control["compute_notebooks"]["atm"]["link_to_ADF"]["parameter_groups"][
+        "none"
+    ]["adf_run"]


Could generalize for all components. Also use .get() to set default run_external_pkg value.

TeaganKing · 2024-11-12T18:21:03Z

We should also add documentation on how to run ADF until #105 is resolved (Done 11/13)

instead of adf_run: True adf_vars: ['...', '...'] Introduced external_tool: tool_name: 'ADF' vars: ['...', '...'] I also moved this out of parameter_group since it's not needed by the link_to_ADF.ipynb notebook (and added plotting_scripts: ['...', '...'] as an extra option to set in config.yml)

TeaganKing · 2024-11-15T23:15:00Z

examples/external_diag_packages/config.yml

-            adf_run: True
-            adf_vars: ['PRECT', 'SST']
+        external_tool:
+          tool_name: 'ADF'


This is great! One thought-- if we eventually want to loop through multiple tools, should the ADF-specific parameters be a subset of the ADF tool?

TeaganKing · 2024-11-18T16:33:39Z

We may want to remove the dependency on CESM (rather than just CUPiD and ADF)-- see note in #142.

ADF will write output to ${CUPiD}/examples/{example}/ADF_output; if external_tool == "ADF" then that directory will be copied into _build/html/ADF

mnlevy1981 · 2024-11-20T00:12:50Z

This runs for me with

$ cd ${CESM_ROOT}/tools/CUPiD/helper_scripts
$ ./generate_adf_config_file.py --cesm-root ${CESM_ROOT} --cupid-example external_diag_packages --adf-template ../externals/ADF/config_amwg_default_plots.yaml --out-file adf_config.yml
$ conda activate cupid-analysis
$ ../externals/ADF/run_adf_diag adf_config.yml # TOOK 75 seconds
$ cd ../examples/external_diag_packages/
$ conda activate cupid-dev
$ cupid-run
$ cupid-build

(assumes cupid-dev was points to ${CESM_ROOT}/tools/CUPiD/cupid`)

TeaganKing

This looks great and runs properly! Here's a screenshot of the output with the link to ADF working smoothly.

mnlevy1981 and others added 7 commits October 10, 2024 15:33

New example, currently identical to key_metrics

ebf6e24

rename key metrics items in config

29a6fa3

add docstring to generate_cupid_config_for_cesm_case.py

30e3cb9

Add link_to_ADF notebook

8f1653b

This notebook should inject a link to the actual ADF output webpage into the jupyter book

Pass pre-commit tests

43814b9

cleaned up a couple files that were triggering pre-commit fails

preliminary inclusion of ADF plots

00a545b

Update link to ADF

11722aa

TeaganKing mentioned this pull request Nov 4, 2024

generate ADF config file #142

Merged

4 tasks

mnlevy1981 and others added 6 commits November 8, 2024 13:35

Remove executed cells from nblibrary dir

3c3a984

hide input for cell generating link

7a92d6e

hide input and output for imports, and input for cell that will eventually have plots as output

Fix adf_root in notebook and config file

83ad515

Notebook was linking to adf_root before updating the value from config.yml; config.yml should set the default to where we expect ADF to write webpage (which is in the same directory tree that CUPiD writes webpage)

links to ADF working now

2655ce8

adf website copied

354be97

update build to copy adf pages

ce10c84

use adf flag

152ce31

TeaganKing reviewed Nov 12, 2024

View reviewed changes

TeaganKing and others added 4 commits November 13, 2024 10:51

update case name and include list of vars for ADF to run

836f4cf

include temporary instructions for running ADF in ipynb

66ac415

Update link_to_ADF.ipynb to execute generate_adf_config helper script

16da053

TeaganKing reviewed Nov 15, 2024

View reviewed changes

TeaganKing mentioned this pull request Nov 18, 2024

Include capabilities for adding ADF comparisons with observations #150

Open

mnlevy1981 added 2 commits November 18, 2024 15:43

Merge branch 'main' into link_ADF_webpage

653bcd8

Updates to account for some schema changes

90ff73a

ADF will write output to ${CUPiD}/examples/{example}/ADF_output; if external_tool == "ADF" then that directory will be copied into _build/html/ADF

Linting fix

5e4e0be

mnlevy1981 marked this pull request as ready for review November 20, 2024 00:13

TeaganKing approved these changes Nov 20, 2024

View reviewed changes

TeaganKing merged commit 33e9705 into NCAR:main Nov 20, 2024
2 checks passed

mnlevy1981 mentioned this pull request Nov 21, 2024

Cleanup PR #1 #153

Merged

5 tasks

mnlevy1981 deleted the link_ADF_webpage branch November 22, 2024 21:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Create a new example that will run full ADF package (not from notebook) #141

Create a new example that will run full ADF package (not from notebook) #141

mnlevy1981 commented Oct 10, 2024 •

edited

Loading

TeaganKing commented Nov 8, 2024 •

edited

Loading

TeaganKing Nov 12, 2024

TeaganKing commented Nov 12, 2024 •

edited

Loading

TeaganKing Nov 15, 2024

TeaganKing commented Nov 18, 2024

mnlevy1981 commented Nov 20, 2024

TeaganKing left a comment •

edited

Loading

Create a new example that will run full ADF package (not from notebook) #141

Create a new example that will run full ADF package (not from notebook) #141

Conversation

mnlevy1981 commented Oct 10, 2024 • edited Loading

All Submissions:

New Feature Submissions:

Changes to Core Features:

TeaganKing commented Nov 8, 2024 • edited Loading

TeaganKing Nov 12, 2024

Choose a reason for hiding this comment

TeaganKing commented Nov 12, 2024 • edited Loading

TeaganKing Nov 15, 2024

Choose a reason for hiding this comment

TeaganKing commented Nov 18, 2024

mnlevy1981 commented Nov 20, 2024

TeaganKing left a comment • edited Loading

Choose a reason for hiding this comment

mnlevy1981 commented Oct 10, 2024 •

edited

Loading

TeaganKing commented Nov 8, 2024 •

edited

Loading

TeaganKing commented Nov 12, 2024 •

edited

Loading

TeaganKing left a comment •

edited

Loading