Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cesm2_albedo artifact #48

Merged
merged 5 commits into from
Oct 7, 2024
Merged

Add cesm2_albedo artifact #48

merged 5 commits into from
Oct 7, 2024

Conversation

imreddyTeja
Copy link
Contributor

@imreddyTeja imreddyTeja commented Oct 4, 2024

See related ClimaLand PR here

Originally, ClimaLand was using esw_albedo_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412_v2.nc, which is now replaced with sw_albedo_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412_v2_no-nans.nc, which is exactly the same.

There was a python script we had that created a similar file from different datasets. The script calculate_sw_alb.jl is based on this. It has a some manual assignment of variable attributes to make the output file the exact same as the existing
sw_albedo_Amon_CESM2_historical_r1i1p1f1_gn_185001-201412_v2_no-nans.nc, for which we do not have a generating script. The file the script generates in nearly identical to the original, except some of the variable attributes are saved in different orders, and some of the overall file attributes are different. This is possibly because this script uses Julia's NCDatsets, while the other script uses python's xarray. This should have no impact on ClimaLand's use of the file.

We had an existing Julia script to create bareground_albedo.nc. Slight modification was required to make it run without error. In particular
ds2.attrib = ds.attrib was removed, and
sw_alb = ds["sw_alb"][:] was changed to sw_alb = Array(ds["sw_alb"])

The resulting file is still exactly the same as the original. File equivalence was checked using ncompare and with a for loop, which checks each variable's data, dimensions, and attributes for equivalence. The overall file attributes were also compared.
Checklist:

  • I created a new folder $artifact_name
    • I added a README.md in that that folder that
      • describes the data and processing done to it
      • lists the sources of the raw data
      • lists the required citation, licenses
    • If applicable (e.g., for Creative Commons), I added a LICENSE file
    • I added the scripts that retrieve, process, and produce the artifact
    • I added the environment used for such scripts (typically, Project.toml
      and Manifest.toml)
    • I added the OutputArtifacts.toml file containing the information
      needed for package developers to add $artifact_name to their package
  • I uploaded the artifact folder to the Caltech cluster (in
    /groups/esm/ClimaArtifacts/artifacts/$artifact_name)
  • I added the relevant code to the Overides.toml on the Caltech Cluster
    (in /groups/esm/ClimaArtifacts/artifacts/Overrides.toml)
  • I added a link to the main README.md to point to the new artifact

cesm2_albedo/README Outdated Show resolved Hide resolved
cesm2_albedo/README Outdated Show resolved Hide resolved
@imreddyTeja imreddyTeja marked this pull request as ready for review October 4, 2024 23:41
@imreddyTeja imreddyTeja marked this pull request as draft October 4, 2024 23:41
Copy link
Member

@juliasloan25 juliasloan25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks great, thank you Teja! After adding the CESM license and updating some typos/outdated text this will be good to merge

README.md Outdated Show resolved Hide resolved
cesm2_albedo/README Outdated Show resolved Hide resolved
cesm2_albedo/README Outdated Show resolved Hide resolved
cesm2_albedo/README Outdated Show resolved Hide resolved
cesm2_albedo/README Outdated Show resolved Hide resolved
cesm2_albedo/README Outdated Show resolved Hide resolved
Dates.DateTime.(
reinterpret.(Ref(NCDatasets.DateTimeStandard), ds["time"][:])
)
latitude = ds["lat"][:]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm surprised we can still use this syntax - I thought it got replaced with Array(ds["lat"]), but maybe that's for variables not dimensions? No need to change it, just something I noticed

cesm2_albedo/calculate_sw_alb.jl Show resolved Hide resolved
cesm2_albedo/calculate_sw_alb.jl Outdated Show resolved Hide resolved
@imreddyTeja imreddyTeja marked this pull request as ready for review October 7, 2024 21:39
Copy link
Member

@juliasloan25 juliasloan25 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to squash and merge! Thank you Teja :)

@imreddyTeja imreddyTeja merged commit 380bc76 into main Oct 7, 2024
@imreddyTeja imreddyTeja deleted the tr/add-cesm_albedo branch October 7, 2024 22:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants