Skip to content

Commit

Permalink
Merge pull request #2895 from CliMA/ne/calibrate
Browse files Browse the repository at this point in the history
Integrate calibration pipeline into atmos
  • Loading branch information
nefrathenrici authored Jul 5, 2024
2 parents 3c157b0 + 3199df1 commit fd2fcf2
Show file tree
Hide file tree
Showing 20 changed files with 3,500 additions and 1 deletion.
13 changes: 13 additions & 0 deletions .buildkite/pipeline.yml
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,11 @@ steps:
- "julia --project=perf -e 'using Pkg; Pkg.precompile()'"
- "julia --project=perf -e 'using Pkg; Pkg.status()'"

- echo "--- Instantiate calibration/test"
- "julia --project=calibration/test -e 'using Pkg; Pkg.develop(;path=\".\"); Pkg.instantiate(;verbose=true)'"
- "julia --project=calibration/test -e 'using Pkg; Pkg.precompile()'"
- "julia --project=calibration/test -e 'using Pkg; Pkg.status()'"

- echo "--- Download artifacts"
- "julia --project=examples artifacts/download_artifacts.jl"

Expand Down Expand Up @@ -487,6 +492,14 @@ steps:
agents:
slurm_mem: 20GB

- group: "Calibration"
steps:
- label: "Calibration interface unit tests"
command: julia --project=calibration/test calibration/test/interface.jl
- label: "end to end test"
command: julia --project=calibration/test calibration/test/e2e_test.jl
artifact_paths: "calibration_end_to_end_test/*"

- group: "Diagnostic EDMFX"
steps:

Expand Down
31 changes: 31 additions & 0 deletions .github/workflows/calibration_test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
name: Calibration
on:
push:
tags: '*'
pull_request:
merge_group:

concurrency:
group: ${{ github.workflow }}-${{ github.ref }}
cancel-in-progress: true

# Needed to allow julia-actions/cache to delete old caches that it has created
permissions:
actions: write
contents: read

jobs:
test:
name: ClimaCalibrate E2E Test
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: julia-actions/cache@v2
- uses: julia-actions/julia-buildpkg@v1
- uses: actions/checkout@v4
- uses: julia-actions/setup-julia@v2
with:
version: '1.10'
- run: |
julia --project=calibration/test -e 'using Pkg; Pkg.develop(;path="."); Pkg.instantiate(;verbose=true)'
julia --project=calibration/test calibration/test/e2e_test.jl
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ docs/site/
# environment.
/Manifest.toml
test/Manifest.toml
calibration/test/Manifest.toml

# ignore vscode artifacts
*.vscode
Expand All @@ -43,6 +44,8 @@ test/Manifest.toml
*.hdf5
*.h5
*.g
*.lock
*.loc

# misc
.DS_Store
2 changes: 1 addition & 1 deletion Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ AtmosphericProfilesLibrary = "0.1"
ClimaComms = "0.6.2"
ClimaCore = "0.14.6"
ClimaDiagnostics = "0.2"
ClimaParams = "0.10.4"
ClimaParams = "0.10.8"
ClimaTimeSteppers = "0.7.32"
ClimaUtilities = "0.1.3"
CloudMicrophysics = "0.22"
Expand Down
23 changes: 23 additions & 0 deletions calibration/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# ClimaAtmos Calibration Experiments

This directory contains the model interface (`model_interface.jl`) for ClimaCalibrate and folders (within `experiments/`) for reproducing calibration experiments.

## Current Experiments

### sphere_held_suarez_rhoe_equilmoist

A perfect-model calibration, using ClimaAtmos v0.24.0.

- Configuration: equilmoist, 0-moment microphysics scheme, spherical with Held-Suarez forcing.
- Observational data: zonal and 60-day time average air temperature at 242m altitude. Provided by [ClimaArtifacts](https://github.com/CliMA/ClimaArtifacts/tree/main/atmos_held_suarez_obs).
- Parameter being calibrated: `equator_pole_temperature_gradient_wet`

For more details, see the experiment directory.

To reproduce the results, on the Caltech central cluster run

```sbatch calibration/experiments/sphere_held_suarez_rhoe_equilmoist/pipeline.sbatch```

## New Experiments

To set up your own experiment, please follow the [setup guide](https://clima.github.io/ClimaCalibrate.jl/dev/atmos_setup_guide/).
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
[atmos_held_suarez_obs]
git-tree-sha1 = "d93ff2958e12d6bba6e8343bfa73554390df52e4"

[[atmos_held_suarez_obs.download]]
sha256 = "f5d63df2bab849632bd2fcd5a4e3165465d1898e1a5fbee6e5fe07f5b3e57fc3"
url = "https://caltech.box.com/shared/static/sbn6afsgn2xzxi5n0ffs928otstshjvi.gz"
Loading

0 comments on commit fd2fcf2

Please sign in to comment.