Add prescribed aerosols #2815
Add prescribed aerosols #2815
Conversation
Sbozzolo
force-pushed
the
gb/aerosol
branch
9 times, most recently
from
ce98880 to
ec02b58
Compare
|
Thanks Gabriele! Just two quick comments from reading the description. I don't have a strong preference for any of them though, and we can discuss.
|
|
Fantastic to see this land! I'll have a closer look later. Just one detail now: We'll probably need extrapolation in z at the top and bottom in some cases, so this should be allowed. Linear extrapolation should be relatively safe. |
Sbozzolo
force-pushed
the
gb/aerosol
branch
14 times, most recently
from
0acbc40 to
62a1b60
Compare
|
Code (+ documentation) to create the artifact: https://github.com/CliMA/ClimaArtifacts/tree/main/aerosol2005 |
| @@ -684,7 +684,6 @@ steps: | |||
| command: | |||
| - mkdir -p gpu_aquaplanet_dyamond | |||
| - > | |||
| nsys profile --trace=nvtx,cuda --output=gpu_aquaplanet_dyamond/report | |||
There was a problem hiding this comment.
I removed this, otherwise the job would stall.
|
|
||
| [[aerosol2005.download]] | ||
| sha256 = "952d723b8462439c266dc0df79d0d4e71f774559edb941e03b7979e8ad743f56" | ||
| url = "https://caltech.box.com/shared/static/chmel1vdthfvfac0yl61ayw2jqnjzr2c.gz" |
There was a problem hiding this comment.
I have to learn more about Artifacts and how do they interact with extensions. Will this file be downloaded always, or only if we are running with aerosols?
There was a problem hiding this comment.
If you are using the cluster, this file is already on the cluster, so we won't have to download it.
If you are not using the cluster, this file is downloaded the first time you initialize your package.
I wrote extensive documentation on artifacts here
| @@ -61,6 +63,12 @@ NVTX.@annotate function rrtmgp_model_callback!(integrator) | |||
| (; idealized_insolation, idealized_h2o, idealized_clouds) = p.radiation | |||
| (; ᶠradiation_flux, radiation_model) = p.radiation | |||
|
|
|||
| # If we have prescribed aerosols, we need to update them | |||
| for (key, tv) in pairs(p.tracers.prescribed_aerosol_timevaryinginputs) | |||
There was a problem hiding this comment.
This is very elegant. It's awesome how all the details are hidden in utilities.
At some point (once we have a 2-moment scheme) we might want to put the aerosol loading in a separate place, to ensure it's happening before both the radiation and microphysics. But that will probably not happen in the next couple of months.
There was a problem hiding this comment.
That's easy, we can update them in a different callback. We can also change the rate at which we update them.
| TimeVaryingInput( | ||
| joinpath( | ||
| @clima_artifact("aerosol2005", ClimaComms.context(Y.c)), | ||
| "aero_2005.nc", |
There was a problem hiding this comment.
Would it be useful to add to the docs somewhere what is the expected structure of the aerosol file? We might want to swap the data files we are using now with something different.
This PR leverages the infrastructure for data handling I built for ClimaLand to read aerosol data from file.
I moved the code to ClimaUtilities. I didn't port over the tests yet because they are tied to ClimaLand. Also, we don't want ClimaUtilities to depend on TempestRemap/NCDatasets/Interpolations/XXX, so everything there should be moved to extensions.
Here's what happens:
High level:
New Fields are added to the cache. Every time the radiation callback is called, the values of the Fields is updated using data read from file. All the complexity is hidden away in ClimaUtilities. Users can choose what aerosols to add by specifying their names in the yml file (e.g,
["CB1", "CB2"]). These aerosols have to exist in a predefined and suitably preprocessed file.The input file
The original input file with the aerosol concentrations is defined in$P = P_0 exp(-z/H)$ , where I hardcoded $P_0 = 1e5$ and $H = 7000$ . Next, I resampled the entire file to be defined on the same $z$s (defined equispaced in hydrostatic pressure levels from 10 m to 50 km). This interpolation is linear and for those points for which data is not available (e.g., because they are below mountains), I used "copy" boundary conditions. I repeat this for every time in the original file. At the end, this procudes a new NetCDF file with the aerosol concentration as a function of
lon, lat, lvl, time. First I converted this to pressure coordinates using the formula described in the CF conventions. Then, I assumed a simple hydristatic balance and converted to z withlon, lat, z, time.All of this is described in the
artifacts/aerosolfolder, where I provide a sketch of readme and the code.(as discussed with @szy21)
Regridder
I perform linear interpolation separately across the different dimensions (lon, lat, z). I do this greedly: ie, every MPI process read the entire file to CPU and prepares an interpolation object for all the points (even those that do not belong to that process). This is memory intensive, but it won't be a problem for the aerosols (I think). Then, I move the object to the GPU, and I evaluate it for the points owned by the process, yielding a Field defined on the computational grid. I validated this regridder against TempestRemap and found that differences can be significant (20% relative error for the variable I was looking at). Interpolation is time is a linear interpolation across two snapshots (nearest neighbor interpolation is already implemented too).
Extrapolation conditions for interpotions in lon is periodic, in lat is "copy", in z and in time is "throw an error" (ie, no extrapolation is allowed).
Everything runs and is GPU and MPI compatible.
cc: @tapios @szy21 (@trontrytel too)
Steps to be completed: