add longrun gpu pipeline #2441
Merged
add longrun gpu pipeline #2441
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
szy21
force-pushed
the
zs/ci_longrun_gpu
branch
from
b98f4c9 to
844ebd8
Compare
szy21
commented
Dec 17, 2023
| @@ -6,7 +6,7 @@ z_elem: 63 | |||
| dz_bottom: 30.0 | |||
| dz_top: 3000.0 | |||
| z_max: 55000.0 | |||
| kappa_4: 2.0e16 | |||
There was a problem hiding this comment.
This run is stable on CPU but unstable on GPU. I decreased the hyperdiffusivity and it became stable. I think we are probably just close to the unstable regime.
szy21
force-pushed
the
zs/ci_longrun_gpu
branch
from
844ebd8 to
791f033
Compare
szy21
force-pushed
the
zs/ci_longrun_gpu
branch
from
791f033 to
bf6c69a
Compare
simonbyrne
approved these changes
Dec 18, 2023
|
@szy21 out of curiousity, what are the corresponding GPU times? |
For the same moist held suarez run (h_elem=16, z_elem=63, t_end=300days), it takes 3.5 hours on 1 GPU and 14 hours on 64 CPUs. I think it's pretty good:) |
|
I'll probably extend t_end for the GPU pipeline later. Hope it will be stable! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
Adds a new pipeline for GPU longruns. For now I only moved dycore simulations (dry and moist baroclinic wave, dry and moist held suarez). I'll move the ones with radiation when the performance issue is solved.
The build is here: https://buildkite.com/clima/climaatmos-gpulongruns/builds/7
To-do
Content