Splitting up the archive task #3242
Conversation
…o-NOAA/global-workflow into feature/arch-split
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
AntonMFernando-NOAA
force-pushed
the
feature/arch-split
branch
2 times, most recently
from
3d5f792 to
c283e87
Compare
AntonMFernando-NOAA
force-pushed
the
feature/arch-split
branch
from
06560cc to
cb836f8
Compare
|
|
workflow/rocoto/gfs_tasks.py
Outdated
| 'maxtries': '&MAXTRIES;' | ||
| } | ||
|
|
||
| metatask_dict = {'task_name': f'{self.run}_eamn', |
There was a problem hiding this comment.
There are now two eamn metatasks. One for earc_vrfy and one for earc_tars. Also, I think earc_vrfy doesn't need to be run in groups. It's going to be pretty lightweight (just local copies). My suggestions here are 1. change earc_vrfy to a task, not a metatask, and 2. let earc_tars have the eamn metatask name.
There was a problem hiding this comment.
Ugh, keep eamn in the trash where it belongs. One of these days, I'm doing to rename all those terrible four-letter names.
| else: | ||
| dep_dict = {'type': 'task', 'name': f'{self.run}_arch'} |
There was a problem hiding this comment.
This else should not have been removed. The ${RUN}_arch_vrfy task isn't valid for ensembles.
workflow/rocoto/gfs_tasks.py
Outdated
| else: | ||
| if self.options['do_archtar']: | ||
| dep_dict = {'type': 'task', 'name': f'{self.run}_arch_tars'} | ||
| deps.append(rocoto.add_dependency(dep_dict)) |
There was a problem hiding this comment.
This shouldn't be an else. It needs to be its own if block.
Description
In this PR the archive task will be split into two parts. The first will always run and will just be for the copying of verification data to the
VRFY_ARCandARCDIRdirectories. The second will only run whenHPSSARCHorLOCALARCHis set to YES and will generate and store tarballs in theATARDIR, either onHPSSor locally.Resolves Split up the archive task #3152
Type of change
Change characteristics
How has this been tested?
HeraExample:
Checklist