GSI changes to read and assimilate IASI-NG #805
Conversation
…on. In this case it is METImage. Specifically, the CRTM coefficent files are needed by CADS. The CRTM coefficient files used are determined by the satinfo file. The METImage entry in the satinfo file generates error messages in various parts of the GSI. These changes silence the error messages. At this time, there is no METImage assimilation code in the GSI.
Conflicts: src/gsi/gsimod.F90 src/gsi/mrmsmod.f90 src/gsi/read_cris.f90 src/gsi/read_iasi.f90 src/gsi/read_obs.F90 src/gsi/statsrad.f90
Merge branch 'IASI-NG' of https://github.com/wx20jjung/GSI into IASI-NG
…n angle calculation.
Merge remote-tracking branch 'emc/develop' into IASI-NG
|
@ADCollard , @DavidHuber-NOAA , @InnocentSouopgui-NOAA would you be willing to review these changes? |
|
Thank you @DavidHuber-NOAA for reviewing this PR. @wx20jjung , who is the second peer reviewer we should assign to this PR? |
Co-authored-by: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com>
|
@russ Treadon - NOAA Federal ***@***.***>
I asked @andrew Collard - NOAA Federal ***@***.***> and
@Innocent
Souopgui - NOAA Affiliate ***@***.***> as well.
…On Tue, Nov 5, 2024 at 9:36 AM RussTreadon-NOAA ***@***.***> wrote:
Thank you @DavidHuber-NOAA <https://github.com/DavidHuber-NOAA> for
reviewing this PR.
@wx20jjung <https://github.com/wx20jjung> , who is the second peer
reviewer we should assign to this PR?
—
Reply to this email directly, view it on GitHub
<#805 (comment)>, or
unsubscribe
<https://github.com/notifications/unsubscribe-auth/AMPASAY4BXPUL3S5YNBKCYLZ7DJXNAVCNFSM6AAAAABRFRPISKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINJXGM2DINRUG4>
.
You are receiving this because you were mentioned.Message ID:
***@***.***>
|
Yes, thanks Co-authored-by: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com>
… code that is no longer used.
|
I have pushed @DavidHuber-NOAA changes to github. I am not able to test them until Hera is returned to users tonight. |
@RussTreadon-NOAA These are the changes I made:
|
|
@RussTreadon-NOAA Jet changes were pushed to github and has passed internal checks. |
There was a problem hiding this comment.
Approve. Thank you @wx20jjung for updating the Jet job configuration for the rrfs ctest.
|
This PR is awaiting the return of WCOSS2 to developers so WCOSS2 ctests can be run. Assuming acceptable WCOSS2 results, this PR can be merged into |
|
WCOSS2 ctests Unfortunately each of the above failures is due to the fact that the updat and contrl analyses differ. The initial penalties are identical. Differences arise in the minimization. It's odd that this behavior is only observed on WCOSS2. ctests pass on other platforms. The WCOSS2 GSI build uses |
|
Create stand-alone script to run global_4denvar on Cactus. Simplify configuration to 3dvar with no FGAT & no constraints. Only assimilate CrIS and IASI. Non-reproducible results between cntrl ( @wx20jjung , I see that you have a WCOSS2 account. Can you log onto Cactus and investigate? The stand-alone script I have is |
|
global_4denvar debugging on Cactus Recompile Does optimization alter the instruction order between the two source codes - |
|
Build |
|
Rebuild |
|
I do not have access to restricted data on wcoss yet. Removing only the restricted data from both develop and update generated the following results: 100% tests passed, 0 tests failed out of 6 Total Test time (real) = 1443.27 sec I can't do further tests until I am added to the rstprod group. |
|
As shown above, comment out these dump files from The initial step size on the first iteration of the first loop differ between the updat and contrl. The same behavior is observed when all dump files are processed. |
|
@wx20jjung , your passed case does not assimilate microwave radiances. I repeat the no_rstprod test with the link for microwave radiance dump files also commented out in |
|
Modify Install Repeat this test using Comparison of the Since |
|
Odd WCOSS2 (Dogwood) behavior The WCOSS2 Ticket#2024112010000039 has been opened reporting this behavior. |
|
WCOSS test using spack-stack/1.6.0` Recompile Rerun all 6 ctests with following results The environment for |
|
@RussTreadon-NOAA It's worth noting that by making the changes that you did for spack-stack, the majority of the GDIT-installed libraries are still being used to build and run the GSI. The main differences in libraries are PrgEnv 8.1.0 -> 8.3.3 I wonder if just incrementing PrgEnv, cray-mpich, craype, and prod_util would have the same result. |
|
WCOSS2 update Another data point regarding WCOSS2 behavior for this PR. If In this test only the source for crtm was changed. All other modules remain untouched. Changing from the production to spack-stack The number of assimilated avhrr data differs between the two crtms. These differences are a bit surprising since both the production and spack-stack crtm are supposedly version |
|
Thanks for that data point, Russ. I will forward that on to Alex and Wei. |
|
updat and contrl results on WCOSS2 are reproducible as is for some developers. For other developers results are reproducible if they build with |
Before opening a PR, please note these guidelines:
-->
Description
Metop-sg-a1 will be launched next year. There are several new instruments on this satellite. This pull request addresses some of the changes necessary to read and use IASI-NG data in the GSI and ultimately the global-workflow.
This code adds the read routine for IASI-NG (read_iasing.f90) and adds logic throughout the GSI to assimilate these data. The code is setup to use data from the standard operational feed and both direct broadcast feeds. The cloud and aerosol detection software (CADS) is also setup and can be turned on/off with a flag, similar to the current IASI and CrIS instruments.
There should be no dependencies needed to incorporate these changes. There are several dependencies to be able to use the IASI-NG data including; connecting to the data, various CRTM coefficient files, and choosing to use CADS.
This pull request is addressing issue #804
Type of change
How Has This Been Tested?
I have tested these changes on about 6 hours of proxy IASI-NG data using a modified version of the global-workflow, including the CRTM with IASI-NG coefficient files. I also modified the satinfo and scaninfo files to monitor up to 10,000 channels as well as a potential 500 channel subset. This was conducted on S4 and Hera. These changes do not affect the analysis when the IASI-NG data are not available.
The ctests were conducted on Hera with reproducible results.
The Hera tests:
The global_enkf failed due to time limits on the first try. It passed on the second try.
[Jim.Jung@hfe10 build]$ ctest -j 6
Test project /scratch1/NCEPDEV/jcsda/Jim.Jung/save/ctests/update/build
Start 1: global_4denvar
Start 2: rtma
Start 3: rrfs_3denvar_rdasens
Start 4: hafs_4denvar_glbens
Start 5: hafs_3denvar_hybens
Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens ............. Passed 1526.62 sec
2/6 Test #5: hafs_3denvar_hybens .............. Passed 2256.26 sec
3/6 Test #4: hafs_4denvar_glbens .............. Passed 2323.86 sec
4/6 Test #2: rtma ............................. Passed 2414.80 sec
5/6 Test #6: global_enkf ......................***Failed 2446.27 sec
6/6 Test #1: global_4denvar ................... Passed 2985.02 sec
83% tests passed, 1 tests failed out of 6
Total Test time (real) = 2985.07 sec
The following tests FAILED:
6 - global_enkf (Failed)
Errors while running CTest
Output from these tests are in: /scratch1/NCEPDEV/jcsda/Jim.Jung/save/ctests/update/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.
[Jim.Jung@hfe10 build]$ ctest -R global_enkf
Test project /scratch1/NCEPDEV/jcsda/Jim.Jung/save/ctests/update/build
Start 6: global_enkf
1/1 Test #6: global_enkf ...................... Passed 1234.16 sec
100% tests passed, 0 tests failed out of 1
Total Test time (real) = 1234.18 sec
My Jet time is limited. I will try again when it is returned from maint.
Running the standard ctests should reproduce results.
None
Checklist