forked from NOAA-EMC/global-workflow
-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Allow building of the ufs-weather-model, WW3 pre/post execs for GFS, …
…GEFS, SFS in the same clone of global-workflow (NOAA-EMC#3098) GFS and GEFS (and now SFS) uses different compile time options for the UFS-weather-model. For the purposes of CI testing, a multi-build pipeline under Jenkins was created by @TerrenceMcGuinness-NOAA. This served well, until now. With the inclusion of SFS, a third variety of the model is being built. Under the multi-build pipeline paradigm, a second (or third) clone and build of the global-workflow is required. This adds cloning and compilation time of the global-workflow. This PR allows compiling the ufs-weather-model in a single clone of the global-workflow. The compiled executables based on the options for GFS, GEFS, and SFS results in a model executable as `gfs_model.x`, `gefs_model.x`, and `sfs_model.x`. The forecast script uses the right executable. This PR also differentiates the WW3 pre/post executables based on `gfs_` or `gefs_`. SFS variants need to be introduced, when `NET=sfs` is added. In the process of updating the WW3 pre/post executable names, it was discovered (via `grep`) the following are not used: - `ww3_prep` - `ww3_outf` - `ww3_ounf` - `ww3_ounp` These need to be confirmed by running the workflow to ensure they are indeed not used at runtime. This PR also updates the `Jenkinsfile` to use the multi-build from the single location. --------- Co-authored-by: Walter Kolczynski - NOAA <Walter.Kolczynski@noaa.gov> Co-authored-by: David Huber <69919478+DavidHuber-NOAA@users.noreply.github.com>
- Loading branch information
3 people
authored
Dec 16, 2024
1 parent
bef3e0b
commit be7626d
Showing
18 changed files
with
359 additions
and
374 deletions.
There are no files selected for viewing
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -18,4 +18,4 @@ arguments: | |
| yaml: {{ HOMEgfs }}/ci/cases/yamls/gefs_defaults_ci.yaml | ||
|
|
||
| skip_ci_on_hosts: | ||
| - wcoss2 | ||
| - None | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -19,4 +19,4 @@ arguments: | |
| icsdir: {{ 'ICSDIR_ROOT' | getenv }}/C96mx100/20240610 | ||
|
|
||
| skip_ci_on_hosts: | ||
| - wcoss2 | ||
| - None | ||
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,132 +1,107 @@ | ||
| #! /usr/bin/env bash | ||
| set -x | ||
|
|
||
| script_dir=$(dirname "${BASH_SOURCE[0]}") | ||
| cd "${script_dir}" || exit 1 | ||
| # shellcheck disable=SC2312 | ||
| _HOMEgfs=$(cd "$(dirname "$(readlink -f -n "${BASH_SOURCE[0]}" )" )/.." && pwd -P) | ||
| cd "${_HOMEgfs}/sorc" || exit 1 | ||
|
|
||
| # Default settings | ||
| APP="S2SWA" | ||
| PDLIB="ON" | ||
|
|
||
| while getopts ":j:a:dvw" option; do | ||
| case "${option}" in | ||
| a) APP="${OPTARG}";; | ||
| d) BUILD_TYPE="Debug";; | ||
| j) BUILD_JOBS="${OPTARG}";; | ||
| v) export BUILD_VERBOSE="YES";; | ||
| w) PDLIB="OFF";; | ||
| :) | ||
| echo "[${BASH_SOURCE[0]}]: ${option} requires an argument" | ||
| usage | ||
| ;; | ||
| *) | ||
| echo "[${BASH_SOURCE[0]}]: Unrecognized option: ${option}" | ||
| usage | ||
| ;; | ||
| d) BUILD_TYPE="Debug" ;; | ||
| j) BUILD_JOBS="${OPTARG}" ;; | ||
| v) export BUILD_VERBOSE="YES" ;; | ||
| w) PDLIB="OFF" ;; | ||
| :) | ||
| echo "[${BASH_SOURCE[0]}]: ${option} requires an argument" | ||
| usage | ||
| ;; | ||
| *) | ||
| echo "[${BASH_SOURCE[0]}]: Unrecognized option: ${option}" | ||
| usage | ||
| ;; | ||
| esac | ||
| done | ||
|
|
||
| # Determine which switch to use | ||
| if [[ "${APP}" == "ATMW" ]]; then | ||
| ww3switch="model/esmf/switch" | ||
| else | ||
| if [[ "${PDLIB}" == "ON" ]]; then | ||
| ww3switch="model/bin/switch_meshcap_pdlib" | ||
| else | ||
| ww3switch="model/bin/switch_meshcap" | ||
| fi | ||
| fi | ||
|
|
||
| # Check final exec folder exists | ||
| if [[ ! -d "../exec" ]]; then | ||
| mkdir ../exec | ||
| fi | ||
|
|
||
| finalexecdir="$( pwd -P )/../exec" | ||
|
|
||
| #Determine machine and load modules | ||
| # Determine machine and load modules | ||
| set +x | ||
| source "${script_dir}/ufs_model.fd/tests/detect_machine.sh" | ||
| source "${script_dir}/ufs_model.fd/tests/module-setup.sh" | ||
| module use "${script_dir}/ufs_model.fd/modulefiles" | ||
| source "${_HOMEgfs}/sorc/ufs_model.fd/tests/detect_machine.sh" | ||
| source "${_HOMEgfs}/sorc/ufs_model.fd/tests/module-setup.sh" | ||
| module use "${_HOMEgfs}/sorc/ufs_model.fd/modulefiles" | ||
| module load "ufs_${MACHINE_ID}.intel" | ||
| set -x | ||
|
|
||
| #Set WW3 directory, switch, prep and post exes | ||
| cd ufs_model.fd/WW3 || exit 1 | ||
| WW3_DIR=$( pwd -P ) | ||
| #Set WW3 directory | ||
| cd "${_HOMEgfs}/sorc/ufs_model.fd/WW3" || exit 1 | ||
| WW3_DIR=$(pwd -P) | ||
| export WW3_DIR | ||
| export SWITCHFILE="${WW3_DIR}/${ww3switch}" | ||
|
|
||
| # Build exes for prep jobs and post jobs: | ||
| prep_exes="ww3_grid ww3_prep ww3_prnc ww3_grid" | ||
| post_exes="ww3_outp ww3_outf ww3_outp ww3_gint ww3_ounf ww3_ounp ww3_grib" | ||
| # Determine which switch to use | ||
| if [[ "${PDLIB}" == "ON" ]]; then | ||
| ww3switch="model/bin/switch_meshcap_pdlib" | ||
| path_build="${WW3_DIR}/build/pdlib_ON" | ||
| path_install="${WW3_DIR}/install/pdlib_ON" | ||
| else | ||
| ww3switch="model/bin/switch_meshcap" | ||
| path_build="${WW3_DIR}/build/pdlib_OFF" | ||
| path_install="${WW3_DIR}/install/pdlib_OFF" | ||
| fi | ||
| export SWITCHFILE="${WW3_DIR}/${ww3switch}" | ||
|
|
||
| #create build directory: | ||
| path_build="${WW3_DIR}/build_SHRD" | ||
| [[ -d "${path_build}" ]] && rm -rf "${path_build}" | ||
| mkdir -p "${path_build}" || exit 1 | ||
| cd "${path_build}" || exit 1 | ||
| echo "Forcing a SHRD build" | ||
|
|
||
| buildswitch="${path_build}/switch" | ||
|
|
||
| cat "${SWITCHFILE}" > "${path_build}/tempswitch" | ||
|
|
||
| sed -e "s/DIST/SHRD/g"\ | ||
| -e "s/OMPG / /g"\ | ||
| -e "s/OMPH / /g"\ | ||
| -e "s/MPIT / /g"\ | ||
| -e "s/MPI / /g"\ | ||
| -e "s/PIO / /g"\ | ||
| -e "s/B4B / /g"\ | ||
| -e "s/PDLIB / /g"\ | ||
| -e "s/SCOTCH / /g"\ | ||
| -e "s/METIS / /g"\ | ||
| -e "s/NOGRB/NCEP2/g"\ | ||
| "${path_build}/tempswitch" > "${path_build}/switch" | ||
| cat "${SWITCHFILE}" >"${path_build}/tempswitch" | ||
|
|
||
| sed -e "s/DIST/SHRD/g" \ | ||
| -e "s/OMPG / /g" \ | ||
| -e "s/OMPH / /g" \ | ||
| -e "s/MPIT / /g" \ | ||
| -e "s/MPI / /g" \ | ||
| -e "s/PIO / /g" \ | ||
| -e "s/B4B / /g" \ | ||
| -e "s/PDLIB / /g" \ | ||
| -e "s/SCOTCH / /g" \ | ||
| -e "s/METIS / /g" \ | ||
| -e "s/NOGRB/NCEP2/g" \ | ||
| "${path_build}/tempswitch" >"${path_build}/switch" | ||
| rm "${path_build}/tempswitch" | ||
|
|
||
| echo "Switch file is ${buildswitch} with switches:" | ||
| cat "${buildswitch}" | ||
|
|
||
| #define cmake build options | ||
| MAKE_OPT="-DCMAKE_INSTALL_PREFIX=install" | ||
| MAKE_OPT="-DCMAKE_INSTALL_PREFIX=${path_install}" | ||
| [[ ${BUILD_TYPE:-"Release"} = "Debug" ]] && MAKE_OPT+=" -DCMAKE_BUILD_TYPE=Debug" | ||
|
|
||
| #Build executables: | ||
| # shellcheck disable=SC2086 | ||
| cmake "${WW3_DIR}" -DSWITCH="${buildswitch}" ${MAKE_OPT} | ||
| rc=$? | ||
| if (( rc != 0 )); then | ||
| if ((rc != 0)); then | ||
| echo "Fatal error in cmake." | ||
| exit "${rc}" | ||
| fi | ||
|
|
||
| make -j "${BUILD_JOBS:-8}" | ||
| rc=$? | ||
| if (( rc != 0 )); then | ||
| if ((rc != 0)); then | ||
| echo "Fatal error in make." | ||
| exit "${rc}" | ||
| fi | ||
|
|
||
| make install | ||
| if (( rc != 0 )); then | ||
| rc=$? | ||
| if ((rc != 0)); then | ||
| echo "Fatal error in make install." | ||
| exit "${rc}" | ||
| fi | ||
|
|
||
| # Copy to top-level exe directory | ||
| for prog in ${prep_exes} ${post_exes}; do | ||
| cp "${path_build}/install/bin/${prog}" "${finalexecdir}/" | ||
| rc=$? | ||
| if (( rc != 0 )); then | ||
| echo "FATAL: Unable to copy ${path_build}/${prog} to ${finalexecdir} (Error code ${rc})" | ||
| exit "${rc}" | ||
| fi | ||
| done | ||
|
|
||
| #clean-up build directory: | ||
| echo "executables are in ${finalexecdir}" | ||
| echo "cleaning up ${path_build}" | ||
| rm -rf "${path_build}" | ||
|
|
||
| exit 0 |
Oops, something went wrong.