Skip to content

notes: setting up for PEST on Atlas

Jump to bottom Edit New page
tobey edited this page Aug 4, 2016 · 3 revisions

Problem

We need to install PEST++, and it needs to be compiled using GCC version 5.2 or greater. On Atlas, the GCC installed is ~ v4.4. So we need to install a newer version of GCC and then use it to compile PEST++.

Solution

We will use a tool called EasyBuild to install GCC and also to compile PEST++. The "recipe" for GCC is provided with EasyBuild, and then we will write our own EasyBuild "recipe" (also known as an "easyconfig" file).

Setup

First, log on, use git to update both the ~/dvm-dos-tem repo and ~/pestdemo.

Make sure the model builds and runs in ~/dvm-dos-tem (I can't get scons to work; make worked).

Install EasyBuild

Install EasyBuild using the bootstrapping method: (http://easybuild.readthedocs.io/en/latest/Installation.html#bootstrapping-easybuild)

Download, the script to ~/:

tcarman2@atlas ~ $ wget https://raw.githubusercontent.com/hpcugent/easybuild-framework/develop/easybuild/scripts/bootstrap_eb.py

Follow the install instructions:

tcarman2@atlas ~ $ # pick an installation prefix to install EasyBuild to (change this to your liking)
tcarman2@atlas ~ $ EASYBUILD_PREFIX=$HOME/.local/easybuild
tcarman2@atlas ~ $ # bootstrap EasyBuild
tcarman2@atlas ~ $ python bootstrap_eb.py $EASYBUILD_PREFIX

Load the module:

# update $MODULEPATH, and load the EasyBuild module
module use $EASYBUILD_PREFIX/modules/all
module load EasyBuild

Check that it all works:

tcarman2@atlas ~ $ eb --version
This is EasyBuild 2.8.2 (framework: 2.8.2, easyblocks: 2.8.2) on host atlas.snap.uaf.edu.

NOTE: It might be helpful to add these two lines to your .bashrc so that you don't have to remember to type them each time you start a new shell on atlas:

 EASYBUILD_PREFIX=$HOME/.local/easybuild
 module use $EASYBUILD_PREFIX/modules/all

Install GCC-5.3.0

First look to see that there is a recipe for GCC, version 5.3.0:

tcarman2@atlas ~ $ eb --search gcc | grep --color GCC-5.3.0

The do the dry-run build:

tcarman2@atlas ~ $ eb GCC-5.3.0.eb -D
== temporary log file in case of crash /tmp/eb-CePIeB/easybuild-NW51Jc.log
Dry run: printing build status of easyconfigs and dependencies
CFGS=/home/UA/tcarman2/.local/easybuild/software/EasyBuild/2.8.2/lib/python2.7/site-packages/easybuild_easyconfigs-2.8.2-py2.7.egg/easybuild/easyconfigs
 * [ ] $CFGS/m/M4/M4-1.4.17.eb (module: M4/1.4.17)
 * [ ] $CFGS/g/GCC/GCC-5.3.0.eb (module: GCC/5.3.0)
== Temporary log file(s) /tmp/eb-CePIeB/easybuild-NW51Jc.log* have been removed.
== Temporary directory /tmp/eb-CePIeB has been removed.

The do the real build with the --robot to resolve dependencies:

tcarman2@atlas ~ $ eb GCC-5.3.0.eb --robot
== temporary log file in case of crash /tmp/eb-pqfnF_/easybuild-btsaiw.log
....lots and lots of messages...

== COMPLETED: Installation ended successfully
== Results of the build can be found in the log file(s) /home/UA/tcarman2/.local/easybuild/software/GCC/5.3.0/easybuild/easybuild-GCC-5.3.0-20160725.151838.log
== Build succeeded for 2 out of 2
== Temporary log file(s) /tmp/eb-pqfnF_/easybuild-btsaiw.log* have been removed.
== Temporary directory /tmp/eb-pqfnF_ has been removed.

Make sure it all looks good:

tcarman2@atlas ~ $ module avail

----------- /home/UA/tcarman2/.local/easybuild/modules/all -----------
EasyBuild/2.8.2 GCC/5.3.0       M4/1.4.17
----------- /usr/share/Modules/modulefiles ---------------------------
dot         module-git  module-info modules     null        use.own
----------- /etc/modulefiles -----------------------------------------
compat-openmpi-x86_64 openmpi-x86_64

Load the module:

tcarman2@atlas ~ $ module load GCC/5.3.0
tcarman2@atlas ~ $ module list
Currently Loaded Modulefiles:
  1) EasyBuild/2.8.2   2) GCC/5.3.0

Build PEST++ on top of GCC-5.3.0

First we have to get the right version of PEST++. Looks like they have adopted github, but like us, have not fully settled on a complete workflow/pattern to follow. By looking at the network graph on github.com, and hovering over the commits, I can find a recent one on master that is called 'v3.3' in the commit message. Because they don't seem to have officially tagged this commit it does't show up in github's 'tags, or 'releases' lists for this repo. No problem, I just click on that commit to see the diff, then click on "browse files" to get to github's checkout of that commit. Then I right click on the "download" button to get the URL of the .zip file which we can then get on atlas with wget:

tcarman2@atlas ~ $ wget https://github.com/dwelter/pestpp/archive/d411f98942417395ae3d70ffbbfe0ea93d2f1d17.zip

Then I move the resulting file into the right place in EasyBuild's directory:

tcarman2@atlas~ $ mv d411f98942417395ae3d70ffbbfe0ea93d2f1d17.zip .local/easybuild/sources/p/pestpp/pestpp-d411f98.zip

Now make some files and such for EasyBuild:

tcarman2@atlas~ $ mkdir myeasyconfigs
tcarman2@atlas~ $ mkdir myeasyconfigs/pestpp
tcarman2@atlas~ $ touch myeasyconfigs/pestpp/tbc-fix-make.patch
tcarman2@atlas~ $ touch myeasyconfigs/pestpp/pestpp-3.3-GCC-5.3.0.eb

Now put some content in those files. Here is my config file, arrived at after much trial and error:

tcarman2@atlas ~ $ cat myeasyconfigs/pestpp/pestpp-3.3-gcc-5.3.0.eb 
# The easy config file for PEST++ (pestpp)
import glob

# Mandatory

#easyblock = 'EB_PESTPP'
easyblock = 'MakeCp'
#install_cmd = 'make -f makefile_linux'

# A short description of the software [default: None]
description = 'Some program...'

#  The homepage of the software [default: None]
homepage = 'http://inversemodeler.com'

#  Name of software [default: None]
name = "pestpp"

#  Software license [default: None]
#  List of software license locations [default: None]

#  Name and version of toolchain [default: None]
toolchain = {'name': 'GCC', 'version': '5.3.0'}

#  Version of software [default: None]
version = '3.3'

#source_urls = ['https://github.com/dwelter/pestpp/archive/master.zip']
#source_urls =['https://github.com/dwelter/pestpp/archive/d411f98942417395ae3d70ffbbfe0ea93d2f1d17.zip']
source_urls = ['https://codeload.github.com/dwelter/pestpp/zip/master']
sources = ['pestpp-d411f98.zip']
patches = ['tbc-fix-make.patch']

prebuildopts = 'mkdir -p %(builddir)s/pestpp-d411f98942417395ae3d70ffbbfe0ea93d2f1d17/exe/linux && export EXE_DIR="%(builddir)s/pestpp-d411f98942417395ae3d70ffbbfe0ea93d2f1d17/exe/linux" && '
start_dir = 'src'
buildopts = '-f makefile_linux'
sanity_check_paths = {'files':[], 'dirs':['%(installdir)s/pestpp-d411f98942417395ae3d70ffbbfe0ea93d2f1d17/exe/linux/']}

modextrapaths = {'PATH':'pestpp-d411f98942417395ae3d70ffbbfe0ea93d2f1d17/exe/linux/'}

# List of files or dirs to copy [default: []]
files_to_copy = ['../pestpp-d411f98942417395ae3d70ffbbfe0ea93d2f1d17']

And the patch that I created by hand for the makefile (use diff -Nru <original> <new> > <outfile.patch> and then I had to edit the file to make the paths correct at the top (src/makefile_linux):

tcarman2@atlas ~ $ cat myeasyconfigs/pestpp/tbc-fix-make.patch 
--- src/makefile_linux.orig 2016-07-25 15:56:35.099171530 -0800
+++ src/makefile_linux  2016-07-25 15:53:39.616167963 -0800
@@ -1,21 +1,23 @@
 CXX = 'g++'
 CCC = 'g++'
 FC = gfortran
-GCCLIBDIR :='/opt/local/gcc-6.1.0/lib64'
+GCCLIBDIR :='/home/UA/tcarman2/.local/easybuild/software/GCC/5.3.0/lib64/'
+#GCCLIBDIR :='/usr/lib/gcc/x86_64-redhat-linux/4.8.3'
+#GCCLIBDIR :='/opt/local/gcc-6.1.0/lib64'
 #GCCLIBDIR :='/opt/local/gcc-4.9.0/lib64'
 CUR_DIR = $(CURDIR)
 LIBS_DIR = $(CURDIR)/libs
 RUN_MNG_DIR = $(CURDIR)/libs/run_managers
-EXE_DIR = ../exe/linux
+#EXE_DIR = ../exe/linux
 INCLUDES := '-I $(LIBS_DIR)/Eigen -I $(LIBS_DIR)/common -I $(LIBS_DIR)/run_managers/abstract_base -I $(RUN_MNG_DIR)/yamr -I $(RUN_MNG_DIR)/serial -I $(RUN_MNG_DIR)/genie_wrapper -I $(RUN_MNG_DIR)/external  -I $(RUN_MNG_DIR)/wrappers  -I $(LIBS_DIR)/pestpp_common -I $(LIBS_DIR)/linear_analysis'
 LIBLDIR := '-L $(GCCLIBDIR) -L /usr/lib64 -L/opt/local/lib64 -L $(LIBS_DIR)/common  -L$(RUN_MNG_DIR)/wrappers -L$(RUN_MNG_DIR)/serial -L$(RUN_MNG_DIR)/external -L$(RUN_MNG_DIR)/yamr -L$(RUN_MNG_DIR)/genie_wrapper -L$(RUN_MNG_DIR)/abstract_base -L $(LIBS_DIR)/pestpp_common -L $(LIBS_DIR)/mio -L $(LIBS_DIR)/propack -L $(LIBS_DIR)/linear_analysis'
 LIBS :=  '-lpestpp_com -lrm_wrappers -lrm_yamr -lrm_serial -lrm_external -lrm_genie_wrapper -lrm_abstract -lmio -lcommon -l propack -llapack -lblas -llinear_analysis -lgfortran $(GCCLIBDIR)/libquadmath.a'
 
-#CFLAGS := '-pthread -std=c++11 -Wl,--no-as-needed -g -gdwarf-2' 
-#FFLAGS := '-g -gdwarf-2 -c -cpp'
+CFLAGS := '-pthread -static-libstdc++ -std=c++11 -Wl,--no-as-needed -g -gdwarf-2' 
+FFLAGS := '-g -gdwarf-2 -c -cpp'
 
-CFLAGS := '-pthread  -Wl,--no-as-needed -O2' 
-FFLAGS := '-O2 -c -cpp'
+#CFLAGS := '-pthread  -Wl,--no-as-needed -O2' 
+#FFLAGS := '-O2 -c -cpp'
 #LFLAGS := '-static -static-libgcc -static-libgfortran'
 
 all:

With any luck now, you can install like this:

tcarman2@atlas ~ $ eb --force  myeasyconfigs/pestpp/pestpp-3.3-gcc-5.3.0.eb --debug

If that succeeds, then you need to load the module "pestpp":

tcarman2@atlas ~ $ module load pestpp

And you should be able to run the program:

tcarman2@atlas ~ $ pestpp


             PEST++ Version 3.3.0

                 by Dave Welter
     Computational Water Resource Engineering


--------------------------------------------------------
usage:

    serial run manager:
        pest++ pest_ctl_file.pst

    YAMR master:
        pest++ control_file.pst /H :port

    YAMR runner:
        pest++ /H hostname:port 

    GENIE:
        pest++ control_file.pst /G hostname:port

    external run manager:
        pest++ control_file.pst /E
--------------------------------------------------------

Success!

Add a custom footer
Add a custom sidebar
Clone this wiki locally