-
Notifications
You must be signed in to change notification settings - Fork 1
Building DGSWEM HPX on Stampede
Before starting the following steps, follow the link below for installing HPX:
module swap intel/15.0.2 gcc/4.9.1
module load boost/1.55.0
module load cmake/3.1.0
wget --no-check-certificate https://libgeodecomp.org/archive/libgeodecomp-0.4.0-ra22d890554f5fb00b7d88c1e1f7c4716a364511b.tar.gz
tar -xzvf libgeodecomp-0.4.0-ra22d890554f5fb00b7d88c1e1f7c4716a364511b.tar.gz
mv libgeodecomp-0.4.0-ra22d890554f5fb00b7d88c1e1f7c4716a364511b libgeodecomp-0.4.0
Download build script: build_lgd_release_stampede.sh
chmod +x build_lgd_release_stampede.sh
./build_lgd_release_stampede.sh
git clone -b dgswem-hpx git@github.com:foci/dgswem.git dgswem-hpx
or:
git clone https://github.com/foci/dgswem.git
cd dgswem
git checkout -b dgswem-hpx remotes/origin/dgswem-hpx
then:
mkdir dgswem-hpx/build_release
cd dgswem-hpx/build_release
cmake -DCMAKE_CXX_COMPILER=/opt/apps/gcc/4.9.1/bin/g++ -DCMAKE_C_COMPILER=/opt/apps/gcc/4.9.1/bin/gcc -DCMAKE_PREFIX_PATH=$WORK/local_install/ -DCMAKE_Fortran_COMPILER=/opt/apps/intel/15/composer_xe_2015.2.164/bin/intel64/ifort -DCMAKE_BUILD_TYPE=Release ..
make dgswem_lgd_dataflow
You need to build and run adcprep (make adcprep) and run it in the directory as you normally would for a parallel DGSWEM run. However, the number of subdomains you divide the grid into does not need to match the number of compute cores you plan to use (but generally you should break it into at least that number).
before running, verify that you have the same modules loaded as above:
module swap intel/15.0.2 gcc/4.9.1
module load boost/1.55.0
module load cmake/3.1.0
and add the ifort libraries to your LD_LIBRARY_PATH:
export LD_LIBRARY_PATH=/opt/apps/intel/15/composer_xe_2015.2.164/compiler/lib/intel64:$LD_LIBRARY_PATH
and then finally from the directory of the problem you want to run:
/PATH/TO/dgswem-hpx-dev/build_release/src/dgswem_lgd_dataflow
--n_timesteps <number of timesteps>
The application will run into problems if the ulimit -n is not set high enough. (This hasn't been a problem on Stampede so far.)
When running an HPX application, instead of using -n , use -N -n . Also, there is no restriction on the number of compute nodes or cores and the number of subdomains the problem is divided into.