Contributing to SPH EXA

In order to contribute to the project, create a pull request to merge your changes to the code.

Continuous integration

Whenever you open a pull request on github, .gitlab/gitlab-ci.yml will trigger a pipeline on CSCS gitlab server. After building and running all the steps of pipeline, gitlab will report back to github the status of the tests, in this case a success:

Clicking on the icon next to the git commit on the pull request page will take you to the details of the gitlab pipeline:

In gitlab, you can inspect each step of the pipeline and find out what has happened (in this case, all the steps have passed):

If 1 or more tests fail, the failure probably comes from the changes that you have made to the code. You must fix the code and push another commit before merging the pull request. The new commit will trigger a new pipeline. When all the tests pass, the pull request can be merged (if approved).

External developpers can open pull requests but only approved team members can launch CI jobs by adding the phrase cscs-ci run as a comment in the PR whenever needed.

Known Issues

This is a list of commonly encountered problems, known issues, and their solutions.

CMake and libpthread.a

Trying to compile sph_exa using CMake will sometimes fail with an error message about libpthread.a:

[ 87%] Linking CUDA device code CMakeFiles/sedov-cuda.dir/cmake_device_link.o
nvlink fatal   : Could not open input file '/usr/lib/x86_64-linux-gnu/libpthread.a'

This is a known CMake issue and it will be fixed in future releases of CMake (we tested with versions from 3.17 up to 3.22). The issue comes from the version of the libc package where libpthread.a is coming from:

• if your system has a libc version older or equal to 2.31 (such as Ubuntu/20.04), then you should not experience the issue. HPC systems such as Piz Daint/Alps/Lumi have older versions of libc (i.e 2.26),

• if your system has a libc version newer than 2.31 (such as Ubuntu/21.10), then you should add an extra flag to your cmake command line:

cmake -DOpenMP_pthread_LIBRARY=`dpkg -L libc6 | grep libpthread.so`
      -DCUDAToolkit_rt_LIBRARY=`dpkg -L libc6 | grep librt.so`
      -DHDF5_C_LIBRARY_dl=`dpkg -L libc6 | grep libdl.so`

More details about supported versions here.

CUDA and GNU compilers

Trying to compile sph_exa using an unsupported gnu compiler version (for example gcc/11 and nvcc/11.0) will raise an error message:

#error -- unsupported GNU version! gcc versions later than 9 are not supported!
The nvcc flag '-allow-unsupported-compiler' can be used to override this
version check; however, using an unsupported host compiler may cause
compilation failure or incorrect run time execution. Use at your own risk.

More recent nvcc versions support more recent gnu compiler versions. As a result, when using a Linux desktop (or a container image), it is necessary to install a version of nvcc (with apt install cuda-nvcc-11.0) that matches the right gnu compiler:

GNU compiler support	gcc/9.x	gcc/10.x	gcc/11.x
CUDA/11.0 (11/2020)	✅	❌	❌
CUDA/11.2 (12/2020)	✅	✅	❌
CUDA/11.4 (06/2021)	✅	✅	✅
CUDA/11.6 (01/2022)	✅	✅	✅

More details in host_config.h in /usr/local/cuda-11.0/targets/x86_64-linux/include/crt/.

Git hooks and clang-format

PRs are checked for clang-format violations before being merged, but it can be tedious to find that a PR failed checks due to simple formatting errors. To help reduce this kind of problem you can install a git hook that will check your files when you commit them. A pre-commit hook exists in the repo (in scripts/pre-commit)and you can activate it by copying the file into your git hooks folder and making it executable as follows

cp scripts/pre-commit .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit

Now when you commit files, you will receive a warning/error - example

git commit -m'Test pre-commit hook'
# clang-format error pre-commit : To fix run the following (use git commit --no-verify to bypass)
clang-format -i domain/include/cstone/tree/definitions.h
clang-format -i src/sqpatch/sqpatch.cpp

# To commit the corrected files, run

git add src/sqpatch/sqpatch.cpp domain/include/cstone/tree/definitions.h 

You can simply copy the commands generated by the hook and paste them into the terminal to run clang-format on any files that have failed the check, and re-add them to the commit with the fixed formatting.

clang-format -i will directly correct your files so that they have the correct format.

Installing clang-format in Ubuntu

In order to fix the format as explained above, you will need to have clang-format installed in your system. To do so, go to https://apt.llvm.org/ and add the corresponding sources to your /etc/apt/sources.list

Download the archive signature:

wget -O - https://apt.llvm.org/llvm-snapshot.gpg.key|sudo apt-key add -

Update apt:

sudo apt update

And finally, install clang-format:

sudo apt install clang-format

Code coverage

Code coverage is a percentage measure of the degree to which the source code of a program is executed when a particular test suite is run. A program with high test coverage has more of its source code executed during testing, which suggests it has a lower chance of containing undetected software bugs. One way is to build the code with the gnu compiler (clang is another option)

cmake -S SPH-EXA.git -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_CXX_FLAGS=--coverage
make sphexa  # will create .gcno files
srun sphexa  # will create .gcda files 

and launch the coverage tools to get an html report:

spack load lcov
cd build
lcov --capture --gcov-tool gcov --base-directory ../SPH-EXA.git/ --directory . --output-file coverage.raw
lcov --extract coverage.raw --quiet "`dirname $PWD`/SPH-EXA.git/*" -o coverage.raw.filtered
genhtml --demangle-cpp --output-directory coverage.report coverage.raw.filtered

firefox coverage.report/index.html

Depending on the number and configuration of the tests, the code coverage will vary. An example coverage report will look like this: