Skip to content

Benchmarks

Jump to bottom
Luc Berger edited this page Feb 15, 2019 · 18 revisions

The features benchmarked for performance tracking are tested on multiple platforms and instructions to perform standard build and tests are gathered here as they are common to all benchmarks. If a particular kernel needs specific settings to be benchmarked, it will be specified on the kernel benchmark page.

You should use the latest version of Kokkos and Kokkos-Kernels to run benchmarks and ideally use the master branch but if you are looking for the very latest updates, you can also attempt the develop branch.

In these instructions, the assumption is that the benchmarks will be setup in $HOME/kokkoskernels_benchmark.

1. Get Kokkos

mkdir $HOME/kokkoskernels_benchmark
cd $HOME/kokkoskernels_benchmark
git clone git@github.com:kokkos/kokkos.git

2. Get KokkosKernels

cd $HOME/kokkoskernels_benchmark
git clone git@github.com:kokkos/kokkos-kernels.git

3. Update the compileKokkosKernels.sh located at example/buildlib.

cd $HOME/kokkoskernels_benchmark/kokkos-kernels/example/buildlib
vi compileKokkosKernels.sh

KNL setup (Bowman)

Below is the example of compileKokkosKernels.sh for KNLs.

KOKKOS_PATH=${HOME}/kokkoskernels_benchmark/kokkos #path to kokkos source
KOKKOSKERNELS_SCALARS='double' #we only need double
KOKKOSKERNELS_LAYOUTS=LayoutLeft #the layout types to instantiate.
KOKKOSKERNELS_ORDINALS=int #ordinal types to instantiate
KOKKOSKERNELS_OFFSETS=int #offset types to instantiate
KOKKOSKERNELS_PATH=${HOME}/kokkoskernels_benchmark/kokkos-kernels #path to kokkos-kernels top directory.
KOKKOSKERNELS_OPTIONS=eti-only #options for kokkoskernels  
CXXFLAGS="-Wall -pedantic -Werror -O3 -g -Wshadow -Wsign-compare -Wignored-qualifiers -Wempty-body -Wclobbered -Wuninitialized"
CXX=icpc
KOKKOS_DEVICES=OpenMP #we only need openmp execution space.
KOKKOS_ARCHS=KNL        #!!!!!!!!!!!!!!!!!!specify the architecture for compilation!!!!!!!!!!!!!!!!!!!
KOKKOSKERNELS_TPLS=mkl #if you want to be able to run mkl functionalities as well. otherwise place "".

../../scripts/generate_makefile.bash --kokkoskernels-path=${KOKKOSKERNELS_PATH} --with-scalars=${KOKKOSKERNELS_SCALARS} --with-ordinals=${KOKKOSKERNELS_ORDINALS} --with-offsets=${KOKKOSKERNELS_OFFSETS} --kokkos-path=${KOKKOS_PATH} --with-devices=${KOKKOS_DEVICES} --arch=${KOKKOS_ARCHS} --compiler=${CXX} --with-options=${KOKKOSKERNELS_OPTIONS}  --cxxflags="${CXXFLAGS}" --with-tpls=${KOKKOSKERNELS_TPLS}

Set the compiler and if you want to use mkl, export the mkl path.

module load intel/compilers/18.2.199
export MKL_PATH=/home/projects/x86-64-knl/intel/compilers/2018/mkl

Skylake setup (Blake)

Below is the example of compileKokkosKernels.sh for SkyLake.

KOKKOS_PATH=${HOME}/kokkoskernels_benchmark/kokkos #path to kokkos source
KOKKOSKERNELS_SCALARS='double' #we only need double
KOKKOSKERNELS_LAYOUTS=LayoutLeft #the layout types to instantiate.
KOKKOSKERNELS_ORDINALS=int #ordinal types to instantiate
KOKKOSKERNELS_OFFSETS=int #offset types to instantiate
KOKKOSKERNELS_PATH=${HOME}/kokkoskernels_benchmark/kokkos-kernels #path to kokkos-kernels top directory.
KOKKOSKERNELS_OPTIONS=eti-only #options for kokkoskernels  
CXXFLAGS="-Wall -pedantic -Werror -O3 -g -Wshadow -Wsign-compare -Wignored-qualifiers -Wempty-body -Wclobbered -Wuninitialized"
CXX=icpc
KOKKOS_DEVICES=OpenMP #we only need openmp execution space.
KOKKOS_ARCHS=SKL        #!!!!!!!!!!!!!!!!!!specify the architecture for compilation!!!!!!!!!!!!!!!!!!!
KOKKOSKERNELS_TPLS=mkl #if you want to be able to run mkl functionalities as well. otherwise place "".

../../scripts/generate_makefile.bash --kokkoskernels-path=${KOKKOSKERNELS_PATH} --with-scalars=${KOKKOSKERNELS_SCALARS} --with-ordinals=${KOKKOSKERNELS_ORDINALS} --with-offsets=${KOKKOSKERNELS_OFFSETS} --kokkos-path=${KOKKOS_PATH} --with-devices=${KOKKOS_DEVICES} --arch=${KOKKOS_ARCHS} --compiler=${CXX} --with-options=${KOKKOSKERNELS_OPTIONS}  --cxxflags="${CXXFLAGS}" --with-tpls=${KOKKOSKERNELS_TPLS}

Set the compiler and if you want to use mkl, export the mkl path.

module load intel/compilers/18.2.199
export MKL_PATH=/home/projects/x86-64-knl/intel/compilers/2018/mkl

Power8 setup (White)

Below is the example of compileKokkosKernels.sh for Power8 with OpenMP execution space.

KOKKOS_PATH=${HOME}/kokkoskernels_benchmark/kokkos #path to kokkos source
KOKKOSKERNELS_SCALARS='double' #we only need double
KOKKOSKERNELS_LAYOUTS=LayoutLeft #the layout types to instantiate.
KOKKOSKERNELS_ORDINALS=int #ordinal types to instantiate
KOKKOSKERNELS_OFFSETS=int #offset types to instantiate
KOKKOSKERNELS_PATH=${HOME}/kokkoskernels_benchmark/kokkos-kernels #path to kokkos-kernels top directory.
KOKKOSKERNELS_OPTIONS=eti-only #options for kokkoskernels  
CXXFLAGS="-Wall -pedantic -Werror -O3 -g -Wshadow -Wsign-compare -Wignored-qualifiers -Wempty-body -Wclobbered -Wuninitialized"
CXX=g++
KOKKOS_DEVICES=OpenMP #we only need openmp execution space.
KOKKOS_ARCHS=Power8        #!!!!!!!!!!!!!!!!!!specify the architecture for compilation!!!!!!!!!!!!!!!!!!!
KOKKOSKERNELS_TPLS="" 

../../scripts/generate_makefile.bash --kokkoskernels-path=${KOKKOSKERNELS_PATH} --with-scalars=${KOKKOSKERNELS_SCALARS} --with-ordinals=${KOKKOSKERNELS_ORDINALS} --with-offsets=${KOKKOSKERNELS_OFFSETS} --kokkos-path=${KOKKOS_PATH} --with-devices=${KOKKOS_DEVICES} --arch=${KOKKOS_ARCHS} --compiler=${CXX} --with-options=${KOKKOSKERNELS_OPTIONS}  --cxxflags="${CXXFLAGS}" --with-tpls=${KOKKOSKERNELS_TPLS}

Set the compiler.

module load gcc/7.2.0

P100 setup (White)

Below is the example of compileKokkosKernels.sh for Power8 with OpenMP execution space.

KOKKOS_PATH=${HOME}/kokkoskernels_benchmark/kokkos #path to kokkos source
KOKKOSKERNELS_SCALARS='double' #we only need double
KOKKOSKERNELS_LAYOUTS=LayoutLeft #the layout types to instantiate.
KOKKOSKERNELS_ORDINALS=int #ordinal types to instantiate
KOKKOSKERNELS_OFFSETS=int #offset types to instantiate
KOKKOSKERNELS_PATH=${HOME}/kokkoskernels_benchmark/kokkos-kernels #path to kokkos-kernels top directory.
KOKKOSKERNELS_OPTIONS=eti-only #options for kokkoskernels  
CXXFLAGS="-Wall -pedantic -Werror -O3 -g -Wshadow -Wsign-compare -Wignored-qualifiers -Wempty-body -Wclobbered -Wuninitialized"
CXX=g++
KOKKOS_DEVICES=OpenMP,Cuda  #we need both cuda and openmp/serial execution space.
KOKKOS_ARCHS=Pascal60,Power8
KOKKOSKERNELS_TPLS="cusparse" ###to enable cusparse 

../../scripts/generate_makefile.bash --kokkoskernels-path=${KOKKOSKERNELS_PATH} --with-scalars=${KOKKOSKERNELS_SCALARS} --with-ordinals=${KOKKOSKERNELS_ORDINALS} --with-offsets=${KOKKOSKERNELS_OFFSETS} --kokkos-path=${KOKKOS_PATH} --with-devices=${KOKKOS_DEVICES} --arch=${KOKKOS_ARCHS} --compiler=${CXX} --with-options=${KOKKOSKERNELS_OPTIONS}  --cxxflags="${CXXFLAGS}" --with-tpls=${KOKKOSKERNELS_TPLS}

Set the compiler.

module load gcc/7.2.0
module load cuda/9.2.148

V100 (?)

4. Compile KokkosKernels.

cd $HOME/kokkoskernels_benchmark/kokkos-kernels/example/buildlib
./compileKokkosKernels.sh
make build-test -j

Home:

Using KokkosKernels

API Reference

Examples

Benchmarks

Clone this wiki locally