Skip to content

Advanced multi threading support in CH4

Hajime Fujita edited this page Nov 9, 2018 · 4 revisions

MPICH 3.3rc1 (released on 11/9/2018) introduced several improvements to MPI_THREAD_MULTIPLE. As of 3.3rc1, this feature is not enabled by default, therefore a special build configuration is needed.

Prerequisites

The new improvements are implemented inside the CH4 device. So CH4 needs to be selected when building MPICH.

Example configure lines:

./configure --with-device=ch4:ofi --with-libfabric=<path/to/ofi/install>
./configure --with-device=ch4:ucx --with-ucx=<path/to/ucx/install> 

Refer to MPICH README for more details on how to build CH4.

New threading models

3.3rc1 introduced two threading models.

  • Trylock-enqueue (trylock)
  • Handoff (handoff)

Trylock-enqueue

Build time configurations:

--enable-thread-cs=per-vni --enable-izem=queue --with-zm-prefix=yes --enable-ch4-mt=trylock --with-hwloc=<path/to/hwlocinstall>

Runtime configurations:

  • None needed

Handoff

Build time configurations:

--enable-thread-cs=per-vni --enable-izem=queue --with-zm-prefix=yes --enable-ch4-mt=handoff --with-hwloc=<path/to/hwlocinstall>

Runtime configurations:

  • Set the environment variable:
MPIR_CVAR_ASYNC_PROGRESS=1