Quicker compile test cycle for MPICH

A short compile-test cycle is critical for developer productivity.

We are very bad at interruptions. When we have to wait for a build to finish for more than a few minutes, we'll shift our attention to somewhere else. Then it will be difficult for us to remind ourselves to check back. And when we do get back, we will have a hard time recall the context and slow to resume. Of course the action of getting back to our old task interrupts the new tasks that we have shifted into ...

Baseline - 10 minutes and up

The official build process for MPICH is very slow and tedious:

1. git submodule init && git submodule update
2. ./autogen.sh
3. ./configure --with-prefix=$PWD/_inst --with-device=ch4:ofi --disable-fortran --disable-romio --enable-g=dbg
4. make install
5. [...test...]

Note that on step 3 I have listed the typical configure options that is typical during development. It skips Fortran bindings and ROMIO to cut some precious build time.

On my relatively powerful work station:

1. [submodule] 11 sec
2. [autogen] 3 min
3. [configure] 2 min 46 sec
4. [make-all] 2 min 10 sec

If you are building on your laptop, your timing will be longer, maybe much longer...

Hack sub modules and autogen -- 40 sec config + 1 min compile

It is easy to recognize that there are a lot of things happen during this build process that an opportunist like me would feel like skipping. However, much of the steps to accelerate the build will involve large churn to build scripts and shortcut the long established autotool paradigm. Thus, it is slow and difficult to get real acceleration into the official MPICH. So meanwhile, I resolve to hacking --

First prebuild sub modules.

$ git clone https://github.com/hzhou/mpich-modules

sh build-all.sh will build module.tar.gz if you are lucky. If you are targeting specific platform and compiler, you'll need tinker the build scripts and build specific tarballs.

Export MODTARBALL=path/to/module.tar.gz

Second, get mymake

$ git clone https://github.com/hzhou/mymake_mpich mymake

Alias `mymake="perl path/to/mymake.pl"

There are two ways of using mymake.

• Use vanilla configure

$ mymake --with-prefix=$PWD/_inst --with-device=ch4:ofi --disable-fortran --disable-romio --enable-g=dbg
$ make install
$ make hydra-install
$ make testing

Hydra is completely separate from the mpich build since most of the time we don't want to rebuild hydra or even checking hydra. make testing only sets up the testing. It just configures test/mpi. It is separate for the same reason of more modular control of build time.

With this approach, step 1 takes 42 sec. It is faster than official configure (let alone skipping of autogen) because it skips most of the sub modules. make install is also faster because it uses a lean Makefile rather than the monster from automake. mymake parses the automake Makefile and simplifies it.

Hack Autoconf and Automake -- instant config + 1 min compile

$ mymake --with-prefix=$PWD/_inst --with-device=ch4:ofi --disable-fortran --disable-romio --enable-g=dbg -quick
$ make install
$ make hydra-install
$ make test

The only difference is the -quick in the first step, but it is fundamentally different from previous approach. It skips the configure and directly parses automake. It is quick as it should take less than 1 sec.

A few convenience bash alias will also help the build-test cycle:

• mpich_pr #### -- Clone the pull request into your test folder and cd into it.
• mymake_ofi -- alias to your common configure line
• set_lib -- set up paths to your newly installed mpich ($PWD/_inst) and ready to test

What's left -- build in seconds

The compile time of MPICH is largely spent in repeatedly compiling nested inline headers. If we allow rearrangement of the source code then we potentially can accelerate the compilation significantly. This is possible if we put the source code in a meta form -- similar to what is done by Knuth in his literate programming -- WEB. If we eliminate the repeated compiling and eliminate linking, then we should be able to compile MPICH instantaneously.

Unfortunately, the transforming of the source code will interfere with my development work, so for now, the 1 minute overhead need be tolerated.