Installation problem: How to set MKLROOT when I use command conda install -c intel mkl mkl-devel mkl-static mkl-include to install MKL`

[dongwhfdyer]

I am trying to install aihwkit's gpu version on ubuntu20.04. But I got stuck with MKL library.
Here are the commands I used:

conda create -n aihwkit
conda install cmake openblas pybind11
conda install -c conda-forge scikit-build
conda install -c pytorch pytorch
conda install -c intel mkl mkl-devel mkl-static mkl-include
 make build_cuda

cuda11.6 is preinstalled.

I used conda install -c intel mkl mkl-devel mkl-static mkl-include to install MKL library. It went smoothly. But I don't know where MKL library is installed so I cannot set the ENV variable MKLROOT said in the docs. When I just simplely use make build_cuda trying to compile the aihwkit source code, I always got the following output.

I have been stuck here for two days. If there's anything helpful, I will be very grateful!!

CMake Warning at cmake/Modules/FindMKL.cmake:367 (MESSAGE):
  MKL libraries files are found, but MKL header files are not.  You can get
  them by `conda install mkl-include` if using conda (if it is missing, run
  `conda upgrade -n root conda` first), and `pip install mkl-devel` if using
  pip.  If build fails with header files available in the system, please make
  sure that CMake will search the directory containing them, e.g., by setting
  CMAKE_INCLUDE_PATH.
Call Stack (most recent call first):
  cmake/dependencies.cmake:61 (find_package)
  CMakeLists.txt:39 (include)


CMake Error at cmake/Modules/FindMKL.cmake:380 (MESSAGE):
  MKL library not found.  Please specify library location
Call Stack (most recent call first):
  cmake/dependencies.cmake:61 (find_package)
  CMakeLists.txt:39 (include)


-- Configuring incomplete, errors occurred!
See also "/home/szh2/kuhn/fairseq/aihwkit/_skbuild/linux-x86_64-3.11/cmake-build/CMakeFiles/CMakeOutput.log".
Traceback (most recent call last):
  File "/home/szh2/miniconda3/envs/aihwkit2/lib/python3.11/site-packages/skbuild/setuptools_wrap.py", line 634, in setup
    env = cmkr.configure(
          ^^^^^^^^^^^^^^^
  File "/home/szh2/miniconda3/envs/aihwkit2/lib/python3.11/site-packages/skbuild/cmaker.py", line 332, in configure
    raise SKBuildError(

An error occurred while configuring with CMake.
  Command:
    /home/szh2/miniconda3/envs/aihwkit2/bin/cmake /home/szh2/kuhn/fairseq/aihwkit -G Ninja -DCMAKE_INSTALL_PREFIX:PATH=/home/szh2/kuhn/fairseq/aihwkit/_skbuild/linux-x86_64-3.11/cmake-install -DPYTHON_VERSION_STRING:STRING=3.11.0 -DSKBUILD:INTERNAL=TRUE -DCMAKE_MODULE_PATH:PATH=/home/szh2/miniconda3/envs/aihwkit2/lib/python3.11/site-packages/skbuild/resources/cmake -DPYTHON_EXECUTABLE:PATH=/home/szh2/miniconda3/envs/aihwkit2/bin/python -DPYTHON_INCLUDE_DIR:PATH=/home/szh2/miniconda3/envs/aihwkit2/include/python3.11 -DPYTHON_LIBRARY:PATH=/home/szh2/miniconda3/envs/aihwkit2/lib/libpython3.11.so -DPython_EXECUTABLE:PATH=/home/szh2/miniconda3/envs/aihwkit2/bin/python -DPython_ROOT_DIR:PATH=/home/szh2/miniconda3/envs/aihwkit2 -DPython_INCLUDE_DIR:PATH=/home/szh2/miniconda3/envs/aihwkit2/include/python3.11 -DPython_FIND_REGISTRY:STRING=NEVER -DPython_NumPy_INCLUDE_DIRS:PATH=/home/szh2/miniconda3/envs/aihwkit2/lib/python3.11/site-packages/numpy/core/include -DPython3_EXECUTABLE:PATH=/home/szh2/miniconda3/envs/aihwkit2/bin/python -DPython3_ROOT_DIR:PATH=/home/szh2/miniconda3/envs/aihwkit2 -DPython3_INCLUDE_DIR:PATH=/home/szh2/miniconda3/envs/aihwkit2/include/python3.11 -DPython3_FIND_REGISTRY:STRING=NEVER -DPython3_NumPy_INCLUDE_DIRS:PATH=/home/szh2/miniconda3/envs/aihwkit2/lib/python3.11/site-packages/numpy/core/include -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE -DRPU_BLAS=MKL -DINTEL_MKL_DIR= -DUSE_CUDA=ON -DCMAKE_BUILD_TYPE:STRING=Release
  Source directory:
    /home/szh2/kuhn/fairseq/aihwkit
  Working directory:
    /home/szh2/kuhn/fairseq/aihwkit/_skbuild/linux-x86_64-3.11/cmake-build
Please see CMake's output for more information.
make[2]: *** [Makefile:22: build] Error 1
make[2]: Leaving directory '/home/szh2/kuhn/fairseq/aihwkit'
make[1]: *** [Makefile:25: build_mkl] Error 2
make[1]: Leaving directory '/home/szh2/kuhn/fairseq/aihwkit'
make: *** [Makefile:28: build_cuda] Error 2

[matifali]

As a workaround, you may consider using the docker image.
See https://aihwkit.readthedocs.io/en/latest/advanced_install.html#cuda-enabled-docker-image

[maljoras]

The MKL library is a little tricky since it can be installed in different locations in principle. Since you installed it with conda though you can try

MKLROOT=$CONDA_PREFIX/lib CMAKE_PREFIX_PATH=$CONDA_PREFIX make build_cuda

which should be similar to this:

python setup.py install --user -j8 -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXPORT_COMPILE_COMMANDS=TRUE -DUSE_CUDA=ON  -DRPU_BLAS=MKL -DINTEL_MKL_DIR=$CONDA_PREFIX/lib -DCMAKE_PREFIX_PATH=$CONDA_PREFIX

[dongwhfdyer]

Really thank you!!! I finally solve the problem which I have been stuck for 5 days.
Here is the command I use to install the aihwkit, hope it might help someone.

conda create -n aihwkit python=3.9
conda activate aihwkit
conda install -y cmake openblas pybind11
conda install -y -c conda-forge scikit-build
pip install torch torchvision
git clone https://github.com/IBM/aihwkit.git
cd aihwkit
conda install -y -c intel mkl mkl-devel mkl-static mkl-include
MKLROOT=$CONDA_PREFIX/lib CMAKE_PREFIX_PATH=$CONDA_PREFIX make build_cuda flags="-DCMAKE_CUDA_COMPILER=/usr/local/cuda/bin/nvcc"

But it will output an error generally.(I have tested it on two machine).
ImportError: libmkl_intel_lp64.so.2: cannot open shared object file: No such file or directory
After searching online, I have to explictly set the conda lib path export LD_LIBRARY_PATH=/home/zx/miniconda3/envs/th13/lib:$LD_LIBRARY_PATH in the ~./bashrc.

It's not an elegant way to solve it. But I don't know how to do it better.

Then It seems i can use aihwkit(cuda version).

[dongwhfdyer]

As a workaround, you may consider using the docker image. See https://aihwkit.readthedocs.io/en/latest/advanced_install.html#cuda-enabled-docker-image

Actually, it failed when i use docker. 😭

[Zhaoxian-Wu]

@dongwhfdyer Thanks for your valuable experience! I encounter a similar libmkl_intel_lp64.so.2 issue. Your suggestion of adding LD_LIBRARY_PATH makes me manage to do that.

[maljoras]

Right, the MKL library certainly has to be in the LD_LIBRARY_PATH