Add Dockerfiles for cuQuantum, cudaQuantum, and HPC-X to the Ella Project

[Quleaf]

Overview

This pull request introduces three new Dockerfiles to the Ella project repository:

1. cuQuantum Dockerfile
2. cudaQuantum Dockerfile
3. HPC-X Dockerfile

These Dockerfiles establish robust and reproducible containerized environments tailored for different components of the Ella project, facilitating seamless development, testing, and deployment.

Dockerfile Details

1. cuQuantum Dockerfile

• Base Image: ubuntu:22.04
• CUDA Version: 12.6.0
• HPC-X Version: v2.20
• Python Version: 3.12
• Key Features:
  • Integration of the cuQuantum library for quantum computing.
  • Incorporation of the HPC-X MPI library for high-performance computing.
  • Setup of a Python virtual environment with essential dependencies and Qiskit integration.
  • Configuration of environment variables for CUDA, HPC-X, and Python.
  • Creation of symbolic links for cuTensor and cuQuantum libraries to ensure proper linking within the environment.
  • Inclusion of MPI configuration files for mpi4py.

2. cudaQuantum Dockerfile

• Base Image: ubuntu:22.04
• CUDA Version: 12.6.0
• HPC-X Version: v2.20
• Python Version: 3.12
• Key Features:
  • Integration of NVIDIA's CUDA Quantum tools.
  • Compilation support using nvq++ and nvq-py from CUDA Quantum.
  • Set up a Python virtual environment with necessary dependencies and Qiskit integration.
  • Install CUDA Quantum packages with temporary patches to ensure compatibility.
  • Configuration of environment variables for CUDA, HPC-X, and CUDA Quantum.
  • MPI configuration files for mpi4py should be included.

3. HPC-X Dockerfile

• Base Image: ubuntu:22.04 and rocky:9.4
• CUDA Version: 12.6.0
• HPC-X Version: v2.20
• Key Features:
  • Setup of the HPC-X MPI library with OpenMPI for the Ella MPI environment.
  • Installation of the CUDA Toolkit 12.6.0 along with necessary dependencies.
  • Configuration of environment variables for CUDA and HPC-X to facilitate MPI applications.
  • Creation of symbolic links for OpenMPI installation to ensure correct MPI path configurations.
  • Inclusion of MPI configuration files for mpi4py.

Usage Instructions

Build from quay.io

singularity build <docker-file-name-tag> docker://quay.io/pawsey/<docker-file-name>:<tag>
Example: singularity build docker://quay.io/pawsey/hpcx-2.20-rocky-arm:11-11

Execution

• ** For single instance execution and MPI compilation**:
  singularity shell --nv <docker-file-name-tag>
• **For Ella cluster MPI execution:
  You may need the env file and --env-file parameter to support the MPI running environment. It will automatically inject the environment into the container. A batch script example:

#!/bin/bash
#SBATCH --job-name=my_mpi_job
#SBATCH --nodes=4
#SBATCH --ntasks-per-node=1
#SBATCH --gres=gpu:1
#SBATCH --time=00:10:00
#SBATCH --partition=gpu
#SBATCH --output=mpi_job_%j.out
#SBATCH --error=mpi_job_%j.err

module load hpcx-mt-ompi

mpirun singularity exec --nv \
        -B /run/slurm:/run/slurm \
        -B /etc/slurm:/etc/slurm \
        -B /run/munge:/run/munge \
        -B /software:/software \
        --env-file hpcx-mt-ompi.env\
        cuquantum-appliance-next-ubuntu-arm.sif bash -c <command>

Additional Notes

-Dependencies: All necessary dependencies are installed to support CUDA, HPC-X, MPI, and Python functionalities.
-Python Environment: A virtual environment (python-venv) is configured to isolate Python dependencies, enhancing reproducibility and reducing conflicts. We highly recommend NOT to use conda for this project.
-Qiskit Integration: Qiskit is integrated to provide quantum computing capabilities within the containers. We rebuild the qiskit for cuquantum and MPI usage.
-MPI Configuration: cuda-aware MPI is configured with mpi4py to enable Python-based MPI applications; You don't need to use hpcx script to load environment variables.

[pelahi]

The recipes that rely on an entry point in docker should have a compatible one for singularity so you don't need to source a script. Consider using something like

RUN mkdir -p /.singularity.d/env/ && \
    cp -p <script_to_source> /.singularity.d/env/91-environment.sh

[Quleaf]

Thanks, Pascal @pelahi , for your advice.

1. I have repaired singularity entrypoint using /.singularity.d/env
2. The folder name has been replaced correctly.
3. QISKIT with cuquantum generation image has been uploaded.

[pelahi]

If possible updates would be useful but are not critical. Just leave a comment saying that these changes will happen in the future and I am happy to approve.

[Quleaf]

All modifications from comments have been uploaded. Please check it again. Thanks.

[pelahi]

Thanks for all this work @Quleaf