firedrake installation issues on HPC cluster #3466
Replies: 19 comments 10 replies
-
The error here looks to be building pastix:
You are using a recent version of OpenMPI which is not compatible. If an older version of OpenMPI exists on your machine you could try that. Otherwise @JDBetteridge is there a suitable |
Beta Was this translation helpful? Give feedback.
-
Pastix is not required for a functional Firedrake, you could try passing In the future, please use triple backticks (`) to format your code output so that is is readable. |
Beta Was this translation helpful? Give feedback.
-
openmpi-4.1.0 seems hardly to be a recent version of OpenMPI :-) Petsc seems to be extremely problematic when installing firedrake. Would I need to change the spack@yaml and/or another file in order to input the Please advice Kind regards |
Beta Was this translation helpful? Give feedback.
-
OK before I go then. |
Beta Was this translation helpful? Give feedback.
-
Thanks. It is a bit of a convoluted process requiring the use of Docker which we do not have on our HPC. I can see many things that can go wrong. is a python/pip installation conceivable with a more universal script? Kind regards |
Beta Was this translation helpful? Give feedback.
-
Thanks.
Will try the different options for firedrake-install.At some point will also try the singularity way.
Kind regardsRoberto
Il giovedì 21 marzo 2024 alle ore 14:05:44 GMT, Jack Betteridge ***@***.***> ha scritto:
- The singularity build explicitly does not use docker, it just pulls the container from dockerhub as there is no centralised repository for singularity containers. We have many users who use the singularity container no issue, I'm not sure what you mean by "I can see many things that can go wrong." If you post an error message maybe we could help.
- The install script supports many different install paths, you do not require sudo and you can avoid calling any package manager (using --no-package-manager) if all the Firedrake dependencies are already installed. This information is all available by running python3 firedrake-install --help.
- Creating a pip installable Firedrake is certainly desirable, but is an incredibly involved process to get something working due to our heavy dependency on PETSc and it's compiled external packages.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Hi,
After a million hours ...I got this error
==================================Installing libsupermesh2024-03-21 18:07:31,951 INFO Cloning libsupermesh
2024-03-21 18:07:31,951 DEBUG Running command 'git clone -q --recursive ***@***.***:firedrakeproject/libsupermesh.git'2024-03-21 18:07:33,167 DEBUG Permission denied (publickey).^Mfatal: Could not read from remote repository.
Please make sure you have the correct access rightsand the repository exists.
=========================================and of course now I have to run everything again after deleting a very big directory :-(Can you please improve the installation script so that people do not have to spend the entire afternoon to make it work?
Many thanks
Roberto
Il giovedì 21 marzo 2024 alle ore 14:05:44 GMT, Jack Betteridge ***@***.***> ha scritto:
- The singularity build explicitly does not use docker, it just pulls the container from dockerhub as there is no centralised repository for singularity containers. We have many users who use the singularity container no issue, I'm not sure what you mean by "I can see many things that can go wrong." If you post an error message maybe we could help.
- The install script supports many different install paths, you do not require sudo and you can avoid calling any package manager (using --no-package-manager) if all the Firedrake dependencies are already installed. This information is all available by running python3 firedrake-install --help.
- Creating a pip installable Firedrake is certainly desirable, but is an incredibly involved process to get something working due to our heavy dependency on PETSc and it's compiled external packages.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Apologies!It is just that is taking such a long time... I will try again ...I think there was an issue with the version of CMakeThank you for your help and again my apologies.Regards
Il giovedì 21 marzo 2024 alle ore 14:05:44 GMT, Jack Betteridge ***@***.***> ha scritto:
- The singularity build explicitly does not use docker, it just pulls the container from dockerhub as there is no centralised repository for singularity containers. We have many users who use the singularity container no issue, I'm not sure what you mean by "I can see many things that can go wrong." If you post an error message maybe we could help.
- The install script supports many different install paths, you do not require sudo and you can avoid calling any package manager (using --no-package-manager) if all the Firedrake dependencies are already installed. This information is all available by running python3 firedrake-install --help.
- Creating a pip installable Firedrake is certainly desirable, but is an incredibly involved process to get something working due to our heavy dependency on PETSc and it's compiled external packages.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Hi,
Now I got an MPI error
===================================
2024-03-21 20:31:22,200 DEBUG Running command 'make'
2024-03-21 20:31:31,723 DEBUG Scanning dependencies of target supermesh
[ 1%] Building Fortran object CMakeFiles/supermesh.dir/src/Debug.F90.o
[ 2%] Building Fortran object CMakeFiles/supermesh.dir/src/Graphs.F90.o
[ 4%] Building Fortran object CMakeFiles/supermesh.dir/src/Integer_Set.F90.o
[ 5%] Building Fortran object CMakeFiles/supermesh.dir/src/Intersections.F90.o
[ 7%] Building Fortran object CMakeFiles/supermesh.dir/src/Precision.F90.o
[ 8%] Building Fortran object CMakeFiles/supermesh.dir/src/Quadtree_Intersection_Finder.F90.o
[ 10%] Building Fortran object CMakeFiles/supermesh.dir/src/Octree_Intersection_Finder.F90.o
[ 11%] Building Fortran object CMakeFiles/supermesh.dir/src/Intersection_Finder.F90.o
[ 13%] Building Fortran object CMakeFiles/supermesh.dir/src/Interval_Intersection.F90.o
[ 14%] Building Fortran object CMakeFiles/supermesh.dir/src/Tetrahedron_Intersection.F90.o
[ 15%] Building Fortran object CMakeFiles/supermesh.dir/src/Triangle_Intersection.F90.o
[ 17%] Building Fortran object CMakeFiles/supermesh.dir/src/Supermesh.F90.o
[ 18%] Building Fortran object CMakeFiles/supermesh.dir/src/C_bindings.F90.o
[ 20%] Building Fortran object CMakeFiles/supermesh.dir/src/Read_Halos.F90.o
[ 21%] Building Fortran object CMakeFiles/supermesh.dir/src/Halo_Ownership.F90.o
[ 23%] Building Fortran object CMakeFiles/supermesh.dir/src/Integer_Map.F90.o
[ 24%] Building Fortran object CMakeFiles/supermesh.dir/src/Parallel_Supermesh.F90.o
/campaign/CA-CS1HGN-001/rs3198/firedrake/src/libsupermesh/src/Parallel_Supermesh.F90:494:58:
477 | call MPI_Unpack(recv_buffer, buffer_size, position, nelements, &
| 2
......
494 | call MPI_Unpack(recv_buffer, buffer_size, position, recv_enlist, &
| 1
Error: Rank mismatch between actual argument at (1) and actual argument at (2) (scalar and rank-1)
/campaign/CA-CS1HGN-001/rs3198/firedrake/src/libsupermesh/src/Parallel_Supermesh.F90:496:58:
477 | call MPI_Unpack(recv_buffer, buffer_size, position, nelements, &
| 2
......
496 | call MPI_Unpack(recv_buffer, buffer_size, position, recv_positions, &
| 1
Error: Type mismatch between actual argument at (1) and actual argument at (2) (REAL(8)/INTEGER(4)).
/campaign/CA-CS1HGN-001/rs3198/firedrake/src/libsupermesh/src/Parallel_Supermesh.F90:501:58:
477 | call MPI_Unpack(recv_buffer, buffer_size, position, nelements, &
==============================================================
Do I need a special compilation option or change the MPI version?I am using MPICH/3.3.2-GCC-10.2
Any advice?
Many thanksRoberto============================
Il giovedì 21 marzo 2024 alle ore 13:09:00 GMT, Jack Betteridge ***@***.***> ha scritto:
https://github.com/firedrakeproject/firedrake/wiki/singularity
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Will run it again and attach logs when it crashesThanksRoberto
Il venerdì 22 marzo 2024 alle ore 11:30:24 GMT, Connor Ward ***@***.***> ha scritto:
Can you share the full firedrake-install.log file? Also firedrake/src/petsc/configure.log? Please note that we are also updating our fork of PETSc, ideally today, which includes changes to bump the version of MPICH installed by default.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Hi, I have attached the firedrake-install and configure.log for petsc (I think they installed fine). module load Python/3.8.6-GCCcore-10.2.0 which read gcc==10.2.0 and the end of the error message is: ============================== Error: Type mismatch between actual argument at (1) and actual argument at (2) (REAL(8)/INTEGER(4)). 290 | call MPI_Pack(nelements_send, 1, MPI_INTEGER, & 2024-03-22 14:19:15,045 DEBUG Popping path '/campaign/CA-CS1HGN-001/rs3198/firedrake/src/libsupermesh/build' ================================ Which if I understand it correctly is about the versions of MPI and GCC not being compatible? Please advice |
Beta Was this translation helpful? Give feedback.
-
Before getting into the error it is doing .... =========================== 2024-03-22 14:19:05,862 DEBUG Running command 'make' ================================================================== Roberto |
Beta Was this translation helpful? Give feedback.
-
Hi,
I will do that. Will let you know if that works (or not).ThanksRoberto
Il venerdì 22 marzo 2024 alle ore 15:07:22 GMT, Jack Betteridge ***@***.***> ha scritto:
It seems that there is some issue with the version of CMake or the compilers you're using. CMake should automatically pass the right flags (in this case -fallow-argument-mismatch) to the FORTRAN compiler, but hasn't. You should be able to remedy the situation by:
source /campaign/CA-CS1HGN-001/rs3198/firedrake/bin/activate
cd /campaign/CA-CS1HGN-001/rs3198/firedrake/src/libsupermesh
cmake .. \
-DBUILD_SHARED_LIBS=ON \
-DCMAKE_INSTALL_PREFIX=/campaign/CA-CS1HGN-001/rs3198/firedrake \
-DMPI_C_COMPILER=/apps/x86_64/software/MPICH/3.3.2-GCC-10.2.0/bin/mpicc \
-DMPI_CXX_COMPILER=/apps/x86_64/software/MPICH/3.3.2-GCC-10.2.0/bin/mpicxx \
-DMPI_Fortran_COMPILER=/apps/x86_64/software/MPICH/3.3.2-GCC-10.2.0/bin/mpif90 \
-DCMAKE_Fortran_COMPILER=/apps/x86_64/software/MPICH/3.3.2-GCC-10.2.0/bin/mpif90 \
-DMPIEXEC_EXECUTABLE=/apps/x86_64/software/MPICH/3.3.2-GCC-10.2.0/bin/mpiexec \
-DCMAKE_Fortran_FLAGS="-ffree-line-length-none -std=f2008 -fPIC -fallow-argument-mismatch"
make
make install
You are suitably far through the installation that you can finish by running firedrake-update. Explicitly:
source /campaign/CA-CS1HGN-001/rs3198/firedrake/bin/activate
firedrake-update
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
Success!
I guess these warning are not important. Though do we need to remind users to doexport OMP_NUM_THREADS=1 ?
Please advice
Roberto
========
Il venerdì 22 marzo 2024 alle ore 15:07:22 GMT, Jack Betteridge ***@***.***> ha scritto:
It seems that there is some issue with the version of CMake or the compilers you're using. CMake should automatically pass the right flags (in this case -fallow-argument-mismatch) to the FORTRAN compiler, but hasn't. You should be able to remedy the situation by:
source /campaign/CA-CS1HGN-001/rs3198/firedrake/bin/activate
cd /campaign/CA-CS1HGN-001/rs3198/firedrake/src/libsupermesh
cmake .. \
-DBUILD_SHARED_LIBS=ON \
-DCMAKE_INSTALL_PREFIX=/campaign/CA-CS1HGN-001/rs3198/firedrake \
-DMPI_C_COMPILER=/apps/x86_64/software/MPICH/3.3.2-GCC-10.2.0/bin/mpicc \
-DMPI_CXX_COMPILER=/apps/x86_64/software/MPICH/3.3.2-GCC-10.2.0/bin/mpicxx \
-DMPI_Fortran_COMPILER=/apps/x86_64/software/MPICH/3.3.2-GCC-10.2.0/bin/mpif90 \
-DCMAKE_Fortran_COMPILER=/apps/x86_64/software/MPICH/3.3.2-GCC-10.2.0/bin/mpif90 \
-DMPIEXEC_EXECUTABLE=/apps/x86_64/software/MPICH/3.3.2-GCC-10.2.0/bin/mpiexec \
-DCMAKE_Fortran_FLAGS="-ffree-line-length-none -std=f2008 -fPIC -fallow-argument-mismatch"
make
make install
You are suitably far through the installation that you can finish by running firedrake-update. Explicitly:
source /campaign/CA-CS1HGN-001/rs3198/firedrake/bin/activate
firedrake-update
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
OK,
Thank you very much for your help!Kind regardsRoberto
Il venerdì 22 marzo 2024 alle ore 17:41:01 GMT, Jack Betteridge ***@***.***> ha scritto:
I guess these warning are not important.
What warnings?
Though do we need to remind users to doexport OMP_NUM_THREADS=1
Yes, although really it should be placed in .bashrc so that it is always set and doesn't destroy the performance of other programs they may want to run.
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you commented.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
I have attempted to run the installation for a different user and I got a different error: I believe it is something to do with an incompatible or not found mkl library. Why did it work for the other user though? ======================== Installing h5py Successfully cloned repository h5py Removing existing h5py installations Installing h5py/ ========================================== I attached also the diff obtained comparing the outputs of the env command for the two users Can you make sense of it ? Many thanks Roberto |
Beta Was this translation helpful? Give feedback.
-
/opt/intel/oneapi/mkl/2021.1.1/lib/intel64 |
Beta Was this translation helpful? Give feedback.
-
Now I got this: ==================== Installing tsfc/ ===================================================== These errors seem to be random and indicative of an unreliable script (sorry to say this) Any clue as to what this may be? |
Beta Was this translation helpful? Give feedback.
-
Somehow it turned out to be an issue with environment related to to many users changes. |
Beta Was this translation helpful? Give feedback.
-
Hey There,
I am trying to install Firedrake. I am having issues during installation. the latest one copied below. Please share your advise how to resolve these issues.
'''
running build_ext
PETSC_DIR: /campaign/CA-CS1HGN-001/firedrake/src/petsc
PETSC_ARCH: arch-linux-c-debug
version: 3.20.4 development
integer-size: 32-bit
scalar-type: real
precision: double
language: CONLY
compiler: mpicc
linker: mpicc
building 'PETSc' extension
creating build/temp.linux-x86_64-cpython-38
creating build/temp.linux-x86_64-cpython-38/arch-linux-c-debug
creating build/temp.linux-x86_64-cpython-38/arch-linux-c-debug/src
creating build/temp.linux-x86_64-cpython-38/arch-linux-c-debug/src/petsc4py
mpicc -pthread -fPIC -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -fstack-protector -g3 -O0 -fPIC -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DOPENSSL_NO_SSL3 -DMPICH_SKIP_MPICXX=1 -DOMPI_SKIP_MPICXX=1 -DNPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -Isrc -Isrc/petsc4py/include -I/campaign/CA-CS1HGN-001/firedrake/lib/python3.8/site-packages/numpy/core/include -I/campaign/CA-CS1HGN-001/firedrake/src/petsc/include -I/campaign/CA-CS1HGN-001/firedrake/src/petsc/arch-linux-c-debug/include -I/campaign/CA-CS1HGN-001/firedrake/include -I/campaign/CA-CS1HGN-001/pyenv/versions/3.8.10/include/python3.8 -c src/petsc4py/PETSc.c -o build/temp.linux-x86_64-cpython-38/arch-linux-c-debug/src/petsc4py/PETSc.o
src/petsc4py/PETSc.c: In function ‘__pyx_pymod_exec_PETSc’:
src/petsc4py/PETSc.c:522959:30: note: variable tracking size limit exceeded with ‘-fvar-tracking-assignments’, retrying without
522959 | static CYTHON_SMALL_CODE int __pyx_pymod_exec_PETSc(PyObject *__pyx_pyinit_module)
| ^~~~~~~~~~~~~~~~~~~~~~
creating build/lib.linux-x86_64-cpython-38/petsc4py/lib/arch-linux-c-debug
mpicc -pthread -fPIC -Wall -Wwrite-strings -Wno-unknown-pragmas -Wno-lto-type-mismatch -fstack-protector -g3 -O0 -shared -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O3 -Wall -DOPENSSL_NO_SSL3 -L/campaign/CA-CS1HGN-001/pyenv/versions/3.8.10/lib -Wl,-rpath,/campaign/CA-CS1HGN-001/pyenv/versions/3.8.10/lib -L/campaign/CA-CS1HGN-001/pyenv/versions/3.8.10/lib -Wl,-rpath,/campaign/CA-CS1HGN-001/pyenv/versions/3.8.10/lib build/temp.linux-x86_64-cpython-38/arch-linux-c-debug/src/petsc4py/PETSc.o -L/campaign/CA-CS1HGN-001/firedrake/src/petsc/arch-linux-c-debug/lib -L/campaign/CA-CS1HGN-001/pyenv/versions/3.8.10/lib -Wl,--enable-new-dtags,-R/campaign/CA-CS1HGN-001/firedrake/src/petsc/arch-linux-c-debug/lib -Wl,--enable-new-dtags,-R/campaign/CA-CS1HGN-001/pyenv/versions/3.8.10/lib -lpetsc -o build/lib.linux-x86_64-cpython-38/petsc4py/lib/arch-linux-c-debug/PETSc.cpython-38-x86_64-linux-gnu.so
/bin/ld: cannot find -lpetsc
collect2: error: ld returned 1 exit status
error: command '/campaign/CA-CS1HGN-001/firedrake/bin/mpicc' failed with exit code 1
error: subprocess-exited-with-error
× python setup.py bdist_wheel did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
full command: /campaign/CA-CS1HGN-001/firedrake/bin/python -u -c '
exec(compile('"'"''"'"''"'"'
This is -- a caller that pip uses to run setup.py
- It imports setuptools before invoking setup.py, to enable projects that directly
import from
distutils.core
to work with newer packaging standards.- It provides a clear error message when setuptools is not installed.
- It sets
sys.argv[0]
to the underlyingsetup.py
, when invokingsetup.py
sosetuptools doesn'"'"'t think the script is
-c
. This avoids the following warning:manifest_maker: standard file '"'"'-c'"'"' not found".
- It generates a shim setup.py, for handling setup.cfg-only projects.
import os, sys, tokenize
try:
import setuptools
except ImportError as error:
print(
"ERROR: Can not execute
setup.py
since setuptools is not available in ""the build environment.",
file=sys.stderr,
)
sys.exit(1)
file = %r
sys.argv[0] = file
if os.path.exists(file):
filename = file
with tokenize.open(file) as f:
setup_py_code = f.read()
else:
filename = ""
setup_py_code = "from setuptools import setup; setup()"
exec(compile(setup_py_code, filename, "exec"))
'"'"''"'"''"'"' % ('"'"'/campaign/CA-CS1HGN-001/firedrake/src/petsc/src/binding/petsc4py/setup.py'"'"',), "", "exec"))' bdist_wheel -d /tmp/pip-wheel-z3vwlt8x
cwd: /campaign/CA-CS1HGN-001/firedrake/src/petsc/src/binding/petsc4py/
error
ERROR: Failed building wheel for petsc4py
Running setup.py clean for petsc4py
Running command python setup.py clean
running clean
invalid PETSC_ARCH: arch-linux-c-opt (ignored)
removing 'build/temp.linux-x86_64-cpython-38' (and everything under it)
removing 'build/lib.linux-x86_64-cpython-38' (and everything under it)
'build/bdist.linux-x86_64' does not exist -- can't clean it
'build/scripts-3.8' does not exist -- can't clean it
removing 'build'
Failed to build petsc4py
ERROR: Could not build wheels for petsc4py, which is required to install pyproject.toml-based projects
Exception information:
Traceback (most recent call last):
File "/campaign/CA-CS1HGN-001/firedrake/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
status = run_func(*args)
File "/campaign/CA-CS1HGN-001/firedrake/lib/python3.8/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
return func(self, options, args)
File "/campaign/CA-CS1HGN-001/firedrake/lib/python3.8/site-packages/pip/_internal/commands/install.py", line 429, in run
raise InstallationError(
pip._internal.exceptions.InstallationError: Could not build wheels for petsc4py, which is required to install pyproject.toml-based projects
Remote version of pip: 24.0
Local version of pip: 24.0
Was pip installed by pip? True
Removed build tracker: '/tmp/pip-build-tracker-brrp7awq'
I am sharing configure.log and firedrake-install.log for your reference
configure.log
firedrake-install.log
firedrake-update.log
.
Kind Regards
Beta Was this translation helpful? Give feedback.
All reactions