Compilation of ExaFMM on MacOS

[DGCaprace]

Compilation of ExaFMM using GCC triggers an unstable behavior of the related package.

After compiling CxxWrap and ExaFMM with GCC v10, I encountered the following. In a fresh Julia REPL, I can do

julia> import FLOWExaFMM
julia> fmm = FLOWExaFMM
FLOWExaFMM
julia> fmm.greet()
"hello, world"

(or equivalently ] test FLOWExaFMM) which seems to show that the package is ready and working. However, when proceeding further with FLOWVPM, this occurs:

[ Info: Precompiling FLOWVPM [6e19019d-7c31-4940-9d16-c3f15dfe6020]
julia(73232,0x114c47dc0) malloc: *** error for object 0x12bc1eb60: pointer being freed was not allocated
julia(73232,0x114c47dc0) malloc: *** set a breakpoint in malloc_error_break to debug
signal (6): Abort trap: 6
in expression starting at /Users/dg/.julia/packages/FLOWVPM/qfQKX/src/FLOWVPM.jl:27
__pthread_kill at /usr/lib/system/libsystem_kernel.dylib (unknown line)
Allocations: 7799290 (Pool: 7795337; Big: 3953); GC: 9

Guidelines for compilation under MacOS could be included in the documentation.

[DGCaprace]

We recommend the use of LLVM Clang to compile the binaries of ExaFMM. The manual procedure explained here can be adapted as follows. An automated procedure is also under development (see branch macos ).

1. Install LLVM suite (e.g. brew install llvm), and make sure you have an OpenMPI wrapper pointing to CLang. You will also need cmake. Note: the default system install of Clang does not support openMP.

EDIT: MPI in not required per se, see below.

2. (This step is not mandatory, but will ensure the compatibility between the compiled ExaFMM and the Julia binding library). Clone libcxxwrap-julia and build it with llvm. Inspired from instructions here, you want to do something like:

git clone https://github.com/JuliaInterop/libcxxwrap-julia.git
cd libcxxwrap-julia 
mkdir build_llvm && cd build_llvm
CC=/path/to/clang CXX=/path/to/clang++ cmake -DJulia_EXECUTABLE=/path/to/julia/bin/julia  ..
cmake --build . --config Release

Then, you must edit ~/.julia/artifacts/Overrides.toml (or create it if needed), and add:

[3eaa8342-bff7-56a5-9981-c04077f7cee7]
libcxxwrap_julia = "/path/to/libcxxwrap-julia/build_llvm"

3. Clone FLOWExaFMM and compile the binaries:

EDIT: first open FLOWExaFMM/deps/3d/makefile.in and edit lines 580-583 (see discussion below):

JLCXX_H=/path/to/libcxxwrap-julia/include
JULIA_H=/path/to/julia/1.6.2_1/include/julia
JLCXX_LIB=/path/to/libcxxwrap-julia/build_llvm/lib
JULIA_LIB=/path/to/julia/1.6.2_1/lib

This should match the location where you installed Julia and what you did in step 2.

Then,

cd FLOWExaFMM/deps
CC=/path/to/clang CXX=/path/to/clang++ MPICXX=/path/to/mpicxx LDFLAGS="-L/path/to/llvm/lib -Wl,-rpath,/path/to/llvm/lib" ./configure
cd 3d
make
cp hmm ../../src/fmm.dylib

EDIT: if you are having troubles related to MPI, add --disable-mpi after ./configure. This will not affect our execution and gets rid of the dependency.

4. Start Julia and try adding and testing the FLOWExaFMM package:

] add FLOWExaFMM
] test FLOWExaFMM

This should return a "hello, world" message. If you get an error for a missing fmm library in the ~/.julia/packages directory, you must copy /path/to/FLOWExaFMM/src/fmm.dylib to the corresponding directory under ~/.julia.

[EdoAlvarezR]

NOTE TO SELF: Use the flag -march=broadwell when compiling for the Fulton supercomputer m9 nodes.

[Eric-P-Green]

When I run the configure line in step 3, I get an error:

...
checking for OpenMP flag of C++ compiler... -fopenmp
checking for mpiicpc... //usr/local/Cellar/open-mpi/4.1.1_2/share/man/man1/mpicxx.1
checking for mpxlf90_rftn... no
checking for mpiifort... no
checking for mpif90... mpif90
checking for MPI_Init... no
checking for MPI_Init in -lmpi... no
checking for MPI_Init in -lmpich... no
checking for MPI_Init in -lmpicxx... no
checking for MPI_Init in -lfmpi... no
checking for MPI_Init in -lmpichf90... no
checking for MPI_Init in -lmpi_cxx... no
configure: error: could not find mpi library for --enable-mpi

This is probably related to the instructions from the first step that indicate that OpenMPI should point to CLang, which I was unable to figure out how to do.

---

After working with DeeGee, here are the solutions we found:

First, if OpenMPI causes problems, add --disable-mpi to the configure command in step 3. eg
CC=/path/to/clang CXX=/path/to/clang++ MPICXX=/path/to/mpicxx LDFLAGS="-L/path/to/llvm/lib -Wl,-rpath,/path/to/llvm/lib" ./configure --disable-mpi

After running this command, the makefile in FLOWExaFMM/deps/3d needs to be modified to work properly.
Lines 580-583 should be modified to match the file paths described in the Linux installation guide. For me, this looks like:

JLCXX_H=/Users/ericgreen/libcxxwrap-julia/include
JULIA_H=/usr/local/Cellar/julia/1.6.2_1/include/julia
JLCXX_LIB=/Users/ericgreen/libcxxwrap-julia/build_llvm/lib
JULIA_LIB=/usr/local/Cellar/julia/1.6.2_1/lib

Lines 585-592 should be changed to match the following:

cxxwrap_OBJFLAGS = -DJULIA_ENABLE_THREADING -Dhello_EXPORTS
cxxwrap_OBJFLAGS += -I${JLCXX_H} -I${JULIA_H}
cxxwrap_OBJFLAGS += -march=native -Wunused-parameter -Wextra -Wreorder -std=gnu++1z -O3 -DNDEBUG -fPIC

cxxwrap_LINKFLAGS = -L${JLCXX_LIB} -lcxxwrap_julia
# Comments this line out to make it executable
cxxwrap_LINKFLAGS += -fPIC  -march=native -Wunused-parameter -Wextra -Wreorder -std=gnu++1z -O3 -DNDEBUG  -shared
cxxwrap_LINKFLAGS += -Wl,-rpath,: -L${JLCXX_LIB} -lcxxwrap_julia -L${JULIA_LIB} -ljulia

Finally, the installation location for FLOWExaVPM needs to be /.julia/packages so that the Julia compiler reads it.

[DGCaprace]

Thanks for documenting this Eric!
I made a commit on FLOWExaFMM so that the change at line 585-592 is built in.
I also edited my message above so that the full process is up to date if anyone needs it.
Though, I think @rymanderson was also working on an automated builder with Julia.

[tylercritchfield]

Compilation notes on my mac with an M2 chip with Ventura 13.2.1:

• make sure llvm is installed: brew install llvm
• in build_macos.sh:
  • I was using a Julia environment for the entire FLOWUnsteady installation, so line 8 did not work for me. I tried including commands to activate the environment using import Pkg; Pkg.activate("path_to_environment") but couldn't get that to work either, so I manually found this path and inserted it in line 8.
  • make these line changes due to using homebrew for other installations (at least this is what worked for me):
    • line 3: CXX=/opt/homebrew/bin/gcc-12 (the version number is the key change here)
    • line 23: MPIHOME=/opt/homebrew/bin/mpicxx
    • line 26: LDFLAGS="-L/opt/homebrew/opt/llvm/lib" (this should be the default, you can ignore the other commented out options in lines 27-28)
  • fix typo in line 40 from .so to .dylib
  • then in the configure step in line 48, use this option: ./configure CC=$CC CXX=$CXX MPICXX=$MPIHOME LDFLAGS=$LDFLAGS (you can comment out line 48 and uncomment line 50)
• add to ~/.zprofile:
  • export PATH="/opt/homebrew/bin/mpicxx:$PATH"
  • export PATH="/opt/homebrew/opt/llvm/bin:$PATH"
• then finally build in a new terminal window: sh build_macos.sh

[EdoAlvarezR]

@tylercritchfield , did you do the export step that brew asks for after installing llvm?

[tylercritchfield]

Yes, I did include that first export PATH... line to my ~/.zprofile - I forgot to include that earlier but it's there now in the comment above. As for the other two - the LDFLAGS one is verbatim line 26 in my build_macos.sh so I didn't do it again. I don't think I did the last one at all as it doesn't show up in my terminal history.

[EdoAlvarezR]

@tylercritchfield, would you mind running sh build_macos.sh and putting here the verbose that you see in the screen?

After following your instructions I'm still getting clang: fatal error: unsupported option '-fopenmp':

Rachels-MBP:FLOWExaFMM.jl edoalvar$ sh build_macos.sh 
Removing existing build
rm: src/fmm.dylib: No such file or directory
Copying files
Configuring build
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... build-aux/install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether /opt/homebrew/bin/gcc-12/ accepts -g... yes
checking for style of include used by make... GNU
checking dependency style of /opt/homebrew/bin/gcc-12/... gcc3
checking for ftn... no
checking for ifort... no
checking for gfortran... gfortran
checking whether we are using the GNU Fortran compiler... yes
checking whether gfortran accepts -g... yes
checking for ranlib... ranlib
checking for gcc... gcc
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking for OpenMP flag of C++ compiler... -fopenmp
checking for mpiicpc... /opt/homebrew/bin/mpicxx
checking for mpxlf90_rftn... no
checking for mpiifort... no
checking for mpif90... mpif90
checking for MPI_Init... yes
checking for MPI_Init in -lmpi... yes
checking for MPI_Init in -lmpich... no
checking for MPI_Init in -lmpicxx... no
checking for MPI_Init in -lfmpi... no
checking for MPI_Init in -lmpichf90... no
checking for MPI_Init in -lmpi_cxx... no
checking for mpirun... mpirun
checking build system type... arm-apple-darwin20.6.0
checking host system type... arm-apple-darwin20.6.0
checking for x86 cpuid  output... unknown
checking for x86-AVX xgetbv  output... unknown
checking for C++ compiler vendor... clang
checking for C++ compiler vendor... (cached) clang
checking for a sed that does not truncate output... /usr/bin/sed
checking whether C++ compiler accepts -Werror=unknown-warning-option... yes
checking whether C++ compiler accepts -diag-error warn... no
checking whether C++ compiler accepts -O0... yes
checking whether C++ compiler accepts -g... yes
checking whether C++ compiler accepts -Werror=unknown-warning-option... yes
checking whether C++ compiler accepts -check all... no
checking whether C++ compiler accepts -debug all... no
checking whether C++ compiler accepts -diag-disable remark... no
checking whether C++ compiler accepts -fmudflap... yes
checking whether C++ compiler accepts -fno-strict-aliasing... yes
checking whether C++ compiler accepts -fsanitize=address... yes
checking whether C++ compiler accepts -fsanitize=leak... no
checking whether C++ compiler accepts -fstack-protector... yes
checking whether C++ compiler accepts -ftrapuv... no
checking whether C++ compiler accepts -ftrapv... yes
checking whether C++ compiler accepts -traceback... no
checking whether C++ compiler accepts -Wall... yes
checking whether C++ compiler accepts -Warray-bounds... yes
checking whether C++ compiler accepts -Wbad-function-cast... yes
checking whether C++ compiler accepts -Wcast-align... yes
checking whether C++ compiler accepts -Wcast-qual... yes
checking whether C++ compiler accepts -Wextra... yes
checking whether C++ compiler accepts -Wfatal-errors... yes
checking whether C++ compiler accepts -Wformat=2... yes
checking whether C++ compiler accepts -Wformat-nonliteral... yes
checking whether C++ compiler accepts -Wformat-security... yes
checking whether C++ compiler accepts -Winit-self... yes
checking whether C++ compiler accepts -Winline... yes
checking whether C++ compiler accepts -Wmissing-format-attribute... yes
checking whether C++ compiler accepts -Wmissing-include-dirs... yes
checking whether C++ compiler accepts -Wmissing-noreturn... yes
checking whether C++ compiler accepts -Wnested-externs... yes
checking whether C++ compiler accepts -Wno-missing-field-initializers... yes
checking whether C++ compiler accepts -Wno-overloaded-virtual... yes
checking whether C++ compiler accepts -Wno-unused-local-typedefs... yes
checking whether C++ compiler accepts -Wno-unused-parameter... yes
checking whether C++ compiler accepts -Wno-unused-variable... yes
checking whether C++ compiler accepts -Wpointer-arith... yes
checking whether C++ compiler accepts -Wredundant-decls... yes
checking whether C++ compiler accepts -Wreturn-type... yes
checking whether C++ compiler accepts -Wshadow... yes
checking whether C++ compiler accepts -Wsign-compare... yes
checking whether C++ compiler accepts -Wstrict-aliasing... yes
checking whether C++ compiler accepts -Wstrict-overflow=5... yes
checking whether C++ compiler accepts -Wstrict-prototype... no
checking whether C++ compiler accepts -Wswitch-enum... yes
checking whether C++ compiler accepts -Wuninitialized... yes
checking whether C++ compiler accepts -Wunreachable-code... yes
checking whether C++ compiler accepts -Wunused-but-set-variable... no
checking whether C++ compiler accepts -Wwrite-strings... yes
checking whether C++ compiler accepts -Wno-error=missing-field-initializers... yes
checking whether C++ compiler accepts -Wno-error=overloaded-virtual... yes
checking whether C++ compiler accepts -Wno-error=unused-local-typedefs... yes
checking whether C++ compiler accepts -Wno-error=unused-parameter... yes
checking whether C++ compiler accepts -Wno-error=unused-variable... yes
configure: OpenMP   : yes
configure: MPI      : yes
configure: CUDA     : yes
configure: FP32     : no
configure: SIMD     : 
configure: Assert   : yes
configure: Debug    : no
configure: Vendor   : clang
configure: CXX      : /opt/homebrew/bin/mpicxx
configure: FC       : mpif90
configure: CXXFLAGS : -g -O2 
configure: FCFLAGS  : -g -O2
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating 3d/Makefile
config.status: creating config.h
config.status: executing depfiles commands
Compiling 3d
/opt/homebrew/bin/mpicxx -DHAVE_CONFIG_H -DJULIA_ENABLE_THREADING -Dhello_EXPORTS -I/Users/edoalvar/.julia/artifacts/434257acf70ce7407e3946f2c8452e814d7cca64/include -I/Users/edoalvar/Programs/Julia-1.8.app/Contents/Resources/julia/include/julia -Wunused-parameter -Wextra -Wreorder -std=gnu++1z -O3 -DNDEBUG -fPIC   -I. -I..  -DEXAFMM_WITH_OPENMP   -DNDEBUG -DEXAFMM_EAGER  -ffast-math -funroll-loops -Wfatal-errors -fopenmp  -g -O2  -MT fmm-fmm.o -MD -MP -MF .deps/fmm-fmm.Tpo -c -o fmm-fmm.o `test -f 'fmm.cxx' || echo './'`fmm.cxx
clang: fatal error: unsupported option '-fopenmp'
make: *** [fmm-fmm.o] Error 1
cp: build/3d/fmm: No such file or directory
Done!

[tylercritchfield]

@tylercritchfield, would you mind running sh build_macos.sh and putting here the verbose that you see in the screen?

tcritchfield@tylers-mbp FLOWExaFMM % sh build_macos.sh
Removing existing build
Copying files
Configuring build
checking for a BSD-compatible install... /usr/bin/install -c
checking whether build environment is sane... yes
checking for a thread-safe mkdir -p... build-aux/install-sh -c -d
checking for gawk... no
checking for mawk... no
checking for nawk... no
checking for awk... awk
checking whether make sets $(MAKE)... yes
checking whether make supports nested variables... yes
checking whether to enable maintainer-specific portions of Makefiles... no
checking whether the C++ compiler works... yes
checking for C++ compiler default output file name... a.out
checking for suffix of executables... 
checking whether we are cross compiling... no
checking for suffix of object files... o
checking whether we are using the GNU C++ compiler... yes
checking whether /opt/homebrew/bin/gcc-12 accepts -g... yes
checking for style of include used by make... GNU
checking dependency style of /opt/homebrew/bin/gcc-12... gcc3
checking for ftn... no
checking for ifort... no
checking for gfortran... gfortran
checking whether we are using the GNU Fortran compiler... yes
checking whether gfortran accepts -g... yes
checking for ranlib... ranlib
checking for gcc... gcc
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking whether gcc understands -c and -o together... yes
checking dependency style of gcc... gcc3
checking for ar... ar
checking the archiver (ar) interface... ar
checking for OpenMP flag of C++ compiler... -fopenmp
checking for mpiicpc... /opt/homebrew/bin/mpicxx
checking for mpxlf90_rftn... no
checking for mpiifort... no
checking for mpif90... mpif90
checking for MPI_Init... yes
checking for MPI_Init in -lmpi... yes
checking for MPI_Init in -lmpich... no
checking for MPI_Init in -lmpicxx... no
checking for MPI_Init in -lfmpi... no
checking for MPI_Init in -lmpichf90... no
checking for MPI_Init in -lmpi_cxx... no
checking for mpirun... mpirun
checking build system type... arm-apple-darwin22.3.0
checking host system type... arm-apple-darwin22.3.0
checking for x86 cpuid  output... unknown
checking for x86-AVX xgetbv  output... unknown
checking for C++ compiler vendor... clang
checking for C++ compiler vendor... (cached) clang
checking for a sed that does not truncate output... /usr/bin/sed
checking whether C++ compiler accepts -Werror=unknown-warning-option... yes
checking whether C++ compiler accepts -diag-error warn... no
checking whether C++ compiler accepts -O0... yes
checking whether C++ compiler accepts -g... yes
checking whether C++ compiler accepts -Werror=unknown-warning-option... yes
checking whether C++ compiler accepts -check all... no
checking whether C++ compiler accepts -debug all... no
checking whether C++ compiler accepts -diag-disable remark... no
checking whether C++ compiler accepts -fmudflap... yes
checking whether C++ compiler accepts -fno-strict-aliasing... yes
checking whether C++ compiler accepts -fsanitize=address... yes
checking whether C++ compiler accepts -fsanitize=leak... yes
checking whether C++ compiler accepts -fstack-protector... yes
checking whether C++ compiler accepts -ftrapuv... no
checking whether C++ compiler accepts -ftrapv... yes
checking whether C++ compiler accepts -traceback... no
checking whether C++ compiler accepts -Wall... yes
checking whether C++ compiler accepts -Warray-bounds... yes
checking whether C++ compiler accepts -Wbad-function-cast... yes
checking whether C++ compiler accepts -Wcast-align... yes
checking whether C++ compiler accepts -Wcast-qual... yes
checking whether C++ compiler accepts -Wextra... yes
checking whether C++ compiler accepts -Wfatal-errors... yes
checking whether C++ compiler accepts -Wformat=2... yes
checking whether C++ compiler accepts -Wformat-nonliteral... yes
checking whether C++ compiler accepts -Wformat-security... yes
checking whether C++ compiler accepts -Winit-self... yes
checking whether C++ compiler accepts -Winline... yes
checking whether C++ compiler accepts -Wmissing-format-attribute... yes
checking whether C++ compiler accepts -Wmissing-include-dirs... yes
checking whether C++ compiler accepts -Wmissing-noreturn... yes
checking whether C++ compiler accepts -Wnested-externs... yes
checking whether C++ compiler accepts -Wno-missing-field-initializers... yes
checking whether C++ compiler accepts -Wno-overloaded-virtual... yes
checking whether C++ compiler accepts -Wno-unused-local-typedefs... yes
checking whether C++ compiler accepts -Wno-unused-parameter... yes
checking whether C++ compiler accepts -Wno-unused-variable... yes
checking whether C++ compiler accepts -Wpointer-arith... yes
checking whether C++ compiler accepts -Wredundant-decls... yes
checking whether C++ compiler accepts -Wreturn-type... yes
checking whether C++ compiler accepts -Wshadow... yes
checking whether C++ compiler accepts -Wsign-compare... yes
checking whether C++ compiler accepts -Wstrict-aliasing... yes
checking whether C++ compiler accepts -Wstrict-overflow=5... yes
checking whether C++ compiler accepts -Wstrict-prototype... no
checking whether C++ compiler accepts -Wswitch-enum... yes
checking whether C++ compiler accepts -Wuninitialized... yes
checking whether C++ compiler accepts -Wunreachable-code... yes
checking whether C++ compiler accepts -Wunused-but-set-variable... yes
checking whether C++ compiler accepts -Wwrite-strings... yes
checking whether C++ compiler accepts -Wno-error=missing-field-initializers... yes
checking whether C++ compiler accepts -Wno-error=overloaded-virtual... yes
checking whether C++ compiler accepts -Wno-error=unused-local-typedefs... yes
checking whether C++ compiler accepts -Wno-error=unused-parameter... yes
checking whether C++ compiler accepts -Wno-error=unused-variable... yes
configure: OpenMP   : yes
configure: MPI      : yes
configure: CUDA     : yes
configure: FP32     : no
configure: SIMD     : 
configure: Assert   : yes
configure: Debug    : no
configure: Vendor   : clang
configure: CXX      : /opt/homebrew/bin/mpicxx
configure: FC       : mpif90
configure: CXXFLAGS : -g -O2 
configure: FCFLAGS  : -g -O2
checking that generated files are newer than configure... done
configure: creating ./config.status
config.status: creating Makefile
config.status: creating 3d/Makefile
config.status: creating config.h
config.status: executing depfiles commands
Compiling 3d
/opt/homebrew/bin/mpicxx -DHAVE_CONFIG_H -DJULIA_ENABLE_THREADING -Dhello_EXPORTS -I/Users/tcritchfield/.julia/artifacts/8bc680b87ac3437cdc7ed58f906f8a09e033d6e9/include -I/Applications/Julia-1.8.app/Contents/Resources/julia/include/julia -Wunused-parameter -Wextra -Wreorder -std=gnu++1z -O3 -DNDEBUG -fPIC   -I. -I..  -DEXAFMM_WITH_OPENMP   -DNDEBUG -DEXAFMM_EAGER  -ffast-math -funroll-loops -Wfatal-errors -fopenmp  -g -O2  -MT fmm-fmm.o -MD -MP -MF .deps/fmm-fmm.Tpo -c -o fmm-fmm.o `test -f 'fmm.cxx' || echo './'`fmm.cxx
In file included from fmm.cxx:22:
In file included from ./exafmm.h:9:
./complexify.h:35:34: warning: unused parameter 'r' [-Wunused-parameter]
inline real_t imag(const real_t& r) {
                                 ^
In file included from fmm.cxx:23:
./args.h:100:9: warning: field 'verbose' will be initialized after field 'phi' [-Wreorder-ctor]
        verbose(1),
        ^~~~~~~~~~
        phi(.5)
In file included from fmm.cxx:25:
./dataset.h:50:29: warning: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'int' [-Wsign-compare]
      for (size_t b=begin; b!=end; ++b) {
                           ~^ ~~~
./dataset.h:67:29: warning: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'int' [-Wsign-compare]
      for (size_t b=begin; b!=end; ++b) {
                           ~^ ~~~
./dataset.h:86:29: warning: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'int' [-Wsign-compare]
      for (size_t b=begin; b!=end; ++b) {
                           ~^ ~~~
./dataset.h:106:16: warning: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'int' [-Wsign-compare]
      while (b != end) {
             ~ ^  ~~~
./dataset.h:137:31: warning: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'int' [-Wsign-compare]
        for (size_t b=begin; b!=end; ++b) {
                             ~^ ~~~
./dataset.h:142:31: warning: comparison of integers of different signs: 'size_t' (aka 'unsigned long') and 'int' [-Wsign-compare]
        for (size_t b=begin; b!=end; ++b) {
                             ~^ ~~~
8 warnings generated.
mv -f .deps/fmm-fmm.Tpo .deps/fmm-fmm.Po
/opt/homebrew/bin/mpicxx -ffast-math -funroll-loops -Wfatal-errors -fopenmp  -g -O2   -L/opt/homebrew/opt/llvm/lib -o fmm fmm-fmm.o   -L/Users/tcritchfield/.julia/artifacts/8bc680b87ac3437cdc7ed58f906f8a09e033d6e9/include/../lib -lcxxwrap_julia -fPIC  -Wunused-parameter -Wextra -Wreorder -std=gnu++1z -O3 -DNDEBUG  -shared -Wl,-rpath,: -L/Users/tcritchfield/.julia/artifacts/8bc680b87ac3437cdc7ed58f906f8a09e033d6e9/include/../lib -lcxxwrap_julia  -L/Applications/Julia-1.8.app/Contents/Resources/julia/include/julia/../../lib -ljulia
Done!

[khuspreet13]

@tylercritchfield What is the version of julia are you using?

[khuspreet13]

OMP: Error #15: Initializing libomp.dylib, but found libomp.dylib already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/

This is the error I am getting when I run the example cases for FLOWUnsteady. What are the possible solutions for this?

[kevmoor]

OMP: Error #15: Initializing libomp.dylib, but found libomp.dylib already initialized. OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/

This is the error I am getting when I run the example cases for FLOWUnsteady. What are the possible solutions for this?

I had this same issue, I had tried to use the python installation behind .julia as opposed to the top level python, and installed the suggested python packages etc. When I switched, per the installation instructions for setting up pycall, it worked - precompilation was broken, but it all worked including good parallel resource utilization on my m1 processor.

[EdoAlvarezR]

Thanks for sharing your solution, Kevin! Also, it's great to hear from you!