Compiling on M1 Mac

[wilrop]

Hi,

First of all, I would like to stress that I'm very new at using C/C++/Ada/Fortran and have limited experience in actually building software. My main "expertise" is in Python. I've been toying around with the Python interface of the library on the online portal and wanted to install it locally for usage in my research. However, I have a new MacBook Pro which runs on the M1 processor and it appears as though the compilation process just will not work. The exact problem:

• I can build and install the QD library for the arm64 architecture
• When I update the makefile, I run make phcpy2c3.so
• This builds everything but crashes at what I believe is one of the final steps namely: ar ruv phcpy2c3.a *.o

The compiler error is that phcpy2c3.a is a FAT file and has two conflicting architectures in it. See below for a complete error message:

ar: phcpy2c3.a is a fat file (use libtool(1) or lipo(1) and ar(1) on it)
ar: phcpy2c3.a: Inappropriate file type or format
make[1]: *** [phcpy2c3.so] Error 1
make: *** [phcpy2c3.so] Error 2

The problem is that some parts appear to be compiled to run natively on arm64, while others are compiled to x86_64.

Have you ever encountered this problem before and what can be done to resolve it? Also, if it helps, it also fails to compile make hello (with a different error) so I'm probably doing something very elementary wrong. The error when building hello is the following:

gnatlink hello.ali -o ../bin/hello
ld: library not found for -lSystem
collect2: error: ld returned 1 exit status
gnatlink: error when calling /Users/willemropke/opt/GNAT/2020/bin/gcc
gnatmake: *** link failed.
make[1]: *** [hello] Error 4
make: *** [hello] Error 2

[janverschelde]

Thanks for reporting this issue. I do not have an M1 Mac so I cannot replicate your problem.
From your output on the "make hello" it seems the GNAT 2020 Community edition was used,
but this version work will not work for the recent M1 Macs.
One can also install gcc on macs with brew, but I am not sure if that installation will contain gnatmake.

[wilrop]

The GNAT version should work, as it can compile to x86_64 which is able to run on the newer macs. I will just try some more and post a solution here if I ever find one so that others may avoid this problem in the future.

As a side note, have you ever considered publishing a pip package for phcpy or is that simply impossible with the current architecture?

[wilrop]

Hi contributors and future phcpy users with an M1 MacBook!

I have managed to build and install everything that was needed and I can now import phcpy into a Python terminal. I am going to briefly explain the steps that I’ve taken to accomplish this, to hopefully save others time in the future. In addition, I think I’ve found some bugs in the project that should be resolved.

Bugs

• The official final release points to (I think) an outdated PHCpack version. The reason I say this is because of this issue: "error: reference to 'data' is ambiguous" when building with C++17 #55. When compiling the latest release with gcc-11, it will immediately crash. I resolved this issue by downloading the source from GitHub. I think a better solution would be to update the latest release, but I’m not sure if the maintainer(s) have an official release schedule they are following.
• In src/Objects/makefile_mac line 3671: $(PHCLib)/phcpy2c3.c is given as a flag but phcpy2c3.c is not actually present in $(PHCLib), at least not in the current source as available on GitHub. I fixed this by copying the phcpy2c3.c from the Mod folder to the C folder.
• In src/Objects/makefile_mac line 3671 as well as 3678 to 3685: Multiple times there are flags given that look something like -I/opt/miniconda3 or -L/opt/miniconda3/lib but I had to change these to my own folder for the Python version of my choosing. Given that this happens multiple times, maybe /opt/miniconda3 could be a user defined folder. Alternatively the PYTHON38 variable that is already defined, but now points to the actual python program, could be set to the top level python directory. This way all other subfolders and files can be inferred.
• In src/Objects/makefile_mac line 3677: g++ is used but this should be $(gpp) to use the user provided g++ compiler. This actually proved to be the final problem in my build process.

Note that I am completely open to these issues not actually being bugs but my own mistakes. I still listed them in the hopes of improving the library for everyone.

Build and installation steps

I will now give a quick overview of the build and installation steps I took to install phcpy on my 2021 MacBook Pro with the M1 chip. I am running on macOS Monterey 12.1.

First of all, it’s important to highlight that because these MacBooks work on a newer chip architecture, it is impossible to run everything natively. Apple has however foreseen this and written a really good emulator (Rosetta) that can run Intel apps. Therefore, throughout the build process you should compile everything for x86_64 and not arm64 or aarch64. Having a subset built for x86_64 and another section for arm64 will not work and give you headaches.

• First of all, you should install the latest GNAT Community Edition from AdaCore that works on mac. I downloaded “x86 Mac OS X (64 bits)”.
• Add GNAT to your path. I use ZSH so it can be done by adding this at the top of your .zshrc file: path+=('/Users/willemropke/opt/GNAT/2020/bin')
• Install the latest version of GCC, g++ and gfortran that run on macs with an intel chip. I downloaded it here: https://github.com/fxcoudert/gfortran-for-macOS/releases
• Make sure your Python version is built for x86_64. I use Anaconda to manage my Python environments and luckily they do this for you.
• Build QD using the following configuration: ./configure CXX=/usr/local/gfortran/bin/g++ CXXFLAGS='-fPIC -O3' CC=/usr/local/gfortran/bin/gcc FC=/usr/local/gfortran/bin/gfortran Ofcourse, you have to change the exact location but importantly, use the GCC, g++ and gfortran compilers that build for x86_64.
• Set the variables in makefile_mac also to the x86_64 compilers. If the above problems under "Bugs" have not been resolved, you should also fix them yourself.
• Normally everything should work now

Good luck to everyone and thanks for creating this library. I'm hoping to put it to good use now!

[janverschelde]

Thank you very much for reporting the bugs and the installation steps for the M1.
I will fix the bugs in the next release.

[EliseAWalker]

Hi, I wanted to see what the latest is in installing phcpy on M1s. I am not versed in compiling anything from source, and working with an M1 makes everything more complicated. So far I have downloaded the executable macairM1_phcv24p.tar.gz from release v2.4.88, and I have it working just fine with Macaualy2. However, I want to integrate PHCpack in with python, so I need phcpy. From the documentation it looks like I need to compile from source to make the .dylib library. I cloned the repo, changed any "windows" reference in the config folder to "macosx" and tried 'alr build' on the repo. The library builds fine, and moving the .dylib into PHCpack/src/Python/PHCpy and running 'pip install .' technically also builds fine. However, when I import the phcpy library into python, it's clear that something went amiss as I get a warning that the library is not suitable for my architecture, and then errors when I try to solve anything.
I figure things went wrong for a few reasons - one is that alr probably is not suitable for M1s. Is there a different command/software that I can use to build the .dylib?
The second reason things went wrong is that phcpy might not be ready for M1s, and so I need to force everything to use x86_64 instead of arm64, as suggested by wilrop. Is this still true? I am a little wary of these instructions, as my brew and conda are set up to use miniforge, which does pull from arm64 compilations when possible. This is a behavior that I want, so I am hesitant to install anything that would conflict with using the latest arm64 compilations. Is there a way to separate x86_64 builds from arm64, other than perhaps a Docker container?
Again, I am not well versed in library management and compilation from source, so any tips would be appreciated. Thanks!

[janverschelde]

The libPHCpack.dylib has been made successfully to work on M1 mac, using alr 1.2.2, provided by Simon Wright, available at

https://github.com/simonjwright

The encapsulated library is not supported, commenting in phclib.gpr out the line

for Library_Standalone use "encapsulated";

made libPHCpack work in phcpy on the M1.