How to get path to cibuildwheel's Python location?

[DBraun]

If on macos-latest, I do

- name: Setup Python
  uses: actions/setup-python@v2
  with:
    python-version: 3.8

I can refer to ${{env.pythonLocation}} to get /Library/Frameworks/Python.framework/Versions/3.8

Suppose that after setting up Python like that, I use cibuildwheel with CIBW_BUILD: cp39-macosx_universal2. Then in CIBW_BEFORE_ALL_MACOS I want to call xcodebuild -configuration Release -project my-project.xcodeproj. This project needs to link against the Python 3.9 headers and libraries that come with cibuildwheel (this is the easiest why to make universal builds I guess...). Can I get an environment variable that's easy to use like ${{env.pythonLocation}}?

If I do

export pythonLocation=$(python3-config --prefix)
echo "pythonLocation: ${{env.pythonLocation}}" 

inside CIBW_BEFORE_ALL_MACOS it shows pythonLocation: /Users/runner/hostedtoolcache/Python/3.8.12/x64 which is the actions/setup-python@v2 location rather than cibuildwheel's location

[joerick]

python3-config --prefix looks right to me. But ${{env.pythonLocation}} is Github Actions syntax, which might be expanded before the script is run.

In your test script, try doing echo "pythonLocation: $pythonLocation" instead.

[DBraun]

Thanks, good suggestion. In order to not overlap with pythonLocation from setup-python, I tried to export pythonLocation2:

CIBW_BEFORE_ALL_MACOS: |
    echo $(which python3-config)
    export pythonLocation2=$(python3-config --prefix)
    echo "pythonLocation2 is $pythonLocation2"

and got

/Users/runner/hostedtoolcache/Python/3.8.12/x64/bin/python3-config
pythonLocation2 is /Users/runner/hostedtoolcache/Python/3.8.12/x64

But it was a Python 3.10 cibuildwheel choice. So python3-config resolves to the wrong one inside CIBW_BEFORE_ALL_MACOS. Any other suggestions?

Update:
As @mayeut said below, it works if it's inside CIBW_BEFORE_BUILD

[mayeut]

You should be able to get it in CIBW_BEFORE_BUILD but not with CIBW_BEFORE_ALL.
Per documentation:

On Windows and macOS, the version of Python available inside CIBW_BEFORE_ALL is whatever is available on the host machine. On Linux, a modern Python version is available on PATH.

[DBraun]

What would be the equivalent for Windows with GitHub Actions?

CIBW_BEFORE_BUILD: |
  set pythonLocation=$(python3-config --prefix)

won't work because python3-config isn't available right?

[DBraun]

Edit: With version cibuildwheel@2.3.1, I tried

CIBW_BEFORE_BUILD: >
  echo "Saving Python location to tmp_pythonLocation.txt" &&
  python -c "import os, sys; print(os.path.dirname(sys.executable))" > tmp_pythonLocation.txt &&
  set /p pythonLocation=<tmp_pythonLocation.txt

and that might be a sufficient trick for setting the environment variable. The logs look like this:

2022-03-21T01:41:14.8375279Z + where python
2022-03-21T01:41:14.8780352Z C:\cibw\python\python.3.7.9\tools\python.exe
2022-03-21T01:41:14.8950121Z C:\hostedtoolcache\windows\Python\3.7.9\x64\python.exe
2022-03-21T01:41:14.9617160Z C:\ProgramData\Chocolatey\bin\python.exe

And fortunately Python.h is at C:\cibw\python\python.3.7.9\tools\include\Python.h

However, I tried this trick with uses: pypa/cibuildwheel@f3e608e42cade59fc8e27816f0971606d6dd3f20 and it doesn't work. These are some of the logs:

2022-03-21T02:08:01.9981113Z + where python
2022-03-21T02:08:02.0316988Z C:\Users\runneradmin\AppData\Local\Temp\cibw-run-ydjncob9\cp37-win_amd64\build\venv\Scripts\python.exe
2022-03-21T02:08:02.0479597Z C:\hostedtoolcache\windows\Python\3.7.9\x64\python.exe
2022-03-21T02:08:02.1113516Z C:\ProgramData\Chocolatey\bin\python.exe

It seems to be setting pythonLocation to C:\Users\runneradmin\AppData\Local\Temp\cibw-run-ydjncob9\cp37-win_amd64\build\venv\Scripts whereas I want it to be C:\hostedtoolcache\windows\Python\3.7.9\x64\ in this newer version of cibuildwheel.