CMake find_package pythoninterp is deprecated

[VincentRouvreau]

Replaces #1007
Modification taken from #151
Fix #139
CMake 3.12 is now 5 years old 3.15 is now 4 years old (july 2019), so it should be ok for every one.
CMake 3.15 is required for cmake_policy(SET CMP0094 NEW)
CMake 3.16 (nov 2019) is required if the user wants to specify a python interpreter (cmake -DPython_EXECUTABLE=/custom/path/to/python3 ...)

[mglisse]

Why not the following, to make it less likely to pick a bad version of Python?

Suggested change

	find_package( Python COMPONENTS Interpreter)
	find_package( Python COMPONENTS Interpreter Development.Module NumPy)

[VincentRouvreau]

I do not see a lot of added value with Development.Module or NumPy. In the documentation, you can get Python_NumPy_INCLUDE_DIRS, that I get in setup.py.in with the python code from numpy import get_include as numpy_get_include, and Python_NumPy_VERSION, that I get with the CMake function find_python_module (that performs ${Python_EXECUTABLE} -c "import ${PYTHON_MODULE_NAME}; print(${PYTHON_MODULE_NAME}.__version__), and by the way I also need this function for the other python dependencies to decide if a test must be passed or not. Maybe we could also skip tests when import throw ImportError.

I am not sure this would help to understand this ugly CMakeLists.txt... But tell me if you disagree, I can try

[mglisse]

I do not see a lot of added value with Development.Module or NumPy.

I didn't suggest them for the variables they define, but because if they are present, find_package is supposed to find a version of Python that has the dev stuff and numpy installed. If there are several versions of Python installed, this makes it less likely to pick one that is unusable for our purpose. (we do need the dev stuff to compile a plugin)

[VincentRouvreau]

Ok, that makes sense. I added your proposal on 23a393b

[mglisse]

Er, this sets PYTHON_EXE to the last entry returned by where python? That's a bit strange.

[VincentRouvreau]

Because it seems to be the good way to do it in batch 😅 cf. https://devblogs.microsoft.com/oldnewthing/20120731-00/?p=7003

If you prefer, I can do:

          for /f %%i in ('where python') do (set PYTHON_EXE=%%i & goto :next)
          :next

[mglisse]

I guess my question is not so much the syntax (this is CI, I don't much care), it is more trying to understand why we need to specify python now when we didn't before. What versions of Python exist under what names on that machine, which one do we want (apparently the only one called plain python?), and which one does CMake choose (and ideally why). Again, I don't really care about the CI case in particular, I care more that this is a fallout of the cmake change which could also affect users.

[VincentRouvreau]

You are right, and I did not talk too much about my experiments around this new feature, my fault too.
If you have a look to https://github.com/VincentRouvreau/play_with_azure_pipelines/blob/159a010d99352edd169949c700af23940b5eabe0/azure-pipelines.yml#L44-L58

In a CMakeLists.txt, I just call find_package( Python COMPONENTS Interpreter).

1. I first call cmake without any arguments, I get on Azure CI (you may not have access to the logs):
   -- Found Python: C:/hostedtoolcache/windows/Python/3.12.4/x64/python3.exe (found version "3.12.4") found components: Interpreter
   Note this is not correct, because I said in the yaml file:

    steps:
      - task: UsePythonVersion@0
        displayName: Use Python $(pythonVersion)
        inputs:
          versionSpec: 3.9
          addToPath: true
          architecture: "x64"

2. Then I call:

for /f %%i in ('where python') do echo %%i
# Prints: C:\hostedtoolcache\windows\Python\3.9.13\x64\python.exe
for /f %%i in ('where python') do (set PYTHON_EXE=%%i & goto :next)
:next
cmake -DPython_EXECUTABLE=%PYTHON_EXE% ..
# -- Found Python: C:\hostedtoolcache\windows\Python\3.9.13\x64\python.exe (found version "3.9.13") found components: Interpreter

I don't know how it will affect the users, but for sure it won't affect main users that are only using conda or pip packages.
I also don't know how the python installation is performed on the Azure Windows virtual machines, but the behaviour is quite strange... I can have a look if there is something that could replace UsePythonVersion or if its installation is not well performed.

[mglisse]

So the system has other versions of Python installed in addition to the one we request, that's fine. The one we request is the only one in the default search path, which is good. However, Python_FIND_REGISTRY is FIRST by default, meaning "Try to use registry before environment variables.", and the registry probably knows both versions of Python and not our preferences. It looks like we could pass -DPython_FIND_REGISTRY=LAST (or NEVER) to cmake in this CI. Does that work?

[VincentRouvreau]

Ok, I got it and tested it on https://github.com/VincentRouvreau/play_with_azure_pipelines it works !

[VincentRouvreau]

I used -DPython_FIND_REGISTRY=LAST and -DPython_FIND_FRAMEWORK=LAST on d190e7d

[mglisse]

It looks like Development.Module (as opposed to plain Development which also includes Development.Embed) requires CMake 3.18, so we should either require that, or say Development (and put a comment next to it to remind us to update it in a couple years), or remove it.

[VincentRouvreau]

find_package( Python COMPONENTS Interpreter Development NumPy) and a comment on 630094b

[mglisse]

So the system has other versions of Python installed in addition to the one we request, that's fine. The one we request is the only one in the default search path, which is good. However, Python_FIND_REGISTRY is FIRST by default, meaning "Try to use registry before environment variables.", and the registry probably knows both versions of Python and not our preferences. It looks like we could pass -DPython_FIND_REGISTRY=LAST (or NEVER) to cmake in this CI. Does that work?

[mglisse]

I would expect it uses PATH, not "standard libraries or headers", since it is looking for a python exe?