Could not find toolchain file when calling CMake on Windows, Linux

[ibis-hdl]

What is your question?

I refer here to issue #17209 and like to restart.

I'm a consumer of catch2 package and want to compile my project/app on Linux, Windows and macOS (even I don't own one) using CMake's default build tool ninja.

For this I have the following conanfile.py

from conan import ConanFile
from conan.tools.cmake import CMakeToolchain, CMakeDeps, CMake, cmake_layout

class CompressorRecipe(ConanFile):
    # Binary configuration
    settings = "os", "compiler", "build_type", "arch"

    def requirements(self):
        self.requires("catch2/3.7.1")

    def layout(self):
        cmake_layout(self)

    def generate(self):
        deps = CMakeDeps(self)
        deps.generate()
        tc = CMakeToolchain(self)
        tc.user_presets_path = "ConanCMakePresets.json"
        tc.generate()

Further, I have these (shortened) presets:

    "include": [
        ...
        "cmake/presets/gnuc.json",
        ...
        "cmake/presets/msvc.json",
        "ConanCMakePresets.json"
    ],
    "configurePresets": [
        ...
        {
            "name": "ninja-default-settings",
            "binaryDir": "${sourceDir}/build/${presetName}",
            "cacheVariables": { "CMAKE_EXPORT_COMPILE_COMMANDS": true }
        },
        {
            "name": "ninja-config",
            "generator": "Ninja",
            "inherits": [ "ninja-default-settings" ]
        },
        {
            "name": "ninja-multi-config",
            "generator": "Ninja Multi-Config",
            "inherits": [ "ninja-default-settings" ],
            "cacheVariables": { "CMAKE_CONFIGURATION_TYPES": "Debug;Release" }
        },
        ...
        {
            "name": "gcc-release",
            "displayName": "GnuC: Release",
            "description": "GnuC compiler: Release",
            "inherits": [
                "condition-linux",
                "ninja-config",
                "gcc-build-type-release",
                "conan-release",
                "cmake-default-configure-settings",
                "default-install-settings",
                "default-environment"
            ]
        }
        ...

As you see I like to have the buildDir ${sourceDir}/build/${presetName}. Creating Conan profiles is working as expected, also installing by Conan:

On Linux:

$ bash -c conan install . --settings build_type=Release --conf tools.cmake.cmaketoolchain:generator=Ninja --build=missing --profile:all=gcc
...

After running this, I have the default build/{Debug,Release}/generators structure from conan, where "toolchainFile": "generators/conan_toolchain.cmake". The point is here "binaryDir": "${sourceDir}/build/${presetName}" from CMakePresets.json. This is my preferred build tree for Linux, Windows and Darwin. Hence, this error rises:

$ cmake --preset gcc-release
Preset CMake variables:

  CMAKE_BUILD_TYPE="Release"
  CMAKE_CXX_COMPILER="g++"
  CMAKE_C_COMPILER="gcc"
  CMAKE_EXPORT_COMPILE_COMMANDS:BOOL="TRUE"
  CMAKE_INSTALL_PREFIX="/workspaces/gh-actions-test/build/gcc-release/install"
  CMAKE_MAKE_PROGRAM="ninja"
  CMAKE_POLICY_DEFAULT_CMP0091="NEW"
  CMAKE_POLICY_DEFAULT_CMP0167="NEW"
  CMAKE_TOOLCHAIN_FILE:FILEPATH="generators/conan_toolchain.cmake"

Preset environment variables:

  CC="gcc"
  CXX="g++"

CMake Error at /opt/cmake/share/cmake-3.30/Modules/CMakeDetermineSystem.cmake:152 (message):
  Could not find toolchain file: generators/conan_toolchain.cmake

I would assume, set I can the build_dir at conanfile.py (how to do in detail? cmake_layout?). Is it useful to use --output-folder to point to "binaryDir": "${sourceDir}/build/${presetName}"?
Here I have some concerns related to my typically workflow:

• conan install ...: Conan should nothing know about cmake's presets to be used later (it's a provider for 3rd party libraries), but this would be important for the output path.
• cmake --preset xyz
• cmake --build --preset xyz

To make things more complicated, Windows and XCode? uses Ninja's multi-config feature, where Conan's directory hierarchy is different. How to cope with this situation? How is the intended process for multi-platform projects on consumer side?

During writing/preparing/testing this question I face just another (related? Ninja's multi-config single-config feature) problem related to CMake presets holding Linux and Windows settings and build tool (conan-relese, conan-debug vs. conan-default on Windows) which I have to tackle down next, but basically the same toolchain path file problem as on Linux.

Have you read the CONTRIBUTING guide?

• I've read the CONTRIBUTING guide

[memsharded]

Hi @ibis-hdl

Thanks for the details.

I see now what is the issue. The default cmake_layout() will not be useful at this moment, because it implements a /Release or /Debug subfolder for non multi-config systems.

The solution would be to encode your layout accordingly in the layout() method of your recipe. I think with your assumptions, writing such a layout() method shouldn't be difficult, just 5 lines (and you can re-use them in multiple recipes if necessary with python-requires.

I'll have a look to see if cmake_layout() can be parameterized or something, but at first sight seems non trivial.

[memsharded]

Trying to allow this, I have realized that settings build_folder_vars might be enough:

def test_recipe_build_folders_vars_empty():
    client = TestClient()
    conanfile = textwrap.dedent("""
        from conan import ConanFile
        from conan.tools.cmake import cmake_layout

        class Conan(ConanFile):
            name = "pkg"
            version = "0.1"
            settings = "os", "arch", "compiler", "build_type"
            generators = "CMakeToolchain"

            def layout(self):
                self.folders.build_folder_vars = ["settings.compiler", "settings.build_type"]
                cmake_layout(self)
        """)
    client.save({"conanfile.py": conanfile})
    client.run("install . -s os=Windows -s compiler=gcc -s compiler.version=10 "
               "-s compiler.libcxx=libstdc++11 -s arch=armv8 -s build_type=Debug")
    presets = client.load("build/gcc-debug/generators/CMakePresets.json")
    assert "conan-gcc-debug" in presets

So if your preset name is gcc-release, then self.folders.build_folder_vars = ["settings.compiler", "settings.build_type"] seems is what you are looking for, can you please give it a try?

[ibis-hdl]

Thank you for your help. Using this works on Linux now. I changed the conanfile.py to:

    ...
    def layout(self):
        # conan.io #17324
        self.folders.build_folder_vars = ["settings.compiler", "settings.build_type"]
        cmake_layout(self)
    ...

Now, the problems rise as described on the bottom of the first thread on Windows. To exclude some side effect from Linux settings on Windows build, I temporarily removed the inheritance of conan-gcc-{release,debug} in my CMakeMakePreset.json (I'll come back to this later). Further, I commented out the change self.folders.build_folder_vars in conanfile.py

On Windows, I have now a CMakePreset using like this:

{
    "version": 8,
    "include": [
        "cmake/presets/os.json",
        "cmake/presets/gnuc.json",
        ...
        "cmake/presets/msvc.json",
        ...
        "ConanCMakePresets.json"
    ],
    "configurePresets": [
        ...
        {
            "name": "gcc-release",
            "displayName": "GnuC: Release",
            "description": "GnuC compiler: Release",
            "inherits": [
                "condition-linux",
                "ninja-config",
                "gcc-release-build-type",
                "cmake-default-configure-settings",
                "default-install-settings",
                "default-environment"
            ]
        },
        {
            "name": "gcc-debug",
            "displayName": "GnuC: Debug",
            "description": "GnuC compiler: Debug",
            "inherits": [
                "condition-linux",
                "ninja-config",
                "gcc-debug-build-type",
                "cmake-default-configure-settings",
                "default-install-settings",
                "default-environment"
            ]
        },
        {
            "name": "msvc-release",
            "displayName": "MSVC: Release",
            "description": "Microsoft Visual Studio C++ compiler: Release",
            "inherits": [
                "condition-windows",
                "windows-x64-arch",
                "ninja-multi-config",
                "msvc-release-build-type",
                "cmake-default-configure-settings",
                "default-install-settings",
                "default-environment",
                "conan-default"
            ]
        },
        {
            "name": "msvc-debug",
            "displayName": "MSVC: Debug",
            "description": "Microsoft Visual Studio C++ compiler: Debug",
            "inherits": [
                "condition-windows",
                "windows-x64-arch",
                "ninja-multi-config",
                "msvc-debug-build-type",
                "cmake-default-configure-settings",
                "default-install-settings",
                "default-environment",
                "conan-default"
            ]
        }

I did run on Windows terminal:

> pwsh -Command conan install . --settings build_type=Release --conf tools.cmake.cmaketoolchain:generator='Ninja Multi-Config' --build=missing --profile:all=default

All went fine. Now:

> cmake --preset msvc-release
...
CMake Error at CMakeLists.txt:14 (find_package):
  By not providing "FindCatch2.cmake" in CMAKE_MODULE_PATH this project has
  asked CMake to find a package configuration file provided by "Catch2", but
  CMake did not find one.

  Could not find a package configuration file provided by "Catch2" (requested
  version 3) with any of the following names:

    Catch2Config.cmake
    catch2-config.cmake

  Add the installation prefix of "Catch2" to CMAKE_PREFIX_PATH or set
  "Catch2_DIR" to a directory containing one of the above files.  If "Catch2"
  provides a separate development package or SDK, be sure it has been
  installed.

There is a file build\generators\Catch2Config.cmake, even the generated build\generators\conan_toolchain.cmake contains (shortened)

list(PREPEND CMAKE_MODULE_PATH "C:/Users/Olaf/.conan2/p/catche1d9fee334212/p/lib/cmake/Catch2")
list(PREPEND CMAKE_MODULE_PATH ${CMAKE_CURRENT_LIST_DIR})
list(PREPEND CMAKE_PREFIX_PATH "C:/Users/Olaf/.conan2/p/catche1d9fee334212/p/lib/cmake/Catch2")
list(PREPEND CMAKE_PREFIX_PATH ${CMAKE_CURRENT_LIST_DIR} )

Anyway, now I'll come back to the conan-gcc-{release,debug,default} issue. Depend on the preset, there are conan-gcc-{release,debug} configurePresets on Gcc/Linux, but not msvc/Windows; or conan-{default} on msvc/Windows but no on Gcc/Linux. This results into misleading error messages from CMake due to limited JSON parser used by CMake. I tried simply to add e.g.

        {
            "name": "conan-gcc-release",
            "hidden": true
        },
        {
            "name": "conan-gcc-debug",
            "hidden": true
        },

to CMakePresets.json and use the msvc preset on Windows, but there seems to be more (didn't work as expected). How to cope with this related problem? In my conanfile.py at def layout() I can switch OS depend on configuration by using e.g. if self.settings.os == "Windows": but the problem above remains (btw, can I check on the generator used? which would be the better choice)

Finally, probably I have a wrong understanding about Ninja's Multi Config but over writing here and researching - there is no need to have debug/release configuration, hence the (conan) default. The build config than is debug or release, would be e.g. cmake --build --config Release --preset msvc e.g. with

        {
            "name": "msvc",
            "displayName": "MSVC",
            "description": "Microsoft Visual Studio C++ compiler using 'Ninja Multi-Config'",
            "inherits": [
                "condition-windows",
                "windows-x64-arch",
                "ninja-multi-config",
                "msvc-debug-build-type",
                "conan-default",
                "cmake-default-configure-settings",
                "default-install-settings",
                "default-environment"
            ]
        },

isn't it?

For reference (and hopefully to make my intention clear), I attach the complete CmakePresets.json from the beginning of this conversation.
CMakePresets.json

[memsharded]

Hi @ibis-hdl

Thanks for your feedback.

The CMakeToolchain has a is_multi_configuration property that can be used to check if the current generator is multi-config or not, and act accordingly. But this is for later, in the generate() method. If we need conditionals in the layout() method, this is not available right now. The internal logic in cmake_layout() is:

    gen = conanfile.conf.get("tools.cmake.cmaketoolchain:generator", default=generator)
    if gen:
        multi = "Visual" in gen or "Xcode" in gen or "Multi-Config" in gen
    else:
        multi = conanfile.settings.get_safe("compiler") == "msvc"

If that doesn't help, I'd suggest the following, as this is getting a bit too difficult to understand and follow as text.
I think the best would be to put it as code as a toy Github project with a build.py script in the root that executes the exact steps that should succeed. Then we can collaborate on exactly the same code.

[ibis-hdl]

Thank you, I'm preparing an special Github repository ...