Can shims only be present when the version/environment is actually in use?

[samvv]

I have a project that requires the PyPI cmake package during build. This creates a new shim ~/.pyenv/bin/cmake, which gets added to my PATH.

Problem is that when I'm not using the project, the ~/.pyenv/bin/cmake is still there. When I try to run $ cmake, I'm getting either an error that the correct Python version/environment is not active or it just uses the version from the shim.

Is this a bug/feature? How can I avoid this, aside from uninstalling all the conflicting packages?

[native-api]

When a Python installation that has the program represented by a shim is not active, the shim is supposed to fall through to whatever is further on PATH. If there's nothing with this name further on PATH (i.e. you don't have CMake installed on your system), you would get an error trying to run it anyway -- only it would be plain cmake: command not found instead of pyenv: cmake: command not found.