Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ModuleNotFoundError: No module named 'setuptools_scm._integration' #145

Open
diazona opened this issue Dec 21, 2023 · 9 comments
Open

ModuleNotFoundError: No module named 'setuptools_scm._integration' #145

diazona opened this issue Dec 21, 2023 · 9 comments
Assignees
@diazona
Labels
bug Something isn't working
Milestone
v0.4

Comments

@diazona
Copy link
Owner

diazona commented Dec 21, 2023

I'm seeing an error ModuleNotFoundError: No module named 'setuptools_scm._integration' when running the package distribution tests for aioax25, but only under very specific circumstances: it only happens with Python 3.12, and only when running all the tests or some large subset of them - if I try to rerun with the specific tests that fail, the error doesn't show up.

$ tox -e py312 --force-dep setuptools==66.1 --force-dep setuptools_scm==3.4.1 --force-dep typing-extensions==4 --
[...]
__________________________________________________________________________________________ ERROR at setup of TestExternalProject.test_name[aioax25==0.0.11.post0] __________________________________________________________________________________________

self = <test_distribution_packages.TestExternalProject object at 0x7f8b21d78530>, distribution_package = <test_support.distribution.PyPiPackagePreparation object at 0x7f8b1ce44380>

    @pytest.fixture(scope="class")
    def actual(self, distribution_package: DistributionPackagePreparation) -> StandardMetadata:
>       return StandardMetadata.from_pyproject(distribution_package.project.generate())

/home/diazona/programming/setuptools-pyproject-migration/tests/distribution/test_distribution_packages.py:91: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
/home/diazona/programming/setuptools-pyproject-migration/test_support/test_support/__init__.py:192: in generate
    distribution: distutils.dist.Distribution = distutils.core.run_setup(
/home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/_distutils/core.py:267: in run_setup
    exec(code, g)
<string>:19: in <module>
    ???
/home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/__init__.py:108: in setup
    return distutils.core.setup(**attrs)
/home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/_distutils/core.py:147: in setup
    _setup_distribution = dist = klass(attrs)
/home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/dist.py:475: in __init__
    _Distribution.__init__(
/home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/_distutils/dist.py:283: in __init__
    self.finalize_options()
/home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/dist.py:903: in finalize_options
    for ep in sorted(loaded, key=by_order):
/home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/dist.py:902: in <lambda>
    loaded = map(lambda e: e.load(), filtered)
/usr/lib/python3.12/importlib/metadata/__init__.py:205: in load
    module = import_module(match.group('module'))
/usr/lib/python3.12/importlib/__init__.py:90: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
<frozen importlib._bootstrap>:1387: in _gcd_import
    ???
<frozen importlib._bootstrap>:1360: in _find_and_load
    ???
<frozen importlib._bootstrap>:1310: in _find_and_load_unlocked
    ???
<frozen importlib._bootstrap>:488: in _call_with_frames_removed
    ???
<frozen importlib._bootstrap>:1387: in _gcd_import
    ???
<frozen importlib._bootstrap>:1360: in _find_and_load
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

name = 'setuptools_scm._integration', import_ = <function _gcd_import at 0x7f8b272380e0>
``
>   ???
E   ModuleNotFoundError: No module named 'setuptools_scm._integration'

<frozen importlib._bootstrap>:1324: ModuleNotFoundError
[...]

but

$ tox -e py312 --force-dep setuptools==66.1 --force-dep setuptools_scm==3.4.1 --force-dep typing-extensions==4 -- 'tests/distribution/test_distribution_packages.py::TestExternalProject' -k aioax25
.pkg-cpython312: _optional_hooks> python /home/diazona/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-cpython312: get_requires_for_build_editable> python /home/diazona/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-cpython312: build_editable> python /home/diazona/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
py312: install_package> python -I -m pip install --force-reinstall --no-deps /home/diazona/programming/setuptools-pyproject-migration/.tox/.tmp/package/244/setuptools_pyproject_migration-0.1.1.dev71+g43d9538-0.editable-py3-none-any.whl
py312: commands[0]> pytest tests/distribution/test_distribution_packages.py::TestExternalProject -k aioax25
=================================================================================================================== test session starts ====================================================================================================================
platform linux -- Python 3.12.1, pytest-7.4.3, pluggy-1.3.0 -- /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/bin/python
cachedir: .tox/py312/.pytest_cache
rootdir: /home/diazona/programming/setuptools-pyproject-migration
configfile: pytest.ini
plugins: console-scripts-1.4.1, enabler-2.3.1, cov-4.1.0
collected 42 items / 28 deselected / 14 selected                                                                                                                                                                                                           

tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[aioax25==0.0.11.post0] PASSED                                                                                                                                       [  7%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_version[aioax25==0.0.11.post0] PASSED                                                                                                                                    [ 14%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_description[aioax25==0.0.11.post0] PASSED                                                                                                                                [ 21%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_license[aioax25==0.0.11.post0] PASSED                                                                                                                                    [ 28%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_readme[aioax25==0.0.11.post0] PASSED                                                                                                                                     [ 35%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_requires_python[aioax25==0.0.11.post0] PASSED                                                                                                                            [ 42%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_dependencies[aioax25==0.0.11.post0] PASSED                                                                                                                               [ 50%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_optional_dependencies[aioax25==0.0.11.post0] PASSED                                                                                                                      [ 57%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_authors[aioax25==0.0.11.post0] PASSED                                                                                                                                    [ 64%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_maintainers[aioax25==0.0.11.post0] PASSED                                                                                                                                [ 71%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_urls[aioax25==0.0.11.post0] PASSED                                                                                                                                       [ 78%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_classifiers[aioax25==0.0.11.post0] PASSED                                                                                                                                [ 85%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_keywords[aioax25==0.0.11.post0] PASSED                                                                                                                                   [ 92%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_dynamic[aioax25==0.0.11.post0] PASSED                                                                                                                                    [100%]

===================================================================================================================== warnings summary =====================================================================================================================
tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[aioax25==0.0.11.post0]
  <string>:30: EncodingWarning: 'encoding' argument not specified

tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[aioax25==0.0.11.post0]
  <string>:30: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.md' mode='r' encoding='UTF-8'>
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/distribution/test_distribution_packages.py::TestExternalProject::test_readme[aioax25==0.0.11.post0]
  /home/diazona/programming/setuptools-pyproject-migration/tests/distribution/test_distribution_packages.py:132: EncodingWarning: 'encoding' argument not specified
    assert actual.readme.text == actual.readme.file.read_text()

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform linux, python 3.12.1-final-0 -----------
Name                                                                                                      Stmts   Miss  Cover   Missing
---------------------------------------------------------------------------------------------------------------------------------------
/tmp/pytest-of-diazona/pytest-2/aioax25==0.0.11.post00/project/aioax25-0.0.11.post0/aioax25/__init__.py       5      0   100%
src/setuptools_pyproject_migration/__init__.py                                                              155     34    78%   47-51, 72-77, 138, 155, 231-241, 288, 292, 300, 311, 321, 329, 345, 362, 371-377, 379, 387, 390, 394, 403
test_support/test_support/__init__.py                                                                        77     26    66%   23-24, 68, 112-123, 132, 141-147, 159-162, 176-177, 186, 220-222
test_support/test_support/distribution.py                                                                   277     65    77%   35-46, 51-52, 67, 71, 83, 87, 93, 125, 132, 135-136, 138, 176, 311-314, 350, 356-357, 361, 412-413, 444-449, 472, 503-516, 533-542, 558-559, 564-567, 570-572, 582-583, 586-587, 592
test_support/test_support/metadata.py                                                                       135     57    58%   15-18, 23-41, 52-56, 58-60, 63, 65, 100-110, 121, 123, 136, 147, 153, 164, 176, 184, 189-190, 196-215, 222
---------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                       649    182    72%

====================================================================================================== 14 passed, 28 deselected, 3 warnings in 1.19s =======================================================================================================
.pkg-cpython312: _exit> python /home/diazona/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
  py312: OK (3.72=setup[1.09]+cmd[2.64] seconds)
  congratulations :) (3.82 seconds)

This originally showed up during a manual workflow run.
@diazona diazona added this to the v0.2 milestone Dec 21, 2023
@diazona diazona added the bug Something isn't working label Dec 23, 2023
@diazona
Copy link
Owner Author

diazona commented Dec 25, 2023 •
edited
Loading

Hmm okay this is useful: running just the external project tests (tests/distribution/test_distribution_packages.py::TestExternalProject) is enough to reproduce the issue. And every single one of those tests fails with the same error.

$ tox -e py312 --force-dep setuptools==66.1 --force-dep setuptools_scm==3.4.1 --force-dep typing-extensions==4 -- 'tests/distribution/test_distribution_packages.py::TestExternalProject' --tb=no --runxfail
.pkg-cpython312: _optional_hooks> python /home/diazona/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-cpython312: get_requires_for_build_editable> python /home/diazona/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
.pkg-cpython312: build_editable> python /home/diazona/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
py312: install_package> python -I -m pip install --force-reinstall --no-deps /home/diazona/programming/setuptools-pyproject-migration/.tox/.tmp/package/400/setuptools_pyproject_migration-0.1.1.dev71+g43d9538-0.editable-py3-none-any.whl
py312: commands[0]> pytest tests/distribution/test_distribution_packages.py::TestExternalProject --tb=no --runxfail
=================================================================================================================== test session starts ====================================================================================================================
platform linux -- Python 3.12.1, pytest-7.4.3, pluggy-1.3.0 -- /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/bin/python
cachedir: .tox/py312/.pytest_cache
rootdir: /home/diazona/programming/setuptools-pyproject-migration
configfile: pytest.ini
plugins: console-scripts-1.4.1, cov-4.1.0, enabler-3.0.0
collected 42 items                                                                                                                                                                                                                                         

tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[pytest==7.3.0] ERROR                                                                                                                                                [  2%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_version[pytest==7.3.0] ERROR                                                                                                                                             [  4%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_description[pytest==7.3.0] ERROR                                                                                                                                         [  7%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_license[pytest==7.3.0] ERROR                                                                                                                                             [  9%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_readme[pytest==7.3.0] ERROR                                                                                                                                              [ 11%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_requires_python[pytest==7.3.0] ERROR                                                                                                                                     [ 14%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_dependencies[pytest==7.3.0] ERROR                                                                                                                                        [ 16%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_optional_dependencies[pytest==7.3.0] ERROR                                                                                                                               [ 19%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_authors[pytest==7.3.0] ERROR                                                                                                                                             [ 21%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_maintainers[pytest==7.3.0] ERROR                                                                                                                                         [ 23%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_urls[pytest==7.3.0] ERROR                                                                                                                                                [ 26%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_classifiers[pytest==7.3.0] ERROR                                                                                                                                         [ 28%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_keywords[pytest==7.3.0] ERROR                                                                                                                                            [ 30%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_dynamic[pytest==7.3.0] ERROR                                                                                                                                             [ 33%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[pytest-localserver==0.8.0] ERROR                                                                                                                                    [ 35%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_version[pytest-localserver==0.8.0] ERROR                                                                                                                                 [ 38%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_description[pytest-localserver==0.8.0] ERROR                                                                                                                             [ 40%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_license[pytest-localserver==0.8.0] ERROR                                                                                                                                 [ 42%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_readme[pytest-localserver==0.8.0] ERROR                                                                                                                                  [ 45%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_requires_python[pytest-localserver==0.8.0] ERROR                                                                                                                         [ 47%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_dependencies[pytest-localserver==0.8.0] ERROR                                                                                                                            [ 50%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_optional_dependencies[pytest-localserver==0.8.0] ERROR                                                                                                                   [ 52%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_authors[pytest-localserver==0.8.0] ERROR                                                                                                                                 [ 54%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_maintainers[pytest-localserver==0.8.0] ERROR                                                                                                                             [ 57%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_urls[pytest-localserver==0.8.0] ERROR                                                                                                                                    [ 59%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_classifiers[pytest-localserver==0.8.0] ERROR                                                                                                                             [ 61%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_keywords[pytest-localserver==0.8.0] ERROR                                                                                                                                [ 64%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_dynamic[pytest-localserver==0.8.0] ERROR                                                                                                                                 [ 66%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[aioax25==0.0.11.post0] ERROR                                                                                                                                        [ 69%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_version[aioax25==0.0.11.post0] ERROR                                                                                                                                     [ 71%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_description[aioax25==0.0.11.post0] ERROR                                                                                                                                 [ 73%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_license[aioax25==0.0.11.post0] ERROR                                                                                                                                     [ 76%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_readme[aioax25==0.0.11.post0] ERROR                                                                                                                                      [ 78%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_requires_python[aioax25==0.0.11.post0] ERROR                                                                                                                             [ 80%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_dependencies[aioax25==0.0.11.post0] ERROR                                                                                                                                [ 83%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_optional_dependencies[aioax25==0.0.11.post0] ERROR                                                                                                                       [ 85%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_authors[aioax25==0.0.11.post0] ERROR                                                                                                                                     [ 88%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_maintainers[aioax25==0.0.11.post0] ERROR                                                                                                                                 [ 90%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_urls[aioax25==0.0.11.post0] ERROR                                                                                                                                        [ 92%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_classifiers[aioax25==0.0.11.post0] ERROR                                                                                                                                 [ 95%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_keywords[aioax25==0.0.11.post0] ERROR                                                                                                                                    [ 97%]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_dynamic[aioax25==0.0.11.post0] ERROR                                                                                                                                     [100%]

===================================================================================================================== warnings summary =====================================================================================================================
tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[pytest==7.3.0]
  /home/diazona/programming/setuptools-pyproject-migration/test_support/test_support/distribution.py:446: UserWarning: No hash available for pytest==7.3.0
    warnings.warn(f"No hash available for {self._distribution.package_spec}")

tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[pytest==7.3.0]
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/config/setupcfg.py:515: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
    warnings.warn(msg, warning_class)

tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[pytest==7.3.0]
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/dist.py:885: UserWarning: Module setuptools_scm was already imported from /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools_scm/__init__.py, but /tmp/pytest-of-diazona/pytest-150/pytest==7.3.00/project/pytest-7.3.0/.eggs/setuptools_scm-8.0.4-py3.12.egg is being added to sys.path
    pkg_resources.working_set.add(dist, replace=True)

tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[pytest-localserver==0.8.0]
  <string>:10: EncodingWarning: 'encoding' argument not specified

tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[pytest-localserver==0.8.0]
  <string>:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/pytest-of-diazona/pytest-150/pytest-localserver==0.8.00/project/pytest-localserver-0.8.0/README.rst' mode='r' encoding='UTF-8'>
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[aioax25==0.0.11.post0]
  <string>:30: EncodingWarning: 'encoding' argument not specified

tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[aioax25==0.0.11.post0]
  <string>:30: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.md' mode='r' encoding='UTF-8'>
  Enable tracemalloc to get traceback where the object was allocated.
  See https://docs.pytest.org/en/stable/how-to/capture-warnings.html#resource-warnings for more info.

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

---------- coverage: platform linux, python 3.12.1-final-0 -----------
Name                                                                                                        Stmts   Miss  Cover   Missing
-----------------------------------------------------------------------------------------------------------------------------------------
/tmp/pytest-of-diazona/pytest-150/aioax25==0.0.11.post00/project/aioax25-0.0.11.post0/aioax25/__init__.py       5      0   100%
src/setuptools_pyproject_migration/__init__.py                                                                155    119    23%   47-51, 69-77, 116-127, 135, 138, 153-157, 181-192, 226-241, 248-396, 403
test_support/test_support/__init__.py                                                                          77     29    62%   23-24, 68, 112-123, 132, 141-147, 159-162, 176-177, 186, 205-207, 220-222
test_support/test_support/distribution.py                                                                     277     39    86%   35-46, 51-52, 67, 87, 125, 132, 138, 176, 311-314, 350, 356-357, 361, 412-413, 447-448, 472, 507, 509, 515-516, 536-540, 558-559, 564-567, 570-572, 592
test_support/test_support/metadata.py                                                                         135     30    78%   15-18, 23-41, 53, 58-60, 63, 65, 110, 121, 123, 136, 147, 176, 184, 199, 210-211, 215, 222
-----------------------------------------------------------------------------------------------------------------------------------------
TOTAL                                                                                                         649    217    67%

================================================================================================================= short test summary info ==================================================================================================================
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_version[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_description[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_license[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_readme[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_requires_python[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_dependencies[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_optional_dependencies[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_authors[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_maintainers[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_urls[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_classifiers[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_keywords[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_dynamic[pytest==7.3.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_version[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_description[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_license[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_readme[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_requires_python[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_dependencies[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_optional_dependencies[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_authors[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_maintainers[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_urls[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_classifiers[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_keywords[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_dynamic[pytest-localserver==0.8.0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_version[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_description[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_license[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_readme[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_requires_python[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_dependencies[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_optional_dependencies[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_authors[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_maintainers[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_urls[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_classifiers[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_keywords[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
ERROR tests/distribution/test_distribution_packages.py::TestExternalProject::test_dynamic[aioax25==0.0.11.post0] - ModuleNotFoundError: No module named 'setuptools_scm._integration'
============================================================================================================= 7 warnings, 42 errors in 10.21s ==============================================================================================================
py312: exit 1 (11.69 seconds) /home/diazona/programming/setuptools-pyproject-migration> pytest tests/distribution/test_distribution_packages.py::TestExternalProject --tb=no --runxfail pid=7202
.pkg-cpython312: _exit> python /home/diazona/.local/pipx/venvs/tox/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta
  py312: FAIL code 1 (12.85=setup[1.16]+cmd[11.69] seconds)
  evaluation failed :( (12.95 seconds)

@diazona
Copy link
Owner Author

diazona commented Dec 25, 2023

If I run just one test, this warning shows up in the output - I have a feeling it might be related, at least indirectly (in the sense that it indicates how the packages under test are messing with the import system):

$ tox -e py312 --force-dep setuptools==66.1 --force-dep setuptools_scm==3.4.1 --force-dep typing-extensions==4 -- 'tests/distribution/test_distribution_packages.py::TestExternalProject' --exitfirst --runxfail
[...]
-------------------------------------------------------------------------------------------------------------------- Captured log setup --------------------------------------------------------------------------------------------------------------------
[...]
tests/distribution/test_distribution_packages.py::TestExternalProject::test_name[pytest==7.3.0]
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/dist.py:885: UserWarning: Module setuptools_scm was already imported from /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools_scm/__init__.py, but /tmp/pytest-of-diazona/pytest-152/pytest==7.3.00/project/pytest-7.3.0/.eggs/setuptools_scm-8.0.4-py3.12.egg is being added to sys.path
    pkg_resources.working_set.add(dist, replace=True)

@diazona
Copy link
Owner Author

diazona commented Dec 26, 2023

setuptools_scm._integration was added in pypa/setuptools-scm@2f24235, which first landed in setuptools_scm 7.0.3. So I tried upgrading to setuptools_scm 7.0.3 and the tests run fine (at least one does)... but weirdly, it works with 7.0.2 as well, and as far back as setuptools_scm 6.0.0 (but not before that).

I still can't account for why the setuptools_scm._integration module is being imported when we pin to an older version of setuptools_scm, but it does suggest that one possible solution would be to require a newer version, at least 6. Anyway, I want to take a bit of time to try understanding this better before giving up and just bumping the version requirement.

@diazona
Copy link
Owner Author

diazona commented Dec 26, 2023 •
edited
Loading

Aha, I found something by loading up the debugger when this failure occurs:

(Pdb) bt
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/_pytest/runner.py(341)from_call()
-> result: Optional[TResult] = func()
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/_pytest/runner.py(262)<lambda>()
-> lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/pluggy/_hooks.py(493)__call__()
-> return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/pluggy/_manager.py(115)_hookexec()
-> return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/pluggy/_callers.py(152)_multicall()
-> return outcome.get_result()
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/pluggy/_result.py(114)get_result()
-> raise exc.with_traceback(exc.__traceback__)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/pluggy/_callers.py(77)_multicall()
-> res = hook_impl.function(*args)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/_pytest/runner.py(157)pytest_runtest_setup()
-> item.session._setupstate.setup(item)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/_pytest/runner.py(497)setup()
-> raise exc
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/_pytest/runner.py(494)setup()
-> col.setup()
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/_pytest/python.py(1795)setup()
-> self._request._fillfixtures()
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/_pytest/fixtures.py(566)_fillfixtures()
-> item.funcargs[argname] = self.getfixturevalue(argname)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/_pytest/fixtures.py(585)getfixturevalue()
-> fixturedef = self._get_active_fixturedef(argname)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/_pytest/fixtures.py(607)_get_active_fixturedef()
-> self._compute_fixture_value(fixturedef)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/_pytest/fixtures.py(693)_compute_fixture_value()
-> fixturedef.execute(request=subrequest)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/_pytest/fixtures.py(1069)execute()
-> result = ihook.pytest_fixture_setup(fixturedef=self, request=request)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/pluggy/_hooks.py(493)__call__()
-> return self._hookexec(self.name, self._hookimpls, kwargs, firstresult)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/pluggy/_manager.py(115)_hookexec()
-> return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/pluggy/_callers.py(152)_multicall()
-> return outcome.get_result()
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/pluggy/_result.py(114)get_result()
-> raise exc.with_traceback(exc.__traceback__)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/pluggy/_callers.py(77)_multicall()
-> res = hook_impl.function(*args)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/_pytest/fixtures.py(1123)pytest_fixture_setup()
-> result = call_fixture_func(fixturefunc, request, kwargs)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/_pytest/fixtures.py(902)call_fixture_func()
-> fixture_result = fixturefunc(**kwargs)
  /home/diazona/programming/setuptools-pyproject-migration/tests/distribution/test_distribution_packages.py(91)actual()
-> return StandardMetadata.from_pyproject(distribution_package.project.generate())
  /home/diazona/programming/setuptools-pyproject-migration/test_support/test_support/__init__.py(192)generate()
-> distribution: distutils.dist.Distribution = distutils.core.run_setup(
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/_distutils/core.py(267)run_setup()
-> exec(code, g)
  <string>(4)<module>()
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/__init__.py(108)setup()
-> return distutils.core.setup(**attrs)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/_distutils/core.py(147)setup()
-> _setup_distribution = dist = klass(attrs)
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/dist.py(475)__init__()
-> _Distribution.__init__(
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/_distutils/dist.py(283)__init__()
-> self.finalize_options()
> /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/dist.py(903)finalize_options()
-> for ep in sorted(loaded, key=by_order):
  /home/diazona/programming/setuptools-pyproject-migration/.tox/py312/lib/python3.12/site-packages/setuptools/dist.py(902)<lambda>()
-> loaded = map(lambda e: e.load(), filtered)
  /usr/lib/python3.12/importlib/metadata/__init__.py(205)load()
-> module = import_module(match.group('module'))
  /usr/lib/python3.12/importlib/__init__.py(90)import_module()
-> return _bootstrap._gcd_import(name[level:], package, level)
  <frozen importlib._bootstrap>(1387)_gcd_import()
  <frozen importlib._bootstrap>(1360)_find_and_load()
  <frozen importlib._bootstrap>(1310)_find_and_load_unlocked()
  <frozen importlib._bootstrap>(488)_call_with_frames_removed()
  <frozen importlib._bootstrap>(1387)_gcd_import()
  <frozen importlib._bootstrap>(1360)_find_and_load()
  <frozen importlib._bootstrap>(1324)_find_and_load_unlocked()
(Pdb) defined
(EntryPoint(name='setuptools_scm', value='setuptools_scm._integration.setuptools:infer_version', group='setuptools.finalize_distribution_options'), EntryPoint(name='keywords', value='setuptools.dist:Distribution._finalize_setup_keywords', group='setuptools.finalize_distribution_options'), EntryPoint(name='parent_finalize', value='setuptools.dist:_Distribution.finalize_options', group='setuptools.finalize_distribution_options'))
(Pdb) e = next(iter(defined))
(Pdb) e.dist._path
PosixPath('/tmp/pytest-of-diazona/pytest-166/pytest==7.3.00/project/pytest-7.3.0/.eggs/setuptools_scm-8.0.4-py3.12.egg/EGG-INFO')

Evidently the pytest installation includes an egg of setuptools_scm 8.0.4 (maybe just whatever happens to be the latest version) outside of the normal package installation system. And that egg includes the entry point causing the error:

$ grep integration /tmp/pytest-of-diazona/pytest-166/pytest\=\=7.3.00/project/pytest-7.3.0/.eggs/setuptools_scm-8.0.4-py3.12.egg/EGG-INFO/entry_points.txt 
use_scm_version = setuptools_scm._integration.setuptools:version_keyword
setuptools_scm = setuptools_scm._integration.setuptools:infer_version

I suspect that when pytest's setup.py runs, it tries to install the build dependencies, which include setuptools_scm>6, but there is already an older version of setuptools_scm earlier on sys.path, so when it tries to find the entry point mentioned in the configuration of the latest setuptools_scm, it goes looking in the code for the older version. If so, then perhaps the proper way to solve this is to create a virtual environment to use when running setup.py pyproject in the various Project methods. But a shorter-term solution would be to skip the distribution package tests unless the right dependencies are already installed for them. I'll see what I can do about that.

@diazona
Copy link
Owner Author

diazona commented Dec 27, 2023

Here is a small script that reproduces the error in an isolated environment (should be run in an empty directory):

#!/bin/bash

mkdir downloads
pushd downloads
curl 'https://files.pythonhosted.org:443/packages/2b/a6/22c1138c2a7b60c9eb9ddeac017d82a58dfd9661651c721e7466af648764/pytest-7.3.0.tar.gz' -O -fsSL
popd
python3.12 -m venv venv
source venv/bin/activate
python -m pip install packaging pep508-parser 'setuptools==66.1' 'setuptools_scm[toml]==3.4.1' tomlkit 'typing-extensions>=4,<5'
mkdir project
cd project
tar -zxf ../downloads/pytest-7.3.0.tar.gz
tree -a >../before.txt
cd pytest-7.3.0
python -c 'import distutils.core; distutils.core.run_setup("setup.py", script_args=["pyproject"], stop_after="commandline")'
tree -a >../../after.txt

@diazona diazona mentioned this issue Dec 27, 2023
Merged
@diazona diazona modified the milestones: v0.2, v0.3 Dec 29, 2023
@diazona
Copy link
Owner Author

diazona commented Aug 14, 2024

I've been looking into this in more detail and basically confirmed my earlier suspicion. An overview of what happens is:

  1. pytest's call to setup() tries to install the build dependencies from the setup_requires argument
  2. It (indirectly) calls setuptools.installer._fetch_build_eggs() with those build dependencies as an argument
  3. That (indirectly) calls pkg_resources.WorkingSet._resolve_dist() with replace_conflicting=True
  4. Because there is a conflicting package already installed (setuptools_scm==3.4.1, which conflicts with pytest's setup requirement setuptools_scm>=6) and replace_conflicting is set, _resolve_dist() creates a brand new empty pkg_resources.Environment, effectively pretending that no packages are installed, and within that environment tries to find a distribution it can install to satisfy setuptools_scm>=6, which will be whatever happens to be the latest version of setuptools-scm available at the time
  5. It downloads the wheel file for the latest version of setuptools-scm to a temporary directory, unpacks it inside the .eggs directory within the current directory, and calls setuptools.wheel.install_as_egg() to install it
  6. Installing the distribution involves creating a setuptools.dist.Distribution object for it, and in the process of doing so it tries to resolve the entry points, but the setuptools_scm module which already exists in sys.modules comes from the old version and doesn't have the required attribute

What I take from all this is that we really shouldn't be trying to run pytest's setup script in an environment which already has an older version of setuptools-scm (or, really, any of its build dependencies) installed, because it will involve two different versions of the same package potentially clashing. So skipping the tests for pytest when the installed version of setuptools-scm is too old, as we do now, is the way to go, at least in the short term. I'm inclined to just leave that like it is for the 0.3 release.

There are a couple of alternatives we could explore in the future:

  • Try more fully resetting the state of the Python interpreter when we invoke another package's setup script. In particular, this would mean removing setuptools_scm from sys.modules to the default so the old version of setuptools_scm isn't found, but based on some quick testing we'd need to do more than that, and I'm not sure it's worth the effort to make that work.
  • Alternatively, we could install the distribution package (pytest or whatever) in a virtual environment and run the setup script in that environment as a subprocess. We wouldn't be able to access the raw Distribution object created (well, maybe it could be pickled), but we don't actually need it, all we need is the content of the pyproject.toml file it generates, and that could easily be parsed from the subprocess's stdout or a temporary file. I think this is a promising approach but it's best left for later.

TL;DR I think we should do nothing about this issue in version 0.3, and for a future release look into running distribution package tests using a virtual environment. That clears us up to release version 0.3 soon. @sjlongland any thoughts?

@diazona diazona removed this from the v0.3 milestone Aug 14, 2024
@diazona diazona self-assigned this Aug 14, 2024
@sjlongland
Copy link
Collaborator

@diazona That seems sane… I'm actually surprised setuptools-scm is even involved with aioax25, as I didn't think I was using it… but as you mention, pytest seems to pull it in.

Truly is a bizzare edge case. I'd be fine with shelving this until after v0.3.

@diazona
Copy link
Owner Author

diazona commented Aug 16, 2024

Oh it's not - I'm pretty confident at this point that aioax25 has nothing to do with this, it's all about pytest.

@diazona
Copy link
Owner Author

diazona commented Aug 27, 2024

Apparently this happens on Python versions less than 3.8 as well. (example). Which I guess makes more sense than it only happening on Python 3.12, but I'm kind of kicking myself for not having noticed that before.

@diazona diazona added this to the v0.4 milestone Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@diazona diazona

Labels
bug Something isn't working
Projects
None yet
Milestone
v0.4
Development

No branches or pull requests
2 participants
@diazona @sjlongland