Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

5.3.1: pytest is failing in tests/test_win32pipes.py::Test_NamedPipe::test_rpyc unit #544

Open
kloczek opened this issue Sep 14, 2023 · 6 comments
Assignees
Labels
Triage Investigation by a maintainer has started

Comments

@kloczek
Copy link
Contributor

kloczek commented Sep 14, 2023

I'm packaging your module as an rpm package so I'm using the typical PEP517 based build, install and test cycle used on building packages from non-root account.

  • python3 -sBm build -w --no-isolation
  • because I'm calling build with --no-isolation I'm using during all processes only locally installed modules
  • install .whl file in </install/prefix> using 'installer` module
  • run pytest with $PYTHONPATH pointing to sitearch and sitelib inside </install/prefix>
  • build is performed in env which is cut off from access to the public network (pytest is executed with -m "not network")

Here is pytest output:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.18, pytest-7.4.2, pluggy-1.3.0
rootdir: /home/tkloczko/rpmbuild/BUILD/rpyc-5.3.1
collected 88 items

tests/test_affinity.py .                                                                                                                                                              [  1%]
tests/test_async.py .....                                                                                                                                                             [  6%]
tests/test_attr_access.py .........                                                                                                                                                   [ 17%]
tests/test_attributes.py .                                                                                                                                                            [ 18%]
tests/test_brine.py .                                                                                                                                                                 [ 19%]
tests/test_classic.py .....                                                                                                                                                           [ 25%]
tests/test_context_managers.py ..                                                                                                                                                     [ 27%]
tests/test_custom_service.py .......                                                                                                                                                  [ 35%]
tests/test_dataclass.py .                                                                                                                                                             [ 36%]
tests/test_deploy.py ...s                                                                                                                                                             [ 40%]
tests/test_gdb.py .                                                                                                                                                                   [ 42%]
tests/test_get_id_pack.py ....                                                                                                                                                        [ 46%]
tests/test_gevent_server.py sss                                                                                                                                                       [ 50%]
tests/test_ipv6.py s                                                                                                                                                                  [ 51%]
tests/test_magic.py ..                                                                                                                                                                [ 53%]
tests/test_netref_hierachy.py ......                                                                                                                                                  [ 60%]
tests/test_oneshot_server.py .                                                                                                                                                        [ 61%]
tests/test_refcount.py .                                                                                                                                                              [ 62%]
tests/test_registry.py ......                                                                                                                                                         [ 69%]
tests/test_remote_exception.py ..                                                                                                                                                     [ 71%]
tests/test_remoting.py s.s.s                                                                                                                                                          [ 77%]
tests/test_rpyc_over_rpyc.py ...                                                                                                                                                      [ 80%]
tests/test_service_pickle.py .                                                                                                                                                        [ 81%]
tests/test_ssh.py ..                                                                                                                                                                  [ 84%]
tests/test_ssl.py .                                                                                                                                                                   [ 85%]
tests/test_teleportation.py ......                                                                                                                                                    [ 92%]
tests/test_threaded_server.py ...                                                                                                                                                     [ 95%]
tests/test_threads.py .                                                                                                                                                               [ 96%]
tests/test_win32pipes.py ssF                                                                                                                                                          [100%]

========================================================================================= FAILURES ==========================================================================================
_________________________________________________________________________________ Test_NamedPipe.test_rpyc __________________________________________________________________________________

self = <test_win32pipes.Test_NamedPipe object at 0x7f29827e5790>

    def test_rpyc(self):
>       assert self.client.root.get_service_name() == "VOID"
E       AttributeError: 'Test_NamedPipe' object has no attribute 'client'

tests/test_win32pipes.py:57: AttributeError
===================================================================================== warnings summary ======================================================================================
tests/test_ssh.py::Test_Ssh::test_connect
  /usr/lib/python3.8/site-packages/_pytest/threadexception.py:73: PytestUnhandledThreadExceptionWarning: Exception in thread RpycSpawnThread-builtins.method-139822925896352-139816296592192

  Traceback (most recent call last):
    File "/usr/lib64/python3.8/threading.py", line 932, in _bootstrap_inner
      self.run()
    File "/usr/lib64/python3.8/threading.py", line 870, in run
      self._target(*self._args, **self._kwargs)
    File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/netref.py", line 240, in __call__
      return syncreq(_self, consts.HANDLE_CALL, args, kwargs)
    File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/netref.py", line 63, in syncreq
      return conn.sync_request(handler, proxy, *args)
    File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/protocol.py", line 718, in sync_request
      return _async_res.value
    File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/async_.py", line 106, in value
      self.wait()
    File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/async_.py", line 51, in wait
      self._conn.serve(self._ttl)
    File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/protocol.py", line 447, in serve
      self._dispatch(data)  # Dispatch will unbox, invoke callbacks, etc.
    File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/protocol.py", line 399, in _dispatch
      self._dispatch_request(seq, args)
    File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/protocol.py", line 373, in _dispatch_request
      self._send(consts.MSG_REPLY, seq, self._box(res))
    File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/protocol.py", line 298, in _send
      self._channel.send(data)
    File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/channel.py", line 82, in send
      self.stream.write(header + data[:part1])
    File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/stream.py", line 288, in write
      count = self.sock.send(data[:self.MAX_IO_CHUNK])
    File "/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-5.3.1-2.fc35.x86_64/usr/lib/python3.8/site-packages/rpyc/core/stream.py", line 96, in __getattr__
      raise EOFError("stream has been closed")
  EOFError: stream has been closed

    warnings.warn(pytest.PytestUnhandledThreadExceptionWarning(msg))

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================== short test summary info ==================================================================================
SKIPPED [1] tests/test_deploy.py:82: Paramiko is not available
SKIPPED [1] tests/test_gevent_server.py:27: Gevent is not available
SKIPPED [1] tests/test_gevent_server.py:33: Gevent is not available
SKIPPED [1] tests/test_gevent_server.py:49: Gevent is not available
SKIPPED [1] tests/test_ipv6.py:19: requires IPv6
SKIPPED [1] tests/test_remoting.py:34: TODO: upload a package and a module
SKIPPED [1] tests/test_remoting.py:38: Requires manual testing atm
SKIPPED [1] tests/test_remoting.py:43: Requires manual testing atm
SKIPPED [1] tests/test_win32pipes.py:11: Requires windows
SKIPPED [1] tests/test_win32pipes.py:24: Requires windows
FAILED tests/test_win32pipes.py::Test_NamedPipe::test_rpyc - AttributeError: 'Test_NamedPipe' object has no attribute 'client'
============================================================== 1 failed, 77 passed, 10 skipped, 1 warning in 83.96s (0:01:23) ===============================================================

Here is list of installed modules in build env

Package                       Version
----------------------------- --------
alabaster                     0.7.13
Babel                         2.12.1
build                         1.0.3
charset-normalizer            3.2.0
distro                        1.8.0
docutils                      0.20.1
editables                     0.5
exceptiongroup                1.1.3
gpg                           1.21.0
hatchling                     1.18.0
idna                          3.4
imagesize                     1.4.1
importlib-metadata            6.8.0
iniconfig                     2.0.0
installer                     0.7.0
Jinja2                        3.1.2
libcomps                      0.1.19
MarkupSafe                    2.1.3
numpy                         1.24.4
packaging                     23.1
pandas                        2.0.3
pathspec                      0.11.2
pkg                           0.0.0
pluggy                        1.3.0
plumbum                       1.8.2
Pygments                      2.16.1
pyproject_hooks               1.0.0
pytest                        7.4.2
python-dateutil               2.8.2
pytz                          2023.3
requests                      2.31.0
six                           1.16.0
snowballstemmer               2.2.0
Sphinx                        7.0.1
sphinxcontrib-applehelp       1.0.4
sphinxcontrib-devhelp         1.0.2
sphinxcontrib-htmlhelp        2.0.3
sphinxcontrib-jsmath          1.0.1
sphinxcontrib-qthelp          1.0.3
sphinxcontrib-serializinghtml 1.1.9
tomli                         2.0.1
trove-classifiers             2023.8.9
urllib3                       1.26.16
wheel                         0.41.1
zipp                          3.16.2
comrumino added a commit that referenced this issue Oct 12, 2023
@comrumino
Copy link
Collaborator

I pushed a commit that should resolve this failure. However, I don't have a windows machine currently configured for development. Let me know if that does the trick! Thanks!

@comrumino comrumino self-assigned this Oct 12, 2023
@comrumino comrumino added the Triage Investigation by a maintainer has started label Oct 12, 2023
@kloczek
Copy link
Contributor Author

kloczek commented Feb 24, 2024

Just tested 6.0.0 and looks like now pytest fails because test suite cannot find support and cfg_tests modules

Here is pytest output:
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.18, pytest-8.0.1, pluggy-1.3.0
rootdir: /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0
plugins: anyio-4.2.0, nbval-0.10.0
collected 93 items / 2 errors

========================================================================================== ERRORS ===========================================================================================
__________________________________________________________________________ ERROR collecting tests/test_affinity.py __________________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/tests/test_affinity.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_affinity.py:4: in <module>
    import support
E   ModuleNotFoundError: No module named 'support'
_______________________________________________________________________ ERROR collecting tests/test_service_pickle.py _______________________________________________________________________
ImportError while importing test module '/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/tests/test_service_pickle.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
/usr/lib64/python3.8/importlib/__init__.py:127: in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
tests/test_service_pickle.py:7: in <module>
    import cfg_tests
E   ModuleNotFoundError: No module named 'cfg_tests'
================================================================================== short test summary info ==================================================================================
ERROR tests/test_affinity.py
ERROR tests/test_service_pickle.py
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! Interrupted: 2 errors during collection !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
===================================================================================== 2 errors in 0.89s =====================================================================================

@kloczek
Copy link
Contributor Author

kloczek commented Feb 24, 2024

Below quick patch fixed scanning

--- a/tests/test_service_pickle.py
+++ b/tests/test_service_pickle.py
@@ -4,7 +4,7 @@
 import timeit
 import rpyc
 import unittest
-import cfg_tests
+import tests.cfg_tests
 try:
     import pandas as pd
     import numpy as np
--- a/tests/test_affinity.py
+++ b/tests/test_affinity.py
@@ -1,7 +1,7 @@
 import sys
 import time
 import unittest
-import support
+import tests.support
 import rpyc

But pytest started failing in more units

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.8.18, pytest-8.0.1, pluggy-1.3.0
rootdir: /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0
plugins: anyio-4.2.0, nbval-0.10.0
collected 95 items

tests/test_affinity.py E                                                                                                                                                              [  1%]
tests/test_async.py .....                                                                                                                                                             [  6%]
tests/test_attr_access.py .........                                                                                                                                                   [ 15%]
tests/test_attributes.py F                                                                                                                                                            [ 16%]
tests/test_brine.py .                                                                                                                                                                 [ 17%]
tests/test_classic.py ...F.                                                                                                                                                           [ 23%]
tests/test_context_managers.py ..                                                                                                                                                     [ 25%]
tests/test_custom_service.py .......                                                                                                                                                  [ 32%]
tests/test_dataclass.py .                                                                                                                                                             [ 33%]
tests/test_deploy.py ...s                                                                                                                                                             [ 37%]
tests/test_gdb.py .                                                                                                                                                                   [ 38%]
tests/test_get_id_pack.py ....                                                                                                                                                        [ 43%]
tests/test_gevent_server.py sss                                                                                                                                                       [ 46%]
tests/test_ipv6.py s                                                                                                                                                                  [ 47%]
tests/test_magic.py FF                                                                                                                                                                [ 49%]
tests/test_netref_hierachy.py ......F....                                                                                                                                             [ 61%]
tests/test_oneshot_server.py .                                                                                                                                                        [ 62%]
tests/test_race.py .                                                                                                                                                                  [ 63%]
tests/test_refcount.py .                                                                                                                                                              [ 64%]
tests/test_registry.py ......                                                                                                                                                         [ 70%]
tests/test_remote_exception.py ..                                                                                                                                                     [ 72%]
tests/test_remoting.py s.s.s                                                                                                                                                          [ 77%]
tests/test_rpyc_over_rpyc.py ...                                                                                                                                                      [ 81%]
tests/test_service_pickle.py F                                                                                                                                                        [ 82%]
tests/test_ssh.py ..                                                                                                                                                                  [ 84%]
tests/test_ssl.py .                                                                                                                                                                   [ 85%]
tests/test_teleportation.py ......                                                                                                                                                    [ 91%]
tests/test_threaded_server.py ...                                                                                                                                                     [ 94%]
tests/test_threads.py .                                                                                                                                                               [ 95%]
tests/test_urllib3.py s                                                                                                                                                               [ 96%]
tests/test_win32pipes.py sss                                                                                                                                                          [100%]

========================================================================================== ERRORS ===========================================================================================
_____________________________________________________________________ ERROR at setup of Test_Affinity.test_pinned_to_0 ______________________________________________________________________

cls = <class 'tests.test_affinity.Test_Affinity'>

    @classmethod
    def setUpClass(cls):
        """Construct the a copy of ClassicServer that embeds a sleep(0) into _dispatch and set affinity"""
        cls._orig_func = rpyc.core.protocol.Connection._dispatch

        def _sleepy_dispatch(self, data):
            time.sleep(0.0)
            return cls._orig_func(self, data)
        setattr(rpyc.core.protocol.Connection, '_dispatch', _sleepy_dispatch)
        cls.cfg = {'sync_request_timeout': 5}
        if sys.platform != "linux":
            print("Running Test_Affinity is less productive on non-linux systems...")
        try:
            cls._skip = None
            cls._os = None
            cls._supported = True
>           cls._os = support.import_module('os', fromlist=('sched_setaffinity', 'sched_getaffinity'))
E           NameError: name 'support' is not defined

tests/test_affinity.py:34: NameError
========================================================================================= FAILURES ==========================================================================================
______________________________________________________________________________ TestAttributes.test_properties _______________________________________________________________________________

self = <tests.test_attributes.TestAttributes testMethod=test_properties>

    def test_properties(self):
>       p = self.conn.modules["test_attributes"].Properties()

tests/test_attributes.py:29:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rpyc/core/service.py:132: in __getitem__
    self.__cache[name] = self.__getmodule(name)
rpyc/core/netref.py:239: in __call__
    return syncreq(_self, consts.HANDLE_CALL, args, kwargs)
rpyc/core/netref.py:63: in syncreq
    return conn.sync_request(handler, proxy, *args)
rpyc/core/protocol.py:744: in sync_request
    return _async_res.value
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <AsyncResult object (ready) at 0x7f7d6c7d30e0>

    @property
    def value(self):
        """Returns the result of the operation. If the result has not yet
        arrived, accessing this property will wait for it. If the result does
        not arrive before the expiry time elapses, :class:`AsyncResultTimeout`
        is raised. If the returned result is an exception, it will be raised
        here. Otherwise, the result is returned directly.
        """
        self.wait()
        if self._is_exc:
>           raise self._obj
E           _get_exception_class.<locals>.Derived: No module named 'test_attributes'
E
E           ========= Remote Traceback (1) =========
E           Traceback (most recent call last):
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 369, in _dispatch_request
E               res = self._HANDLERS[handler](self, *args)
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 863, in _handle_call
E               return obj(*args, **dict(kwargs))
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/service.py", line 162, in getmodule
E               return importlib.import_module(name)
E             File "/usr/lib64/python3.8/importlib/__init__.py", line 127, in import_module
E               return _bootstrap._gcd_import(name[level:], package, level)
E             File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
E             File "<frozen importlib._bootstrap>", line 991, in _find_and_load
E             File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
E           ModuleNotFoundError: No module named 'test_attributes'

rpyc/core/async_.py:111: ModuleNotFoundError
_________________________________________________________________________________ ClassicMode.test_modules __________________________________________________________________________________

self = <tests.test_classic.ClassicMode testMethod=test_modules>

    def test_modules(self):
>       self.assertIn('test_magic', self.conn.modules)
E       AssertionError: 'test_magic' not found in <rpyc.core.service.ModuleNamespace object at 0x7f7d6d0a0040>

tests/test_classic.py:55: AssertionError
____________________________________________________________________________ TestContextManagers.test_hash_class ____________________________________________________________________________

self = <rpyc.core.service.ModuleNamespace object at 0x7f7d6c79b900>, name = 'test_magic'

    def __getattr__(self, name):
        """Provides dot notation access to modules"""
        try:
>           return self[name]

rpyc/core/service.py:138:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rpyc/core/service.py:132: in __getitem__
    self.__cache[name] = self.__getmodule(name)
rpyc/core/netref.py:239: in __call__
    return syncreq(_self, consts.HANDLE_CALL, args, kwargs)
rpyc/core/netref.py:63: in syncreq
    return conn.sync_request(handler, proxy, *args)
rpyc/core/protocol.py:744: in sync_request
    return _async_res.value
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <AsyncResult object (ready) at 0x7f7d6c7bc0e0>

    @property
    def value(self):
        """Returns the result of the operation. If the result has not yet
        arrived, accessing this property will wait for it. If the result does
        not arrive before the expiry time elapses, :class:`AsyncResultTimeout`
        is raised. If the returned result is an exception, it will be raised
        here. Otherwise, the result is returned directly.
        """
        self.wait()
        if self._is_exc:
>           raise self._obj
E           _get_exception_class.<locals>.Derived: No module named 'test_magic'
E
E           ========= Remote Traceback (1) =========
E           Traceback (most recent call last):
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 369, in _dispatch_request
E               res = self._HANDLERS[handler](self, *args)
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 863, in _handle_call
E               return obj(*args, **dict(kwargs))
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/service.py", line 162, in getmodule
E               return importlib.import_module(name)
E             File "/usr/lib64/python3.8/importlib/__init__.py", line 127, in import_module
E               return _bootstrap._gcd_import(name[level:], package, level)
E             File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
E             File "<frozen importlib._bootstrap>", line 991, in _find_and_load
E             File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
E           ModuleNotFoundError: No module named 'test_magic'

rpyc/core/async_.py:111: ModuleNotFoundError

During handling of the above exception, another exception occurred:

self = <tests.test_magic.TestContextManagers testMethod=test_hash_class>

    def test_hash_class(self):
        hesh = self.conn.builtins.hash
>       mod = self.conn.modules.test_magic

tests/test_magic.py:40:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <rpyc.core.service.ModuleNamespace object at 0x7f7d6c79b900>, name = 'test_magic'

    def __getattr__(self, name):
        """Provides dot notation access to modules"""
        try:
            return self[name]
        except ImportError:
>           raise AttributeError(name)
E           AttributeError: test_magic

rpyc/core/service.py:140: AttributeError
_____________________________________________________________________________ TestContextManagers.test_hash_obj _____________________________________________________________________________

self = <rpyc.core.service.ModuleNamespace object at 0x7f7d6c663780>, name = 'test_magic'

    def __getattr__(self, name):
        """Provides dot notation access to modules"""
        try:
>           return self[name]

rpyc/core/service.py:138:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rpyc/core/service.py:132: in __getitem__
    self.__cache[name] = self.__getmodule(name)
rpyc/core/netref.py:239: in __call__
    return syncreq(_self, consts.HANDLE_CALL, args, kwargs)
rpyc/core/netref.py:63: in syncreq
    return conn.sync_request(handler, proxy, *args)
rpyc/core/protocol.py:744: in sync_request
    return _async_res.value
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <AsyncResult object (ready) at 0x7f7d6c7894a0>

    @property
    def value(self):
        """Returns the result of the operation. If the result has not yet
        arrived, accessing this property will wait for it. If the result does
        not arrive before the expiry time elapses, :class:`AsyncResultTimeout`
        is raised. If the returned result is an exception, it will be raised
        here. Otherwise, the result is returned directly.
        """
        self.wait()
        if self._is_exc:
>           raise self._obj
E           _get_exception_class.<locals>.Derived: No module named 'test_magic'
E
E           ========= Remote Traceback (1) =========
E           Traceback (most recent call last):
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 369, in _dispatch_request
E               res = self._HANDLERS[handler](self, *args)
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 863, in _handle_call
E               return obj(*args, **dict(kwargs))
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/service.py", line 162, in getmodule
E               return importlib.import_module(name)
E             File "/usr/lib64/python3.8/importlib/__init__.py", line 127, in import_module
E               return _bootstrap._gcd_import(name[level:], package, level)
E             File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
E             File "<frozen importlib._bootstrap>", line 991, in _find_and_load
E             File "<frozen importlib._bootstrap>", line 973, in _find_and_load_unlocked
E           ModuleNotFoundError: No module named 'test_magic'

rpyc/core/async_.py:111: ModuleNotFoundError

During handling of the above exception, another exception occurred:

self = <tests.test_magic.TestContextManagers testMethod=test_hash_obj>

    def test_hash_obj(self):
        hesh = self.conn.builtins.hash
>       mod = self.conn.modules.test_magic

tests/test_magic.py:56:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <rpyc.core.service.ModuleNamespace object at 0x7f7d6c663780>, name = 'test_magic'

    def __getattr__(self, name):
        """Provides dot notation access to modules"""
        try:
            return self[name]
        except ImportError:
>           raise AttributeError(name)
E           AttributeError: test_magic

rpyc/core/service.py:140: AttributeError
________________________________________________________________ Test_Netref_Hierarchy.test_instancecheck_across_connections ________________________________________________________________

self = <tests.test_netref_hierachy.Test_Netref_Hierarchy testMethod=test_instancecheck_across_connections>

    def test_instancecheck_across_connections(self):
        self.conn2 = rpyc.classic.connect('localhost', port=18878)
>       self.conn.execute('import test_magic')

tests/test_netref_hierachy.py:111:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rpyc/core/netref.py:239: in __call__
    return syncreq(_self, consts.HANDLE_CALL, args, kwargs)
rpyc/core/netref.py:63: in syncreq
    return conn.sync_request(handler, proxy, *args)
rpyc/core/protocol.py:744: in sync_request
    return _async_res.value
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <AsyncResult object (ready) at 0x7f7d6d01f950>

    @property
    def value(self):
        """Returns the result of the operation. If the result has not yet
        arrived, accessing this property will wait for it. If the result does
        not arrive before the expiry time elapses, :class:`AsyncResultTimeout`
        is raised. If the returned result is an exception, it will be raised
        here. Otherwise, the result is returned directly.
        """
        self.wait()
        if self._is_exc:
>           raise self._obj
E           _get_exception_class.<locals>.Derived: No module named 'test_magic'
E
E           ========= Remote Traceback (1) =========
E           Traceback (most recent call last):
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 369, in _dispatch_request
E               res = self._HANDLERS[handler](self, *args)
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 863, in _handle_call
E               return obj(*args, **dict(kwargs))
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/service.py", line 152, in execute
E               execute(text, self.namespace)
E             File "<string>", line 1, in <module>
E           ModuleNotFoundError: No module named 'test_magic'

rpyc/core/async_.py:111: ModuleNotFoundError
_________________________________________________________________________ TestServicePickle.test_dataframe_pickling _________________________________________________________________________

self = <tests.test_service_pickle.TestServicePickle testMethod=test_dataframe_pickling>

    def setUp(self):
        self.cfg = {'allow_pickle': True}
        self.server = rpyc.utils.server.ThreadedServer(MyService, port=0, protocol_config=self.cfg.copy())
        self.server.logger.quiet = False
        self.thd = self.server._start_in_thread()
        self.conn = rpyc.connect("localhost", self.server.port, config=self.cfg)
        self.conn2 = rpyc.connect("localhost", self.server.port, config=self.cfg)
        # globals are made available to timeit, prepare them
>       cfg_tests.timeit['conn'] = self.conn
E       NameError: name 'cfg_tests' is not defined

tests/test_service_pickle.py:61: NameError
================================================================================== short test summary info ==================================================================================
SKIPPED [1] tests/test_deploy.py:86: Paramiko is not available
SKIPPED [1] tests/test_gevent_server.py:27: Gevent is not available
SKIPPED [1] tests/test_gevent_server.py:33: Gevent is not available
SKIPPED [1] tests/test_gevent_server.py:49: Gevent is not available
SKIPPED [1] tests/test_ipv6.py:19: requires IPv6
SKIPPED [1] tests/test_remoting.py:34: TODO: upload a package and a module
SKIPPED [1] tests/test_remoting.py:38: Requires manual testing atm
SKIPPED [1] tests/test_remoting.py:43: Requires manual testing atm
SKIPPED [1] tests/test_urllib3.py:28: urllib3 not available
SKIPPED [1] tests/test_win32pipes.py:11: Requires windows
SKIPPED [1] tests/test_win32pipes.py:24: Requires windows
SKIPPED [1] tests/test_win32pipes.py:56: Requires windows
ERROR tests/test_affinity.py::Test_Affinity::test_pinned_to_0 - NameError: name 'support' is not defined
FAILED tests/test_attributes.py::TestAttributes::test_properties - _get_exception_class.<locals>.Derived: No module named 'test_attributes'
FAILED tests/test_classic.py::ClassicMode::test_modules - AssertionError: 'test_magic' not found in <rpyc.core.service.ModuleNamespace object at 0x7f7d6d0a0040>
FAILED tests/test_magic.py::TestContextManagers::test_hash_class - AttributeError: test_magic
FAILED tests/test_magic.py::TestContextManagers::test_hash_obj - AttributeError: test_magic
FAILED tests/test_netref_hierachy.py::Test_Netref_Hierarchy::test_instancecheck_across_connections - _get_exception_class.<locals>.Derived: No module named 'test_magic'
FAILED tests/test_service_pickle.py::TestServicePickle::test_dataframe_pickling - NameError: name 'cfg_tests' is not defined
=============================================================== 6 failed, 76 passed, 12 skipped, 1 error in 69.26s (0:01:09) ================================================================

Looks like more fixes around imports are needed.

@kloczek
Copy link
Contributor Author

kloczek commented Apr 6, 2024

Here is improved version of above patch.

--- a/tests/test_service_pickle.py
+++ b/tests/test_service_pickle.py
@@ -4,7 +4,7 @@
 import timeit
 import rpyc
 import unittest
-import cfg_tests
+from tests import cfg_tests
 try:
     import pandas as pd
     import numpy as np
--- a/tests/test_affinity.py
+++ b/tests/test_affinity.py
@@ -1,7 +1,7 @@
 import sys
 import time
 import unittest
-import support
+from tests import support
 import rpyc


--- a/tests/test_attributes.py
+++ b/tests/test_attributes.py
@@ -26,7 +26,7 @@
         self.conn.close()

     def test_properties(self):
-        p = self.conn.modules["test_attributes"].Properties()
+        p = self.conn.modules["tests.test_attributes"].Properties()
         print(p.counter)  # 1
         print(p.counter)  # 2
         print(p.counter)  # 3
--- a/tests/test_classic.py
+++ b/tests/test_classic.py
@@ -52,7 +52,7 @@
         self.assertEqual(conn.eval("2+3"), 5)

     def test_modules(self):
-        self.assertIn('test_magic', self.conn.modules)
+        self.assertIn('tests.test_magic', self.conn.modules)
         self.assertNotIn('test_badmagic', self.conn.modules)
         self.assertIsNone(self.conn.builtins.locals()['self']._last_traceback)

--- a/tests/test_magic.py
+++ b/tests/test_magic.py
@@ -37,7 +37,7 @@

     def test_hash_class(self):
         hesh = self.conn.builtins.hash
-        mod = self.conn.modules.test_magic
+        mod = self.conn.modules.tests.test_magic
         self.assertEqual(hash(mod.Base), 4321)
         self.assertEqual(hash(mod.Foo), 4321)
         self.assertEqual(hash(mod.Bar), 4321)
@@ -53,7 +53,7 @@

     def test_hash_obj(self):
         hesh = self.conn.builtins.hash
-        mod = self.conn.modules.test_magic
+        mod = self.conn.modules.tests.test_magic
         obj = mod.Base()

         self.assertNotEqual(hash(obj), 1234)
--- a/tests/test_netref_hierachy.py
+++ b/tests/test_netref_hierachy.py
@@ -108,15 +108,15 @@

     def test_instancecheck_across_connections(self):
         self.conn2 = rpyc.classic.connect('localhost', port=18878)
-        self.conn.execute('import test_magic')
-        self.conn2.execute('import test_magic')
-        foo = self.conn.modules.test_magic.Foo()
-        bar = self.conn.modules.test_magic.Bar()
-        self.assertTrue(isinstance(foo, self.conn.modules.test_magic.Foo))
-        self.assertTrue(isinstance(bar, self.conn2.modules.test_magic.Bar))
-        self.assertFalse(isinstance(bar, self.conn.modules.test_magic.Foo))
+        self.conn.execute('import tests.test_magic')
+        self.conn2.execute('import tests.test_magic')
+        foo = self.conn.modules.tests.test_magic.Foo()
+        bar = self.conn.modules.tests.test_magic.Bar()
+        self.assertTrue(isinstance(foo, self.conn.modules.tests.test_magic.Foo))
+        self.assertTrue(isinstance(bar, self.conn2.modules.tests.test_magic.Bar))
+        self.assertFalse(isinstance(bar, self.conn.modules.tests.test_magic.Foo))
         with self.assertRaises(TypeError):
-            isinstance(self.conn.modules.test_magic.Foo, bar)
+            isinstance(self.conn.modules.tests.test_magic.Foo, bar)

     def test_classic(self):
         x = self.conn.builtin.list((1, 2, 3, 4))

With above pytest still fails with:

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages
+ /usr/bin/pytest -ra -m 'not network'
==================================================================================== test session starts ====================================================================================
platform linux -- Python 3.10.14, pytest-8.1.1, pluggy-1.4.0
rootdir: /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0
configfile: pyproject.toml
plugins: timeout-2.3.1
collected 95 items

tests/test_affinity.py .                                                                                                                                                              [  1%]
tests/test_async.py .....                                                                                                                                                             [  6%]
tests/test_attr_access.py .........                                                                                                                                                   [ 15%]
tests/test_attributes.py .                                                                                                                                                            [ 16%]
tests/test_brine.py .                                                                                                                                                                 [ 17%]
tests/test_classic.py .....                                                                                                                                                           [ 23%]
tests/test_context_managers.py ..                                                                                                                                                     [ 25%]
tests/test_custom_service.py .......                                                                                                                                                  [ 32%]
tests/test_dataclass.py .                                                                                                                                                             [ 33%]
tests/test_deploy.py ....                                                                                                                                                             [ 37%]
tests/test_gdb.py F                                                                                                                                                                   [ 38%]
tests/test_get_id_pack.py ....                                                                                                                                                        [ 43%]
tests/test_gevent_server.py ...                                                                                                                                                       [ 46%]
tests/test_ipv6.py s                                                                                                                                                                  [ 47%]
tests/test_magic.py ..                                                                                                                                                                [ 49%]
tests/test_netref_hierachy.py ...........                                                                                                                                             [ 61%]
tests/test_oneshot_server.py .                                                                                                                                                        [ 62%]
tests/test_race.py .                                                                                                                                                                  [ 63%]
tests/test_refcount.py .                                                                                                                                                              [ 64%]
tests/test_registry.py ......                                                                                                                                                         [ 70%]
tests/test_remote_exception.py ..                                                                                                                                                     [ 72%]
tests/test_remoting.py s.s.s                                                                                                                                                          [ 77%]
tests/test_rpyc_over_rpyc.py ...                                                                                                                                                      [ 81%]
tests/test_service_pickle.py .                                                                                                                                                        [ 82%]
tests/test_ssh.py EE                                                                                                                                                                  [ 84%]
tests/test_ssl.py .                                                                                                                                                                   [ 85%]
tests/test_teleportation.py ......                                                                                                                                                    [ 91%]
tests/test_threaded_server.py ...                                                                                                                                                     [ 94%]
tests/test_threads.py .                                                                                                                                                               [ 95%]
tests/test_urllib3.py s                                                                                                                                                               [ 96%]
tests/test_win32pipes.py sss                                                                                                                                                          [100%]

========================================================================================== ERRORS ===========================================================================================
__________________________________________________________________________ ERROR at setup of Test_Ssh.test_connect __________________________________________________________________________

cls = <class 'tests.test_ssh.Test_Ssh'>

    @classmethod
    def setUpClass(cls):
        if sys.platform == "win32":
            cls.server = None
            os.environ["HOME"] = os.path.expanduser("~")
        else:
            # assume "ssh localhost" is configured to run without asking for password
            # `.ssh/config`
            # Host localhost
            #   HostName 127.0.0.1
            #   User <username>
            #   IdentityFile <id_rsa>
            cls.server = ThreadedServer(SlaveService, hostname="localhost",
                                        ipv6=False, port=18888, auto_register=False)
            cls.server._start_in_thread()
>       cls.remote_machine = SshMachine("localhost")

tests/test_ssh.py:34:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:159: in __init__
    BaseRemoteMachine.__init__(
/usr/lib/python3.10/site-packages/plumbum/machines/remote.py:172: in __init__
    self._session = self.session(new_session=new_session)
/usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:243: in session
    self.popen(
/usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:190: in popen
    return self._ssh_command[tuple(cmdline)].popen(**kwargs)
/usr/lib/python3.10/site-packages/plumbum/commands/base.py:332: in popen
    return self.cmd.popen(self.args + list(args), **kwargs)
/usr/lib/python3.10/site-packages/plumbum/machines/local.py:115: in popen
    return self.machine._popen(
/usr/lib/python3.10/site-packages/plumbum/machines/local.py:295: in _popen
    proc = PlumbumLocalPopen(
/usr/lib/python3.10/site-packages/plumbum/machines/local.py:29: in __init__
    self._proc = Popen(*args, **kwargs)  # pylint: disable=consider-using-with
/usr/lib64/python3.10/subprocess.py:971: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Popen: returncode: None args: ['/usr/bin/ssh', '-T', 'localhost', '/bin/sh']>, args = ['/usr/bin/ssh', '-T', 'localhost', '/bin/sh'], executable = b'/usr/bin/ssh', preexec_fn = None
close_fds = True, pass_fds = (), cwd = '/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0'
env = {'AR': '/usr/bin/gcc-ar', 'ASMFLAGS': '-m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-...cc1 -flto=auto -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -Wall -Werror=format-security', ...}
startupinfo = None, creationflags = 0, shell = False, p2cread = 14, p2cwrite = 17, c2pread = 19, c2pwrite = 20, errread = 21, errwrite = 22, restore_signals = True, gid = None, gids = None
uid = None, umask = -1, start_new_session = False

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       restore_signals,
                       gid, gids, uid, umask,
                       start_new_session):
        """Execute program (POSIX version)"""

        if isinstance(args, (str, bytes)):
            args = [args]
        elif isinstance(args, os.PathLike):
            if shell:
                raise TypeError('path-like args is not allowed when '
                                'shell is true')
            args = [args]
        else:
            args = list(args)

        if shell:
            # On Android the default shell is at '/system/bin/sh'.
            unix_shell = ('/system/bin/sh' if
                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')
            args = [unix_shell, "-c"] + args
            if executable:
                args[0] = executable

        if executable is None:
            executable = args[0]

        sys.audit("subprocess.Popen", executable, args, cwd, env)

        if (_USE_POSIX_SPAWN
                and os.path.dirname(executable)
                and preexec_fn is None
                and not close_fds
                and not pass_fds
                and cwd is None
                and (p2cread == -1 or p2cread > 2)
                and (c2pwrite == -1 or c2pwrite > 2)
                and (errwrite == -1 or errwrite > 2)
                and not start_new_session
                and gid is None
                and gids is None
                and uid is None
                and umask < 0):
            self._posix_spawn(args, executable, env, restore_signals,
                              p2cread, p2cwrite,
                              c2pread, c2pwrite,
                              errread, errwrite)
            return

        orig_executable = executable

        # For transferring possible exec failure from child to parent.
        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()

                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
>               self.pid = _posixsubprocess.fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session,
                        gid, gids, uid, umask,
                        preexec_fn)
E                       AttributeError: 'NoneType' object has no attribute 'fork_exec'

/usr/lib64/python3.10/subprocess.py:1796: AttributeError
__________________________________________________________________________ ERROR at setup of Test_Ssh.test_simple ___________________________________________________________________________

cls = <class 'tests.test_ssh.Test_Ssh'>

    @classmethod
    def setUpClass(cls):
        if sys.platform == "win32":
            cls.server = None
            os.environ["HOME"] = os.path.expanduser("~")
        else:
            # assume "ssh localhost" is configured to run without asking for password
            # `.ssh/config`
            # Host localhost
            #   HostName 127.0.0.1
            #   User <username>
            #   IdentityFile <id_rsa>
            cls.server = ThreadedServer(SlaveService, hostname="localhost",
                                        ipv6=False, port=18888, auto_register=False)
            cls.server._start_in_thread()
>       cls.remote_machine = SshMachine("localhost")

tests/test_ssh.py:34:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:159: in __init__
    BaseRemoteMachine.__init__(
/usr/lib/python3.10/site-packages/plumbum/machines/remote.py:172: in __init__
    self._session = self.session(new_session=new_session)
/usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:243: in session
    self.popen(
/usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:190: in popen
    return self._ssh_command[tuple(cmdline)].popen(**kwargs)
/usr/lib/python3.10/site-packages/plumbum/commands/base.py:332: in popen
    return self.cmd.popen(self.args + list(args), **kwargs)
/usr/lib/python3.10/site-packages/plumbum/machines/local.py:115: in popen
    return self.machine._popen(
/usr/lib/python3.10/site-packages/plumbum/machines/local.py:295: in _popen
    proc = PlumbumLocalPopen(
/usr/lib/python3.10/site-packages/plumbum/machines/local.py:29: in __init__
    self._proc = Popen(*args, **kwargs)  # pylint: disable=consider-using-with
/usr/lib64/python3.10/subprocess.py:971: in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <Popen: returncode: None args: ['/usr/bin/ssh', '-T', 'localhost', '/bin/sh']>, args = ['/usr/bin/ssh', '-T', 'localhost', '/bin/sh'], executable = b'/usr/bin/ssh', preexec_fn = None
close_fds = True, pass_fds = (), cwd = '/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0'
env = {'AR': '/usr/bin/gcc-ar', 'ASMFLAGS': '-m64 -mtune=generic -fasynchronous-unwind-tables -fstack-clash-protection -fcf-...cc1 -flto=auto -Wp,-U_FORTIFY_SOURCE,-D_FORTIFY_SOURCE=3 -Wp,-D_GLIBCXX_ASSERTIONS -Wall -Werror=format-security', ...}
startupinfo = None, creationflags = 0, shell = False, p2cread = 14, p2cwrite = 17, c2pread = 19, c2pwrite = 20, errread = 21, errwrite = 22, restore_signals = True, gid = None, gids = None
uid = None, umask = -1, start_new_session = False

    def _execute_child(self, args, executable, preexec_fn, close_fds,
                       pass_fds, cwd, env,
                       startupinfo, creationflags, shell,
                       p2cread, p2cwrite,
                       c2pread, c2pwrite,
                       errread, errwrite,
                       restore_signals,
                       gid, gids, uid, umask,
                       start_new_session):
        """Execute program (POSIX version)"""

        if isinstance(args, (str, bytes)):
            args = [args]
        elif isinstance(args, os.PathLike):
            if shell:
                raise TypeError('path-like args is not allowed when '
                                'shell is true')
            args = [args]
        else:
            args = list(args)

        if shell:
            # On Android the default shell is at '/system/bin/sh'.
            unix_shell = ('/system/bin/sh' if
                      hasattr(sys, 'getandroidapilevel') else '/bin/sh')
            args = [unix_shell, "-c"] + args
            if executable:
                args[0] = executable

        if executable is None:
            executable = args[0]

        sys.audit("subprocess.Popen", executable, args, cwd, env)

        if (_USE_POSIX_SPAWN
                and os.path.dirname(executable)
                and preexec_fn is None
                and not close_fds
                and not pass_fds
                and cwd is None
                and (p2cread == -1 or p2cread > 2)
                and (c2pwrite == -1 or c2pwrite > 2)
                and (errwrite == -1 or errwrite > 2)
                and not start_new_session
                and gid is None
                and gids is None
                and uid is None
                and umask < 0):
            self._posix_spawn(args, executable, env, restore_signals,
                              p2cread, p2cwrite,
                              c2pread, c2pwrite,
                              errread, errwrite)
            return

        orig_executable = executable

        # For transferring possible exec failure from child to parent.
        # Data format: "exception name:hex errno:description"
        # Pickle is not used; it is complex and involves memory allocation.
        errpipe_read, errpipe_write = os.pipe()
        # errpipe_write must not be in the standard io 0, 1, or 2 fd range.
        low_fds_to_close = []
        while errpipe_write < 3:
            low_fds_to_close.append(errpipe_write)
            errpipe_write = os.dup(errpipe_write)
        for low_fd in low_fds_to_close:
            os.close(low_fd)
        try:
            try:
                # We must avoid complex work that could involve
                # malloc or free in the child process to avoid
                # potential deadlocks, thus we do all this here.
                # and pass it to fork_exec()

                if env is not None:
                    env_list = []
                    for k, v in env.items():
                        k = os.fsencode(k)
                        if b'=' in k:
                            raise ValueError("illegal environment variable name")
                        env_list.append(k + b'=' + os.fsencode(v))
                else:
                    env_list = None  # Use execv instead of execve.
                executable = os.fsencode(executable)
                if os.path.dirname(executable):
                    executable_list = (executable,)
                else:
                    # This matches the behavior of os._execvpe().
                    executable_list = tuple(
                        os.path.join(os.fsencode(dir), executable)
                        for dir in os.get_exec_path(env))
                fds_to_keep = set(pass_fds)
                fds_to_keep.add(errpipe_write)
>               self.pid = _posixsubprocess.fork_exec(
                        args, executable_list,
                        close_fds, tuple(sorted(map(int, fds_to_keep))),
                        cwd, env_list,
                        p2cread, p2cwrite, c2pread, c2pwrite,
                        errread, errwrite,
                        errpipe_read, errpipe_write,
                        restore_signals, start_new_session,
                        gid, gids, uid, umask,
                        preexec_fn)
E                       AttributeError: 'NoneType' object has no attribute 'fork_exec'

/usr/lib64/python3.10/subprocess.py:1796: AttributeError
========================================================================================= FAILURES ==========================================================================================
_____________________________________________________________________________________ Test_GDB.test_gdb _____________________________________________________________________________________

self = <tests.test_gdb.Test_GDB testMethod=test_gdb>

    def test_gdb(self):
        print(0)
        parent_gdb_conn = rpyc.connect(host='localhost', port=18878)
        print(1)
        gdb = parent_gdb_conn.root.get_gdb()
        print(2)
>       gdb.execute('file {}'.format(self.a_out))

tests/test_gdb.py:63:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
rpyc/core/netref.py:239: in __call__
    return syncreq(_self, consts.HANDLE_CALL, args, kwargs)
rpyc/core/netref.py:63: in syncreq
    return conn.sync_request(handler, proxy, *args)
rpyc/core/protocol.py:744: in sync_request
    return _async_res.value
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <AsyncResult object (ready) at 0x7f7c1e967bf0>

    @property
    def value(self):
        """Returns the result of the operation. If the result has not yet
        arrived, accessing this property will wait for it. If the result does
        not arrive before the expiry time elapses, :class:`AsyncResultTimeout`
        is raised. If the returned result is an exception, it will be raised
        here. Otherwise, the result is returned directly.
        """
        self.wait()
        if self._is_exc:
>           raise self._obj
E           _get_exception_class.<locals>.Derived: connection closed by peer
E
E           ========= Remote Traceback (1) =========
E           Traceback (most recent call last):
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 369, in _dispatch_request
E               res = self._HANDLERS[handler](self, *args)
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 863, in _handle_call
E               return obj(*args, **dict(kwargs))
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/netref.py", line 239, in __call__
E               return syncreq(_self, consts.HANDLE_CALL, args, kwargs)
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/netref.py", line 63, in syncreq
E               return conn.sync_request(handler, proxy, *args)
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 744, in sync_request
E               return _async_res.value
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/async_.py", line 109, in value
E               self.wait()
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/async_.py", line 51, in wait
E               self._conn.serve(self._ttl, waiting=self._waiting)
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/protocol.py", line 464, in serve
E               data = self._channel.poll(timeout) and self._channel.recv()
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/channel.py", line 55, in recv
E               header = self.stream.read(self.FRAME_HEADER.size)
E             File "/home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/core/stream.py", line 280, in read
E               raise EOFError("connection closed by peer")
E           EOFError: connection closed by peer

rpyc/core/async_.py:111: EOFError
----------------------------------------------------------------------------------- Captured stdout call ------------------------------------------------------------------------------------
0
1
36539
2
===================================================================================== warnings summary ======================================================================================
tests/test_gevent_server.py::Test_GeventServer::test_connection
  /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/tests/test_gevent_server.py:18: MonkeyPatchWarning: Monkey-patching ssl after ssl has already been imported may lead to errors, including RecursionError on Python 3.6. It may also silently lead to incorrect behaviour on Python 3.7. Please monkey-patch earlier. See https://github.com/gevent/gevent/issues/1016. Modules that had direct imports (NOT patched): ['urllib3.util.ssl_ (/usr/lib/python3.10/site-packages/urllib3/util/ssl_.py)', 'urllib3.util (/usr/lib/python3.10/site-packages/urllib3/util/__init__.py)'].
    monkey.patch_all()

tests/test_threaded_server.py::Test_ThreadPoolServer::test_connection
  /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/utils/server.py:349: DeprecationWarning: setName() is deprecated, set the name attribute instead
    self.polling_thread.setName('PollingThread')

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
================================================================================== short test summary info ==================================================================================
SKIPPED [1] tests/test_ipv6.py:19: requires IPv6
SKIPPED [1] tests/test_remoting.py:34: TODO: upload a package and a module
SKIPPED [1] tests/test_remoting.py:38: Requires manual testing atm
SKIPPED [1] tests/test_remoting.py:43: Requires manual testing atm
SKIPPED [1] tests/test_urllib3.py:28: urllib3 not available
SKIPPED [1] tests/test_win32pipes.py:11: Requires windows
SKIPPED [1] tests/test_win32pipes.py:24: Requires windows
SKIPPED [1] tests/test_win32pipes.py:56: Requires windows
ERROR tests/test_ssh.py::Test_Ssh::test_connect - AttributeError: 'NoneType' object has no attribute 'fork_exec'
ERROR tests/test_ssh.py::Test_Ssh::test_simple - AttributeError: 'NoneType' object has no attribute 'fork_exec'
FAILED tests/test_gdb.py::Test_GDB::test_gdb - _get_exception_class.<locals>.Derived: connection closed by peer
========================================================= 1 failed, 84 passed, 8 skipped, 2 warnings, 2 errors in 99.44s (0:01:39) ==========================================================

kloczek added a commit to kloczek/rpyc that referenced this issue May 17, 2024
Signed-off-by: Tomasz Kłoczko <kloczek@github.com>
kloczek added a commit to kloczek/rpyc that referenced this issue May 17, 2024
Signed-off-by: Tomasz Kłoczko <kloczek@github.com>
@kloczek
Copy link
Contributor Author

kloczek commented May 17, 2024

I've added #557

@kloczek
Copy link
Contributor Author

kloczek commented May 17, 2024

After send prod build request to my build infra I found that units which requires running ssh server on local hosts are failing

Here is pytest output:
+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc37.x86_64/usr/lib64/python3.10/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-rpyc-6.0.0-2.fc37.x86_64/usr/lib/python3.10/site-packages
+ /usr/bin/pytest -ra -m 'not network' --deselect tests/test_gdb.py::Test_GDB::test_gdb
============================= test session starts ==============================
platform linux -- Python 3.10.14, pytest-8.1.1, pluggy-1.4.0
rootdir: /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0
configfile: pyproject.toml
collected 95 items / 1 deselected / 94 selected

tests/test_affinity.py .                                                 [  1%]
tests/test_async.py .....                                                [  6%]
tests/test_attr_access.py .........                                      [ 15%]
tests/test_attributes.py .                                               [ 17%]
tests/test_brine.py .                                                    [ 18%]
tests/test_classic.py .....                                              [ 23%]
tests/test_context_managers.py ..                                        [ 25%]
tests/test_custom_service.py .......                                     [ 32%]
tests/test_dataclass.py .                                                [ 34%]
tests/test_deploy.py FFFs                                                [ 38%]
tests/test_get_id_pack.py ....                                           [ 42%]
tests/test_gevent_server.py sss                                          [ 45%]
tests/test_ipv6.py s                                                     [ 46%]
tests/test_magic.py ..                                                   [ 48%]
tests/test_netref_hierachy.py ...........                                [ 60%]
tests/test_oneshot_server.py .                                           [ 61%]
tests/test_race.py .                                                     [ 62%]
tests/test_refcount.py .                                                 [ 63%]
tests/test_registry.py ......                                            [ 70%]
tests/test_remote_exception.py ..                                        [ 72%]
tests/test_remoting.py s.s.s                                             [ 77%]
tests/test_rpyc_over_rpyc.py ...                                         [ 80%]
tests/test_service_pickle.py .                                           [ 81%]
tests/test_ssh.py ss                                                     [ 84%]
tests/test_ssl.py .                                                      [ 85%]
tests/test_teleportation.py ......                                       [ 91%]
tests/test_threaded_server.py ...                                        [ 94%]
tests/test_threads.py .                                                  [ 95%]
tests/test_urllib3.py s                                                  [ 96%]
tests/test_win32pipes.py sss                                             [100%]

=================================== FAILURES ===================================
________________________ TestDeploy.test_close_timeout _________________________

self = <tests.test_deploy.TestDeploy testMethod=test_close_timeout>

    def test_close_timeout(self):
        expected_timeout = 4
        observed_timeouts = []
        original_communicate = subprocess.Popen.communicate

        def replacement_communicate(self, input=None, timeout=None):
            observed_timeouts.append(timeout)
            return original_communicate(self, input, timeout)

        try:
            subprocess.Popen.communicate = replacement_communicate
>           rem = SshMachine("localhost")

tests/test_deploy.py:47:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:159: in __init__
    BaseRemoteMachine.__init__(
/usr/lib/python3.10/site-packages/plumbum/machines/remote.py:172: in __init__
    self._session = self.session(new_session=new_session)
/usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:242: in session
    return ShellSession(
/usr/lib/python3.10/site-packages/plumbum/machines/session.py:233: in __init__
    self._startup_result = self.run("")
/usr/lib/python3.10/site-packages/plumbum/machines/session.py:316: in run
    return run_proc(self.popen(cmd), retcode)
/usr/lib/python3.10/site-packages/plumbum/commands/processes.py:318: in run_proc
    stdout, stderr = proc.communicate()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <plumbum.machines.session.SessionPopen object at 0x7f03d5bd1a50>
input = None

    def communicate(self, input=None):  # pylint: disable=redefined-builtin
        """Consumes the process' stdout and stderr until the it terminates.

        :param input: An optional bytes/buffer object to send to the process over stdin
        :returns: A tuple of (stdout, stderr)
        """
        stdout = []
        stderr = []
        sources = [("1", stdout, self.stdout)]
        if not self.isatty:
            # in tty mode, stdout and stderr are unified
            sources.append(("2", stderr, self.stderr))
        i = 0
        while sources:
            if input:
                chunk = input[:1000]
                self.stdin.write(chunk)
                self.stdin.flush()
                input = input[1000:]
            i = (i + 1) % len(sources)
            name, coll, pipe = sources[i]
            try:
                line = pipe.readline()
                shell_logger.debug("%s> %r", name, line)
            except EOFError as err:
                shell_logger.debug("%s> Nothing returned.", name)

                self.proc.poll()
                returncode = self.proc.returncode
                stdout = b"".join(stdout).decode(self.custom_encoding, "ignore")
                stderr = b"".join(stderr).decode(self.custom_encoding, "ignore")
                argv = self.argv.decode(self.custom_encoding, "ignore").split(";")[:1]

                if returncode == 5:
                    raise IncorrectLogin(
                        argv,
                        returncode,
                        stdout,
                        stderr,
                        message="Incorrect username or password provided",
                        host=self.host,
                    ) from None
                if returncode == 6:
                    raise HostPublicKeyUnknown(
                        argv,
                        returncode,
                        stdout,
                        stderr,
                        message="The authenticity of the host can't be established",
                        host=self.host,
                    ) from None
                if returncode != 0:
>                   raise SSHCommsError(
                        argv,
                        returncode,
                        stdout,
                        stderr,
                        message="SSH communication failed",
                        host=self.host,
                    ) from None
E                   plumbum.machines.session.SSHCommsError: SSH communication failed
E                   Return code:  | 255
E                   Command line: | 'true '
E                   Host:         | localhost
E                   Stderr:       | ssh: connect to host localhost port 22: Connection refused

/usr/lib/python3.10/site-packages/plumbum/machines/session.py:149: SSHCommsError
__________________ TestDeploy.test_close_timeout_default_none __________________

self = <tests.test_deploy.TestDeploy testMethod=test_close_timeout_default_none>

    def test_close_timeout_default_none(self):
        observed_timeouts = []
        original_communicate = subprocess.Popen.communicate

        def replacement_communicate(self, input=None, timeout=None):
            observed_timeouts.append(timeout)
            return original_communicate(self, input, timeout)

        try:
            subprocess.Popen.communicate = replacement_communicate
>           rem = SshMachine("localhost")

tests/test_deploy.py:71:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:159: in __init__
    BaseRemoteMachine.__init__(
/usr/lib/python3.10/site-packages/plumbum/machines/remote.py:172: in __init__
    self._session = self.session(new_session=new_session)
/usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:242: in session
    return ShellSession(
/usr/lib/python3.10/site-packages/plumbum/machines/session.py:233: in __init__
    self._startup_result = self.run("")
/usr/lib/python3.10/site-packages/plumbum/machines/session.py:316: in run
    return run_proc(self.popen(cmd), retcode)
/usr/lib/python3.10/site-packages/plumbum/commands/processes.py:318: in run_proc
    stdout, stderr = proc.communicate()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <plumbum.machines.session.SessionPopen object at 0x7f03d4f1b2e0>
input = None

    def communicate(self, input=None):  # pylint: disable=redefined-builtin
        """Consumes the process' stdout and stderr until the it terminates.

        :param input: An optional bytes/buffer object to send to the process over stdin
        :returns: A tuple of (stdout, stderr)
        """
        stdout = []
        stderr = []
        sources = [("1", stdout, self.stdout)]
        if not self.isatty:
            # in tty mode, stdout and stderr are unified
            sources.append(("2", stderr, self.stderr))
        i = 0
        while sources:
            if input:
                chunk = input[:1000]
                self.stdin.write(chunk)
                self.stdin.flush()
                input = input[1000:]
            i = (i + 1) % len(sources)
            name, coll, pipe = sources[i]
            try:
                line = pipe.readline()
                shell_logger.debug("%s> %r", name, line)
            except EOFError as err:
                shell_logger.debug("%s> Nothing returned.", name)

                self.proc.poll()
                returncode = self.proc.returncode
                stdout = b"".join(stdout).decode(self.custom_encoding, "ignore")
                stderr = b"".join(stderr).decode(self.custom_encoding, "ignore")
                argv = self.argv.decode(self.custom_encoding, "ignore").split(";")[:1]

                if returncode == 5:
                    raise IncorrectLogin(
                        argv,
                        returncode,
                        stdout,
                        stderr,
                        message="Incorrect username or password provided",
                        host=self.host,
                    ) from None
                if returncode == 6:
                    raise HostPublicKeyUnknown(
                        argv,
                        returncode,
                        stdout,
                        stderr,
                        message="The authenticity of the host can't be established",
                        host=self.host,
                    ) from None
                if returncode != 0:
>                   raise SSHCommsError(
                        argv,
                        returncode,
                        stdout,
                        stderr,
                        message="SSH communication failed",
                        host=self.host,
                    ) from None
E                   plumbum.machines.session.SSHCommsError: SSH communication failed
E                   Return code:  | 255
E                   Command line: | 'true '
E                   Host:         | localhost
E                   Stderr:       | ssh: connect to host localhost port 22: Connection refused

/usr/lib/python3.10/site-packages/plumbum/machines/session.py:149: SSHCommsError
____________________________ TestDeploy.test_deploy ____________________________

self = <tests.test_deploy.TestDeploy testMethod=test_deploy>

    def test_deploy(self):
>       rem = SshMachine("localhost")

tests/test_deploy.py:18:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:159: in __init__
    BaseRemoteMachine.__init__(
/usr/lib/python3.10/site-packages/plumbum/machines/remote.py:172: in __init__
    self._session = self.session(new_session=new_session)
/usr/lib/python3.10/site-packages/plumbum/machines/ssh_machine.py:242: in session
    return ShellSession(
/usr/lib/python3.10/site-packages/plumbum/machines/session.py:233: in __init__
    self._startup_result = self.run("")
/usr/lib/python3.10/site-packages/plumbum/machines/session.py:316: in run
    return run_proc(self.popen(cmd), retcode)
/usr/lib/python3.10/site-packages/plumbum/commands/processes.py:318: in run_proc
    stdout, stderr = proc.communicate()
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <plumbum.machines.session.SessionPopen object at 0x7f03d5a88160>
input = None

    def communicate(self, input=None):  # pylint: disable=redefined-builtin
        """Consumes the process' stdout and stderr until the it terminates.

        :param input: An optional bytes/buffer object to send to the process over stdin
        :returns: A tuple of (stdout, stderr)
        """
        stdout = []
        stderr = []
        sources = [("1", stdout, self.stdout)]
        if not self.isatty:
            # in tty mode, stdout and stderr are unified
            sources.append(("2", stderr, self.stderr))
        i = 0
        while sources:
            if input:
                chunk = input[:1000]
                self.stdin.write(chunk)
                self.stdin.flush()
                input = input[1000:]
            i = (i + 1) % len(sources)
            name, coll, pipe = sources[i]
            try:
                line = pipe.readline()
                shell_logger.debug("%s> %r", name, line)
            except EOFError as err:
                shell_logger.debug("%s> Nothing returned.", name)

                self.proc.poll()
                returncode = self.proc.returncode
                stdout = b"".join(stdout).decode(self.custom_encoding, "ignore")
                stderr = b"".join(stderr).decode(self.custom_encoding, "ignore")
                argv = self.argv.decode(self.custom_encoding, "ignore").split(";")[:1]

                if returncode == 5:
                    raise IncorrectLogin(
                        argv,
                        returncode,
                        stdout,
                        stderr,
                        message="Incorrect username or password provided",
                        host=self.host,
                    ) from None
                if returncode == 6:
                    raise HostPublicKeyUnknown(
                        argv,
                        returncode,
                        stdout,
                        stderr,
                        message="The authenticity of the host can't be established",
                        host=self.host,
                    ) from None
                if returncode != 0:
>                   raise SSHCommsError(
                        argv,
                        returncode,
                        stdout,
                        stderr,
                        message="SSH communication failed",
                        host=self.host,
                    ) from None
E                   plumbum.machines.session.SSHCommsError: SSH communication failed
E                   Return code:  | 255
E                   Command line: | 'true '
E                   Host:         | localhost
E                   Stderr:       | ssh: connect to host localhost port 22: Connection refused

/usr/lib/python3.10/site-packages/plumbum/machines/session.py:149: SSHCommsError
=============================== warnings summary ===============================
tests/test_threaded_server.py::Test_ThreadPoolServer::test_connection
  /home/tkloczko/rpmbuild/BUILD/rpyc-6.0.0/rpyc/utils/server.py:349: DeprecationWarning: setName() is deprecated, set the name attribute instead
    self.polling_thread.setName('PollingThread')

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
=========================== short test summary info ============================
SKIPPED [1] tests/test_deploy.py:84: Paramiko is not available
SKIPPED [1] tests/test_gevent_server.py:27: Gevent is not available
SKIPPED [1] tests/test_gevent_server.py:33: Gevent is not available
SKIPPED [1] tests/test_gevent_server.py:49: Gevent is not available
SKIPPED [1] tests/test_ipv6.py:19: requires IPv6
SKIPPED [1] tests/test_remoting.py:34: TODO: upload a package and a module
SKIPPED [1] tests/test_remoting.py:38: Requires manual testing atm
SKIPPED [1] tests/test_remoting.py:43: Requires manual testing atm
SKIPPED [1] tests/test_ssh.py:53: Requires paramiko_machine to localhost
SKIPPED [1] tests/test_ssh.py:48: Requires paramiko_machine to localhost
SKIPPED [1] tests/test_urllib3.py:28: urllib3 not available
SKIPPED [1] tests/test_win32pipes.py:11: Requires windows
SKIPPED [1] tests/test_win32pipes.py:24: Requires windows
SKIPPED [1] tests/test_win32pipes.py:56: Requires windows
FAILED tests/test_deploy.py::TestDeploy::test_close_timeout - plumbum.machine...
FAILED tests/test_deploy.py::TestDeploy::test_close_timeout_default_none - pl...
FAILED tests/test_deploy.py::TestDeploy::test_deploy - plumbum.machines.sessi...
= 3 failed, 77 passed, 14 skipped, 1 deselected, 1 warning in 90.22s (0:01:30) =

I think that it would be better if test suite in session will start sshd on some high port and use it to test module remote calls.

comrumino added a commit that referenced this issue May 22, 2024
fix pytest issues reported in #544
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Triage Investigation by a maintainer has started
Projects
None yet
Development

No branches or pull requests

2 participants