Async Worker (python RQ) Are Stuck #334

adiarbili · 2021-12-16T17:18:44Z

adiarbili
Dec 16, 2021

For general questions please use the mail group.

Describe the bug
I'm using PSSH package in async workers such as python RQ (https://python-rq.org/).
The worker is getting stuck when it tries to connect to a remote server via SSHClient (native).

To Reproduce

Steps to reproduce the behavior:
The issue is reproduced only if I import the module that contains PSSH import (e.g. tasks see below) and you need to import it in the worker code (worker.py) at the beginning of the module. If I put it in main() function, everything is working just fine.

Example code that produces error:
worker.py:

import redis
from rq import Queue, Worker, Connection
import tasks

def main():
    with Connection(redis.from_url("redis://redis:6379/0")):
        queue = Queue('default_prio_queue')
        worker = Worker([queue])
        worker.work()

if __name__ == "__main__":
    main()

main.py:

import redis
from rq import Queue, Worker, Connection

from tasks import remote


def queue_tasks():
    print("queue_tasks")
    with Connection(redis.from_url("redis://redis:6379/0")):
        queue = Queue('default_prio_queue', default_timeout=10)
        queue.enqueue(remote)


queue_tasks()

tasks.py:

import logging
from pssh.clients import SSHClient

def remote():
    output_list = []
    logging.error("Before connection")
    ssh_client = SSHClient("xxx", user="xxx", password="xxx", pkey='xxx')
    logging.error("After connection")
    logging.error("Before Execution")
    host_output_obj = ssh_client.run_command("ls -la", timeout=10)
    for line in host_output_obj.stdout:
        output_list.append(line)
    logging.error(output_list)
    logging.error("After Execution")

Stack trace or error messages:
I added prints in order to track where the command get stuck, it seems that it is stuck on the function socket.getaddrinfo():

    def _get_addr_info(self, host, port):
        logger.error("Before getaddrinfo")
        addr_info = socket.getaddrinfo(host, port, proto=socket.IPPROTO_TCP)

Here are the lines with my print and after the last line the code is stuck:

2021-12-16T17:06:21.062561270Z ERROR:root:Before connection
2021-12-16T17:06:21.063960028Z ERROR:pssh.clients.base.single:Before init
2021-12-16T17:06:21.064232437Z ERROR:pssh.clients.base.single:Before _connect
2021-12-16T17:06:21.064278272Z ERROR:pssh.clients.base.single:Before _get_addr_info
2021-12-16T17:06:21.064367307Z ERROR:pssh.clients.base.single:Before getaddrinfo

The following is a full trace, I think that not all of the modules were loaded:

2021-12-16T17:06:16.412304937Z import _frozen_importlib # frozen
2021-12-16T17:06:16.412374578Z import _imp # builtin
2021-12-16T17:06:16.412844866Z import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.412873545Z import '_warnings' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.413008438Z import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.413704156Z import '_io' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.413779752Z import 'marshal' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.414205201Z import 'posix' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.414511877Z import '_frozen_importlib_external' # <class '_frozen_importlib.FrozenImporter'>
2021-12-16T17:06:16.414560188Z import _thread # previously loaded ('_thread')
2021-12-16T17:06:16.414568246Z import '_thread' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.414574574Z import _weakref # previously loaded ('_weakref')
2021-12-16T17:06:16.414580698Z import '_weakref' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.414612923Z # installing zipimport hook
2021-12-16T17:06:16.415182549Z import 'time' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.415287586Z import 'zipimport' # <class '_frozen_importlib.FrozenImporter'>
2021-12-16T17:06:16.415298976Z # installed zipimport hook
2021-12-16T17:06:16.421698170Z # /usr/local/lib/python3.8/encodings/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/encodings/init.py
2021-12-16T17:06:16.421715915Z # code object from '/usr/local/lib/python3.8/encodings/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.422044701Z # /usr/local/lib/python3.8/pycache/codecs.cpython-38.pyc matches /usr/local/lib/python3.8/codecs.py
2021-12-16T17:06:16.422224316Z # code object from '/usr/local/lib/python3.8/pycache/codecs.cpython-38.pyc'
2021-12-16T17:06:16.422315842Z import '_codecs' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.422576938Z import 'codecs' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa491c1a60>
2021-12-16T17:06:16.423122823Z # /usr/local/lib/python3.8/encodings/pycache/aliases.cpython-38.pyc matches /usr/local/lib/python3.8/encodings/aliases.py
2021-12-16T17:06:16.423240500Z # code object from '/usr/local/lib/python3.8/encodings/pycache/aliases.cpython-38.pyc'
2021-12-16T17:06:16.423249917Z import 'encodings.aliases' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa49154670>
2021-12-16T17:06:16.423349421Z import 'encodings' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa491c17c0>
2021-12-16T17:06:16.424046494Z # /usr/local/lib/python3.8/encodings/pycache/utf_8.cpython-38.pyc matches /usr/local/lib/python3.8/encodings/utf_8.py
2021-12-16T17:06:16.424072858Z # code object from '/usr/local/lib/python3.8/encodings/pycache/utf_8.cpython-38.pyc'
2021-12-16T17:06:16.424077393Z import 'encodings.utf_8' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa491c18e0>
2021-12-16T17:06:16.430382785Z import '_signal' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.430852198Z # /usr/local/lib/python3.8/encodings/pycache/latin_1.cpython-38.pyc matches /usr/local/lib/python3.8/encodings/latin_1.py
2021-12-16T17:06:16.430869257Z # code object from '/usr/local/lib/python3.8/encodings/pycache/latin_1.cpython-38.pyc'
2021-12-16T17:06:16.431094478Z import 'encodings.latin_1' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4916b160>
2021-12-16T17:06:16.431361652Z # /usr/local/lib/python3.8/pycache/io.cpython-38.pyc matches /usr/local/lib/python3.8/io.py
2021-12-16T17:06:16.431373064Z # code object from '/usr/local/lib/python3.8/pycache/io.cpython-38.pyc'
2021-12-16T17:06:16.431677598Z # /usr/local/lib/python3.8/pycache/abc.cpython-38.pyc matches /usr/local/lib/python3.8/abc.py
2021-12-16T17:06:16.431788112Z # code object from '/usr/local/lib/python3.8/pycache/abc.cpython-38.pyc'
2021-12-16T17:06:16.431875476Z import '_abc' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.432013333Z import 'abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4916b610>
2021-12-16T17:06:16.432299609Z import 'io' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4916b340>
2021-12-16T17:06:16.453349456Z Python 3.8.12 (default, Dec 3 2021, 02:23:43)
2021-12-16T17:06:16.453366746Z [GCC 8.3.0] on linux
2021-12-16T17:06:16.453372457Z Type "help", "copyright", "credits" or "license" for more information.
2021-12-16T17:06:16.457725210Z # /usr/local/lib/python3.8/pycache/site.cpython-38.pyc matches /usr/local/lib/python3.8/site.py
2021-12-16T17:06:16.457750334Z # code object from '/usr/local/lib/python3.8/pycache/site.cpython-38.pyc'
2021-12-16T17:06:16.457757928Z # /usr/local/lib/python3.8/pycache/os.cpython-38.pyc matches /usr/local/lib/python3.8/os.py
2021-12-16T17:06:16.457765198Z # code object from '/usr/local/lib/python3.8/pycache/os.cpython-38.pyc'
2021-12-16T17:06:16.457773148Z # /usr/local/lib/python3.8/pycache/stat.cpython-38.pyc matches /usr/local/lib/python3.8/stat.py
2021-12-16T17:06:16.457780450Z # code object from '/usr/local/lib/python3.8/pycache/stat.cpython-38.pyc'
2021-12-16T17:06:16.457787600Z import '_stat' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.457795758Z import 'stat' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa49112f40>
2021-12-16T17:06:16.457803158Z # /usr/local/lib/python3.8/pycache/_collections_abc.cpython-38.pyc matches /usr/local/lib/python3.8/_collections_abc.py
2021-12-16T17:06:16.457810608Z # code object from '/usr/local/lib/python3.8/pycache/_collections_abc.cpython-38.pyc'
2021-12-16T17:06:16.457817800Z import '_collections_abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa49112e50>
2021-12-16T17:06:16.457843360Z # /usr/local/lib/python3.8/pycache/posixpath.cpython-38.pyc matches /usr/local/lib/python3.8/posixpath.py
2021-12-16T17:06:16.457851334Z # code object from '/usr/local/lib/python3.8/pycache/posixpath.cpython-38.pyc'
2021-12-16T17:06:16.457858300Z # /usr/local/lib/python3.8/pycache/genericpath.cpython-38.pyc matches /usr/local/lib/python3.8/genericpath.py
2021-12-16T17:06:16.457865589Z # code object from '/usr/local/lib/python3.8/pycache/genericpath.cpython-38.pyc'
2021-12-16T17:06:16.457872701Z import 'genericpath' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa49144910>
2021-12-16T17:06:16.457880009Z import 'posixpath' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4911c910>
2021-12-16T17:06:16.457887225Z import 'os' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa491846d0>
2021-12-16T17:06:16.457894411Z # /usr/local/lib/python3.8/pycache/_sitebuiltins.cpython-38.pyc matches /usr/local/lib/python3.8/_sitebuiltins.py
2021-12-16T17:06:16.457901679Z # code object from '/usr/local/lib/python3.8/pycache/_sitebuiltins.cpython-38.pyc'
2021-12-16T17:06:16.457908589Z import '_sitebuiltins' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa49184f70>
2021-12-16T17:06:16.457917315Z # /usr/local/lib/python3.8/pycache/_bootlocale.cpython-38.pyc matches /usr/local/lib/python3.8/_bootlocale.py
2021-12-16T17:06:16.457924939Z # code object from '/usr/local/lib/python3.8/pycache/_bootlocale.cpython-38.pyc'
2021-12-16T17:06:16.457932109Z import '_locale' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.457939312Z import '_bootlocale' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa49144d60>
2021-12-16T17:06:16.457946635Z # /usr/local/lib/python3.8/pycache/types.cpython-38.pyc matches /usr/local/lib/python3.8/types.py
2021-12-16T17:06:16.457953679Z # code object from '/usr/local/lib/python3.8/pycache/types.cpython-38.pyc'
2021-12-16T17:06:16.457960742Z import 'types' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa49144f10>
2021-12-16T17:06:16.457967982Z # /usr/local/lib/python3.8/importlib/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/importlib/init.py
2021-12-16T17:06:16.457975139Z # code object from '/usr/local/lib/python3.8/importlib/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.457992115Z # /usr/local/lib/python3.8/pycache/warnings.cpython-38.pyc matches /usr/local/lib/python3.8/warnings.py
2021-12-16T17:06:16.457999659Z # code object from '/usr/local/lib/python3.8/pycache/warnings.cpython-38.pyc'
2021-12-16T17:06:16.458006622Z import 'warnings' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa490c9be0>
2021-12-16T17:06:16.458013769Z import 'importlib' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa490c9850>
2021-12-16T17:06:16.458020995Z # /usr/local/lib/python3.8/importlib/pycache/util.cpython-38.pyc matches /usr/local/lib/python3.8/importlib/util.py
2021-12-16T17:06:16.458036045Z # code object from '/usr/local/lib/python3.8/importlib/pycache/util.cpython-38.pyc'
2021-12-16T17:06:16.458043325Z # /usr/local/lib/python3.8/importlib/pycache/abc.cpython-38.pyc matches /usr/local/lib/python3.8/importlib/abc.py
2021-12-16T17:06:16.458050572Z # code object from '/usr/local/lib/python3.8/importlib/pycache/abc.cpython-38.pyc'
2021-12-16T17:06:16.458057795Z # /usr/local/lib/python3.8/importlib/pycache/machinery.cpython-38.pyc matches /usr/local/lib/python3.8/importlib/machinery.py
2021-12-16T17:06:16.458065102Z # code object from '/usr/local/lib/python3.8/importlib/pycache/machinery.cpython-38.pyc'
2021-12-16T17:06:16.458072202Z import 'importlib.machinery' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa490dc460>
2021-12-16T17:06:16.458079535Z import 'importlib.abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa490ce760>
2021-12-16T17:06:16.458087512Z # /usr/local/lib/python3.8/pycache/contextlib.cpython-38.pyc matches /usr/local/lib/python3.8/contextlib.py
2021-12-16T17:06:16.458094765Z # code object from '/usr/local/lib/python3.8/pycache/contextlib.cpython-38.pyc'
2021-12-16T17:06:16.458101839Z # /usr/local/lib/python3.8/collections/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/collections/init.py
2021-12-16T17:06:16.458109589Z # code object from '/usr/local/lib/python3.8/collections/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.458116776Z # /usr/local/lib/python3.8/pycache/operator.cpython-38.pyc matches /usr/local/lib/python3.8/operator.py
2021-12-16T17:06:16.458124033Z # code object from '/usr/local/lib/python3.8/pycache/operator.cpython-38.pyc'
2021-12-16T17:06:16.458131116Z import '_operator' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.458138343Z import 'operator' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa49085970>
2021-12-16T17:06:16.458145653Z # /usr/local/lib/python3.8/pycache/keyword.cpython-38.pyc matches /usr/local/lib/python3.8/keyword.py
2021-12-16T17:06:16.458152700Z # code object from '/usr/local/lib/python3.8/pycache/keyword.cpython-38.pyc'
2021-12-16T17:06:16.458159706Z import 'keyword' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa49085a60>
2021-12-16T17:06:16.458167027Z # /usr/local/lib/python3.8/pycache/heapq.cpython-38.pyc matches /usr/local/lib/python3.8/heapq.py
2021-12-16T17:06:16.458174107Z # code object from '/usr/local/lib/python3.8/pycache/heapq.cpython-38.pyc'
2021-12-16T17:06:16.458181260Z # extension module '_heapq' loaded from '/usr/local/lib/python3.8/lib-dynload/_heapq.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.458188500Z # extension module '_heapq' executed from '/usr/local/lib/python3.8/lib-dynload/_heapq.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.458195680Z import '_heapq' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa4908e3d0>
2021-12-16T17:06:16.458202947Z import 'heapq' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa49085be0>
2021-12-16T17:06:16.458216473Z import 'itertools' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.458223747Z # /usr/local/lib/python3.8/pycache/reprlib.cpython-38.pyc matches /usr/local/lib/python3.8/reprlib.py
2021-12-16T17:06:16.458230817Z # code object from '/usr/local/lib/python3.8/pycache/reprlib.cpython-38.pyc'
2021-12-16T17:06:16.458237787Z import 'reprlib' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa49085f70>
2021-12-16T17:06:16.458244930Z import '_collections' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.458253567Z import 'collections' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa490e6b50>
2021-12-16T17:06:16.458261013Z # /usr/local/lib/python3.8/pycache/functools.cpython-38.pyc matches /usr/local/lib/python3.8/functools.py
2021-12-16T17:06:16.458268133Z # code object from '/usr/local/lib/python3.8/pycache/functools.cpython-38.pyc'
2021-12-16T17:06:16.458275057Z import '_functools' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.458282213Z import 'functools' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa490e6d00>
2021-12-16T17:06:16.458291257Z import 'contextlib' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa490cee20>
2021-12-16T17:06:16.458298547Z import 'importlib.util' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa490c9a30>
2021-12-16T17:06:16.458306000Z # possible namespace for /usr/local/lib/python3.8/site-packages/zope
2021-12-16T17:06:16.458313310Z # possible namespace for /usr/local/lib/python3.8/site-packages/zope
2021-12-16T17:06:16.458320460Z # destroy zope
2021-12-16T17:06:16.458327423Z import 'site' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4916bf70>
2021-12-16T17:06:16.458334957Z # /usr/local/lib/python3.8/site-packages/redis/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/init.py
2021-12-16T17:06:16.458342333Z # code object from '/usr/local/lib/python3.8/site-packages/redis/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.458349473Z # /usr/local/lib/python3.8/site-packages/redis/pycache/client.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/client.py
2021-12-16T17:06:16.458356817Z # code object from '/usr/local/lib/python3.8/site-packages/redis/pycache/client.cpython-38.pyc'
2021-12-16T17:06:16.458364033Z # /usr/local/lib/python3.8/pycache/copy.cpython-38.pyc matches /usr/local/lib/python3.8/copy.py
2021-12-16T17:06:16.458371084Z # code object from '/usr/local/lib/python3.8/pycache/copy.cpython-38.pyc'
2021-12-16T17:06:16.458378068Z # /usr/local/lib/python3.8/pycache/weakref.cpython-38.pyc matches /usr/local/lib/python3.8/weakref.py
2021-12-16T17:06:16.458385378Z # code object from '/usr/local/lib/python3.8/pycache/weakref.cpython-38.pyc'
2021-12-16T17:06:16.458392698Z # /usr/local/lib/python3.8/pycache/_weakrefset.cpython-38.pyc matches /usr/local/lib/python3.8/_weakrefset.py
2021-12-16T17:06:16.458406451Z # code object from '/usr/local/lib/python3.8/pycache/_weakrefset.cpython-38.pyc'
2021-12-16T17:06:16.458413941Z import '_weakrefset' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48fec7c0>
2021-12-16T17:06:16.458421264Z import 'weakref' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4905f640>
2021-12-16T17:06:16.474487137Z # /usr/local/lib/python3.8/pycache/copyreg.cpython-38.pyc matches /usr/local/lib/python3.8/copyreg.py
2021-12-16T17:06:16.474522044Z # code object from '/usr/local/lib/python3.8/pycache/copyreg.cpython-38.pyc'
2021-12-16T17:06:16.474531337Z import 'copyreg' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48ff5520>
2021-12-16T17:06:16.474539410Z import 'copy' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4905f190>
2021-12-16T17:06:16.474547267Z # /usr/local/lib/python3.8/pycache/datetime.cpython-38.pyc matches /usr/local/lib/python3.8/datetime.py
2021-12-16T17:06:16.474554757Z # code object from '/usr/local/lib/python3.8/pycache/datetime.cpython-38.pyc'
2021-12-16T17:06:16.474561994Z # extension module 'math' loaded from '/usr/local/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.474569307Z # extension module 'math' executed from '/usr/local/lib/python3.8/lib-dynload/math.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.474576554Z import 'math' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa4901b460>
2021-12-16T17:06:16.474583874Z # extension module '_datetime' loaded from '/usr/local/lib/python3.8/lib-dynload/_datetime.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.474591094Z # extension module '_datetime' executed from '/usr/local/lib/python3.8/lib-dynload/_datetime.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.474598388Z import '_datetime' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa4901bac0>
2021-12-16T17:06:16.474605784Z import 'datetime' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4905f520>
2021-12-16T17:06:16.474613058Z # /usr/local/lib/python3.8/pycache/hashlib.cpython-38.pyc matches /usr/local/lib/python3.8/hashlib.py
2021-12-16T17:06:16.474620320Z # code object from '/usr/local/lib/python3.8/pycache/hashlib.cpython-38.pyc'
2021-12-16T17:06:16.474627568Z # extension module '_hashlib' loaded from '/usr/local/lib/python3.8/lib-dynload/_hashlib.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.474634950Z # extension module '_hashlib' executed from '/usr/local/lib/python3.8/lib-dynload/_hashlib.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.474642394Z import '_hashlib' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa4901be50>
2021-12-16T17:06:16.474649729Z # extension module '_blake2' loaded from '/usr/local/lib/python3.8/lib-dynload/_blake2.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.474715245Z # extension module '_blake2' executed from '/usr/local/lib/python3.8/lib-dynload/_blake2.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.474746095Z import '_blake2' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa4901bf10>
2021-12-16T17:06:16.474754225Z # extension module '_sha3' loaded from '/usr/local/lib/python3.8/lib-dynload/_sha3.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.474761475Z # extension module '_sha3' executed from '/usr/local/lib/python3.8/lib-dynload/_sha3.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.474768891Z import '_sha3' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48f9e040>
2021-12-16T17:06:16.474776215Z import 'hashlib' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48ff57c0>
2021-12-16T17:06:16.474783525Z # /usr/local/lib/python3.8/pycache/re.cpython-38.pyc matches /usr/local/lib/python3.8/re.py
2021-12-16T17:06:16.474790672Z # code object from '/usr/local/lib/python3.8/pycache/re.cpython-38.pyc'
2021-12-16T17:06:16.474797739Z # /usr/local/lib/python3.8/pycache/enum.cpython-38.pyc matches /usr/local/lib/python3.8/enum.py
2021-12-16T17:06:16.474804849Z # code object from '/usr/local/lib/python3.8/pycache/enum.cpython-38.pyc'
2021-12-16T17:06:16.474811852Z import 'enum' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4900dd00>
2021-12-16T17:06:16.474819139Z # /usr/local/lib/python3.8/pycache/sre_compile.cpython-38.pyc matches /usr/local/lib/python3.8/sre_compile.py
2021-12-16T17:06:16.474826339Z # code object from '/usr/local/lib/python3.8/pycache/sre_compile.cpython-38.pyc'
2021-12-16T17:06:16.474833475Z import '_sre' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.474840819Z # /usr/local/lib/python3.8/pycache/sre_parse.cpython-38.pyc matches /usr/local/lib/python3.8/sre_parse.py
2021-12-16T17:06:16.474848055Z # code object from '/usr/local/lib/python3.8/pycache/sre_parse.cpython-38.pyc'
2021-12-16T17:06:16.474855139Z # /usr/local/lib/python3.8/pycache/sre_constants.cpython-38.pyc matches /usr/local/lib/python3.8/sre_constants.py
2021-12-16T17:06:16.474862312Z # code object from '/usr/local/lib/python3.8/pycache/sre_constants.cpython-38.pyc'
2021-12-16T17:06:16.474869385Z import 'sre_constants' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48f9e910>
2021-12-16T17:06:16.474876705Z import 'sre_parse' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa49003490>
2021-12-16T17:06:16.474883929Z import 'sre_compile' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa49003c10>
2021-12-16T17:06:16.474891985Z import 're' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48ff58e0>
2021-12-16T17:06:16.474900190Z # /usr/local/lib/python3.8/pycache/threading.cpython-38.pyc matches /usr/local/lib/python3.8/threading.py
2021-12-16T17:06:16.474907456Z # code object from '/usr/local/lib/python3.8/pycache/threading.cpython-38.pyc'
2021-12-16T17:06:16.474914480Z import 'threading' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4900dbe0>
2021-12-16T17:06:16.474928960Z # /usr/local/lib/python3.8/site-packages/redis/commands/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/commands/init.py
2021-12-16T17:06:16.474936803Z # code object from '/usr/local/lib/python3.8/site-packages/redis/commands/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.474944200Z # /usr/local/lib/python3.8/site-packages/redis/commands/pycache/core.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/commands/core.py
2021-12-16T17:06:16.474951926Z # code object from '/usr/local/lib/python3.8/site-packages/redis/commands/pycache/core.cpython-38.pyc'
2021-12-16T17:06:16.474959190Z # /usr/local/lib/python3.8/site-packages/redis/commands/pycache/helpers.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/commands/helpers.py
2021-12-16T17:06:16.474966640Z # code object from '/usr/local/lib/python3.8/site-packages/redis/commands/pycache/helpers.cpython-38.pyc'
2021-12-16T17:06:16.474973836Z # /usr/local/lib/python3.8/pycache/random.cpython-38.pyc matches /usr/local/lib/python3.8/random.py
2021-12-16T17:06:16.474981056Z # code object from '/usr/local/lib/python3.8/pycache/random.cpython-38.pyc'
2021-12-16T17:06:16.474988170Z # /usr/local/lib/python3.8/pycache/bisect.cpython-38.pyc matches /usr/local/lib/python3.8/bisect.py
2021-12-16T17:06:16.474995360Z # code object from '/usr/local/lib/python3.8/pycache/bisect.cpython-38.pyc'
2021-12-16T17:06:16.475002400Z # extension module '_bisect' loaded from '/usr/local/lib/python3.8/lib-dynload/_bisect.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.475009663Z # extension module '_bisect' executed from '/usr/local/lib/python3.8/lib-dynload/_bisect.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.475016983Z import '_bisect' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48ad69d0>
2021-12-16T17:06:16.475024313Z import 'bisect' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48ad6790>
2021-12-16T17:06:16.475031550Z # extension module '_sha512' loaded from '/usr/local/lib/python3.8/lib-dynload/_sha512.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.475039997Z # extension module '_sha512' executed from '/usr/local/lib/python3.8/lib-dynload/_sha512.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.475047453Z import '_sha512' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48ad6880>
2021-12-16T17:06:16.475054913Z # extension module '_random' loaded from '/usr/local/lib/python3.8/lib-dynload/_random.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.475062297Z # extension module '_random' executed from '/usr/local/lib/python3.8/lib-dynload/_random.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.475069677Z import '_random' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48ad68b0>
2021-12-16T17:06:16.475077003Z import 'random' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48bb7be0>
2021-12-16T17:06:16.475084310Z # /usr/local/lib/python3.8/pycache/string.cpython-38.pyc matches /usr/local/lib/python3.8/string.py
2021-12-16T17:06:16.475091597Z # code object from '/usr/local/lib/python3.8/pycache/string.cpython-38.pyc'
2021-12-16T17:06:16.475104660Z import '_string' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.475112120Z import 'string' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48ad6a90>
2021-12-16T17:06:16.475119470Z import 'redis.commands.helpers' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48bb7880>
2021-12-16T17:06:16.475127030Z # /usr/local/lib/python3.8/site-packages/redis/pycache/exceptions.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/exceptions.py
2021-12-16T17:06:16.475134480Z # code object from '/usr/local/lib/python3.8/site-packages/redis/pycache/exceptions.cpython-38.pyc'
2021-12-16T17:06:16.475141740Z import 'redis.exceptions' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48bb7a90>
2021-12-16T17:06:16.475149138Z import 'redis.commands.core' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48fc3f70>
2021-12-16T17:06:16.475156604Z # /usr/local/lib/python3.8/site-packages/redis/commands/pycache/redismodules.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/commands/redismodules.py
2021-12-16T17:06:16.475164104Z # code object from '/usr/local/lib/python3.8/site-packages/redis/commands/pycache/redismodules.cpython-38.pyc'
2021-12-16T17:06:16.475171291Z # /usr/local/lib/python3.8/json/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/json/init.py
2021-12-16T17:06:16.475178678Z # code object from '/usr/local/lib/python3.8/json/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.475185844Z # /usr/local/lib/python3.8/json/pycache/decoder.cpython-38.pyc matches /usr/local/lib/python3.8/json/decoder.py
2021-12-16T17:06:16.500597599Z # code object from '/usr/local/lib/python3.8/json/pycache/decoder.cpython-38.pyc'
2021-12-16T17:06:16.500621721Z # /usr/local/lib/python3.8/json/pycache/scanner.cpython-38.pyc matches /usr/local/lib/python3.8/json/scanner.py
2021-12-16T17:06:16.500628443Z # code object from '/usr/local/lib/python3.8/json/pycache/scanner.cpython-38.pyc'
2021-12-16T17:06:16.500634212Z # extension module '_json' loaded from '/usr/local/lib/python3.8/lib-dynload/_json.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.500640070Z # extension module '_json' executed from '/usr/local/lib/python3.8/lib-dynload/_json.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.500646058Z import '_json' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48ae8fd0>
2021-12-16T17:06:16.500652585Z import 'json.scanner' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48ae8dc0>
2021-12-16T17:06:16.500681724Z import 'json.decoder' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48ae8a30>
2021-12-16T17:06:16.500688444Z # /usr/local/lib/python3.8/json/pycache/encoder.cpython-38.pyc matches /usr/local/lib/python3.8/json/encoder.py
2021-12-16T17:06:16.500694293Z # code object from '/usr/local/lib/python3.8/json/pycache/encoder.cpython-38.pyc'
2021-12-16T17:06:16.500699944Z import 'json.encoder' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48ae8e80>
2021-12-16T17:06:16.500719513Z import 'json' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48ae8790>
2021-12-16T17:06:16.500725760Z import 'redis.commands.redismodules' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48bb7a30>
2021-12-16T17:06:16.500732055Z # /usr/local/lib/python3.8/site-packages/redis/commands/pycache/sentinel.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/commands/sentinel.py
2021-12-16T17:06:16.500738153Z # code object from '/usr/local/lib/python3.8/site-packages/redis/commands/pycache/sentinel.cpython-38.pyc'
2021-12-16T17:06:16.500743899Z import 'redis.commands.sentinel' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48ae8490>
2021-12-16T17:06:16.500749711Z import 'redis.commands' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48fc3520>
2021-12-16T17:06:16.500755748Z # /usr/local/lib/python3.8/site-packages/redis/pycache/connection.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/connection.py
2021-12-16T17:06:16.500761702Z # code object from '/usr/local/lib/python3.8/site-packages/redis/pycache/connection.cpython-38.pyc'
2021-12-16T17:06:16.500768873Z # /usr/local/lib/python3.8/distutils/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/distutils/init.py
2021-12-16T17:06:16.500775037Z # code object from '/usr/local/lib/python3.8/distutils/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.500780817Z import 'distutils' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48aa05b0>
2021-12-16T17:06:16.500786591Z # /usr/local/lib/python3.8/distutils/pycache/version.cpython-38.pyc matches /usr/local/lib/python3.8/distutils/version.py
2021-12-16T17:06:16.500792520Z # code object from '/usr/local/lib/python3.8/distutils/pycache/version.cpython-38.pyc'
2021-12-16T17:06:16.500798873Z import 'distutils.version' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48aa05e0>
2021-12-16T17:06:16.500805157Z # /usr/local/lib/python3.8/pycache/queue.cpython-38.pyc matches /usr/local/lib/python3.8/queue.py
2021-12-16T17:06:16.500810917Z # code object from '/usr/local/lib/python3.8/pycache/queue.cpython-38.pyc'
2021-12-16T17:06:16.500816655Z # extension module '_queue' loaded from '/usr/local/lib/python3.8/lib-dynload/_queue.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.500822400Z # extension module '_queue' executed from '/usr/local/lib/python3.8/lib-dynload/_queue.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.500828333Z import '_queue' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48aab700>
2021-12-16T17:06:16.500834273Z import 'queue' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48aa0bb0>
2021-12-16T17:06:16.500840144Z # /usr/local/lib/python3.8/urllib/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/urllib/init.py
2021-12-16T17:06:16.500846022Z # code object from '/usr/local/lib/python3.8/urllib/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.500857005Z import 'urllib' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48aab7f0>
2021-12-16T17:06:16.500862876Z # /usr/local/lib/python3.8/urllib/pycache/parse.cpython-38.pyc matches /usr/local/lib/python3.8/urllib/parse.py
2021-12-16T17:06:16.500868668Z # code object from '/usr/local/lib/python3.8/urllib/pycache/parse.cpython-38.pyc'
2021-12-16T17:06:16.500882267Z import 'urllib.parse' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48aab8b0>
2021-12-16T17:06:16.500888357Z import 'errno' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.500894193Z # /usr/local/lib/python3.8/pycache/socket.cpython-38.pyc matches /usr/local/lib/python3.8/socket.py
2021-12-16T17:06:16.500906085Z # code object from '/usr/local/lib/python3.8/pycache/socket.cpython-38.pyc'
2021-12-16T17:06:16.500911824Z # extension module '_socket' loaded from '/usr/local/lib/python3.8/lib-dynload/_socket.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.500917559Z # extension module '_socket' executed from '/usr/local/lib/python3.8/lib-dynload/_socket.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.500923220Z import '_socket' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48a5a340>
2021-12-16T17:06:16.500928988Z # /usr/local/lib/python3.8/pycache/selectors.cpython-38.pyc matches /usr/local/lib/python3.8/selectors.py
2021-12-16T17:06:16.500934637Z # code object from '/usr/local/lib/python3.8/pycache/selectors.cpython-38.pyc'
2021-12-16T17:06:16.500940321Z # /usr/local/lib/python3.8/collections/pycache/abc.cpython-38.pyc matches /usr/local/lib/python3.8/collections/abc.py
2021-12-16T17:06:16.500946034Z # code object from '/usr/local/lib/python3.8/collections/pycache/abc.cpython-38.pyc'
2021-12-16T17:06:16.500951603Z import 'collections.abc' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48a6a670>
2021-12-16T17:06:16.500957345Z # extension module 'select' loaded from '/usr/local/lib/python3.8/lib-dynload/select.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.500963225Z # extension module 'select' executed from '/usr/local/lib/python3.8/lib-dynload/select.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.500968934Z import 'select' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48a6a6a0>
2021-12-16T17:06:16.500974663Z import 'selectors' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48a5a4c0>
2021-12-16T17:06:16.500980428Z import 'socket' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48aabf70>
2021-12-16T17:06:16.500986317Z # /usr/local/lib/python3.8/site-packages/redis/pycache/utils.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/utils.py
2021-12-16T17:06:16.500992086Z # code object from '/usr/local/lib/python3.8/site-packages/redis/pycache/utils.cpython-38.pyc'
2021-12-16T17:06:16.500997854Z import 'redis.utils' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48ac4b50>
2021-12-16T17:06:16.501003663Z # /usr/local/lib/python3.8/site-packages/redis/pycache/backoff.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/backoff.py
2021-12-16T17:06:16.501014619Z # code object from '/usr/local/lib/python3.8/site-packages/redis/pycache/backoff.cpython-38.pyc'
2021-12-16T17:06:16.501020483Z import 'redis.backoff' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48a73970>
2021-12-16T17:06:16.501027077Z # /usr/local/lib/python3.8/site-packages/redis/pycache/retry.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/retry.py
2021-12-16T17:06:16.501033059Z # code object from '/usr/local/lib/python3.8/site-packages/redis/pycache/retry.cpython-38.pyc'
2021-12-16T17:06:16.501038700Z import 'redis.retry' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48a73be0>
2021-12-16T17:06:16.501044632Z # /usr/local/lib/python3.8/pycache/ssl.cpython-38.pyc matches /usr/local/lib/python3.8/ssl.py
2021-12-16T17:06:16.501050237Z # code object from '/usr/local/lib/python3.8/pycache/ssl.cpython-38.pyc'
2021-12-16T17:06:16.501055734Z # extension module '_ssl' loaded from '/usr/local/lib/python3.8/lib-dynload/_ssl.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.501061334Z # extension module '_ssl' executed from '/usr/local/lib/python3.8/lib-dynload/_ssl.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.501066992Z import '_ssl' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa489fb3d0>
2021-12-16T17:06:16.501072686Z # /usr/local/lib/python3.8/pycache/base64.cpython-38.pyc matches /usr/local/lib/python3.8/base64.py
2021-12-16T17:06:16.501078308Z # code object from '/usr/local/lib/python3.8/pycache/base64.cpython-38.pyc'
2021-12-16T17:06:16.501083859Z # /usr/local/lib/python3.8/pycache/struct.cpython-38.pyc matches /usr/local/lib/python3.8/struct.py
2021-12-16T17:06:16.501089641Z # code object from '/usr/local/lib/python3.8/pycache/struct.cpython-38.pyc'
2021-12-16T17:06:16.501095185Z # extension module '_struct' loaded from '/usr/local/lib/python3.8/lib-dynload/_struct.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.501100957Z # extension module '_struct' executed from '/usr/local/lib/python3.8/lib-dynload/_struct.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.501106674Z import '_struct' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa4899b1f0>
2021-12-16T17:06:16.501112366Z import 'struct' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4898ffd0>
2021-12-16T17:06:16.501118077Z # extension module 'binascii' loaded from '/usr/local/lib/python3.8/lib-dynload/binascii.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.501123950Z # extension module 'binascii' executed from '/usr/local/lib/python3.8/lib-dynload/binascii.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.501129694Z import 'binascii' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa4899b0d0>
2021-12-16T17:06:16.501135456Z import 'base64' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4898f1f0>
2021-12-16T17:06:16.523786664Z import 'ssl' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48a7b280>
2021-12-16T17:06:16.523860738Z import 'redis.connection' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48fb6670>
2021-12-16T17:06:16.523872942Z # /usr/local/lib/python3.8/site-packages/redis/pycache/lock.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/lock.py
2021-12-16T17:06:16.523882088Z # code object from '/usr/local/lib/python3.8/site-packages/redis/pycache/lock.cpython-38.pyc'
2021-12-16T17:06:16.523890798Z # /usr/local/lib/python3.8/pycache/uuid.cpython-38.pyc matches /usr/local/lib/python3.8/uuid.py
2021-12-16T17:06:16.523898502Z # code object from '/usr/local/lib/python3.8/pycache/uuid.cpython-38.pyc'
2021-12-16T17:06:16.523907228Z # /usr/local/lib/python3.8/pycache/platform.cpython-38.pyc matches /usr/local/lib/python3.8/platform.py
2021-12-16T17:06:16.523916892Z # code object from '/usr/local/lib/python3.8/pycache/platform.cpython-38.pyc'
2021-12-16T17:06:16.523925762Z import 'platform' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa489af670>
2021-12-16T17:06:16.523933762Z # /usr/local/lib/python3.8/pycache/subprocess.cpython-38.pyc matches /usr/local/lib/python3.8/subprocess.py
2021-12-16T17:06:16.523941526Z # code object from '/usr/local/lib/python3.8/pycache/subprocess.cpython-38.pyc'
2021-12-16T17:06:16.523949293Z # /usr/local/lib/python3.8/pycache/signal.cpython-38.pyc matches /usr/local/lib/python3.8/signal.py
2021-12-16T17:06:16.523957043Z # code object from '/usr/local/lib/python3.8/pycache/signal.cpython-38.pyc'
2021-12-16T17:06:16.523964826Z import 'signal' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa487136d0>
2021-12-16T17:06:16.523976693Z # extension module '_posixsubprocess' loaded from '/usr/local/lib/python3.8/lib-dynload/_posixsubprocess.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.523984636Z # extension module '_posixsubprocess' executed from '/usr/local/lib/python3.8/lib-dynload/_posixsubprocess.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.523992586Z import '_posixsubprocess' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48713ac0>
2021-12-16T17:06:16.524000693Z import 'subprocess' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa489bd4f0>
2021-12-16T17:06:16.524008563Z # extension module '_uuid' loaded from '/usr/local/lib/python3.8/lib-dynload/_uuid.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.524019326Z # extension module '_uuid' executed from '/usr/local/lib/python3.8/lib-dynload/_uuid.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.524027946Z import '_uuid' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa489bd250>
2021-12-16T17:06:16.524036156Z import 'uuid' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa489a53d0>
2021-12-16T17:06:16.524044023Z import 'redis.lock' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48a7b5e0>
2021-12-16T17:06:16.524051966Z import 'redis.client' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa490e6d90>
2021-12-16T17:06:16.524067433Z # /usr/local/lib/python3.8/site-packages/redis/pycache/sentinel.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/redis/sentinel.py
2021-12-16T17:06:16.524075766Z # code object from '/usr/local/lib/python3.8/site-packages/redis/pycache/sentinel.cpython-38.pyc'
2021-12-16T17:06:16.524083506Z import 'redis.sentinel' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4905f070>
2021-12-16T17:06:16.524091389Z import 'redis' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa490dce50>
2021-12-16T17:06:16.524099326Z # /usr/local/lib/python3.8/site-packages/rq/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/init.py
2021-12-16T17:06:16.524107776Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.524115456Z # /usr/local/lib/python3.8/pycache/future.cpython-38.pyc matches /usr/local/lib/python3.8/future.py
2021-12-16T17:06:16.524123439Z # code object from '/usr/local/lib/python3.8/pycache/future.cpython-38.pyc'
2021-12-16T17:06:16.524131089Z import 'future' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48720700>
2021-12-16T17:06:16.524139006Z # /usr/local/lib/python3.8/site-packages/rq/pycache/connections.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/connections.py
2021-12-16T17:06:16.524165446Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/connections.cpython-38.pyc'
2021-12-16T17:06:16.524173456Z # /usr/local/lib/python3.8/site-packages/rq/pycache/local.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/local.py
2021-12-16T17:06:16.524181350Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/local.cpython-38.pyc'
2021-12-16T17:06:16.524189326Z # /usr/local/lib/python3.8/site-packages/greenlet/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/greenlet/init.py
2021-12-16T17:06:16.524198311Z # code object from '/usr/local/lib/python3.8/site-packages/greenlet/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.524206481Z # extension module 'greenlet._greenlet' loaded from '/usr/local/lib/python3.8/site-packages/greenlet/_greenlet.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.524214374Z # extension module 'greenlet._greenlet' executed from '/usr/local/lib/python3.8/site-packages/greenlet/_greenlet.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.524222414Z import 'greenlet._greenlet' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa4873dd00>
2021-12-16T17:06:16.524230394Z import 'greenlet' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4873d790>
2021-12-16T17:06:16.524238397Z import 'rq.local' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48736400>
2021-12-16T17:06:16.524246241Z import 'rq.connections' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48720ee0>
2021-12-16T17:06:16.524255664Z # /usr/local/lib/python3.8/site-packages/rq/pycache/job.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/job.py
2021-12-16T17:06:16.524270811Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/job.cpython-38.pyc'
2021-12-16T17:06:16.524278661Z # /usr/local/lib/python3.8/pycache/inspect.cpython-38.pyc matches /usr/local/lib/python3.8/inspect.py
2021-12-16T17:06:16.524286557Z # code object from '/usr/local/lib/python3.8/pycache/inspect.cpython-38.pyc'
2021-12-16T17:06:16.524294297Z # /usr/local/lib/python3.8/pycache/dis.cpython-38.pyc matches /usr/local/lib/python3.8/dis.py
2021-12-16T17:06:16.524301994Z # code object from '/usr/local/lib/python3.8/pycache/dis.cpython-38.pyc'
2021-12-16T17:06:16.524309601Z # /usr/local/lib/python3.8/pycache/opcode.cpython-38.pyc matches /usr/local/lib/python3.8/opcode.py
2021-12-16T17:06:16.524317494Z # code object from '/usr/local/lib/python3.8/pycache/opcode.cpython-38.pyc'
2021-12-16T17:06:16.524325114Z # extension module '_opcode' loaded from '/usr/local/lib/python3.8/lib-dynload/_opcode.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.524332914Z # extension module '_opcode' executed from '/usr/local/lib/python3.8/lib-dynload/_opcode.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.524341461Z import '_opcode' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa486ed760>
2021-12-16T17:06:16.524349411Z import 'opcode' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa486ed430>
2021-12-16T17:06:16.524357374Z import 'dis' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa486decd0>
2021-12-16T17:06:16.524366404Z # /usr/local/lib/python3.8/pycache/linecache.cpython-38.pyc matches /usr/local/lib/python3.8/linecache.py
2021-12-16T17:06:16.524374577Z # code object from '/usr/local/lib/python3.8/pycache/linecache.cpython-38.pyc'
2021-12-16T17:06:16.524382524Z # /usr/local/lib/python3.8/pycache/tokenize.cpython-38.pyc matches /usr/local/lib/python3.8/tokenize.py
2021-12-16T17:06:16.524390277Z # code object from '/usr/local/lib/python3.8/pycache/tokenize.cpython-38.pyc'
2021-12-16T17:06:16.524397997Z # /usr/local/lib/python3.8/pycache/token.cpython-38.pyc matches /usr/local/lib/python3.8/token.py
2021-12-16T17:06:16.524405881Z # code object from '/usr/local/lib/python3.8/pycache/token.cpython-38.pyc'
2021-12-16T17:06:16.524413517Z import 'token' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa486795e0>
2021-12-16T17:06:16.524421371Z import 'tokenize' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa486edc70>
2021-12-16T17:06:16.524429564Z import 'linecache' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa486ed910>
2021-12-16T17:06:16.524437391Z import 'inspect' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48748c10>
2021-12-16T17:06:16.524445267Z # extension module 'zlib' loaded from '/usr/local/lib/python3.8/lib-dynload/zlib.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.524453136Z # extension module 'zlib' executed from '/usr/local/lib/python3.8/lib-dynload/zlib.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.524460962Z import 'zlib' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa486c1c70>
2021-12-16T17:06:16.524475146Z # /usr/local/lib/python3.8/site-packages/rq/compat/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/compat/init.py
2021-12-16T17:06:16.524483842Z # code object from '/usr/local/lib/python3.8/site-packages/rq/compat/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.524491576Z import 'rq.compat' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48691b80>
2021-12-16T17:06:16.524500036Z # /usr/local/lib/python3.8/site-packages/rq/pycache/exceptions.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/exceptions.py
2021-12-16T17:06:16.524507978Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/exceptions.cpython-38.pyc'
2021-12-16T17:06:16.524515806Z import 'rq.exceptions' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48691c40>
2021-12-16T17:06:16.524523846Z # /usr/local/lib/python3.8/site-packages/rq/pycache/utils.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/utils.py
2021-12-16T17:06:16.544684666Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/utils.cpython-38.pyc'
2021-12-16T17:06:16.544722768Z # /usr/local/lib/python3.8/pycache/calendar.cpython-38.pyc matches /usr/local/lib/python3.8/calendar.py
2021-12-16T17:06:16.544732864Z # code object from '/usr/local/lib/python3.8/pycache/calendar.cpython-38.pyc'
2021-12-16T17:06:16.544739418Z # /usr/local/lib/python3.8/pycache/locale.cpython-38.pyc matches /usr/local/lib/python3.8/locale.py
2021-12-16T17:06:16.544745747Z # code object from '/usr/local/lib/python3.8/pycache/locale.cpython-38.pyc'
2021-12-16T17:06:16.544752178Z import 'locale' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa486ab580>
2021-12-16T17:06:16.544759918Z import 'calendar' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4869a9d0>
2021-12-16T17:06:16.544766769Z # /usr/local/lib/python3.8/logging/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/logging/init.py
2021-12-16T17:06:16.544773698Z # code object from '/usr/local/lib/python3.8/logging/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.544780062Z # /usr/local/lib/python3.8/pycache/traceback.cpython-38.pyc matches /usr/local/lib/python3.8/traceback.py
2021-12-16T17:06:16.544806375Z # code object from '/usr/local/lib/python3.8/pycache/traceback.cpython-38.pyc'
2021-12-16T17:06:16.544814121Z import 'traceback' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48664700>
2021-12-16T17:06:16.544820849Z import 'atexit' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.544827561Z import 'logging' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4869ae80>
2021-12-16T17:06:16.544834076Z # /usr/local/lib/python3.8/pycache/numbers.cpython-38.pyc matches /usr/local/lib/python3.8/numbers.py
2021-12-16T17:06:16.544840513Z # code object from '/usr/local/lib/python3.8/pycache/numbers.cpython-38.pyc'
2021-12-16T17:06:16.544878467Z import 'numbers' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4864c430>
2021-12-16T17:06:16.544886464Z import 'rq.utils' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48691eb0>
2021-12-16T17:06:16.544892858Z # /usr/local/lib/python3.8/pycache/pickle.cpython-38.pyc matches /usr/local/lib/python3.8/pickle.py
2021-12-16T17:06:16.544899295Z # code object from '/usr/local/lib/python3.8/pycache/pickle.cpython-38.pyc'
2021-12-16T17:06:16.544905369Z # /usr/local/lib/python3.8/pycache/_compat_pickle.cpython-38.pyc matches /usr/local/lib/python3.8/_compat_pickle.py
2021-12-16T17:06:16.545056837Z # code object from '/usr/local/lib/python3.8/pycache/_compat_pickle.cpython-38.pyc'
2021-12-16T17:06:16.545063519Z import '_compat_pickle' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48621d00>
2021-12-16T17:06:16.545070188Z # extension module '_pickle' loaded from '/usr/local/lib/python3.8/lib-dynload/_pickle.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.545076579Z # extension module '_pickle' executed from '/usr/local/lib/python3.8/lib-dynload/_pickle.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.545082953Z import '_pickle' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48621e50>
2021-12-16T17:06:16.545106457Z import 'pickle' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4860e280>
2021-12-16T17:06:16.545113464Z import 'rq.job' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4873d670>
2021-12-16T17:06:16.545119793Z # /usr/local/lib/python3.8/site-packages/rq/pycache/queue.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/queue.py
2021-12-16T17:06:16.545126461Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/queue.cpython-38.pyc'
2021-12-16T17:06:16.545132866Z # /usr/local/lib/python3.8/site-packages/rq/pycache/defaults.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/defaults.py
2021-12-16T17:06:16.545139304Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/defaults.cpython-38.pyc'
2021-12-16T17:06:16.545145959Z import 'rq.defaults' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485b5fd0>
2021-12-16T17:06:16.545168652Z import 'rq.queue' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48621be0>
2021-12-16T17:06:16.545302387Z # /usr/local/lib/python3.8/site-packages/rq/pycache/version.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/version.py
2021-12-16T17:06:16.545311091Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/version.cpython-38.pyc'
2021-12-16T17:06:16.545317563Z import 'rq.version' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485b5ee0>
2021-12-16T17:06:16.545324225Z # /usr/local/lib/python3.8/site-packages/rq/pycache/worker.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/worker.py
2021-12-16T17:06:16.545330653Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/worker.cpython-38.pyc'
2021-12-16T17:06:16.545362896Z # /usr/local/lib/python3.8/site-packages/rq/pycache/worker_registration.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/worker_registration.py
2021-12-16T17:06:16.545371672Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/worker_registration.cpython-38.pyc'
2021-12-16T17:06:16.545378359Z import 'rq.worker_registration' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485d8910>
2021-12-16T17:06:16.545385027Z # /usr/local/lib/python3.8/site-packages/rq/pycache/logutils.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/logutils.py
2021-12-16T17:06:16.545391341Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/logutils.cpython-38.pyc'
2021-12-16T17:06:16.545414790Z import 'rq.logutils' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485d8a90>
2021-12-16T17:06:16.545421954Z # /usr/local/lib/python3.8/site-packages/rq/pycache/registry.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/registry.py
2021-12-16T17:06:16.545428321Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/registry.cpython-38.pyc'
2021-12-16T17:06:16.545434613Z import 'rq.registry' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485d8b80>
2021-12-16T17:06:16.545441401Z # /usr/local/lib/python3.8/site-packages/rq/pycache/suspension.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/suspension.py
2021-12-16T17:06:16.545447692Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/suspension.cpython-38.pyc'
2021-12-16T17:06:16.545453952Z import 'rq.suspension' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485e22b0>
2021-12-16T17:06:16.545556706Z # /usr/local/lib/python3.8/site-packages/rq/pycache/timeouts.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/rq/timeouts.py
2021-12-16T17:06:16.545565593Z # code object from '/usr/local/lib/python3.8/site-packages/rq/pycache/timeouts.cpython-38.pyc'
2021-12-16T17:06:16.545571953Z import 'rq.timeouts' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485e24c0>
2021-12-16T17:06:16.545578582Z import 'rq.worker' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485c9130>
2021-12-16T17:06:16.545584895Z import 'rq' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa490e6e80>
2021-12-16T17:06:16.545591393Z # /usr/src/app/pycache/tasks.cpython-38.pyc matches /usr/src/app/tasks.py
2021-12-16T17:06:16.545597504Z # code object from '/usr/src/app/pycache/tasks.cpython-38.pyc'
2021-12-16T17:06:16.545603708Z # /usr/local/lib/python3.8/site-packages/pssh/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/init.py
2021-12-16T17:06:16.545611622Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.545618293Z # /usr/local/lib/python3.8/site-packages/pssh/pycache/_version.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/_version.py
2021-12-16T17:06:16.545641965Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/pycache/_version.cpython-38.pyc'
2021-12-16T17:06:16.545649228Z import 'pssh._version' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485e2e80>
2021-12-16T17:06:16.545655704Z import 'pssh' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485e2c10>
2021-12-16T17:06:16.545691018Z # /usr/local/lib/python3.8/site-packages/pssh/clients/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/clients/init.py
2021-12-16T17:06:16.545698264Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/clients/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.545705118Z # /usr/local/lib/python3.8/site-packages/pssh/clients/native/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/clients/native/init.py
2021-12-16T17:06:16.545711775Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/clients/native/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.545718191Z # /usr/local/lib/python3.8/site-packages/pssh/clients/native/pycache/parallel.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/clients/native/parallel.py
2021-12-16T17:06:16.545724668Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/clients/native/pycache/parallel.cpython-38.pyc'
2021-12-16T17:06:16.545731180Z # /usr/local/lib/python3.8/site-packages/pssh/clients/native/pycache/single.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/clients/native/single.py
2021-12-16T17:06:16.545738063Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/clients/native/pycache/single.cpython-38.pyc'
2021-12-16T17:06:16.545744561Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/init.py
2021-12-16T17:06:16.545751552Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.545757934Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_config.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_config.py
2021-12-16T17:06:16.563811953Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_config.cpython-38.pyc'
2021-12-16T17:06:16.563842803Z # /usr/local/lib/python3.8/pycache/textwrap.cpython-38.pyc matches /usr/local/lib/python3.8/textwrap.py
2021-12-16T17:06:16.563852469Z # code object from '/usr/local/lib/python3.8/pycache/textwrap.cpython-38.pyc'
2021-12-16T17:06:16.563860549Z import 'textwrap' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48575100>
2021-12-16T17:06:16.563869279Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_compat.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_compat.py
2021-12-16T17:06:16.563877593Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_compat.cpython-38.pyc'
2021-12-16T17:06:16.563885696Z import 'gevent._compat' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48575730>
2021-12-16T17:06:16.563894586Z # /usr/local/lib/python3.8/site-packages/gevent/libev/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/libev/init.py
2021-12-16T17:06:16.563916599Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/libev/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.563924936Z import 'gevent.libev' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48586dc0>
2021-12-16T17:06:16.563933456Z # extension module 'gevent.libev.corecext' loaded from '/usr/local/lib/python3.8/site-packages/gevent/libev/corecext.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.563942173Z import 'gc' # <class '_frozen_importlib.BuiltinImporter'>
2021-12-16T17:06:16.563950389Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/exceptions.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/exceptions.py
2021-12-16T17:06:16.563961036Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/exceptions.cpython-38.pyc'
2021-12-16T17:06:16.563969309Z import 'gevent.exceptions' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48514820>
2021-12-16T17:06:16.563977309Z # possible namespace for /usr/local/lib/python3.8/site-packages/zope
2021-12-16T17:06:16.563985126Z # /usr/local/lib/python3.8/site-packages/zope/interface/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/zope/interface/init.py
2021-12-16T17:06:16.563993256Z # code object from '/usr/local/lib/python3.8/site-packages/zope/interface/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.564002664Z # /usr/local/lib/python3.8/site-packages/zope/interface/pycache/interface.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/zope/interface/interface.py
2021-12-16T17:06:16.564010954Z # code object from '/usr/local/lib/python3.8/site-packages/zope/interface/pycache/interface.cpython-38.pyc'
2021-12-16T17:06:16.564018940Z # /usr/local/lib/python3.8/site-packages/zope/interface/pycache/_compat.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/zope/interface/_compat.py
2021-12-16T17:06:16.564027140Z # code object from '/usr/local/lib/python3.8/site-packages/zope/interface/pycache/_compat.cpython-38.pyc'
2021-12-16T17:06:16.564035540Z import 'zope.interface._compat' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485373d0>
2021-12-16T17:06:16.564044057Z # /usr/local/lib/python3.8/site-packages/zope/interface/pycache/exceptions.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/zope/interface/exceptions.py
2021-12-16T17:06:16.564052290Z # code object from '/usr/local/lib/python3.8/site-packages/zope/interface/pycache/exceptions.cpython-38.pyc'
2021-12-16T17:06:16.564060448Z import 'zope.interface.exceptions' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48537760>
2021-12-16T17:06:16.564069174Z # /usr/local/lib/python3.8/site-packages/zope/interface/pycache/ro.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/zope/interface/ro.py
2021-12-16T17:06:16.564077184Z # code object from '/usr/local/lib/python3.8/site-packages/zope/interface/pycache/ro.cpython-38.pyc'
2021-12-16T17:06:16.564085008Z import 'zope.interface.ro' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48537fd0>
2021-12-16T17:06:16.564099904Z # extension module 'zope.interface._zope_interface_coptimizations' loaded from '/usr/local/lib/python3.8/site-packages/zope/interface/_zope_interface_coptimizations.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.564108578Z # extension module 'zope.interface._zope_interface_coptimizations' executed from '/usr/local/lib/python3.8/site-packages/zope/interface/_zope_interface_coptimizations.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.564117094Z import 'zope.interface._zope_interface_coptimizations' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa4853ea90>
2021-12-16T17:06:16.564126844Z # /usr/local/lib/python3.8/site-packages/zope/interface/pycache/declarations.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/zope/interface/declarations.py
2021-12-16T17:06:16.564135050Z # code object from '/usr/local/lib/python3.8/site-packages/zope/interface/pycache/declarations.cpython-38.pyc'
2021-12-16T17:06:16.564162278Z # /usr/local/lib/python3.8/site-packages/zope/interface/pycache/advice.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/zope/interface/advice.py
2021-12-16T17:06:16.564171228Z # code object from '/usr/local/lib/python3.8/site-packages/zope/interface/pycache/advice.cpython-38.pyc'
2021-12-16T17:06:16.564179010Z import 'zope.interface.advice' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48b38220>
2021-12-16T17:06:16.564186920Z import 'zope.interface.declarations' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48b19760>
2021-12-16T17:06:16.564194860Z import 'zope.interface.interface' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485254f0>
2021-12-16T17:06:16.564202940Z # /usr/local/lib/python3.8/site-packages/zope/interface/pycache/interfaces.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/zope/interface/interfaces.py
2021-12-16T17:06:16.564210968Z # code object from '/usr/local/lib/python3.8/site-packages/zope/interface/pycache/interfaces.cpython-38.pyc'
2021-12-16T17:06:16.564218724Z import 'zope.interface.interfaces' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4852fc10>
2021-12-16T17:06:16.564226840Z import 'zope.interface' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48514d90>
2021-12-16T17:06:16.564235170Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_interfaces.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_interfaces.py
2021-12-16T17:06:16.564243195Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_interfaces.cpython-38.pyc'
2021-12-16T17:06:16.564250991Z import 'gevent._interfaces' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48b38670>
2021-12-16T17:06:16.564258989Z # extension module 'gevent.libev.corecext' executed from '/usr/local/lib/python3.8/site-packages/gevent/libev/corecext.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.564266952Z import 'gevent.libev.corecext' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48586eb0>
2021-12-16T17:06:16.564274969Z import 'gevent._config' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485662b0>
2021-12-16T17:06:16.564290275Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_hub_local.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_hub_local.py
2021-12-16T17:06:16.564298665Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_hub_local.cpython-38.pyc'
2021-12-16T17:06:16.564306859Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_util.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_util.py
2021-12-16T17:06:16.564315282Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_util.cpython-38.pyc'
2021-12-16T17:06:16.564323032Z import 'gevent._util' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48586af0>
2021-12-16T17:06:16.564331072Z # extension module 'gevent._gevent_c_hub_local' loaded from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_hub_local.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.564339162Z # extension module 'gevent._gevent_c_greenlet_primitives' loaded from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_greenlet_primitives.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.564347322Z # extension module 'gevent._gevent_c_greenlet_primitives' executed from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_greenlet_primitives.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.564355449Z import 'gevent._gevent_c_greenlet_primitives' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48b44a30>
2021-12-16T17:06:16.564363722Z # extension module 'gevent._gevent_c_hub_local' executed from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_hub_local.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.564371742Z import 'gevent._gevent_c_hub_local' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48b38fd0>
2021-12-16T17:06:16.564379642Z import 'gevent._hub_local' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485669a0>
2021-12-16T17:06:16.564387899Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_hub_primitives.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_hub_primitives.py
2021-12-16T17:06:16.564395919Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_hub_primitives.cpython-38.pyc'
2021-12-16T17:06:16.564403725Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_greenlet_primitives.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_greenlet_primitives.py
2021-12-16T17:06:16.575912275Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_greenlet_primitives.cpython-38.pyc'
2021-12-16T17:06:16.575936130Z import 'gevent._greenlet_primitives' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48b44f10>
2021-12-16T17:06:16.575942987Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_waiter.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_waiter.py
2021-12-16T17:06:16.575947374Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_waiter.cpython-38.pyc'
2021-12-16T17:06:16.575951534Z # extension module 'gevent._gevent_c_waiter' loaded from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_waiter.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.575965673Z # extension module 'gevent._gevent_c_waiter' executed from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_waiter.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.575970230Z import 'gevent._gevent_c_waiter' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa47fdac40>
2021-12-16T17:06:16.575974202Z import 'gevent._waiter' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48b4f3a0>
2021-12-16T17:06:16.575978134Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/timeout.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/timeout.py
2021-12-16T17:06:16.575982010Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/timeout.cpython-38.pyc'
2021-12-16T17:06:16.575986137Z import 'gevent.timeout' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48b4f700>
2021-12-16T17:06:16.575990064Z # extension module 'gevent._gevent_c_hub_primitives' loaded from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_hub_primitives.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.575993966Z # extension module 'gevent._gevent_c_hub_primitives' executed from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_hub_primitives.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.575998225Z import 'gevent._gevent_c_hub_primitives' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa47ff03a0>
2021-12-16T17:06:16.576002593Z import 'gevent._hub_primitives' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48566a00>
2021-12-16T17:06:16.576006524Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/greenlet.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/greenlet.py
2021-12-16T17:06:16.576011358Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/greenlet.cpython-38.pyc'
2021-12-16T17:06:16.576015288Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_tblib.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_tblib.py
2021-12-16T17:06:16.576019231Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_tblib.cpython-38.pyc'
2021-12-16T17:06:16.576022994Z import 'gevent._tblib' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa47ffaaf0>
2021-12-16T17:06:16.576026936Z # extension module 'gevent._gevent_cgreenlet' loaded from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_cgreenlet.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.576030759Z # extension module 'gevent._gevent_c_ident' loaded from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_ident.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.576034653Z # extension module 'gevent._gevent_c_ident' executed from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_ident.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.576038466Z import 'gevent._gevent_c_ident' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48005be0>
2021-12-16T17:06:16.576042468Z # extension module 'gevent._gevent_cgreenlet' executed from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_cgreenlet.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.576050485Z import 'gevent._gevent_cgreenlet' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa48b4f2b0>
2021-12-16T17:06:16.576054483Z import 'gevent.greenlet' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa48566d30>
2021-12-16T17:06:16.576058275Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/hub.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/hub.py
2021-12-16T17:06:16.576062040Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/hub.cpython-38.pyc'
2021-12-16T17:06:16.576065908Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_ident.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_ident.py
2021-12-16T17:06:16.576069680Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_ident.cpython-38.pyc'
2021-12-16T17:06:16.576073674Z import 'gevent._ident' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa477324c0>
2021-12-16T17:06:16.576077938Z import 'gevent.hub' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa47ffa790>
2021-12-16T17:06:16.576082037Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/os.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/os.py
2021-12-16T17:06:16.576085927Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/os.cpython-38.pyc'
2021-12-16T17:06:16.576089786Z # extension module 'fcntl' loaded from '/usr/local/lib/python3.8/lib-dynload/fcntl.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.576093487Z # extension module 'fcntl' executed from '/usr/local/lib/python3.8/lib-dynload/fcntl.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.576097554Z import 'fcntl' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa47732d30>
2021-12-16T17:06:16.576101498Z import 'gevent.os' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa47732550>
2021-12-16T17:06:16.576105285Z import 'gevent' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4855e730>
2021-12-16T17:06:16.576109106Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/lock.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/lock.py
2021-12-16T17:06:16.576113173Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/lock.cpython-38.pyc'
2021-12-16T17:06:16.576116850Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/monkey.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/monkey.py
2021-12-16T17:06:16.576120646Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/monkey.cpython-38.pyc'
2021-12-16T17:06:16.576124337Z import 'gevent.monkey' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4773c760>
2021-12-16T17:06:16.576128087Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_semaphore.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_semaphore.py
2021-12-16T17:06:16.576131887Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_semaphore.cpython-38.pyc'
2021-12-16T17:06:16.576139267Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_abstract_linkable.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_abstract_linkable.py
2021-12-16T17:06:16.576155250Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_abstract_linkable.cpython-38.pyc'
2021-12-16T17:06:16.576159166Z # extension module 'gevent._gevent_c_abstract_linkable' loaded from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_abstract_linkable.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.576163648Z # extension module 'gevent._gevent_c_abstract_linkable' executed from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_abstract_linkable.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.576167646Z import 'gevent._gevent_c_abstract_linkable' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa47753130>
2021-12-16T17:06:16.576171597Z import 'gevent._abstract_linkable' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa47ff0a00>
2021-12-16T17:06:16.576175408Z # extension module 'gevent._gevent_c_semaphore' loaded from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_semaphore.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.576179279Z # extension module 'gevent._gevent_c_semaphore' executed from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_semaphore.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.576183117Z import 'gevent._gevent_c_semaphore' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa47749e80>
2021-12-16T17:06:16.576186909Z import 'gevent._semaphore' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa47ffa460>
2021-12-16T17:06:16.576190719Z import 'gevent.lock' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4855efd0>
2021-12-16T17:06:16.576194479Z # /usr/local/lib/python3.8/site-packages/ssh2/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/ssh2/init.py
2021-12-16T17:06:16.576198279Z # code object from '/usr/local/lib/python3.8/site-packages/ssh2/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.576202093Z # /usr/local/lib/python3.8/site-packages/ssh2/pycache/_version.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/ssh2/_version.py
2021-12-16T17:06:16.576205890Z # code object from '/usr/local/lib/python3.8/site-packages/ssh2/pycache/_version.cpython-38.pyc'
2021-12-16T17:06:16.576209672Z import 'ssh2._version' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa477536a0>
2021-12-16T17:06:16.576213677Z import 'ssh2' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa47ff0ca0>
2021-12-16T17:06:16.576217466Z # extension module 'ssh2.error_codes' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/error_codes.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.576221274Z # extension module 'ssh2.error_codes' executed from '/usr/local/lib/python3.8/site-packages/ssh2/error_codes.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588313554Z import 'ssh2.error_codes' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa47ffa460>
2021-12-16T17:06:16.588359749Z # extension module 'ssh2.exceptions' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/exceptions.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588369164Z # extension module 'ssh2.exceptions' executed from '/usr/local/lib/python3.8/site-packages/ssh2/exceptions.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588375818Z import 'ssh2.exceptions' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa47ff0c40>
2021-12-16T17:06:16.588382954Z # extension module 'ssh2.session' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/session.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588389507Z # extension module 'ssh2.agent' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/agent.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588396093Z # extension module 'ssh2.pkey' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/pkey.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588402736Z # extension module 'ssh2.pkey' executed from '/usr/local/lib/python3.8/site-packages/ssh2/pkey.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588409225Z import 'ssh2.pkey' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa477497f0>
2021-12-16T17:06:16.588415745Z # extension module 'ssh2.utils' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/utils.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588422425Z # extension module 'ssh2.utils' executed from '/usr/local/lib/python3.8/site-packages/ssh2/utils.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588436449Z import 'ssh2.utils' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa47ff0d30>
2021-12-16T17:06:16.588443107Z # extension module 'ssh2.agent' executed from '/usr/local/lib/python3.8/site-packages/ssh2/agent.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588449394Z import 'ssh2.agent' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa47749ac0>
2021-12-16T17:06:16.588455982Z # extension module 'ssh2.channel' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/channel.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588463754Z # extension module 'ssh2.sftp' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/sftp.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588471738Z # extension module 'ssh2.sftp_handle' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/sftp_handle.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588478373Z # extension module 'ssh2.sftp_handle' executed from '/usr/local/lib/python3.8/site-packages/ssh2/sftp_handle.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588485510Z import 'ssh2.sftp_handle' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa466429d0>
2021-12-16T17:06:16.588492011Z # extension module 'ssh2.sftp' executed from '/usr/local/lib/python3.8/site-packages/ssh2/sftp.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588498302Z import 'ssh2.sftp' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa46642730>
2021-12-16T17:06:16.588504891Z # extension module 'ssh2.channel' executed from '/usr/local/lib/python3.8/site-packages/ssh2/channel.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588566426Z import 'ssh2.channel' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa47749940>
2021-12-16T17:06:16.588573835Z # extension module 'ssh2.listener' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/listener.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588580351Z # extension module 'ssh2.listener' executed from '/usr/local/lib/python3.8/site-packages/ssh2/listener.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588586715Z import 'ssh2.listener' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa466427c0>
2021-12-16T17:06:16.588593245Z # extension module 'ssh2.statinfo' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/statinfo.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588599593Z # extension module 'ssh2.statinfo' executed from '/usr/local/lib/python3.8/site-packages/ssh2/statinfo.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588624094Z import 'ssh2.statinfo' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa46642be0>
2021-12-16T17:06:16.588642968Z # extension module 'ssh2.knownhost' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/knownhost.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588649645Z # extension module 'ssh2.knownhost' executed from '/usr/local/lib/python3.8/site-packages/ssh2/knownhost.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588656113Z import 'ssh2.knownhost' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa46642c40>
2021-12-16T17:06:16.588714737Z # extension module 'ssh2.fileinfo' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/fileinfo.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588722544Z # extension module 'ssh2.fileinfo' executed from '/usr/local/lib/python3.8/site-packages/ssh2/fileinfo.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588729633Z import 'ssh2.fileinfo' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa46642eb0>
2021-12-16T17:06:16.588736151Z # extension module 'ssh2.publickey' loaded from '/usr/local/lib/python3.8/site-packages/ssh2/publickey.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588742575Z # extension module 'ssh2.publickey' executed from '/usr/local/lib/python3.8/site-packages/ssh2/publickey.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588748933Z import 'ssh2.publickey' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa46642fd0>
2021-12-16T17:06:16.588833378Z # extension module 'ssh2.session' executed from '/usr/local/lib/python3.8/site-packages/ssh2/session.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588841312Z import 'ssh2.session' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa47ffa340>
2021-12-16T17:06:16.588848051Z # /usr/local/lib/python3.8/site-packages/pssh/clients/native/pycache/tunnel.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/clients/native/tunnel.py
2021-12-16T17:06:16.588854588Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/clients/native/pycache/tunnel.cpython-38.pyc'
2021-12-16T17:06:16.588861216Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/server.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/server.py
2021-12-16T17:06:16.588903976Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/server.cpython-38.pyc'
2021-12-16T17:06:16.588912047Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/baseserver.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/baseserver.py
2021-12-16T17:06:16.588918571Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/baseserver.cpython-38.pyc'
2021-12-16T17:06:16.588925370Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/event.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/event.py
2021-12-16T17:06:16.588932028Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/event.cpython-38.pyc'
2021-12-16T17:06:16.588938232Z # extension module 'gevent._gevent_cevent' loaded from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_cevent.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588962056Z # extension module 'gevent._gevent_cevent' executed from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_cevent.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.588969037Z import 'gevent._gevent_cevent' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa46663c40>
2021-12-16T17:06:16.588975685Z import 'gevent.event' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4665cb50>
2021-12-16T17:06:16.588982345Z import 'gevent.baseserver' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa46654fd0>
2021-12-16T17:06:16.588988736Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/socket.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/socket.py
2021-12-16T17:06:16.588995339Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/socket.cpython-38.pyc'
2021-12-16T17:06:16.589017390Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_socket3.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_socket3.py
2021-12-16T17:06:16.589025546Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_socket3.cpython-38.pyc'
2021-12-16T17:06:16.589032071Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_socketcommon.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_socketcommon.py
2021-12-16T17:06:16.589038995Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_socketcommon.cpython-38.pyc'
2021-12-16T17:06:16.589045572Z import 'gevent._socketcommon' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4666cf40>
2021-12-16T17:06:16.589052086Z import 'gevent._socket3' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4666c160>
2021-12-16T17:06:16.589058398Z import 'gevent.socket' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4665ca90>
2021-12-16T17:06:16.589064878Z import 'gevent.server' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa46654730>
2021-12-16T17:06:16.589071295Z # /usr/local/lib/python3.8/site-packages/pssh/pycache/constants.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/constants.py
2021-12-16T17:06:16.589083664Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/pycache/constants.cpython-38.pyc'
2021-12-16T17:06:16.589090366Z import 'pssh.constants' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4665cac0>
2021-12-16T17:06:16.589096977Z import 'pssh.clients.native.tunnel' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa47ffa310>
2021-12-16T17:06:16.606913517Z # /usr/local/lib/python3.8/site-packages/pssh/clients/base/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/clients/base/init.py
2021-12-16T17:06:16.606947233Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/clients/base/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.606957427Z import 'pssh.clients.base' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa46663460>
2021-12-16T17:06:16.606966127Z # bytecode is stale for 'pssh.clients.base.single'
2021-12-16T17:06:16.606976323Z # code object from /usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py
2021-12-16T17:06:16.606984587Z # created '/usr/local/lib/python3.8/site-packages/pssh/clients/base/pycache/single.cpython-38.pyc'
2021-12-16T17:06:16.606993173Z # /usr/local/lib/python3.8/pycache/getpass.cpython-38.pyc matches /usr/local/lib/python3.8/getpass.py
2021-12-16T17:06:16.607001103Z # code object from '/usr/local/lib/python3.8/pycache/getpass.cpython-38.pyc'
2021-12-16T17:06:16.607009923Z # extension module 'termios' loaded from '/usr/local/lib/python3.8/lib-dynload/termios.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.607018193Z # extension module 'termios' executed from '/usr/local/lib/python3.8/lib-dynload/termios.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.607027927Z import 'termios' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa44b28fa0>
2021-12-16T17:06:16.607036208Z import 'getpass' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44b28d90>
2021-12-16T17:06:16.607044204Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/select.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/select.py
2021-12-16T17:06:16.607052458Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/select.cpython-38.pyc'
2021-12-16T17:06:16.607060228Z import 'gevent.select' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44b28f10>
2021-12-16T17:06:16.607068211Z # /usr/local/lib/python3.8/site-packages/pssh/clients/pycache/common.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/clients/common.py
2021-12-16T17:06:16.607076501Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/clients/pycache/common.cpython-38.pyc'
2021-12-16T17:06:16.607084841Z # /usr/local/lib/python3.8/site-packages/pssh/pycache/exceptions.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/exceptions.py
2021-12-16T17:06:16.607093938Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/pycache/exceptions.cpython-38.pyc'
2021-12-16T17:06:16.607101875Z import 'pssh.exceptions' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44b15460>
2021-12-16T17:06:16.607122721Z import 'pssh.clients.common' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44b15280>
2021-12-16T17:06:16.607131565Z # /usr/local/lib/python3.8/site-packages/pssh/clients/pycache/reader.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/clients/reader.py
2021-12-16T17:06:16.607139568Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/clients/pycache/reader.cpython-38.pyc'
2021-12-16T17:06:16.607147445Z import 'pssh.clients.reader' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44b15370>
2021-12-16T17:06:16.607155421Z # /usr/local/lib/python3.8/site-packages/pssh/pycache/output.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/output.py
2021-12-16T17:06:16.607163325Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/pycache/output.cpython-38.pyc'
2021-12-16T17:06:16.607171108Z import 'pssh.output' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44b154f0>
2021-12-16T17:06:16.607179501Z import 'pssh.clients.base.single' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa46663430>
2021-12-16T17:06:16.607187418Z # /usr/local/lib/python3.8/pycache/pprint.cpython-38.pyc matches /usr/local/lib/python3.8/pprint.py
2021-12-16T17:06:16.607195261Z # code object from '/usr/local/lib/python3.8/pycache/pprint.cpython-38.pyc'
2021-12-16T17:06:16.607202935Z import 'pprint' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44b157c0>
2021-12-16T17:06:16.607210815Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/threadpool.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/threadpool.py
2021-12-16T17:06:16.607218705Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/threadpool.cpython-38.pyc'
2021-12-16T17:06:16.607226501Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/pool.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/pool.py
2021-12-16T17:06:16.607234451Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/pool.cpython-38.pyc'
2021-12-16T17:06:16.607242198Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/queue.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/queue.py
2021-12-16T17:06:16.607250968Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/queue.cpython-38.pyc'
2021-12-16T17:06:16.607258858Z # extension module 'gevent._gevent_cqueue' loaded from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_cqueue.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.607266961Z # extension module 'gevent._gevent_cqueue' executed from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_cqueue.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.607275468Z import 'gevent._gevent_cqueue' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa44a16fa0>
2021-12-16T17:06:16.607283461Z import 'gevent.queue' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a10730>
2021-12-16T17:06:16.607297676Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_imap.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_imap.py
2021-12-16T17:06:16.607305906Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_imap.cpython-38.pyc'
2021-12-16T17:06:16.607313679Z # extension module 'gevent._gevent_c_imap' loaded from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_imap.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.607321689Z # extension module 'gevent._gevent_c_imap' executed from '/usr/local/lib/python3.8/site-packages/gevent/_gevent_c_imap.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.607329712Z import 'gevent._gevent_c_imap' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa44a22dc0>
2021-12-16T17:06:16.607337662Z import 'gevent._imap' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a107f0>
2021-12-16T17:06:16.607345596Z import 'gevent.pool' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a093d0>
2021-12-16T17:06:16.607353716Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/util.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/util.py
2021-12-16T17:06:16.607362352Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/util.cpython-38.pyc'
2021-12-16T17:06:16.607370260Z import 'gevent.util' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a105e0>
2021-12-16T17:06:16.607378296Z # /usr/local/lib/python3.8/site-packages/gevent/pycache/_threading.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/_threading.py
2021-12-16T17:06:16.607386352Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/pycache/_threading.cpython-38.pyc'
2021-12-16T17:06:16.607395000Z import 'gevent._threading' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a37dc0>
2021-12-16T17:06:16.607403342Z # /usr/local/lib/python3.8/concurrent/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/concurrent/init.py
2021-12-16T17:06:16.607411332Z # code object from '/usr/local/lib/python3.8/concurrent/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.607419100Z import 'concurrent' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a16b80>
2021-12-16T17:06:16.607427116Z # /usr/local/lib/python3.8/concurrent/futures/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/concurrent/futures/init.py
2021-12-16T17:06:16.607435362Z # code object from '/usr/local/lib/python3.8/concurrent/futures/pycache/init.cpython-38.pyc'
2021-12-16T17:06:16.607443700Z # /usr/local/lib/python3.8/concurrent/futures/pycache/_base.cpython-38.pyc matches /usr/local/lib/python3.8/concurrent/futures/_base.py
2021-12-16T17:06:16.607451682Z # code object from '/usr/local/lib/python3.8/concurrent/futures/pycache/_base.cpython-38.pyc'
2021-12-16T17:06:16.607459586Z import 'concurrent.futures._base' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a16880>
2021-12-16T17:06:16.607467572Z import 'concurrent.futures' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a16550>
2021-12-16T17:06:16.607481832Z # /usr/local/lib/python3.8/concurrent/futures/pycache/thread.cpython-38.pyc matches /usr/local/lib/python3.8/concurrent/futures/thread.py
2021-12-16T17:06:16.607489880Z # code object from '/usr/local/lib/python3.8/concurrent/futures/pycache/thread.cpython-38.pyc'
2021-12-16T17:06:16.607497780Z import 'concurrent.futures.thread' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a16670>
2021-12-16T17:06:16.607505716Z import 'gevent.threadpool' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44b15d30>
2021-12-16T17:06:16.607520892Z import 'pssh.clients.native.single' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4854b3a0>
2021-12-16T17:06:16.607529006Z # /usr/local/lib/python3.8/site-packages/pssh/clients/base/pycache/parallel.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/pssh/clients/base/parallel.py
2021-12-16T17:06:16.607537143Z # code object from '/usr/local/lib/python3.8/site-packages/pssh/clients/base/pycache/parallel.cpython-38.pyc'
2021-12-16T17:06:16.607544974Z import 'pssh.clients.base.parallel' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa449fe8e0>
2021-12-16T17:06:16.612688421Z import 'pssh.clients.native.parallel' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4854b940>
2021-12-16T17:06:16.612712752Z import 'pssh.clients.native' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa4854b790>
2021-12-16T17:06:16.612720346Z import 'pssh.clients' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa485e2b20>
2021-12-16T17:06:16.612726817Z import 'tasks' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa489bdca0>
2021-12-16T17:06:16.612733534Z # /usr/local/lib/python3.8/encodings/pycache/idna.cpython-38.pyc matches /usr/local/lib/python3.8/encodings/idna.py
2021-12-16T17:06:16.612741057Z # code object from '/usr/local/lib/python3.8/encodings/pycache/idna.cpython-38.pyc'
2021-12-16T17:06:16.612747748Z # /usr/local/lib/python3.8/pycache/stringprep.cpython-38.pyc matches /usr/local/lib/python3.8/stringprep.py
2021-12-16T17:06:16.612755106Z # code object from '/usr/local/lib/python3.8/pycache/stringprep.cpython-38.pyc'
2021-12-16T17:06:16.612761360Z # extension module 'unicodedata' loaded from '/usr/local/lib/python3.8/lib-dynload/unicodedata.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.612767877Z # extension module 'unicodedata' executed from '/usr/local/lib/python3.8/lib-dynload/unicodedata.cpython-38-x86_64-linux-gnu.so'
2021-12-16T17:06:16.612775186Z import 'unicodedata' # <_frozen_importlib_external.ExtensionFileLoader object at 0x7faa44a5f880>
2021-12-16T17:06:16.612782722Z import 'stringprep' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a5f130>
2021-12-16T17:06:16.612790245Z import 'encodings.idna' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a42e80>
2021-12-16T17:06:16.612796738Z 17:06:16 RQ worker 'rq:worker:2f2115472bf54b19bbc4db9800f563b6' started, version 1.0
2021-12-16T17:06:16.612980054Z 17:06:16 *** Listening on default_prio_queue...
2021-12-16T17:06:16.613768599Z 17:06:16 Cleaning registries for queue: default_prio_queue
2021-12-16T17:06:21.050921720Z # code object from /usr/local/lib/python3.8/_strptime.py
2021-12-16T17:06:21.050961497Z # created '/usr/local/lib/python3.8/pycache/_strptime.cpython-38.pyc'
2021-12-16T17:06:21.050971566Z import '_strptime' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a5fac0>
2021-12-16T17:06:21.050979675Z 17:06:21 default_prio_queue: tasks.remote() (19d9cf85-8dbd-4db9-ba03-091cec2135e2)
2021-12-16T17:06:21.062561270Z ERROR:root:Before connection
2021-12-16T17:06:21.063960028Z ERROR:pssh.clients.base.single:Before init
2021-12-16T17:06:21.064232437Z ERROR:pssh.clients.base.single:Before _connect
2021-12-16T17:06:21.064278272Z ERROR:pssh.clients.base.single:Before _get_addr_info

2021-12-16T17:06:21.064367307Z ERROR:pssh.clients.base.single:Before getaddrinfo

2021-12-16T17:06:31.071236599Z # /usr/local/lib/python3.8/site-packages/gevent/resolver/pycache/init.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/resolver/init.py
2021-12-16T17:06:31.071289501Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/resolver/pycache/init.cpython-38.pyc'
2021-12-16T17:06:31.071302141Z # /usr/local/lib/python3.8/site-packages/gevent/resolver/pycache/_addresses.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/resolver/_addresses.py
2021-12-16T17:06:31.071311158Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/resolver/pycache/_addresses.cpython-38.pyc'
2021-12-16T17:06:31.071319515Z import 'gevent.resolver._addresses' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a9a280>
2021-12-16T17:06:31.071329021Z import 'gevent.resolver' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a85790>
2021-12-16T17:06:31.071337587Z # /usr/local/lib/python3.8/site-packages/gevent/resolver/pycache/thread.cpython-38.pyc matches /usr/local/lib/python3.8/site-packages/gevent/resolver/thread.py
2021-12-16T17:06:31.071346187Z # code object from '/usr/local/lib/python3.8/site-packages/gevent/resolver/pycache/thread.cpython-38.pyc'
2021-12-16T17:06:31.071354256Z import 'gevent.resolver.thread' # <_frozen_importlib_external.SourceFileLoader object at 0x7faa44a85f10>
2021-12-16T17:06:31.071362533Z 17:06:31 rq.timeouts.JobTimeoutException: Task exceeded maximum timeout value (10 seconds)
2021-12-16T17:06:31.071371358Z Traceback (most recent call last):
2021-12-16T17:06:31.071379339Z File "/usr/local/lib/python3.8/site-packages/rq/worker.py", line 812, in perform_job
2021-12-16T17:06:31.071387616Z rv = job.perform()
2021-12-16T17:06:31.071395595Z File "/usr/local/lib/python3.8/site-packages/rq/job.py", line 588, in perform
2021-12-16T17:06:31.071404696Z self._result = self._execute()
2021-12-16T17:06:31.071412566Z File "/usr/local/lib/python3.8/site-packages/rq/job.py", line 594, in _execute
2021-12-16T17:06:31.071437830Z return self.func(*self.args, **self.kwargs)
2021-12-16T17:06:31.071445699Z File "/usr/src/app/tasks.py", line 17, in remote
2021-12-16T17:06:31.071453213Z ssh_client = SSHClient("xxx", user="xxx", password="xxx", pkey='xxx')
2021-12-16T17:06:31.071461312Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/native/single.py", line 135, in init
2021-12-16T17:06:31.071468712Z super(SSHClient, self).init(
2021-12-16T17:06:31.071475638Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 191, in init
2021-12-16T17:06:31.071485043Z self._init()
2021-12-16T17:06:31.071492061Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 199, in _init
2021-12-16T17:06:31.071499350Z self._connect(self._host, self._port)
2021-12-16T17:06:31.071506416Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 284, in _connect
2021-12-16T17:06:31.071513867Z addr_info = self._get_addr_info(host, port)
2021-12-16T17:06:31.071520734Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 270, in _get_addr_info
2021-12-16T17:06:31.071528087Z addr_info = socket.getaddrinfo(host, port, proto=socket.IPPROTO_TCP)
2021-12-16T17:06:31.071535153Z File "/usr/local/lib/python3.8/site-packages/gevent/_socketcommon.py", line 247, in getaddrinfo
2021-12-16T17:06:31.071542604Z addrlist = get_hub().resolver.getaddrinfo(host, port, family, type, proto, flags)
2021-12-16T17:06:31.071549697Z File "/usr/local/lib/python3.8/site-packages/gevent/resolver/thread.py", line 63, in getaddrinfo
2021-12-16T17:06:31.071557020Z return self.pool.apply(_socket.getaddrinfo, args, kwargs)
2021-12-16T17:06:31.071564084Z File "/usr/local/lib/python3.8/site-packages/gevent/pool.py", line 161, in apply
2021-12-16T17:06:31.071571479Z return self.spawn(func, *args, **kwds).get()
2021-12-16T17:06:31.071578393Z File "src/gevent/event.py", line 329, in gevent._gevent_cevent.AsyncResult.get
2021-12-16T17:06:31.071585793Z File "src/gevent/event.py", line 356, in gevent._gevent_cevent.AsyncResult.get
2021-12-16T17:06:31.071593199Z File "src/gevent/_abstract_linkable.py", line 487, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
2021-12-16T17:06:31.071600754Z File "src/gevent/_abstract_linkable.py", line 490, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
2021-12-16T17:06:31.071608214Z File "src/gevent/_abstract_linkable.py", line 442, in gevent._gevent_c_abstract_linkable.AbstractLinkable._AbstractLinkable__wait_to_be_notified
2021-12-16T17:06:31.071615764Z File "src/gevent/_abstract_linkable.py", line 451, in gevent._gevent_c_abstract_linkable.AbstractLinkable._switch_to_hub
2021-12-16T17:06:31.071623076Z File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
2021-12-16T17:06:31.071637820Z File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
2021-12-16T17:06:31.071645404Z File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch
2021-12-16T17:06:31.071653684Z File "/usr/local/lib/python3.8/site-packages/rq/timeouts.py", line 63, in handle_death_penalty
2021-12-16T17:06:31.071684260Z raise self._exception('Task exceeded maximum timeout value '
2021-12-16T17:06:31.071694076Z rq.timeouts.JobTimeoutException: Task exceeded maximum timeout value (10 seconds)
2021-12-16T17:06:31.071701154Z Traceback (most recent call last):
2021-12-16T17:06:31.071708028Z File "/usr/local/lib/python3.8/site-packages/rq/worker.py", line 812, in perform_job
2021-12-16T17:06:31.071715207Z rv = job.perform()
2021-12-16T17:06:31.071722039Z File "/usr/local/lib/python3.8/site-packages/rq/job.py", line 588, in perform
2021-12-16T17:06:31.071729285Z self._result = self._execute()
2021-12-16T17:06:31.071736071Z File "/usr/local/lib/python3.8/site-packages/rq/job.py", line 594, in _execute
2021-12-16T17:06:31.071743839Z return self.func(*self.args, **self.kwargs)
2021-12-16T17:06:31.071750845Z File "/usr/src/app/tasks.py", line 17, in remote
2021-12-16T17:06:31.071758247Z ssh_client = SSHClient("xxx", user="xxx", password="xxx", pkey='xxx')
2021-12-16T17:06:31.071766554Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/native/single.py", line 135, in init
2021-12-16T17:06:31.071773880Z super(SSHClient, self).init(
2021-12-16T17:06:31.071780949Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 191, in init
2021-12-16T17:06:31.071788320Z self._init()
2021-12-16T17:06:31.071795267Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 199, in _init
2021-12-16T17:06:31.071802629Z self._connect(self._host, self._port)
2021-12-16T17:06:31.071809600Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 284, in _connect
2021-12-16T17:06:31.071816981Z addr_info = self._get_addr_info(host, port)
2021-12-16T17:06:31.072532835Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 270, in _get_addr_info
2021-12-16T17:06:31.072541370Z addr_info = socket.getaddrinfo(host, port, proto=socket.IPPROTO_TCP)
2021-12-16T17:06:31.072546031Z File "/usr/local/lib/python3.8/site-packages/gevent/_socketcommon.py", line 247, in getaddrinfo
2021-12-16T17:06:31.072550551Z addrlist = get_hub().resolver.getaddrinfo(host, port, family, type, proto, flags)
2021-12-16T17:06:31.072555158Z File "/usr/local/lib/python3.8/site-packages/gevent/resolver/thread.py", line 63, in getaddrinfo
2021-12-16T17:06:31.072559534Z return self.pool.apply(_socket.getaddrinfo, args, kwargs)
2021-12-16T17:06:31.072579222Z File "/usr/local/lib/python3.8/site-packages/gevent/pool.py", line 161, in apply
2021-12-16T17:06:31.072588842Z return self.spawn(func, *args, **kwds).get()
2021-12-16T17:06:31.072592721Z File "src/gevent/event.py", line 329, in gevent._gevent_cevent.AsyncResult.get
2021-12-16T17:06:31.072601095Z File "src/gevent/event.py", line 356, in gevent._gevent_cevent.AsyncResult.get
2021-12-16T17:06:31.072605924Z File "src/gevent/_abstract_linkable.py", line 487, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
2021-12-16T17:06:31.072609732Z File "src/gevent/_abstract_linkable.py", line 490, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
2021-12-16T17:06:31.072613721Z File "src/gevent/_abstract_linkable.py", line 442, in gevent._gevent_c_abstract_linkable.AbstractLinkable._AbstractLinkable__wait_to_be_notified
2021-12-16T17:06:31.072617540Z File "src/gevent/_abstract_linkable.py", line 451, in gevent._gevent_c_abstract_linkable.AbstractLinkable._switch_to_hub
2021-12-16T17:06:31.072621513Z File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
2021-12-16T17:06:31.072625324Z File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
2021-12-16T17:06:31.072629253Z File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch
2021-12-16T17:06:31.072633054Z File "/usr/local/lib/python3.8/site-packages/rq/timeouts.py", line 63, in handle_death_penalty
2021-12-16T17:06:31.072636980Z raise self._exception('Task exceeded maximum timeout value '
2021-12-16T17:06:31.072640615Z rq.timeouts.JobTimeoutException: Task exceeded maximum timeout value (10 seconds)
2021-12-16T17:06:31.072644273Z ERROR:rq.worker:rq.timeouts.JobTimeoutException: Task exceeded maximum timeout value (10 seconds)
2021-12-16T17:06:31.072647962Z Traceback (most recent call last):
2021-12-16T17:06:31.072651670Z File "/usr/local/lib/python3.8/site-packages/rq/worker.py", line 812, in perform_job
2021-12-16T17:06:31.072655421Z rv = job.perform()
2021-12-16T17:06:31.072680359Z File "/usr/local/lib/python3.8/site-packages/rq/job.py", line 588, in perform
2021-12-16T17:06:31.072685001Z self._result = self._execute()
2021-12-16T17:06:31.072689833Z File "/usr/local/lib/python3.8/site-packages/rq/job.py", line 594, in _execute
2021-12-16T17:06:31.072696854Z return self.func(*self.args, **self.kwargs)
2021-12-16T17:06:31.072701401Z File "/usr/src/app/tasks.py", line 17, in remote
2021-12-16T17:06:31.072705215Z ssh_client = SSHClient("xxx", user="xxx", password="xxx", pkey='xxx')
2021-12-16T17:06:31.072709412Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/native/single.py", line 135, in init
2021-12-16T17:06:31.072713492Z super(SSHClient, self).init(
2021-12-16T17:06:31.072723274Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 191, in init
2021-12-16T17:06:31.072727638Z self._init()
2021-12-16T17:06:31.072731319Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 199, in _init
2021-12-16T17:06:31.072735170Z self._connect(self._host, self._port)
2021-12-16T17:06:31.072738842Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 284, in _connect
2021-12-16T17:06:31.072742756Z addr_info = self._get_addr_info(host, port)
2021-12-16T17:06:31.072746412Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 270, in _get_addr_info
2021-12-16T17:06:31.072750270Z addr_info = socket.getaddrinfo(host, port, proto=socket.IPPROTO_TCP)
2021-12-16T17:06:31.072754034Z File "/usr/local/lib/python3.8/site-packages/gevent/_socketcommon.py", line 247, in getaddrinfo
2021-12-16T17:06:31.072757895Z addrlist = get_hub().resolver.getaddrinfo(host, port, family, type, proto, flags)
2021-12-16T17:06:31.072761754Z File "/usr/local/lib/python3.8/site-packages/gevent/resolver/thread.py", line 63, in getaddrinfo
2021-12-16T17:06:31.072765574Z return self.pool.apply(_socket.getaddrinfo, args, kwargs)
2021-12-16T17:06:31.072769271Z File "/usr/local/lib/python3.8/site-packages/gevent/pool.py", line 161, in apply
2021-12-16T17:06:31.072773076Z return self.spawn(func, *args, **kwds).get()
2021-12-16T17:06:31.072776891Z File "src/gevent/event.py", line 329, in gevent._gevent_cevent.AsyncResult.get
2021-12-16T17:06:31.072780726Z File "src/gevent/event.py", line 356, in gevent._gevent_cevent.AsyncResult.get
2021-12-16T17:06:31.072784539Z File "src/gevent/_abstract_linkable.py", line 487, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
2021-12-16T17:06:31.072788454Z File "src/gevent/_abstract_linkable.py", line 490, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
2021-12-16T17:06:31.072792642Z File "src/gevent/_abstract_linkable.py", line 442, in gevent._gevent_c_abstract_linkable.AbstractLinkable._AbstractLinkable__wait_to_be_notified
2021-12-16T17:06:31.072797130Z File "src/gevent/_abstract_linkable.py", line 451, in gevent._gevent_c_abstract_linkable.AbstractLinkable._switch_to_hub
2021-12-16T17:06:31.072801064Z File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
2021-12-16T17:06:31.072805453Z File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
2021-12-16T17:06:31.072810242Z File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch
2021-12-16T17:06:31.072814205Z File "/usr/local/lib/python3.8/site-packages/rq/timeouts.py", line 63, in handle_death_penalty
2021-12-16T17:06:31.072818079Z raise self._exception('Task exceeded maximum timeout value '
2021-12-16T17:06:31.072825534Z rq.timeouts.JobTimeoutException: Task exceeded maximum timeout value (10 seconds)
2021-12-16T17:06:31.072829448Z Traceback (most recent call last):
2021-12-16T17:06:31.072833082Z File "/usr/local/lib/python3.8/site-packages/rq/worker.py", line 812, in perform_job
2021-12-16T17:06:31.072836922Z rv = job.perform()
2021-12-16T17:06:31.072840487Z File "/usr/local/lib/python3.8/site-packages/rq/job.py", line 588, in perform
2021-12-16T17:06:31.072844367Z self._result = self._execute()
2021-12-16T17:06:31.072848054Z File "/usr/local/lib/python3.8/site-packages/rq/job.py", line 594, in _execute
2021-12-16T17:06:31.072851938Z return self.func(*self.args, **self.kwargs)
2021-12-16T17:06:31.072855677Z File "/usr/src/app/tasks.py", line 17, in remote
2021-12-16T17:06:31.072859519Z ssh_client = SSHClient("xxx", user="xxx", password="xxx", pkey='xxx')
2021-12-16T17:06:31.072863562Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/native/single.py", line 135, in init
2021-12-16T17:06:31.072867413Z super(SSHClient, self).init(
2021-12-16T17:06:31.072871074Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 191, in init
2021-12-16T17:06:31.072875031Z self._init()
2021-12-16T17:06:31.072878591Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 199, in _init
2021-12-16T17:06:31.072882438Z self._connect(self._host, self._port)
2021-12-16T17:06:31.072886454Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 284, in _connect
2021-12-16T17:06:31.072890413Z addr_info = self._get_addr_info(host, port)
2021-12-16T17:06:31.072894019Z File "/usr/local/lib/python3.8/site-packages/pssh/clients/base/single.py", line 270, in _get_addr_info
2021-12-16T17:06:31.072897888Z addr_info = socket.getaddrinfo(host, port, proto=socket.IPPROTO_TCP)
2021-12-16T17:06:31.072901610Z File "/usr/local/lib/python3.8/site-packages/gevent/_socketcommon.py", line 247, in getaddrinfo
2021-12-16T17:06:31.072905653Z addrlist = get_hub().resolver.getaddrinfo(host, port, family, type, proto, flags)
2021-12-16T17:06:31.072909370Z File "/usr/local/lib/python3.8/site-packages/gevent/resolver/thread.py", line 63, in getaddrinfo
2021-12-16T17:06:31.072913213Z return self.pool.apply(_socket.getaddrinfo, args, kwargs)
2021-12-16T17:06:31.072916879Z File "/usr/local/lib/python3.8/site-packages/gevent/pool.py", line 161, in apply
2021-12-16T17:06:31.072920675Z return self.spawn(func, *args, **kwds).get()
2021-12-16T17:06:31.072925329Z File "src/gevent/event.py", line 329, in gevent._gevent_cevent.AsyncResult.get
2021-12-16T17:06:31.072929213Z File "src/gevent/event.py", line 356, in gevent._gevent_cevent.AsyncResult.get
2021-12-16T17:06:31.072932982Z File "src/gevent/_abstract_linkable.py", line 487, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
2021-12-16T17:06:31.072940366Z File "src/gevent/_abstract_linkable.py", line 490, in gevent._gevent_c_abstract_linkable.AbstractLinkable._wait_core
2021-12-16T17:06:31.072944379Z File "src/gevent/_abstract_linkable.py", line 442, in gevent._gevent_c_abstract_linkable.AbstractLinkable._AbstractLinkable__wait_to_be_notified
2021-12-16T17:06:31.072948317Z File "src/gevent/_abstract_linkable.py", line 451, in gevent._gevent_c_abstract_linkable.AbstractLinkable._switch_to_hub
2021-12-16T17:06:31.072952206Z File "src/gevent/_greenlet_primitives.py", line 61, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
2021-12-16T17:06:31.072956122Z File "src/gevent/_greenlet_primitives.py", line 65, in gevent._gevent_c_greenlet_primitives.SwitchOutGreenletWithLoop.switch
2021-12-16T17:06:31.072960000Z File "src/gevent/_gevent_c_greenlet_primitives.pxd", line 35, in gevent._gevent_c_greenlet_primitives._greenlet_switch
2021-12-16T17:06:31.072963970Z File "/usr/local/lib/python3.8/site-packages/rq/timeouts.py", line 63, in handle_death_penalty
2021-12-16T17:06:31.072967831Z raise self._exception('Task exceeded maximum timeout value '
2021-12-16T17:06:31.072971522Z rq.timeouts.JobTimeoutException: Task exceeded maximum timeout value (10 seconds)

Expected behavior
The worker shouldn't get stuck.

Actual behaviour
The worker is stuck.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional information
Python 3.8

Worker packages and main:
click 8.0.3
Deprecated 1.2.13
gevent 21.12.0
greenlet 1.1.2
parallel-ssh 2.8.0
pip 21.2.4
redis 4.0.2
rq 1.0
setuptools 57.5.0
ssh-python 0.9.0
ssh2-python 0.27.0
wheel 0.37.0
wrapt 1.13.3
zope.event 4.5.0
zope.interface 5.4.0

Redis
Redis server v=6.2.6

pkittenis · 2021-12-20T12:48:56Z

pkittenis
Dec 20, 2021
Maintainer

Hi there,

Thanks for the interest.

What you are describing is expected behaviour when the SSH host cannot be resolved via DNS. socket.getaddrinfo does DNS resolution for a host.

If this fails or times out, the SSHClient will time out. Can use the timeout setting to adjust how long the client waits for DNS resolution.

To verify DNS resolution works, run the following on the host that is running the SSHClient:

from gevent import socket

socket.getaddrinfo(host, 22, proto=socket.IPPROTO_TCP)

Where host is the host name.

Not seeing an issue with the library.

0 replies

adiarbili · 2021-12-21T06:31:34Z

adiarbili
Dec 21, 2021
Author

Hi,

Let me add more info:

I'm using IP address so no DNS is required here.
You missed the point on the "reproduce steps":

Steps to reproduce the behavior:
The issue is reproduced only if I import the module that contains PSSH import (e.g. tasks see below) and you need to import it in the worker code (worker.py) at the beginning of the module. If I put it in main() function, everything is working just fine.

Please take the code and run it, I provided a full example.

0 replies

pkittenis · 2021-12-21T10:11:48Z

pkittenis
Dec 21, 2021
Maintainer

It could be that RQ runner does not play well with gevent. You could try it in blocking mode instead of async. Can also try running the client in its own greenlet.

Your worker.py code does not use the tasks import. You are saying the issue only occurs if worker.py contains an unused import?

import redis
from rq import Queue, Worker, Connection
import tasks

def main():
    with Connection(redis.from_url("redis://redis:6379/0")):
        queue = Queue('default_prio_queue')
        worker = Worker([queue])
        worker.work()

if __name__ == "__main__":
    main()

0 replies

adiarbili · 2021-12-21T10:32:03Z

adiarbili
Dec 21, 2021
Author

Yes, I suspect that it's related to gevent and RQ, I think that RQ is using gevent as well and somehow it causes issues.
I will try that and update.

It doesn't matter if it's used/unused, but it does matter where you place the import statement.
If you comment it or move it to the main function then it runs without issues.

RQ is forking a process and maybe there is some race condition when loading the modules, it's just a guess.

Also, forgot to mention that if I use the native ParallelSSHClient everything is working even with the problematic import.

0 replies

pkittenis · 2021-12-21T10:52:26Z

pkittenis
Dec 21, 2021
Maintainer

Ok, well if an unused import is causing an issue I would suggest to not have that import.

Since the problem is with interacting with RQ, and there are ways to get it working available, not seeing an issue with the library or anything the library can do. The parallel client spawns greenlets for each client. Likely will need to do the same in worker code.

0 replies

adiarbili · 2021-12-21T11:18:57Z

adiarbili
Dec 21, 2021
Author

It's an example, but in a real case, I'm using the "unused" import.

But how come that it's working with ParallelSSHClient? something about the implementation is wrong there.
My reproduction is very simple as you can see, I don't understand why workarounds are needed here especially when
you're saying that your package has an issue interacting with RQ. The package should interact with other packages without any issues.

I cannot use unblocking mode, as you suggested it doesn't meet my needs.
Please advise how to solve it.

0 replies

pkittenis · 2021-12-21T11:42:05Z

pkittenis
Dec 21, 2021
Maintainer

Can also try running the client in its own greenlet.

Whether or not RQ and gevent play well together is not the responsibility of this library. The developer will need to find a way to get the code working for their use case, as above.

You could create a test case with a runner that does socket.getaddrinfo alone and raise that with gevent and/or RQ. This library does not own either.

Moved to discussions.

0 replies

adiarbili · 2021-12-22T22:38:29Z

adiarbili
Dec 22, 2021
Author

I'm pretty sure you still have some unwanted behavior, please look at the code below, it has the same outcome (blocked at connection) without using RQ :

main.py

import os
import logging

from pssh.clients import SSHClient


def remote():
    output_list = []
    logging.error("Before connection")
    ssh_client = SSHClient("10.141.27.23", user="root", password="xxxx")
    logging.error("After connection")
    logging.error("Before Execution")
    host_output_obj = ssh_client.run_command("ls -la", timeout=30)
    for line in host_output_obj.stdout:
        output_list.append(line)
    logging.error(output_list)
    logging.error("After Execution")


def main():
    pid = os.fork()
    if pid > 0:
        print("This is written by the parent process {}".format(os.getpid()))
        os.waitpid(pid, 0)
    else:
        print("This is written by the child process {}".format(os.getpid()))
        remote()


if __name__ == '__main__':
    main()

Run as a regular script: python3 main.py
Please run it multiple times, it seems that there is an issue when using pssh when forking.

4 replies

pkittenis Dec 23, 2021
Maintainer

Thanks, will take a look.

adiarbili Jan 2, 2022
Author

Any update?

pkittenis Jan 13, 2022
Maintainer

Does not seem to be an issue with this library, or anything this library can do.

The developer will need to take care to use gevent correctly when forking or otherwise using multiple processes with gevent. From gevent documentation, emphasis added:

reinit() → None[source]
Prepare the gevent hub to run in a new (forked) process.
This should be called immediately after os.fork() in the child process. This is done automatically by gevent.os.fork() or if the os module has been monkey-patched. If this function is not called in a forked process, symptoms may include hanging of functions like socket.getaddrinfo(), and the hub’s threadpool is unlikely to work.

Your code will need to use the above if that is your use case. If RQ is forking and using gevent without the above, it is a bug with RQ.

adiarbili Jan 16, 2022
Author

I checked your suggestion using the gevent.os.fork() instead of os.fork() and it solved the issue.
If I'm using os.fork() and call gevent.reinit() that also works.

I think it is worth mentioning it in your documentation.

Thank you very much.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Async Worker (python RQ) Are Stuck #334

{{title}}

Replies: 8 comments 4 replies

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

Select a reply

Async Worker (python RQ) Are Stuck #334

adiarbili Dec 16, 2021

Replies: 8 comments · 4 replies

pkittenis Dec 20, 2021 Maintainer

adiarbili Dec 21, 2021 Author

pkittenis Dec 21, 2021 Maintainer

adiarbili Dec 21, 2021 Author

pkittenis Dec 21, 2021 Maintainer

adiarbili Dec 21, 2021 Author

pkittenis Dec 21, 2021 Maintainer

adiarbili Dec 22, 2021 Author

pkittenis Dec 23, 2021 Maintainer

adiarbili Jan 2, 2022 Author

pkittenis Jan 13, 2022 Maintainer

adiarbili Jan 16, 2022 Author

adiarbili
Dec 16, 2021

Replies: 8 comments 4 replies

pkittenis
Dec 20, 2021
Maintainer

adiarbili
Dec 21, 2021
Author

pkittenis
Dec 21, 2021
Maintainer

adiarbili
Dec 21, 2021
Author

pkittenis
Dec 21, 2021
Maintainer

adiarbili
Dec 21, 2021
Author

pkittenis
Dec 21, 2021
Maintainer

adiarbili
Dec 22, 2021
Author

pkittenis Dec 23, 2021
Maintainer

adiarbili Jan 2, 2022
Author

pkittenis Jan 13, 2022
Maintainer

adiarbili Jan 16, 2022
Author