Kathara requires recent Docker / Kathara (docker-py) incompatibility with docker.io with Debian version suffixes incl. backported versions (was: Debian 11 (bullseye) with kathara Ubuntu focal packages fails kathara lstart with ValueError)

[alphanet72]

Operating System

Debian GNU/Linux 11

Kathará Version

3.7.7

Bug Description

Starting an individual container works (kathara vstart -n quicktest)

Starting a lab fails with error: CRITICAL (ValueError) invalid literal for int() with base 10: '5+dfsg1'
at step 4, then stops just after outputing Deploying devices

The "5+dfsg1" is suspiciously similar to the Manager version (see the kathara check below).

Steps To Reproduce

kathara lstart

lab.conf.txt

Expected Behavior

Used to work with the same OS version and kathara 3.7.1. Or its the version of docker.io which triggers this parsing bug.

Check Command Output

Current Manager is:             Docker (Kathara)
Manager version is:             20.10.5+dfsg1
Python version is:              3.11.9 (main, Apr  6 2024, 17:59:24) [GCC 9.4.0]
Kathara version is:             3.7.7
Operating System version is:    Linux-5.10.0-32-amd64-x86_64
[Deploying devices]
[Deleting devices]
Container run successfully.

[tcaiazzi]

Dear @alphanet72,

Thanks for reaching out!

Your error is already covered by issue #301 😇

The error you are encountering is due to the use of an outdated version of Docker: docker.io. Unfortunately, there are some incompatibilities between docker-py (the official Docker APIs for Python) and docker.io.

To fix the error, you simply need to install the newer version of Docker Engine (also known as docker-ce).

You can follow the official installation guide provided by Docker: https://docs.docker.com/engine/install/

Please let me know if this helps.

Thanks,
Tommaso

[alphanet72]

Hello,

On Wed, Sep 11, 2024 at 01:54:58AM -0700, Tommaso Caiazzi wrote: Your error is already covered by issue #301 ????

Did not see (actually I forgot to check the closed issues).

However, the error you are encountering is due to the use of an outdated version of Docker: docker.io. Unfortunately, there are some incompatibilities between docker-py (the official Docker APIs for Python) and docker.io.

No, it is due to the fact that you parse the version string incorrectly. This bug will reappear as soon as people start to use backport versions, etc. Can't you fix your version parser with a regex such as (\d+) ?

To fix the error, you simply need to install the newer version of Docker Engine (also known as docker-ce).

This won't be a Debian-supported version.

[tcaiazzi]

Dear @alphanet72,

To parse the version we use the functionalities offered by 'docker-py', the official Docker APIs for Python.

Since Kathará strongly relies on docker-py, we are afraid that other functionalities (aside from version parsing) offered by the library could be incopatible with docker.io. Do you have any insight on this?

Moreover, your engine version is quite old. Indeed, currently Docker (from official Docker release) is at version 27.2.1.

Current Manager is:             Docker (Kathara)
Manager version is:             27.2.1
Python version is:              3.11.9 (main, Apr  6 2024, 17:59:24) [GCC 11.4.0]
Kathara version is:             3.7.7
Operating System version is:    Linux-5.15.153.1-microsoft-standard-WSL2-x86_64

I'm not sure that all the fucntionalties offered by Kathará works on that version. Or maybe the packages use a completly different versioning?

Probably the right thing to do is to open an issue directly to docker-py and try to ask if it is possible to fix their incopatibility with docker.io. Moreover, we can ask if they know any other incopatibilities.

Sorry for the inconvenience, I hope you can understand.

[alphanet72]

Hello,

On Wed, Sep 11, 2024 at 04:30:58AM -0700, Tommaso Caiazzi wrote: To parse the version we use the functionalities offered by 'docker-py', the official Docker APIs for Python.

Ok, I understand.

I'm not sure that all the fucntionalties offered by Kathará works on that version.

I understand, and I think the risk is too hight, indeed.

Probably the right thing to do is to open an issue directly to `docker-py` and try to ask if it is possible to fix their incopatibility with `docker.io`.

Could you do that for me, as you understand more the internals of docker-py than me? The actual message to pass to them is: what happens when someone backports your version and the Debian package has a subversion, such as: 20.10.5+dfsg1 They should support this naming (or at least output a warning, not fail completely with a int() parse failure). That's my point of view. Of course, docker-py might not like at all the idea to support backported Debian versions, e.g.

Sorry for the inconvenience, I hope you can understand.

Installing the latest Docker CE was not as complicated as I thought, and it seems to not break anything else for now :) So it's fixed on my part. Thank you.

[tcaiazzi]

Could you do that for me, as you understand more the internals of docker-py than me?

I'll handle it as soon as possible :)

Meanwhile, I'll keep this issue open while we wait for their response. I'll also link the new issue here once it is created.

Could you also please update the title of the issue to something like:

"Kathara Incompatibility with Docker.io (and other backported versions)"

If you think of a better title, feel free to go with that! 😄

Thanks a lot!

[alphanet72]

Hello,

On Wed, Sep 11, 2024 at 05:48:33AM -0700, Tommaso Caiazzi wrote: "Kathara Incompatibility with Docker.io (and other backported versions)"

I tried something :)

[przygienda]

that patch is relatively simple in /usr/lib/python3/dist-packages/docker/utils/utils.py but stuff still doesn't work (I cannot move to docker-ce for weird reasons)

`

def compare_version(v1, v2):
"""Compare docker versions

>>> v1 = '1.9'
>>> v2 = '1.10'
>>> compare_version(v1, v2)
1
>>> compare_version(v2, v1)
-1
>>> compare_version(v2, v2)
0
"""

v1 = v1.split('+')[0]
v2 = v2.split('+')[0]
s1 = StrictVersion(v1)
s2 = StrictVersion(v2)
if s1 == s2:
    return 0
elif s1 > s2:
    return -1
else:
    return 1

`

kathara still falls flat on its face. did you compile stuff in via cython or something ?

[tcaiazzi]

Dear @przygienda,

It is a pleasure to hear from you! I hope you are fine!

Since we understand that some people may prefer to use docker.io, we included a temporary fix to handle docker.io versioning, waiting for an official answer from docker-py.

kathara still falls flat on its face. did you compile stuff in via cython or something ?

Yes, Kathará is compiled through Nuitka.

However, I pushed a patched version on a new brach: https://github.com/KatharaFramework/Kathara/tree/fix-version-comparison

If you are working on Linux I can build the patched package for you. In alternative, you can use Kathará from source or you can build the package by your own. I can guide you through the process, if needed.

Anyway, I provide you a python package to install using pip (so that it works on any OS you are working):

kathara-3.7.7.tar.gz

python3 -m pip install kathara-3.7.7.tar.gz

After installing, you can use Kathará from the CLI:

python3 -m kathara lstart

Let me know if this resolves the problem and if I can help further.

[przygienda]

Hey Tomasso ;-) yeah, I fall back on python version I think since I'm on 12.05 ubuntu I have no control over ;-)
I tried to compile from scratch but pacman just got confused with the release as well

your patched python stuff works on venv fine. thanks ...

[przygienda]

almost, you didn't compile the pyuv thing in so connect does not work. I compiled master of pyuv (the one with the python 3.11 fixes) but no cigar, not sure why it doesn't pick it up from my venv

[przygienda]

finally success. building pyuv from master branch like this

~/.venv/bin/python setup.py build_ext --inplace install

installs it and kathara fix works (so far)

[tcaiazzi]

Hi @przygienda,

Apologies for the delayed response!

I'm glad to hear you got it working.

By the way, to install the pyuv module, you can also use this command (from within the virtual environment):

python3 -m pip install git+https://github.com/saghul/pyuv@master#egg=pyuv

I meant to mention that earlier!

Feel free to reach out if you run into any further issues.