Issue with accessing a private JFrog registry

[shaunak-cisco]

I am trying to run the basic pull example provided here: https://oras-project.github.io/oras-py/getting_started/user-guide.html with the python sdk. Following is the code snippet I am trying to run. The file getting pulled is already present in the repo pushed with the oras cli. I have tried pulling with the cli as well as a Golang SDK example which worked fine.

import oras.client
from oras.logger import setup_logger, logger
setup_logger(quiet=False, debug=True)

password = "<>"
user = "<>"
host = "dockerhub.<>.com"
client = oras.client.OrasClient()

print(client.login(username=user, password=password, hostname=host))
client.pull(target="dockerhub.<>.com/<>/<>:<version>")

Following is a snippet of the exception:

✗ python oras_test_sdk.py
{'Status': 'Login Succeeded'}
Retrying in 3 seconds - error: 'TokenAuth' object has no attribute 'prefix'
Retrying in 5 seconds - error: 'TokenAuth' object has no attribute 'prefix'
Retrying in 11 seconds - error: 'TokenAuth' object has no attribute 'prefix'
Retrying in 29 seconds - error: 'TokenAuth' object has no attribute 'prefix'
Retrying in 83 seconds - error: 'TokenAuth' object has no attribute 'prefix'
Traceback (most recent call last):
  File "/path/to/your/script/oras_test_sdk.py", line 11, in <module>
    client.pull(target="dockerhub.<domain>.com/<repo>/<image>:<version>")
  File "/path/to/virtualenv/lib/python3.11/site-packages/oras/provider.py", line 871, in pull
    manifest = self.get_manifest(container, allowed_media_type)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/virtualenv/lib/python3.11/site-packages/oras/decorator.py", line 36, in __call__
    return self.func(cls, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/virtualenv/lib/python3.11/site-packages/oras/provider.py", line 928, in get_manifest
    response = self.do_request(get_manifest, "GET", headers=headers)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/virtualenv/lib/python3.11/site-packages/oras/decorator.py", line 63, in __call__
    return self.func(cls, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/virtualenv/lib/python3.11/site-packages/oras/provider.py", line 978, in do_request
    headers, changed = self.auth.authenticate_request(response, headers)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/virtualenv/lib/python3.11/site-packages/oras/auth/token.py", line 84, in authenticate_request
    token = self.request_token(h)
            ^^^^^^^^^^^^^^^^^^^^^
  File "/path/to/virtualenv/lib/python3.11/site-packages/oras/auth/token.py", line 117, in request_token
    h.realm = f"{self.prefix}://{h.realm}"
                 ^^^^^^^^^^^
AttributeError: 'TokenAuth' object has no attribute 'prefix'

Looking at the above logs, the prefix attribute seems to get referred in /oras/auth/token.py:L117 but I could not see where it is getting set in the codebase.

Any pointers on if my code may not be accurate here? cc: @vsoch

Edit: Version used: 0.2.22

[tarilabs]

can you kindly confirm which version of oras-py are you using?

btw fwiw and maybe it could help, it works for me with latest oras-py for push/pull for the public dockerhub: https://hub.docker.com/repository/docker/matteomortari/demo20241017-oraspy164
with the following snippet:

# ...
host = "registry-1.docker.io"
client = oras.client.OrasClient()

print(client.login(username=user, password=password, hostname=host))
client.push(target="registry-1.docker.io/matteomortari/demo20241017-oraspy164:latest", files=["artifact.txt"])
client.pull(target="registry-1.docker.io/matteomortari/demo20241017-oraspy164:latest", outdir="tmp")

Using user and password per configured PAT which I've also tested from oras cli (go).

---

Personally, I refrain from using programmatic login, and rely on the ~/.docker/config.json or similar auth secret from the environment.

---

Btw, noticed the oras cli (go) does not require the registry-1. prefix and I can just specify docker.io/... as a target when using the cli, but not for oras-py, likely missing some follow-redirect or some convention which is onboarded from oras cli (go). Looks to me a corner case for public dockerhub/docker.io specifically, fwiw. ( Ref #147 )

Hope this helps!

[shaunak-cisco]

Hi @tarilabs, I am using a private registry here for dockerhub. I have tried running on the latest version (0.22.0) where I was getting the above error, updated the version value in original post.

I also tried out with a lower version: 0.2.1 with the same snippet, in this case, the initial request was saying unauthorized, the next request after updating headers still failed with a 404 error even when data was present. The header update seemed to add a bearer token on this version.

No Authorization, requesting anonymous token
Final params are {'service': 'example-service', 'scope': 'repository:example-repo:pull'}
Successfully obtained anonymous token!
The named manifest is not known to the registry.
Traceback (most recent call last):
  File "script.py", line 11, in <module>
    client.pull(target="example-repo")
  File "provider.py", line 871, in pull
    manifest = self.get_manifest(container, allowed_media_type)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "decorator.py", line 35, in __call__
    return self.func(cls, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "provider.py", line 929, in get_manifest
    self._check_200_response(response)
  File "provider.py", line 651, in _check_200_response
    raise ValueError(f"Issue with {response.request.url}: {response.reason}")
ValueError: Issue with https://example.com/v2/example-repo/manifests/example-tag:

[shaunak-cisco]

Also, checking the documentation: https://oras-project.github.io/oras-py/getting_started/user-guide.html where it says As of version 0.1.0 we no longer provide a default client alongside oras Python, and if you need a client you should use [oras ](https://github.com/oras-project/oras)in Go., is our case not supported out of the box by the sdk, do we need a custom client here?

[tarilabs]

Can you provide more details about which registry is used as a "private dockerhub" so to attempt replicate the issue you are seeing?
As per #164 (comment) you can see it works for me using public dockerhub

fwiw I'm also using CNCF Distribution, Zot and Quay (-lite) in another project regularly and that work for me using simply same environment auths.

[tarilabs]

About

As of version 0.1.0 we no longer provide a default client alongside oras Python, and if you need a client you should use oras in Go.

I think that refers to the fact that oras-py originally provided CLI bindings (ie, after pip install, you could use oras in Python as the cli client) but since they are advising to directly use Oras (go) as a cli client to avoid "duplications". I use oras-py as an SDK (ie as a Python library) in another project that way.

But happy to be corrected here.

[shaunak-cisco]

Confirmed on this, we are using a JFrog registry for hosting artifacts.

Can you provide more details about which registry is used as a "private dockerhub" so to attempt replicate the issue you are seeing?

[vsoch]

The bug is here @tarilabs - the prefix is expected to be for the registry, but with the refactor it was removed. It was originally derived here:

oras-py/oras/provider.py

Line 69 in 36ef98a

self.prefix: str = "http" if insecure else "https"

and so an easy fix is to move it, or pass forward.

[tarilabs]

but with the refactor it was removed

I think you mean the

• wip: separate auth into backends #134

refactor?

so an easy fix is to move it, or pass forward

I think you mean like this:

• core: add missing prefix property to auth backend #165

but I wish I understood better which test case to reproduce the failure :/ do you have some suggestions, please?

[shaunak-cisco]

I was able to get my code to work after applying the changes in the PR: #165 with the below snippet:

import oras.client
from oras.logger import setup_logger, logger
setup_logger(quiet=False, debug=True)

password = "<>"
user = "<>"
host = "dockerhub.<>.com"
client = oras.client.OrasClient(auth_backend="basic")
config_path = "<config_path>"
# print(client.login(username=user, password=password, hostname=host, config_path=))
client.push(files=["<local_file>"], target="<registry>/<repository>/<artifact>:<tag>", config_path=[config_path])
client.pull(target="<registry>/<repository>/<artifact>:<tag>", outdir="<output_dir>", config_path=[config_path])
client.logout(host)

One question about the config_path variable in the login vs push / pull functions. The login function in oras/main/login.py has a type annotation dockercfg_path: Optional[str] = None,, but was erroring out when provided with a list, it worked when a str was provided.

    if os.path.exists(dockercfg_path):  # type: ignore
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen genericpath>", line 19, in exists
TypeError: stat: path should be string, bytes, os.PathLike or integer, not list

For the pull and push functions in oras/provider.py, these functions have a str requirement: config_path: Optional[str] = None, but worked when provided with a list of config paths as the subsequent auth functions require a list.

Can this be fixed? Should I open a different issue for this?

[vsoch]

Definitely! I would gladly review a PR to fix these.

AFAIK the typing is just for checking during CI, etc., and nothing is checked or enforced when using a library.

[tarilabs]

Definitely! I would gladly review a PR to fix these.

raised my proposal as #166 :) hope this helps!

[tarilabs]

@shaunak-cisco could you kindly test again with 0.2.24 ?

That should solve original issue of prefix from #164 (comment)
and should solve about wrong type hints and consistency from #164 (comment)

as oras-py 0.2.24 contains both:

• core: add missing prefix property to auth backend #165
• core: align config_path type annotation #166

so that we could resolve this issue accordingly?