Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

items.get throws PydanticSchemaGenerationError #108

Open
gregorias opened this issue Sep 8, 2024 · 3 comments
Open

items.get throws PydanticSchemaGenerationError #108

gregorias opened this issue Sep 8, 2024 · 3 comments
Labels
bug Something isn't working

Comments

@gregorias
Copy link

Scenario & Reproduction Steps

I have an item that looks like this:

Screenshot 2024-09-08 at 18 31 12

And I try to fetch it with op_client.items.get(vault_id, item_id).

Actual Behavior

The get function throws:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<string>", line 41, in ruc
  File "/Users/grzesiek/.pyenv/versions/3.12.4/lib/python3.12/asyncio/base_events.py", line 687, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/onepassword/items.py", line 49, in get
    return Item.model_validate_json(response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/main.py", line 624, in model_validate_json
    return cls.__pydantic_validator__.validate_json(json_data, strict=strict, context=context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_mock_val_ser.py", line 93, in __getattr__
    val_ser = self._attempt_rebuild()
              ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_mock_val_ser.py", line 126, in handler
    if cls.model_rebuild(raise_errors=False, _parent_namespace_depth=5) is not False:
       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/main.py", line 562, in model_rebuild
    return _model_construction.complete_model_class(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_model_construction.py", line 568, in complete_model_class
    schema = cls.__get_pydantic_core_schema__(cls, handler)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/main.py", line 670, in __get_pydantic_core_schema__
    return handler(source)
           ^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 655, in generate_schema
    schema = self._generate_schema_inner(obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 924, in _generate_schema_inner
    return self._model_schema(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 739, in _model_schema
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1115, in _generate_md_field_schema
    common_field = self._common_field_schema(name, field_info, decorators)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1308, in _common_field_schema
    schema = self._apply_annotations(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 2104, in _apply_annotations
    schema = get_inner_schema(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 2085, in inner_handler
    schema = self._generate_schema_inner(obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 929, in _generate_schema_inner
    return self.match_type(obj)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1029, in match_type
    return self._match_generic_type(obj, origin)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1062, in _match_generic_type
    return self._list_schema(self._get_first_arg_or_any(obj))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 431, in _list_schema
    return core_schema.list_schema(self.generate_schema(items_type))
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 650, in generate_schema
    from_property = self._generate_schema_from_property(obj, obj)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 821, in _generate_schema_from_property
    schema = get_schema(
             ^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/main.py", line 670, in __get_pydantic_core_schema__
    return handler(source)
           ^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 924, in _generate_schema_inner
    return self._model_schema(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 739, in _model_schema
    {k: self._generate_md_field_schema(k, v, decorators) for k, v in fields.items()},
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1115, in _generate_md_field_schema
    common_field = self._common_field_schema(name, field_info, decorators)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1308, in _common_field_schema
    schema = self._apply_annotations(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 2104, in _apply_annotations
    schema = get_inner_schema(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_schema_generation_shared.py", line 83, in __call__
    schema = self._handler(source_type)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 2085, in inner_handler
    schema = self._generate_schema_inner(obj)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 929, in _generate_schema_inner
    return self.match_type(obj)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 1038, in match_type
    return self._unknown_type_schema(obj)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/grzesiek/Library/Caches/pypoetry/virtualenvs/findata-fetcher-3hK6JJJX-py3.12/lib/python3.12/site-packages/pydantic/_internal/_generate_schema.py", line 558, in _unknown_type_schema
    raise PydanticSchemaGenerationError(
pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for [<class 'onepassword.types.ItemFieldDetailsOtp'>]. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.9/u/schema-for-unknown-type

Expected Behavior

The SDK should return the relevant item or at least an exception that is relevant to the caller (e.g., that the functionality is not implemented).

SDK version

0.1.1

Additional information

No response

@Marton6
Copy link
Member

Marton6 commented Sep 9, 2024

Hey @gregorias ! Thanks for reaching out about this bug. The functionality you're using should be supported and this is indeed a bug.

We will look into it!

@AndyTitu
Copy link
Contributor

AndyTitu commented Sep 9, 2024

Hey @gregorias, could you share more about the project setup in which this error occurs? We're attempting to reproduce this issue and we were wondering whether:

  • you can share the code of the project you're running our SDK in (given that doesn't contain anything personal or sensitive)?
  • you can share how did you install the onepassword package, and how are you managing dependencies? I'm interested in this since I noticed in your logs you are using poetry.
  • you can check the pydantic version that poetry is pulling for you?
  • you can share how are you running the code that is calling our SDK? Are you using python3 your_main.py or poetry shell && python3 your_main.py or poetry run python your_main.py or are you running the SDK in a test file? If you're running the SDK in tests, what testing framework are you using?

@gregorias
Copy link
Author

gregorias commented Sep 9, 2024

So while working the questions, I made the issue disappear by specifying v0.1.1 specifically in the manifest file:

onepassword = { git = "https://github.com/1Password/onepassword-sdk-python.git", tag = 'v0.1.1' }

It was like this before:

onepassword = { git = "https://github.com/1Password/onepassword-sdk-python.git" }

source

and it was installing d9e45c9. 713e09f, on the other hand, works fine.
I bisected the commits and d9e45c9 is the bad commit that breaks my use-case.

Here’s a script that I am using to reproduce this issue:

# repro.py
from onepassword.client import Client  # type: ignore

async def get_vault_id(client, vault_name: str) -> str | None:
    async for vault in await client.vaults.list_all():
        if vault.title == vault_name:
            return vault.id
    return None

async def get_item_id(client, vault_id, item_name: str) -> str | None:
    async for item in await client.items.list_all(vault_id):
        if item.title == item_name:
            return item.id
    return None


async def main():
    client = await Client.authenticate(
        auth=
        'redacted_for_obvious_reasons',
        integration_name="Findata Fetcher",
        integration_version="1.0.0")
    af_vault_id = await get_vault_id(client, 'Redacted')
    af_item_id = await get_item_id(client, af_vault_id, 'degiro.nl')
    item = await client.items.get(af_vault_id, af_item_id)
    return client, af_vault_id, af_item_id, item

if __name__ == '__main__':
    import asyncio
    asyncio.run(main())

I am running this in the virtual environment created for Findata Fetcher by Poetry.

I added a print to items.py and here’s the JSON response that the code in the stack trace is seeing (with redactions):

{
    "id": "redacted",
    "title": "degiro.nl",
    "category": "Login",
    "vaultId": "redacted",
    "fields": [
        {
            "id": "username",
            "title": "username",
            "sectionId": null,
            "fieldType": "Text",
            "value": "redacted",
            "details": null
        },
        {
            "id": "password",
            "title": "password",
            "sectionId": null,
            "fieldType": "Concealed",
            "value": "redacted",
            "details": null
        },
        {
            "id": "TOTP_redacted",
            "title": "one-time password",
            "sectionId": "add more",
            "fieldType": "Totp",
            "value": "redacted",
            "details": {
                "type": "Otp",
                "content": {
                    "code": "redacted",
                    "errorMessage": null
                }
            }
        },
        {
            "id": "redacted",
            "title": "mobile app code",
            "sectionId": "add more",
            "fieldType": "Concealed",
            "value": "redacted",
            "details": null
        }
    ],
    "sections": [
        {
            "id": "add more",
            "title": ""
        }
    ],
    "version": 4
}

Looks like details is optional after all.

I’m using Pydantic at 2.9.0, but I can also reproduce it at 2.9.1.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants