Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BigQuery's unnest function implementation overrides and breaks this function for other dialects #882

Open
altvod opened this issue Jul 20, 2023 · 4 comments
Labels
api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@altvod
Copy link

altvod commented Jul 20, 2023

If bigquery is installed, the unnest function (defined here) overrides the unnest function for all dialects.

In PostgreSQL usage of unnest is allowed with column identifiers (unnesting an array contained in a table column). But the global re-definition of unnest caused by bigquery makes code like this sa.func.unnest(some_column) fail with TypeError: The argument to unnest must have an ARRAY type. because, obviously, a column is not an array literal.

In my case I really do need many dialects installed at the same time.

@product-auto-label product-auto-label bot added the api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. label Jul 20, 2023
@altvod
Copy link
Author

altvod commented Jul 20, 2023

The best fix I can think of is to move the logic from this class to the visitor that renders functions in the BQ dialect implementation

@chalmerlowe
Copy link
Collaborator

Sorry to hear that you are running into this issue. Thank you submitting this bug/feature request.

If you happen to have code OR a patch that you would like to submit to address this issue feel free to submit a PR.

In the meantime, I will consider this feature request and explore where it should fit in our priority list based on current staffing and workloads.

@chalmerlowe chalmerlowe added the priority: p3 Desirable enhancement or fix. May not be included in next release. label Jul 21, 2023
@Linchin Linchin added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Nov 1, 2023
@fordhoka
Copy link

Is there an ETA on this? This is complicating our upgrade to sqlalchemy 2.

In case this helps anyone else experiencing this issue, our workaround was to always unregister the unnest function before connecting to postgresql and then always register it again before connecting to bigquery.

@r1b
Copy link

r1b commented Dec 13, 2024

Is deprecation acceptable? Couple of options I was poking at:

  1. Set package = "bigquery" as described in the docs - implementation moves to sqlalchemy.func.bigquery.unnest
  2. Export an unnest wrapper that applies this validation, then returns plain old sqlalchemy.func.unnest - the postgres dialect does something similar for array_agg (link here)

I didn't really see a good way to do this without a deprecation period though - I looked at this suggestion:

#882 (comment)

but it seems like once we're about to render the function, its too late to apply this validation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigquery Issues related to the googleapis/python-bigquery-sqlalchemy API. priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

5 participants