-
Notifications
You must be signed in to change notification settings - Fork 85
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ak.from_parquet slower than pa.parquet.read_table + ak.from_arrow #3151
Comments
But is the time difference really seen in the |
The |
Okay. If the speed difference persists after replacing pa.parquet.read_table with pa.parquet.ParquetFile.read_row_groups, then there is something in the Awkward code that's impeding performance, because the Awkward code is supposed to be just pa.parquet.ParquetFile.read_row_groups followed by ak.from_arrow. |
Version of Awkward Array
2.6.5
Description and code to reproduce
In benchmarking GPU resources I ran into a curious performance difference in trying to compare CPU based reads with arrow to GPU-DMA reads via cudf.
Is this expected? A factor of two, coming only from reading (all other bits of code are the same) seems like performance left on the floor.
The text was updated successfully, but these errors were encountered: