When using groupby, change the type to distort the value.

[tommyhj217]

Hi
DataFrame delay_time has values from -276 to 111 and dtype=int64.

The number of values is 206.

df_delaytime_data = vdf_2.groupby(['delay_time'], agg={'cnt':'count'})

However, if you do groupby, the dtype of the column is changed to int8, and the value of -128 or higher is changed to positive.

Is this a problem with groupby?
How can we solve it?

Thanks
Hyunjun

[JovanVeljanoski]

Hi,

I can't reproduce your issue. Can you say anything more about the data, are you loading it from an hdf5 file, do you do any other operations before it etc..

This example works for me:

import vaex
import numpy as np

x = np.arange(-218, 218, dtype=np.int64)
df = vaex.from_arrays(x=x)
df = df.export_hdf5('../tmp.hdf5')
df = vaex.open('../tmp.hdf5')
df.groupby(['x'], agg={'cnt': 'count'})

Can you provide a minimal example as above that showcases the issue?

[tommyhj217]

Hi

Please check below capture and code.
It reproduce same issue.

x = [-129,-120,15,30,40]
df = vaex.from_arrays(x=x)
df.groupby(['x'], agg={'cnt': 'count'})

Thanks

[maartenbreddels]

This seems like a bug, would be great if someone can open a PR with this in a test in tests/groupby_test.py, then I'm happy to fix it!

[JovanVeljanoski]

I confirm this is a bug and I can reproduce it via @tommyhj217 latest example!
Thanks @tommyhj217 ! I will convert this discussion to an issue for better tracking.

[tommyhj217]

Thank you for checking and debugging issues.

As a result of checking more, if you change the type to float, it will not be reproduced.
I don't know, but is it related to the code below?

Thanks

[JovanVeljanoski]

No idea yet, but worth looking. Btw our core team just stared our holidays so we might not reply for the next 10-14 days or so :).