Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backlog: Xarray Private API Workaround #508

Open
cyschneck opened this issue Nov 3, 2023 · 1 comment
Open

Backlog: Xarray Private API Workaround #508

cyschneck opened this issue Nov 3, 2023 · 1 comment
Labels
blocked Work got blocked waiting the output of some other source/work bug Something isn't working

Comments

@cyschneck
Copy link
Contributor

Describe the bug
Xarray transition to a private API had introduced an error addressed in #381

AttributeError: 'DataArray' object has no attribute '_data'

PR #492 fix is non-ideal method to address private API from Xarray, but is a fix as contains_cftime_datetimes does not accept DataArrays. Current fix allows for xarray to be unpinned from version 2023.02.0

Fix required a change in geocat/comp/climatologies.py from d_arr to d_arr.variable

def _contains_datetime_like_objects(d_arr):
    """Check if a variable contains datetime like objects (either
    np.datetime64, or cftime.datetime)"""
    return np.issubdtype(
        d_arr.dtype,
        np.datetime64) or xr.core.common.contains_cftime_datetimes(d_arr.variable)

This required a change in geocat/comp/interpolation.py to specify dims directly as [lev_dim] (to fix ValueError)

# Line 595
for idx, (d, s) in enumerate(zip(data_stacked, sigma_stacked)):
            output[idx, :] = xr.DataArray(_vertical_remap(
                func_interpolate, s.data, sig_coords.data, d.data),
                                          dims=[lev_dim])

....
# Line 606
output[:len(hyam)] = xr.DataArray(_vertical_remap(
            func_interpolate, sigma.data, sig_coords.data, data.data),
                                          dims=[lev_dim])

A conflict between pandas deprecrating loffset and xarray not also required a fix in geocat/comp/climatologies.py (to fix a ValueError) to preserve the "MS" (MonthBegin) resampling with time offset arithmetic

# Line 303
    # Group the months into three and take the mean
    means = data_filter.resample({
        time_coord_name: quarter
    }).mean(keep_attrs=keep_attrs)
    # Set offset for supported array formats
    if isinstance(means.indexes[time_coord_name],
                  xr.coding.cftimeindex.CFTimeIndex):
        means[time_coord_name] = means.indexes[
            time_coord_name] + xr.coding.cftime_offsets.to_offset(freq="MS")
    elif isinstance(means.indexes[time_coord_name], pd.DatetimeIndex):
        means[time_coord_name] = means.indexes[
            time_coord_name] + pd.tseries.frequencies.to_offset(freq="MS")
    else:
        raise ValueError(
            f"unsupported array type - {type(means.indexes[time_coord_name])}. Valid types include: (xr.coding.cftimeindex.CFTimeIndex, pandas.core.indexes.datetimes.DatetimeIndex)"
        )
@cyschneck cyschneck added bug Something isn't working support Support request opened by outside user/collaborator labels Nov 3, 2023
@kafitzgerald
Copy link
Contributor

Thanks for logging this!

This was before both of our times, but I'm pretty sure this was a result of us (likely unintentionally) using private API from xarray that then broke and not necessarily a change to it being private API.

Hopefully adding better upstream testing helps us from digging a hole this deep in the future and noting it here provides some documentation 🤞🏻 .

A bit more on the resample/loffset issue... This is technically a bug as noted in pydata/xarray#8399 (since the loffset parameter hasn't been deprecated from xarray yet). I'll see if I can get a fix in for that, but given that it's deprecated I think the proposed solution in #492 is probably a better solution anyway for the time being. We should keep an eye on pydata/xarray#5687 though for a better long term solution or maybe put in a PR for that.

@kafitzgerald kafitzgerald removed bug Something isn't working support Support request opened by outside user/collaborator labels Nov 3, 2023
@cyschneck cyschneck added bug Something isn't working blocked Work got blocked waiting the output of some other source/work labels Nov 3, 2023
@kafitzgerald kafitzgerald added blocked Work got blocked waiting the output of some other source/work and removed blocked Work got blocked waiting the output of some other source/work labels Nov 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blocked Work got blocked waiting the output of some other source/work bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants