Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

method resolution surprise on EarthAccessFile #610

Open
itcarroll opened this issue Jun 25, 2024 · 1 comment · May be fixed by #620
Open

method resolution surprise on EarthAccessFile #610

itcarroll opened this issue Jun 25, 2024 · 1 comment · May be fixed by #620
Labels
bug Something isn't working

Comments

@itcarroll
Copy link
Collaborator

itcarroll commented Jun 25, 2024

There's something funny going on with the wrapping of fsspec.spec.AbstractBufferedFile, where the method resolution isn't coming out as expected.

The following gives me an AssertionError

import earthaccess

auth = earthaccess.login()
results = earthaccess.search_data(short_name="VIIRSJ1_L2_OC", count=1)
files = earthaccess.open(results)
with files[0] as g:
    assert g.read is getattr(g.f, "read")

I don't know about the internals, but getattr(g, "read") and g.__getattribute__("read") both follow the MRO to the read method of fsspec.spec.AbstractBufferedFile. We don't want that! We want the read method, given by g.__getattr__("read") as fsspec.implementations.http.HTTPFile.read as resulting from getattr(g.f) in the class definition.

class EarthAccessFile(fsspec.spec.AbstractBufferedFile):
def __init__(self, f: fsspec.AbstractFileSystem, granule: DataGranule) -> None:
self.f = f
self.granule = granule
def __getattr__(self, method: str) -> Any:
return getattr(self.f, method)

Is it possible that __getattr__ should be __getattribute__?

Sorry for the multiple edits; confused myself with two fs

@itcarroll itcarroll changed the title method resolution surprise on EarthAccessGranule method resolution surprise on EarthAccessFile Jun 25, 2024
@itcarroll
Copy link
Collaborator Author

No, don't use __getattribute__ b/c it leads to infinite recursion. New idea: this is not a class that should be subclassing at all.

@itcarroll itcarroll added the bug Something isn't working label Jun 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Status: 🆕 New
Development

Successfully merging a pull request may close this issue.

1 participant