Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remote COG support? #75

Open
drwelby opened this issue Jul 28, 2021 · 3 comments
Open

Remote COG support? #75

drwelby opened this issue Jul 28, 2021 · 3 comments

Comments

@drwelby
Copy link

drwelby commented Jul 28, 2021

Can this read COGs via vsis3? The gdal.js examples seem to all be local files.

@ddohler ddohler changed the title COG support? Remote COG support? Jul 28, 2021
@ddohler
Copy link
Collaborator

ddohler commented Jul 28, 2021

Good question! The answer is "kind of". vsis3 support requires libcurl to be compiled into the bundle, and currently that is not done, so you can't run loam.open('/vsis3/bucket_name/path/to/cog.tif) as things stand.

Emscripten provides a lazy file wrapper that can be used to wrap a remote file, and I've tested it out and it issues ranged reads as you would hope. However, it reads in fixed 1MB chunks, so there's a lot of efficiency lost because 1MB is rarely the ideal size for reading in the appropriate parts of the COG, so it either makes too many requests, or needlessly loads extra data. For that reason, I haven't added support for remote files directly to Loam, but it should be an easy change if it would be useful for someone.

The other two options for accomplishing this are

  1. Compile in libcurl support. This would increase the size of the downloaded WebAssembly. I don't know by how much, but as it's already 2.2MB compressed, I would prefer not to take this route. It would also make the build process quite a bit more complex, and I'm not totally certain that libcurl would function properly from WebAssembly, so there is some potential risk here.
  2. Make a PR to GDAL so that it can use Emscripten's fetch API as an alternative to libcurl when being compiled by emcc. This is my preferred option because it should limit bundle size and provide the smoothest integration between gdal.js and the browser's built-in HTTP request capabilities.

Can you tell me more about your use case? This is a high-priority feature for me because lots of people are asking for it, but as you can see, the preferred solution is quite a bit of a lift, so I'm happy to help brainstorm workarounds. Alternatively, if you have the expertise necessary to implement any of these solutions I would be very happy to review a PR!

@drwelby
Copy link
Author

drwelby commented Jul 28, 2021

My use case is the standard "view a COG and metadata in the browser" use case that's handled by geotiff.js, but wasn't sure if this was something gdal.js was able to replace yet.

Your solution #2 sounds like the way to go, but unfortunately I don't have the technical ability to be any help implementing it.

@ddohler
Copy link
Collaborator

ddohler commented Jul 28, 2021

Okay, thanks! I will add this to the stack of requests for this feature. 🙂 The good news is that I'm close to finishing up the current set of features I'm working on, which mostly relate to developer experience, and this will be my next focus. It's not necessarily going to happen quickly, but I am planning to start working on this soon.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants