Jupyter: Geospatial integration with Lonboard

[kylebarron]

👋 This is really cool and I've been a fan of both of your work for a long time!

I saw you added some geospatial support and I'm curious if there could be some benefits to integrations between our projects. I'm working on Lonboard, an interactive geospatial visualization library for Jupyter Notebooks built on (Geo)Arrow, Parquet, and deck.gl (which uses WebGL). Since both projects are very performant, I think they could be quite complementary. So for example, your taxi example is really nice, but it plots the data on a plain canvas. For many users being able to plot data on an interactive map with a basemap is really useful context.

To give an idea of the expected performance, here's 3 million points rendered from a GeoDataFrame in 3 seconds, including calculating a unique color and radius per point.

lonboard-fig1.mov

One potential area of collaboration is for cross filtering between spatial and non-spatial views, since it seems Mosaic really excels at this. Lonboard recently added support for GPU-based data filtering for numeric attributes. Geometries whose attributes are outside the selected ranges are not displayed. Here those same 3 million points (from internet speedtest data) are filtered based on download/upload speeds

Screen.Recording.2024-02-13.at.11.55.00.AM.mov

A natural integration would be through our jupyter APIs. When I looked at mosaic a few weeks ago I thought I saw that the selection state of the widget was passed back to Python as a SQL string, and I had been worried that I'd need to parse the SQL string to interpret ranges. Am I imagining that it used to be SQL-based? But with 0.6.0 it looks like widget.params now holds a dict possibly containing ranges?

So I'm thinking a compelling demo would be to set up a widget connection where expected behavior might be:

• User selects filter in mosaic view
• Mosaic's view updates based on filter
• Mosaic propagates filter parameters to Python via widget
• an observe handler sees the change in filter parameter and sets the filter on lonboard's map
• lonboard's widget sends the new filter range to the frontend
• deck.gl updates the map rendering based on that filter range

We'll have multiple copies of the data, but I think that's ok because mosaic seems quite integrated around duckdb and I'm blocked on duckdb integration pending duckdb/duckdb_spatial#153.

What do you think? Is this of any interest to you?

[jheer]

Thanks for reaching out! I'd love to explore connections between projects. (Also, I moved this out of issues and over to GitHub discussions.)

• Your initial integration idea sounds like a great start. As of Mosaic v0.5.0 we package up and expose the params and selections in a more principled way. @domoritz implemented the Python-integration bits so he can speak to that in more detail. We're also happy to consider alterations that enable better integration.
• Longer term (and assuming solid DuckDB GeoArrow/GeoParquet support), I wonder how feasible it would be to implement a dedicated Mosaic client based on lonboard. I haven't looked to see how deeply tied to Python libs the main pieces are. If it is possible to leverage DuckDB spatial for load and basic processing, and then hand-off to WebGPU in the browser, that could make for a powerful integration, and also open up use of lonboard across the web more generally.

[kylebarron]

Oops, I always forget discussions exist 🙈

I'll try to read through the docs more and come up with a demo or questions. I'm working on an air quality project at the moment and it seems like potentially an interesting crossfilter use case.

I haven't looked to see how deeply tied to Python libs the main pieces are

It's tied to Python as minimally as possible. The stack of lonboard, from lowest to highest level is:

• deck.gl in the browser for handling WebGL rendering of spatial data and synchronizing with a basemap, like Mapbox GL/Maplibre GL. deck.gl v9, currently in beta, paves the way for WebGPU support, I believe coming in v9.1.

• deck.gl has a low-level API where you can pass binary typed arrays directly and those will be copied to the GPU without CPU overhead.

• @geoarrow/deck.gl-layers is a JS project of mine that wraps the upstream deck.gl for usage with GeoArrow tables (An Arrow JS Table object where one of the columns matches the GeoArrow spec). Then a geometry array can be passed directly into the GPU.

  This is the level that some future direct Mosaic integration would use. @geoarrow/deck.gl-layers is not yet as well documented as lonboard because focusing on the Python user has been the best bang for my buck initially.

• parquet-wasm, a project of mine to load Parquet to Arrow via WebAssembly. This is useful in the context of lonboard because we can't really know whether the user's Python session is on the same machine as their browser or on some remote machine, and so I decided Parquet (with ZSTD compression) was a sane default that wouldn't give too much overhead for local users but would give huge download improvements to remote users.

• Lonboard's anywidget-based JS source. This defines the widget models to parse input Parquet data and create @geoarrow/deck.gl-layers objects

• Lonboard's Python side. This handles conversions from non-GeoArrow data sources, writing to Parquet, data validation, reprojection to WGS84, helpers for colormaps, some widget UI helpers.

So ideally once DuckDB spatial supports GeoArrow output, it will just work with @geoarrow/deck.gl-layers. You could get it to work now with some extra steps, as long as you know which column is the geometry column, but that might be less maintainable.

[jheer]

Fantastic. It sounds then like integrating a new first-class Mosaic client would be feasible. I'd be happy to help with this. I also wonder how critical the GeoArrow support is as a blocker. For example, if you put a WKB binary blob column in standard Arrow (which I believe DuckDB + spatial should already support) could the system work now?

[kylebarron]

That's awesome! Happy to help out where I can.

I also wonder how critical the GeoArrow support is as a blocker. For example, if you put a WKB binary blob column in standard Arrow (which I believe DuckDB + spatial should already support) could the system work now?

In GeoArrow there's a distinction between "serialized types", such as WKB and WKT, that provide easy interoperability with existing spatial libraries, and "native types" that use nested Arrow lists and structs directly and have O(1) access to any given coordinate.

@geoarrow/deck.gl-layers only supports the native encodings because those have the raw coordinate buffers that deck.gl can pass to the GPU directly. This means that there needs to be something that converts an Arrow WKB column to a native column.

This conversion has a decent amount of complexity because it both needs to be able to parse WKB but also generate columnar GeoArrow arrays accurately. So there isn't a pure-JS implementation today I don't think.

I believe duckdb spatial's POINT_2D, LINESTRING_2D, and POLYGON_2D types coincide with GeoArrow's native types, so you should be able to cast its Geometry type to one of those. Then Arrow output should work, it'll just be missing the metadata that declares which geometry type it represents and its coordinate reference system.

Apart from duckdb, I've been building out geometry operations on GeoArrow memory in Rust with bindings to both Python and Wasm. So geoarrow-wasm (compiled from Rust) is able to convert from WKB to native types, but I've been spending more time on the Python binding lately, so the Wasm binding isn't production stable right now, and you have the downsides of having another Wasm dependency.