Add geoviews offline features and tiles docs (#649)

Co-authored-by: Andrew Huang <ahuang@Andrews-MacBook-Pro.local>

doc/index.rst

@@ -11,9 +11,13 @@
 
 Please see the `User Guide <user_guide>`_ for further documentation.
 
+If you have any `issues <https://github.com/holoviz/geoviews/issues>`_ or wish
+to `contribute code <https://help.github.com/articles/about-pull-requests>`_., you can visit
+our `GitHub site <https://github.com/holoviz/geoviews>`_ or file a topic on
+the `HoloViz Discourse <https://discourse.holoviz.org/>`_.
 
 Installation
-------------
+============
 
 You can install GeoViews and its dependencies using conda::
 
@@ -75,16 +79,10 @@ in how non-Python packages are built between conda-forge and defaults.
 
 -----
 
-GeoViews itself is also installable using `pip`, but to do that you
-will first need to have installed the `dependencies of cartopy <http://scitools.org.uk/cartopy/docs/v0.17/installing.html#requirements>`_,
+GeoViews itself is also installable using ``pip``, but to do that you
+will first need to have installed the `dependencies of cartopy <http://scitools.org.uk/cartopy/docs/latest/installing.html#requirements>`_,
 or else have set up your system to be able to build them.
 
-
-If you have any `issues <https://github.com/holoviz/geoviews/issues>`_ or wish
-to `contribute code <https://help.github.com/articles/about-pull-requests>`_., you can visit
-our `GitHub site <https://github.com/holoviz/geoviews>`_ or file a topic on
-the `HoloViz Discourse <https://discourse.holoviz.org/>`_.
-
 .. toctree::
  :hidden:
  :maxdepth: 2

doc/user_guide/Using_Features_Offline.md

@@ -0,0 +1,114 @@
+# Using Features Offline
+
+## Creating Environment
+
+Under the hood, GeoViews features simply wrap ``cartopy`` features, so it's a matter of properly
+configuring ``cartopy`` ahead of time.
+
+1. Create a new cartopy environment (or use an existing one):
+
+ ```bash
+ conda create -n cartopy_env python=3.10
+ ```
+
+2. Install the required packages (note that `cartopy_offlinedata` is about 200MBs):
+
+ ```bash
+ conda install -c conda-forge geoviews cartopy cartopy_offlinedata
+ ```
+
+ Or if you have an environment already, you may just need [`cartopy_offlinedata`](https://anaconda.org/conda-forge/cartopy_offlinedata):
+
+ ```bash
+ conda install -c conda-forge cartopy_offlinedata
+ ```
+
+## Verifying Setup
+
+Now, we will verify that the shapefiles are available offline.
+
+1. Ensure offline shapefiles were downloaded:
+
+ ```python
+ from pathlib import Path
+ import cartopy
+
+ data_dir = Path(cartopy.config["pre_existing_data_dir"])
+ shapefiles = data_dir / "shapefiles" / "natural_earth" / "cultural"
+ list(shapefiles.glob("*"))
+ ```
+
+2. Test GeoViews offline (toggle internet off):
+
+ ```python
+ import geoviews as gv
+ from bokeh.resources import INLINE
+
+ gv.extension("bokeh")
+
+ coastline = gv.feature.coastline()
+ borders = gv.feature.borders()
+ world = (coastline * borders).opts(global_extent=True)
+
+ gv.save(world, "world.html", resources=INLINE)
+ ```
+
+ Please ensure to set [`resources=INLINE`](https://docs.bokeh.org/en/latest/docs/reference/resources.html#bokeh.resources.INLINE) if the machine you're using is completely
+ offline and you intend to view the output on that machine.
+ Failure to do so will result in the HTML file appearing empty when opened.
+
+## Changing Directory
+
+If you wish to change the default data directory, follow these steps.
+
+1. Create a new directory and move the data:
+
+ ```python
+ from pathlib import Path
+ import cartopy
+
+ new_data_dir = Path("~/.cartopy").expanduser()
+ new_data_dir.mkdir(exist_ok=True)
+
+ data_dir = Path(cartopy.config["pre_existing_data_dir"])
+ data_dir.rename(new_data_dir / "cartopy")
+ ```
+
+2. Point to the new data directory within the script:
+
+ ```python
+ from pathlib import Path
+
+ import cartopy
+ import geoviews as gv
+ from bokeh.resources import INLINE
+
+ cartopy.config["pre_existing_data_dir"] = str(Path("~/.cartopy/cartopy").expanduser())
+
+ gv.extension("bokeh")
+
+ coastline = gv.feature.coastline()
+ borders = gv.feature.borders()
+ world = (coastline * borders).opts(global_extent=True)
+
+ gv.save(world, "world.html", resources=INLINE)
+ ```
+
+3. Or set an environment variable ``CARTOPY_DATA_DIR``:
+
+ For sh:
+ ```bash
+ export CARTOPY_DATA_DIR="$HOME/.cartopy/cartopy"
+ ```
+
+ For powershell:
+ ```powershell
+ $env:CARTOPY_DATA_DIR = "$HOME/.cartopy/cartopy"
+ ```
+
+ For cmd:
+ ```cmd
+ set CARTOPY_DATA_DIR=%USERPROFILE%\.cartopy\cartopy
+ ```
+
+ Please note using tilde (``~``) in the environment variable will not work.

doc/user_guide/Using_WMTS_Offline.md

@@ -0,0 +1,149 @@
+# Using WMTS Offline
+
+## Caching the Tiles
+
+Web map tile services simply provide tiled images for a given target domain request. So to use them offline, you simply need to copy these images from their server to a preferred local mirror and point to that local mirror.
+
+However, attempting to determine the corresponding tiles to specific target domains can be a daunting task--thankfully, Cartopy provides utilities that can assist you with this task.
+
+When Cartopy is invoked for a given target domain, it retrieves and stores the relevant map tiles in a NumPy, binary file format `.npy`.
+
+```python
+from pathlib import Path
+
+import cartopy.crs as ccrs
+import cartopy.io.img_tiles as cimgt
+import numpy as np
+from PIL import Image
+from shapely import box
+
+
+def cache_tiles(
+ tile_source,
+ max_target_z=1,
+ x_bounds=(-180, 180),
+ y_bounds=(-90, 90),
+ cache_dir="tiles",
+):
+ """
+ Caches map tiles within specified bounds from a given tile source.
+
+ Args:
+ tile_source (str or cartopy.io.img_tiles.Tiles): The tile source to use for caching.
+ It can be a string specifying a built-in tile source, or an instance of a custom tile source class.
+ max_target_z (int, optional): The maximum zoom level to cache. Defaults to 1.
+ x_bounds (tuple, optional): The longitudinal bounds of the tiles to cache. Defaults to (-180, 180).
+ y_bounds (tuple, optional): The latitudinal bounds of the tiles to cache. Defaults to (-90, 90).
+ cache_dir (str, optional): The directory to store the cached tiles. Defaults to "tiles".
+
+ Returns:
+ pathlib.Path: The path to the cache directory.
+ """
+ if not isinstance(tile_source, cimgt.GoogleWTS):
+ tile_source = getattr(cimgt, tile_source)
+ tiles = tile_source(cache=cache_dir)
+
+ bbox = ccrs.GOOGLE_MERCATOR.transform_points(
+ ccrs.PlateCarree(), x=np.array(x_bounds), y=np.array(y_bounds)
+ )[:, :-1].flatten() # drop Z, then convert to x0, y0, x1, y1
+ target_domain = box(*bbox)
+
+ for target_z in range(max_target_z):
+ tiles.image_for_domain(target_domain, target_z)
+ return Path(cache_dir) / tile_source.__name__
+```
+
+As an example, to cache OpenStreetMaps tiles, you can simply call the provided function, ensuring that you specify a maximum zoom level (`max_target_z`).
+
+```python
+cache_dir = cache_tiles("OSM", max_target_z=6)
+```
+
+WARNING: When working with higher zoom levels, it is **highly recommended** to specify the `x_bounds` and `y_bounds` parameters to your region of interest. This is crucial to prevent potential issues such as rate limiting or, in extreme cases, *being banned*.
+
+As the zoom level increases, the time required for downloading and caching the tiles grows exponentially due to the increasing number of fine-grained tiles that need to be retrieved. By setting appropriate boundaries, you can effectively manage the download process and mitigate the risk of encountering problems related to excessive requests.
+
+Here is a table illustrating the number of tiles for *global extents* at different zoom levels:
+
+```
+z=0: 1 tile (entire world)
+z=1: 4 tiles
+z=2: 16 tiles
+z=3: 64 tiles
+z=4: 256 tiles
+z=5: 1,024 tiles
+z=6: 4,096 tiles
+z=7: 16,384 tiles
+z=8: 65,536 tiles
+z=9: 262,144 tiles
+z=10: 1,048,576 tiles
+z=11: 4,194,304 tiles
+z=12: 16,777,216 tiles
+z=13: 67,108,864 tiles
+z=14: 268,435,456 tiles
+z=15: 1,073,741,824 tiles
+z=16: 4,294,967,296 tiles
+z=17: 17,179,869,184 tiles
+z=18: 68,719,476,736 tiles
+z=19: 274,877,906,944 tiles
+z=20: 1,099,511,627,776 tiles
+```
+
+## Converting to PNG
+
+Since GeoViews lacks support for reading cached NumPy binary files, an additional step is required to:
+
+1. convert them to PNG format
+2. update their directories
+3. build a format string containing "{X}/{Y}/{Z}" (or similar variations)
+
+Fortunately, this process only involves a straightforward loop that performs minimal processing on each file.
+
+```python
+def convert_tiles_cache(cache_dir):
+ """
+ Converts cached tiles from numpy format to PNG format.
+
+ Args:
+ cache_dir (str): The directory containing the cached tiles in numpy format.
+
+ Returns:
+ str: The format string representing the converted PNG tiles.
+ """
+ for np_path in Path(cache_dir).rglob("*.npy"):
+ img = Image.fromarray(np.load(np_path))
+ img_path = Path(str(np_path.with_suffix(".png")).replace("_", "/"))
+ img_path.parent.mkdir(parents=True, exist_ok=True)
+ img.save(img_path)
+
+ tiles_fmt = str(cache_dir / "{X}" / "{Y}" / "{Z}.png")
+ return tiles_fmt
+```
+
+```python
+tiles_fmt = convert_tiles_cache(cache_dir)
+```
+
+## Testing Locally
+
+Now, all that's left is passing that generated tiles format string into `gv.WMTS`.
+
+```python
+import geoviews as gv
+
+gv.extension("bokeh")
+
+gv.WMTS(tiles_dir).opts(global_extent=True)
+```
+
+Please keep in mind that when reaching higher zoom levels beyond the cached max_target_z, you might encounter a blank map.
+
+To avoid this issue, it is essential to set the `max_zoom` option to the same value as `max_target_z`.
+
+```python
+import geoviews as gv
+
+gv.extension("bokeh")
+
+gv.WMTS(tiles_dir).opts(global_extent=True, max_zoom=6)
+```

doc/user_guide/index.rst

@@ -25,6 +25,11 @@ Contents:
 * `Annotating Geographic Elements <Annotators.html>`_
  Demonstrates the use of the annotate operation to annotate geographic elements.
 
+* `Using Features Offline <Using_Features_Offline.html>`_
+ Demonstrates how to use the features module offline.
+
+* `Using WMTS Offline <Using_WMTS_Offline.html>`_
+ Demonstrates how to use WMTS (web-map-tile-services) offline.
 
 .. toctree::
  :titlesonly:
@@ -38,3 +43,5 @@ Contents:
  Working with Bokeh <Working_with_Bokeh>
  Resampling Grids <Resampling_Grids>
  Annotating Geographic Elements <Annotators>
+ Using Features Offline <Using_Features_Offline>
+ Using WMTS Offline <Using_WMTS_Offline>