Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MapboxTilesRenderer #1024

Open
wants to merge 13 commits into
base: main
Choose a base branch
from
Open

Commits on Oct 16, 2021

  1. Implement MapboxTilesRenderer

    Create MapboxTilesRenderer for outputting TMS tile sets as a mbtiles file (a SQLite database following the MBTiles 1.3 specification).
    hokieg3n1us committed Oct 16, 2021
    Configuration menu
    Copy the full SHA
    f1abb63 View commit details
    Browse the repository at this point in the history
  2. Undo Formatting

    hokieg3n1us committed Oct 16, 2021
    Configuration menu
    Copy the full SHA
    e508a9a View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    85fb11a View commit details
    Browse the repository at this point in the history
  4. Configuration menu
    Copy the full SHA
    c5d6d09 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2021

  1. Minor updates

    Compute min/max of span in-place instead of building a list of all values, then computing using dask. Use dimensions of data array for label based indexing, instead of hard-coded x, y labels (which forced input coordinate columns to be named x, y).
    hokieg3n1us committed Oct 18, 2021
    Configuration menu
    Copy the full SHA
    64d28dc View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2021

  1. Out-of-Core Processing

    Include capability to provide a local_cache_path. If provided, the aggregation for the super tiles will be stored locally as a NetCDF file (instead of keeping all these in memory, which eventually overflows memory at high zoom levels). This allows most of the processing to now be done completely out of core.
    hokieg3n1us committed Oct 20, 2021
    Configuration menu
    Copy the full SHA
    3b1c9d4 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2021

  1. Use netCDF4 for caching

    Use the netCDF4 library for writing/loading XArray DataArrays from cache, since it properly handles unsigned data types. Optional dependency that will raise an error if not installed when the caching feature is used.
    hokieg3n1us committed Oct 21, 2021
    Configuration menu
    Copy the full SHA
    9ff9ca6 View commit details
    Browse the repository at this point in the history

Commits on Oct 25, 2021

  1. Configuration menu
    Copy the full SHA
    6d0173b View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2021

  1. SQL consistency.

    Allow updates to an existing mbtiles file, inserting or replacing rows in the SQLite database and make SQL statements consistent.
    hokieg3n1us committed Oct 26, 2021
    Configuration menu
    Copy the full SHA
    b6f3e80 View commit details
    Browse the repository at this point in the history

Commits on Jan 21, 2022

  1. Memory Overhead Tradeoff

    calculate_zoom_level_stats moved to a parallel implementation with dask bags, that does not cache the super tiles in memory. If a local_cache_path is provided, the super tiles will be cached to disk as NetCDF files. Super tiles will then be either recomputed during the render_super_tiles function, or loaded from the local cache.
    
    Note: If using the Datashader render_tiles, the scheduler for Dask should be configured to 'threads' (when running on local) to avoid Dask bag computations from copying the input DataFrame to multiple processes, which will increase memory overhead. If outputting tiles to the MBTiles format, the num_workers should be tuned to prevent the SQLite database from locking during transactions. Both of these can be configured using dask.config.set(scheduler='threads', num_workers=4).
    hokieg3n1us committed Jan 21, 2022
    Configuration menu
    Copy the full SHA
    9435272 View commit details
    Browse the repository at this point in the history
  2. Path handling for MBTiles output.

    Handle setup of output paths.
    hokieg3n1us committed Jan 21, 2022
    Configuration menu
    Copy the full SHA
    11bc3d6 View commit details
    Browse the repository at this point in the history

Commits on May 19, 2023

  1. Configuration menu
    Copy the full SHA
    fcaea08 View commit details
    Browse the repository at this point in the history
  2. Update tiles.py

    hokieg3n1us committed May 19, 2023
    Configuration menu
    Copy the full SHA
    a0e6f1c View commit details
    Browse the repository at this point in the history