Paper and readthedocs updates (#14)

* Paper updates

* bib modification

* figure added

* dataset readme added

* python versions added

* new reports added

* artefact version bumped

* citation added

* repo links moved

* package build updated

* action versions updated

* image paths changed

* duplicate lines removed

* error keywords removed

* data tutorial added

* pages re-mapped

* formatting changed

* troubleshooting

* duplicate setuptools removed

* test data moved

* test from temp files

* tests modified

* dataset clean-up

* Delete other/out directory

* clean up

* Delete other/datasets/iiml_2017 directory

* data handling tutorial added

.github/workflows/unit_test.yml

@@ -12,11 +12,11 @@ jobs:
         python-version: ['3.10','3.11','3.12']
     steps:
       - name: Install Python
-        uses: actions/setup-python@v4
+        uses: actions/setup-python@v5
         with:
           python-version: "3.x"        
       - name: Checkout repo
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           token: ${{ secrets.GITHUB_TOKEN }}
       - name: Install dependencies

.gitignore

@@ -7,6 +7,7 @@ other/hagen_brae/*
 other/lake_temperature/*
 other/south_greenland/*
 other/datasets/aoi/*
+other/datasets/iiml_2017/*
 other/reporting/figures/old/*
 src/griml/examples/*
 src/griml/sentinelhub_addon/*

CITATION.cff

@@ -0,0 +1,38 @@
+
+cff-version: 1.2.0
+title: GrIML
+message: >-
+  If you use this software, please cite it using the
+  metadata from this file
+type: software
+authors:
+  - given-names: Penelope
+    family-names: How
+    email: pho@geus.dk
+    affiliation: GEUS
+    orcid: 'https://orcid.org/0000-0002-8088-8497'
+  - given-names: Mads Christian
+    family-names: Lund
+    email: maclu@geus.dk
+    affiliation: GEUS
+    orcid: 'https://orcid.org/0009-0009-0446-8253'
+identifiers:
+  - type: doi
+    value: 10.XXXXX/joss.XXXXXX
+    description: JOSS software paper
+  - type: doi
+    value: 10.5281/zenodo.11395471
+    description: Zenodo DOI for releases
+repository-code: 'https://github.com/GEUS-Glaciology-and-Climate/GrIML'
+url: 'https://griml.readthedocs.io/'
+keywords:
+  - remote-sensing
+  - satellite-imagery
+  - esa
+  - cryosphere
+  - glofs
+  - ice-marginal-lakes
+  - glacial-lakes
+  - python
+  - google-earth-engine
+license: GPL-2.0

README.md

@@ -1,4 +1,4 @@
-# Investigating Greenland's ice marginal lakes under a changing climate (GrIML)
+# GrIML - Investigating Greenland's ice marginal lakes under a changing climate
 
 [![PyPI version](https://badge.fury.io/py/griml.svg)](https://badge.fury.io/py/griml) [![DOI](https://zenodo.org/badge/444752900.svg)](https://zenodo.org/badge/latestdoi/444752900) [![Documentation Status](https://readthedocs.org/projects/griml/badge/?version=latest)](https://griml.readthedocs.io/en/latest/?badge=latest) [![Build Status](https://img.shields.io/endpoint.svg?url=https%3A%2F%2Factions-badge.atrox.dev%2FPennyHow%2FGrIML%2Fbadge%3Fref%3Dmain&style=flat)](https://actions-badge.atrox.dev/PennyHow/GrIML/goto?ref=main)
 
@@ -21,11 +21,11 @@ $ pip install .
 
 ## Workflow outline
 
-<img src="https://github.com/PennyHow/GrIML/blob/main/other/reporting/figures/griml_workflow_with_gee.png?raw=true" alt="The GrIML workflow." width="1500" align="aligncenter" />
+<img src="https://github.com/GEUS-Glaciology-and-Climate/GrIML/blob/main/other/reporting/figures/griml_workflow_with_gee.png?raw=true" alt="The GrIML workflow." width="1500" align="aligncenter" />
 
 **GrIML** proposes to examine ice marginal lake changes across Greenland using a multi-sensor and multi-method remote sensing approach to better address their influence on sea level contribution forecasting.
 
-Ice marginal lakes are detected using a remote sensing approach, based on offline workflows developed within the [ESA Glaciers CCI](https://catalogue.ceda.ac.uk/uuid/7ea7540135f441369716ef867d217519") (Option 6, An Inventory of Ice-Marginal Lakes in Greenland) ([How et al., 2021](https://www.nature.com/articles/s41598-021-83509-1)). Initial classifications are performed on Google Earth Engine with the scripts available [here](https://github.com/PennyHow/GrIML/tree/main/gee_scripts). Lake extents are defined through a multi-sensor approach using:
+Ice marginal lakes are detected using a remote sensing approach, based on offline workflows developed within the [ESA Glaciers CCI](https://catalogue.ceda.ac.uk/uuid/7ea7540135f441369716ef867d217519") (Option 6, An Inventory of Ice-Marginal Lakes in Greenland) ([How et al., 2021](https://www.nature.com/articles/s41598-021-83509-1)). Initial classifications are performed on Google Earth Engine with the scripts available [here](https://github.com/GEUS-Glaciology-and-Climate/GrIML/tree/main/gee_scripts). Lake extents are defined through a multi-sensor approach using:
 
 - Multi-spectral indices classification from Sentinel-2 optical imagery
 - Backscatter classification from Sentinel-1 SAR (synthetic aperture radar) imagery

docs/README-ESA-GRIML-IML-fv1.md

@@ -0,0 +1,89 @@
+# Greenland Ice Marginal Lake Inventory annual time-series Edition 1
+
+Data available at [GEUS Dataverse](https://doi.org/10.22008/FK2/MBKW9N).
+
+Contact [pho@geus.dk](mailto:pho@geus.dk).
+
+## Table of contents
+
+- [Terms of use](#terms-of-use)
+- [Dataset Contents](#dataset-contents)
+	+ [Data format](#data-format)
+	+ [Metadata](#metadata)
+- [Acknowledgements](#acknowledgements)
+- [Relevant links](#relevant-links)
+
+## Terms of use
+
+If the data are presented or used to support results of any kind, please include references to the applicable publications:
+
+- *How, P. et al. (2024) "Greenland Ice Marginal Lake Inventory annual time-series Edition 1". GEUS Dataverse. [https://doi.org/10.22008/FK2/MBKW9N](https://doi.org/10.22008/FK2/MBKW9N)*
+- *How, P. et al. (2021) "Greenland-wide inventory of ice marginal lakes using a multi-method approach". Sci. Rep. 11, 4481. [https://doi.org/10.1038/s41598-021-83509-1](https://doi.org/10.1038/s41598-021-83509-1)*
+- *How, P. et al. "Greenland ice marginal lake inventory series from 2016 to 2023". In Prep.*
+- *How, P. "A Python package for investigating Greenland's ice marginal lakes under a changing climate". In Prep.*
+
+And include the following statement in the acknowledgments:
+
+*"Ice marginal lake data provided by the European Space Agency (ESA), and the Programme for Monitoring of the Greenland Ice Sheet (PROMICE) at the Geological Survey of Denmark and Greenland (GEUS) ([https://doi.org/10.22008/FK2/MBKW9N](https://doi.org/10.22008/FK2/MBKW9N))."*
+
+## Dataset contents
+
+This ice marginal lake dataset is a series of annual inventories, mapping the extent and presence of lakes across Greenland that share a margin with the Greenland Ice Sheet and/or the surrounding ice caps and periphery glaciers. The annual inventories provide a comprehensive record of all identified ice marginal lakes, which have been detected using three independent remote sensing techniques:
+
+- DEM sink detection using the ArcticDEM (mosaic version 3)
+- SAR backscatter classification from Sentinel-1 imagery
+- Multi-spectral indices classification from Sentinel-2 imagery
+
+All data were compiled and filtered in a semi-automated approach, using a modified version of the [MEaSUREs GIMP ice mask](https://nsidc.org/data/NSIDC-0714/versions/1) to clip the dataset to within 1 km of the ice margin. Each detected lake was then verified manually. The methodology is open-source and provided in the associated [Github repository](https://github.com/GEUS-Glaciology-and-Climate/GrIML) for full reproducibility.
+
+The inventory series was created to better understand the impact of ice marginal lake change on the future sea level budget and the terrestrial and marine landscapes of Greenland, such as its ecosystems and human activities. The dataset is a complete inventory series of Greenland, with no absent data.
+
+### Data format
+
+The detected lakes are presented as polygon vector features in shapefile format (.shp), with coordinates provided in the WGS NSIDC Sea Ice Polar Stereographic North (EPSG:3413) projected coordinate system.
+
+### Metadata
+
+Each inventory in the inventory series contains the following metadata information:
+
+| Variable name       | Description         | Format | 
+|---------------------|---------------------|---------|
+| `row_id`  	| Index identifying number for each polygon   | Integer  |
+| `lake_id` 	| Identifying number for each unique lake  	| Integer  |
+| `lake_name`| Lake placename, as defined by the [Oqaasileriffik (Language Secretariat of Greenland)](https://oqaasileriffik.gl) placename database which is distributed with [QGreenland](https://qgreenland.org/)  | String   |
+| `margin`	| Type of margin that the lake is adjacent to (`ICE_SHEET`, `ICE_CAP`)   | String |
+| `region`	| Region that lake is located, as defined by Mouginot and Rignot (2019) (`NW`, `NO`, `NE`, `CE`, `SE`, `SW`, `CW`)       	| String |
+| `area_sqkm`	| Areal extent of polygon/s in square kilometres  | Float |
+| `length_km`	| Length of polygon/s in kilometres         		| Float |
+| `temp_aver`	| Average lake surface temperature estimate (in degrees Celsius), derived from the Landsat 8/9 OLI/TIRS Collection 2 Level 2 surface temperature data product  | Float |
+| `temp_min`	| Minimum pixel lake surface temperature estimate (in degrees Celsius), derived from the Landsat 8/9 OLI/TIRS Collection 2 Level 2 surface temperature data product  | Float |
+| `temp_max`	| Maximum pixel lake surface temperature estimate (in degrees Celsius), derived from the Landsat 8/9 OLI/TIRS Collection 2 Level 2 surface temperature data product  | Float |
+| `temp_stdev`	| Average lake surface temperature estimate standard deviation, derived from the Landsat 8/9 OLI/TIRS Collection 2 Level 2 surface temperature data product  | Float |
+| `method`		| Method of classification (`DEM`, `SAR`, `VIS`)  | String |
+| `source`     | Image source of classification (`ARCTICDEM`, `S1`, `S2`)    | String  |
+| `all_src`     | List of all sources that successfully classified the lake (i.e. all classifications with the same `lake_name` value)  | String         |
+| `num_src`          | Number of sources that successfully classified the lake (`1`, `2`, `3`)     | String | 
+| `certainty`     | Certainty of classification, which is calculated from `all_src` as a score between `0` and `1`          | Float | -                             |
+| `start_date` | Start date for classification image filtering 	| String  |
+| `end_date` 	| End date for classification image filtering     | String |
+| `verified` | Flag to denote if the lake has been manually verified (`Yes`, `No`)   | String |
+| `verif_by`  | Author of verification | String  |
+| `edited`  | Flag to denote if polygon has been manually edited (`Yes`, `No`)  | String   |
+| `edited_by` | Author of manual editing   | String  |
+
+## Acknowledgements
+
+The inventory series of ice marginal lakes in Greenland has been produced as part of the European Space Agency (ESA) Living Planet Fellowship project "Examining GReenland’s Ice Marginal Lakes under a changing climate (GrIML)", which is a follow-on effort to the 2017 inventory of ice marginal lakes created under the European Space Agency (ESA) Climate Change Initiative (CCI) in Option 6 of the Glaciers_cci project (4000109873/14/I-NB). 
+
+Upkeep and continuation of the inventory series is supported by PROMICE, funded by the Geological Survey of Denmark and Greenland (GEUS) and the Danish Ministry of Climate, Energy and Utilities under the Danish Cooperation for Environment in the Arctic (DANCEA), conducted in collaboration with DTU Space (Technical University of Denmark) and Asiaq Greenland Survey.
+
+## Relevant links
+
+- The GrIML code repository on [Github](https://github.com/GEUS-Glaciology-and-Climate/GrIML) and the code readme on [readthedocs](https://griml.readthedocs.io)
+- The ESA GrIML [project outline](https://eo4society.esa.int/projects/griml/) and [fellow information](https://eo4society.esa.int/lpf/penelope-how/)
+- Information about the [ESA Living Planet Fellowship](https://eo4society.esa.int/communities/scientists/living-planet-fellowship/)
+- The GrIML [project description](https://pennyhow.github.io/blog/investigating-griml/)
+- The original 2017 ice marginal lake inventory [Scientific Reports paper](https://www.nature.com/articles/s41598-021-83509-1) and [dataset](https://catalogue.ceda.ac.uk/uuid/7ea7540135f441369716ef867d217519)
+- The [MEaSUREs GIMP ice mask](https://nsidc.org/data/NSIDC-0714/versions/1) and associated publications, [Howat (2017)](https://doi.org/10.5067/B8X58MQBFUPA) and [Howat, Negrete and Smith (2014)](https://doi.org/10.5194/tc-8-1509-2014)
+- The [Oqaasileriffik placename database](https://asiaq.maps.arcgis.com/apps/View/index.html?appid=c5c7d9d52a264980a24911d7d33914b5)
+- QGreenland v3.0.0 dataset reference, [Moon et al. (2023)](https://doi.org/10.5281/zenodo.12823307)

docs/acknowledgements.md

@@ -1,3 +1,19 @@
-# Acknowledgements
+# Acknowledgements and links
 
-This project is supported by the ESA Living Planet Fellowship (4000136382/21/I-DT-lr) entitled "Examining Greenland's Ice Marginal Lakes under a Changing Climate". Further support is provided by [PROMICE](https://promice.org], funded by the [Geological Survey of Denmark and Greenland (GEUS)](https://www.geus.dk/) and the Danish Ministry of Climate, Energy and Utilities under the Danish Cooperation for Environment in the Arctic (DANCEA). 
+## Acknowledgements
+
+The inventory series of ice marginal lakes in Greenland has been produced as part of the European Space Agency (ESA) Living Planet Fellowship project "Examining GReenland’s Ice Marginal Lakes under a changing climate (GrIML)", which is a follow-on effort to the 2017 inventory of ice marginal lakes created under the European Space Agency (ESA) Climate Change Initiative (CCI) in Option 6 of the Glaciers_cci project (4000109873/14/I-NB). 
+
+Upkeep and continuation of the inventory series is supported by [PROMICE](https://promice.org], funded by the [Geological Survey of Denmark and Greenland (GEUS)](https://www.geus.dk/) and the Danish Ministry of Climate, Energy and Utilities under the Danish Cooperation for Environment in the Arctic (DANCEA), conducted in collaboration with DTU Space (Technical University of Denmark) and Asiaq Greenland Survey.
+
+
+## Relevant links
+
+- The GrIML code repository on [Github](https://github.com/GEUS-Glaciology-and-Climate/GrIML) and the code readme on [readthedocs](https://griml.readthedocs.io)
+- The ESA GrIML [project outline](https://eo4society.esa.int/projects/griml/) and [fellow information](https://eo4society.esa.int/lpf/penelope-how/)
+- Information about the [ESA Living Planet Fellowship](https://eo4society.esa.int/communities/scientists/living-planet-fellowship/)
+- The GrIML [project description](https://pennyhow.github.io/blog/investigating-griml/)
+- The original 2017 ice marginal lake inventory [Scientific Reports paper](https://www.nature.com/articles/s41598-021-83509-1) and [dataset](https://catalogue.ceda.ac.uk/uuid/7ea7540135f441369716ef867d217519)
+- The [MEaSUREs GIMP ice mask](https://nsidc.org/data/NSIDC-0714/versions/1) and associated publications, [Howat (2017)](https://doi.org/10.5067/B8X58MQBFUPA) and [Howat, Negrete and Smith (2014)](https://doi.org/10.5194/tc-8-1509-2014)
+- The [Oqaasileriffik placename database](https://asiaq.maps.arcgis.com/apps/View/index.html?appid=c5c7d9d52a264980a24911d7d33914b5)
+- QGreenland v3.0.0 dataset reference, [Moon et al. (2023)](https://doi.org/10.5281/zenodo.12823307)

docs/background.md

@@ -4,7 +4,18 @@ The **GrIML** processing package is for classifying water bodies from satellite
 
 **Project aim:** To examine ice marginal lake changes across Greenland using a multi-method and multi-sensor remote sensing approach, refined with in situ validation.
 
-## Motivation
+## Package structure
+
+The **GrIML** post-processing chain follows a linear workflow. Initial rasterised binary classifications denoting water bodies can be inputted to **convert**, **filter** and **merge** into a cohesive ice marginal lake vector dataset, populated with useful **metadata** and analysed with relevant **statistical information**.
+
+<img src="https://raw.githubusercontent.com/GEUS-Glaciology-and-Climate/GrIML/refs/heads/main/other/reporting/figures/griml_workflow_without_gee.png?raw=true" align="center", width="400">
+
+Each of these post-processing steps is contained within GrIML's modules, and called in turn to perform the entire processing chain. The `griml()` function invokes all post-processing steps.
+
+<img src="https://raw.githubusercontent.com/GEUS-Glaciology-and-Climate/GrIML/refs/heads/main/other/reporting/figures/griml_package_structure.png?raw=true" align="center", width="400">
+
+
+## Project motivation
 
 <img src="https://media.springernature.com/full/springer-static/image/art%3A10.1038%2Fs41598-021-83509-1/MediaObjects/41598_2021_83509_Fig1_HTML.png?raw=true" align="right" width="400">
 
@@ -21,7 +32,7 @@ In 2017, 3347 ice marginal lakes were identified in Greenland along the ice marg
 3. The findings from this work will be validated against in situ observations - namely hydrological measurements and terrestrial time-lapse images - to evaluate whether the remote sensing workflow adequately captures ice marginal lake dynamics
 
 
-## Methodology
+### Methodology
 
 <img src="https://github.com/PennyHow/pennyhow.github.io/blob/master/assets/images/griml_workflow.png?raw=true" alt="The GrIML workflow." width="1500" align="aligncenter" />
 
@@ -32,12 +43,12 @@ Ice marginal lakes are detected using a remote sensing approach, based on offlin
 - Sink detection from ArcticDEM digital elevation models 
 
 
-## Reporting
+### Reporting
 
-Bi-monthly reports of GrIML's progress are available [here](https://github.com/PennyHow/GrIML/tree/main/other/reporting).
+Bi-monthly reports of GrIML's progress are available [here](https://github.com/GEUS-Glaciology-and-Climate/GrIML/tree/main/other/reporting).
 
 
-## Project links
+### Project links
 
 - ESA [project outline](https://eo4society.esa.int/projects/griml/) and [fellow information](https://eo4society.esa.int/lpf/penelope-how/)
 

docs/conf.py

@@ -9,7 +9,7 @@
 project = 'GrIML'
 copyright = '2024, Penelope How'
 author = 'Penelope How'
-release = '0.1.0'
+release = '0.1.1'
 
 # -- General configuration ---------------------------------------------------
 

docs/contributing.md

@@ -2,7 +2,7 @@
 
 ## Bug reports and enhancement requests
 
-Bug reports are essential to improving the stability and usability of the GrIML processing package. These should be raised on [GrIML’s issues forum](https://github.com/PennyHow/GrIML/issues). A complete and reproducible report is essential for bugs to be resolved easily, therefore bug reports must:
+Bug reports are essential to improving the stability and usability of the GrIML processing package. These should be raised on [GrIML’s issues forum](https://github.com/GEUS-Glaciology-and-Climate/GrIML/issues). A complete and reproducible report is essential for bugs to be resolved easily, therefore bug reports must:
 
 - Include a concise and self-contained Python snippet reproducing the problem
 - Include a description of how your GrIML configuration is set up, such as from pip install or repository cloning/forking. If installed from pip or locally built, you can find the version of GrIML you are using with the following function