update api docs

pypsa-meets-earth · Mar 13, 2024 · 767183f · 767183f
1 parent 2ccbcf2
commit 767183f
Show file tree

Hide file tree

Showing 10 changed files with 349 additions and 199 deletions.
diff --git a/Makefile b/Makefile
@@ -109,6 +109,6 @@ docs: api-docs		## Build the documentation.
 	URL="site/index.html"; xdg-open $$URL || sensible-browser $$URL || x-www-browser $$URL || gnome-open $$URL
 
 .PHONY: docs-gh-deploy
-docs-deploy: api-docs		## Serve the documentation.
+docs-gh-deploy: api-docs		## Serve the documentation.
 	@echo "deploying documentation on github ..."
 	@$(ENV_PREFIX)mkdocs gh-deploy --force
diff --git a/docs/api-docs/README.md b/docs/api-docs/README.md
@@ -7,26 +7,31 @@
 - [`args`](./args.md#module-args)
 - [`config`](./config.md#module-config)
 - [`eo`](./eo.md#module-eo)
+- [`export`](./export.md#module-export)
 - [`extract`](./extract.md#module-extract)
 - [`filter`](./filter.md#module-filter)
 - [`gfk_data`](./gfk_data.md#module-gfk_data)
 - [`gfk_download`](./gfk_download.md#module-gfk_download)
 - [`osmpbf`](./osmpbf.md#module-osmpbf)
+- [`tagdata`](./tagdata.md#module-tagdata)
+- [`taginfo`](./taginfo.md#module-taginfo)
 - [`utils`](./utils.md#module-utils)
 
 ## Classes
 
+- [`export.OutFileWriter`](./export.md#class-outfilewriter)
 - [`osmpbf.Node`](./osmpbf.md#class-node): Node(id, tags, lonlat)
 - [`osmpbf.Relation`](./osmpbf.md#class-relation): Relation(id, tags, members)
 - [`osmpbf.Way`](./osmpbf.md#class-way): Way(id, tags, refs)
-- [`utils.OutFileWriter`](./utils.md#class-outfilewriter)
 
 ## Functions
 
 - [`args.main`](./args.md#function-main): The main function executes on commands:
 - [`eo.get_osm_data`](./eo.md#function-get_osm_data)
 - [`eo.process_region`](./eo.md#function-process_region): Process Country
 - [`eo.save_osm_data`](./eo.md#function-save_osm_data): Get OSM Data for a list of regions and features
+- [`export.convert_pd_to_gdf`](./export.md#function-convert_pd_to_gdf)
+- [`export.get_list_slug`](./export.md#function-get_list_slug)
 - [`extract.filter_file_block`](./extract.md#function-filter_file_block)
 - [`extract.filter_pbf`](./extract.md#function-filter_pbf): Parallized pre-Filtering of OSM file by a pre_filter
 - [`extract.id_filter`](./extract.md#function-id_filter)
@@ -46,19 +51,25 @@
 - [`gfk_data.get_region_dict`](./gfk_data.md#function-get_region_dict): Takes a region id (eg. germany) and returns a ditctionary consisting of
 - [`gfk_data.get_region_tuple`](./gfk_data.md#function-get_region_tuple): Takes a region id or code (eg. DE, germany) and returns a named tuple with 
 - [`gfk_data.get_root_list`](./gfk_data.md#function-get_root_list): Returns a list of regions without parents (i.e continents)
+- [`gfk_download.calculate_md5`](./gfk_download.md#function-calculate_md5)
+- [`gfk_download.download_file`](./gfk_download.md#function-download_file): Download file from url to dir
 - [`gfk_download.download_pbf`](./gfk_download.md#function-download_pbf)
 - [`gfk_download.download_sitemap`](./gfk_download.md#function-download_sitemap)
-- [`gfk_download.earth_downloader`](./gfk_download.md#function-earth_downloader): Download file from url to dir
+- [`gfk_download.verify_pbf`](./gfk_download.md#function-verify_pbf)
+- [`tagdata.get_feature_list`](./tagdata.md#function-get_feature_list)
+- [`tagdata.get_popular_features`](./tagdata.md#function-get_popular_features)
+- [`tagdata.get_primary_list`](./tagdata.md#function-get_primary_list)
+- [`tagdata.load_tag_data`](./tagdata.md#function-load_tag_data)
+- [`taginfo.fetch_all_data`](./taginfo.md#function-fetch_all_data)
+- [`taginfo.fetch_data_from_api`](./taginfo.md#function-fetch_data_from_api)
+- [`taginfo.get_data`](./taginfo.md#function-get_data)
+- [`taginfo.get_key_overview`](./taginfo.md#function-get_key_overview)
+- [`taginfo.get_tag_data`](./taginfo.md#function-get_tag_data)
+- [`taginfo.get_wiki_features`](./taginfo.md#function-get_wiki_features)
+- [`taginfo.get_wiki_features_df`](./taginfo.md#function-get_wiki_features_df)
+- [`taginfo.save_data`](./taginfo.md#function-save_data)
 - [`utils.columns_melt`](./utils.md#function-columns_melt)
-- [`utils.convert_pd_to_gdf`](./utils.md#function-convert_pd_to_gdf)
-- [`utils.convert_ways_lines`](./utils.md#function-convert_ways_lines): Convert Ways to Line Coordinates
-- [`utils.convert_ways_points`](./utils.md#function-convert_ways_points): Convert Ways to Point Coordinates
-- [`utils.convert_ways_polygons`](./utils.md#function-convert_ways_polygons): Convert Ways to Polygon and Point Coordinates
-- [`utils.get_list_slug`](./utils.md#function-get_list_slug)
 - [`utils.lonlat_lookup`](./utils.md#function-lonlat_lookup): Lookup refs and convert to list of longlats
-- [`utils.output_creation`](./utils.md#function-output_creation): Save Dataframe to disk
 - [`utils.tags_explode`](./utils.md#function-tags_explode)
 - [`utils.tags_melt`](./utils.md#function-tags_melt)
 - [`utils.way_or_area`](./utils.md#function-way_or_area)
-- [`utils.write_csv`](./utils.md#function-write_csv): Create csv file. Optimized for large files as write on disk in chunks
-- [`utils.write_geojson`](./utils.md#function-write_geojson): Create geojson file. Optimized for large files as write on disk in chunks
diff --git a/docs/api-docs/args.md b/docs/api-docs/args.md
@@ -7,9 +7,6 @@
 
 
 
-**Global Variables**
----------------
-- **primary_feature_element**
 
 ---
 

diff --git a/docs/api-docs/eo.md b/docs/api-docs/eo.md
@@ -10,7 +10,7 @@
 
 ---
 
-<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/eo.py#L27"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/eo.py#L29"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ## <kbd>function</kbd> `process_region`
 
@@ -38,12 +38,12 @@ Process Country
 
 ---
 
-<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/eo.py#L88"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/eo.py#L94"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ## <kbd>function</kbd> `get_osm_data`
 
 ```python
-get_osm_data(region_str, primary_name, feature_name, cached, data_dir)
+get_osm_data(region_str, primary_name, feature_name, data_dir=None, cached=True)
 ```
 
 
@@ -53,18 +53,19 @@ get_osm_data(region_str, primary_name, feature_name, cached, data_dir)
 
 ---
 
-<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/eo.py#L122"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/eo.py#L129"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ## <kbd>function</kbd> `save_osm_data`
 
 ```python
 save_osm_data(
-    region_list=['germany'],
-    primary_name='power',
-    feature_list=['tower'],
+    region_list,
+    primary_name,
+    feature_list=None,
     update=False,
     mp=True,
     data_dir='/home/matin/Projects/earth-osm/earth_data',
+    out_dir='/home/matin/Projects/earth-osm/earth_data',
     out_format='csv',
     out_aggregate=True
 )

diff --git a/docs/api-docs/export.md b/docs/api-docs/export.md
@@ -0,0 +1,65 @@
+<!-- markdownlint-disable -->
+
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/export.py#L0"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+# <kbd>module</kbd> `export`
+
+
+
+
+
+---
+
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/export.py#L26"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+## <kbd>function</kbd> `get_list_slug`
+
+```python
+get_list_slug(str_list)
+```
+
+
+
+
+
+
+---
+
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/export.py#L39"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+## <kbd>function</kbd> `convert_pd_to_gdf`
+
+```python
+convert_pd_to_gdf(pd_df)
+```
+
+
+
+
+
+
+---
+
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/export.py#L67"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+## <kbd>class</kbd> `OutFileWriter`
+
+
+
+
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/export.py#L69"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+### <kbd>method</kbd> `__init__`
+
+```python
+__init__(region_list, primary_name, feature_list, data_dir, out_format)
+```
+
+
+
+
+
+
+
+
+
diff --git a/docs/api-docs/filter.md b/docs/api-docs/filter.md
@@ -7,13 +7,10 @@
 
 
 
-**Global Variables**
----------------
-- **primary_feature_element**
 
 ---
 
-<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/filter.py#L27"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/filter.py#L26"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ## <kbd>function</kbd> `feature_filter`
 
@@ -28,7 +25,7 @@ feature_filter(primary_data, filter_tuple=('power', 'line'))
 
 ---
 
-<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/filter.py#L40"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/filter.py#L39"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ## <kbd>function</kbd> `run_feature_filter`
 
@@ -43,7 +40,7 @@ run_feature_filter(primary_dict, feature_name)
 
 ---
 
-<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/filter.py#L59"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/filter.py#L62"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ## <kbd>function</kbd> `run_primary_filter`
 
@@ -58,7 +55,7 @@ run_primary_filter(PBF_inputfile, primary_file, primary_name, multiprocess)
 
 ---
 
-<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/filter.py#L93"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/filter.py#L96"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ## <kbd>function</kbd> `get_filtered_data`
 

diff --git a/docs/api-docs/gfk_download.md b/docs/api-docs/gfk_download.md
@@ -10,12 +10,12 @@
 
 ---
 
-<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/gfk_download.py#L26"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/gfk_download.py#L28"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
-## <kbd>function</kbd> `earth_downloader`
+## <kbd>function</kbd> `download_file`
 
 ```python
-earth_downloader(url, dir, exists_ok=False)
+download_file(url, dir, exists_ok=False)
 ```
 
 Download file from url to dir 
@@ -26,6 +26,7 @@ Download file from url to dir
 
  - <b>`url`</b> (str):  url to download 
  - <b>`dir`</b> (str):  directory to download to 
+ - <b>`exists_ok`</b> (bool):  Flag to allow skipping download if file exists. 
 
 
 
@@ -36,7 +37,22 @@ Download file from url to dir
 
 ---
 
-<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/gfk_download.py#L61"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/gfk_download.py#L63"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+## <kbd>function</kbd> `download_sitemap`
+
+```python
+download_sitemap(geom, pkg_data_dir)
+```
+
+
+
+
+
+
+---
+
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/gfk_download.py#L73"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
 ## <kbd>function</kbd> `download_pbf`
 
@@ -51,12 +67,27 @@ download_pbf(url, update, data_dir)
 
 ---
 
-<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/gfk_download.py#L79"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/gfk_download.py#L97"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
 
-## <kbd>function</kbd> `download_sitemap`
+## <kbd>function</kbd> `calculate_md5`
 
 ```python
-download_sitemap(geom, pkg_data_dir)
+calculate_md5(fname)
+```
+
+
+
+
+
+
+---
+
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/gfk_download.py#L104"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+## <kbd>function</kbd> `verify_pbf`
+
+```python
+verify_pbf(pbf_inputfile, pbf_md5file)
 ```
 
 

diff --git a/docs/api-docs/tagdata.md b/docs/api-docs/tagdata.md
@@ -0,0 +1,73 @@
+<!-- markdownlint-disable -->
+
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/tagdata.py#L0"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+# <kbd>module</kbd> `tagdata`
+
+
+
+
+**Global Variables**
+---------------
+- **tag_data**
+
+---
+
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/tagdata.py#L13"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+## <kbd>function</kbd> `get_feature_list`
+
+```python
+get_feature_list(primary_name)
+```
+
+
+
+
+
+
+---
+
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/tagdata.py#L16"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+## <kbd>function</kbd> `get_primary_list`
+
+```python
+get_primary_list()
+```
+
+
+
+
+
+
+---
+
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/tagdata.py#L19"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+## <kbd>function</kbd> `load_tag_data`
+
+```python
+load_tag_data()
+```
+
+
+
+
+
+
+---
+
+<a href="https://github.com/pypsa-meets-earth/earth-osm/blob/main/earth_osm/tagdata.py#L33"><img align="right" style="float:right;" src="https://img.shields.io/badge/-source-cccccc?style=flat-square"></a>
+
+## <kbd>function</kbd> `get_popular_features`
+
+```python
+get_popular_features(primary_name)
+```
+
+
+
+
+
+