python/tutorials/GEDI_Finder_Tutorial_Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "43a88b0d",
   "metadata": {},
   "source": [
    "# Spatial Querying of GEDI Version 2 Data in Python\n",
    "\n",
    "The Global Ecosystem Dynamics Investigation ([GEDI](https://lpdaac.usgs.gov/data/get-started-data/collection-overview/missions/gedi-overview/)) mission aims to characterize ecosystem structure and dynamics to enable radically improved quantification and understanding of the Earth's carbon cycle and biodiversity. The Land Processes Distributed Active Archive Center (LP DAAC) distributes the GEDI Level 1 and Level 2 Version 1 and Version 2 products. The LP DAAC created the GEDI Finder _Web Service_ to allow users to perform spatial queries of GEDI _Version 1_ L1-L2 full-orbit granules. One of the updates for GEDI _Version 2_ included additional spatial metadata that allows users to perform spatial queries via a graphical user interface (GUI) using NASA's [Earthdata Search](https://search.earthdata.nasa.gov/search) or programmatically using NASA's [Common Metadata Repository](https://cmr.earthdata.nasa.gov/search) (CMR). Another update is that each GEDI V1 full-orbit granule has been divided into 4 sub-orbit granules in V2.   \n",
    "\n",
    "### This tutorial will show how to use Python to perform a spatial query for GEDI V2 data using NASA's CMR, how to reformat the CMR response into a list of links pointing to the intersecting sub-orbit granules, and how to export the list of links as a text file. \n",
    "---\n",
    "## Use Case Example:  \n",
    "This tutorial was developed using an example use case for a current GEDI Finder user who has been using the GEDI Finder web service in Python to find intersecting GEDI L2A Version 1 full-orbit granules over the Amazon Rainforest. The user is now looking to use the same workflow to find intersecting GEDI L2A V2 sub-orbit granules. \n",
    "\n",
    "\n",
    "***    \n",
    "## Applicable Data Products: \n",
    "### This tutorial can be used to perform spatial queries on the following products:\n",
    "- **GEDI L1B Geolocated Waveform Data Global Footprint Level - [GEDI01_B.002](https://doi.org/10.5067/GEDI/GEDI01_B.002)**\n",
    "- **GEDI L2A Elevation and Height Metrics Data Global Footprint Level - [GEDI02_A.002](https://doi.org/10.5067/GEDI/GEDI02_A.002)**\n",
    "- **GEDI L2B Canopy Cover and Vertical Profile Metrics Data Global Footprint Level - [GEDI02_B.002](https://doi.org/10.5067/GEDI/GEDI02_B.002)**\n",
    "\n",
    "***  \n",
    "# Topics Covered:\n",
    "1. [**Import Packages**](#importpackages)  \n",
    "2. [**Define Function to Query CMR**](#definefunction)      \n",
    "3. [**Execute GEDI_Finder Function**](#executefunction)      \n",
    "4. [**Export Results**](#exportresults)        \n",
    "***\n",
    "# Before Starting this Tutorial:\n",
    "\n",
    "This tutorial requires a compatible Python Environment. To setup the Python environment, follow the steps in sections 1 of the [set-up instruction](https://github.com/nasa/GEDI-Data-Resources/Setup/setup_instructions.md). \n",
    " \n",
    "***\n",
    "## Source Code used to Generate this Tutorial:\n",
    "The repository containing all of the required files is located at: https://github.com/nasa/GEDI-Data-Resources  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2aa9271b",
   "metadata": {},
   "source": [
    "\n",
    "# 1. Import Packages <a id=\"importpackages\"></a>\n",
    "#### All of the packages required to execute this tutorial are included in the Python Standard Library."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "58cf1146",
   "metadata": {},
   "outputs": [],
   "source": [
    "import requests as r\n",
    "from datetime import datetime\n",
    "import os"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "693a3d94",
   "metadata": {},
   "source": [
    "# 2. Define Function to Query CMR <a id=\"definefunction\"></a>\n",
    "#### In the code cell below, define a function called `gedi_finder` that takes two *user-submitted* input values, a `product` and a `bbox`.\n",
    "#### There are three available products for this function, including 'GEDI01_B.002', 'GEDI02_A.002' and 'GEDI02_B.002'. \n",
    "#### A dictionary is set up to relate each product \"shortname.version\" to its associated \"concept_id\", which is a value used by NASA's CMR to retrieve data for a specific product. Additional information on concept ID's can be found in the [CMR Search API Documentation](https://cmr.earthdata.nasa.gov/search/site/docs/search/api.html#c-concept-id).\n",
    "\n",
    "#### The second user-submitted input value, `bbox` is a string of bounding box coordinate values (decimal degrees) in the following format:\n",
    "#### Lower Left Longitude, Lower Left Latitude, Upper Right Longitude, Upper Right Latitude (\"LLLon,LLLat,URLon,URLat\")  \n",
    "> #### Example: `'-73.65,-12.64,-47.81,9.7'`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "456d47f1",
   "metadata": {},
   "outputs": [],
   "source": [
    "def gedi_finder(product, bbox):\n",
    "    \n",
    "    # Define the base CMR granule search url, including LPDAAC provider name and max page size (2000 is the max allowed)\n",
    "    cmr = \"https://cmr.earthdata.nasa.gov/search/granules.json?pretty=true&provider=LPCLOUD&page_size=2000&concept_id=\"\n",
    "    \n",
    "    # Set up dictionary where key is GEDI shortname + version\n",
    "    concept_ids = {'GEDI02_B.002': 'C2142776747-LPCLOUD', \n",
    "                   'GEDI02_A.002': 'C2142771958-LPCLOUD', \n",
    "                   'GEDI01_B.002': 'C2142749196-LPCLOUD'}\n",
    "    \n",
    "    # CMR uses pagination for queries with more features returned than the page size\n",
    "    page = 1\n",
    "    bbox = bbox.replace(' ', '')  # remove any white spaces\n",
    "    try:\n",
    "        # Send GET request to CMR granule search endpoint w/ product concept ID, bbox & page number, format return as json\n",
    "        cmr_response = r.get(f\"{cmr}{concept_ids[product]}&bounding_box={bbox}&pageNum={page}\").json()['feed']['entry']\n",
    "        # If 2000 features are returned, move to the next page and submit another request, and append to the response\n",
    "        while len(cmr_response) % 2000 == 0:\n",
    "            page += 1\n",
    "            cmr_response += r.get(f\"{cmr}{concept_ids[product]}&bounding_box={bbox}&pageNum={page}\").json()['feed']['entry']\n",
    "        # CMR returns more info than just the download links, below use list comprehension to return a list of DP links\n",
    "        return [c['links'][0]['href'] for c in cmr_response]\n",
    "    except:\n",
    "        # If the request did not complete successfully, print out the response from CMR\n",
    "        print(r.get(f\"{cmr}{concept_ids[product]}&bounding_box={bbox.replace(' ', '')}&pageNum={page}\").json())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3d76a504",
   "metadata": {},
   "source": [
    "#### The function returns a list of links to download the intersecting GEDI sub-orbit V2 granules. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "99c60e32",
   "metadata": {},
   "source": [
    "# 3. Execute GEDI Finder Function <a id=\"executefunction\"></a>\n",
    "### Below is a demonstration of how to set the two required inputs to the `gedi_finder` function to variables."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f05ebfc6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# User-provided inputs (UPDATE FOR YOUR DESIRED PRODUCT AND BOUNDING BOX REGION OF INTEREST)\n",
    "product = 'GEDI02_B.002'           # Options include 'GEDI01_B.002', 'GEDI02_A.002', 'GEDI02_B.002'\n",
    "bbox = '-73.65,-12.64,-47.81,9.7'  # bounding box coordinates in LL Longitude, LL Latitude, UR Longitude, UR Latitude format"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "48730717",
   "metadata": {},
   "source": [
    "#### Above, the variables are defined to query the `GEDI02_B.002` product for a bounding box covering the Amazon Rainforest."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "862ccddd",
   "metadata": {},
   "source": [
    "#### Next, call the `gedi_finder` function for the desired product and bounding box region of interest defined above, and set the output to a variable."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8cc79b3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "granules = gedi_finder(product, bbox)\n",
    "print(f\"{len(granules)} {product} Version 2 granules found.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1695e6d8",
   "metadata": {},
   "source": [
    "#### Notice the print statement above will notify you how many granules intersected your bounding box for the product requested."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1bb0847d",
   "metadata": {},
   "source": [
    "# 4. Export Results <a id=\"exportresults\"></a>\n",
    "#### Below is a demonstration of how to take the `granules` list of download links for intersecting GEDI V2 granules and export as a text file. The text file will be written to your current working directory, and will be named based on the date and time that the file was created. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0f6ecff",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set up output text file name using the current datetime\n",
    "outName = f\"{product.replace('.', '_')}_GranuleList_{datetime.now().strftime('%Y%m%d%H%M%S')}.txt\"\n",
    "\n",
    "# Open file and write each granule link on a new line\n",
    "with open(outName, \"w\") as gf:\n",
    "    for g in granules:\n",
    "        gf.write(f\"{g}\\n\")\n",
    "print(f\"File containing links to intersecting {product} Version 2 data has been saved to:\\n {os.getcwd()}\\{outName}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4b1717ca",
   "metadata": {},
   "source": [
    "## Contact Info:  \n",
    "\n",
    "Email: LPDAAC@usgs.gov  \n",
    "Voice: +1-866-573-3222  \n",
    "Organization: Land Processes Distributed Active Archive Center (LP DAAC)¹  \n",
    "Website: <https://lpdaac.usgs.gov/>  \n",
    "Date last modified: 02-20-2024  \n",
    "\n",
    "¹Work performed under USGS contract G15PD00467 for NASA contract NNG14HH33I.  "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "303454083be38f4f4ded9b6a78c3d461d42e357933f26159a9c36c3a6745d9b7"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}