Skip to content

Commit

Permalink
FarmVibes.AI 28.05 (#169)
Browse files Browse the repository at this point in the history
This is a sync of our repo. Here's the summary of the changes:

## Added 🎉

- New workflow to perform spectral extension. The workflow runs a model
to combine information from an RGB drone image with a Sentinel-2 product
to generate a high-resolution (0.125m/px) image with 8 Sentinel-2 bands.
- New notebook showcasing the spectral extension workflow. The user may
provide their own UAV image in order to generate the image.
- Two automatic segmentation workflows based on SAM ([Segment Anything
Model](https://segment-anything.com/)). These will automatically segment
entities in Sentinel-2 rasters (`farm_ai/segmentation/auto_segment_s2`)
and BingMaps basemaps (`farm_ai/segmentation/auto_segment_basemap`)
without requiring the user to explictly define a set of prompts (i.e.,
points and/or bounding box).
- Two notebooks showcasing the automatic segmentation workflows over
Sentinel-2 rasters and BingMaps basemaps. The notebooks explores several
of the workflow parameters, highlighting how they affect the output
segmentation masks.

## Changed ↪️

- **[💔 BREAKING CHANGE]** We created a new module `vibe_notebook` with
all utility methods that are shared by the example notebooks in
FarmVibes.AI. Previously these methods were organized in the
`notebooks/shared_nb_lib` directory, which was imported by the notebooks
using them. Because of this, most notebook environments
will need to be updated.
- **[💔 BREAKING CHANGE]** With the new segmentation workflows, we
updated `scripts/export_sam_models.py` so the exported ONNX models can
process multiple prompts in a batch. Because of this change, the
segmentation workflows (both prompt-based and automatic segmentation)
will not work with ONNX models exported prior to this release. Please,
make sure to delete the old ONNX models in the `onnx_resources` folder
in your storage, and re-run the exportation script.
- We updated `scripts/export_sam_models.py` to support adding the ONNX
models to remote clusters. The script now has a new `--cluster`
parameter to specify the cluster type (local or remote) to which the
script will export the model to.
- The workflow `data_processing/clip/clip` now allows hard clip
(explicitly clipping the raster to the input geometry) in addition to
soft clip (which would only update the geometry metadata of the raster,
without actually cropping the raster).
- We improved the post request of ADMA by accessing a skip token in the
response. This enhancement allows users to download response results in
smaller, manageable chunks.
- Both the worker and the cache services now use gRPC for internal
communication, hopefully reducing latency a bit and improving
reliability as well. This should fix #163.
- We updated the description of workflows that download Sentinel-1, as
well as the [documentation page on
secrets](https://microsoft.github.io/farmvibes-ai/docfiles/markdown/SECRETS.html),
to reflect the need for a Planetary Computer API key.

### Fixed ⚒️

- Fixed micromamba environment creation on Windows (see #164).
- Stopped logging the azurerm provider's key when creating or updating
AKS clusters.

---------

Co-authored-by: Bruno Silva <brunosilva@microsoft.com>
Co-authored-by: Eduardo Rodrigues <edrodrigues@microsoft.com>
Co-authored-by: Naga Bilwanth Gangarapu <Naga@zensa.co>
Co-authored-by: Rafael Padilha <rpadilha@microsoft.com>
Co-authored-by: Renato Luiz de Freitas Cunha
<renato.cunha@microsoft.com>
Co-authored-by: Roberto de Moura Estevão Filho <robertode@microsoft.com>

Co-authored-by: FarmVibes Release Pipeline <farmvibesaicd@microsoft.com>
Co-authored-by: Eduardo Rodrigues <edrodrigues@microsoft.com>
Co-authored-by: Naga Bilwanth Gangarapu <Naga@zensa.co>
Co-authored-by: Renato Luiz de Freitas Cunha <renato.cunha@microsoft.com>
Co-authored-by: Roberto de Moura Estevão Filho <robertode@microsoft.com>
  • Loading branch information
6 people authored May 28, 2024
1 parent 5228adf commit cb2772b
Show file tree
Hide file tree
Showing 103 changed files with 13,627 additions and 9,151 deletions.
10 changes: 5 additions & 5 deletions docs/requirements.txt
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
docutils==0.18.1
sphinx==6.1.3
sphinx_rtd_theme==1.2.0
sphinx==7.1.2
sphinx_rtd_theme==2.0.0
sphinxcontrib.mermaid==0.8.1
myst_parser==1.0.0
sphinx_autodoc_typehints==1.22
sphinxcontrib-openapi==0.8.4
myst_parser==2.0.0
sphinx_autodoc_typehints==2.0.1
sphinxcontrib-openapi==0.8.4
11 changes: 11 additions & 0 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,13 +23,24 @@
"myst_parser",
"sphinx_autodoc_typehints",
"sphinxcontrib.openapi",
"sphinx.ext.napoleon",
]

autosummary_generate = True
autodoc_member_order = "groupwise"
autodoc_typehints = "description"
autodoc_typehints_description_target = "documented"
autodoc_inherit_docstrings = False
autoclass_content = "class"
myst_heading_anchors = 3
typehints_document_rtype = False
typehints_use_rtype = False
typehints_defaults = "comma"
always_use_bars_union = True
napoleon_use_rtype = False
napoleon_google_docstring = True
napoleon_numpy_docstring = False
napoleon_include_init_with_doc = False

sys.path.insert(0, os.path.abspath("../../src"))

Expand Down
22 changes: 22 additions & 0 deletions docs/source/docfiles/markdown/NOTEBOOK_LIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,8 @@ We organize available notebooks in the following topics:
<details>
<summary> BingMaps </summary>

- [`Automatic Segmentation of BingMaps basemaps` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/basemap_automatic_segmentation.ipynb)

- [`Segment Anything Model Workflow (BingMaps basemap)` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/basemap_segmentation.ipynb)


Expand Down Expand Up @@ -152,6 +154,10 @@ We organize available notebooks in the following topics:
<details>
<summary> Remote Sensing </summary>

- [`Automatic Segmentation of BingMaps basemaps` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/basemap_automatic_segmentation.ipynb)

- [`Automatic Segmentation of Sentinel-2` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/sentinel2_automatic_segmentation.ipynb)

- [`Crop cycle detection` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/crop_cycles/crop_cycles.ipynb)

- [`Crop land segmentation (1/4) - dataset generation` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/crop_segmentation/01_dataset_generation.ipynb)
Expand Down Expand Up @@ -192,6 +198,8 @@ We organize available notebooks in the following topics:

- [`Segment Anything Model Workflow (Sentinel-2)` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/sentinel2_segmentation.ipynb)

- [`Spectral Extension` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/spectral_extension/spectral_extension.ipynb)

- [`Spectral indices` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/sentinel/spectral_indices.ipynb)

- [`Timelapse visualization` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/sentinel/timelapse_visualization.ipynb)
Expand All @@ -203,6 +211,10 @@ We organize available notebooks in the following topics:
<details>
<summary> Segment Anything Model </summary>

- [`Automatic Segmentation of BingMaps basemaps` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/basemap_automatic_segmentation.ipynb)

- [`Automatic Segmentation of Sentinel-2` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/sentinel2_automatic_segmentation.ipynb)

- [`Field boundary segmentation (SAM exploration)` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/sam_exploration.ipynb)

- [`Field-level Irrigation Classification` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/irrigation/field_level_irrigation_classification.ipynb)
Expand All @@ -220,6 +232,8 @@ We organize available notebooks in the following topics:
<details>
<summary> Sentinel </summary>

- [`Automatic Segmentation of Sentinel-2` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/sentinel2_automatic_segmentation.ipynb)

- [`Field boundary segmentation (SAM exploration)` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/sam_exploration.ipynb)

- [`Field-level spectral indices` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/sentinel/field_level_spectral_indices.ipynb)
Expand All @@ -238,6 +252,8 @@ We organize available notebooks in the following topics:

- [`Segment Anything Model Workflow (Sentinel-2)` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/sentinel2_segmentation.ipynb)

- [`Spectral Extension` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/spectral_extension/spectral_extension.ipynb)

- [`Spectral indices` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/sentinel/spectral_indices.ipynb)


Expand Down Expand Up @@ -356,6 +372,10 @@ We organize available notebooks in the following topics:

## Notebooks description

- [`Automatic Segmentation of BingMaps basemaps` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/basemap_automatic_segmentation.ipynb) : Automatically segment BingMaps basemaps with SAM workflow.

- [`Automatic Segmentation of Sentinel-2` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/sentinel2_automatic_segmentation.ipynb) : Automatically segment Sentinel-2 imagery with SAM workflow.

- [`Carbon sequestration evaluation with Microsoft Azure Data Manager for Agriculture (ADMAg) and COMET-Farm API` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/admag/azure_data_manager_for_agriculture_and_comet_farm_api_example.ipynb) : Derive carbon sequestration information with COMET-Farm API for agricultural fields, using Azure Data Manager for Agriculture to retrieve farming data.

- [`Crop cycle detection` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/crop_cycles/crop_cycles.ipynb) : Run time-series analysis over NDVI data to detect the number of crop cycles in a year for a certain region.
Expand Down Expand Up @@ -414,6 +434,8 @@ We organize available notebooks in the following topics:

- [`Segment Anything Model Workflow (Sentinel-2)` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/segment_anything/sentinel2_segmentation.ipynb) : Segment Sentinel-2 imagery with SAM workflow.

- [`Spectral Extension` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/spectral_extension/spectral_extension.ipynb) (~10 minutes) : Generate high-resolution multispectral data by combining high-resolution drone imagery and multispectral Sentinel-2 data.

- [`Spectral indices` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/sentinel/spectral_indices.ipynb) : Compute and visualize spectral indices from Sentinel-2 imagery.

- [`Timelapse visualization` 📓](https://github.com/microsoft/farmvibes-ai/blob/main/notebooks/sentinel/timelapse_visualization.ipynb) : Generate a timelapse of SpaceEye and NDVI rasters.
Expand Down
156 changes: 55 additions & 101 deletions docs/source/docfiles/markdown/REST_API.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,108 +112,62 @@ Which would generate the following json:

```json
{
"name": "SAM segmentation",
"workflow": "farm_ai/segmentation/segment_s2",
"parameters": null,
"user_input": {
"user_input": {
"type": "Feature",
"stac_version": "1.0.0",
"id": "f6465ad0-5e01-4792-ad99-a0bd240c1e7d",
"properties": {
"start_datetime": "2020-05-01T00:00:00+00:00",
"end_datetime": "2020-05-05T00:00:00+00:00",
"datetime": "2020-05-01T00:00:00Z"
},
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-119.14896203939314,
46.51578909859286
],
[
-119.14896203939314,
46.37578909859286
],
[
-119.28896203939313,
46.37578909859286
],
[
-119.28896203939313,
46.51578909859286
],
[
-119.14896203939314,
46.51578909859286
]
]
]
},
"links": [],
"assets": {},
"bbox": [
-119.28896203939313,
46.37578909859286,
-119.14896203939314,
46.51578909859286
],
"stac_extensions": [],
"terravibes_data_type": "DataVibe"
},
"prompts": {
"type": "Feature",
"stac_version": "1.0.0",
"id": "geo_734c6441-cb25-4c40-8204-6b7286f24bb9",
"properties": {
"urls": [
"/mnt/734c6441-cb25-4c40-8204-6b7286f24bb9_geometry_collection.geojson"
],
"start_datetime": "2020-05-01T00:00:00+00:00",
"end_datetime": "2020-05-05T00:00:00+00:00",
"datetime": "2020-05-01T00:00:00Z"
},
"geometry": {
"type": "Polygon",
"coordinates": [
[
[
-119.14896203939314,
46.51578909859286
],
[
-119.14896203939314,
46.37578909859286
],
[
-119.28896203939313,
46.37578909859286
],
[
-119.28896203939313,
46.51578909859286
],
[
-119.14896203939314,
46.51578909859286
]
]
]
},
"links": [],
"assets": {},
"bbox": [
-119.28896203939313,
46.37578909859286,
-119.14896203939314,
46.51578909859286
],
"stac_extensions": [],
"terravibes_data_type": "ExternalReferenceList"
'name': 'SAM segmentation',
'workflow': 'farm_ai/segmentation/segment_s2',
'parameters': None,
'user_input': {
'user_input': {
'type': 'Feature',
'stac_version': '1.0.0',
'id': 'f6465ad0-5e01-4792-ad99-a0bd240c1e7d',
'properties': {
'start_datetime': '2020-05-01T00:00:00+00:00',
'end_datetime': '2020-05-05T00:00:00+00:00',
'datetime': '2020-05-01T00:00:00Z'
},
'geometry': {'type': 'Polygon',
'coordinates': (((-119.14896203939314, 46.51578909859286),
(-119.14896203939314, 46.37578909859286),
(-119.28896203939313, 46.37578909859286),
(-119.28896203939313, 46.51578909859286),
(-119.14896203939314, 46.51578909859286)),)
},
'links': [],
'assets': {},
'bbox': [-119.28896203939313,
46.37578909859286,
-119.14896203939314,
46.51578909859286],
'stac_extensions': [],
'terravibes_data_type': 'DataVibe'
},
'prompts': {
'type': 'Feature',
'stac_version': '1.0.0',
'id': 'geo_734c6441-cb25-4c40-8204-6b7286f24bb9',
'properties': {
'urls': ['/mnt/734c6441-cb25-4c40-8204-6b7286f24bb9_geometry_collection.geojson'],
'start_datetime': '2020-05-01T00:00:00+00:00',
'end_datetime': '2020-05-05T00:00:00+00:00',
'datetime': '2020-05-01T00:00:00Z'
},
'geometry': {'type': 'Polygon',
'coordinates': (((-119.14896203939314, 46.51578909859286),
(-119.14896203939314, 46.37578909859286),
(-119.28896203939313, 46.37578909859286),
(-119.28896203939313, 46.51578909859286),
(-119.14896203939314, 46.51578909859286)),)
},
'links': [],
'assets': {},
'bbox': [-119.28896203939313,
46.37578909859286,
-119.14896203939314,
46.51578909859286],
'stac_extensions': [],
'terravibes_data_type': 'ExternalReferenceList'
}
}
}
}
```

Expand Down
15 changes: 6 additions & 9 deletions docs/source/docfiles/markdown/SECRETS.md
Original file line number Diff line number Diff line change
Expand Up @@ -82,26 +82,23 @@ farmvibes-ai <local | remote> delete-secret <key>
- **NOAA GFS SAS token** (parameter `noaa_gfs_token` with default secret key `noaa-gfs-sas`).
- `data_ingestion/weather/get_forecast`

- **SciHub username** and **password** (parameters `scihub_user` and `scihub_password`, and default secret keys `scihub-user` and `scihub-password`, respectively).
- `data_ingestion/sentinel1/preprocess_s1`

- **Planetary computer API key**. By default, FarmVibes.AI workflows access the Planetary Computer catalog anonymously, when possible. However, we recommend registering for an API key [(see more information here)](https://planetarycomputer.microsoft.com/docs/overview/about/) to avoid being throttled.
- **Planetary computer API key**. The API key is required for workflows that download Sentinel-1 data. For other workflows, the key is optional and avoids being throttled. If not provided, FarmVibes.AI access the Planetary Computer catalog anonymously. To register for an API key, [(see more information here)](https://planetarycomputer.microsoft.com/docs/overview/about/).
- `data_ingestion/dem/download_dem`
- `data_ingestion/landsat/preprocess_landsat`
- `data_ingestion/naip/download_naip`
- `data_ingestion/sentinel1/preprocess_s1`
- `data_ingestion/sentinel1/preprocess_s1` (Required)
- `data_ingestion/sentinel2/preprocess_s2`
- `data_ingestion/sentinel2/preprocess_s2_improved_mask`
- `data_ingestion/spaceeye/spaceeye`
- `data_ingestion/spaceeye/spaceeye` (Required)
- `data_ingestion/spaceeye/spaceeye_interpolation`
- `data_ingestion/spaceeye/spaceeye_preprocess`
- `data_ingestion/spaceeye/spaceeye_preprocess` (Required)
- `farm_ai/agriculture/canopy_cover`
- `farm_ai/agriculture/change_detection`
- `farm_ai/agriculture/change_detection` (Required)
- `farm_ai/agriculture/emergence_summary`
- `farm_ai/agriculture/methane_index`
- `farm_ai/agriculture/ndvi_summary`
- `farm_ai/agriculture/conservation_practices`
- `farm_ai/agriculture/landsat_ndvi_trend`
- `farm_ai/land_cover_mapping/conservation_practices`
- `ml/dataset_generation/datagen_crop_segmentation`

```{eval-rst}
Expand Down
12 changes: 9 additions & 3 deletions docs/source/docfiles/markdown/WORKFLOW_LIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -118,7 +118,7 @@ Below is a list of all available workflows within the FarmVibes.AI platform. For

- [`chunk_onnx/chunk_onnx_sequence` 📄](workflow_yaml/data_processing/chunk_onnx/chunk_onnx_sequence.md): Runs an Onnx model over all rasters in the input to produce a single raster.

- [`clip/clip` 📄](workflow_yaml/data_processing/clip/clip.md): Performs a soft clip on an input raster based on a provided reference geometry.
- [`clip/clip` 📄](workflow_yaml/data_processing/clip/clip.md): Performs a clip on an input raster based on a provided reference geometry.

- [`gradient/raster_gradient` 📄](workflow_yaml/data_processing/gradient/raster_gradient.md): Computes the gradient of each band of the input raster with a Sobel operator.

Expand Down Expand Up @@ -172,6 +172,10 @@ Below is a list of all available workflows within the FarmVibes.AI platform. For

- [`land_degradation/ndvi_linear_trend` 📄](workflow_yaml/farm_ai/land_degradation/ndvi_linear_trend.md): Computes the pixel-wise NDVI linear trend over the input raster.

- [`segmentation/auto_segment_basemap` 📄](workflow_yaml/farm_ai/segmentation/auto_segment_basemap.md): Downloads basemap with BingMaps API and runs Segment Anything Model (SAM) automatic segmentation over them.

- [`segmentation/auto_segment_s2` 📄](workflow_yaml/farm_ai/segmentation/auto_segment_s2.md): Downloads Sentinel-2 imagery and runs Segment Anything Model (SAM) automatic segmentation over them.

- [`segmentation/segment_basemap` 📄](workflow_yaml/farm_ai/segmentation/segment_basemap.md): Downloads basemap with BingMaps API and runs Segment Anything Model (SAM) over them with points and/or bounding boxes as prompts.

- [`segmentation/segment_s2` 📄](workflow_yaml/farm_ai/segmentation/segment_s2.md): Downloads Sentinel-2 imagery and runs Segment Anything Model (SAM) over them with points and/or bounding boxes as prompts.
Expand All @@ -196,8 +200,10 @@ Below is a list of all available workflows within the FarmVibes.AI platform. For

- [`driveway_detection` 📄](workflow_yaml/ml/driveway_detection.md): Detects driveways in front of houses.

- [`segment_anything/basemap_prompt_segmentation` 📄](workflow_yaml/ml/segment_anything/basemap_prompt_segmentation.md): Runs Segment Anything Model (SAM) over BingMaps basemap rasters with points and/or bounding boxes as prompts.
- [`segment_anything/automatic_segmentation` 📄](workflow_yaml/ml/segment_anything/automatic_segmentation.md): Runs a Segment Anything Model (SAM) automatic segmentation over input rasters.

- [`segment_anything/prompt_segmentation` 📄](workflow_yaml/ml/segment_anything/prompt_segmentation.md): Runs Segment Anything Model (SAM) over input rasters with points and/or bounding boxes as prompts.

- [`segment_anything/s2_prompt_segmentation` 📄](workflow_yaml/ml/segment_anything/s2_prompt_segmentation.md): Runs Segment Anything Model (SAM) over Sentinel-2 rasters with points and/or bounding boxes as prompts.
- [`spectral_extension` 📄](workflow_yaml/ml/spectral_extension.md): Generates high-resolution Sentinel-2 bands by combining UAV and Sentinel-2 data.


Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ classDiagram
}
class CarbonOffsetInfo {
}
class ChipWindow {
}
class DataSequence {
}
class DataSummaryStatistics {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ classDiagram
}
class RasterSequence {
}
class SamMaskRaster {
}
DataSequence --|> DataVibe
DataVibe --|> BaseVibe
DemProduct --|> DataVibe
Expand All @@ -60,6 +62,7 @@ classDiagram
RasterIlluminance --|> DataVibe
RasterSequence --|> DataSequence
RasterSequence --|> Raster
SamMaskRaster --|> CategoricalRaster


</div>
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Runs the SpaceEye preprocessing pipeline with an ensemble of cloud segmentation

## Parameters

- **pc_key**: Optional Planetary Computer API key.
- **pc_key**: Planetary Computer API key.

## Tasks

Expand Down Expand Up @@ -82,7 +82,7 @@ description:
s1_raster: Sentinel-1 rasters.
cloud_mask: Cloud and cloud shadow mask.
parameters:
pc_key: Optional Planetary Computer API key.
pc_key: Planetary Computer API key.


```
Loading

0 comments on commit cb2772b

Please sign in to comment.