Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Site Image Viewer #229

Merged
merged 88 commits into from
Jul 25, 2023
Merged

Site Image Viewer #229

merged 88 commits into from
Jul 25, 2023

Conversation

BryonLewis
Copy link
Contributor

@BryonLewis BryonLewis commented Jul 6, 2023

Overview:

  • Add the capability to download S2/L8 images with a set time in between them (Default=14 days) instead of every image in this list.
  • Add downloading of images for an entire model run with status and cancelling.
  • ImageViewer with two modes. One mode is for basic viewing while in reviewing mode, and then another mode is for the annotation viewer. Added some fields to display additional information in the ImageViewer like S3 file path.
  • Vue Router installed so there is an /annotation route for the annotation mode. This also contains a new back-end endpoint to get a list of the SiteEvaluations in a ModelRun. Vue router also has a /:regionId? route to automatically zoom into a region instead of requiring using the dropdown.

Backend Changes:

  • ModelRun level generate-images and cancel-generate-images which will execute generating images for all siteEvaluations of a model-run.
    • Includes progress indicator which lists of the number of currently downloading items as well as a cancel button to cancel the process
  • Database Changes
    • SiteImage
      • image_bbox - stores coordinates of the bounding box so the image space polygons can be generated
      • image_dimensions - stores the image width/height so it's easier to convert the polygon into geoJSON for image space rendering
    • SatteliteFetction
      • configuration - foreign key to hyperparameters so we know the total number of images that are being downloaded for a model-run
  • Added the capability to download and view Landsat8 images (don't know why you would because they are very low resolution). This capability is in the individual site downloads as well as the mode-run wide download capabilities
    • This required adding the accenture version for landsat8 images so they can be downloaded.
  • Task Modifications
    • Some minor edits to get Landsat8 downloads to work
    • The set_siteobservation_images now has optional parameters:
    • dayRange - defaults to 14, but is used for S2 and L8 image to specify the gap that should be used between images. It prevents the taskf from downloading all images and will skip over the 14 day range.
    • NoDataLimit - defaults to 50, is used to skip over images in S2 and L8 that have over 50% NODATA or percentblack
  • Views Changes:
    • site_image - returns a list of images download along with all necessary corresponding data to generate a pixel-space polygon. Also includes filtering data for filtering by source, cloudcover, percent black
    • site_observation - added bbox and image_demiensions as well as aws_location to the image data included on a siteEvaluation. This is for the image renderer
    • model-run:
      • The endpoints for generating/cancelling download of a all model-run images
      • GET /ID/evaluations - returns a list of evaluations within a model run with their ID, displayName, and image counts for display in the annotation viewer.

Client Side:

  • Vue Router
    • Added Vue Router with a route for /annotation which changes the behavior of the system to only selecting one model run. That modelrun then displays a list of site evaluations and clicking on them will bring up the image viewer.
    • The App.vue has been modified to support the default route and the new Annotation route
    • Added /:region_id?/:selected? to the routes. Only region_id is connected right now. When a regionId is provided in the URL it will automatically select that region
  • Image Viewer
    • Image viewer has two modes: One is a dialog for generic viewing in the main interface. Other is for Annotation viewing and places the image viewer at the bottom of the screen.
    • Image Viewer will download the specific modelEvluation data and generate local geoJSON polygons to display on the image.
    • Provides filtering capability that can be toggled using the filter icon
  • Annotation Route
    • New behavior for model-run lists where only a single model-run can be selected
    • The model run display is condensed using the 'compact; property to provide less information
    • The Map has a compact option that will resize the map to a smaller view.
    • When selecting a model-run on the right side a new component will list the siteEvaluations (ModelRunSiteEvalList.vue)

BryonLewis and others added 30 commits June 7, 2023 15:29
Co-authored-by: Mike VanDenburgh <37340715+mvandenburgh@users.noreply.github.com>
Co-authored-by: Mike VanDenburgh <37340715+mvandenburgh@users.noreply.github.com>
Co-authored-by: Mike VanDenburgh <37340715+mvandenburgh@users.noreply.github.com>
Co-authored-by: Mike VanDenburgh <37340715+mvandenburgh@users.noreply.github.com>
Co-authored-by: Mike VanDenburgh <37340715+mvandenburgh@users.noreply.github.com>
Co-authored-by: Mike VanDenburgh <37340715+mvandenburgh@users.noreply.github.com>
Co-authored-by: Mike VanDenburgh <37340715+mvandenburgh@users.noreply.github.com>
@BryonLewis BryonLewis marked this pull request as ready for review July 20, 2023 16:10
django/src/rdwatch/models/satellite_fetching.py Outdated Show resolved Hide resolved
django/src/rdwatch/tasks/__init__.py Outdated Show resolved Hide resolved
django/src/rdwatch/tasks/__init__.py Outdated Show resolved Hide resolved
django/src/rdwatch/tasks/__init__.py Outdated Show resolved Hide resolved
django/src/rdwatch/urls.py Outdated Show resolved Hide resolved
geom_serializer = SiteObservationGeomListSerializer(geom_queryset)
output['images'] = image_serializer.data
output['geoJSON'] = geom_serializer.data
return Response(output)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use ninja here instead of DRF. I'm not opposed to merging this first if you'd prefer that, but the goal is to eventually use ninja exclusively and drop DRF as a dependency all together.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at this I thing your branch #239 should be merged first, because really these are /evaluations/images/:id and I can do the transfer to that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: I don't think I needed that merged, because really this lives under viewes.site_observations, because site_observations is technically /evaluations/:id which makes things wonderfully confusing.

BryonLewis and others added 11 commits July 24, 2023 13:14
Co-authored-by: Mike VanDenburgh <37340715+mvandenburgh@users.noreply.github.com>
Co-authored-by: Mike VanDenburgh <37340715+mvandenburgh@users.noreply.github.com>
Co-authored-by: Mike VanDenburgh <37340715+mvandenburgh@users.noreply.github.com>
Co-authored-by: Mike VanDenburgh <37340715+mvandenburgh@users.noreply.github.com>
),
status=SatelliteFetching.Status.RUNNING,
)
.annotate(count=Func(F('id'), function='Count'))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
.annotate(count=Func(F('id'), function='Count'))
.annotate(count=Count('id'))

@BryonLewis BryonLewis merged commit ae607e1 into main Jul 25, 2023
7 checks passed
@BryonLewis BryonLewis deleted the limit-image-downloads branch July 25, 2023 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants