Skip to content

Code to use PyTorch to train a CNN to predict various outcome measures based on satellite imagery.

Notifications You must be signed in to change notification settings

aiddata/geo-cnn

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Using Convolutional Neural Networks to Predict Survey Estimates and other Outcome Measures

Stage 0 - Settings and Data Preparation

Files:

  • push.sh - shell script to push local data to HPC (script args define dir on HPC and settings JSON to use)
  • settings_builder.py - class to manage settings for different scripts
  • settings/ - folder containing settings JSONs for jobs
  • scripts/ - folder containing one off scripts for preparing survey data and anything else not a core utility used in following stages

Notes:

Stage 1 - Training CNN

Files:

  • s1_jobscripts - jobscript
  • main.py - primary script called by jobscript
  • data_prep.py - class/functions to generate sample data for training CNN
  • create_grid.py - class to generate point grid
  • runscript.py - core class/functions for running PyTorch CNN
  • load_data.py - classes for reading Landsat imagery for CNN training data and extracting NTL imagery values for training data labels
  • load_survey_data.py - class to load survey data for use with CNN predictions
  • resnet.py - modified ResNet class (based on PyTorch ResNet class)
  • vgg.py - modified VGG class (based on PyTorch VGG class) - NOT FUNCTION

Notes:

Stage 2 - Training Secondary Models

Files:

  • s2_jobscript - jobscript

  • second_stage_model.py - primary script called by jobscript

  • model_prep.py - functions and classes for building second stage models

  • merge_outputs.py - script to merge second stage model metrics

Stage 3 - Generating Predictive Surface

Files:

  • build_surface_grid.py

S3A - Creating Point Grid

S3B - Generating CNN Features for Surface Grid

S3C - Generating Secondary Model Values for Surface Grid

S4 - Building Raster Surface

Files:

  • s4_main.py -

Notes:

  • x

Usage Notes

Stage 1 Training

Stage 1 Predict

Stage 2 Train and Predict

  • running with full set of cnn features (or more; ntl, etc.) will make stage 2 take significantly longer. If testing, start with PCA instead of full feature set.
  • when using MLP Classifier model, single variable models (e.g., only NTL) will not converge

Stage 3

Stage 4

About

Code to use PyTorch to train a CNN to predict various outcome measures based on satellite imagery.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published