Getting Started Guide

Prerequisites

To run development modules etc., users will need to have installed R, Rstudio and SpaDES.

Install R

Windows

1. Download and run the R installer from https://cran.r-project.org/bin/windows/base.
2. Download and install the latest Rtools from http://cran.r-project.org/bin/windows/Rtools/.

macOS

1. Download and run the R installer from https://cran.r-project.org/bin/macosx.

2. Install Xcode commandline tools from the terminal:

   xcode-select install

3. Install homebrew from the terminal (instructions at https://brew.sh/)

4. Install GDAL and other spatial packages (instructions at http://www.alexchubaty.com/post/2016-12-13-using-latest-gdal-macos/)

5. Install OpenMP support for data.table:

   brew update && brew install llvm

6. Install GLPK (needed for full igraph functionality; used with SpaDES.core::moduleGraph):

   brew install glpk

   Then, reinstall igraph from source so it is compiled with GLPK:

   install.packages("igraph", type = "source")

Linux

Install R (r-base) and various dependency packages via apt or yum depending on your distribution.

Install Rstudio

Download and run the installer for your operating system from https://www.rstudio.com/products/rstudio/download/#download.

Install SpaDES and other packages

1. Update package versions and install the SpaDES package via CRAN:

   # Restart your R session so it is clear (Ctrl-Shift-F10 in Rstudio)
   
   # If you have any of our packages or their dependencies, please update them first
   # Get latest versions of key SpaDES packages from CRAN
   deps <- tools::package_dependencies("SpaDES", recursive = TRUE)
   update.packages(oldPkgs = unlist(deps), ask = FALSE) 
   
   # install the latest version of the SpaDES packages, including "Suggested"
   install.packages("SpaDES", dependencies = TRUE) 

   Since SpaDES is still a package in the early stages of maturity, the development branch on GitHub may contain useful bug fixes that are not in the CRAN version. To install this development version:

   library("devtools")
   install_github("PredictiveEcology/SpaDES@development")
   install_github("PredictiveEcology/SpaDES.shiny@development") ## optional

2. Using the instructions above, devtools::install_github() will automatically try to install the optional fastshp package. This additional package requires OS development tools (e.g., Rtools for Windows). If the above installation doesn't work for you, be sure to install the necessary development tools before reinstalling the package.

   install.packages("fastshp", repos = "https://rforge.net", type = "source")

Getting Started with SpaDES

Attach the SpaDES package in your R session using:

library(SpaDES)

Set your default working directories

Simulations make use of several working directories:

1. an inputs directory, inputPath, where SpaDES looks to find simulation inputs;
2. an outputs directory, outputPath, where simulation outputs are saved;
3. a cache directory, cachePath, where simulation outputs are cached;
4. a modules directory, modulePath, where modules and their data are downloaded and saved.

Unless otherwise specified during simInit (by passing a paths argument), the default working directories are set via options. Unless these options are changed by the user, the temporary locations are used.

To configure the location of these working directories:

## use 'setPaths' to quickly set all paths to a default location
setPaths() ## set all paths to defaults

## alternatively, custom paths can be set as arguments to 'setPaths'
setPaths(inputPath = 'path/to/my/inputs') ## set custom inputPath; all others set to defaults
setPaths(inputPath = 'path/to/my/inputs', outputPath = 'path/to/my/outputs',
         cachePath = 'path/to/my/cache', modulePath = 'path/to/my/modules') ## set all paths custom

## or by chaging the global options directly
options(spades.inputPath = 'path/to/my/inputs')
options(spades.outputPath = 'path/to/my/outputs')
options(spades.cachePath = 'path/to/my/cache')
options(spades.modulePath = 'path/to/my/modules')

Remember that once custom paths are set by the user, calling setPaths will reset the the directories to default. So if custom set paths have to be called, use

myinputPaths <- getPaths()$inputPath

Using pre-existing modules

1. Browse locally available modules:

   openModules(path = "/path/to/my/modules") # opens all modules in a directory
   openModules("moduleName", "/path/to/my/modules") # opens only the named module

2. Browse modules at https://github.com/PredictiveEcology/SpaDES-modules

3. Download modules for use:

   downloadModule("moduleName", path = "/path/to/my/modules", data = TRUE)
   openModules("moduleName", "/path/to/my/modules")

   If no path is specified, modules and data will be downloaded and saved in the location returned by getOption('spades.modulePath'). See above to change this default location.

   Try the LCC2005 module tutorial to see SpaDES at work.

Creating new modules

1. Create an empty module template:

   newModule("moduleName", path = "/path/to/my/modules")

2. Read the modules vignette for more details.

Module development checklist

Metadata

• are module metadata fully and correctly specified (module description, authorship and citation info, parameters and inputs/outputs, etc.)?
• citation should specify how to cite the module, or if published, the paper that describes the module.
• module object dependencies: use moduleDiagram and objectDiagram to confirm how data objects are passed among modules.

Events

• are all event types defined in doEvent?
• use function(sim) to access event functions from within a module: functions calls are correctly namespaced (i.e., it looks first inside the functions built in the module)
• use sim$object to access and make "global" data objects, shared among events and modules
• use `sim$$object to access and make module-specific functions, not intended to be shared with other modules
• use e.g., sim[[globals(sim)$objectName]] to access variable-named objects

Documentation

• have you provided useful (meaningful) documentation in the module's .Rmd file and README?
• have you built (knitted) the .Rmd file to generate a .pdf or .html version?
• have you specified the terms under which your module code can be reused and/or modified? Add a license!

Data

• we suggest that data you wish to include with your module are saved in data/; this makes modules more easily shareable with other people. Access those data with dataPath(sim)
• verify that external data sources are included in the sourceURL metadata field
• verify that any additional data preparation/transformation steps used in .inputObjects are correct; SpaDES.tools::prepInputs( ) may be very useful
• write CHECKSUMS.txt file for all data using checksums(..., write = TRUE)

Distributing your module

• where will your module code/data be hosted? Currently Google Drive and Dropbox appear to be easy places which can be private or public, and can now be easily accessed with googledrive and rdrop2 packages
• test downloadModule and downloadData from a temp dir to ensure your module can be downloaded correctly by others

Strategies for module development

Speed considerations

Since modules will often have to run many, many times because of replication, there are a few strategies that should be followed:

• Always write fast code. This likely means using data.table (usually fastest) or dplyr (not quite as fast) for data and data wrangling.

  • Avoid data.frame if possible.
  • Matrices and vectors are generalyl fastest, if they provide the necessary features.
  • Avoid loops.

• Use memoise or reproducible package to Cache functions for speed.

• For computationally intensive functions, consider writing them in C++, via the Rcpp package.

• For large (out of RAM) situations, use ff or bigMemory. Sometimes, these can be done seamlessly inside functions using the getOption("spades.lowMemory"), where two alternatives a provided, one "in Memory" the other "on disk". See "if (lowMemory)" code block about 20 lines from start of spread function for one way to do this with ff.

Other best practices

• Don't write modules that depend internally on other modules. Instead, pass data via the inputObjects and outputObjects in the metadata. This means avoid scheduling one event in module A from module B, if possible.
• Use and push publicly sharable modules from and to the SpaDES-Modules repository (https://github.com/PredictiveEcology/SpaDES-modules) using downloadModule() or via pull request.

Types of modules

The concept of a "module" can be very broadly defined, i.e., what a particular module does can vary widely. The only components that must exist are the metadata and the init event. This means that many, many types of modules can be written. As we slowly build a SpaDES ecosystem of modules designed to be used and re-used, we can consider writing our entire work flow -- raw data, data wrangling, data analysis, calibration of simulation model, simulation, output analysis, decision support -- all in one chain. We can cache everything along the way, so that if something must run again, but its inputs are identical to a previous run, then it can just read from disk.

This is an evolving list of types of modules that would be useful to have in this "re-use" cycle:

• dynamic forecasting

  • "classical" simulation models
  • NetLogo-type models
  • SELES-type models
  • time is a component of the model

• static forecasting

  • e.g., predict methods from statistical outputs

• agent based models

  • animals, plants
  • processes, such as fire

• raster models

  • e.g., forest succession, cellular automata

• statistical

  • Bayesian

• calibration and optimization

  • taking outputs from other modules and rescheduling those other modules again, iterating through a heuristic optimization

• translators

  • from one data type to another to allow two different modules to talk

• GIS

  • reprojection, crop, mask etc.

• data fetching

  • modules that go to specific web resources (e.g., Dryad etc.)

• data manipulation

  • simplifying, joining etc.
  • interpolators

• output analysis

  • e.g., takes time series of rasters and visualizes them

• quality scanning - e.g., from external databases

Current modules on the SpaDES-Modules repository (see above) include simple versions of dynamic forecasting (Forest Succession, fireSpreadLcc, forestAge), GIS (cropReprojectLccAge), translators (LccToBeaconsReclassify),