update

day5.qmd

@@ -76,7 +76,7 @@ plot(nb_q, coords = coords, add = TRUE, points = FALSE)
 
 Alternative approaches to form neighbourhood matrices:
 
--   based on distance
+-   based on distance, e.g. setting a distance threshold or selecting a fixed number of nearest neighbours
 -   based on triangulating points, for instance polygon centroids
 -   sphere of influence, a modification of triangulation
 -   include neighbours from neighbours
@@ -108,13 +108,31 @@ pol_pres15$I_turnout |>
 plot(pol_pres15["I_turnout"])
 ```
 
+A simple linear regression model, assuming independent observations, can be carried out using `lm`:
+
 ```{r}
 summary(pol_pres15$I_entitled_to_vote)
 (lm0 <- lm(I_turnout ~ I_entitled_to_vote, pol_pres15)) |> summary()
 pol_pres15$res = residuals(lm0)
 plot(pol_pres15["res"])
 ```
 
+A spatial linear regression model (SEM: spatial error model),
+assuming independent observations, can be carried out using `lm`:
+
+```{r}
+form = I_turnout ~ I_entitled_to_vote
+library(spatialreg)
+SEM_pres <- errorsarlm(form, data = pol_pres15, Durbin = FALSE,
+      listw = lw_q_B, zero.policy = TRUE) |> summary()
+```
+
+## Exercises
+
+1. Compare the results of the simple linear regression with the spatial error model
+2. Fit a spatial Durbin model, using `Durbin = TRUE` in the same call to `errorsarlm`
+3. carry out a likelyhood ratio test to compare both SEM models (`lmtest::lrtest()`, see the SDS book Ch 17)
+
 ## Big data: resource constraints in data science projects
 
 Constraints concern the availability of:
@@ -151,12 +169,6 @@ it's just someone else's computer!
     -   too big to fit on the hard drive, or local file storage (10 Tb)
     -   too big to move (copy) to your institution (100 Tb - Pb)
 
-::: {.callout-note title="Breakout session 1"}
-Discuss:
-
--   Have you used datasets obtained from cloud storage? For which case(s)?
--   Have you used cloud processing? For which case(s)
-:::
 
 ## R for big, tabular datasets
 
@@ -325,6 +337,11 @@ m = st_read("/vsizip/vsicurl/https://minedbuildings.z5.web.core.windows.net/lega
 
 A key paper comparing different approaches is Heaton, Matthew J., Abhirup Datta, Andrew O. Finley, Reinhard Furrer, Joseph Guinness, Rajarshi Guhaniyogi, Florian Gerber, et al. 2018. “A Case Study Competition Among Methods for Analyzing Large Spatial Data.” Journal of Agricultural, Biological and Environmental Statistics, December. [DOI](https://doi.org/10.1007/s13253-018-00348-w).
 
+## Exercises:
+
+1. Discuss: Have you used datasets obtained from cloud storage? For which case(s)?
+2. Discuss: Have you used cloud processing? For which case(s)
+
 ## If time is left
 
 -   [Exercises chapter 9](https://r-spatial.org/book/09-Large.html#exercises)

docs/day4.html

@@ -710,8 +710,7 @@
 <span><span class="co"># [201] "d_184"      "d_185"      "d_186"      "d_187"     </span></span>
 <span><span class="co"># [205] "d_188"      "d_189"      "d_190"      "d_191"     </span></span>
 <span><span class="co"># [209] "d_192"      "d_193"      "d_194"      "d_195"     </span></span>
-<span><span class="co"># [213] "d_196"      "d_197"      "d_198"      "d_199"     </span></span>
-<span><span class="co"># [217] "d_200"      "d_201"      "d_202"      "d_203"</span></span>
+<span><span class="co"># [213] "d_196"</span></span>
 <span><span class="va">rf</span> <span class="op">=</span> <span class="fu"><a href="https://rdrr.io/pkg/randomForest/man/randomForest.html">randomForest</a></span><span class="op">(</span><span class="va">NO2</span><span class="op">~</span><span class="va">.</span>, <span class="fu"><a href="https://rdrr.io/r/base/as.data.frame.html">as.data.frame</a></span><span class="op">(</span><span class="va">no2.sf</span><span class="op">)</span><span class="op">[</span><span class="fu"><a href="https://rdrr.io/r/base/c.html">c</a></span><span class="op">(</span><span class="st">"NO2"</span>, <span class="va">n</span><span class="op">[</span><span class="fu"><a href="https://rdrr.io/r/base/grep.html">grepl</a></span><span class="op">(</span><span class="st">"d_"</span>, <span class="va">n</span><span class="op">)</span><span class="op">]</span><span class="op">)</span><span class="op">]</span><span class="op">)</span></span>
 <span><span class="va">g4</span><span class="op">$</span><span class="va">rf_dm</span> <span class="op">=</span> <span class="fu"><a href="https://rdrr.io/r/stats/predict.html">predict</a></span><span class="op">(</span><span class="va">rf</span>, <span class="fu"><a href="https://rdrr.io/r/base/as.data.frame.html">as.data.frame</a></span><span class="op">(</span><span class="va">g4</span><span class="op">)</span><span class="op">)</span></span>
 <span><span class="fu"><a href="https://rdrr.io/r/graphics/plot.default.html">plot</a></span><span class="op">(</span><span class="va">g4</span><span class="op">[</span><span class="st">"rf_dm"</span><span class="op">]</span>, breaks <span class="op">=</span> <span class="st">"equal"</span>, reset <span class="op">=</span> <span class="cn">FALSE</span>, main <span class="op">=</span> <span class="st">"random forest"</span><span class="op">)</span></span>