3_exploratory_gam_analysis_ndvi.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "0ba9f552",
   "metadata": {},
   "source": [
    "# Exploratory GAM analysis NDVI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "e28a8f15",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import geopandas as gpd\n",
    "import matplotlib.pyplot as plt\n",
    "import warnings\n",
    "from pygam import GAM, LinearGAM, s, l\n",
    "from sklearn import linear_model, metrics\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score, mean_squared_error\n",
    "from PIL import Image as im"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "755b432b",
   "metadata": {},
   "source": [
    "GAM consists of 3 components:\n",
    "- Distribution from the exponential familiy\n",
    "- Link Function\n",
    "- Functional Form"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "94ef206d",
   "metadata": {},
   "source": [
    "## 1. Distribution and link function"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36713449",
   "metadata": {},
   "source": [
    "**Distribution**\n",
    "\n",
    "Specified via GAM(distribution = \"...\")\n",
    "\n",
    "- **\"normal\"**: symmetric about the mean, data near the mean are more frequent in occurrence than data far from the mean. The distribution takes a bell-curve shape. Type of continuous probability distribution for real-valued random variables, also called Gaussian distribution;\n",
    "- **\"binomial\"**: used to model the number of successes (yes/no) in a sample of size N;\n",
    "- **\"poisson\"**: expresses the distribution of a given number of events occuring in a fixed interval of time or space if these events occur with a constant mean rate and independelty of time since the last event;\n",
    "- **\"gamma\"**: is a two-parameter family of continuous probability distributions. It has a shape parameter k and scale parameter (theta); \n",
    "- **\"inverse gaussian\"**: is a two-parameter family of conitnuous probability distributions with support on (0, ∞). For the distribution 1/x when x is Gaussian. Has several properties analogous to a Gaussian ditribution."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b16cd27",
   "metadata": {},
   "source": [
    "**Link function**\n",
    "\n",
    "Specified via GAM(link = \"...\")\n",
    "\n",
    "Link functions take the distribution mean to the linear prediction. A link function in a GLM maps a non-linear relationship to a linear one, which means you can a fit a linear model to the data. Which link function you use depends upon what analysis you're performing. A link function is simply a function of the mean of the response variable Y that we use as the response instead of Y itself.\n",
    "\n",
    "- **\"identity\"**: is link function used for linear regression. An identity function maps every element in a set into itself. The linear model directly predicts the outcome. The link function for a normal distribution is also the identity function; \n",
    "- **\"logit\"**: used for logistic regression. When Y is categorical, we use the logit of Y as the response in our regression equation instead of just Y. The logit function is the natural log of the odds that Y equals one of the categories;\n",
    "- **\"inverse\"**: used for regression with gamma distributed response. Inverse link functions are used to make linear predictors map to predicted values that are on a different scale;\n",
    "- **\"log\"**: used along with poisson log-linear models for counts. A natural fit for count variables that follow Poisson or negative binomial distribution is the log link. The log link exponentiates the linear predictors. It does not log transform the outcome variable;\n",
    "- **\"inverse-squared\"**: canonical link associated with family of inverse-gaussian probability distributions."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7aa7c620",
   "metadata": {},
   "source": [
    "### 1.1. Normal distribution and identity link"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "74534c21",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>NDVI_100</th>\n",
       "      <th>Greenspace%_100</th>\n",
       "      <th>Canopy%_100</th>\n",
       "      <th>Grass%_100</th>\n",
       "      <th>Shrubs%_100</th>\n",
       "      <th>geometry</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.536433</td>\n",
       "      <td>0.414039</td>\n",
       "      <td>0.201814</td>\n",
       "      <td>0.097506</td>\n",
       "      <td>0.106576</td>\n",
       "      <td>POINT (377971.171 402346.176)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.571965</td>\n",
       "      <td>0.551457</td>\n",
       "      <td>0.108844</td>\n",
       "      <td>0.072562</td>\n",
       "      <td>0.340136</td>\n",
       "      <td>POINT (364310.461 403435.100)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.455587</td>\n",
       "      <td>0.256395</td>\n",
       "      <td>0.129252</td>\n",
       "      <td>0.092971</td>\n",
       "      <td>0.043084</td>\n",
       "      <td>POINT (369566.841 406840.228)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.816160</td>\n",
       "      <td>0.942891</td>\n",
       "      <td>0.009070</td>\n",
       "      <td>0.249433</td>\n",
       "      <td>0.068027</td>\n",
       "      <td>POINT (354567.818 412168.416)</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.526997</td>\n",
       "      <td>0.388459</td>\n",
       "      <td>0.217687</td>\n",
       "      <td>0.061224</td>\n",
       "      <td>0.111111</td>\n",
       "      <td>POINT (388526.382 403175.927)</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   NDVI_100  Greenspace%_100  Canopy%_100  Grass%_100  Shrubs%_100  \\\n",
       "0  0.536433         0.414039     0.201814    0.097506     0.106576   \n",
       "1  0.571965         0.551457     0.108844    0.072562     0.340136   \n",
       "2  0.455587         0.256395     0.129252    0.092971     0.043084   \n",
       "3  0.816160         0.942891     0.009070    0.249433     0.068027   \n",
       "4  0.526997         0.388459     0.217687    0.061224     0.111111   \n",
       "\n",
       "                        geometry  \n",
       "0  POINT (377971.171 402346.176)  \n",
       "1  POINT (364310.461 403435.100)  \n",
       "2  POINT (369566.841 406840.228)  \n",
       "3  POINT (354567.818 412168.416)  \n",
       "4  POINT (388526.382 403175.927)  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load greenspace metrics at 100m geodataframe\n",
    "gdf_100 = gpd.read_file(\"thesis_project/output/dataframes/greenspace_metrics_100_df.json\")\n",
    "gdf_100.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "20655690",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Grass, Canopy, and Shrubs as predictors\n",
    "cols = ['Grass%_100', 'Canopy%_100', 'Shrubs%_100']\n",
    "df = pd.DataFrame(gdf_100, columns=cols)\n",
    "\n",
    "# target as NDVI_100\n",
    "target = pd.DataFrame(gdf_100, columns=['NDVI_100'])\n",
    "\n",
    "# define X and Y\n",
    "#X = df\n",
    "y = target['NDVI_100']\n",
    "X = df.iloc[:, 0:3].values\n",
    "\n",
    "# split independent and dependent variables into training and test data\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "9eda8c8b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GAM                                                                                                       \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     44.9874\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -403714.1815\n",
      "Number of Samples:                         4673 AIC:                                           807520.3378\n",
      "                                                AICc:                                           807521.272\n",
      "                                                GCV:                                                0.0056\n",
      "                                                Scale:                                              0.0055\n",
      "                                                Pseudo R-Squared:                                   0.7479\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                20           15.3         1.11e-16     ***         \n",
      "s(1)                              [0.6]                20           14.3         1.11e-16     ***         \n",
      "s(2)                              [0.6]                20           15.4         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:752: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  np.fill_diagonal(Dinv, d**-1) # invert the singular values\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/1287724214.py:5: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_dist_n.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model with normal distribution of Y and identity link\n",
    "gam_100_dist_n = GAM(distribution='normal', link='identity').fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_dist_n.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee18edd3",
   "metadata": {},
   "source": [
    "Fitting a GAM model with normal distribution and identity link function to NDVI data outputs the following statistic coefficients; Effective DoF ~ 44, Log likelihood ~ -182983.3579, AIC and AICc ~ 366058 (e.g. indicating that the sample size is large enough), GCV ~ 0.0118, Scale ~ 0.0116, and Pseudo R-squared ~ 0.5514."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8ba452ac",
   "metadata": {},
   "source": [
    "### 1.2. Gamma distribution and inverse/log link"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "4b97585e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/links.py:242: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return mu ** -1.\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/links.py:258: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return lp ** -1.\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:529: RuntimeWarning: divide by zero encountered in log\n",
      "  dev = 2 * ((y - mu) / mu - np.log(y / mu))\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:752: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  np.fill_diagonal(Dinv, d**-1) # invert the singular values\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GAM                                                                                                       \n",
      "=============================================== ==========================================================\n",
      "Distribution:                         GammaDist Effective DoF:                                     42.3757\n",
      "Link Function:                      InverseLink Log Likelihood:                                       -inf\n",
      "Number of Samples:                         4673 AIC:                                                   inf\n",
      "                                                AICc:                                                  inf\n",
      "                                                GCV:                                                   inf\n",
      "                                                Scale:                                               0.012\n",
      "                                                Pseudo R-Squared:                                      nan\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                20           14.5         1.11e-16     ***         \n",
      "s(1)                              [0.6]                20           13.5         1.11e-16     ***         \n",
      "s(2)                              [0.6]                20           14.4         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:1111: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  r2['explained_deviance'] = 1. - full_d.sum()/null_d.sum()\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:1112: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  r2['McFadden'] = full_ll/null_ll\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:1113: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  r2['McFadden_adj'] = 1. - (full_ll - self.statistics_['edof'])/null_ll\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/189087857.py:6: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_dist_g.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model with gamma distribution of Y and inverse/log link\n",
    "#gam_100_dist_g = GAM(distribution='gamma', link='log').fit(X_train, y_train)\n",
    "gam_100_dist_g = GAM(distribution='gamma', link='inverse').fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_dist_g.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4bef3675",
   "metadata": {},
   "source": [
    "Fitting a GAM model with gamma distribution and inverse or log link function to NDVI data outputs statistics coefficients too small, too large or/and null. For instange, Log likelihood statistic takes -∞, AIC, AICc, and GCV take +∞, and Pseudo R-squared takes nan, which might indicate that the data we are trying to fit (e.g. NDVI) does not follow a gamma probability distribution nor that predictors map to a response variable using an inverse or log link function. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adc6a62b",
   "metadata": {},
   "source": [
    "### 1.3. Inverse gaussian distribution and inverse-squared/log link"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "f0759fd6",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/links.py:320: RuntimeWarning: invalid value encountered in power\n",
      "  return lp ** -0.5\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:639: RuntimeWarning: divide by zero encountered in true_divide\n",
      "  dev = ((y - mu)**2) / (mu**2 * y)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:752: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  np.fill_diagonal(Dinv, d**-1) # invert the singular values\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GAM                                                                                                       \n",
      "=============================================== ==========================================================\n",
      "Distribution:                      InvGaussDist Effective DoF:                                     33.5639\n",
      "Link Function:                   InvSquaredLink Log Likelihood:                                       -inf\n",
      "Number of Samples:                         4673 AIC:                                                   inf\n",
      "                                                AICc:                                                  inf\n",
      "                                                GCV:                                                   inf\n",
      "                                                Scale:                                              0.0206\n",
      "                                                Pseudo R-Squared:                                      nan\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                20           11.7         1.11e-16     ***         \n",
      "s(1)                              [0.6]                20           10.6         1.11e-16     ***         \n",
      "s(2)                              [0.6]                20           11.2         1.11e-16     ***         \n",
      "intercept                                              1            0.0          2.30e-12     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:1111: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  r2['explained_deviance'] = 1. - full_d.sum()/null_d.sum()\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:1112: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  r2['McFadden'] = full_ll/null_ll\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:1113: RuntimeWarning: invalid value encountered in double_scalars\n",
      "  r2['McFadden_adj'] = 1. - (full_ll - self.statistics_['edof'])/null_ll\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/1424524372.py:6: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_dist_i.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model with inverse gaussian distribution of Y and inverse-squared or log link\n",
    "gam_100_dist_i = GAM(distribution='inv_gauss', link='inv_squared').fit(X_train, y_train)\n",
    "#gam_100_dist_i = GAM(distribution='inv_gauss', link='log').fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_dist_i.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ee8ebd5",
   "metadata": {},
   "source": [
    "Fitting a GAM model with inverse gaussian distribution and inverse-squared or log link function to NDVI data outputs statistics coefficients too small, too large or/and null. For instange, Log likelihood statistic takes -∞, AIC, AICc, and GCV take +∞, and Pseudo R-squared takes nan, which might indicate that the data we are trying to fit (e.g. NDVI) does not follow an inverse gaussian probability distribution nor that predictors map to a response variable using an inverse-squared or log link function."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e4f95a6",
   "metadata": {},
   "source": [
    "**In Practice**\n",
    "\n",
    "In pyGAM you can build custom models by specifying these 3 elements, or you can choose from common models:\n",
    "\n",
    "- **LinearGAM**: identity link and normal distribution of response variable Y (and normal error distribution);\n",
    "- **LogisticGAM**: logit link and binomial distribution of Y;\n",
    "- **PoissonGAM**: log link and Poisson distribution of Y;\n",
    "- **GammaGAM**: log link and gamma distribution of Y;\n",
    "- **InvGauss**: log link and inv_gauss distribution of Y."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a03e75ff",
   "metadata": {},
   "source": [
    "From the previous section, we assessed that fitting a GAM model with **normal distribution and identity link function to NDVI data performed best**. So, we can use the common model **LinearGam**, which already uses a normal distribution and identity link, instead of the custom GAM and avoid larger chunks of code."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a5b47fd3",
   "metadata": {},
   "source": [
    "## 2. Functional form"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b7cf0947",
   "metadata": {},
   "source": [
    "**Functional Form**\n",
    "\n",
    "Specified in GAM(terms=\"...\") or simply GAM(...)\n",
    "\n",
    "- **l( ) linear terms**: for terms like Xi. If a term of a polynomial contains a variable with exponent 1, we call it linear term;\n",
    "- **s( ) spline terms**: are normally used when one knows that the relation between an independent variable and dependent/target variable is non-linear in nature and can be best depicted by a polynomial relation. The name of splines are related to its order. For example, cubic splines correspond to n=3 (order). Two types of spline terms; p-spline and cyclic p-spline basis, useful for building periodic functions. For controlling the shape of spline term we use two parameters, which are n_splines which is a integer term, lam, which is short for lambda, the penalization parameter.\n",
    "- **f( ) factor terms**: a factor is a vector whose elements can take on one of a specific set of values. For example, \"Sex\" will usually take on only the values \"M\" and \"F\". Also known as categorical variable;\n",
    "- **te( ) tensor products**: the tensor product of two vector spaces V and W (over the same field) is a vector space to which is associated a bilinear map that maps a pair to an element of. Used to fit interactions in pyGAM;\n",
    "- **intercept**: if intercept=True, the model includes an intercept by default. It is the point where a line crosses the x-axis or y-axis.\n",
    "\n",
    "The following section was used to test different combinations of functional forms: "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e0d9dab",
   "metadata": {},
   "source": [
    "### 2.1. NDVI ~ l(grass) + s(canopy) + s(shrubs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "92ad6d28",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     31.8858\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -403642.3362\n",
      "Number of Samples:                         4673 AIC:                                           807350.4439\n",
      "                                                AICc:                                          807350.9243\n",
      "                                                GCV:                                                0.0055\n",
      "                                                Scale:                                              0.0055\n",
      "                                                Pseudo R-Squared:                                   0.7465\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "l(0)                              [0.6]                1            1.0          1.11e-16     ***         \n",
      "s(1)                              [0.6]                20           15.4         1.11e-16     ***         \n",
      "s(2)                              [0.6]                20           15.5         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/3402436101.py:6: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_funct_1.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model: NDVI ~ l(grass) + s(canopy) + s(shrubs)\n",
    "#gam_100_funct_1 = LinearGAM(l(0) + s(1) + s(2)).gridsearch(X_train, y_train)\n",
    "gam_100_funct_1 = LinearGAM(l(0) + s(1) + s(2)).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_funct_1.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dbbee67b",
   "metadata": {},
   "source": [
    "### 2.2. NDVI ~ s(grass) + l(canopy) + s(shrubs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "39bc01ad",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     31.7169\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -393822.5079\n",
      "Number of Samples:                         4673 AIC:                                           787710.4495\n",
      "                                                AICc:                                          787710.9251\n",
      "                                                GCV:                                                0.0057\n",
      "                                                Scale:                                              0.0056\n",
      "                                                Pseudo R-Squared:                                   0.7404\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                20           15.3         1.11e-16     ***         \n",
      "l(1)                              [0.6]                1            1.0          1.11e-16     ***         \n",
      "s(2)                              [0.6]                20           15.4         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/1395283057.py:6: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_funct_2.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model: NDVI ~ s(grass) + l(canopy) + s(shrubs)\n",
    "#gam_100_funct_2 = LinearGAM(s(0) + l(1) + s(2)).gridsearch(X_train, y_train)\n",
    "gam_100_funct_2 = LinearGAM(s(0) + l(1) + s(2)).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_funct_2.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "288b27da",
   "metadata": {},
   "source": [
    "### 2.3. NDVI ~ s(grass) + s(canopy) + l(shrubs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "5f9e257c",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     30.4883\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -387476.8992\n",
      "Number of Samples:                         4673 AIC:                                            775016.775\n",
      "                                                AICc:                                          775017.2159\n",
      "                                                GCV:                                                0.0058\n",
      "                                                Scale:                                              0.0057\n",
      "                                                Pseudo R-Squared:                                   0.7361\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                20           15.2         1.11e-16     ***         \n",
      "s(1)                              [0.6]                20           14.3         1.11e-16     ***         \n",
      "l(2)                              [0.6]                1            1.0          1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/3062727570.py:6: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_funct_3.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model: NDVI ~ s(grass) + s(canopy) + l(shrubs)\n",
    "#gam_100_funct_3 = LinearGAM(s(0) + s(1) + l(2)).gridsearch(X_train, y_train)\n",
    "gam_100_funct_3 = LinearGAM(s(0) + s(1) + l(2)).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_funct_3.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c435eb7",
   "metadata": {},
   "source": [
    "### 2.4. NDVI ~ l(grass) + l(canopy) + s(shrubs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "0acbaf82",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     18.6021\n",
      "Link Function:                     IdentityLink Log Likelihood:                                -393418.731\n",
      "Number of Samples:                         4673 AIC:                                           786876.6662\n",
      "                                                AICc:                                          786876.8398\n",
      "                                                GCV:                                                0.0057\n",
      "                                                Scale:                                              0.0056\n",
      "                                                Pseudo R-Squared:                                   0.7386\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "l(0)                              [0.6]                1            1.0          1.11e-16     ***         \n",
      "l(1)                              [0.6]                1            1.0          1.11e-16     ***         \n",
      "s(2)                              [0.6]                20           16.6         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:752: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  np.fill_diagonal(Dinv, d**-1) # invert the singular values\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/3320750960.py:6: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_funct_4.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model: NDVI ~ l(grass) + l(canopy) + s(shrubs)\n",
    "#gam_100_funct_4 = LinearGAM(l(0) + l(1) + s(2)).gridsearch(X_train, y_train)\n",
    "gam_100_funct_4 = LinearGAM(l(0) + l(1) + s(2)).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_funct_4.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d6f57f27",
   "metadata": {},
   "source": [
    "### 2.5. NDVI ~ l(grass) + s(grass) + s(canopy) + s(shrubs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "ddfcbe39",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     44.9874\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -403714.1815\n",
      "Number of Samples:                         4673 AIC:                                           807520.3378\n",
      "                                                AICc:                                           807521.272\n",
      "                                                GCV:                                                0.0056\n",
      "                                                Scale:                                              0.0055\n",
      "                                                Pseudo R-Squared:                                   0.7479\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "l(0)                              [0.6]                1            1.0          1.11e-16     ***         \n",
      "s(0)                              [0.6]                20           14.3         1.11e-16     ***         \n",
      "s(1)                              [0.6]                20           14.3         1.11e-16     ***         \n",
      "s(2)                              [0.6]                20           15.4         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/518690504.py:6: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_funct_5.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model: NDVI ~ l(grass) + s(grass) + s(canopy) + s(shrubs)\n",
    "#gam_100_funct_5 = LinearGAM(l(0) + s(0) + s(1) + s(2)).gridsearch(X_train, y_train)\n",
    "gam_100_funct_5 = LinearGAM(l(0) + s(0) + s(1) + s(2)).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_funct_5.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c633b029",
   "metadata": {},
   "source": [
    "### 2.6. NDVI ~ automatic terms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "67523db7",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     44.9874\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -403714.1815\n",
      "Number of Samples:                         4673 AIC:                                           807520.3378\n",
      "                                                AICc:                                           807521.272\n",
      "                                                GCV:                                                0.0056\n",
      "                                                Scale:                                              0.0055\n",
      "                                                Pseudo R-Squared:                                   0.7479\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                20           15.3         1.11e-16     ***         \n",
      "s(1)                              [0.6]                20           14.3         1.11e-16     ***         \n",
      "s(2)                              [0.6]                20           15.4         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:752: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  np.fill_diagonal(Dinv, d**-1) # invert the singular values\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/430702205.py:6: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_funct_6.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model: NDVI ~ automatic terms\n",
    "#gam_100_funct_6 = LinearGAM().gridsearch(X_train, y_train)\n",
    "gam_100_funct_6 = LinearGAM().fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_funct_6.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd202546",
   "metadata": {},
   "source": [
    "**Functional form tuning** \n",
    "\n",
    "After trying all possible combinations of linear and spline terms, we selected the **models with the lowest GCV score**. It seems best practice to select between GAM models based on the lowest Generalized Cross Validation (GCV) score, which is an **unbiased estimator of MSE**. \n",
    "\n",
    "- **Model 1**: NDVI ~ spline(Grass%) + spline(Canopy%) + spline(Shrubs%) \n",
    "- **Model 2**: NDVI ~ linear(Grass%) + spline(Canopy%) + spline(Shrubs%) \n",
    "- **Model 3**: NDVI ~ linear(Grass%) + linear(Canopy%) + spline(Shrubs%)\n",
    "\n",
    "Additionally, we could argue in favour of **less complex models** and strive for the model performing best **combining GCV, DoF, and AIC**. Unfortunately, we found that adding both linear and spline terms to the same feature did not improve the model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "efc7ec31",
   "metadata": {},
   "source": [
    "## 3. Regularization penalty (lambda)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e5ec999",
   "metadata": {},
   "source": [
    "**Lambda** \n",
    "\n",
    "Lambda controls the strength of the regularization penalty on each term (also called regularization rate). Larger values enforce stronger smoothing. Terms can have multiple penalties, and therefore multiple lambdas. When choosing a lambda value, the goal is to strike the right balance between simplicity and training-data fit. If your lambda is too high, your model will be simple, but you run the risk of underfitting your data. If your  lambda value is too low, your model will be more complex, and you run the risk of overfitting your data. The ideal value of lambda produces a model that generalizes well to new, previously unseen data. \n",
    "\n",
    "Unfortunately, that ideal value of lambda is data-dependent, so some tuning is required. We can **perform a grid-search over multiple lam values to see if we can improve our model**. We will seek the model with the lowest generalized cross-validation (GCV) score. By default in pyGAM, spline terms use 2nd derivative smoothing for numerical dtypes, which encourages the functions to be smoother, and L2/ridge smoothing for categorical dtypes. Factor terms and linear terms have a l2, i.e. ridge penalty, which encourages them to take on smaller values."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "1e84e18e",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:752: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  np.fill_diagonal(Dinv, d**-1) # invert the singular values\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     30.1926\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -401863.4372\n",
      "Number of Samples:                         4673 AIC:                                           803789.2595\n",
      "                                                AICc:                                          803789.6923\n",
      "                                                GCV:                                                0.0056\n",
      "                                                Scale:                                              0.0055\n",
      "                                                Pseudo R-Squared:                                   0.7452\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [10]                 20           10.5         1.11e-16     ***         \n",
      "s(1)                              [10]                 20           9.5          1.11e-16     ***         \n",
      "s(2)                              [10]                 20           10.2         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/1115933263.py:5: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_lam_1.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model with lambda = 10\n",
    "gam_100_lam_1 = LinearGAM(lam=10).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_lam_1.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "5b774bc1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     33.8725\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -402350.7318\n",
      "Number of Samples:                         4673 AIC:                                           804771.2088\n",
      "                                                AICc:                                          804771.7483\n",
      "                                                GCV:                                                0.0056\n",
      "                                                Scale:                                              0.0055\n",
      "                                                Pseudo R-Squared:                                   0.7459\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [5]                  20           11.7         1.11e-16     ***         \n",
      "s(1)                              [5]                  20           10.6         1.11e-16     ***         \n",
      "s(2)                              [5]                  20           11.6         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:752: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  np.fill_diagonal(Dinv, d**-1) # invert the singular values\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/3093255912.py:5: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_lam_2.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model with lambda = 5\n",
    "gam_100_lam_2 = LinearGAM(lam=5).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_lam_2.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f5bd6aeb",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:752: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  np.fill_diagonal(Dinv, d**-1) # invert the singular values\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                      36.632\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -402643.6637\n",
      "Number of Samples:                         4673 AIC:                                           805362.5914\n",
      "                                                AICc:                                          805363.2188\n",
      "                                                GCV:                                                0.0056\n",
      "                                                Scale:                                              0.0055\n",
      "                                                Pseudo R-Squared:                                   0.7464\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [3]                  20           12.5         1.11e-16     ***         \n",
      "s(1)                              [3]                  20           11.5         1.11e-16     ***         \n",
      "s(2)                              [3]                  20           12.6         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/1381891552.py:5: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_lam_3.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model with lambda = 3\n",
    "gam_100_lam_3 = LinearGAM(lam=3).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_lam_3.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "05c81bab",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     49.7028\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -404325.5344\n",
      "Number of Samples:                         4673 AIC:                                           808752.4743\n",
      "                                                AICc:                                          808753.6088\n",
      "                                                GCV:                                                0.0056\n",
      "                                                Scale:                                              0.0054\n",
      "                                                Pseudo R-Squared:                                   0.7488\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.2]                20           16.9         1.11e-16     ***         \n",
      "s(1)                              [0.2]                20           15.9         1.11e-16     ***         \n",
      "s(2)                              [0.2]                20           16.9         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:752: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  np.fill_diagonal(Dinv, d**-1) # invert the singular values\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/3450052795.py:5: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_lam_4.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model with lambda = 0.2\n",
    "gam_100_lam_4 = LinearGAM(s(0) + s(1) + s(2), lam=0.2).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_lam_4.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "076d2e68",
   "metadata": {},
   "source": [
    "**Optimal regularization parameter**\n",
    "\n",
    "Lambda parameters seems to be more flexible, as GCV score maintains low for several values of lambda [0.2 - 10]. In this case, we can study other statistics like DoF or Pseudo R-squared to compare between models and select the best one. For instance, we could prefer a simpler model than a more complex once, meaning lower DoF."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92cc6cbb",
   "metadata": {},
   "source": [
    "## 4. Smoothness (n_splines)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54a7285e",
   "metadata": {},
   "source": [
    "**n_splines (Rank)** \n",
    "\n",
    "The parameter n_splines refers to the number of splines to use for the feature function. The type of basis function to use in the term is p-spline basis, default, or cyclic p-spline basis, used to build periodic functions. The term p-spline stands for \"penalized B-spline\". It refers to using the B-spline representation where the coefficients are partly determined by the data to be fitted, and partly by an additional penalty function that aims to impose smoothness to avoid overfitting. \n",
    "\n",
    "A spline function of order n (spline_order) is a picewise polynomial function (function defined by multiple sub-functions, were each sub-function applies to a different interval in the domain) of degree n-1 in a variable x. B-splines of order n are basis functions (an element of a particular basis for a function space) for spline functions of the same order defined over the same knots, meaning that all possible spline fucntions can be built from a linear combination of B-splines, and there is only one unique combination for each spline function. So, the higher n_spline you fit, the more higher degree polynomial is going to fit to your data for that particular feature."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "21b3ed86",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:752: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  np.fill_diagonal(Dinv, d**-1) # invert the singular values\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                    195.3096\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -400135.2222\n",
      "Number of Samples:                         4673 AIC:                                           800663.0635\n",
      "                                                AICc:                                           800680.372\n",
      "                                                GCV:                                                0.0058\n",
      "                                                Scale:                                              0.0053\n",
      "                                                Pseudo R-Squared:                                   0.7621\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                100          63.6         1.11e-16     ***         \n",
      "s(1)                              [0.6]                100          62.8         1.11e-16     ***         \n",
      "s(2)                              [0.6]                100          69.0         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/1774563792.py:5: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_spl_1.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model with n_splines = 100\n",
    "gam_100_spl_1 = LinearGAM(n_splines = 100).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_spl_1.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "fc64a4a5",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:752: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  np.fill_diagonal(Dinv, d**-1) # invert the singular values\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                    106.4944\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -402462.7439\n",
      "Number of Samples:                         4673 AIC:                                           805140.4767\n",
      "                                                AICc:                                          805145.5868\n",
      "                                                GCV:                                                0.0056\n",
      "                                                Scale:                                              0.0054\n",
      "                                                Pseudo R-Squared:                                   0.7539\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                50           34.9         1.11e-16     ***         \n",
      "s(1)                              [0.6]                50           34.2         1.11e-16     ***         \n",
      "s(2)                              [0.6]                50           37.4         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/4172701601.py:5: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_spl_2.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model with n_splines = 50\n",
    "gam_100_spl_2 = LinearGAM(n_splines = 50).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_spl_2.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "583df55f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                      55.935\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -403890.7062\n",
      "Number of Samples:                         4673 AIC:                                           807895.2823\n",
      "                                                AICc:                                          807896.7118\n",
      "                                                GCV:                                                0.0056\n",
      "                                                Scale:                                              0.0054\n",
      "                                                Pseudo R-Squared:                                   0.7493\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                25           18.8         1.11e-16     ***         \n",
      "s(1)                              [0.6]                25           17.9         1.11e-16     ***         \n",
      "s(2)                              [0.6]                25           19.3         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/2764893127.py:5: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_spl_3.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model with n_splines = 25\n",
    "gam_100_spl_3 = LinearGAM(n_splines = 25).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_spl_3.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "ad1cb9d7",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     33.6437\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -403690.7904\n",
      "Number of Samples:                         4673 AIC:                                           807450.8682\n",
      "                                                AICc:                                          807451.4008\n",
      "                                                GCV:                                                0.0055\n",
      "                                                Scale:                                              0.0055\n",
      "                                                Pseudo R-Squared:                                   0.7467\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                15           11.7         1.11e-16     ***         \n",
      "s(1)                              [0.6]                15           10.6         1.11e-16     ***         \n",
      "s(2)                              [0.6]                15           11.3         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/127136797.py:5: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_spl_4.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model with n_splines = 15\n",
    "gam_100_spl_4 = LinearGAM(n_splines = 15).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_spl_4.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "653d93e7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                      9.2139\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -387903.6261\n",
      "Number of Samples:                         4673 AIC:                                           775827.6802\n",
      "                                                AICc:                                          775827.7293\n",
      "                                                GCV:                                                0.0057\n",
      "                                                Scale:                                              0.0057\n",
      "                                                Pseudo R-Squared:                                    0.734\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                5            3.7          1.11e-16     ***         \n",
      "s(1)                              [0.6]                5            2.7          1.11e-16     ***         \n",
      "s(2)                              [0.6]                5            2.8          1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/926768180.py:5: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_spl_5.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model with n_splines = 5\n",
    "gam_100_spl_5 = LinearGAM(n_splines = 5).fit(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_spl_5.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c99b979",
   "metadata": {},
   "source": [
    "**Optimal smoothness parameter**\n",
    "\n",
    "After tuning the n_splines parameter, we observe that **optimal values of n_splines lay between 15 and 25** since the associated models have lowest GCV score (~0.0118)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a243e21",
   "metadata": {},
   "source": [
    "## 5. How to choose the best model and model tuning"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "46bc95fc",
   "metadata": {},
   "source": [
    "### 5.1. LinearGAM statistics summary"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "08b22174",
   "metadata": {},
   "source": [
    "\n",
    "- **Effective Degrees of Freedom (DoF)**: is the number of values in the final calculation of a statistic that are free to vary. The concern is that if there are more degrees of freedom (model parameters) in machine learning, then the model is expected to overfit the training dataset. This expectation can be overcome through the use of regularization techniques, such as regularization linear regression. The degrees of freedom are an accounting of how many parameters are estimated by the model and, by extension, a measure of complexity for linear regression models. We often **prefer lower complexity models over higher complexity models**. Simpler models generalize better.\n",
    "\n",
    "\n",
    "- **Log Likelihood**: the log-likelihood value of a regression model is a way to measure the goodness of fit for a model. The **higher the value of the log-likelihood, the better a model fits a dataset**. The actual log-likelihood value for a given model is mostly meaningless, but it's useful for comparing two or more models.\n",
    "\n",
    "\n",
    "- **AIC**: is an estimator of prediction error and relative quality of statistical models for a given set of data. AIC is founded on information theory. When a statistical model is used to represent the process that generated the data, the representation will almost never be exact; so some information will be lost by using the model to represent the process. AIC estimates the relative amount of information lost by a given model: the less information a model loses, the higher the quality of that model. Given a set of candidate models for the data, the **preferred model is the one with the minimum AIC value**. Thus, AIC rewards goodness of fit (as assessed by the likelihood function), but it also includes a penalty that is an increasing function of the number of estimated parameters. The penalty discourages overfitting, which is desired because increasing the number of parameters in the model almost always improves the goodness of fit.\n",
    "\n",
    "\n",
    "- **AICc**: is AIC with a correction (extra penalty term for the number of parameters) for small sample sizes. When the sample is small, there is a probability that AIC will select models that have too many parameters, i.e. that AIC will overfit. As n -> ∞, the extra penalty term converges to 0, and thus AICc converges to AIC.\n",
    "\n",
    "\n",
    "- **Generalized cross-validation (GCV)**: is a model validation technique for assessing how good the results of a statistical analysis will generalize to an independent data set. It can be taken as an estimate of the mean square prediction error (MSE) based on a leave-one-out cross validation estimation process. GCV provides an approximately unbiased estimate of the prediction error (MSE). This is one of the most important approaches used to estimate parameters in the context of inverse problems and regularization techniques. A notable example is the determination of the smoothness parameter in splines. The **chosen tuning parameters are the ones giving the smallest GCV error**. \n",
    "\n",
    "\n",
    "- **Scale**: the scale parameter is an estimated model parameter related to the variance of the response. It is a special kind of numerical parameter of a parametric family of probability distributions. The larger the scale parameter, the more spread out the distribution. It measures the statistical dispersion of the distribution. If models with unknown scale, grid search will optimize GCV, and for known scale, it will optimize UBRE.\n",
    "\n",
    "\n",
    "- **Pseudo R-Squared**: measure to evaluate the goodness of fit of certain Nonlinear regression models. While pseudo R-squareds cannot be compared independently or compared across datasets, they are valid and useful in evaluating multiple models predicting the same outcome on the same dataset. A pseudo R-squared statistic without context has little meaning. A pseudo R-squared only has meaning when compared to another pseudo R-squared of the same type, on the same data, predicting the same outcome. Then, the **higher pseudo R-squared indicates which model better predicts the outcome**."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3765d7ff",
   "metadata": {},
   "source": [
    "### 5.2. fit() vs. gridsearch()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90105161",
   "metadata": {},
   "source": [
    "The **fit()** function fits the generalized additive model and returns a fitted GAM object. Whereas **gridsearch()** function performs a grid search over a space of parameters for a given objective. The metric to optimize can be AIC, AICc, GCV or UBRE. If 'auto' is selected, then grid search will optimize GCV for models with unknown scale and UBRE for models with known scale. If no lambda parameter is specified, gridsearch will select a **common lambda** for all features that minimizes GCV score. Furthermore, we can easily allow gridsearch to select a **unique lambda for each feature** minimizing the overall GCV score by passing a lambda iterable into the lambda parameter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "36ddb39a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     44.9874\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -403714.1815\n",
      "Number of Samples:                         4673 AIC:                                           807520.3378\n",
      "                                                AICc:                                           807521.272\n",
      "                                                GCV:                                                0.0056\n",
      "                                                Scale:                                              0.0055\n",
      "                                                Pseudo R-Squared:                                   0.7479\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.6]                20           15.3         1.11e-16     ***         \n",
      "s(1)                              [0.6]                20           14.3         1.11e-16     ***         \n",
      "s(2)                              [0.6]                20           15.4         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:593: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  weights ** -1)**-0.5)\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/pygam.py:752: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  np.fill_diagonal(Dinv, d**-1) # invert the singular values\n",
      "/Applications/anaconda3/envs/ssml/lib/python3.9/site-packages/pygam/distributions.py:86: RuntimeWarning: invalid value encountered in reciprocal\n",
      "  return (np.sum(weights * self.V(mu)**-1 * (y - mu)**2) /\n",
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/3040872960.py:6: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_fit.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model using fit()\n",
    "gam_100_fit = LinearGAM().fit(X_train, y_train)\n",
    "warnings.filterwarnings(\"ignore\", category=RuntimeWarning)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_fit.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "2b40cad8",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (11 of 11) |########################| Elapsed Time: 0:00:00 Time:  0:00:00\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     48.8255\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -404236.9289\n",
      "Number of Samples:                         4673 AIC:                                           808573.5088\n",
      "                                                AICc:                                          808574.6046\n",
      "                                                GCV:                                                0.0056\n",
      "                                                Scale:                                              0.0054\n",
      "                                                Pseudo R-Squared:                                   0.7487\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [0.2512]             20           16.6         1.11e-16     ***         \n",
      "s(1)                              [0.2512]             20           15.6         1.11e-16     ***         \n",
      "s(2)                              [0.2512]             20           16.6         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/2995102486.py:6: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_grid.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model using gridsearch()\n",
    "warnings.filterwarnings(\"ignore\", category=RuntimeWarning)\n",
    "gam_100_grid = LinearGAM(s(0) + s(1) + s(2)).gridsearch(X_train, y_train)\n",
    "\n",
    "# check model statistics\n",
    "gam_100_grid.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7d30858d",
   "metadata": {},
   "source": [
    "It seems that gridsearch() outputs a GAM model with the same GCV score and lower DoF. Therefore, we prefer to use gridsearch() to find the best GAM model."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fada6bd",
   "metadata": {},
   "source": [
    "### 5.3. Optimizing for lowest GCV score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b5c04e6",
   "metadata": {},
   "source": [
    "Using gridsearch() automatically tune a common lambda for all parameters that minimize the GCV score. Additionally, we can pass an **iterable to the lambdas parameter** and get a unique lambda for each term that minimizes GCV score instead of a common lambda, which outputs a model with lower GCV score."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "8e536ead",
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (125 of 125) |######################| Elapsed Time: 0:00:10 Time:  0:00:10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     40.3749\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -405426.8559\n",
      "Number of Samples:                         4673 AIC:                                           810936.4616\n",
      "                                                AICc:                                          810937.2188\n",
      "                                                GCV:                                                0.0055\n",
      "                                                Scale:                                              0.0054\n",
      "                                                Pseudo R-Squared:                                   0.7485\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "s(0)                              [1000.]              20           5.2          1.11e-16     ***         \n",
      "s(1)                              [0.0316]             20           17.4         1.11e-16     ***         \n",
      "s(2)                              [0.0316]             20           17.8         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/4200477633.py:10: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_1.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model gridsearch() iterable lambdas (one for each term)\n",
    "warnings.filterwarnings(\"ignore\", category=RuntimeWarning)\n",
    "gam_100_1 = LinearGAM().fit(X_train, y_train)\n",
    "\n",
    "# using gridsearch to tune the lambdas\n",
    "lam = np.logspace(-3, 3, 5)\n",
    "lams = [lam] * 3\n",
    "\n",
    "gam_100_1.gridsearch(X_train, y_train, lam=lams)\n",
    "gam_100_1.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "8862d6c9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1gAAAEYCAYAAABBWFftAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAB5Z0lEQVR4nO3dd3iTVRvH8e/popQNZUMZsvcSRFFEZSmKCqiAIIri3op7oTh4URFxA4oTUVFZAgIiICBL9t6UPcssdJz3jxOglEIDJOnI73NduZrkeZLcqfbmOes+xlqLiIiIiIiIXLiQjA5AREREREQku1ADS0RERERExEfUwBIREREREfERNbBERERERER8RA0sERERERERH1EDS0RERERExEfUwBIREREREfERNbBERCQgjDGdjDFzjDEHjTFbjTF/GGOaZHRcaTHG1DbGLDHG7DLGPJ7i+XBjzL/GmNJevs/rxphFxphEY8yraRzvZIzZYIw5ZIz5zRhTMMWxHMaYwcaY/caYbcaYJ3zy5UQEY0w3Y8w0H75fWWOMNcaE+eo9JetSA0tOY4y5zXMBccgYs8Nz/wFjjAnQ5+vCRiSb8fwN9QPeBIoCMcDHQNsMDOts3gKeAmoDLxpjinmefwL4xVq7ycv3WQ30BEanPmCMqQ58BnTB/U4O434nx70KVATKAM2AnsaYVuf8TUSCmDGmiTFmujEmzhizxxjzjzHm4oyO6zhjTD5jzDhjzD5jzHfGmNAUx74wxtzk5fvc4vmeh40xk9M4XscYM9dzfK4xpk6q4497rnfiPNc/OS70uwUzNbDkFMaYJ4EPgP8BxXD/6N8HXAZEpHF+aOrnfEAXNiLZiDEmH9ALeNBaO9xae8ham2CtHWmtfdoY09AYM8NzgbHVGDPAGBOR4vXWGHOfMWaVMWavMeaj4x0+xpgQY8yLns6SHcaYrz2fl7JHuYcxZovnvZ/0HCvmudAolOJz6htjdhpjwoFywCRr7WZgFRBjjIkB2gHve/vdrbVDrLV/AAfSONwZGGmtnWKtPQi8BNxsjMnjOd4VeN1au9dauwz4Aujm7WeLBDtjTF5gFPAhUBAoCbwGHD3H9zHGGH9dM98L/Ie7FikL3OT5zMZAcWvtr16+zx5cJ9bbqQ948unvwLdAAWAI8PvxPGuMaQk8C1ztiaE87vck50kNLDkhxUXQA9ban621B6zzn7W2s7X2qDHmK2PMJ8aYMcaYQ0AzY8x1xpj/PKM9m1KOFhljIo0x3xpjdnsunmYbY4p6jnUzxqw1xhwwxqwzxnT2vEwXNiLZS2MgEjjThUIS8DgQ7Tn3auCBVOe0AS7GdbzcArT0PN/Nc2uGuyjIDQxI9dpmuA6TFsCzxphrrLXbgMme9zrudmCotTYBWAy0MMaUwl1wrAH6Az09x32hOrDg+ANr7RrgGFDJGFMAKJHyuOd+dR99tkgwqARgrf3BWptkrT1irR1vrV14/ARjTF9Px806Y0zrFM9PNsb0Nsb8g+uELW+MWW+MuSbFOa8aY75N9Zl3pe7Q8Zzb0Lgp0vuNMduNMe95DpUD/rLWHgWmej4nFHe986i3X9RaO8FaOwzYksbhK4EwoJ+19qi1tj9ggKs8x+8ABllrl1hr9wKvo2ueC6IGlqTUGMiB6+U4m05AbyAPMA04hGuQ5AeuA+43xtzoOfcOIB9QGiiEGw07YozJhbtYaW2tzQNcCsz3vEYXNiLZSyFgl7U2Ma2D1tq51tqZ1tpEa+163Ohy01SnvW2t3Wet3Qj8BdTxPN8ZeM9au9bTWfIccJs5dR3Ea55Rs0XAl0BHz/NDcI2q46PxHYFvPMeeAu4HRuAaf5fhOmvWGmN+N8b8bYzpcD6/jBRyA3GpnovD5dbcKR6nPiYi3lkJJBljhhhjWnv+fU+pEbAC17nTBxh0fHTcowvQA/d3t8HLzzytQ8fz/AfAB9bavMBFwDDP84uBa4wxOYHLgSXAI8AfnmsTX6gOLLTW2hTPLeTkdc0p10Se+0VTjvDLuVEDS1KKJtVFkHHzefcZY44YY67wPP27tfYfa22ytTbeWjvZWrvI83gh8AMnL44ScBdXFTy9R3Ottfs9x5KBGsaYnNbardbaJZ7ndWEjkr3sBqLNGRZ/G2MqGWNGGTf/fz9unVZ0qtO2pbh/mJN/pyU49cJnA66ntmiK5zalOl7Cc/93oJoxpjzQHIiz1s4CsNZusNZea62t5zmvFy439QV+BG4A3jMp1m6eh4NA3lTP5cXlu4MpHqc+JiJe8FxvNAEsbibKTmPMiOMzaYAN1tovrLVJuA6X4pyaO77yjOoknkMH75k6dBKACsaYaGvtQWvtTM/zg3Ad0f/iRrAW4Bp2/TwzhqYYY944r1/ASWe75knr+PH7uu45T2pgSUqnXQRZay+11ub3HDv+/8spa6CMMY2MMX951i7E4Uapjl8cfQOMA4Z6hsz7GGPCrbWHgFs95241xow2xlTxfKYubESylxlAPHDjGY5/AiwHKnp6d5/HTV/xxhbcWsnjYoBEYHuK50qnOr4FwFobj+tF7oy7oPmGtL0MDLTWbgdqAnOstXFALFDByzjTsgQ35REAT0MvB7DSM01na8rjnvtLEBGvWWuXWWu7WWtLATVwHSz9PIe3pTjvsOdu7hQv93bNd0pn6tDpjpuyuNyzXKKN53PjrbU9rLW1rLXP4qYGPo/LS6G4DutG5sLWgZ/tmiet48fv67rnPKmBJSnNwC38TK+ql031+HvcaFNpa20+4FM8F0eeheyvWWur4aYBtsFNJ8RaO85a2xzXY7Qc17uUmi5sRLI4z9/sy8BHxpgbjTFRxlUFbW2M6YPrJd0PHPR0tNx/Dm//A/C4MaacMSY3bvTrx1TTEV/yfGZ14E5cR81xX+PWGtyAWwB+CmNMNdz6hU88T60DrvL0gFcENp4tOM/3jMT9exvmWZd6vDjQd8D1xpjLPdOmewHDrbXHL2q+xhX6KeD5vdwDfJXub0RE0mStXY77G6rh7UtSPT4ERKV4XIzTnalDZ5W1tiNQBHgH+Nnzd3+CpxFlrLVjOXnNY4E5QC0vY07LEqBWqumPtTh5XXPKNZHn/nZr7e4L+MygpgaWnGCt3YerGvOxMaa9MSa3cRW66gC5zvLSPMAea228MaYhbo0WAMaYZsaYmp4Liv24IfIkY0xRY8wNnuRyFNd7kpTyTXVhI5J9WGvfw1UCfRHYievlfQj4DTdC3QnXW/oFpzaA0jMYN/I0BZcj4oGHU53zN66q6ESgr7V2fIq4/sFNV57nWf+V2kfAo54pRODWeD2CuyB501Ms42y+AI7gpgm94LnfxfPZS3Cj+N8BO3C5NGVxj1dwa1A3eL7D/zwXXiLiBWNMFWPMk5413Ri3zUtHYObZX3lG83FrPMONMQ2A9mmck2aHjjHmdmNMYWttMrDPc+6J6x7P9crbuKUR4PLZlcZV+rsMWHu2wIwxoZ73CANCPNc84Z7Dkz2f9Yhx29A85Hl+kufn10B3Y0w1zzq1F9E1z4Wx1uqm2yk33LD0LNw6h524ecE9cGXavwLeSHV+e9wFwAFcOdQBwLeeYx1xC0gP4abs9Mf98RfHXTDE4RLNZKBaqvf9C2iU4nFtYCmwC3jCi+/xFa73KeWtW4rjnXCNtEO4qYgFUxzLgbtw2++JO93P00033TLXDVckxwJh6Zw3Cbg7o+PVTTfdfHvDlWUfBmz2/Fu/GVdEJy9u5HpaqvMtbs04nuuSu1MdL++5JjqI2wKmf4rrneP5pgdu1GobrjjX8dd+i+tIOYjroLkx1Xv3Ap5O8TgfMN5znfQ9EJrOd+2WxjXPVymO1wXm4jp55gF1U73+Cc/1zn7c2rEcGf3fLyvfjOeXKiIikq0YY8rieoHD7RkqGBq34eifuCnOWm8gIiIXTFMERUQkKBljhgATgMfOt3HlmWZ8MK2bb6MVEZGsQiNYkmUZYy4H/kjrmLU2d1rPi4iIiGRFZ+m4aW2tnRrQYOSs1MASERERERHxkTQ3fcxOoqOjbdmyZTM6DJGgNXfu3F3W2sIZHUcgKN+IZCzlGxEJpDPlnGzfwCpbtixz5szJ6DBEgpYxZkNGxxAoyjciGUv5RkQC6Uw5R0UuREREREREfEQNLBERERERER9RA0tERERERMRH1MASERERERHxETWwREREREREfEQNLBERERERER9RA0tERERERMRHzroPljEmDLgBuA6oDeQH9gELgD+A36y1if4NUUSChXKOiJzm338hd26oXt2nb6t8IyL+csYGljHmXuAFYBnwNzAKOADkAaoC9wDvGWPetNZ+GoBYRSQbU84RkTQ1agSJvm3nKN+IiD+dbQSrEtDQWrstjWO/Am8aY4oDT/olMhEJNso5IpK2sLNOuDkfyjci4jdnXINlrX3yDIkn5TlbrbVP+T4sEQk2yjkicpo+faBlS0hK8unbKt+IiD953SVkjMkHVAZyp3zeWjvJ10GJiCjniAhjx0JcHISG+vVjlG9ExJe8amAZY7oBHwEHgcMpDlmgvO/DEpFgppwjIhw7BjNmwH33+fVjlG9ExNe8HcHqDbS31v7hz2BERDyUc0SC3ezZEB8PTZv6+5OUb0TEp7zdBysMGO/PQEREUlDOEQl2U6a4n02a+PuTlG9ExKe8bWC9A7xojNHGxCISCMo5IsGuTBm4806Ijvb3JynfiIhPeTtF8HGgGNDTGLM75QFrbYzPoxKRYKecIxLsOnVyN/9TvhERn/K2gXW7X6MQETmVco5IMNu4EfLlczf/U74REZ/yqoFlrf3b34GIiBynnCMS5J55BqZOhU2bwBi/fpTyjYj42rnsg1UHuByIBk5kO2vty74IxBjTCvgACAUGWmvfPsN5FwMzgVuttT/74rNFJPNRzhEJUvHxMGoUdOzo98bVcco3IuJLXi3oNMb0AP4BrgKeAWoCTwIVfBGEMSYUtwdFa6Aa0NEYU+0M570DjPPF54pI5qScIxLExo+HgwehffuAfJzyjYj4mrcVc3oCray1NwFHPD/bAwk+iqMhsNpau9ZaewwYCrRN47yHgV+AHT76XBHJnJRzRILVzz9DgQLQrFmgPlH5RkR8ytsGVhFr7VTP/WRjTIhnQ77rfRRHSWBTisexnudOMMaUBG4CPk3vzYwxPYwxc4wxc3bu3OmjEEUkgLJMzlG+EfGhI0dg5Eho2xbCwwP1qco3IuJT3jawYo0xZT33VwJtjTGXA8d8FEdak6xtqsf9gGestUnpvZm19nNrbQNrbYPChQv7Ij4RCawsk3OUb0R8KGdOmD4dXnjhxFMJScn+/lTlGxHxKW+LXPQBqgLrgV7Az0AE8IiP4ogFSqd4XArYkuqcBsBQ4xa8RgPXGmMSrbW/+SgGEck8lHNEgo21rqhF1aqnPP3gd/PIExnOu7fU9tcnK9+IiE95W6b9qxT3/zDGFAAirLUHfRTHbKCiMaYcsBm4DThld0Frbbnj940xXwGjlHhEsiflHJEg1Ls3LFwI3313Ynrg4s1xjF+6ncevqeS3j1W+ERFf83aKIMaYQsaYLsaYnp5FmnmNMaV8EYS1NhF4CFc5ZxkwzFq7xBhznzHmPl98hohkLco5IkFk40bo2xeOHTtl7dUHE1eRNzKMbpeV9evHK9+IiC95NYJljGmKq2wzB7gMN5xeEXgKHy0CtdaOAcakei7NxZ7W2m6++EwRyZyUc0SCSEIC3HYbJCe7RpbH4s1x/OkZvcqX038FL5RvRMTXvB3B6ofb9K4VkOh57l9c6VEREV/rh3KOSHB47jmYMQMGDYIKJ7ee6h+g0SuUb0TEx7xtYJW11k703D9e+eYY3hfJEBE5F8o5IsFg3z74/nt44AHo0OHE0wtj9zF+6Xa6Nynv19ErD+UbEfEpbxtYS40xLVM9dw2wyMfxiIiAco5IcMifH/79F95//5Sn+45fSYGocO5qUjYQUSjfiIhPeds78yQwyhgzGshpjPkMNy85rZ3IRUQulHKOSHY2YwaMGOEqB5YufcqhWev2MGXlTp5rXYU8kQHZbFj5RkR8yqsRLGvtTKA2sAQYDKwDGlprZ/sxNhEJUso5ItnY2rXQti389BPs33/KIWstfcetoHCeHHRtXDYg4SjfiIiveT2/2Fq7GVdZR0TE75RzRLKhuDho0wYSE2H0aDdFMIWJy3Ywa/0eXm9bnZwRoQELS/lGRHzJ2zLt+XA7mtcFcqc8Zq1t4Ye4RCSIKeeIZEPWwl13wcqVMGECVK58yuHEpGTe+mMZ5aNzcVvDmICFpXwjIr7m7QjWT0Ao8CtwxH/hiIgAyjki2c+iRTBqFLzzDlx55WmHh82JZc3OQ3zWpT7hod7W4PIJ5RsR8SlvG1iXAIWstQn+DEZExEM5RyS7qVULFi6ESpVOO3QgPoH3/lxJgzIFaFGtaKAjU74REZ/ytotoGlDVn4GIiKSgnCOSXWzfDj//7O5XrgzGnHbKh5NWs+vgUV5sUw2TxnE/U74REZ/ydgSrGzDGGPMvsD3lAWttL18HJSJBrxvKOSJZX2Ii3Hab2+vqssugePHTTlm94wCDp63jlgalqFM6f+BjVL4RER/ztoHVGygNrAfypnjepnm2iMiFUc4Ryeqshfvug8mTYciQNBtX1lpeG7mUnBGh9GxVJfAxOso3IuJT3jawbgMqWWu3+jMYEREP5RyRrMxaePJJGDQIXnoJunZN87QRC7YwddUuXrm+GtG5cwQ4yBOUb0TEp7xdg7UW0OJPEQkU5RyRrGzePHj/fXjkEXjttTRP2X3wKK+OWEKd0vlP31R42zYYOtT/cTrKNyLiU96OYH0DjDDGfMjp85Mn+TwqEQl2yjkiWdG+fW7z4Pr1XUn21q3TLGoB8OrIpRw8mkif9rUIDUlxzvTp0L49HDwI11wD0dH+jlr5RkR8ytsG1oOen2+met4C5X0XjogIoJwjkrXEx0OfPu42bRrUqQPXXXfG00cu2MLIBVt4onklKhXNc/LApk3QsiUULQpjxwaicQXKNyLiY141sKy15fwdiIjIcco5IlnIvHnQpQssXQq33gpFipz19A27D/Hc8EXUL1OA+6+86NSDjz3mKg/++SeUC0waUL4REV8L6FbpIiIiko307w+NGsHevfDHH27dVIkSZzz9aGISD//wHyEGPritDuGhKS5DFi6E4cNdUYwANa5ERPzhjA0sY8xsY0wHY0zEGY5HGGNu8ewbISJyQZRzRLKguDg3FXDxYmjV6qynWmt57pdFLIyNo0/72pQqEHXqCbVquemFTz3lx4Ad5RsR8aezTRG8A+gFfGKMmQesAA4AeYBKQD1gEm6DPhGRC6WcI5IVJCbC2rVQqRK8+KIryR6S/oSY/hNXM/y/zTzRvBKtahQ79eCxYxAR4TYjDgzlGxHxmzNmRGvtUmtte6AGrsLOESAaOAx8DVS31t5qrV0WkEhFJFtTzhHJAo4dc+usGjeGnTtdhUAvGlffztzA+xNWcnO9kjx8VYVTDyYmQu3a8Pbbfgr6dMo3IuJP6Ra5sNZuwyUfERG/U84RyaT27HGNqwkToF8/KFzYq5d9M2M9L/2+hKuqFOHtm2thUpdt//FHWL4cqlb1fczpUL4REX/wtky7iIiIBKv58+Hmm2HzZvjyS+jWLd2XWGv5ePIa/jduBddULcJHnesREZZqtCspCXr3hpo14frr/RK6iEigqYElIt4ZPx4mTQroNB4RySS++AKOHoW//4ZLLkn39KOJSTw3fBHD523mhtol+F+HWuQICz39xJ9+gmXL3CiWF1MNRUSyAmUzETm7+fOhRQu3+eewYbBvX0ZHJCL+tngx3HknTJ7sHvfqBQsWeNW4Wr3jADd+NJ3h81xBiw9uq5N248paeOstqFED2rf3bfwiImk5eBA++wzuvtuvH6MGloikbccOuOMOqFcP5s6F9993Pc3582d0ZCLiL4cPw9NPu6ITw4bBypXu+UKFIDr6rC9NTEpm4NS1tPlwGtv3xzPojgY8cnXF09dcHWcMjBjhphxq9EpE/G3pUqhbF+67D2bNcttMJCXBm2+6Dh8fSneKoDGmENAOqI4rX3oAWAL8Yq3d7dNoRCTjWesufMLC4M8/oWdPePbZEw2rDbsPMXDqOh6+ugJF8kT6/OOVc0QyyJo1cO21rlHVo4cbXSpY0KuXzlizm9dGLmH5tgOeYhY1KZL3LPkhKck1qsqUcbcMonwjEiR+/RW6doVcudy1zdVXu2udUaPghRfg8svdzUfO2mVkjLkaWA3c7jl3C2CAzsAqY0wzn0UiIhkrPh769oXmzSE52V1YrV3r1lzlz8/izXE89P08mvWdzI+zNzFvwz6fh6CcI5KBfv0Vdu2CiRPdFBovGlcLNu2j6+BZdPxiJnFHEvj09noMuqPB2RtXAO++66YeHz7so+DPnfKNSBbzzDNumnLz5m6GzQ8/eJdD4uPh8cddpdI5c+Caa1zjCuCqq1yu69fPp6GmN4L1IdDdWjs89QFjzE3Ax0Dg66qKiO8kJcF338FLL8HGjdCqlVtnVbAgNkcOZqzZxSeT1zB11S5y5wjjnivK0/2yculfQJ0f5RyRQEtIgPBwePJJ6NwZihc/6+nWWqav2c0nk9cwbfUu8keF8/y1VejauCyR4WmstUpt1Sp45RVo3Rqionz0Jc6L8o1IZrZ6tSuw9cAD7vG6dZAzp1tHNXYsfP01tGkDI0e62TdHj0Kk59pk1y745hu3ljR/fre9REyM29A8pagoN2Lfpw+sXw9ly/ok9PQaWGWA0Wc4Ngb4zidRiEjGWL8e2raFhQuhfn23FuKqq0hKtvy5eCuf/L2WBZv2EZ07Bz1bVeb2S8qQNzLcnxEp54gE0j//uGkzv//uik2cpXGVkJTM6IVb+XzKWpZu3U/hPDl4rnUVOl9Shtw5vCxKnJwM99wDOXLARx/56EucN+Ubkcxmzx4YPdp1/I4b5zp/brwRSpRw60KPS06GadMg1NOpExvrGkcxMXDokNsIHaBoUejUCSq4Dc73HjrGjLW7WbHtALlzhNH10jLkePBB+N//YMAAN5PHB9LLiP8CbxhjXrXWHjr+pDEmF/CK57iIZDX79rkenRIloFgxt8bq1ls5mmz5bfZGPvt7LWt3HaJMoSh631SDdvVKneyZTk52Q/K5c/sjMuUckUAZNQpuuw1KloQCBc542pFjSfwwayMDp65lS1w8FYvkpk+7WtxQp4R3I1bHWQsPPeRKvQ8cmO5IWQAo34j4UkKCG1n69184dgzy5nWNnsaN4aKLTj8/Ls6NaJcs6fLB+PFuHWhSEpQqBa+95jpk0soVISFwxRUnH0dEwIsvujWkefJAxYpunVW9eoArwvP1jA28P2ElB+ITMcalpJ/nxvLuLbWpceutbvrg8XXoFyi9BtadwA/ALmPMWiAOyAuUB+YDt11wBCISOKtWucWc//zj7kdFwbhxHIhP4Idp6xg0bR3b9x+leom8DOhUl9Y1ihMa4kk0Gze6Ea7Bg6FdO3jvPX9EqJwj4m/Wul7aZ55xFx+jRrmOllSOJSbzzcwNfDJ5DbsOHqVhuYK8cVMNrqxUhJCQ87gA2b3bXXz17Al33eWDL3LBlG9EfCEpyY0kWQu33AKJia5QVny8O/7ss65oTlwcNGzoju/Zc3LblwED4MEH3Sj6s8/CDTdAgwZpVheNO5JAYlIyIcaQL2f4yVxUtKhrkKXhn9W7eG3kElZuP8jlFaN5vHklqhXPy/Q1u3j2l0Xc+tkMJn8yiMI+XPpw1gaWtXYDcKkxphJQDcgNHASWWGtX+SwKEfGvbdvcPjZffOGm5jzxBFjLzgNH+fKfdXwzcwMH4hO59KJC9O1QmyYVok+WVh43zjWm/vzTPb7mGrjySr+EqZwjEgCffeYaObfc4jpN0lgHNX7JNnqPWcaG3Ye5rEIhPr66Hg3LeVdR8DQ7drjKXdHRroe4QAGf9BBfKOUbkQt07JhbuzRyJMyY4UaRZs6ESpXcWqmjR906quMzXg4ccGXSw8NPjm5ddJEb4QI3q+aNN068vbWWZVsP8NeKHfyzehcrth1g96FjJ47nCAshpmAUFYvmpkqxvJQpFEWpAjkJDw0hPiGZhbH7mLBsOzPX7qF0wZx8ent9WlYveuL65qoqRfmhRy6av/c3n01Zy4ttqvnsV+PVpGlr7Upgpc8+VUQCZ/Vqt6fNsWNuqP3ll9kQnofPx6/lp7mxJCQl07pGMe5rehG1SuV3r1m82M1XjoyEefPc/lcvvwzduvlsAejZKOeI+FhcnBuFrlnTrbnKmdP9TNXQ2X3wKC+PWMLohVupWCQ3Q+5qSNNKhc/vM611ayYefhhuvRU+/NDrsu+BpHwjch6mT3fXFEuXus6aI0dcR0rt2ifPyZEDqlc/+bhUKRg6NN23XrHtAKMWbmHkgi2s3+2qBFYrnpdrqhalQpHc5AgPITHJsjXuCOt3H2bJlv2MWbQtzfeqWCQ3z7Sqwp2XpV2E56LCubmpbim+mbmBJ37tR9Sh/V7FmB5v9sHqAXTD7RFxoncH+NJa+8UFRyAivhcf7zYHvuwy1zvUsyd07szinIX5dNIaxizaSlhICO3ql+Sey8tTvnBu17M0cKC7/fuvK396223w2GPu9aHnsNbiAijniPhQcrL7m37+eTeCtHSpG7G6447TTp26aieP/zifuCMJPNWiEvc2vYjw0PPcAHjdOreZ5/jxbhriffdd4BfxD+UbkXN0+LCbxjdgAJQu7QpSXHvtKackJCWz99Ax9scnkGwhMiyU3JFh5MsZfnLZQQpxhxNYtDmOqat3MmnZDlbtOEiIgcsqRHNf04u4qkqRdCsXHz6WSOzeI2zed4SkJEtEWAgXFclNyfw5Tz1x5Up3y5vXjbQVK8ajV1fk9/mbWRq7lwb//nlyyuMFOGsDyxjzDtAGeBdYwMn5yXWAJ4wx5a21z11QBCLiO4mJrizpK6+4EqUbN2ILFWJG5wf55O81TF21gjypS60fOgTdu8OPP7r71au7KYHXXOPeM2fOs3+mDynniPjQkiWuh3nGDLcY/P3301zTkJRs6T9xFf0nraJC4dx8e3cjqhTLe/6f+9tvrtx7aCj07+9KLAeog+ZcKN+InIfQUPjrL1ewpndvV1ACWLIljt/+28ys9XtZtmU/x5KST3upMZAnRxh5IsPJER5CUrIl7kgC+w4nABAeaqhfpgBdG1enVY3iFM6Tw+uwoiLCqFQ0D5WK5jnzSbNnuzVgx+XMCf36EXPPPbSvX4ofF5ehQVycy521ann92WlJbwTrLqCWtXZrqufnGWPGAgsBJR+RjGYtDB/uKugsXw4NG5I0+EvGbU3gs6H/sCA27tRS64f2w5zpboO9qCg3JfC221xD65JLMnJ9hHKOiC/MnQuXXup6aYcMgS5d0vy7jjuSwGND/+OvFTu5uV5J3rixBlERXpZcTy052TXgataEli3hgw9cD3fmpXwj4o0jR+DNN91sljx5XEMlMpLkZMu4RVsZ8NdqlmzZT0RoCHVj8nNnk7KUKhDlilAYiE9I5kB8AnsPJ7D/iLsdTUomLMSQO0cYZQvlokLR3DQsW5BcaW35cDy3gFsXvmSJW1u+Y4frTC5VCj791B3/8ENXQKNwYTczZ/Vqt7ThhRfcdjQDBrifBw64Yj8PPghNm3JT3ZI8OcGzBmvqVL83sNK7yvLZVZgxphXwARAKDLTWvp3qeGfgGc/Dg8D91toFvvp8kSxt5Uro0AEqVyZh2E/8HHMxn09dx7pd8yh7vNR6nRJETpsCd77kGmORkS5B5czpFqVmgkXnKOeIXJi9e10Ribp14bnn3OhRkSJpnrpy+wF6fD2HzfuO0PumGnRqGHOyuE16rHXTaA4ccFOKhwxxF2G//uqmJQ8/be/ezEj5RiQ9K1bAzTe76cU1arj1lJGRzFm/h1dGLGHJlv2UL5yL19tW5/raJcgfFZH+e6bn8GE3vXjcOJgyxZV/X+lZJvnhh25aYo4cLrdFR7ttZ44bNsztj3VcdDRcf727HxLiGlTHXX21K7xTuTJ1E5PZG12cfYWKkn/q1FPPOw/pNbAGAZOMMamHz2sDTwA+mZ9sjAkFPgKaA7HAbGPMCGvt0hSnrQOaWmv3GmNaA58DjXzx+SJZ0owZbj+ZZ5+FypU5NO5PvgkpxeCZm9gxdzE1S+bjo071aFWjGKF/jofqV8OaNS4R3Xsv3H33yel/maNxBco5IufOWrf1wocfwoQJrihNkSLw6qtnfMmYRVt56qcF5MoRxg/3XEKDsmcoPmGtK63er58reLN1qyu//Oij7vOOK1DAretKTHQVwrIG5RuRs/n7b7jpJvc3P3YstGzJ/vgE3hqzjB9mbaJEvkjeu6U2beuUTHNtFeBGn7ZvdwUw8nox9fiTT9wWEgcOuOqDV1wBdeqc3J9q4EDXQZwvX9rXLlOnugbZrl3pf2ZIiJsyaC0Rvw2nedF8/NTgOu65pKZXv56zSa9M+zOevSHu5PQFoP2ttZ9dcAROQ2C1tXYtgDFmKNAWOJF8rLXTU5w/Eyjlo88WyVrmz3dTAUePhmLF2Nn5TgYu3sP3MxM5cHQ1l1eM5r32tbhs3TxM6B4IKe56cI5v2nfzzQFdV3UulHNEzsGqVW66y+jRp3aeRJ55MfixxGTeGbucQdPWUS8mP5/cXp+iZ1o8vmiRqxw6b57bCPSWW1wBndy53RrNQoXcRU758m5KYA7v10tkBso3ImcxYgS0b+9GpMeMgXLlmLN+D4/9OJ+tcfH0uKI8j15d8fQpfbt2uS1h/vvPXa+sW+c6XgYNcvvfzZ8Pbdq4aXvFi7sOmc2b4fPPoXJl93yHDtCpE1x+uSv9nlIae/adJjz83DYyX78eOnbkwVY307xOV1p0uZIy3r86TelOtPYkGF8lmTMpCWxK8TiWs/fcdAf+ONNBT1WgHgAxMTG+iE8k423aBE8+CT/9BPnzs+eFV3i/ait+/HQeicnJtK5ZnIcrR1Fl7C/QaqBLGA88AB995OYbT56c0d/AK1kt5yjfSEBZ66bP5MrlLlq++MKtpXzuObeOMleuM7503a5DPPbjfBZs2scdjcvw/HVVyRF2huITa9a4jT7z54evvoKOHU+90LnhBnfL4pRvRM6gTh3XwProI5Lz5eeTv1bz7vgVlCyQk2H3NqZ+mQLuvMWL4euv3fmdOrmpw88/7zpe6tSBdu3cWszj+2fmyOGm5m3a5F579KjrAF63zjWwWrd2t0AqVw4ee4yKfftSonQrpqzaRZdCZ86l3jjPlayOMSbGWrvxgiLwvFUaz9kzfGYzXPJpcqY3s9Z+jhtep0GDBmm+j0iWcXxH9PBwmDKFbY88xTtVr+O3DYeIWLaXDg1K0eOK8pR55lHo+pVLblddBe+8A23bZnT0PpUZc47yjQTMzp2uBxhc73LVqm4x91lGrAASk5L58p/19B2/goiwED7pXI/WNc/Qu3vwoBuhuugiV3WwQwe3WDwIKd9I0LHWraO88UaIiYHvv2d/fAJPfDOXCcu206ZWcd66uSZ5IsPdVLxnn3X7YYWHw9NPu/coWtStBU25LiqlqlXdms3M5p57oG9fOmyay7RVNelyyYWNYZ3nBhdgjMmBmzPsC7FAylJDpYAtaXxmLWAg0NZau9tHny2SOa1d6y6mrroKm5zM5LgQOr88jEtyXsnEbUd5uk4B5hZaRe+21SlTKBeUKQNPPOEWgk6c6KbzZLEpO2ejnCNBbdUqN6I0fvzJLRTgrI2r5GTLyAVbaPH+FHqPWcblFQsz4YmmaTeujhyB1193Pc2LFrnnHnggmBtXyjcSXJKSXOn1du3gu+8AWLvzIDd+9A+TV+zgleur8WHHuq5x9dxzbm3Upk1uW5ctW1zJ9uPO1LjKzCpVgho1aLvuX6av3k1iGmXmz0V6+2BdcZbDvrxymw1UNMaUAzYDtwGdUsUSAwwHunh2XRfJntatc4lqyBBsaChrb76dx96bxKJdRymWJwcDSh6g5bTfCX/zF7eQs0ljuPhieOmljI78ginniKRhyRLXqEpMdL3F9euf9fQd++P5eV4s3/+7kdi9R6hcNA+fd6lP82pF064SOGaMu7Bat86NWBU8Q8GLbEb5RsRj5063lcO4ca4Ue+fOTF21kwe/m0dYaAjf3d2IRuULnTz/sstc2fPnn3dbvWQX7dpR5KtvyEUim/YeoVz0+U8TTG+K4GRgK3Bhzbh0WGsTjTEPAeNwJUwHW2uXGGPu8xz/FHgZKAR87PkHItFa28CfcYkE3IQJ0Lo1NjSUZW078Vyl61iQnIuKIeF8fEkkrV68j5AlS9zC8vvvh/vuc8Pt2cdklHNETkpOdqPRxriKXtWqpXna/vgExi7axu8LNjN9zW6shcblC/Fc66q0rlGMkLQqfCUnuzUTP/4IVaq4ke+rrvLzF8pUJqN8I8Fuxgy31mr3bldo4p57+HrGel4buZSKRXLzRdcGlC4YBd9+64pRPPOMK1LRpk1GR+57zz9P5IsvMT0kJO2ceQ6MtWeewmuMWQd0TlXd5vixSOCQtTbzbc+eQoMGDeycOXMyOgyRM1u92iWtpk3ZtXMf6x58mhdimrEyLB83h++lS6kQat/TkZCkRDcvul07t5g9i/QaGWPmenuhkNVzjvKN+MXixW4qYIUKpzxtrWX2+r18M3MD45Zs41hiMmUKRdG2Tkna1inBRYVzp//evXu7dRdPP50tphQr34icozlz3OjVDz+QULMWr41cwrczN3JN1aL0u60OucND3EjVO++44hTjxkFopv2z8I2UGxun40w5J70RrDlAA+C05IPr8fHF4k+R4LRkCbz1FvzwA8cqVqLXWz/y09zNJJdtw1OHl9Fp9kjyzJruLqp6dHSLSEePzuio/U05RwRcr/L48fDKK25zz1SmrdpFn3HLWRgbR57IMDo1jOHGuiWpXSpf+psFf/21K4V8xRVumk/wUr6R4JOY6Eaq1qyBd991azsXL2ZvfBIPDJrFjLW7ubdpeXq2rELo3j1w++1uD6z77oP+/bN/42rYMFeEY8SIC/qu6TWwOp3pgLX2GFDuvD9ZJFgtWuQ2AB0+nKSoXExq1Znny7cgbs5mXolfzK3fv0/Ytq2ubGifPq7QRebZCNjflHNERoyAW291pYsfffSUBePb98fz3PBFTFq+g5L5c/LmTTW5sW4JoiK8KApsrdsL77XX3FqrK862BCkoKN9IcJkwAR5/3I2KX321W8cdHs7ibQe579u57Nh/lPduqc3N9UrBsWPQuDFs2OA2/7333uC4FmnXzk3LvkDpbTSc4O0bGWNGW2uvu+CIRLIja12vUXg4yStWkjRhIiOu7cbrFVpQ5fBOujUqTYcbGlFkZigsrglffO72gcjuPUWpKOdIUEtOdqPaL73kepVHjTqlcTV28VaeHb6I+IQkXri2Kl0vLXPmfaxS27cP7rwTfvvNbR78mb+3fsr8lG8kaKxdCw8/fGLDYH7+GW6+GQv8NGcTL/22mAJREfx47yXULZ3fvSYiwq23qlkTGjbMyOgDy0fXXRe0D1Yql/vwvUSyh+Rk+P13eOcdElu0YPgN9zBwfUF23PEJ1+9YwriJ/6Pokv+gYE+4vSm0aOFu4g3lHMleunZ15ZE7dnQbCHs2DU5Otrw/YSUfTlpN7VL5eO/WOt6trzru0CFXeXDjRldS+bHHgqMn2reUbyRrOXQItm1ze9rlzAnz58Pbb7tR8chI9scn8MKvixm5YAuNyxfiw051iU447PbP7N795E85L75sYInIcfHxruJO376wYgVxJUrz/tKjfHVsIc8vG0OXmcPJuWObW1/Vr5/rWRaR4Hb99W7LhUceOdEAik9I4vEf5/PH4m3c0qAUr99Y48yjVps2wdy5sHWru5+U5Bam58oFTZq4KT6XXhrALyQiF+TIEdc4Apg3D2Jj4cABt5Hvvn1uU/DHHnPH33jDLUE4cMCtr1q92k0D/usvKF4c1q93a7mBcUu28crvS9h58ChPt6zMfU0vInT+f2563ObNLhfJBVEDS8Qf7rsPhgxha/mqvHvTM8wqXIEyDWvyzRXladJrKKZWDXj0C7j2Wq8r1YhINvTjj3DwoOspvvXWUw7FHUngnq/nMGvdHl68rirdm5Q7cwGL++47ddpfWBjUrn3y8ZAhfgheRHwqKQkmTYKvvoI//3TVQzd6aq28/PLpha6qVz/ZwFq0CBYudBWGa9Z064iaNz95bng4S7fsp+/4FUxavoMqxfLwaZf61Cmd3+WhO++E6GiYMgUuuSQAXzZ7UwNLxBfWrnUjUQ8/zMKchRlTpTVLbqtIiLU8s2IsfX99Bx6YDxULw+DBJ3qRRCRIJSe7YhO9ernF5qmK2ew4EE/XQbNYs/Mg/TvW5YbaJU59vbUwdKhbq5k/P7Rq5UbEr7gCSpeGIkWCbg2nSJY2f77raJk3z/1Nt23rKoha63LD//7nckbu3FCggDsnIuLk63/8Mc23TU62TF+zm29mrmfcku3kiQzjudZVuKtJOcJDQ2DWLLf1y2WXwfDhLnfIBfNlA0sTuiW4WAvTp8N772F//RUbGsYnB/LTL7oet6ycRd+Foyi6fiUULeouokqVcq9T48pXlHMkazp0yBWa+Pln9/PTT09pXG3ac5jbB/3LzgNHGXTHxVxRqfCpr9+61U33GzkS3n/f9WDfeGMAv0BQUr4R/9q92/1tDxniRp8iI089XrWq12+1Pz6BBZv28dfynYxfuo3YvUcoEBXOQ80qcM/l5ckXFe6uYcBNS/7mG1dZNBvshZdZ+LKB9aYP30skc0tOhmbNYMoUjubNxy/NOvJB5eaEli7Ni3Wi6fpZV0zZsvDll27BupKWPyjnSNaTmAhNm7pe6v/9D5588pTG1crtB+g6aBaHjyXy7d2NqBdT4ORrrYXvv3fVwI4ccWs8H344A75EUFK+Ed9bvx4mT3YdLVdf7dZOedZcHTmWxKLNcazcfoBdB4+y73ACydZiLYSFGiLCQgg1BmPgyLFkDh5NYGtcPJv2HGb97sMARISF0KRCNE+2qETrGsWJDPeMas+YAQ8+6BpzNWu6va7Ep87awDLG3JXeG1hrB3t+vuWroEQypT17XI/xHXew63ACK8vWZkqeWkwqUoWH10xizPSPyDtnJuFhoW6heYUKqtR1jpRzJNs63lscFuam/vTq5dZgpjBn/R7u+mo2keGhDLuvMVWK5T31PXr1cnvoXXKJW6NRuXJAQs+ulG8kQ/30E9xzj1uH3bYtFCjAsfAc/LlwKz/O2cQ/q3eRlGxPnJ4nMoywEHdNkZhkOZqUTHKyJdlaoiLCiIoIpXi+SKqXyEf7+qWoVSo/DcoWOHWPvE2bXMfORx+5qcQHDgT6WweN9EawuqRz3AKDfRSLSOa0fLnbvXzIEDh8mPf25uXTXZFUTy7DC9vH8MyYjyAsDNO5MxyNh7BcULFiRkedVSnnSPZirdvc85VXXAPpmmvcPlepjFywhad+WkDJ/DkZcldDSheMOnkwOdldhHXt6ioCPv641lf5hvKNBN6+fW7kevBgaNQIvv8emz8/YxZu5Z2xy9m45zAl8kVyz+XlaVCmANVL5iU6dw63XupCPPoofPyxy0n33AN9+kDevOm/Ts5LehsNNwtUICKZzsaNbp3D2LEkR+RgWsMWvFmxBet2RfJi8mq6fvuUW2T67LPw0ENQokS6bylnp5wj2cauXfD11+4iaskStwbz8OHTTktOtvSbsJL+k1ZzcdkCfHp7fQrl9kwpttbtWzNrFvzyi9sg9KmnAvxFsi/lGwm4o0ehXj3YsMFdO/TqxaYDCfT84l9mrN1NlWJ5GNi1Ac2qFCE05DxnwBw54qYATprkZtOMHu06aPLkgQcegCeegDJlfPu95DTpTRGcBXwF/GCt3RuQiEQy0qFDbk509eoczZefoyvW8Fvru/ikbBOu27KQx4rEc/GzV1Eo4mooHQZduriKPuITyjmSLVjr9ptatcr1UH/+uRt9SrUWc8u+IzwxbD4z1+6hQ/1SvHFTij2uEhLcuozvv4dOndxjreX0KeUbCZhVq9yygRw53Ah2zZrY+vX5YdYm3hi9lBBjeOPGGnRsGHOyYbV2Lfzxhxv1rlzZNZjuvdeVYS9Vyk3xK1zYVR4sW9aVdX/+eViwwOWL0FBXwGL3bnfeG29k6K8g2KQ33vgdcCew1Rgz3BhzgzFGpd0l+9m4EZ55BkqVIunmm/lo4kouHzCLK9q8SjKGv37syYu/96PV8mmudzkiAu6/X40r31POkazpwAG3qW9Cglt72b+/25dm5kw3HSdF4yg52fLDrI206jeFRbFx9Glfiz7ta51sXB09Cu3bu8ZV795u03I1rvxB+Ub8a9kyt79dpUon97C68072VavF/d/O4/lfF1E3Jj/jHr+C2y8pQ+jhQ/DFF65hdNFFbnbM/PnudfnyuedjYly1wV9+gTffhB073PGDB90o1RNPwKhRbt34jBmucSUBZ6y16Z9kTFXcXOXOQE5gKDDEWjvXv+FduAYNGtg5c+ZkdBiSWc2f7xLU8OFYa1nS6Gp6V2rBjKJVeHXLVLr8MoDQI4fdHjM9e8KVV6pwxTkyxsy11jY4x9dkyZyjfBOkpk51I1Tr17se51atznjq3A17eH3UMuZv2kfDcgXp064WZaNznXpSp07www9uIfoDD/g39mxG+UYynLWuYfO//8Hvv7t1k48+6tZdFSjA9DW7eOLHBew+dJSeLavQvUk5QkIM7N/v1m/v2OH2v7rzTrj++rOv6U5Odj9DLnB9lpy3M+Ucr3pqrLXLgOeNMS8AzYDbgYnGmFhrbQ3fhiriZ8eOuVLJUVHY5ctJGjeeCS068UbZZhSOP0DdS2vwapuLqbwgHEK2wdNPu2QnAaOcI1nCsWOueMU770D58jBtmtusMw1LtsTx/p+rmLBsO9G5c/D+rbW5sU5JTFodNk88Ac2buwss8TvlG/GJ+Hi3d1Vyshu1OnQIXnwRHnkEoqOJT0ji/THL+HzqWsoVysUXXS+jZsm8bo1lo0au4MRTT7npxZde6l1nrhpWmdY5DYVba60x5iBwBEjE9fSIZA27d7u1EAMGkHTvfYxu253B24ux4q6BXLVrBd/P+JyY+TOh+PNQ7Coo1hJatszoqIOaco5kat26uZGmu+92G/6mMWV43sa9fPzXaiYs20GeyDCeblmZOy8re2rpZHCNtTFj3IbBDRq4mwSU8o2cs1274NdfYdgwV8xm40a3FcNvv7l1U56cMG3VLl74bREbdh+mY8MYXmpTlaid26F5O7e2av58qFXLdehKtuBVA8sYUxo3fN4VKAb8DNxsrZ3ix9hEfGPFCujXz5VZP3KE2AZNeGdTFCN/+I+7N83ky1k/U2D1clcFsE8f6NEjoyMOeso5kmklJLjGUK5c7mLo1lvdHjYpWGv5e+VOPp68hlnr9pA/Kpwnm1ei66VlyZcz/PT3jIuDdu1g4kS3ZqtRowB9GQHlGzkPy5fD66+7hlViolsvdeedbhQrd26oXx9rLbPW7uaDiauYvmY3ZQpF8f3djbi0QjSMGHHy/I8/1iyZbCi9KoLdcAmnCTAZeB0Ybq094vfIRHzlySexEyawoGkbXivfnCW5ilKnYlEGXl6eq1/9GpMjFL780q17iIjI6GiDmnKOZGpz5rjRqkaN4LPPoG5dd/Ow1jJuyXY+nLSKJVv2UzxfJC+3qcZtDUufPmJ13H//wR13uMXwX32lxlUAKd/Iedu6FUaOhAcfdH+/deqcmNK3cfdhxizeys9zY1m94yDRuSN48bqq3H5JGSLDQ920wd69Xbn2H35wBTAk20lvBOsZYAjQxVq7OQDxiFyYhAT4+Wc3YjV0KMsiC/Fr8x78flFHLIZeGyZxzd/DCZs2FaoVhS8+d1V3NI85s1DOkczn6FF3UfTee1C06GlFLKy1/LViB++OX8mSLfspWyiKPu1qcWPdkkSEnSW3vPyyK50cHe2mBzZv7ucvIqko34j3ZsyA6dNdsYorr4TY2BMb9a7YdoCxi7cxdsk2lm3dD0D9MgXo064WbWoXP7WDpVAhuO8+d52i6qDZVnobDVcFMMYUMMa0BAoCe4BZ2jNCMpUDB2DgQJewNm7kcPkK9P1iIoOTi1Pp0G4GrBlHg0m/YY4dc2scQj3lkPPly8ioJRXlHMl0Vq+GW25xI009eriCFvnznzi8MHYfvUcv4991e4gpGMW7HWrTtk4JwkLTaFjFxbkOoJtugoIFoWpVV13slVdOeU8JDOUb8dovv0Dnzm6D3oceghw5OBCRk2HT1vHz3FiWbd2PMVA/pgAvXleVltWLUbpglHuttW7LhVy53HTixx5TNeIgkO4aLGPMS8DznnN3AYWBBGPM29ba1/wcn0j6jhxxZUy3b2d3/Uv4uOX9DC5QnUJROXmuflF63NoRc/SoK6P81FNu4alkWso5kqmEhrqG0e+/ww03nHh618Gj9Bm7nGFzYimUK4LX21bntoYxhKfVsFq9Gvr2detA4+MhPNzlo44d3U0yjPKNpOvDD11HSOPG8PvvHDahDJy4ikHT1hF3JIHapfLRq211WtUoRpE8kSdfd/iwG5nu3dsVsWjd2jWw1LgKCumtwboFeBhXsvR3a22iZxO+G4EBxpjl1tof/R+mSCqrV7tFok88QXxYBAu7PsTgIwUZGxXDTftXM3bDb5T5+jMiI8LcJp0NGrgiFpKpKedIpjF+vJuyV66cK5QT5v65TE62DJuziTfHLONIQhL3XlGeh66qQJ7INIpXHD3qNiQfMsS9vmtXt4arYcMAfxlJi/KNpKt3bzc9+MYb4fvvmbn1MD0HT2XjnsNcU7Uoj1xdgVql8rtzk5PdaJUx8MEH8NxzrgO4QgX4+mt1pgSZ9Eaw7gGesNb+cvwJa20i8LMxJgfQA1DykcBZsADeegt++gkbHs63JS/mg1VH2W1qc/expcyf8ir5F86FIkVgxzYoVeqUXmfJ9JRzJGMlJ7s1Fp51nNx664nG1aY9h3nqpwX8u24PjcoVpPdNNalQ5PTS7CQkuFGqiAi3aehjj7nR8+LFA/pVJF3KN3J2hQtD164kfzGQ9/5ay4C/VlOmUBRDe1zCJeULuQbVtGkwaBCMHu02Ha9cGcqWhe7d3fVHs2YncogEj/T+i9cBbj3DsTHABz6NRuRMNmyABx6AMWNIzpOH2e3u5NnSV7Huv/20z3uYXr+9QtTKZS6pffyx258mp7YwyYLqoJwjGeXwYTfK9MsvbkpQ+/aAK2IxdPYmXh+1lFBjeKddTW5pUPr0TYI3b4ZPP3XrQWfNgtKlXaUxTQnKrOqgfCOpHTsGCxe6mS89enCgSzce+34+E5fv4NYGpXnlhmpEhYe6PNGrlzs3Xz647jrXQQNuKmCq7RskuKTXwMphrd2T1gFr7V5jjGpai/9Y6zYHjo6GAgVIXLGSKbc/zDPFLmc/4XQsnET/Tk2oGZ0DJn4KLz0Pt92mnqKsTTlHMsbGjW4a0Pz5rlrg448DEHckgeeGL2TMom1cVqEQfdrXpmT+FJ03SUluOuHAgW6dVnIyXH+9u0gDNa4yN+UbOdXOndChA8yeDWvWsC1nfrp9OYvVOw7yetvq3H5JGdexsn07dOniphAPHOiuPXLlyujoJRNJ70rUGGPKAWf6F0L/cojvWQvjxrmeof372TR5Jp9M3cAvt/QjNOEYvXf9S5vx3xIeEQ5PrHINqokTMzpq8Q3lHMkYq1e7kfKRI11PNLB4cxz3fzeXrfviebZ1FXpcXp6QkDRGrW64AQoUcI2y+++H8uUz4AvIeVC+kZPmzoWbb3bTegcOZHVILrp+/A9xRxIY3O1irojJA4MHw113ue0apk2D2rVPViUWSSG9BlYuYDVnTjLWt+FIULMW/vgDXnsNZs0isVRpRrTqwnN9/yIyKZH3d/5Dyz++IXTHDrj8crfwVIktu1HOkcA5dAgmTHBTea66CtatO7Gvzc9zY3nh10UUzBXBj/c2pn6ZAidf9/vv8OefMGAAxMS4C626dbVRedajfCNO377w/PMnGk5zo8tz1yczCA8N4cd7G1Pj4DZodI2bDlitmqsoWK9eRkctmVh6+2Bp91UJnN9/h5tuIiGmDL/2eImX89fHhkfQsWEMjx5YQoHb34VrrnENq6ZNMzpa8QPlHAmYsWPdaNPGjbByJVx0EeTNS0JSMr1HL+Or6etpXL4QH3aqS3Ruz2agu3bBPffAb79BjRqwb5/bv6pRowz8InK+lG/khE2b3NTezz9n4o5EHhw4k2J5I/mmeyNK//WHW9edI4crZNG4cUZHK1mAFqtIxrHW9R7v2wcdOrD+kmb881BvekXVILdN4OOtf3FxleLkuaE1JFeFKnOgfv2MjlpEsrItW+CJJ+DHH6FKFZg82TWucHtbPfjdPP5dt4fuTcrxXOsqJzcM/usvt9Ho7t1us+HHH3eVAkUk69m2Dd580623uvxyePddCA3ly+nreX3UUqqXyMeXd15M9Ocfuaqil1wCP/3kKhOLeCG9fbD+4uxD5NZae7VvQ5KgMHWqG4maMoVjderxgq3I8PlbyJ+zMp9s+4srR35NyN490KmTOz8kRI2rIKCcI34VH+/yyN69biryM8+4XmlgYew+7v1mLnsOHeP9W2tzU90UF1KHDrlF7AUKuI1D69TJmPjFp5RvgtDatfDRR/DJJ64QTYkScPnlxFtDr98W8/2/G2lerSgf3FaHqIgwNx2wa1f47DOIjEz//UU80hvB+vYMz5cEHgGifBuOZHuLF8PTT8PYsSQVLcrou5/j2QKNSFy4lTftSjoMeoOQPbvh2mvh1Vfh4oszOmIJLOUc8b05ntHvyEjo39+tl6pQAThZgv2VEUsonDsHv9x/KTVK5nOv27bN7YOTK5ebUlipkiqFZS/KN8HkoYfcNi4hIa7D5JVXoGJF1uw8yMPf/8fSrfu5t2l5el4eQ+if41yxm1at3E3kHKW3BmtQysfGmELAc7jN+X4EevkvNMlWju9uvmMHyf/OYtKdT/JE4ctIwtC5RiG639CAYiujYV59Vz1QaxqCknKO+NTixfDCCzBihFvjecMNbkqQx8Gjibz022J+/W8zl1eM5oPb6lIwl6dQxa+/wt13u1Gunj1do0yyFeWbbCw+3v0N//orDBni9sWsXh1eegl69ICSJTl4NJEBfyxn8LR15MoRyuBuDbiqSDi0bOH2sVu50pVhFzkPXq3BMsbkBZ4GHgJGAfWstWv8GZhkE7GxbiSqYEH29+rNwMSSfN9jEPuTDb33zeWmUYMJa9USujRxo1XjxmV0xJIJKOfIebMWpkxxe1mNGAF58ri1Fs2bn3La3A17efzH+cTuPcwTzSvxYLMKhIYYOHLErdH69FO30ehNN2XQF5FAUb7JRvbudeupPv/c7WlVsiSsWeOK0tx/PwA79sfz9bgVfPvvBvYdTqBdvVI806oyRfbvgsuvcls2/PCDGldyQdJbg5UTeAx4EpgMNLHWLvF/WJLl7dsHb78NH3yATU5mYbtudOvzF/sOHeWlg0u5/Y/BRKxb4xaOdu6c0dFKJqGcI+ctMdHtiZec7Cp+7d/vOnceeggKFTpx2oH4BPpNWMWX/6yjRP6c/HhvYy4uW9AdXLEC2rc/OZX5jTdUej0bU77JZjZscFOB9+xxWy88+KDbfiEkhENHE5m4fAe/zotlyqpdJFtL86pFeaBZBeqUzu/Whd9yi1tvOXYsNGuW0d9Gsrj0RrDWAaFAH2AOUNQYUzTlCdbaSX6KTbKqkSPhjjuw+/axofVNPFH9ZuaF5Ofykvnou3g4RT/uA7Vqud7lNm3c1EERRzlHzs2GDfDWW+6iaMUKV7RixAhXGTDq5BKaxKRkhv+3mb7jVrDz4FE6NYzh2dZVyBOZohLgjh2uSuDYsdCyZQZ8GQkw5ZvsID7era+MiXGbAHfuDLVrcywxmb+X7+S3/zYzcfl24hOSKZEvknuvKM8tDUpTNjrFesp589weeH/+6Ua7RC5Qeg2seFyFnfvPcNwC2rJeXK/xgQOQLx+2YkV216zHixd3ZGxYMW5J2MTLTUtT57pGEFsSLq7tFpiGaAsSOY1yjnhnzx54+WU3FcgYV+nr4EHXwKpZ88RpCUnJjFq4hQGTVrNm5yFql87PF10bULt0fndCcrIr1X7VVa5c85o1br2GBAPlm6zMWvjyS7fO8p9/oHx56NOHHQfi+Xb8Cr77dyO7Dx2jUK4IbmlQmutqFufisgUJCTFupGrwYNcw69QJHnnErblUERvxkfSKXJQNUBySlU2Z4vaJKF2aBR8M5s2/9/Jv48e5Ink3M2cPoNiksbCxPVx3hdtD4njpdZFUlHPEK8enAu3d6zb+feEFKF36lFP2xycwbPYmvvxnPZv3HaFy0Tx81qU+LaoVxRwfNY+Lgy5d3Kj78UqDalwFDeWbLGzfPrj3Xhg2zE3ni4gg7nACH01ezVf/rCchOZmrqxShU6MYLq9YmPDQEDfSNWY0DB8OP//sOoVbtHDXJMaocSU+lWk2GjbGtAI+wA3XD7TWvp3quPEcvxY4DHSz1s4LeKBy0sqVrsLWb7+RWKIk3198Ay8PmEZle5DxK0dQcdQwTFSUqwr4+OMZHa3IKZRzsqC9e91eVDEx0L073H77KaNVAGt2HuTr6ev5eW4sh44l0bBsQXq1rU6zykVcz/Vx8+e79Vbr18OHH0K9egH9KhJclG98aPp01yiKjYU338Q+/TQ/z9/Km4P/Yt+RBG6uW4qHrqpAuehckJQExzcL794dvv/eTQW8+Wb3uEmTjP0ukm1ligaWMSYU+AhoDsQCs40xI6y1S1Oc1hqo6Lk1Aj7x/JSMMHw43HorNjKSv7s+ysNFryQhRw4ealKeh2cMJcfon+CBB1xJ1MKFMzpakVMo52Qxe/e66YBffw2LFrkG1jvvnDhsrWX6mt18MXUtk1fsJCI0hDa1i3PXZeVO7mmV0scfu1H3ggXd9EBdZIkfKd/42DffuCUG//zD5sq1eHbIXKau2sXFZQvw2g01qFYiL2zZAs/3hq++cgUsLroIHnvMjVhfdZWK14jfZYoGFtAQWG2tXQtgjBkKtAVSJp+2wNfWWgvMNMbkN8YUt9ZuDXy4QerYMbfxZkwMCZdfwaobb+ehcq3YaHLy1r7ZXH11XQq2rAxNnoIut7uEJpI5KedkBceOwWefwWuvuUbW/fe7suse1lomLNtB/4mrWLQ5jujcETx+TSU6NYqhcJ4cp75XYqJbbxUR4aYDtWzp1m8VKRLgLyVBSPnmQm3c6Kb0Va/uyrC/8w4j1x3k+X5TSE62vH5jDTo3jCFk4wa4+wnXGZOU5DYLjo9373HxxRn7HSS4WGvTvAE3pLgffqbzfHED2uOGzI8/7gIMSHXOKFwJ1eOPJwINzvB+PXAVgebExMRYb7zyyisWt6DVAnbOnDl2zpw5pzz3yiuvWGutLV68+Inn6tWrZ6219p577jnl3M2bN9sRI0ac8txnn31mPcnzxK1NmzbWWmvbtGlzyvPWWvvZZ5+d8tyIESPs5s2bT3nunnvusdZaW69evRPPFS9e3D/fqVw5+3v58qc8d/NV7e2RSlWy7nfKjv+dAvSdjseUHmCO9S4PZMmcg/KNf77T6tV2RMmS6X6nnBddbK/oM8nWa3J1+t/pvvuC4m8zO34n5ZvzzzfWZt3/7vfcc4+1w4bZeiEhJ54rVry4ffqn+TbfZR1P/U6TJ9s5UVFZ4zvZ7PX3md2+k7f5xvOZaeacsyWE/Wnd98cN6MDpyefDVOeM5vTkUz+9965fv77XvyRJw8KF1l59tbVg4ytUsu88/K4t03Ok7f7kILu78RXuf6EKFaz95Rdrk5MzOlrJhM6UfFLfskPOUb7xgSVLTt5/7TVrR48+Jbcs37rf3j5wpi3zzCh76VsT7bDZG21CYtKp75Gc7HJSy5bWGuPy1JVXWjthQoC+hGQU5Zts5OBBa+++2/39Nmpk7Zo1dsW2/faadyfbss+Osn3HLbcJCYnWTpx48jVDh1q7aVPGxSxB50w552xTBLcZYx7CDWGHGWOaAadtWGR9s0dELJCyBFQpYMt5nCO+NHEitGhBct58jLqzJ09GX0qu3Dl57ZpKdF71N2FfLoZ+/dy0Hc1nlgunnBPMliyBnj1hzJiTFf1efvnE4T2HjvHu+BX8MGsjuXOE8eJ1Vbn9kjJEhoeefI+EBAgPd1MA333XTSt68UW44w5NWZbUlG8ys9hYaN7c7W333HPw2mv8tGAbL305jdw5wvjmrkY0yZ0A17dx+9ZNmeK2Wbj11oyOXAQ4+xqsbkAv4FEgAhicxjkW3+wRMRuoaIwpB2wGbgNS1/IeATzkmbvcCIizmpvse8eOwerVUK0a8ZdcyqKuD/Fo4cuJs6F8tu0vGte7iJyXtoBLYuCGG1xFLxHf6IZyTvCJjYVXX3X72eTJA336uHUWHglJyXwzYwP9Jqzk0LEkujYuy6NXV6RArhSdOnv3Qv/+br3W/PluXdVPP0HRohAaetpHiqB8k7kVLQpVqsCAARxs0pSXf13M8HmbaVy+EB/cVocif/8J3brBkSOuCuhll2V0xCKnOGMDy1o7HbgGwBiz2lpbwV9BWGsTPT1J43AlTAdba5cYY+7zHP8UGIMrX7oaV8L0Tn/FE5SsdT3HTzyBPXCA8b9P442/1rOpSHOeO7CQu0Z+SnjsJgi5w50fEqLGlfiUck4QOnoU6tZ1+1E98ogbbSpU6MThySt28PqopazZeYjLK0bzcptqVCx6ssgFu3a5UfQPP4T9++HGG90FF0CJEgH9KpK1KN9kQitWuBzwyScQHQ2//sqCTft4tP9UNu45zGPXVOThqyoS+nov1ylTr54ru165ckZHLnIar6oIHk88xpgYoCQQa63d5MtArLVjcAkm5XOfprhvgQd9+ZnisWyZ26dq3DiOXlSR925+ks9+WUbrxO2M/utT8s6bDbVrw9dD3IZ+In6mnJONrVwJP/zgpv/lyAFffOEaWWXKnDhlxbYD9B6zjCkrd1K2UBSD7mjAVVWKnNwgGGDPHihfHg4ehHbt3IVZ7doZ8IUkq1O+yWCHD0PfvvDmm26j74ULSWh6JR9OXMVHk9dQNE8OhvZoTMNyBd35FSu6TYb79YPIyAwNXeRMvGpgGWOKAT8CjYHdQCFjzEzgNmtt8M4Rzg4WL4Y6dUjOnZsxdz7N49GXkTt3Tl5vUZmO8bkJ+3YjDBzohuI11UYCRDknm7HWrZF4910YOdI1rDp0gGrV3KiTx6Y9h+k3YRW//hdLnshwXryuKl0alyFHmCf37N4N48a5TUYLFoTevd2eNimmFIqcK+WbDPTtt26NVWysywn9+7MoMSfPDPiHpVv3c3PdkrxyQ3Xy/TcbZm6Ajh3d33+n1DMsRTIXb/fB+hRYAFxrrT1kjMkFvOl5/gZ/BSd+kpjo1ik0aMCxylWZe98zPBNZi90mgs+3TqRx0RxEXtICKAPr16uHSDKCck52sW4d3HYbzJrlpv28/LLbhLxo0ROnbNl3hI8nr+bH2ZswxtC9STkebFaB/FGedVZHj8KAAfD6627EqmlTKFkSHn44g76UZDPKN4FkrStEA/D7727N5LffcvCSy+g/cRWDps2jYK4IPr29Pq2qFnabir/8spsK2L69K2Qjksl528BqAhS31iYAeBJQT9xiTclKJk6Exx/Hrl7N1D/+5eVZu1ifqzFP719Ij9GfEb5po6vCk5zs1lmpcSUZQzknK7PWbUpevLhbC5Url1tXcccdbgqQx4bdh/j07zX8PDcWgFsalOahqypQPN/Jc/jjD7c+a/VquPZaVwSjZMlAfyPJ3pRvAiE+Hr75Bv73P/jtNzeCPXAgNnduRi3eTu93/2bb/nhuu7g0z11blXwb1sCVHWDaNDdy9cknalxJluFtA2svUA3Xw3NcZWCfrwMSP1mzBp56Cn77jYTSZfi420u8P2YDV7KH3yZ9Qv45M0+us7ryyoyOVkQ5J6uaOROefdY1iFatcg2qSadWul6+bT+fTF7DqIVbCQ0x3Hpxae6/sgIl8+c89b127nQ91qVLu1LMLVsG8ItIEFG+8afdu12Fz/79Yft2aNDAjUQD/8Ul88b3/zJ3w16qFc/LR53rUb9MAdixw63NzJkThgyBLl1OjnqJZAHeNrD6ABOMMYOADUAZXIWbl/wVmPjQ9u1QowY2NJSJtz/Kw8WvJDwqJy9fU4kuRZMI/+FFl/y6d9c6K8kslHOymoUL4aWXYMQIN+XnpZdOyydzN+zh47/WMHH5DqIiQrnrsrLcfXl5iuZNMVK+bx8MGwb33AOFC7tR93r1tNee+JPyjb8cPerKre/aBS1awDPPQLNmrNxxkL5fz2H80u1E587BO+1q0r5eKUL/mwdlGrgc8uGHcP31p0wnFskqvK0i+IUxZg1u34ZauM3vOvpoAz7xh6QkN6zetCkJ0YWZ+dhrvHqsFJtDctJ/2980PbKFHK/+6M5duxbCvG1ri/ifck4WM3eu65XOl88VnnjkEcidGwBrLVNX7eKjv1bz77o9FIgK5/FrKnHHpWVOrrECVxVwwAB47z1Xtr1OHWjYEC65JGO+kwQN5RsfSkx066rGj4dPP3UFbfr1g1q1oGZNlm3dz4Dv/2PM4q3kjgjjieaVuKtJOXIvXgDNbnfXLf/952bU3H13Rn8bkfPm9VW1J9Eo2WQFkyfDY4/BggXM+v0vnlsFa5Kr8fjhRdw/5nMiNq53vUJHjrjhdzWuJBNSzsnkZsxwUwC7dnUjTAMGuMpenv3xrLVMWr6D/hNXsSA2jmJ5I3mpTTU6NixNVESKnLNvnxvtGjzYlWu+8UZ45RXXwBIJEOWbC3TkiKs43LcvbNwIZcu6Kb5FimA7deLfdXv47MtZ/LViJ7lzhPHglRXo3qQcBfbthB7d4euv3Yj1p59CjRoZ/W1ELpiurLOT1avh6afdOqtSMXx+7xv875+DXBJ6kJ/+6k/B2TNc4ho/Hpo3z+hoRSQr+vtv6NXLrasqW9Y1qsLC4EG3hY+1lonLdtBv4koWb95P6YI5efvmmtxUr+TJcuvg1mUUKgRRUa50e4cO8OSTULNmxnwvETk/8+bBdde5wjZNmri1Vm3akIBhzPzNDJy6jkWb4yiUK4Inm1eiS2PP6HV8/MmNxp95xpVrz5cvo7+NiE+ogZVdHD4MjRphjx1jQudHeLT4lYRG5eTFayrRtWp+In7r5Srw3H23RqxE5NwtXOhGxv/6y62J6NvXbfbpySfWWv5asYP3/1zFos1xxBSMok/7WtxUtyThoSF4TnIj7G+84Ua/Vq92a6tWrHBTiUQk60hIcFX9qlZ1Wyfcfz80bcrBo4kMnb6BwdPWsSUunvLRueh9Uw3a1StF5LF4GPa9qygaGelGrOrWhXLlMvrbiPiUrrSzsmPH4Jdf4LbbOBqRg0k93+GdnbnYYSLpv3UyTdfMJvylKe4CaPFiV3ZdRORcHD3qGj/WwrJlbj1Fjx4nyq1ba5m8cif9JqxiwaZ9lC6YM+2G1aRJ8NprMHWqK9/+5JNuOwhQ40okK0lKcpuGDxkC//7r1lsOHcqeQ8cYPG4FX89Yz/74RBqVK8hrbWtwdZUihMTtg3fecqNbO3e6Pa0aN4abb87obyPiF+fUwDLGhABFrbVb/RSPeMNat4i0Z09YtYqZh8N5endBYvcU5dkD87lrzBeEx25yaxn274eCBdW4kixJOSeDJCe7/afefhvKlIFvv3WLzjduPLEPzfHiFe9PWMl/G/dRMr+bCtiufqmTDavjpkyBa65xe2J9+KEbSdcee5LJKN94YcMGVzJ96lTXODp2jD2HjvHZlDV8M2MDRxKSaFW9GPc1vYjapfO7a5CnnoTPP4dDh9xedi+84BpXItmYVw0sY0x+4GOgPZAA5DLG3AA0tNa+6L/w5DSzZrn9rKZO5UiFSrx7//8YuDKSS6P2MXpsb/IunOcWh3/ztfazkixLOSeD7NvnNgL98EM3ha90abfx+HHh4Vhr+XvlTvpPXMW8jfsokS+S3jfVoEP90kSEeRpWSUnw669undW998IVV7hGWrt2alhJpqN84wVr4fvv4YEH3P0hQzhyaycG/bOOz/6ezaFjiVxfuwQPX1WBCkXyuGUL4P7ehw+Hm25ya8Rr1crY7yESIN6OYH2K24ivDLDU89wM4F1AySdQEhOhQweSjhxh6F3P83KhRhSPCuXta2vSoU5xQv/5FL780vUuaT8rydqUcwLFWncLCYF33nGjVo0auQbRLbecGLFKTraMX7qdjyevZmFsHCXz5+SNG2vQoUGpk8UrEhJcA+2dd2DlSqhf300nNAY6d87ALylyVso36bH2RIU/+803/HEoJ73fn8LmfUdoXq0oPVtWpmLRPG495ROvu3LrK1e6NZbLl6tjRYKOtw2sq4ES1toEY4wFsNbuNMYU8V9oArjN+fr3hxdeYE+i4afH3+WTWMhpk/h51c/Unj2JkCeWQkS4qw4okj0o5/jb9u2uNPKgQW7vqWuvhYcegvbtXcPI41hiMr/P38znU9ayasdByhSK4u2ba3JzvVInR6wAxo1zo1UbNrhF68OGuSlExmTAlxM5J8o3ZzJqlNvjrlgx+O03NiVH8OLIZfy9cidViuVhaI9LuKR8IdeweuoNN8oVGemqih496hpYalxJEPK2gRUHRAMn5iUbY2JSPhYfO3TILSZ/5x3soUOMyFOeFw8WJ/FQCO9tn0LLEV8ScvAgdO/uRrZEshflHH9ITnYNoYEDYcQIlzsuu+xEwQpKlnQ3IO5wAj/M3shX/6xn2/54qhTLwwe31eG6msUJC00xFfDgQVdauUgRV7zio49cY00NK8k6lG9S278fHn/c7U/3+OMk932Xr5fv552xKzAGXrm+Gl0uKeNywX//uUZYZCQ88YSbClhEbVMJbt42sAYCvxhjXgBCjDGNgTdxw+riS0lJbjFor16wbRuxTVvyZJ0O/Lu7MDeVhD4DHyF8y2Zo08ZN5alePaMjFvEH5Rxfio93Fz/WulGm+Hh49FFXbKJKlVNOXb3jIEOmr+eXebEcPpbEZRUK8U77WlxRMRpzvNGUnOzWVbz8Mlx8sasmVreu23xYJOtRvknp779dGfVNm+D559nyaE+eHvwv/6zezZWVC9P7ppqUXLsMfpzh9sGrUwf69HHLE9SwEgG8b2C9A8QDHwHhwGDgM+ADP8UVvEJCsF99xd7iMbzS/gVGRpXl+hxxDLuzMQ3LFYT1t8INN7g9J0SyL+UcX5g5003/mznz5J5T48bBRRe5+x4JSclMXLadb2Zu4J/Vu4kIDeH62iW4q0lZqpdIsfFnYiL8/LPbx2rJErf/zY03Bv57ifiW8s1x333nGkoXXQTTpjEubzl6fvIvCUnJvHVzTW6LTsTc09XlgZgYt04zLMxtuyAiJ3jVwLLWWqCf5ya+ZO3Jcsi//MI/cYaPb36Ff/Ykc+Ohdfw34XXyL56P6b4SKOj2nhDJ5pRzLtCMGW50acIEKFDAjVTFx7tGVdWqJ07bsPsQP87exE9zY9l54Cgl8kXyVItK3NYwhujcaexN1asXvP66e4/vv3cXVyqoI1mc8k0KLVvCU0+R8OJLvPn3Rr78fS41SuZlQPualP2iP/Tu7RpUr7ziphCGaTtVkbR4W6b9qjMcOgrEWms3+C6kIPL3324/iH/+4WhMWXr1G8V3SUVoemQL0+cNpcS0SW5NwwcfuP1jRIKEcs4FmD0bLr0UihaFvn3hvvsgV64Th+MTkhi7eBtDZ29k5to9hBhoVrkItzWMoVnlwifXV4HboHzAANeQuuoq11CrXx+uv15760m2EfT5ZupUeP99GDoUoqPZ8WIvHvhuHnM27KXbpWV57toq5Jj/H7z6qtu2oW/fE2s1RSRt3nY9DAKOX+HvBgp57u8AihljFgK3WWtX+Ti+7OnYMTfNb9w4EooVZ2jXZ3itSGPyRUbxdu183Nr+JkyuXPDWW/DIIxAVldERiwSacs65sNZN2atRwy02/+IL6NjxlIbV6h0H+XbmBobPi2V/fCIxBaN4qkUl2tUvRfF8OU++V1ISjBzpOnYmT3Zrt2rUcA2smBh3E8legjffzJ3rRq1iYmDHDhaRh7u/ns3+I4n071iXG3IehLBQl1cWLnS5QETSdS4NrHzAy9baI8aYnMBruMo7/XB7RXwMNPdHkNlGbCyUKgUREcQVLcmkTo/ybLErKJN0kEFH59Gg50vkyhHmepGaNXNTe0SCk3KOt5Yvd5t/zpzpSiWXLu1GmuDEpsCDpq1j6qpdhIcaWtUoTseGpbmkXCFCQtKo9HfllW4Pm5gYt59V9+5QqNDp54lkH8GZbzZvdp29hQvD338zbpfl0aHTKZQrB8Pvb0zVj//nOnonTXLrvtW4EvGacVOP0znJmJ1AcWttYornwoEt1trCxphcuGH0TNciaNCggZ0zZ07GBrF0qRtaHz6cLVNn8e6mUH79L5bS8fvot2YMdcb+hAkJgTVrNOwu2Y4xZq61tsE5viZL5pyA5pvdu+F//3NFLHLlco2hu++GkBDPpsDb6DdhFcu3HaBo3hx0bVyWWy8uffraqnnz3ObAffq4TYW/+86NWrVtq/UVkuUo33gpKQkaN4Zly2D6dEbZQjw6dD41S+bji1trUvjR+906y+7d3dYLOdJYkykiZ8w53v7reQi4GLez+XH1gcOe+8kXFl42tXKlWxT+/fck58rF5Jvu4smfVhNu4Ps1o2k0bhgmMRHuvBNeekmNK5GTlHPOJi4OKlRwP7t0cY2jokUBmLl2N2+MXsrizfspH52Lvh1qc0PtEqduCrxnj7t4GjzY7WGTMyfcfrtbX9W5cwZ9KZEME3z5ZsMG2LYNPvuMP4jm0aH/US8mP1/dWoNct7RzBXLeegueeUZ72omcB28bWC8D440xI4BNQCngeuBhz/GrgZ99H14WduAA1K9PcnIy027sxlNlmrMvZz46NorhwdoFKFLvPrdY9OWXoXz5jI5WJLMJzpyzerVrOAH06AGjR7tpxSVKuEZR0aIwbJjb2PeNN06ZthO79zBvjFrG2CXbKJEvknc71KZtnRKnFq0AN6Jerx4cPep+DhjgGlX58wf2u4pkHsGXb8qXh5UrWbwrnkc/nUGd0vn58s6G5Br2g5sS+NVXbi8sETkv3pZp/9oYMwdoh1sIuhJobK1d6jk+ChjltyizinXr3MVPz55st+FMeuQt+h8sSIIJ4X8bJ3DpthXk6D3F9QatW+cukkTkNEGZc779Frp2dZX7qlVzlQCTktxmnytXQsGCbppOQoKbyvfgg4CrCvj5lLV8PHk1BsNTLSpx9+XliQz3lE9PSIAffnCjXQ8/7DYWfvppaNfObRAqEuSCLt/MmQM1arCfUB784T8KRkXweZf65M4R5kbE69SBmjUzOkqRLM3rCfaeRLPUj7FkXevXu70hvvoKGxrKR/lr8eEmyHW4MP+LnUiz8UMJOXTQjVgdPAh58qhxJZKOoMo5mzfDQw/BJZe4ESuAbt3c7QySki2jFm6hz9gVbN53hGtrFuOF66pRMr+nIuDRozBwoJs+uHEjXHGF+4yQELeXlYicEDT55vBhaNECbryR565+iNi9R/jxnkYUeucNt+6yQQM1rkR8wNt9sAoCTwF1gNwpj1lrr/B9WFnEnj3w3HMweDA2JIQ5rW7hyfKt2LzB8nDefTzS/35CDuyH9u3dpnyqwCPilaDKOdbCvfe67RuGDIG8ec96+qGjifyxeBuf/r2G1TsOUrV4Xvp2qE3ji1JU+hs71k0x3LQJLrsMPvkEWrfWWgqRNARVvvntN9i7lwVXXs/oRVt5umVlGoz+wU05ttY1sETkgnk7gvU9kAMYxslFn8Hr+BSdyEiSxvzB/JYdePKi1uwJjaRb9DFu6tGUcnnDYd3fbhpPrVoZHbFIVhM8Oef3391aq/feg4oViTucwKhFWzgQn0hiUjJREWFERYSyNS6eFdsO8PfKnRxJSKJy0TwM6FSX1jWKExpi3MXR4cOuomDRom4kbPBguPpqNaxEzi548s2QIdgyZei5Mz9lCsE9uffB44+7zcN79cro6ESyDW8bWJcCha21R/0ZTKa3caOrqjNlCvtmzGbgjE1822UAifFHeWvLFK4d9x2huXPB07e4Bthnn2V0xCJZVfDknNhYqFaNpIceZsi0dfSftIp9hxNOO80YKFUgJzfXK8kNtUtwcdmCJ/ex+ucft66qQgX4+muoWxemTw/wFxHJsoIj32zeDBMmsLjbQ6zYeZjPO1QjotsNEB0NX37ppg+LiE9428BaiKuqs8aPsWRexxtWgwZhgUUt29HjzbHsS7C8uW0qN4z9lrA9u6FNG7ffVXh4RkcsktUFT8556CF48EE+nLiKfhNW0aRCNM+0qsJFRXIRFhLC4WOJHDyaSHTuHCcLVxy3eDE8/zyMHOkqDXo2GBaRcxIc+WbECEhO5sV89WhYriDN/xnpcsjo0dpMXMTHvG1gTQLGGmO+BLalPGCtHezzqDKTuXOhcWMssKRVe5686FpW5ChAy+pFeTFkPaVv/8AtGO3VCxo1yuhoRbKL4Mg5hw5BVBTLth1gwKTVtK1Tgn631sGkmNIXERZB/qiI0187eLDbBDRfPnjzTXj0UYiKCmDwItlGcOSb++5jbIGKLJh/lO+vqYgp2xAqVnTrM0XEp7xtYF0OxALNUz1vgeyTfI7bsAGWLIFrryW+Ri2Wdr6Plwo2YnloHp7fNZtry0RRvMtzYOtDlTluc04R8aXgyDndu2M3b6HnLW+SPyqcV6+vfrJxZS3s3etKtScmwvLlMHs2XHwxNGsGLVu6DcoffVS9zyIXJijyjQU+3JGDSkXCaFwoDEJD1bgS8RNv98Fq5u9AMoX1611P8FdfYQsW5Pufp/HhlA1sK9KCx/Yv4uc/vyTnmlVw5ZXw8rNuUYQaVyI+FxQ558gRGDWKNS1uZNHmOD7qVI8CuSJco6pAAUhOdsUqEhNPfd2TT7oGVsmSWpQu4gNBkW/69mXXvMUsKd2eb3OtxVTsAFOnun3xRMTnvN4H6zjjuldPzF+x1ib7NKKMsGGD2xdmyBBsSAhrb+xIzwqtmTtqBR3Mdl4ZO4DcC/+DqlXhl1/gpptUlUskQLJlzgEYNw4OHeLHMg0pXzgX15aJcls6LFkC//0HkZHw6acQH+/Or1gRatd2jS4R8Ytsm28GDWJvaB6iyyZz6ed9ICbG5RQR8Qtv98EqCQwArgDypzocetoLsprNm7HffsuGDl14tkJrZh6LomaRXHzZuhpXHtmCGbrXrXfo2tUNqYuIX2X7nAMwaRLJuXLxVXgZHqhVAvPYY26PmjfegDBPau7ePSMjFAkK2T7fLF0Ky5fzXcv7eW33LEI2b3bVRnU9I+I33o5gfYrbG+Jq4G9cEnoVGOOfsALHWsvfhSrw2as/M2Of4bLkA/yzeDAltuTCPPY1UARWr1YiEgmsbJtzTpgzhx0VqpEQEsYte5fCV1+5ioDPPpvRkYkEm+ydb37+GWsMf5ZrwAsjXnGbjzfL/rMiRTLSueyDFWOtPWSMsdbaBcaY7sB04Av/hed/h44l8ejQ+ZS0iYyPHUPFn4ZgQkLgiSfcInNj1LgSCbxsm3NOeOwxhkxcQ50CoZTs+SBUruyKVohIoGXvfPPLL6yqUIvLD20mYkssDB6oZQ4ifuZtAysJOL7Sep8xpjCwHyjpl6gCKHeOMH6vfJgy99+JiYuDbt3cwvFSpTI6NJFglm1zznGbW1zPJ/Mm8XqpCLjiCujRw627EpFAy775JiGBY40vY9DmSHLf0g4ebgUNGmR0VCLZnrfbdv8LXOu5Pw74ERgOzLnQAIwxBY0xfxpjVnl+FkjjnNLGmL+MMcuMMUuMMY9e6OemVPbKSzDNmsGCBW6tlRpXIhktW+ccVqxg9vCJGJvM5S0bufUQTZr49CNExGvZN9+EhzP2/hf5sfrVtKpRzG3zoNErEb/ztoHVBTcvGeAx4C9gMdDJBzE8C0y01lYEJnoep5YIPGmtrQpcAjxojKnmg892ihWD4cOhZk2fvaWIXJDsnXPef58WD3ekjjlI2a1rffa2InJesnW+Gbd4G8//O5T6vZ9xSx9ExO+83QdrX4r7R4DXfRhDW+BKz/0hwGTgmVSfvxXY6rl/wBizDDd0v9SHcYhIJpHtc86cOSwtUYn7l4yDul1g+3ZtFiySQbJzvolPSOKv5dt5Y/FEQvI00OiVSIB4W6Y9AugG1AFypzxmre16gTEU9SQXrLVbjTFF0omlLFAXN6R/pnN6AD0AYmJiLjA8EQm0rJRzzjnfHD2KXbiQ2fVu4PZZ46FlSzWuRDJQds43sXsP02T/Rgrs2ur28BSRgPC2yMUQoDYwEth+rh9ijJkAFEvj0Avn+D65gV+Ax6y1+890nrX2c+BzgAYNGmg8XCTryTI555zzzcKFmIQEDodHkmfHVujc91xCEhHfy7b5pkKRPHyWeyM2JARz/fXnEo6IXABvG1itgHIph9HPhbX2mjMdM8ZsN8YU9/TsFAd2nOG8cFzi+c5aO/x84hCRLCP75pw5bt18sYO7sBERmLZtffbWInJesm++Acxvv7lKpdHRvnxbETkLb4tcbARy+CmGEcAdnvt3AL+nPsEYY4BBwDJr7Xt+ikNEMo/sm3M6duR/T35Ijf1bMXXqQK5cPn17ETln2TffJCRAixbQvbtP31ZEzu6MI1jGmKtSPPwa+N0Y8wGphs+ttZMuMIa3gWGeTf02Ah08n18CGGitvRa4DFflZ5ExZr7ndc9ba7PHLusiEjQ5x+bLx9B8lYl7+WNqX3rW5Rgi4ifBkm8ID4f337/gtxGRc3O2KYKD0njuzVSPLVD+QgKw1u4Grk7j+S149qWw1k4DVPpGJHsLipwTu/cIuw8do0rlSlCxjD8+QkTSFxT5RkQyxhkbWNbacoEMRESCW7DknAWx+2i6di7Nf5wCdV6DyMiMDkkk6ARLvhGRjOHVGixjTB1jTOlUz5U2xtT2T1giEsyyc85ZGBtHm5X/UOSbQZDDX8s+RMRb2TnfiEjG8LbIxbdAeKrnIoBvfBuOiAiQjXNO50YxtD60AdOwoTb9FMkcsm2+EZGM4W0DK8ZauzblE9baNUBZn0ckIpKNc06Z8CRyr1kJjRpldCgi4mTbfCMiGcPbBlasMaZeyic8j7f4PiQRkWycc+bMAWvVwBLJPLJvvhGRDOHtRsPv40qY9gHWABcBTwG9/RWYiAS17JtztmyBPHmgYcOMjkREnOybb0QkQ3jVwLLWfmGM2Qd0B0oDm4AnrbU/+zE2EQlS2Trn3H47dOoEId5OIBARf8rW+UZEMoS3I1hYa38CfvJjLCIiJ2TrnKPGlUimkq3zjYgEnP6VFxERERER8RE1sERERERERHxEDSwREREREREfUQNLRERERETER7xqYBljchhjehtj1hpj4jzPtTDGPOTf8EQkGCnniEigKN+IiK95O4L1PlAD6AxYz3NLgPv9EZSIBD3lHBEJFOUbEfEpb8u03wRUsNYeMsYkA1hrNxtjSvovNBEJYso5IhIoyjci4lPejmAdI1VjzBhTGNjt84hERJRzRCRwlG9ExKe8bWD9BAwxxpQDMMYUBwYAQ/0VmIgENeUcEQkU5RsR8SlvG1jPA+uBRUB+YBWwBXjNL1GJSLBTzhGRQFG+ERGf8moNlrX2GPAY8Jhn2HyXtdae/VUiIudHOUdEAkX5RkR8zdsy7b8ZYzoYY3JYa3cq8YiIPynniEigKN+IiK95O0Xwb+BpYIcxZogxpqUxRpsUi4i/KOeISKAo34iIT3mVQKy171trGwINgLVAP2CLMaa/H2MTkSClnCMigaJ8IyK+dk49NNbaVdba14DbgIXAg36JSkQE5RwRCRzlGxHxFa8bWMaYi4wxLxpjlgDjcVV2mvotMhEJaso5IhIoyjci4kteVRE0xswGKgEjgKeA8dbaJH8GJiLBSzlHRAJF+UZEfM2rBhbQFxhhrT3iz2BERDyUc0QkUJRvRMSnztjAMsaYFKVKf/I8d9qUQmttsp9iE5EgopwjIoGifCMi/nS2Eaw4IK/nfiKQel8I43ku1A9xiUjwUc4RkUBRvhERvzlbA6t6ivvl/B2IiAQ95RwRCRTlGxHxmzNWEbTWbkrxsIO1dkPqG9DO/yGKSDBQzhGRQFG+ERF/8rZM+8tneP5FXwUiIpKCco6IBIryjYj41FmrCBpjrvLcDTXGNMPNST6uPHDAX4GJSPBRzhGRQFG+ERF/Sa9M+yDPz0hgcIrnLbANeNgfQYlI0FLOEZFAUb4REb84awPLWlsOwBjztbW2a2BCEpFgpZwjIoGifCMi/uLVGiwlHhEJJOUcEQkU5RsR8bX0pggCYIzJC7wKNAWiSTFP2Vob45fIRCRoKeeISKAo34iIr3lbRfBjoB7QCyiIm5e8EXjfT3GJSHBTzhGRQFG+ERGf8raB1QJoZ639HUjy/LwV6HKhARhjChpj/jTGrPL8LHCWc0ONMf8ZY0Zd6OeKSKamnCMigaJ8IyI+5W0DKwSI89w/aIzJD2wFKvgghmeBidbaisBEz+MzeRRY5oPPFJHMTTlHRAJF+UZEfMrbBtYC3NxkgKnAR8AnwEofxNAWGOK5PwS4Ma2TjDGlgOuAgT74TBHJ3JRzRCRQlG9ExKe8bWDdA6z33H8EOALkB3xReaeotXYrgOdnkTOc1w/oCSSn94bGmB7GmDnGmDk7d+70QYgiEmBZJuco34hkeco3IuJTXlURtNauTXF/J3D3uXyIMWYCUCyNQy94+fo2wA5r7VxjzJXpnW+t/Rz4HKBBgwbW+0hFJDPISjlH+UYka1O+ERFfO2MDyxhzlzdvYK0d7MU515zlc7YbY4pba7caY4oDO9I47TLgBmPMtbgd1/MaY7611t7uTYwikvkp54hIoCjfiIg/nW0Ey5vqORZIN/mkYwRwB/C25+fvp32Itc8BzwF4eneeUuIRyXaUc0QkUJRvRMRvztjAstY2C1AMbwPDjDHdcftOdAAwxpQABlprrw1QHCKSgZRzRCRQlG9ExJ+8WoMFYIwpBFwLFLPW/s+THEKstbEXEoC1djdwdRrPb/F8XurnJwOTL+QzRSTzU84RkUBRvhERX/KqiqAxpimwAugMvOx5uiKujKmIiE8p54hIoCjfiIiveVumvR9wq7W2FZDoee5foKE/ghKRoNcP5RwRCYx+KN+IiA9528Aqa62d6Ll/vCzoMc5hiqGIyDlQzhGRQFG+ERGf8raBtdQY0zLVc9cAi3wcj4gIKOeISOAo34iIT3nbO/MkMMoYMxrIaYz5DLgeaOu3yEQkmCnniEigKN+IiE95NYJlrZ0J1AaW4PaEWAc0tNbO9mNsIhKklHNEJFCUb0TE17yeX2yt3Qz0Of7YGFPLGPO+tbaDXyITkaCmnCMigaJ8IyK+dNYRLGNMlDHmdWPMSGPMe8aYvMaY8saYX4HpwI7AhCkiwUA5R0QCRflGRPwlvRGsj4C6wDigNVATqAIMAe6x1u7yb3giEmSUc0QkUJRvRMQv0mtgtQTqWGt3GGM+BDYCTa21U/0fmogEIeUcEQkU5RsR8Yv0ilzkttbuALDWxgIHlXhExI+Uc0QkUJRvRMQv0hvBCjPGNAPM8SdSP7bWTvJTbCISfJRzRCRQlG9ExC/Sa2DtwJUsPW53qscWKO/roEQkaCnniEigKN+IiF+ctYFlrS0boDhERJRzRCRglG9ExF+82mhYRERERERE0qcGloiIiIiIiI+ogSUiIiIiIuIjamCJiIiIiIj4iLHWZnQMfmWM2Qls8OLUaCCz79quGH1DMfqGtzGWsdYW9ncwmYHyTcApRt/ITjEq36QtO/03zkiZPcbMHh9kvxjTzDnZvoHlLWPMHGttg4yO42wUo28oRt/ICjFmVlnhd6cYfUMx+kZWiDEzywq/P8V44TJ7fBA8MWqKoIiIiIiIiI+ogSUiIiIiIuIjamCd9HlGB+AFxegbitE3skKMmVVW+N0pRt9QjL6RFWLMzLLC708xXrjMHh8ESYxagyUiIiIiIuIjGsESERERERHxETWwREREREREfCSoGljGmFbGmBXGmNXGmGfTOG6MMf09xxcaY+plwhg7e2JbaIyZboypndliTHHexcaYJGNM+0DG5/nsdGM0xlxpjJlvjFlijPk7s8VojMlnjBlpjFngifHODIhxsDFmhzFm8RmOZ/jfTGamnBOYGFOcp5xzATFmdM5RvrkwyjeBiTHFeco3FxBjts831tqguAGhwBqgPBABLACqpTrnWuAPwACXAP9mwhgvBQp47rfOjDGmOG8SMAZon9liBPIDS4EYz+MimTDG54F3PPcLA3uAiADHeQVQD1h8huMZ+jeTmW/KOYGLMcV5yjkXFmOG5hzlG7//91W+8UGMKc5TvrmwGLN1vgmmEayGwGpr7Vpr7TFgKNA21Tltga+tMxPIb4wpnplitNZOt9bu9TycCZQKYHxexejxMPALsCOQwXl4E2MnYLi1diOAtTbQcXoTowXyGGMMkBuXfBIDGaS1dornc88ko/9mMjPlnADF6KGcc+ExZmjOUb65IMo3AYrRQ/nmwmPM1vkmmBpYJYFNKR7Hep4713P86Vw/vzuudR1I6cZojCkJ3AR8GsC4UvLm91gJKGCMmWyMmWuM6Rqw6BxvYhwAVAW2AIuAR621yYEJz2sZ/TeTmSnn+IZyjm9kh5yT0X8vmZnyjW8o3/hG0OebMJ+Hk3mZNJ5LXaPem3P8yevPN8Y0wyWfJn6NKI2PTuO51DH2A56x1ia5jomA8ybGMKA+cDWQE5hhjJlprV3p7+A8vImxJTAfuAq4CPjTGDPVWrvfz7Gdi4z+m8nMlHN8QznHN7JDzsnov5fMTPnGN5RvfCPo800wNbBigdIpHpfCtZrP9Rx/8urzjTG1gIFAa2vt7gDFdpw3MTYAhnoSTzRwrTEm0Vr7W0Ai9P6/9S5r7SHgkDFmClAbCFTy8SbGO4G3rZsMvNoYsw6oAswKTIheyei/mcxMOcc3lHN8IzvknIz+e8nMlG98Q/nGN5RvzmXBVla+4RqTa4FynFxwVz3VOddx6oK2WZkwxhhgNXBpZv09pjr/KwK/ANSb32NVYKLn3ChgMVAjk8X4CfCq535RYDMQnQH/zcty5kWgGfo3k5lvyjmBizHV+co55x9jhucc5Ru//vdVvvFBjKnOV745/xizdb4JmhEsa22iMeYhYByuuslga+0SY8x9nuOf4qrBXIv74z6Ma11nthhfBgoBH3t6TxKttQ0yWYwZypsYrbXLjDFjgYVAMjDQWptmqc6MihF4HfjKGLMI9wf+jLV2V6BiBDDG/ABcCUQbY2KBV4DwFDFm6N9MZqacE9AYM5Ryjm8o35w/5ZuAxpihlG98w9/5xnhaaSIiIiIiInKBgqmKoIiIiIiIiF+pgSUiIiIiIuIjamCJiIiIiIj4iBpYIiIiIiIiPqIGloiIiIiIiI+ogSUiIiIiIuIjamCJiIiIiIj4yP8BgVNjcVPNsFUAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x288 with 3 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "## plotting\n",
    "plt.figure();\n",
    "fig, axs = plt.subplots(1,3, figsize=(12,4));\n",
    "\n",
    "titles = ['Grass%_100', 'Canopy%_100', 'Shrubs%_100']\n",
    "for i, ax in enumerate(axs):\n",
    "    XX = gam_100_1.generate_X_grid(term=i)\n",
    "    ax.plot(XX[:, i], gam_100_1.partial_dependence(term=i, X=XX))\n",
    "    ax.plot(XX[:, i], gam_100_1.partial_dependence(term=i, X=XX, width=.95)[1], c='r', ls='--')\n",
    "    ax.set_ylim(-0.5, 0.5)\n",
    "    ax.set_xticks(list(ax.get_xticks()))\n",
    "    ax.hlines(y=0, xmin=0, xmax=1, linewidth=1, color='black', linestyle='--')\n",
    "    ax.set_ylabel('Relative change of NDVI_100 (mean)', fontsize=12)\n",
    "    ax.set_title(titles[i], fontsize=12);\n",
    "    \n",
    "# non-overlapping subplots\n",
    "plt.tight_layout()\n",
    "    \n",
    "# save the figure\n",
    "plt.savefig('thesis_project/visualizations/eda/gam_ndvi_100_gcv_plot.png', dpi=300, bbox_inches='tight')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07788251",
   "metadata": {},
   "source": [
    "**Partial dependence plot: interpretating negative values of y axis**\n",
    "\n",
    "Above, we ploted partial dependence between dependent variables (Grass%_100, Canopy%_100, and Shrubs%_100) and independent variable (NDVI_100). We are observing **changes relative to a central tendency (mean)**. Therefore, negative values on y axis are relative to the mean of NDVI_100. For instance, lower levels of Grass%_100 predict NDVI_100 values below its mean. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "caf19fcf",
   "metadata": {},
   "source": [
    "- **GAM model NDVI_100 optimizing for Generalized Cross Validation (GCV) score**\n",
    "    - **Distribution**: Normal\n",
    "    - **Link Function**: Identity\n",
    "    - **Functional Form**: NDVI_100 ~ spline(grass%_100) + spline(canopy%_100) + spline(shrubs%_100)\n",
    "    - **Regularization penalty**: lambda = [31.6228, 0.0316, 31.6228]\n",
    "    - **Smoothness**: n_splines = 20\n",
    "    \n",
    "GCV score is ~0.0117, indicating a low probability of overfitting."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a80c6f2",
   "metadata": {},
   "source": [
    "### 5.4. Optimizing for combination of GCV, AIC, and DoF (simpler-version)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0e12f769",
   "metadata": {},
   "source": [
    "Otherwise, if we are interested in selecting a simpler model, we could also choose the model that performs best combining the scores for GCV, AIC, and DoF. For instance, one could prefer a model with slightly higher GCV, but with considerable less degrees of freedom. **Less complex models imply lower probability of overfitting**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "1f60f20a",
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "100% (125 of 125) |######################| Elapsed Time: 0:00:03 Time:  0:00:03\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "LinearGAM                                                                                                 \n",
      "=============================================== ==========================================================\n",
      "Distribution:                        NormalDist Effective DoF:                                     21.9796\n",
      "Link Function:                     IdentityLink Log Likelihood:                               -394042.6295\n",
      "Number of Samples:                         4673 AIC:                                           788131.2183\n",
      "                                                AICc:                                          788131.4554\n",
      "                                                GCV:                                                0.0057\n",
      "                                                Scale:                                              0.0056\n",
      "                                                Pseudo R-Squared:                                   0.7394\n",
      "==========================================================================================================\n",
      "Feature Function                  Lambda               Rank         EDoF         P > x        Sig. Code   \n",
      "================================= ==================== ============ ============ ============ ============\n",
      "l(0)                              [0.001]              1            1.0          1.11e-16     ***         \n",
      "l(1)                              [0.001]              1            1.0          1.11e-16     ***         \n",
      "s(2)                              [0.001]              20           20.0         1.11e-16     ***         \n",
      "intercept                                              1            0.0          1.11e-16     ***         \n",
      "==========================================================================================================\n",
      "Significance codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1\n",
      "\n",
      "WARNING: Fitting splines and a linear function to a feature introduces a model identifiability problem\n",
      "         which can cause p-values to appear significant when they are not.\n",
      "\n",
      "WARNING: p-values calculated in this manner behave correctly for un-penalized models or models with\n",
      "         known smoothing parameters, but when smoothing parameters have been estimated, the p-values\n",
      "         are typically lower than they should be, meaning that the tests reject the null too readily.\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/var/folders/6g/kgz5rml118s6lqw25w5sv5mr0000gn/T/ipykernel_74951/3678749054.py:10: UserWarning: KNOWN BUG: p-values computed in this summary are likely much smaller than they should be. \n",
      " \n",
      "Please do not make inferences based on these values! \n",
      "\n",
      "Collaborate on a solution, and stay up to date at: \n",
      "github.com/dswah/pyGAM/issues/163 \n",
      "\n",
      "  gam_100_2.summary()\n"
     ]
    }
   ],
   "source": [
    "# GAM model candidate 2\n",
    "warnings.filterwarnings(\"ignore\", category=RuntimeWarning)\n",
    "gam_100_2 = LinearGAM(l(0) + l(1) + s(2)).fit(X_train, y_train)\n",
    "\n",
    "# using gridsearch to tune the lambdas\n",
    "lam = np.logspace(-3, 5, 5)\n",
    "lams = [lam] * 3\n",
    "\n",
    "gam_100_2.gridsearch(X_train, y_train, lam=lams)\n",
    "gam_100_2.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "c82fcd9e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Figure size 432x288 with 0 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAA1gAAAEYCAYAAABBWFftAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAABuC0lEQVR4nO3dd9yN9R/H8dfnnvbee2dUJJRKovxEUmhSoqK0t6RUSkMDUckKaW/tZTQRyiw7e+95z+/vj++ROxlHzrjH+/l4nIdzznWdcz7n5n67vtf1HeacQ0RERERERI5fTLQLEBERERERyS7UwBIREREREQkRNbBERERERERCRA0sERERERGREFEDS0REREREJETUwBIREREREQkRNbBERERERERCRA0sERGJCDPraGbTzWyXma01sy/M7Kxo13UoZlbXzOaZ2SYzuzPD8/FmNtXMygf5Po+Z2RwzSzWzRw6xvaOZLTez3Wb2kZkVybAt0cxGmdkOM1tnZneF5MuJCGbWxcx+DOH7VTIzZ2ZxoXpPybrUwJJ/MbMrAgcQu81sQ+D+TWZmEfp8HdiIZDOB36GBwBNASaAC8BJwURTLOpIngXuAusCDZlYq8PxdwPvOuZVBvs9i4D7gs4M3mFkd4BXgavzPZA/+Z7LfI0B1oCLQDLjPzM4/5m8ikoOZ2Vlm9rOZbTezLWb2k5k1jHZd+5lZQTP7ysy2mdnrZhabYdtwM2sX5PtcFviee8xs0iG21zOzGYHtM8ys3kHb7wwc72wPHP8kHu93y8nUwJJ/MLO7gUHAM0Ap/H/6NwJnAgmH2D/24OdCQAc2ItmImRUE+gI3O+c+cM7tds6lOOc+cc7da2aNzOyXwAHGWjMbYmYJGV7vzOxGM1tkZlvN7MX9J3zMLMbMHgycLNlgZmMDn5fxjHJ3M1sTeO+7A9tKBQ40imb4nFPNbKOZxQOVgQnOudXAIqCCmVUAOgADgv3uzrkxzrkvgJ2H2NwJ+MQ5971zbhfwENDezPIHtncGHnPObXXO/QEMB7oE+9kiOZ2ZFQA+BQYDRYCywKNA0jG+j5lZuI6ZbwB+wx+LVALaBT6zMVDaOfdhkO+zBX8S66mDNwTy9GNgHFAYGAN8vD9nzawlcD9wbqCGKvifk/xHamDJ3zIcBN3knHvPObfTeb855zo555LMbLSZvWxmn5vZbqCZmV1gZr8FrvaszHi1yMxymdk4M9scOHj61cxKBrZ1MbOlZrbTzJaZWafAy3RgI5K9NAZyAYc7UEgD7gSKBfY9F7jpoH3aAA3xJ14uA1oGnu8SuDXDHxTkA4Yc9Npm+BMm/wPuN7PznHPrgEmB99rvKuAt51wKMBf4n5mVwx9wLAFeAO4LbA+FOsCs/Q+cc0uAZKCGmRUGymTcHrhfJ0SfLZIT1ABwzr3pnEtzzu11zn3tnJu9fwczezZw4maZmbXK8PwkM+tnZj/hT8JWMbO/zOy8DPs8YmbjDvrMaw8+oRPYt5H5LtI7zGy9mT0f2FQZmOicSwJ+CHxOLP545/Zgv6hz7lvn3DvAmkNsPgeIAwY655Kccy8ABjQPbL8GGOmcm+ec2wo8ho55josaWJJRYyARf5bjSDoC/YD8wI/AbnyDpBBwAdDDzC4O7HsNUBAoDxTFXw3ba2Z58QcrrZxz+YEzgN8Dr9GBjUj2UhTY5JxLPdRG59wM59wU51yqc+4v/NXlpgft9pRzbptzbgUwEagXeL4T8LxzbmngZEkv4Ar75ziIRwNXzeYArwJXBp4fg29U7b8afyXwWmDbPUAPYDy+8Xcm/mTNUjP72Mwmm9ml/+WHkUE+YPtBz23HZ2u+DI8P3iYiwVkIpJnZGDNrFfj/PaPTgAX4kzv9gZH7r44HXA10x//eLQ/yM/91Qifw/CBgkHOuAFAVeCfw/FzgPDPLDTQB5gG3AV8Ejk1CoQ4w2znnMjw3mwPHNf84JgrcL5nxCr8cGzWwJKNiHHQQZL4/7zYz22tmZwee/tg595NzLt05t885N8k5NyfweDbwJgcOjlLwB1fVAmePZjjndgS2pQMnmllu59xa59y8wPM6sBHJXjYDxewwg7/NrIaZfWq+//8O/DitYgftti7D/T0c+D0twz8PfJbjz9SWzPDcyoO2lwnc/xiobWZVgBbAdufcNADn3HLnXGvnXP3Afn3x2fQs8DbQFnjeMozd/A92AQUOeq4APu92ZXh88DYRCULgeOMswOF7omw0s/H7e9IAy51zw51zafgTLqX5Z3aMDlzVST2GE7yHO6GTAlQzs2LOuV3OuSmB50fiT0RPxV/BmoVv2A0M9Bj63swe/08/gAOOdMxzqO377+u45z9SA0sy+tdBkHPuDOdcocC2/f9e/jEGysxOM7OJgbEL2/FXqfYfHL0GfAW8Fbhk3t/M4p1zu4HLA/uuNbPPzKxm4DN1YCOSvfwC7AMuPsz2l4E/geqBs7sP4LuvBGMNfqzkfhWAVGB9hufKH7R9DYBzbh/+LHIn/AHNaxxaH2CEc249cBIw3Tm3HVgFVAuyzkOZh+/yCECgoZcILAx001mbcXvg/jxEJGjOuT+cc12cc+WAE/EnWAYGNq/LsN+ewN18GV4e7JjvjA53Quc6fJfFPwPDJdoEPnefc667c+5k59z9+K6BD+BzKRZ/wvo0O75x4Ec65jnU9v33ddzzH6mBJRn9gh/4ebRZvdxBj9/AX20q75wrCAwlcHAUGMj+qHOuNr4bYBt8d0Kcc18551rgzxj9iT+7dDAd2IhkcYHf2T7Ai2Z2sZnlMT8raCsz648/S7oD2BU40dLjGN7+TeBOM6tsZvnwV7/ePqg74kOBz6wDdMWfqNlvLH6sQVv8APB/MLPa+PELLweeWgY0D5wBrw6sOFJxge+ZC///bVxgXOr+yYFeBy40syaBbtN9gQ+cc/sPasbiJ/opHPi5dANGH/UnIiKH5Jz7E/87dGKwLzno8W4gT4bHpfi3w53QWeScuxIoATwNvBf4vf9boBFlzrkvOXDM44DpwMlB1nwo84CTD+r+eDIHjmv+cUwUuL/eObf5OD4zR1MDS/7mnNuGnzXmJTO7xMzymZ+hqx6Q9wgvzQ9scc7tM7NG+DFaAJhZMzM7KXBAsQN/iTzNzEqaWdtAuCThz56kZXxTHdiIZB/OuefxM4E+CGzEn+W9BfgIf4W6I/5s6XD+2QA6mlH4K0/f4zNiH3DrQftMxs8q+h3wrHPu6wx1/YTvrjwzMP7rYC8Ctwe6EIEf43Ub/oDkicBkGUcyHNiL7ybUO3D/6sBnz8NfxX8d2IDP0oyTezyMH4O6PPAdngkceIlIEMysppndHRjTjfllXq4Ephz5lYf1O36MZ7yZNQAuOcQ+hzyhY2ZXmVlx51w6sC2w79/HPYHjlafwQyPA59k55mf6OxNYeqTCzCw28B5xQEzgmCc+sHlS4LNuM78MzS2B5ycE/hwLXGdmtQPj1B5ExzzHxzmnm27/uOEvS0/Dj3PYiO8X3B0/Tfto4PGD9r8EfwCwEz8d6hBgXGDblfgBpLvxXXZewP/yl8YfMGzHB80koPZB7zsROC3D47rAfGATcFcQ32M0/uxTxluXDNs74htpu/FdEYtk2JaIP3DbEaj7qJ+nm266Za4bfpIcB8QdZb8JwPXRrlc33XQL7Q0/Lfs7wOrA//Wr8ZPoFMBfuf7xoP0dfsw4geOS6w/aXiVwTLQLvwTMCxmOd/bnTXf8Vat1+Mm59r92HP5Eyi78CZqLD3rvvsC9GR4XBL4OHCe9AcQe5bt2OcQxz+gM208BZuBP8swETjno9XcFjnd24MeOJUb77y8r3yzwQxUREclWzKwS/ixwvDvMDIbmFxz9Bt/FWeMNRETkuGWaLoJmdr6ZLTCzxWZ2/2H2OcfMfjezeWY2OdI1ikj2ocwRMxsDfAvc8V8bV4FuxrsOdQtttZKVKW9EcpZMcQUrMC5mIX6a3FXAr8CVzrn5GfYpBPwMnO+cW2FmJZxzG6JRr2QOZtYE+OJQ25xz+Q71vAgoc0QkcpQ3EipHOHHTyjn3Q0SLkSM65JokUdAIWOycWwpgZm/hZ7Kbn2GfjvhJCFYAKHgkECZqSMl/ocwRkUhR3khI6ORx1pFZGlhl+ee6Aavwq2tnVAOIN7NJ+JmWBjnnxh7qzcysO36QIXnz5j21Zs2aIS9YRIIzY8aMTc654tGu4yAhyxzljUjmobwRkUg6XOZklgbWoRaUPLjvYhxwKnAukBv4xcymOOcW/uuFzg0DhgE0aNDATZ8+PcTlikiwzGx5tGs4hJBljvJGJPNQ3ohIJB0uczJLA2sV/1yYrRyBhdkO2meTc243sNvMvsdP2/2vBpaIyFEoc0QkUpQ3IjlMZplF8FeguplVDiyodgUw/qB9PgaamFmcmeXBX17/I8J1ikj2oMwRkUhR3ojkMJniCpZzLjWwqvRXQCwwyjk3z8xuDGwf6pz7w8y+BGYD6cAI59zc6FUtIlmVMkdEIkV5I5LzZIpp2sNJfZRFosvMZjjnGkS7jkhQ3ohEl/JGRCLpcJmTWboIioiIiIiIZHlqYImIiIiIiISIGlgiIiIiIiIhcsRJLswsDmgLXICfLrQQsA2YBXwBfOScSw1viSKSUyhzRCRSlDciEi6HvYJlZjcAS4EbgCVAP+DGwJ9LgG7A0v2z4IiIHA9ljohEivJGRMLpSFewagCNnHPrDrHtQ+AJMysN3B2WykQkp1HmiEikKG9EJGwO28Byzh01VJxza4F7QlqRiORIyhwRiRTljYiEU9ALDZtZQeAEIF/G551zE0JdlIiIMkdEIkV5IyKhFFQDy8y6AC8Cu4A9GTY5oEroyxKRnEyZIyKRorwRkVAL9gpWP+AS59wX4SxGRCRAmSMikaK8EZGQCnYdrDjg63AWIiKSgTJHRCJFeSMiIRVsA+tp4EEz08LEIhIJyhwRiRTljYiEVLBdBO8ESgH3mdnmjBuccxVCXpWI5HTKHBGJFOWNiIRUsA2sq8JahYjIPylzRCRSlDciElJBNbCcc5PDXYiIyH7KHBGJFOWNiITasayDVQ9oAhQDbP/zzrk+oS9LRHI6ZY6IRIryRkRCKagBnWbWHfgJaA70BE4C7gaqha80EcmplDkiEinKGxEJtWBnzLkPON851w7YG/jzEiAlbJWJSE6mzBHJ6RYtgsWLI/FJyhsRCalgG1glnHM/BO6nm1lMYEG+C8NUl4jkbMockZxq6VLo2hVq1oTevSPxicobEQmpYBtYq8ysUuD+QuAiM2sCJIelKhHJ6ZQ5IjnN8uXQrRuccAK89Rbcfju88AI79qWwYvOecH6y8kZEQirYSS76A7WAv4C+wHtAAnBbeMoSkUxp+3YYPBhOPx3OOy+cn6TMEclphg6FsWPhxhuhVy+2FyrOqJ+WMeqn3zmpbEHe6HZ6uD5ZeSMiIRXsNO2jM9z/wswKAwnOuV3hKkxEMpGtW+GFF2DgQNi2De6/P6wNLGWOSA6wdi089RS0agXnnw/33Qc33cS2oiUZ+eMyRv80h51JqfyvdkluO7d62MpQ3ohIqB3LNO1FgdZAaedcfzMrZmaFnHOrwleeiETdwIHw8MOwYwdcdBH06cO+k+qSK8wfq8wRyaY2bICnn4aXXoKUFChTBs4/ny0JeRkxbz1jfp7A7uQ0Wp1YilubV6d2mQJhL0l5IyKhFOw07U2BBUAn4KHA09WBl8NUl4hE08aNkBwYfhAXBy1awO+/s/3NdxmyLT9nPDWBGcu3hO3jlTki2dRzz0Hlyv7EzWWXwYIFbLr5Dp784g/OenoCL09ewjk1S/DVHWfz8hV1qT3xE7j0UkhPD1tJyhsRCbVgJ7kYCFzunDsfSA08NxVoFI6iRCRK1q/33XQqV4YxY/xzN9/MptGv88y6RM56agLPfr2QuuUKkich6Avg/8VAlDki2cPWrQdO2OTN66+Ez5/PhiGv8PgfSZz19ASGf7+UFrVL8s2dZ/Nihzqc8PEbUKMGXHUV/PknrFsXzgoHorwRkRAK9gipknPuu8B9F/gz+RheLyKZ2dq18MwzfpB5UhJceSWcfTZrtu1l2PdLeevXFSSlptP6xNLc1KwqdcoUDHdFyhyRrG77dhg0CJ5/3o+1uvFGuPFG1nfswtDJS3jjtYmkpKVzcb2y3Ny8GlWL5/OvW7fOzyBYty4MGAAXXggxwZ4P/k+UNyISUsGGx3wza+mc+yrDc+cBc8JQk4hEWocOMG0adOoEvXuzrEhZhk5awgdjJ+IctDulLDeeU5WqcSnw8otQpQpccUU4K1LmiGRVO3f62UaffdZfvbroIjjzTNZu38vQSUt489eVpKU72p1SlluaVaNSbDIMGQAzZsCHH0KpUjB7NlSvDmaRqFh5IyIhFWwD627gUzP7DMhtZq/gF+C7KGyViUj4rFjhx0I8/DAUKeJnCCxcmPm5i/PSpMV8PmcS8bExdGxUge5Nq1K2UG748ks/FmLXLujRI9wNLGWOSFbVvj18+y1ccAE8+iirq9bmpYmLefetSaQ7xyWnluOmc6pRIXUnPPuYn+xi505o0wZ274Z8+Xz3wMhR3ohISAU7TfsUM6uLHwA6ClgJNNLsOiJZzF9/wZNPwquv+sfnnQcXXsiM4lV5aeJivvvzT/IlxtH97Kpcd1Zliq9cAovnQYMGcOqp/sDprrt8150wUuaIZCF798KwYXD11f6EzaOPwuOPs7L6Sbw0aQnvfTARgEsblOemc6pSrnAemDgRWrf2Y7MuvRR69Qp7rhyO8kZEQi3o/sXOudX4xfhEJKtJTfXjH8aM8WMZunXD9ezJz8l5GDJsCr8s3UyhPPHc1aIG15xekYK/TYOO98Knn0KzZjBhAhQvfmDiiwhQ5ohkcklJMGIEPPEErFkDuXND9+4sP6EuL05czAcfTyLGjCsaVqDHOVUps2ElzJkOZ58NjRrB9dfDbbf5roBRprwRkVAKqoFlZgXxK5qfAuTLuM05978w1CUiobBxo28YxcX5Aec9epB+7318uyOOFz9ZwqyV2yiRP5EHL6jFlY0qkPfHydDiGvjlFyhaFB55BG65JeJlK3NEMjHnYPhweOwxWLUKmjSB119n2YkNGfLOLD76fTVxMcZVp1fkxqZVKbXsT7ihC7z7Lpx0Evz+u59NcPDgaH8TQHkjIqEX7BWsd4FY4ENgb/jKEZGQmD8f+vWD997z96tWJfXNt/h0zjpeem8xC9fvokKRPDzR7iQ6nFiMxPg4SIiDBQv8jIIvvADXXQd58kTrGyhzRDIb5/ykE2bw2WdQrhy8+iqLTzqNFyct4eOvJpEQF0OXMypxw9lVKLF4PnS6xO9boAD07Al33hmpiSuOhfJGREIq2AbW6UBR51xKOIsRkeM0d64/q/zuu75xdPvtJOXNx/tTVzB08hJWbNlDjZL5GHB5XS6slI+4kSOg/QB46CHfhfD666F7d3/FK7qUOSKZRWoqvPGG7wr4ySe+S9+4cSza7Rg8cQmfDPyeXHGxXHdWZbo3qULx3LEQHw9LlsCUKfD443DzzVCoULS/yeEob0QkpII9ivoRqAXMDmMtInI8tmzxk1HEx0PPnuy59XbeWLKH4aPmsH5HEnXLFaT3BafSorAjZvAL8PLLvttg8+ZQp45/j4SE6H6HA5Q5ItGWlgbvvOO7Ci9cCPXqwfbt/LluB4O/W8znc9eSOz6W7mdXoduZlSj23Zdw7jXQtq0/adO+vZ/IIm/eaH+To1HeiEhIBdvA6gJ8bmZTgfUZNzjn+oaiEDM7HxiEv0w/wjn31GH2awhMwa+6/l4oPlsky5oxw59RfuQRP3vXO++wo/5pjF6wg1dHzGbrnhROr1KEZy+ty1nVimFmfoD5Tz/5ta/uu883yjKfLihzRKInNRVOOw1mzoQTT4QPPmB+o+a8MGExX773A/kS47jpnKpcd3oFinzyATS+xHdHrlLF3wBiY7NC4wqUNyISYsE2sPoB5YG/gAIZnneH3PsYmVks8CLQAlgF/Gpm451z8w+x39PAV/9+F5EcZNo06NvXj20oVAhuuIGN+YowMr4G44b/zq6kVM6tWYKbmlXl1PWLoc9tfkB5sWIwYAAULAjVqkX7WxyJMkck0pyDH3/0k1bExfm17u67j7ln/I9BE5fwzeAfyZ8Yx23Nq3HtWZUplCcBrr3WL/tQpw6MGweXX54ZuhgfK+WNiIRUsCl4BVDDObc2THU0AhY755YCmNlb+AX+5h+0363A+0DDMNUhkrktX+7HSn35pb9i1a8fa666jlembeStX38nOS2dC04qzU1Nq1J79s/Q5T6YNMk3wmbP9t0BTz012t8iGMockUhxDr74Avr08VfFf/wRzjyTWVd044XvFvHdiz9TIFccd55Xgy71ilPwtVeh+sVQubKfZfSii+DCC/0SEFmT8kZEQirYBtZSIJyDP8viF/bbbxVwWsYdzKws0A5ozlHCx8y6A90BKlSoENJCRaJi61YoXNg3qpYtg6eeYullnXl5+gY+HDoDM2h/SjluaFqFKnljoHFj36AqWxaeew66dYP8+aP9LY5Flskc5Y1kWc7BN9/4htXUqb7B9OqrzCxdgxdencakBRsplCeee/5Xg861ClJgxCtwxSDYvNm/9q67oH59f8valDciElLBNrBeA8ab2WD+3T95QgjqONScrQdfmh8I9HTOpdlRpnh1zg0DhgE0aNAgJJf4RSLOOX/1qW9fWL/ezxCYPz/zvv2Fl75fxufDZpIYF8NVp1ek+6klKfP7VCh+sn9t8+b+4OfKKzPTxBXHIstkjvJGsqw9e6BjRz9OatgwZpzTloHf/8UPw6ZROE88951/Ap0bVyJfv75w8UDYuRPatIEHHvAncbIP5Y2IhFSwDaybA38+cdDzDqgSgjpW4fs/71cOWHPQPg2AtwLBUwxobWapzrmPQvD5IpmHc/Ddd/Doo76rTunS0LMnM5ZsZMiPy5m4YCP5E+O4sWlVrj8hH0VHD4euQ2DbNvjrLyhf3o+zytqUOSLh8MMPMGoUjBjhG1bffsu0XCUZ+MNyfh45naJ5E+jVqiZXVclN3vJl/GvWrfOzAfbqBXXrRrf+8FDeiEhIBdXAcs5VDnMdvwLVzawysBrfH7rj4Wows9HApwoeyZY+/dRPc1yuHG7wYH5sehFDfl7F1JHTKZI3wXfXqZyLAs89De1Hwd69fgxEz56+cZUNKHNEQuyXX3xXwG+/hVKlcIsX80tsUQZN3cvUZTMpli+RBy+oxVWF9pFrQD947TU/22ijRjB0aFYeX3VUyhsRCbVMMdWPcy7VzG7Bz5wTC4xyzs0zsxsD24dGtUCRcHLOzwa4c6fv0teqFemjXuWbus148eeVzH59NqUK5OKhNrW58qRi5CmYH9auhbFj/f733AO1akX7W2QpyhzJMTZtgs6d/SQWxYvjnnmGX/53GQMmruLXvxZTIn8ifdrUplPiFhKfuR/eew8SE6FHDz+GE7J14yoSlDciOY85d+guvGb2K9Af+Ng5l3yI7QnAxcDdzrnTDt6eWTRo0MBNnz492mWI/Ft6Oowf78dY/fYbnH46qT/8yCdz1vLSxCUs2rCLikXzcOPZVeiwfQEJzz4Lycl+XBb4BlkWmLjCzGY454662FZ2yBzljWQa+yfGSUuDpk1xF1zAjy2vYMCUNcxcsY1SBXJxU7OqXNagPLlSk31jKjUVbr4Z7rgDSpaM9jf4T5Q3IhJJh8ucI13BugboC7xsZjOBBcBOID9QA6gPTMAv0Ccix2LyZLjtNj/TX7VqpIwYyTsnnM3Q5yezcsteTiiZnxcuOZELFv9C7M33+qmTS5XyBz5paX4BzyzQuDpGyhyR4zV3rl94/PvvYelSXN68TBz+HoMmLGHWO/MpWyg3/S6uw6Xb/iRhxGNw2nCIzw0ffggnn+wbZTmD8kZEwuawDazAAniXmFkp/OJ4J+EHXm4FxgJXO+c2RKRKkewgLQ327fMDy9PTYd8+kka+ymuVzmDYLyvYsGgB9coXok+bOpxbswQxw4f5Na+qV4dhw+DqqyFXrmh/i7BR5ogchz//9A2rd96BfPlwd9zBxD82MGDKGuas3k65wrl58uI6XLJ6JvG3XQa//gplyvi19SpXhqZNo/0NIkp5IyLhdNQxWM65dfgpTEXkv0hNhbffhscfh5YtYeBAtjc6kzEvfMyoqSvYtnARZ1YrypCWFWn45dvY3NVQ+zLo1AlKlPATXsTGRvtbRIwyR+QY/fkn1KkDefLgevViQutOPDt9M3+8/wcVi+ah/yUn065QMvEXt4F586BKFXjlFbjmGj/eKgdT3ohIOGSKSS5EsqXUVHj9dejXDxYtgpNOYlujxrz8+R+Mm7Kc3clpnFerJLfXSOSkd1+FO4bB7t1wyy1w2WWQLx+0axftbyEimdHSpTBlil/HqmZN0gcP4bsTm/DczC38+ckyKhfLy4CLanJhrp3EnVIeUlKgUiW/htVll0Gc/vsXEQkXJaxIuNx7LwwcCPXqsWnsm7yQtzZvzVxNatpS2pxchpuaVaXm8EFwfV8/k+CVV/rXnHxytCsXkcxq+XJ/0ubVVyFvXtIuaMNny3YyeF9tFn2+nCrF8zK4bQ1a//IJsZd08t2Rly3z3Ys//TTa1YuI5AhqYImESlISjBkDjRvDSSfBTTex5pTTeS7xBD6etQazVXQ4pSy3Ja6nTJPKUKgA1K4NN90Ed90FFStG+xuISGa1bh089hgMHw5mpN9wA1+37cozI2ayZONuqpfIx8sXVKHlpPeIaTsINm+GZs38Fasc3g1QRCTStLiFyPFKSoKXXvKTUdxwA7zxBnNXb6fH1B2c+Uc+Ppu7lqtPK8/U2jt56rkbKdO2JYwY4V/boQMMGqTGlYgc2v6lVHbsgFGjSO/alc8/+J5zK13CjRPWERcTw4sd6/PVHWfTKmk1MX36+JM8P/0EEybAeeeBWXS/g4hIDnPUK1hmVhToANTBT1+6E5gHvO+c2xze8kQyuREj/Mxdq1fDGWew4LHneSK1PJMH/0j+xDhualqFG5d+T/6ed8KCBX62rhdfhC5dol15pqXMEQE2boT+/WHNGnj9dVKqVuOzT6bx/MxNrPhhE7VLF2D0uSVp+vFobFNBOLkfnHsu/PEH1KwZ7eqzDOWNiITDEa9gmdm5wGLgqsC+awADOgGLzKxZ2CsUyWz27DlwVnnxYly1avz+6rtcelV/Wv6Rm7lrdtCzWSV+vL85955fi/zjP4Q8eeCtt2DhQt8lME+e6H6HTEqZIzne5s3Qq5c/GfP886TFxPLWz0tp9uwk7vh2BQVyx/H62YX57PdRnNO6MfbKK/7qFvgrVWpcBU15IyLhcrQrWIOB65xzHxy8wczaAS8BtcJRmEims2sXvPwyPPssvPoq6ee34uvLbmJI2QuY++cOShfcx5OnF+OSH98jvtOrfnHgihX9FO0FCqibTnCUOZJzffsttG8Pu3aRdtllfHZxN57+y1g9/g/qlitI34vq0Oybd7A2t/txVT16wD33QIUK0a48q1LeiEhYHK2BVRH47DDbPgdeD205IpnQjh2+W99zz8HmzaS3aMGErcaTAyazZONuKhXNw0sN8tDyi9eJfeQ1Pz37pZf6hYUBChaMbv1ZizJHcpYdO2DtWjjhBDjlFNLaXMhnrTvz5Mo41v6+j1MqFGJwxb2cckoxrGZJSGsOPXvCHXdAyZLRrj6rU96ISFgcrYE1FXjczB5xzu3e/6SZ5QUeDmwXyb6cgyZNYPZs0lq14uv23ei3uSCr5uylZqkYXrjyFC4om0hs2TIQEwPXXQd33w1Vq0a78qxKmSM5w65dMHiwvyJepQr7fp7Cm3/uYOiJXVk/N4kGFfIxrPQWThz9DDZ5MnTtCqNG+QWFn3wy2tVnF8obEQmLozWwugJvApvMbCmwHSgAVAF+B64Ia3Ui0bB1KwwbBnfeCQkJ7O37OF+uS+WJTfnZuDiJU8onMLjgWuotnYzV7e9f88YbviFWokR0a8/6lDmSve3Z46+I9+8PmzaR1qo149t3p9/TE9m0K4nTKhdhdIkN1Bz2KPbrr1CmDDz/PHTrFu3KsyPljYiExREbWM655cAZZlYDqA3kA3YB85xziyJQn0jkbN4MAwbACy/Azp3sqlmH4XlqMPr3RLbvjaFppdw8mLiQakNfxubOhfLl/WD0woX9dOty3JQ5km3tnxhn3Di47z5Sz2vBJ+1v4PFNBdi8OJmzKuXilivrcXrVYnDvBz6Phg2Dzp21jlWYKG9EJFyCWmjYObcQWBjmWkSiIykJHn7Yn1XevZt9F7fntXOvYsCMOPYkL+J/tUtyd5GdnHDTlbByJZx4IowdC1dcAfHx0a4+W1LmSLaRlAQjR0KhQtCxI7uu6MTXyYV4fGthtixPplmlXDwcO5dKzwyByi9B1ZZ+6Ycnn4S4oP6LluOkvBGRUAtmHazuQBf8GhF/n90BXnXODQ9rdSLhlJTkzwwnJMB337G7ZStGNLmSFzfmInVVOh0rxXN91fxUOq8BbN8OJ50EQ4dCq1aaETCMlDmSLSQnw+jR8PjjsHIlKZdcyiulGjLix2Vs25OXFhUT6JM8jfKPveTXumrY8MDyDXnzRrX0nER5IyLhcMQGlpk9DbQBngNmcaB/cj3gLjOr4pzrFe4iRUJqzRp45hl4802YP59FKfEMu/1FPpi/idgNxvVlocf0j8g/cJxfU2bmTD8T4GeHm2xKQkWZI9nCp5/CrbfCX3+R2ug0Prm5Lw/vKcWOrxdybs0S3Nq8GvXaNoNZs+Ccc3xD7LzzdOImwpQ3IhIuR7uCdS1wsnNu7UHPzzSzL4HZgMJHsoZVq+Dpp2H4cEhNZUv7y3nq9V95d00queJiubd0MtdMHEfupz70XXM6d/ZrzOigJ5KUOZI1paX5q1a5cwOQWqQo469/gIeTyrNzaxodyhr3rviFUh37+KvmfftCsWJwxhlRLjxHU96ISFgcrYF1tCNLHXlK1rB6NVStiktPZ2P7y3nqlPZ8sC2RApsct51diWvOrk6R996E776Be++F22+H0qWjXXVOpMyRrCU9Hd55x4+bat+erb0fYURcVcZc8Di7dqbRsVQad8/+mKKDXvPdkls1h+bNoW3baFcuyhsRCZOjNbBGAhPM7ODL53WBuwD1T5bMa9kymDQJunbFlSnDop6PMiChOl/sykXJpFiGJyyg+fjRxBa4ClrVgSuvhIsv1sLA0aXMkawhPR0+/NBPkDNvHqm1avNRfDn6PD2BvSlptKtekD7fvEKh597y+3fu7BcIrlEjunVLRsobEQmLo03T3jOwNkRX/j0A9AXn3CvhL1HkGC1eDE88AWPH4nLl4psapzNoxibmJZ9E5Vh4K3Umjca8SsyyZXDCCVChgn9dQoK/SdQocyTLuPtuGDiQ1Oo1+PS+Z3kgvhZ7kx1XVEqg68UNqVEiHwy/F3r08F2N9+eMZBrKGxEJl6POIhgIGIWMZH4rV0Lv3vD667iEBBZd1oXe1c7n10+WUqVYXp655GQ69LuNmI8+gtNP94t3tm0LMTHRrlwyUOZIpuQcfPGFPylTtSqbL+vE91aS3rlPZF+6cVviRrp9/yZ5X/oFLl/ux27+9JPyJZNT3ohIOBzXIhtmVsE5tyJUxYj8Jykpfj0qM9ynnzL/8mu5v3JL5qTlpqntYeKqj6hwyyPElisLDzwAd90FZ52lySuyIGWORJxz8O230KcPTJnC7htv5plWPXhj2kbSEk/ivphVdJ78Orl/+gGKFvVjOGNj/WvVuMrSlDci8l/95waWmSUCy4DY0JUjcgzmzIHHHoNt29g5/jPGLU5i3K1jWZ1ktGczw+e/RcnPP8LMoM25fmHghg2jXbX8R8ocibjJk+Ghh+CHH0gtW47PevThgQKnkjRlOR3ql+P20kmUOasNlCnjr4h37641rLIJ5Y2IHI+jrYN19hE2J4a4FpHg/P67b1h98AEuf36mXdCRG574hm3JjrOrFuGTt/pSZMJX/kDn1lvhzjs1/iGLUOZIpvLee6QuWsxn1z/AA0UakRYTwyP75tEq124KXvKY3+eTT6BFC79ouWQpyhuRLCIlxXe5TkqCli2jXU1QjnYFaxKwFkgPfykiQXj3XbjsMtILFuTHK3rQs1wz1lsiN6avoeXNl1C3fCGYXR2aN/GDy4sUiXbFcmwmocyRaJkyxc8K+MADrKjTgJENLufd3OdiFkO/nTNo8+U44pYthXr1oG8f3zW5TZtoVy3/3SSUNyKZ16ZN8NJL8MorsGYN1KlzoIF11VVw9dWZtsF1tAbWcqCTc+7ngzeYWS5gd1iqEslo6lTYsweaNWNlw7OZ1+k2epc8iz2x8TyybSbtvnuDhKVL4MrGQCEYPDjaFct/p8yRyJs+3TesPv+ctKJFeePjaTzy5S5iY2PoVWALnV96kNi1a30X4wHPw4UXanxV9qC8Ecmsxo6Fm2+GXbt8I2rwYDj1VL9tzx6YORPeeMOPrX/kEYgLctRTSgqMHw/DhsHOnfDmm1CxYsjLP1o104EGwL/CB3/GR4M/JXx++sl3BfzqK/ac1pgH7niR8bPWUKB0U55e9z3nff0msRs2QP368PbbULNmtCuW46fMkci6/noYOZK0woX58opb6FmmKblJ5/YqcVx+eVNKbtsAk9/0/9mfe64mx8lelDciofLbb/Dyy7B9uz8mA3j2Wd+gadsWatc+cn7u2wcTJ0KtWlCpElSv7nsIPPSQf21GefL4E2O33Qb9+sG0af4zCxc+co0rVvj1Tn/7DcqXhx07fG+nzz8/nm9+SEdrYHU83AbnXDJQObTliOCvWPXuDd99R0rRYoy/7FYeKns2MXPXcN1ZlelWpxAlTuwMZ5zhF+5s3lwHPdmHMkfC748//IK/sbFsqFGH6Zf24P5yzcjvUnhl+Zc0/vodYk4/Hbq1hAIV4Jtvol2xhIfyRuR4zZrl1wX87jvIlw/OOcfPvmrmu12//76/ylSuHDRtCu3aQYcOfp/Bg+Gvv/zY+mnTYPduuOMOGDAAGjf2t8PJkwdGjPD79OjhG2M//njk48E//oBly+Ctt+CSS2DhQihQILQ/j4CjLTScEuwbmdlnzrkLjr8kyZGcg/R0iI3FLVxIyuw5vHvZbTxe9mxqJG3hnflvUXP7GuIe+8n/8ixeDKVKRbtqCTFljoTVn3/Co4/C22+z9oVX6FfkVD7bWoPKpYsxavHHnPrN+1hSElx6KfTqFe1qJcyUNyLH6YsvfJfpwoWhf3/o1g0KFTqw/b33YPVq+PRTmDDBL3kRE+MbWGb+ZHpqKpx8MnTpAhdcAM2a/etj1mzby28rtrFlTzL7ktOoXjIfp5QvTME88XDddf6qV3q6f889e2Dp0gNXzJYv9z2iOnXyXQ3/+gsKFvRvXKuW/3PHDti2LaQToh3XOlgHaRLC95Kcwjl/drhvX1zbtkxsew0vb6/I7Ktf5oydq/h82lAq//Qt5M4N117rZ5DJlUuNKwFljgRr8WLo2xdef530XLn5uk0Xei7KT2rBDfRoWpVbpv5Gnpfe9gOm77/fX90S+Sfljch+O3b4Kz/nnAP33Qf33HP4ScXKloUbbvA353yXwf1WrPDvE/vv1RCWbtzFh7+t5rPZa1m66d/DIc2gRa2SdD+7Cqc2buyX5AF45hk/Jit/fv95u3b548bzzoOSJQ80rvZzDurWhQYN/ERqIRLKBpZI8JzzZz769oWpU9lbsjQvztrOkC3TKVsoN68U38g5z9/uf2EfftgPdCxePNpVi0hW4xy0a0f64iV807Ijvaq1onLSdt7+fSzlru1EvvPPhzNug66dtZyDiMiRbN/uuwNOmODXIs2bF5544u/N+1LSWL55D8s27SY5LZ2EWKNYvkQqFMlD8fyJvhGUkHDg/Q4aM7VjXwpfzl3HO7+uZPryrcQYnFmtGB1Pq8DpVYpSokAi8TEx/LF2Bz8s3sSb01bw9fz1nFmtKL1a1eLEsgWha1eoXNl3OYyL8yfMWrTwjatAjSu37CE13VGqQC4K503w3QtHjPCTXuTPH5IflRpYEh233govvsju0uV4pf2djKzQmM5rZ/BWweWceu+NxKecAQXS4JprtHCniByblSvhuefgsceYvT2Njy+5l/EbY6i5dyPvTRtOlamT/H+iuwI9vgoUCFs/fBGRbGHCBH9MtmYN3Hvv31edUtPS+WreesbPWs2kBRtJSj30qgf5E+OoUSo/VYvnpWLRvJTIn0juhFj2paSzfPNufluxjSlLN5Oa7qhSPC89z69J+/plKVkg17/e64xqxTijWjFubV6NN6auYMjExVw45EfanFyG28+tRrXOnaFz53+8Zse+FF77ZTkjf1zGlt3JAORLjGN014Y0uPxyGDLEr2vY8bBDM49Jpmlgmdn5wCD8qukjnHNPHbS9E9Az8HAX0MM5NyuyVcp/lp4OH38MjRqxr0QpvjupGbPa5+Ltsqdy64ofmfHG7eTasBb2XAixN0FsLrjppmhXLdmYMicbWrPGn00dPpx05xgQW4XB8VUpmLsUbywdQZ0v3oWiRf3spDfffPQZp0RCRHkjWVZKiu9J9NRT/mrQL79Ao0akpqXz3rQVvDRpCSu27KFE/kSuaFie+hULU6VYPnInxJCUms7GnUms2LKHxRt28ee6nUz4cyObdq36x0fExhhViuXluiaVaVmnFKeUL3Sgy98R5EmI4/omVbi0QXmGTl7CmJ//4tPZazirWjH+V6cUlYvmJSUtnQl/buCDmavYnZzGOScU5+J6ZYmPjeG5rxfQedQ0Xu18KqeVLesnv8iEDaz/PI2bmcUCLwItgFXAr2Y23jk3P8Nuy4CmzrmtZtYKGAacdjwFSwSkp/sZZB57DObM4ddrbqVHtQvZtCsXPfPEM2P0jcTt3OH78Y4ZlWkXjJNMSZkjXkqKHwfw8su4tDQmntmGh2pdSKO1K3jgqqZc2bw2+T/fBS0a+0HY+fJFu2LJepQ3kjM5Bz//7CeTGDgQ8uZl5oqtPPjhXOav3UHdcgV5qE0Dzq1ZgpiY4H5NdiWlsmVXMvtS00iIjaFs4dzExx5mbcFff/UTU6xff2Aii4oV/dTvgfoK5o6n5/k16dakCqN/Wsans9fy0Edz/36LhLgYLjy5DF3PrOS7EQY0rFSYjiOmcv1rM5nR4RISXn7Jz2QYgp5ToWxgPXH0XQ6rEbDYObcUwMzeAi4C/g6fgxYCnAKUO47Pk0h45x0/Y9f8+WwuX4Vn293L1PgqNCoAXTo1ptE8B3sX+QOjRo2iXa1kPcqcnG7fPj94OT6ebTNm8XuD83ikdlvO3LqM8V89TdFlC6FJSchVF9q3j3a1krUpbyRnWbLETwhRrBh8/TUkJJCUmsazn81nxI/LKJk/Fy92rE/rk0od+WpTejrs3QuJiX8vBpwvMY58iYdpgixf7te46tDBP77tNj/de0ZnnHGggdWokf9/4PzzKXLFFdz1vxO4s0UNlm7azZbdycSYvzpWOG8CBytRIBfPXHIy7V76mc/OvZx23a737xUCR2xgmdm1R3sD59yowJ9PHkcdZYGVGR6v4shnbq4DvjjcRjPrDnQHqKBBy5GVnu6n4AT2vP8Ru3Yl83S7nizJV4KH/vycJz56FuvdGyq3gMpt/MBCkYCsmDnKmyjYvBmeew73yivM/Ggiz/yxm2mn3c51Cycx/rN+FFizwk/RO24cXH55tKuVTEp5I3IY06dD69Zw5pnw4YeQkMDiDTu55Y3f+HPdTjqdVoFerWsdaCSlpcH8+X4d0/nzYcEC6NMHTjvNj2u6+GJ/5alECahaFU491U+WUbGin8Z99my/FtaHH/orVomJ8L//+bGyI0b49y9VyjfQ0tL81O77tWoFn30GDz7oFyVu3Rrr1YuqZ55J1YPnRktN9TMXFi78dxfxeuULUbNUfkb8lcrFF9YJqmtiMI52Bevqo2x3wKgQ1HGob+MOuaNZM3z4nHXYopwbhr+8ToMGDQ75PhJiKSnw+uvwxBOseWU0g7fm48vy7alLTXrP+5Tq8371Z0Luv9+PfRA5tCyXOcqbCNq2DQYMwA0YALt28fOp59Lz9V9JLl+R3hedTJfezxNbpgS8OMif3Yw5TJcTEU95I3Kwzz+Hyy7zMzc/5YcKfvz7anp9MIfc8bGM6tKA5jVLHtj/99/92lXbtvnHuXP7sVq7A1Orn3giPP20f7xmjV+PcNQo6N7db3/vPb+4MPirUU884U+M7Z/Nr06dI9fbt6+/rVoFw4fDK6/4zzjzTL+Q8AcfwLp1MHkyzJvnj1cXLPANLOcwMzqdVoGHPp7HyiEjqFCqkF8L8TgdbaHhf6/2FR6rgPIZHpcD1hy8k5mdDIwAWjnnNkeoNjmS5GQYO9b/QixbxspKNblz1E/Mrngil59Vk14jx5Fn62p49lk/9kEzdckRKHPksHbuxFWrhm3ezJRTzuGZ2hfQcv08vvrgQWJ/nUauiuX9RDoFC/ozpSJHobwRycA5eP55P0NgvXrw2WeklChJv/HzGP3zXzSoWJghHetTqmAu31BZscJfPapVCy65BJo29Vesqlb958mtqlX9UJCM0tIO5HSHDtCwIVSpcnxrnJYr54el9O594Llff/WLxufODY0bw113+QZbtWq+i3mnTtCuHRddcjlPfP4n6S+9BMXyhb+BZWbTgNHAm865rcf9aYf3K1DdzCoDq4ErgH9M42FmFYAPgKudcwvDWIsEKz3dX+adO5dllWvRv+39lNu3leHfDib9s08pWv9EaPyKX9U7MTHa1UoWoMyRf9i1C778EtehA5NW72Vh885MiilGq3VzeOvDR0nYswsuuABSkvz+hQpFtVzJWpQ3Ihns3OmnKm/fHsaMYZOL46YRU5m2bAvXnlmZXq1rEo+Dfv38rILVqvmJyRIT/ZWjY5FxYeFy5fwtVDKus9WpE1x0kX8u4aAxWGawdSt07UqBokW5sG4Zfv6yHBVnf485d9wn6o7WRfB1oCvwvJl9jg+iz51zqUd81TFyzqWa2S3AV/gpTEc55+aZ2Y2B7UOBPkBR4KVA/8hU51yDUNYhQdi3D95/H3fllfyydAvz6l/I71Vac+K21Tw/aSi5d2zzl2UJ/BMpWfJI7yZyMGWOwJ49fkbAp5/GNm7kxgdf56uUgpxQ6xw+fbYjcXv3YJdeCg88AHXrRrtaybqUN5KzOQdffeVnci5QwM8WWLIks9fs4IbXprBldzIDLq9Lu1PKwYYNfgrz776DK6+EwYOzRjfsw80am5gIH33kv/vll9Pqiyl8U7QitmOHn2ijUqXj+lhz7uhdeM2sFr6vcicgN/AWMMY5N+O4Pj0CGjRo4KZPnx7tMrK+vXth2DBc//7YmjU8cOeLvJFQkbK5jO8GXkWu7VvhwguhZ89AA0vEM7MZx3qgkFUzR3lznPbtg1dewT35JLZ+PTNPaMCIE5pTgz2UfvBe2tcvR/zIEf4/xBo1ol2tZELKG5Eg/fqr77o3aZIfa9XTL8P23oxVPPDhHIrnS+SVq0/105pv3Aj168OmTfDii9C1a/bpij1vHpx4Irvv68VVa4ry4bh7fMProouCevnhMieopqdz7g/n3ANAJfyl7XzAd2Y294gvlKxv3z4/W1flynDHHczJXYL7W95C1VlTeOziE/mud0tyDR4Ec+fC+PFqXElIKHNypvRdu0np/SCz85fmnla3sy1Xfl789Flu/2Ykl1dI9OukdO+uxpWElPJGcpSff/YzBDZq5I/dhgyBu+4iKTWN3h/O4Z53Z3FqhcKMv+XMA2tGFSsG11zjX3vttdmncQV+TFb79uR97232VK9OuhksWnTcb3tM1/acv9y1C9iL7wOW+7grkMwpcGUzOc2xp/9zzMxflkebX09ybDxPfTWEa6d9yNVljFzxsXD11Uef5UXkP1DmZHMpKTB8OO7idnzy+2pavTaPKy56iD3xuXj2i0E0Wzodu/debNkydTeWsFPeSLaVnn7g/qOP+qtXTz7p17q6+WaWbU/mkpd/4fWpK7ihaRVeu64RRUnxk5PNnesbVI8/DqecEr3vEE4vvACzZnFC9fL8r/eHcM89x/2WQTWwzKy8mT1gZn8CXwOJQHvnXNXjrkAyl+3boV8/0k85hdET/qTpCz9zZct7KJC8h4cnjODU7SvgiSewFStAa3BImChzsrnUVBg9GnfCCdC9O3/+toDnXvqUNOe4rltrTt+zFh57DFu+3HddUeNKwkh5I1lOWpqfBGjbNtiyxU+BnrERlZ7ux0x9+qlfb6pCBT+uCPyEFH/9Bfffj8ufn3d+XUmbF35gxZY9DLv6VHq1qkXczBm+S+DIkfDTT9H4hpFVtizky8cpFQqxODmWtdv3HvdbHm0WwS5AZ/x6DJOAx4APnHPH/8mSuWzbBi+8gBswANu2jV+qNeDTER9TvunZ3Hp3B6otfg96v4J17hyyVa5FDqbMyQEWLsRdcAG2eDELy9bgyzOuoOXK3/no55cp8PJMYmJj/FnVjLNMiYSB8kYyrVmz4Isv/KK9S5f6Y7T77oPOnWHOHDj55H+/ZsQIuO46+PFHP2X6/gZXQoJftHfPHv84cHJ85ZY9PPDhHH5YtInTKhdhwOX1KFMg0Z/UeughP2X6hAl+zGtOMH8+V3Zpz/IKZ5Ny3VswbsRxHe8ebRbBnsAY/LShq//zp0jmtmgR6Q0aELNjBz9Wa8DSyiW5eMnPjPtlGLleu8cf6Pz8c7SrlJxBmZMdpafDsmWkVKrM+E0xlIkrxtyGJ3Leyt+5/ee3cJUrY7fdyd9rr6pxJZGhvJHMZ/Vq3xXPOT99ebVqftxpsWJ+e+nS8MgjkCcPxMf77ntJSX5MFUDlyn4tqKJF/Syrp53m14EK2Lo7mRcnLmbsL8uJjzUeu6gOnU6rSEyMwdChft2oSy6BYcP8Yrw5RfnyJP61hHJFT2TsHY/y4HFeTDjaQsO1AMyssJm1BIoAW4BpYV4zQsJt0yaYOZM1jZowbH4SJ1Q7k4SkfVywZCpN9k33axv07Jk1puCUbEOZk82kp8NHH5He52H2bdjEhbe/ypKdadx2ajPuev1JXO3a8Ow47PLLIe5o5/tEQkt5I5nGqlV+ofSbb/bd1d5911+F2t+oyqhYMb8O1eGULQt9+/7jqfR0x6xV23hr2ko+nrWa5NR0OtQvx13/q0Hpgrn9hGa5cvkJLIoW9Q2s7DSRRTDy58fOPJNzl8zmnlXbj/vtjvo/mpk9BDwQ2HcTUBxIMbOnnHOPHncFElkbNsBzz5E+5EWSiKHlTa+yNz43D53djCsG9/IHOvfdp7VlJGqUOdmAc/DJJ6T3eZiYWb+zqlg5ppesResFP1Gv1y00r9QcLjsda9NGJ3EkqpQ3EnXjx0OXLv4qVLt2UKYMdOjw92bnHDv2pbJldzJp6Q5wJMbFkjshlrwJceSKjyGwdhrOOZJS09mwI4nV2/by57odzFm1ne8XbWLTriRyx8fS7pRydD2zEjVK5vcnwfr391erpk71jatLL43OzyEzaNmSyg88wJoFf5GcejoJcf/9/6ejjcG6DLgVuAr4OLBYXhxwMTDEzP50zr39nz9dImf9eujfn/SXXoakJH6ucDJxLo2BayZxwpCnKVegJdxxGVSsGO1KJQdT5mQPyd9+R8JFF7G2cClm1TiDs1bNpf28ibj65bBagQkr2raNbpGS4ylvJKqSknxPoUGD/IQSb78NZcrgnGPG8q18NW8d05dv5Y+1O9iXkn7YtzGD+JgYYmIgOTWd9IOWty2WL5HTqhThvFolaF6zJAVzx/sNmzb5MV1ffOGvWMXHh/HLZhEtW8IDD9B67RzWbr+IikXz/ue3OtoVrG7AXc659/c/EVjh/D0zSwS6AwqfzMw5MOOP6X9QY+AgpparQ6Gk3Zy1/HfSixQlplE1KJzH76vGlUSfMicrcg6++46UJUsZd2ILhk5z3FyvFR3+mEzrhT/jWreG3r2xM86IdqUiGSlvJDqcg/PO8xNS3HorPPMM6fEJfPzbKoZOWsqC9TtJiI2hbvmCdDqtIqUL5qJI3gTiYv0VlX0paexNTmNPchp7klNJTXekpTsS42LInRBLsXyJlC6Yi+ol8lOyQOLfV7j+/uyRI/1Yqx07/MLBPXrkvC6Bh1KvHu6663iocys4jsYVHL2BVQ+4/DDbPgcGHdenS/isWIF76inWb9nFnc1v4pelu3nhpGa0nfUt6RUrweDBxFx7rR8kKZJ51EOZk7VMnkzagw8S++OPrC1Shhc65qN6nco07HEVeb7NBw88gNWrF+0qRQ6lHsobiaSUFD+JT0wM3HSTH5Jx4YXMWrmNPuOnM2vlNmqVLsBT7U/iwrplyJsYorGpzsHatb77IfgxXjVrwksvwUknheYzsoOYGGzEiJC81dH+5hKdc1sOtcE5t9XMEkJShYTOX3/hnnwSN+pV0p1jVclq7C2zkAfbnUWLto/B8muJufRSDSiXzEqZk1XMmUPabbcTO2ki2/IW4q8yNam7diGfpM+g3I0dgcbQvWO0qxQ5EuWNRIZz8OWXvkF1++1w/fVw5ZUkpaYx6Ms/GTp5CcXyJfLcpXVpd0pZP6Mf+PWuFiyARYtgzRpo3hxOOAH27oV16/yEFgmH+Ge6dKn/vIUL/etnzvRj8Jcu9bMMvvMOFCigq1ZhdLSjbDOzysDh/gb0N5OJpL3+OtalC2kO/ixWkcrb1tBgzZ+8V3YTcU2qAFXgjNOjXabIkShzMruUFHalG59PXU6LaTNYXaIKtTcso3DqPmJu6kG5e2+LdoUiwVLeSHg5B19/DU8+CZMnQ9Wqfn0pYPGGndz65u/8sXYHl55ajocurE2BXIFxUCtWQJ8+8PnnsHHjgfcbN843sKZNO7A+VbFi/qR5Sgq8/76ffXD6dD8jYd68for31q2hYcMDvZYKFozczyCHOloDKy+wmMOHjDvM8xIpixaRvHsP7ycX5v358fQtXI5qW1Zx4sZluHbt4P77iWvQINpVigRLmZNZzZhB6kN9WJISx+VNbmbbHuP9GvWot3gmMffdC3fdBSVLRrtKkWOhvJHwuv56GDXKX2kaPBi6d8fFx/PG1OU89ul88iTEMbxzA1rULgnJyf4KU5Uqfv2piROhRQu/SHCtWlC+PBQq5N+3enW/sPCqVf5KVnq673q4f92qVq38tjJldJUqSo62Dpbmz82s/vyT1MceI+att5hXvg69rniSk8uVId95zYjPH4/dey9WrVq0qxQ5JsqcTGjWLFIe6kP8J+NJSshNQp5CtG62nctubkG9m16D/Plz1mKUkm0obyRknPNd8SZM8OtZvfqqb9xccgk0aQIdO0JCAht27uP+N6Yz4c8NNKlejOcurUuJ/Inwxht+ceDcuWHOHJ+rf/11+MZRmTJw3XWHryd/fn+TqNFAnKxm/nySH3mU+PfeJS0mjg15i3DK8jl81CCeuh3OxG45K9oVikg2sefl4eS5qTsp8YlsyleU0rs2U65wIZ5oUBDKFwIKRblCEZEoWrQIHnzQd/9bv94/V7267+JXpoy/koRf6PfNqcvp/+UC9qWk8ciFtencuBIxixbCxT381ar69aFfvwNrA+rKU5Z2tHWwJnLkS+TOOXduaEuSw9m4M4n5/Ydz+ocfsjlXAYrt3U6xfAnwyHPUa9lYv4yS5SlzMoEFC9i+bTfDt+Xl8wW5eSNfEUrt2kJc6VLwbD/iu3SBxMRoVyly3JQ38p9MmQJ79vgJJ3Lnhp9/9lOun3OOv1Wt+vfxWFq648u563h58mLmrt7BaZWL0K/diVQrkR9mzPBXtxIT4eWXoVs3381PsoWjXcEad5jnywK3AZrjO9xmzmRvn0f4rMYZ9M5TlwL56zM5PpF85UvDA4NIuPLKQ88gI5I1KXOiZfFikh5+lPi33mBN8UoM6fICrRvWwvLdDjWrkHDFFZp9VLIb5Y0Eb/NmuOceGD3ad/1r3hzKlfNXqzKc4N6XksZvK7bxzfz1fDVvHau37aVS0TwMvLweF9Urc2DAX716cNttflbB0qWj8IUknI42BmtkxsdmVhTohV+c722gb/hKy+GmTWP3Qw+T9+svsdh4Tv15Ohe9/Bk3Nm9CnltPg2rVDlxGFskmlDlRsGwZex9+lMTXxxED7IlNoNb6pUw+Jy8Vzz8VODXaFYqEhfJGgjZzJrRv76dK79nTdwsM2JmUypSlW5i+fAsz/trK7FXbSU5LJyEuhibVivFQm1q0qF2K2Bjzs//dfjt8+KGfTfCpp6L4pSScgjodaWYFgHuBW4BPgfrOuSXhLCzHco7N195A0dHDiYuNJyUmllxpKZRqfCr9/1cJCueD4jWiXaVIWClzImPDzn3MfXggZ497jaSYOHKnJZN6emN4pA8Vz2sa7fJEIkJ5I0c0bZqf+rxYMfjpJ2jYkL3JaXwyfSUfzFzF9L+2kpruiI816pQpSJczK9GwUhEaVy1Kvv0LBSclwWOP+QZVuXJ+6vXAdO2SPR1tDFZu4A7gbmAScJZzbl74y8qBfvyR3wqX54Vp66m1dB/3YMThSOl0FfG9epK7Vq1oVygSdsqcCFizhj2PPsYXeSvxQJ6TyV/4dCblHYM1Ph0e6UPuM86IdoUiEaG8kaDUrw933w233UZSkaKMnLSYoZOWsGNfKlWK56Xb2VU4u3pxTqlQiFzxhxhDNWaMXwdrwQLo0gUGDDgw3bpkW0e7grUMiAX6A9OBkmb2j4VOnHMTwlRbjuAmTWL7/Q9SaOpPLKnTnN8v68mpvXuRfHZlct3YndiyZaNdokgkKXPCZf16dvXtR+LwV8iVmkKTPAW5eOR39Dj3bPL1+R8ULRrtCkUiTXkjh5acDA89BHfe6a80Pf44vyzZzP1jvmf55j2cV6sE3ZpUoVHlItj+8VfO+fFY8+bBwoV+fJUZfPONnwzjs8/8gr+SIxytgbUPP8NOj8Nsd0CVkFaUQ6R/8y3b7n+QIjOnkjfGn/E4Z99aWvdsRp7EePjfo1GuUCQqlDlhsO3p58j70APkSUnBAS4mhrxt2/D0+VWhYF78eqsiOY7yRv5t82bo0MFPvV69Oqldr+WFCYsZPGERlYrm5bXrGtGkevED+0+cCM8957sSbtx44Pk2bfyMgkOHQt68muk5hznaJBeVIlRHjpGals6ns9dy8jU3UGHdXwDsLV0OHuxFsS7XQGJ8dAsUiSJlTght2cKq3Wm8+Otain0+iztTkkmLjyfpmmvJ92Av8lasGO0KRaJKeSP/Mn06XHoprF0L48axvd1l3DL6V35YtIlLTi3Ho23rkDchFn74AapUgbJlYedO3/3vggugQQM4+WSoXftAr4B8+aL7nSQqNOduJDhHyiefsK33o9x7Xg8mJZam+7md6DHzYwo8+hAF2rXT2gciEhrbtrHtif7kfmEgM6s05P3299Px9rvZdU51CtzUnfiSJY/+HiIiOc033/irTiVLwuTJLKlSh+tf+olVW/fQv8PJXNawvO/+d9dd8PXX0KsXPPGEf82FF+oKlfyDGljh5Bz73v+QPff1osiyhRTFuC7hTTqOfZ3zarYiJuZR/UKKSGjs2MGWJ54h76DnKbRvDwBNNi/h+7ubUqpwHmhfN8oFiohkQunpftmbhg2hY0d49ll+2uro8eJPJMTF8Ga302lQOq+fnv3ZZ6FAAXj+ebjhBv96LZkjh6AGVphs37ab1Hr1KLp8MYlAuhmb2rTnrMf7YHU0NaeIhM7iDTtx5zSn+h8zANhdoBDpPe+n8G03Qz6tlSoi8i/79vmG0ief+C5/hQrBq6/y5rQVPPTRXKoUz8vIaxpSvkgeP2HF4MFw3XXw9NOaFEiO6rANLDNr65wbH7gf75xLiVxZWVRaGtu+mchQV45xU5bz3u4UCsTGs/XqLpR4pDclNOZB5LCUOcdo7142PPUcw/KcwMhtebi82tk8sGEV9tCD5L+xGyQmRrtCkUxLeZODJSXBq6/6qdNXrPALCO/cSUqBgjz26XzG/rKcpjWKM7jjKRSwdP+aXr2gVSt/EwnCka5gjQMKBO5vznBfDpaWxuYRY4jr/QCFNq9nddv7aHbFFcR+9SXxFYtTQmc6RIKhzAnGvn2sf2oABfr3o8Te3bQuV4v44R9wfe/zKJD7GY3nFAmO8iYnWrjQLxq8bh2cfjqMGgXnnsvKLXu4Y9gUZizfyg1nV+G+FtWJ7XkfzJgB334LpUv7m0iQjtTAWmdmtwDzgTgzawb8a8BQjl4jIjWV9YOHkfvRPhTdvhmAnQUK80DHxpS+/JQoFyeS5ShzjmJNv2co1K8vJffuAmBzuUrUeP5x6p9fM8qViWQ5ypucYPFi+OgjyJULbrnFT5vesiV06gTnnUe6g/d+Xcljn84H4IUrT6Ft1QJwSQcYP/7AWlYix+hIDawuQF/gdiABGHWIfXLsGhFzV29nzPhfefquW4jBsbVEWXiiH4Wv6UT+OA1tE/kPuqDM+be0NOas3cULExZx34DBlNm7iw1VapL3+Wcp2ra1/vMX+W+6oLzJntavh9deg7FjYc4c/1zbtr6BFRsLo0fjnOOnxZt56ss/mLt6Bw0rFeb5y+pRfu9Wf4Xr999hyBC4+eaofhXJug7bEnDO/QycB2Bmi51z1SJWVWaVnMyKR57Exr5Gm44DyJ87gbadb6Vuh/9R+EId6IgcD2XOQVJTWfXEcxR+uh+jzrmeqQ1b8svTr1C6Um5KnHtOtKsTydKUN9nYbbfBO+/4LoADBsDFF0OlSgBs35vCZ7PXMvaXv/hz3U7KFsrNoCvqceHJZYgx4KKWvhvhJ59A69bR/BaSxQV1qWV/8JhZBaAssMo5tzKchWUmbt8+lt/zEKWGD6FC8j4cMKD0ds7tcTkFcrWMdnki2U6Ozpy0NFY89gxF+/ej3N5dOKBL4ib63t+c/Lm0ELlIqOXovMkOdu3y61F17QrVq/tZ/h55BGrVAmDHvhQm/Laaz+esZdKCjSSnpVOzVH76X3IybeuWIVdcDKSmQnw8DB3q79fVshZyfIJqYJlZKeBtoDF+MGhRM5sCXOGcWxPG+qIqPd3x65gPqXfDlVRKSSYdY0XjZpQY8TLtap8Q7fJEsq2cmjm/LttMtVNOoML2zThg1SmnUfjV4dSte1K0SxPJtnJq3mR5zsGHH/orVqtXQ6lSvoFVqRL7UtL4etYaxv++hu8X+kZVyQKJdDq9Au1OKctJZQtiZvDnn3DffVCmjG9c1akT7W8l2USwg4WGArOA1s653WaWF3gi8HzbcBUXLSk7djJ1zEc8klKBNavSmBYbz9rmrSgz8mUqlNUsMiIRkHMyxzmWPjGAB4ucxs/LtzG6dA2qnhBHsbEjKXdC9WhXJ5IT5Jy8yS5WrfJjqj7+GOrVg3ffhcaNWb1tLyN/WMb7M1exfW8KpQrk4qrTK3LByaU4pXxhYmLMN8x++AFGjoTXX4fcueHhh/3zGuohIRJsA+ssoPT+dSICAXQfsDpUhZjZ+cAgIBYY4Zx76qDtFtjeGtgDdHHOzQzV5wPs27CJtZ26Un7C5zR2kPfJz3my8xnk6r+dSnGa+lgkgrJ95rj0dP7q1ZeSLzxLlX27aX5OJ5r37ctpfX8gd4LyRiSCsn3eZDsDB8LXX8Mzz8Add7BudyqDPpjNezNWAdCyTimubFSBxlWK+kZVWhq4dCDWdyHs1Qvy5fOTWPTuDSVKRPXrSPYTbANrK1Abf4ZnvxOAbaEowsxigReBFsAq4FczG++cm59ht1ZA9cDtNODlwJ/HbddfK9nc8RrKT5lEZefYm5ibdZ2789FdzbF4jXkQiYJsmzkuLY2/7utDyRcHUjlpD2lm/NW0JVe9/iy5ypQ63rcXkWOXbfMmW/njD9i7F+rX91ecbrqJfeUr8sqkpQydvIS0dMeVjSpwQ9OqlC2UG9LT4ccfYNw4P1X7mDF+oeCLL/ZdAjt0gLx5o/2tJJsKtoHVH/jWzEYCy4GKQFfgoRDV0QhY7JxbCmBmbwEX4den2O8iYKxzzgFTzKyQmZV2zq09ng/enZTKuK69ufGXiezKk5+Nt91NpX4PUTkm5njeVkSOT7bNnH4f/Mb9A57CgGXNL6D0m6OpVKLY8byliByfbJs32UJKCvTvD337QqNGvntf/vx8v3YfDw74nhVb9nDBSaW5v1VNyhfJ4xthTz0Fw4bBsmW+EdW2LRQL5GzNmv4mEk7OuaBuQHNgBPB54M/mwb42iPe+BH/JfP/jq4EhB+3zKXBWhsffAQ0O837dgenA9AoVKrijGTF5sbvt/Asdfs0LB7jp06e76dOn/+O5hx9+2DnnXOnSpf9+rn79+s4557p16/aPfVevXu3Gjx//j+deeeUVFwjPv29t2rRxzjnXpk2bfzzvnHOvvPLKP54bP368W7169T+e69atm3POufr16//9XOnSpZ1zzj388MP6TvpOYftO+2s6GmC6+2+5kCUyh2PMmxnLt7jOLdpmyb/z7PjvWN8pa3wn5c1/y5v9surfe7cOHZyrX9/Vz/BcqVKl3Z1v/eYKnnnlP7/Tjz/++zu1a+fcrl2Z6ztlw9/P7Padgs2bwGceMnPMb4suM7sUaOmcuz7w+GqgkXPu1gz7fAY86Zz7MfD4O+A+59yMI713gwYN3PTp08NXvIgckZnNcM41iHYdGYUrc5Q3ItGlvMlGpkzxi/4WLAgvvwwdOvD1vHU88OFctu1J5qZzqnJz82okbtwAjz4KX3zh17BKTITt2/3rRMLscJkTbBfBcFsFlM/wuBxw8NSowewjIhIMZY6IRIry5lgkJ0NCAjRoAPfcA3fcwfa8hXjk7d/58LfV1CpdgDHXNqROsdzwTH94/HH/mhtugH37fANLjSuJsswy0OhXoLqZVTazBOAKYPxB+4wHOpt3OrDdqW+yiPw3yhwRiRTlTTBSU/3sgDVrwqZNEBcH/frx7cZ0WgyYzCez1nD7udX5+OYzqWN7/PTsDzwALVv69ayGDFHDSjKNTHEFyzmXama3AF/hpzAd5ZybZ2Y3BrYPxfeLbg0sxk9h2jVa9YpI1qbMEZFIUd4EYepU6NEDfvsNWreG1FS27E6m7yfz+Oj3NdQslZ9RXRpyYtlAA6pUKWjc2E/TfsEF0a1d5BCOaQyWmcUAJbPSWZVs30dZJJM7njERWS1zlDci0aW8yWKSk6FzZ3j7bShdGgYNwnXowEez1vDYp3+wc18KPc6pxi3NqpHwxzy49VZ47TUoX/7o7y0SAYfLnKC6CAamC30D2Ic/u4KZtTWzx0NbpoiIMkdEIkd5EwVrAsPLEhLADB56CBYsYMk5reg0chp3vj2LCkXy8OmtTbireVUSBj4PDRv6tbBWrIhu7SJBCHYM1lBgO35tiOTAc78Al4ejKBHJ8ZQ5IhIpyptISE2Fjz+GFi2gQgVYutQ//8Yb7HjgIR6fvJKWA75n7urtPH7xibzf4wxO2LoKmjSBe++F88+HOXPgzDOj+z1EghDsGKxzgTLOuRQzcwDOuY1mViJ8pYlIDqbMEZFIUd6E04YN8PzzMHYsrF0L5crBY49B4cIkp6bz+tTlDJ6wmK17krm8QXnuaXkCxfIl+tcOHOgnsBg3Djp29Fe7RLKAYBtY24FiwN/9ks2sQsbHIiIhpMwRkUhR3oTaggWwezfUr+8fDxwI//sfXHcdtG5NssXy/sxVvDjxN1Zt3csZVYvSq1UtTiqdD8aMgRNPhEaN4MknfWOshNq6krUE28AaAbxvZr2BGDNrDDyBv6wuIhJqyhwRiRTlTShs3OgbR2+84WcDPPdc+PZb3zjasAEKFGD73hTe/mUFo3/6izXb91G3XEEev/hEmlYvhn31FVxwP8ye7WcUbNQIihSJ9rcS+U+CbWA9jR/8+SIQD4wCXgEGhakuEcnZlDkiEinKm+P1yCPw1FOQlOQnoxgwAC69FADnHLO2p/PWN7MZP2sNe5LTOL1KEfq1P4lzahTHZsyAFlfChAlQpQq89RZcdll0v4/IcQqqgeX8XO4DAzcRkbBS5ohIpChv/qPNm/3CvnFxULGi7/53881QuzYAG3bu4+Pvl/LujJUsXL+L3PGxtK1bhqsbVzywnhXAd9/5ySsGDYIbb/QzC4pkcUE1sMys+WE2JQGrnHPLQ1eSiOR0yhwRiRTlzTFyDt58E26/He6/H+6+G7p2ha5dSUlL57u563hn+komL9xIWrqjXvlC9Gt3IhfWLUOBXPG+C+B9T0ObNnDllXDbbXDTTZA/f7S/mUjIBNtFcCRQJnB/M1A0cH8DUMrMZgNXOOcWhbg+EcmZlDkiEinKm2CtXOmvMn3+OZx2mp+4Ali3fR+vT13Om9NWsmlXEiULJNL97Cp0qF+OaiXyQVoafPEFDBkCX30F+fL5roQAuXNH8QuJhMexNLAKAn2cc3vNLDfwKH7mnYHAc8BLQItwFCkiOY4yR0QiRXkTjHfegeuv942lgQPhlltYsmUvL787i49+W02aczQ/oQQdT6tA0xrFiYvNsNTqJZfARx9B6dLw+OP+ilXhwtH6JiJhF2wD63agtHMuFSAQQL2BNc65fmZ2N7AqXEWKSI6jzBGRSFHeHEl6OsTEQNmycPrpMHQoqwqX4vn35/Dhb6tJjIvhqtMr0vXMSlQsmtd3IZw0CYYPh5degkKFoFs36NQJLroI4uOj/Y1Ewi7YBtZuoCF+ZfP9TgX2BO6nh7IoEcnxlDkiEinKm0NZvBh69/bTrA8eDGeeyY7xnzFkwmJG/zQZDLo1qUK3JlUonj/Rr3s1bBi88ALMm+enWJ8zB5o0gdato/1tRCIq2AZWH+BrMxsPrATKARcCtwa2nwu8F/ryRCSHUuaISKQobzL64w8/5frrr0NiIvTqRXq6490ZK+n/5QK27EnmkvrluOt/NShdMDB+atMmqFEDtm6FU06BUaPgiis0vkpyrGCnaR9rZtOBDviBoAuBxs65+YHtnwKfhq1KEclRlDkiEinKmwwGDYI77vANo9tvh3vv5feUXDz80k/MWrWdBhULM/rCRpxUtgB8842/QnX33VCsmH9ds2Zw1llgFu1vIhJVwV7BIhA088NYi4jI35Q5IhIpOTpvvvgCypSBunWhZUt48km47jo25y7AM18t4O3pKymWL5GBl9fjojrFsTfegP79/ZWucuX82le5ckGfPtH+JiKZRrDrYBUB7gHqAfkybnPOnR36skQkJ1PmiEik5Oi8GTIEbr0VuneHV16BmjVJufc+xk1ZzoBvZrInOY1uTapwa/Nq5J8+FWqdDUuX+sbY2LFw2WW+G6GI/EOwV7DeABKBdzgw6FNEJFyUOSISKTkzb554wk9icdFFvmsgMHnhRh7/dD6LNuyiSfViPHxhbaoVzQOxsVCxor/SNWgQXHCBugGKHEGwDawzgOLOuaRwFiMiEqDMEZFIyXl5M22ab1x17AhjxrAlKZ1er03nq3nrqVg0D8OuPpUWFfNhDz/sZwT84gsoXx5++CHalYtkCTFH3wWA2fhZdUREIkGZIyKRkvPyZuBAKFkShg7l57+2cf7A75n450Z6nl+Tr+88m/9tXojVrQvPPw+VKkFycrQrFslSgr2CNQH40sxeBdZl3OCcGxXyqkQkp1PmiEik5Ly8efVVWLiQH9clce3oXylfJDejuzaidtFE6Hmfb1hVqeIXDG7aNNrVimQ5wTawmuBXMW9x0PMOyJ7hIyLRpMwRkUjJOXmTlub/TExkVqHydB8+hSrF8/J298YUzBPv17F67z246SY/U2DevNGtVySLCnYdrGbhLkREZD9ljohESo7Km2++gWuvZecnn3PdF5sokjeBMdc2ouCKJVCtGhQuDLNnQ8GC0a5UJEsLdgzW38yL2X8LR1EiIvspc0QkUrJ93rzxBuzdy4Bl6WzZncTQq06l5G9ToWFDePBBv48aVyLHLajwMLOyZvahmW0GUoGUDDcRkZBS5ohIpOSYvNmzBz78kC0XXMToGWu56vSKnLh9NbRp42cIvPXWaFcokm0Ee3ZmKJAMnAvsAuoD44Ebw1SXiORsyhwRiZSckTeffAK7djGoRAMK50ng7oYl4eKLIU8e+PprKFs22hWKZBvHsg5WBefcbjNzzrlZZnYd8DMwPHzliUgOpcwRkUjJGXnzxhsklyrN2PiK9D2vOgVv6gbLlsHEiVAuZ81SLxJuwTaw0vCXzQG2mVlxYAeg0x0iEg7KHBGJlJyRN7feylufziBvrgTa1y8HPXvCZZfBWWdFuzKRbCfYBtZUoDXwIfAV8DawF5geprpEJGdT5ohIpOSIvNl2xtn0m5TCpfVKkzcxDho39jcRCblgG1hXc2C81h3APUA+YGDoSxIRUeaISMRk/7z55BMmbYCk1Bju+nAQTMwLgwaBWbQrE8mWgl0Ha1uG+3uBx8JVkIiIMkdEIiXb541zuB49KFy0Ghe07U6Rca/CHXeocSUSRkE1sMwsAegC1MOf1fmbc65zyKsSkRxNmSMikZLt82buXGz1aj47qQN3//YR5MoFvXpFuyqRbC3YLoJjgLrAJ8D68JUjIgIoc0QkcrJ33nz5JQALileg8psvQY8eULx4lIsSyd6CbWCdD1TOeBldRCSMlDkiEinZO2++/JIVZarQdfkUDOCee6JdkUi2F+xCwyuAxHAUYGZFzOwbM1sU+LPwIfYpb2YTzewPM5tnZreHoxYRyTSUOSISKdk3b5KTcTNm8GX5U1jf9QYYORIqVAjZ24vIoR32CpaZNc/wcCzwsZkN4qDL5865CcdZw/3Ad865p8zs/sDjngftkwrc7ZybaWb5gRlm9o1zbv5xfraIZBLKHBGJlByTNwkJfP3NTF4cN43hTetD5fOO+y1F5OiO1EVw5CGee+Kgxw6ocpw1XAScE7g/BpjEQeHjnFsLrA3c32lmf+AXANTBjkj2ocwRkUjJMXkzYfkuLlz+K6fOzgOV24biLUXkKA7bwHLOVY5QDSUD4YJzbq2ZlTjSzmZWCTgFvzDg4fbpDnQHqKBL4SJZQlbNHOWNSNaTU/LGOcekBev5avJYYhNXwUVqYIlEQrDTtNcDNjvnVmZ4rjxQxDk3K4jXfwuUOsSm3kHWuf998gHvA3c453Ycbj/n3DBgGECDBg3csXyGiERfVsoc5Y1I1pad8+aPtTspuvhPCm3bBK1aHUs5InIcgp1FcBxw8GmPBOA14OSjvdg5d9hOv2a23sxKB87slAY2HGa/eHzwvO6c+yDIukUka1LmiEikZNu8OaFUfkaUCHzk+eeH6m1F5CiCnUWwgnNuacYnnHNLgEohqGE8cE3g/jXAxwfvYGaG7y/9h3Pu+RB8pohkbsocEYmUbJs3sTFGmZ8mwqmnQqlDXWQTkXAItoG1yszqZ3wi8HhNCGp4CmhhZouAFoHHmFkZM/s8sM+ZwNVAczP7PXBrHYLPFpHMSZkjIpGSffMmORnWroXWii+RSAq2i+AA/BSm/YElQFXgHqDf8RbgnNsMnHuI59cArQP3fwS/Pp6I5AjKHBGJlOybNwkJsHgxpKSE5e1F5NCCamA554ab2TbgOqA8sBK/ZsN7YaxNRHIoZY6IREq2zxsz39ASkYgJ9goWzrl3gXfDWIuIyN+UOSISKcobEQmlYMdgiYiIiIiIyFGogSUiIiIiIhIiamCJiIiIiIiEiBpYIiIiIiIiIRJUA8vMEs2sn5ktNbPtgef+Z2a3hLc8EcmJlDkiEinKGxEJtWCvYA0ATgQ6AS7w3DygRziKEpEcT5kjIpGivBGRkAp2mvZ2QDXn3G4zSwdwzq02s7LhK01EcjBljohEivJGREIq2CtYyRzUGDOz4sDmkFckIqLMEZHIUd6ISEgF28B6FxhjZpUBzKw0MAR4K1yFiUiOpswRkUhR3ohISAXbwHoA+AuYAxQCFgFrgEfDUpWI5HTKHBGJFOWNiIRUUGOwnHPJwB3AHYHL5pucc+7IrxIR+W+UOSISKcobEQm1YKdp/8jMLjWzROfcRgWPiISTMkdEIkV5IyKhFmwXwcnAvcAGMxtjZi3NTIsUi0i4KHNEJFKUNyISUkEFiHNugHOuEdAAWAoMBNaY2QthrE1EcihljohEivJGRELtmM7QOOcWOeceBa4AZgM3h6UqERGUOSISOcobEQmVoBtYZlbVzB40s3nA1/hZdpqGrTIRydGUOSISKcobEQmloGYRNLNfgRrAeOAe4GvnXFo4CxORnEuZIyKRorwRkVALqoEFPAuMd87tDWcxIiIByhwRiRTljYiE1GEbWGZmGaYqfTfw3L+6FDrn0sNUm4jkIMocEYkU5Y2IhNORrmBtBwoE7qcCB68LYYHnYsNQl4jkPMocEYkU5Y2IhM2RGlh1MtyvHO5CRCTHU+aISKQob0QkbA47i6BzbmWGh5c655YffAM6hL9EEckJlDkiEinKGxEJp2Cnae9zmOcfDFUhIiIZKHNEJFKUNyISUkecRdDMmgfuxppZM3yf5P2qADvDVZiI5DzKHBGJFOWNiITL0aZpHxn4MxcwKsPzDlgH3BqOokQkx1LmiEikKG9EJCyO2MByzlUGMLOxzrnOkSlJRHIqZY6IRIryRkTCJagxWAoeEYkkZY6IRIryRkRC7WhdBAEwswLAI0BToBgZ+ik75yqEpTIRybGUOSISKcobEQm1YGcRfAmoD/QFiuD7Ja8ABoSpLhHJ2ZQ5IhIpyhsRCamgrmAB/wNqOec2m1mac+5jM5sOfIICSERCT5kjIpGivBGRkAr2ClYMsD1wf5eZFQLWAtXCUZSI5HjKHBGJFOWNiIRUsA2sWfi+yQA/AC8CLwMLj7cAMytiZt+Y2aLAn4WPsG+smf1mZp8e7+eKSKamzBGRSFHeiEhIBdvA6gb8Fbh/G7AXKASEYuad+4HvnHPVge8Cjw/nduCPEHymiGRuyhwRiRTljYiEVLDTtC91zi0J3N/onLveOXe5c25+CGq4CBgTuD8GuPhQO5lZOeACYEQIPlNEMjFljohEivJGRELtsJNcmNm1wbyBc27U0fc6opLOubWB91prZiUOs99A4D4g/3F+nohkQsocEYkU5Y2IhNORZhG8OojXO+Co4WNm3wKlDrGpdxCfgZm1ATY452aY2TlB7N8d6A5QoYKWsBDJIrJk5ihvRLIk5Y2IhM1hG1jOuWah+hDn3HmH22Zm682sdODMTmlgwyF2OxNoa2atgVxAATMb55y76jCfNwwYBtCgQQN3/N9ARMItq2aO8kYk61HeiEg4BTvJBWZW1MyuNrN7A4/LBPoMH6/xwDWB+9cAHx+8g3Oul3OunHOuEnAFMOFwjSsRyR6UOSISKcobEQmloBpYZtYUWAB0AvoEnq6On8b0eD0FtDCzRUCLwOP94fZ5CN5fRLIYZY6IRIryRkRC7UhjsDIaCFzunPvOzLYGnpsKNDreApxzm4FzD/H8GqD1IZ6fBEw63s8VkUxtIMocEYmMgShvRCSEgu0iWMk5913g/v4+v8kE30ATETkWyhwRiRTljYiEVLANrPlm1vKg584D5oS4HhERUOaISOQob0QkpII9O3M38KmZfQbkNrNXgAvxC+iJiISaMkdEIkV5IyIhFdQVLOfcFKAuMA+/JsQyoJFz7tcw1iYiOZQyR0QiRXkjIqEWdP9i59xqoP/+x2Z2spkNcM5dGpbKRCRHU+aISKQob0QklI54BcvM8pjZY2b2iZk9b2YFzKyKmX0I/MyhF8wTEflPlDkiEinKGxEJl6NdwXoROAX4CmgFnATUBMYA3Zxzm8JbnojkMMocEYkU5Y2IhMXRGlgtgXrOuQ1mNhhYATR1zv0Q/tJEJAdS5ohIpChvRCQsjjbJRT7n3AYA59wqYJeCR0TCSJkjIpGivBGRsDjaFaw4M2sG2P4nDn7snJsQptpEJOdR5ohIpChvRCQsjtbA2oCfsnS/zQc9dkCVUBclIjmWMkdEIkV5IyJhccQGlnOuUoTqEBFR5ohIxChvRCRcglpoWERERERERI5ODSwREREREZEQUQNLREREREQkRNTAEhERERERCRFzzkW7hrAys43A8iB2LQZk9lXbVWNoqMbQCLbGis654uEuJjNQ3kScagyN7FSj8ubQstPfcTRl9hoze32Q/Wo8ZOZk+wZWsMxsunOuQbTrOBLVGBqqMTSyQo2ZVVb42anG0FCNoZEVaszMssLPTzUev8xeH+ScGtVFUEREREREJETUwBIREREREQkRNbAOGBbtAoKgGkNDNYZGVqgxs8oKPzvVGBqqMTSyQo2ZWVb4+anG45fZ64McUqPGYImIiIiIiISIrmCJiIiIiIiEiBpYIiIiIiIiIZKjGlhmdr6ZLTCzxWZ2/yG2m5m9ENg+28zqZ8IaOwVqm21mP5tZ3cxWY4b9GppZmpldEsn6Ap991BrN7Bwz+93M5pnZ5MxWo5kVNLNPzGxWoMauUahxlJltMLO5h9ke9d+ZzEyZE5kaM+ynzDmOGqOdOcqb46O8iUyNGfZT3hxHjdk+b5xzOeIGxAJLgCpAAjALqH3QPq2BLwADTgemZsIazwAKB+63yow1ZthvAvA5cElmqxEoBMwHKgQel8iENT4APB24XxzYAiREuM6zgfrA3MNsj+rvTGa+KXMiV2OG/ZQ5x1djVDNHeRP2v1/lTQhqzLCf8ub4aszWeZOTrmA1AhY755Y655KBt4CLDtrnImCs86YAhcysdGaq0Tn3s3Nua+DhFKBcBOsLqsaAW4H3gQ2RLC4gmBo7Ah8451YAOOciXWcwNTogv5kZkA8fPqmRLNI5933gcw8n2r8zmZkyJ0I1Bihzjr/GqGaO8ua4KG8iVGOA8ub4a8zWeZOTGlhlgZUZHq8KPHes+4TTsX7+dfjWdSQdtUYzKwu0A4ZGsK6Mgvk51gAKm9kkM5thZp0jVp0XTI1DgFrAGmAOcLtzLj0y5QUt2r8zmZkyJzSUOaGRHTIn2r8vmZnyJjSUN6GR4/MmLuTlZF52iOcOnqM+mH3CKejPN7Nm+PA5K6wVHeKjD/HcwTUOBHo659L8iYmIC6bGOOBU4FwgN/CLmU1xzi0Md3EBwdTYEvgdaA5UBb4xsx+cczvCXNuxiPbvTGamzAkNZU5oZIfMifbvS2amvAkN5U1o5Pi8yUkNrFVA+QyPy+Fbzce6TzgF9flmdjIwAmjlnNscodr2C6bGBsBbgeApBrQ2s1Tn3EcRqTD4v+tNzrndwG4z+x6oC0QqfIKpsSvwlPOdgReb2TKgJjAtMiUGJdq/M5mZMic0lDmhkR0yJ9q/L5mZ8iY0lDehobw5lgFbWfmGb0wuBSpzYMBdnYP2uYB/DmiblglrrAAsBs7IrD/Hg/YfTeQHgAbzc6wFfBfYNw8wFzgxk9X4MvBI4H5JYDVQLAp/55U4/CDQqP7OZOabMidyNR60vzLnv9cY9cxR3oT171d5E4IaD9pfefPfa8zWeZNjrmA551LN7BbgK/zsJqOcc/PM7MbA9qH42WBa43+59+Bb15mtxj5AUeClwNmTVOdcg0xWY1QFU6Nz7g8z+xKYDaQDI5xzh5yqM1o1Ao8Bo81sDv4XvKdzblOkagQwszeBc4BiZrYKeBiIz1BjVH9nMjNlTkRrjCplTmgob/475U1Ea4wq5U1ohDtvLNBKExERERERkeOUk2YRFBERERERCSs1sEREREREREJEDSwREREREZEQUQNLREREREQkRNTAEhERERERCRE1sEREREREREJEDSwREREREZEQ+T+FtMN8RW16DgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x288 with 3 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "## plotting\n",
    "plt.figure();\n",
    "fig, axs = plt.subplots(1,3, figsize=(12,4));\n",
    "\n",
    "titles = ['Grass%_100', 'Canopy%_100', 'Shrubs%_100']\n",
    "for i, ax in enumerate(axs):\n",
    "    XX = gam_100_2.generate_X_grid(term=i)\n",
    "    ax.plot(XX[:, i], gam_100_2.partial_dependence(term=i, X=XX))\n",
    "    ax.plot(XX[:, i], gam_100_2.partial_dependence(term=i, X=XX, width=.95)[1], c='r', ls='--')\n",
    "    ax.set_ylim(-0.5, 0.6)\n",
    "    ax.set_xticks(list(ax.get_xticks()))\n",
    "    ax.hlines(y=0, xmin=0, xmax=1, linewidth=1, color='black', linestyle='--')\n",
    "    ax.set_ylabel('Relative change of NDVI_100 (mean)', fontsize=12)\n",
    "    ax.set_title(titles[i], fontsize=12);\n",
    "    \n",
    "# non-overlapping subplots\n",
    "plt.tight_layout()\n",
    "    \n",
    "# save the figure\n",
    "plt.savefig('thesis_project/visualizations/eda/gam_ndvi_100_simple_plot.png', dpi=300, bbox_inches='tight')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab404fd8",
   "metadata": {},
   "source": [
    "- **GAM model NDVI_100 optimizing for combination of GCV, DoF, and AIC (simpler-version)**\n",
    "    - **Distribution**: Normal\n",
    "    - **Link Function**: Identity\n",
    "    - **Functional Form**: NDVI_100 ~ linear(grass%_100) + linear(canopy%_100) + spline(shrubs%_100)\n",
    "    - **Regularization penalty**: lambda = [0.1, 0.1, 10]\n",
    "    - **Smoothness**: n_splines = 20"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}