NHL-Modeling_v2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "dd854976",
   "metadata": {},
   "source": [
    "@author: Betty Au (bettyau), Claire Cai (ClaireCai97)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "05bed39a",
   "metadata": {},
   "source": [
    "# Step 5: Modeling"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2c47b2e8",
   "metadata": {},
   "source": [
    "We will examine supervised machine learning models on sports betting of NHL hockey data. The target and control features are defined as follows: <br>\n",
    "- <b>Target Feature</b>: ```won``` <br>\n",
    "- <b>Control Features</b>: ```pim```, ```powerPlayGoals```, ```faceOffWinPercentage```, ```shots```, ```goals```, ```takeaways```, ```hits```, ```blockedShots```, ```giveaways```, ```missedShots```, ```penalties```, ```timeOnIce```, ```evenTimeOnIce```, ```shortHandedTimeOnIce```, ```powerPlayTimeOnIce```, ```hoa_away```, ```hoa_home```, ```settledIn_OT```, ```settledIn_REG```, ```startRinkSide_left```, ```startRinkSide_right```,\n",
    "```goalieReplacement_No```, ```goalieReplacement_Yes``` <br>\n",
    "\n",
    "The modeling objective is to investigate factors that contribute to the outcome of a hockey game and predict which NHL hockey team will win a game. The results can be used to make data-driven decisions regarding the game's outcome. The following set of models is used:\n",
    "- <b>Logistic Regression</b>\n",
    "- <b>Artificial Neural Networks</b>\n",
    "- <b>Random Forest</b>\n",
    "- <b>Gradient Boosting</b>\n",
    "\n",
    "Specifically, logistic regression will serve as a baseline model for the classification problem and the rest of the classification models will be built and compared against its model performance, as well as feature importance. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "272f6635",
   "metadata": {},
   "source": [
    "## 5.1 Load Libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "e2473c72",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "from sklearn import ensemble, model_selection, metrics\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.neural_network import MLPClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.ensemble import GradientBoostingClassifier\n",
    "\n",
    "from sklearn.metrics import (\n",
    "    confusion_matrix, \n",
    "    accuracy_score, \n",
    "    precision_recall_curve,\n",
    "    average_precision_score,\n",
    "    roc_curve,\n",
    "    roc_auc_score,\n",
    "    classification_report,\n",
    "    plot_confusion_matrix\n",
    ")\n",
    "\n",
    "# Where to save the figures\n",
    "PROJECT_ROOT_DIR = \".\"\n",
    "CHAPTER_ID = \"INSY695_Group_Project\"\n",
    "IMAGES_PATH = os.path.join(PROJECT_ROOT_DIR, \"images\", CHAPTER_ID)\n",
    "os.makedirs(IMAGES_PATH, exist_ok=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fd00c6a5",
   "metadata": {},
   "source": [
    "## 5.2 Data import"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "f96f7b7e",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(\"/Users/bettyau/Downloads/readyformodel_V2.1.csv\")\n",
    "\n",
    "# define x and y\n",
    "X, y = df.drop(['won'], axis=1), df.won\n",
    "\n",
    "# split the data \n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8976d2ca",
   "metadata": {},
   "outputs": [],
   "source": [
    "# standarize the dataset\n",
    "scaler = StandardScaler()\n",
    "\n",
    "X_train_sc = scaler.fit_transform(X_train)\n",
    "X_test_sc= scaler.fit_transform(X_test) # scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58508095",
   "metadata": {},
   "source": [
    "## 5.3 Helpful function"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d444e1bc",
   "metadata": {},
   "outputs": [],
   "source": [
    "def save_fig(fig_id, tight_layout=True, fig_extension=\"png\", resolution=300):\n",
    "    path = os.path.join(IMAGES_PATH, fig_id + \".\" + fig_extension)\n",
    "    print(\"Saving figure\", fig_id)\n",
    "    if tight_layout:\n",
    "        plt.tight_layout()\n",
    "    plt.savefig(path, format=fig_extension, dpi=resolution)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d90cb38e",
   "metadata": {},
   "source": [
    "### Classification report"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6b961ed8",
   "metadata": {},
   "outputs": [],
   "source": [
    "def clf_score(clf, X_train, y_train, X_val, y_val, train=True):\n",
    "    if train:\n",
    "        print(\"Train Result:\\n\")\n",
    "        print(\"accuracy score: {0:.4f}\\n\".format(accuracy_score(y_train, clf.predict(X_train))))\n",
    "        print(\"Classification Report: \\n {}\\n\".format(classification_report(y_train, clf.predict(X_train))))\n",
    "        print(\"Confusion Matrix: \\n {}\\n\".format(confusion_matrix(y_train, clf.predict(X_train))))\n",
    "\n",
    "        res = cross_val_score(clf, X_train, y_train, cv=10, scoring='accuracy')\n",
    "        print(\"Average Accuracy: \\t {0:.4f}\".format(np.mean(res)))\n",
    "        print(\"Accuracy SD: \\t\\t {0:.4f}\".format(np.std(res)))\n",
    "\n",
    "    elif train == False:\n",
    "        print(\"Validation Result:\\n\")\n",
    "        print(\"accuracy score: {0:.4f}\\n\".format(accuracy_score(y_val, clf.predict(X_val))))\n",
    "        \n",
    "        precision, recall, _ = precision_recall_curve(y_val, clf.predict(X_val))\n",
    "        average_precision = average_precision_score(y_val, clf.predict(X_val))\n",
    "        plot_pr_curve(precision, recall, average_precision)\n",
    "        \n",
    "        fpr, tpr, _ = roc_curve(y_val, clf.predict(X_val))\n",
    "        roc_auc = roc_auc_score(y_val, clf.predict(X_val))\n",
    "        print(\"roc auc score: {}\\n\".format(roc_auc))\n",
    "        plot_roc_curve(fpr, tpr, roc_auc)\n",
    "        \n",
    "        print(\"Classification Report: \\n {}\\n\".format(classification_report(y_val, clf.predict(X_val))))\n",
    "        print(\"Confusion Matrix: \\n {}\\n\".format(confusion_matrix(y_val, clf.predict(X_val))))\n",
    "        plot_confusion_matrix(clf,  X_val, clf.predict(X_val))\n",
    "        print(\"End of validation Result\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ca68eaa",
   "metadata": {},
   "source": [
    "### PR Curve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "7b7ca0cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_pr_curve(precision, recall, average_precision):\n",
    "    plt.step(recall, precision, color='b', alpha=0.2, where='post')\n",
    "    plt.fill_between(recall, precision, step='post', alpha=0.2, color='b')   \n",
    "    plt.xlabel('Recall')\n",
    "    plt.ylabel('Precision')\n",
    "    plt.ylim([0.0, 1.05])\n",
    "    plt.xlim([0.0, 1.0])\n",
    "    plt.title('2-class Precision-Recall curve: AP={0:0.2f}'.format(average_precision))\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a1edfe01",
   "metadata": {},
   "source": [
    "### ROC Curve"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "c47ef758",
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_roc_curve(fpr, tpr, roc_auc):\n",
    "    plt.title('Receiver Operating Characteristic')\n",
    "    plt.plot(fpr, tpr, 'b', label = 'AUC = %0.2f' % roc_auc)\n",
    "    plt.legend(loc = 'lower right')\n",
    "    plt.plot([0, 1], [0, 1],'r--')\n",
    "    plt.xlim([0, 1])\n",
    "    plt.ylim([0, 1])\n",
    "    plt.ylabel('True Positive Rate')\n",
    "    plt.xlabel('False Positive Rate')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b255be0a",
   "metadata": {},
   "source": [
    "## 5.4 Build Model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "138afd05",
   "metadata": {},
   "source": [
    "### 5.4.1 Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "4d62111b",
   "metadata": {},
   "outputs": [],
   "source": [
    "lr = LogisticRegression()\n",
    "model1 = lr.fit(X_train_sc, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "48d99ecc",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.6689294026377036\n"
     ]
    }
   ],
   "source": [
    "# cross-validation\n",
    "y_test_pred = model1.predict(X_test_sc)\n",
    "accuracy_lr = metrics.accuracy_score(y_test, y_test_pred)\n",
    "print(accuracy_lr)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "75186230",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>predictor</th>\n",
       "      <th>feature importance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>settledIn_REG</td>\n",
       "      <td>1.214892</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>settledIn_OT</td>\n",
       "      <td>1.181894</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>goals</td>\n",
       "      <td>0.585512</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>powerPlayGoals</td>\n",
       "      <td>0.380196</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>goalieReplacement_No</td>\n",
       "      <td>0.268579</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>hoa_home</td>\n",
       "      <td>0.087368</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>shortHandedTimeOnIce</td>\n",
       "      <td>0.084293</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>shots</td>\n",
       "      <td>0.024857</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>faceOffWinPercentage</td>\n",
       "      <td>0.017338</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>missedShots</td>\n",
       "      <td>0.006951</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>startRinkSide_right</td>\n",
       "      <td>0.005182</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>startRinkSide_left</td>\n",
       "      <td>-0.005182</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>takeaways</td>\n",
       "      <td>-0.016745</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>blockedShots</td>\n",
       "      <td>-0.022160</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>hits</td>\n",
       "      <td>-0.047984</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>giveaways</td>\n",
       "      <td>-0.053044</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>evenTimeOnIce</td>\n",
       "      <td>-0.077128</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>penalties</td>\n",
       "      <td>-0.082075</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>hoa_away</td>\n",
       "      <td>-0.087368</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>timeOnIce</td>\n",
       "      <td>-0.153122</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pim</td>\n",
       "      <td>-0.154291</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>powerPlayTimeOnIce</td>\n",
       "      <td>-0.207012</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>goalieReplacement_Yes</td>\n",
       "      <td>-0.268579</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                predictor  feature importance\n",
       "18          settledIn_REG            1.214892\n",
       "17           settledIn_OT            1.181894\n",
       "4                   goals            0.585512\n",
       "1          powerPlayGoals            0.380196\n",
       "21   goalieReplacement_No            0.268579\n",
       "16               hoa_home            0.087368\n",
       "13   shortHandedTimeOnIce            0.084293\n",
       "3                   shots            0.024857\n",
       "2    faceOffWinPercentage            0.017338\n",
       "9             missedShots            0.006951\n",
       "20    startRinkSide_right            0.005182\n",
       "19     startRinkSide_left           -0.005182\n",
       "5               takeaways           -0.016745\n",
       "7            blockedShots           -0.022160\n",
       "6                    hits           -0.047984\n",
       "8               giveaways           -0.053044\n",
       "12          evenTimeOnIce           -0.077128\n",
       "10              penalties           -0.082075\n",
       "15               hoa_away           -0.087368\n",
       "11              timeOnIce           -0.153122\n",
       "0                     pim           -0.154291\n",
       "14     powerPlayTimeOnIce           -0.207012\n",
       "22  goalieReplacement_Yes           -0.268579"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get importance\n",
    "importance = model1.coef_[0]\n",
    "\n",
    "# summarize feature importance\n",
    "pd.DataFrame(list(zip(X_train.columns, importance)), \n",
    "             columns = ['predictor','feature importance']).sort_values(\"feature importance\", ascending = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "0b82ae7e",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXwAAAD4CAYAAADvsV2wAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAPl0lEQVR4nO3db4xcV33G8e9TO3nRQssfLzTYTu1WpsUvSIBtgkRbQmmonbwwSFRKQCEgIjdSXIHEi1itCpVQJVcISisClhusBIliVSIFl5imNKJNURTkDUqTGCuwMvmz2Io3gADBi8jk1xczaUbLrj27cz2z3vP9SKude87J/Z1cXz9zdXzvTKoKSdLa9yuTnoAkaTwMfElqhIEvSY0w8CWpEQa+JDVi/aQncDYbNmyoLVu2THoaknTBePDBB5+pqqnF+lZ14G/ZsoWZmZlJT0OSLhhJnliqzyUdSWqEgS9JjTDwJakRBr4kNcLAl6RGGPiS1AgDX5Ia0UngJzmY5HSSR5fof3eSh/s/9ye5rIu6kqThdfXg1R3Ap4DPLdH/PeDNVfWjJDuBA8CVHdWWtMZs2Xv30GMf33fteZzJ2tJJ4FfVfUm2nKX//oHNB4BNXdSVJA1vEmv47we+ulRnkt1JZpLMzM/Pj3FakrS2jTXwk7yFXuDfutSYqjpQVdNVNT01tejn/0iSVmBsH56W5LXA7cDOqvrBuOpKknrGEvhJLgXuAm6oqu+Mo6ak1WHYf4D1H1/Pv04CP8kXgKuADUnmgI8AFwFU1X7gw8DLgU8nAThTVdNd1JYkDaeru3SuP0f/TcBNXdSSJK2MT9pKUiMMfElqhIEvSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWpEJ4Gf5GCS00keXaI/Sf4xyWySh5O8vou6kqThdXWFfwew4yz9O4Ft/Z/dwGc6qitJGlIngV9V9wE/PMuQXcDnqucB4CVJLumitiRpOONaw98IPDWwPddv+yVJdieZSTIzPz8/lslJUgvGFfhZpK0WG1hVB6pquqqmp6amzvO0JKkd4wr8OWDzwPYm4OSYakuSGF/gHwbe079b543Aj6vq1JhqS5KA9V3sJMkXgKuADUnmgI8AFwFU1X7gCHANMAv8HHhfF3UlScPrJPCr6vpz9BdwSxe1JEkr45O2ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWqEgS9JjTDwJakRBr4kNcLAl6RGGPiS1IhOAj/JjiSPJZlNsneR/t9I8m9J/jfJsSR+xaEkjdnIgZ9kHXAbsBPYDlyfZPuCYbcA366qy+h99+3Hk1w8am1J0vC6uMK/ApitqhNV9SxwCNi1YEwBL04S4EXAD4EzHdSWJA2pi8DfCDw1sD3Xbxv0KeA1wEngEeADVfVcB7UlSUPqIvCzSFst2P5T4CHgVcDlwKeS/PqiO0t2J5lJMjM/P9/B9CRJ0E3gzwGbB7Y30buSH/Q+4K7qmQW+B/zeYjurqgNVNV1V01NTUx1MT5IE3QT+UWBbkq39f4i9Dji8YMyTwFsBkrwS+F3gRAe1JUlDWj/qDqrqTJI9wD3AOuBgVR1LcnO/fz/wUeCOJI/QWwK6taqeGbW2JGl4Iwc+QFUdAY4saNs/8Pok8LYuakmSVsYnbSWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWqEgS9JjTDwJakRBr4kNaKTwE+yI8ljSWaT7F1izFVJHkpyLMl/d1FXkjS8kb/TNsk64DbgamAOOJrkcFV9e2DMS4BPAzuq6skkrxi1riRpebq4wr8CmK2qE1X1LHAI2LVgzLuAu6rqSYCqOt1BXUnSMnQR+BuBpwa25/ptg14NvDTJfyV5MMl7ltpZkt1JZpLMzM/PdzA9SRJ0E/hZpK0WbK8H3gBcC/wp8NdJXr3YzqrqQFVNV9X01NRUB9OTJEEHa/j0rug3D2xvAk4uMuaZqvoZ8LMk9wGXAd/poH4ztuy9e+ixj++79jzORNKFqIsr/KPAtiRbk1wMXAccXjDmy8AfJlmf5FeBK4HjHdSWJA1p5Cv8qjqTZA9wD7AOOFhVx5Lc3O/fX1XHk/w78DDwHHB7VT06am1J0vC6WNKhqo4ARxa07V+w/THgY13UkyQtn0/aSlIjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWqEgS9JjTDwJakRBr4kNcLAl6RGGPiS1AgDX5IaYeBLUiMMfElqRCeBn2RHkseSzCbZe5Zxv5/kF0ne2UVdSdLwRg78JOuA24CdwHbg+iTblxj3d/S+7FySNGZdXOFfAcxW1YmqehY4BOxaZNxfAF8ETndQU5K0TF0E/kbgqYHtuX7b/0uyEXgHsP9cO0uyO8lMkpn5+fkOpidJgm4CP4u01YLtTwK3VtUvzrWzqjpQVdNVNT01NdXB9CRJAOs72MccsHlgexNwcsGYaeBQEoANwDVJzlTVlzqo35kte+8eeuzj+649jzORpO51EfhHgW1JtgLfB64D3jU4oKq2Pv86yR3AV1Zb2EvSWjdy4FfVmSR76N19sw44WFXHktzc7z/nur0k6fzr4gqfqjoCHFnQtmjQV9V7u6gpSVoen7SVpEYY+JLUCANfkhph4EtSIwx8SWqEgS9JjTDwJakRBr4kNcLAl6RGGPiS1AgDX5Ia0cln6UhSSy7Uj1L3Cl+SGmHgS1IjDHxJaoSBL0mNMPAlqRGdBH6SHUkeSzKbZO8i/e9O8nD/5/4kl3VRV5I0vJEDP8k64DZgJ7AduD7J9gXDvge8uapeC3wUODBqXUnS8nRxhX8FMFtVJ6rqWeAQsGtwQFXdX1U/6m8+AGzqoK4kaRm6CPyNwFMD23P9tqW8H/jqUp1JdieZSTIzPz/fwfQkSdBN4GeRtlp0YPIWeoF/61I7q6oDVTVdVdNTU1MdTE+SBN18tMIcsHlgexNwcuGgJK8Fbgd2VtUPOqgrSVqGLq7wjwLbkmxNcjFwHXB4cECSS4G7gBuq6jsd1JQkLdPIV/hVdSbJHuAeYB1wsKqOJbm5378f+DDwcuDTSQDOVNX0qLUlScPr5NMyq+oIcGRB2/6B1zcBN3VRS5K0Mj5pK0mNMPAlqREGviQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWqEgS9JjejkSVtJa9+WvXcPPfbxfdeex5lopQx8aZUwUHW+uaQjSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjOrkPP8kO4B/ofaft7VW1b0F/+v3XAD8H3ltV3+qi9oXKe64ljdvIgZ9kHXAbcDUwBxxNcriqvj0wbCewrf9zJfCZ/u8LnsEt6ULRxRX+FcBsVZ0ASHII2AUMBv4u4HNVVcADSV6S5JKqOtVBfTXIN1p1ZdhzaS2cR+ll8Ag7SN4J7Kiqm/rbNwBXVtWegTFfAfZV1Tf62/cCt1bVzCL72w3sBrj00kvf8MQTT6xoXi39IZ7LSo7FSgJ1pSE8rj+rcc5vnG9Iq31+47La/5/GNb8kD1bV9GJ9XVzhZ5G2he8iw4zpNVYdAA4ATE9Pj/ZuJKkZF8ob0yR1EfhzwOaB7U3AyRWM0XniXwRJ0E3gHwW2JdkKfB+4DnjXgjGHgT399f0rgR+7fr/2+MYirW4jB35VnUmyB7iH3m2ZB6vqWJKb+/37gSP0bsmcpXdb5vtGrStJWp5O7sOvqiP0Qn2wbf/A6wJu6aKWJGllfNJWkhrhN15pUeNcj3ftXxoPr/AlqREGviQ1wiUd6TxwmUqrkYGvZhjCap2BLzXIN782uYYvSY0w8CWpEQa+JDXCwJekRhj4ktQIA1+SGmHgS1IjDHxJasSaffDKB0vUFc8lrRVe4UtSIwx8SWrESIGf5GVJvpbku/3fL11kzOYkX09yPMmxJB8YpaYkaWVGvcLfC9xbVduAe/vbC50BPlRVrwHeCNySZPuIdSVJyzRq4O8C7uy/vhN4+8IBVXWqqr7Vf/1T4DiwccS6kqRlGjXwX1lVp6AX7MArzjY4yRbgdcA3zzJmd5KZJDPz8/MjTk+S9Lxz3paZ5D+B31yk66+WUyjJi4AvAh+sqp8sNa6qDgAHAKanp2s5NaQWeduohnXOwK+qP1mqL8nTSS6pqlNJLgFOLzHuInph//mqumvFs5UkrdioSzqHgRv7r28EvrxwQJIAnwWOV9UnRqwnSVqhUQN/H3B1ku8CV/e3SfKqJEf6Y94E3AD8cZKH+j/XjFhXkrRMI320QlX9AHjrIu0ngWv6r78BZJQ6kqTR+aStJDXCwJekRhj4ktSINfvxyJK0mqyG5yW8wpekRhj4ktQIA1+SGmHgS1IjDHxJaoSBL0mNMPAlqREGviQ1wsCXpEakavV+qVSSeeCJDne5AXimw/1dyDwWL/BY9HgcXnAhH4vfqqqpxTpWdeB3LclMVU1Peh6rgcfiBR6LHo/DC9bqsXBJR5IaYeBLUiNaC/wDk57AKuKxeIHHosfj8II1eSyaWsOXpJa1doUvSc0y8CWpEc0EfpIdSR5LMptk76TnM0lJHk/ySJKHksxMej7jkuRgktNJHh1oe1mSryX5bv/3Syc5x3FZ4lj8TZLv98+Lh5JcM8k5jkOSzUm+nuR4kmNJPtBvX5PnRROBn2QdcBuwE9gOXJ9k+2RnNXFvqarL1+K9xmdxB7BjQdte4N6q2gbc299uwR388rEA+Pv+eXF5VR0Z85wm4Qzwoap6DfBG4JZ+NqzJ86KJwAeuAGar6kRVPQscAnZNeE4as6q6D/jhguZdwJ3913cCbx/nnCZliWPRnKo6VVXf6r/+KXAc2MgaPS9aCfyNwFMD23P9tlYV8B9JHkyye9KTmbBXVtUp6P3lB14x4flM2p4kD/eXfNbEMsawkmwBXgd8kzV6XrQS+FmkreX7Ud9UVa+nt8R1S5I/mvSEtCp8Bvgd4HLgFPDxic5mjJK8CPgi8MGq+smk53O+tBL4c8Dmge1NwMkJzWXiqupk//dp4F/pLXm16ukklwD0f5+e8HwmpqqerqpfVNVzwD/RyHmR5CJ6Yf/5qrqr37wmz4tWAv8osC3J1iQXA9cBhyc8p4lI8mtJXvz8a+BtwKNn/6/WtMPAjf3XNwJfnuBcJur5gOt7Bw2cF0kCfBY4XlWfGOhak+dFM0/a9m8x+ySwDjhYVX872RlNRpLfpndVD7Ae+OdWjkWSLwBX0fvo26eBjwBfAv4FuBR4Evizqlrz/5i5xLG4it5yTgGPA3/+/Dr2WpXkD4D/AR4Bnus3/yW9dfw1d140E/iS1LpWlnQkqXkGviQ1wsCXpEYY+JLUCANfkhph4EtSIwx8SWrE/wHgTVGrQgPe9QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot feature importance\n",
    "plt.bar([x for x in range(len(importance))], importance)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "4f1a354e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation Result:\n",
      "\n",
      "accuracy score: 0.6689\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAagklEQVR4nO3de9QddX3v8ffHBBG5BBSwEi5BCkW0QDGCdxFRCcWDt6MgiiJKo9K6eno8sLrqhWJbL0etLkCggmhBqBeOokU5ViroUSphycWAQOQaARWBIBeBwPf8MROyfXwyz36eZJ5nJ7xfa+2VPTO/Pfs7v+xnPnsueyZVhSRJq/K4mS5AkjTaDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng2Itl+StSX4w03WsSUkOSfJ/h2h3YpL3TUdN0yHJDUn2bZ9/MMnpM12TBAbFjEiyfpJTktyY5LdJfpJkwUzXNYx2ZXZ/knuS/DLJ55JstCbfo6rOqKqXD9FuYVUduybfe4UkleTedjl/keQTSWb18V7ruiSnJVmeZKsx4z+Y5KG2j+9K8sMkz53C/N/Y/i3dm+RrSZ40Qfv3JLm+bX9Vkp3a8U9Nck6SW9r//3mTrWVdZVDMjNnAzcCLgTnA+4AvrUUfzFdW1UbAHsCzgb8b2yDJ7Gmvas3brV3OFwNvAN42w/WsUdPxf5RkQ+C1wDLgkHGa/Fvbx1sAPwDOTpJJzP8ZwEnAm4GnAPcBJ3S0fztwOPDnwEbAAcDt7eRHgG+39WqAQTEDqureqvpgVd1QVY9U1TeB64Fnreo1SbZJcnaSXyf5TZLjVtHuU0luTnJ3kkuSvHBg2p5JFrXTfpnkE+34JyQ5vZ3vXUkuTvKUIZbjF8C3gGe286kk705yLXBtO+6AJJcOfGPcdaJlGtydlsYnk/wqybIklydZ8X6nJfnQwPzekWRJkjvab4ZbDUyrJAuTXJvkziTHD7tCqqolwP8Ddh+Y31SWa4ck57fjbk9yRpJNh6lhrCQHtu9/d5KfJ9mvHf/o7qt2+NFdWEnmtf1weJKbgPOTfDvJkWPmfVmS17TPd07ynbZPr07y+kmW+lrgLuDvgbesqlFVPQR8Hvgj4MmTmP8hwDeq6sKquofmS9drkmw8tmGSxwEfAP66qq6sxs+r6o62hl9W1QnAxZN4/8cEg2IEtCvlnYDFq5g+C/gmcCMwD5gLnLWK2V1Ms0J7EvBF4MtJntBO+xTwqaraBNgB+FI7/i00Wzbb0PyRLgTuH6LubYD9gZ8MjH4VsBewS5I9gFOBv2jnexJwTppdb8Mu08uBF9H0z6Y03+x/M04t+wD/BLweeGo737HzO4BmC2i3tt0rJlrGdt47Ay8ElrTDU12utDVuBTydpr8/OEwNY+rZE/gC8F6aPnkRcMMkZvHi9v1fQfMZOXhg3rsA2wH/3m4NfKdts2Xb7oT2W/yKXT6XT/BebwHOpOmDndu+G2+Z1gfeCiytqtuTvKAN4VU9XtC+9BnAZSvmU1U/Bx6k+byMtXX7eGb7Zer6JMe0AaIuVeVjBh/AesB/ACd1tHku8Gtg9jjT3gr8oOO1d9LsQgG4EDgG2HxMm7cBPwR2HaLeG4B7aL4l3kizmb9BO62AfQbafgY4dszrr6ZZUQ21TMA+wDXAc4DHjWl3GvCh9vkpwEcHpm0EPATMG6jtBQPTvwQc3bGcBdwN3Ns+PxNYf3WWa5z3eBXwkzF9u2/7/IPA6at43UnAJzv+f/YdGH50PjTBVcDTBqZv3C7jdu3wPwCnts/fAHx/nPf+wJCf7W1pdufs3g6fR/NFZbC2B9vP0q+A84FnTfLv57vAwjHjfgHsPU7b57XL/+80ATuv/Wy9Y0y72W27eZOpZV1+mKQzqP0m8680fyxHDoz/VpoDfPckOYTmm+eNVbV8iHn+TZoDdMuS3EWzpbB5O/lwmm9aP2t3Lx3Qjv9Xmj/is9IcyPtokvU63uZVVbVpVW1XVe+qqsGtj5sHnm8H/M3gN8F2WbYadpmq6nzgOOB44JdJTk6yyThNt6IJrhWvu4dmy2PuQJvbBp7fRxMmJFk80N8vHGizR9vmDTRbSRuuznIl2TLJWWkOjt8NnM7K/5vJ2Ab4+RRet8Kj/0dV9VuaFedB7aiDgDPa59sBe41ZzkNodg8N483AVVV1aTt8BvDGMZ+tL7WfpS2rap+qumSSy3IPMPbzsAnw23HarvicfrSq7qqqG2iCb/9JvudjjkExQ9r946fQHIB7bTX7aAGoqgVVtVH7OIPmD3vbTHDwsV3JHUWzW2WzqtqU5iBi2vleW1UH0+xG+AjwlSQbVtVDVXVMVe1C863rAODQKS7a4OWIbwb+oV0RrHg8sarOHHaZ2ro/XVXPotnNsBPNLpexbqFZsQGPHkR9Ms23y4nm/4yB/v7+mGlVVV8CfgS8fzWX659o+mfXanb/vYn2/2aSbqbZdTiee4EnDgyPt1Ife8noM4GD05xxtAHwnwPvc8GY5dyoqt45ZJ2HAk9LcluS24BP0ATjhGf4JXnhQHiP91gR6ItpdiWueN3TgPVpthTGuprmS5mXzJ4kg2LmfIZmP/Erx3wjH8+PgVuBDyfZMM3B5+eP025jYDntro8k72fg21aSNyXZoqoeodncB3g4yUuS/Gm7f/1uml02D6/OwrX+BViYZK80Nkzy5+2BxqGWKcmz29evR7MS/N0qavsicFiS3dv93f8I/Ff7rXFN+DBwRJI/Wo3l2ph2t12SuYwfeMM4hWZZX5rkcUnmtsdRAC4FDkqyXpL5wOuGmN+5NCH79zRnIT3Sjv8msFOSN7fzW6/9/3j6RDNsQ2cHYE+aY2a705z08EU6DmqvUFXfHwjv8R4rAv0M4JVtsGzYLsPZ7ZbS2HneB/wb8L+SbJxka+Ad7XKuqPsJNEEDsH5WHt97TDMoZkCS7WgOhO4O3DZmN9MfqKqHgVcCfwzcBCyl2R0y1nk0ZyFdQ7Mb5nf8/q6g/YDFSe6hObB9UFX9juZb51doQuIq4AKa3SKrpaoW0fwhHkdzrGQJzfGHySzTJjQr5jvbZfoN8L/Hea/v0pzx8lWaFfUOrNydstqq6gqafnnvaizXMTS7s5bR7O45e4q1/Bg4DPhkO68LWLk19T6aZb+zfb8vDjG/B9pa9h1s365sX07Tj7fQ7Lr7CO2KNM0PI8c9AYMmDL5eVVdU1W0rHjSfuwMywW8dhlVVi2lOvjiD5jjHxsC7VkxP86PMEwdeciRNWN9Cs5X4RZoTE1a4v50O8DOGOKnjsSBVboVJklbNLQpJUieDQpLUyaCQJHUyKCRJnda6C7dtvvnmNW/evJkuQ5LWKpdccsntVbXFVF671gXFvHnzWLRo0UyXIUlrlSQ3TtxqfO56kiR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdeguKJKemuc/xT1cxPUk+neYex5dnFbdIlCTNrD63KE6juaz1qiwAdmwfR9Dcn0GSNGJ6C4qquhC4o6PJgcAX2juIXQRsmuSpE833wQfXVIWSpGHM5DGKufz+TXWW8vv3N35UkiOSLEqy6NZb75yW4iRJjZkMivHuFTzuXZSq6uSqml9V8+fM2aznsiRJg2YyKJYC2wwMb01ze0JJ0giZyaA4Bzi0PfvpOcCyqrp1BuuRJI2jt6vHJjkT2BvYPMlS4APAegBVdSJwLrA/zY3p76O5WbwkacT0FhRVdfAE0wt4d1/vL0laM/xltiSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTr0GRZL9klydZEmSo8eZPifJN5JclmRxksP6rEeSNHm9BUWSWcDxwAJgF+DgJLuMafZu4Mqq2g3YG/h4ksf3VZMkafL63KLYE1hSVddV1YPAWcCBY9oUsHGSABsBdwDLe6xJkjRJfQbFXODmgeGl7bhBxwFPB24BrgDeU1WPjJ1RkiOSLEqyaNmyO/uqV5I0jj6DIuOMqzHDrwAuBbYCdgeOS7LJH7yo6uSqml9V8+fM2WxN1ylJ6tBnUCwFthkY3ppmy2HQYcDZ1VgCXA/s3GNNkqRJ6jMoLgZ2TLJ9e4D6IOCcMW1uAl4KkOQpwJ8A1/VYkyRpkmb3NeOqWp7kSOA8YBZwalUtTrKwnX4icCxwWpIraHZVHVVVt/dVkyRp8noLCoCqOhc4d8y4Ewee3wK8vM8aJEmrx19mS5I6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROvd4KtQ+PPALXXDPTVUhrhyc9CTbffKar0NpurQsKgAsvnOkKpNH3wANNUBx88ExXorXdWhcUs2fDn/3ZTFchjb4bb4Q77pjpKrQu8BiFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoNFRRJnp/kO0muSXJdkuuTXDfE6/ZLcnWSJUmOXkWbvZNcmmRxkgsmuwCSpH4Ne1HAU4C/Bi4BHh7mBUlmAccDLwOWAhcnOaeqrhxosylwArBfVd2UZMtJ1C5JmgbDBsWyqvrWJOe9J7Ckqq4DSHIWcCBw5UCbNwJnV9VNAFX1q0m+hySpZ8Meo/jPJB9L8twke6x4TPCaucDNA8NL23GDdgI2S/K9JJckOXTIeiRJ02TYLYq92n/nD4wrYJ+O12SccTXO+z8LeCmwAfCjJBdV1e/dwy7JEcARAFtuue2QJUuS1oShgqKqXjKFeS8FthkY3hq4ZZw2t1fVvcC9SS4EdgN+Lyiq6mTgZICddpo/NmwkST0a9qynOUk+kWRR+/h4kjkTvOxiYMck2yd5PHAQcM6YNl8HXphkdpIn0my5XDXZhZAk9WfYYxSnAr8FXt8+7gY+1/WCqloOHAmcR7Py/1JVLU6yMMnCts1VwLeBy4EfA5+tqp9OZUEkSf0Y9hjFDlX12oHhY5JcOtGLqupc4Nwx404cM/wx4GND1iFJmmbDblHcn+QFKwaSPB+4v5+SJEmjZNgtincCn2+PSwS4A3hrX0VJkkbHsGc9XQrslmSTdvjuPouSJI2OzqBI8qaqOj3J/xgzHoCq+kSPtUmSRsBEWxQbtv9u3HchkqTR1BkUVXVS++8x01OOJGnUDPuDu48m2STJekm+m+T2JG/quzhJ0swb9vTYl7cHsA+guezGTsB7e6tKkjQyhg2K9dp/9wfOrKo7eqpHkjRihv0dxTeS/IzmR3bvSrIF8Lv+ypIkjYqhtiiq6mjgucD8qnoIuJfmJkSSpHXcRL+j2Keqzk/ymoFxg03O7qswSdJomGjX04uB84FXjjOtMCgkaZ030e8oPtD+e9j0lCNJGjXD/o7iH5NsOjC8WZIP9VaVJGlkDHt67IKqumvFQFXdSXOqrCRpHTfs6bGzkqxfVQ8AJNkAWL+/siStCQ88ANdcM3E7qcuwQXE68N0kn6M5iP024PO9VSVptc2ZA7fdBhdeONOVaDRsvOHEbcY37P0oPprkcmBfmhsXHVtV5031TSX1b9NNYa+9ZroKjY5Zs6b6ymG3KACuApZX1X8keWKSjavqt1N9Y0nS2mHYs57eAXwFOKkdNRf4Wk81SZJGyLBnPb0beD5wN0BVXQts2VdRkqTRMWxQPFBVD64YSDKb5qC2JGkdN2xQXJDkb4ENkrwM+DLwjf7KkiSNimGD4ijg18AVwF8A5wJ/11dRkqTRMeFZT0keB1xeVc8E/qX/kiRJo2TCLYqqegS4LMm201CPJGnEDPs7iqcCi5P8mOamRQBU1X/rpSpJ0sgYNiiO6bUKSdLImugOd08AFgJ/THMg+5SqWj4dhUmSRsNExyg+D8ynCYkFwMd7r0iSNFIm2vW0S1X9KUCSU4Af91+SJGmUTLRF8dCKJ+5ykqTHpomCYrckd7eP3wK7rnie5O6JZp5kvyRXJ1mS5OiOds9O8nCS1012ASRJ/erc9VRVU75+eZJZwPHAy4ClwMVJzqmqK8dp9xHA+1tI0gga9hIeU7EnsKSqrmsvKHgWcOA47f4S+Crwqx5rkSRNUZ9BMRe4eWB4aTvuUUnmAq8GTuyaUZIjkixKsmjZsl+v8UIlSavWZ1BknHFjL03+z8BRVfVw14yq6uSqml9V8+fM2WJN1SdJGsJkboU6WUuBbQaGtwZuGdNmPnBWEoDNgf2TLK+qr/VYlyRpEvoMiouBHZNsD/wCOAh442CDqtp+xfMkpwHfNCQkabT0FhRVtTzJkTRnM80CTq2qxUkWttM7j0tIkkZDn1sUVNW5NDc5Ghw3bkBU1Vv7rEWSNDV9HsyWJK0DDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR16jUokuyX5OokS5IcPc70Q5Jc3j5+mGS3PuuRJE1eb0GRZBZwPLAA2AU4OMkuY5pdD7y4qnYFjgVO7qseSdLU9LlFsSewpKquq6oHgbOAAwcbVNUPq+rOdvAiYOse65EkTUGfQTEXuHlgeGk7blUOB7413oQkRyRZlGTRsmW/XoMlSpIm0mdQZJxxNW7D5CU0QXHUeNOr6uSqml9V8+fM2WINlihJmsjsHue9FNhmYHhr4JaxjZLsCnwWWFBVv+mxHknSFPS5RXExsGOS7ZM8HjgIOGewQZJtgbOBN1fVNT3WIkmaot62KKpqeZIjgfOAWcCpVbU4ycJ2+onA+4EnAyckAVheVfP7qkmSNHl97nqiqs4Fzh0z7sSB528H3t5nDZKk1eMvsyVJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdeg2KJPsluTrJkiRHjzM9ST7dTr88yR591iNJmrzegiLJLOB4YAGwC3Bwkl3GNFsA7Ng+jgA+01c9kqSp6XOLYk9gSVVdV1UPAmcBB45pcyDwhWpcBGya5Kk91iRJmqTZPc57LnDzwPBSYK8h2swFbh1slOQImi0OIA/Nn7/ZDWu21LXVA3Ng/WUzXcVosC9Wsi9Wsi9Wunvrqb6yz6DIOONqCm2oqpOBkwGSLKq6c/7ql7f2a/riPvsC+2KQfbGSfbFSkkVTfW2fu56WAtsMDG8N3DKFNpKkGdRnUFwM7Jhk+ySPBw4CzhnT5hzg0Pbsp+cAy6rq1rEzkiTNnN52PVXV8iRHAucBs4BTq2pxkoXt9BOBc4H9gSXAfcBhQ8z65J5KXhvZFyvZFyvZFyvZFytNuS9S9QeHBCRJepS/zJYkdTIoJEmdRjYovPzHSkP0xSFtH1ye5IdJdpuJOqfDRH0x0O7ZSR5O8rrprG86DdMXSfZOcmmSxUkumO4ap8sQfyNzknwjyWVtXwxzPHStk+TUJL9K8tNVTJ/aerOqRu5Bc/D758DTgMcDlwG7jGmzP/Atmt9iPAf4r5muewb74nnAZu3zBY/lvhhodz7NyRKvm+m6Z/BzsSlwJbBtO7zlTNc9g33xt8BH2udbAHcAj5/p2nvoixcBewA/XcX0Ka03R3WLwst/rDRhX1TVD6vqznbwIprfo6yLhvlcAPwl8FXgV9NZ3DQbpi/eCJxdVTcBVNW62h/D9EUBGycJsBFNUCyf3jL7V1UX0izbqkxpvTmqQbGqS3tMts26YLLLeTjNN4Z10YR9kWQu8GrgxGmsayYM87nYCdgsyfeSXJLk0GmrbnoN0xfHAU+n+UHvFcB7quqR6SlvpExpvdnnJTxWxxq7/Mc6YOjlTPISmqB4Qa8VzZxh+uKfgaOq6uHmy+M6a5i+mA08C3gpsAHwoyQXVdU1fRc3zYbpi1cAlwL7ADsA30ny/aq6u+faRs2U1pujGhRe/mOloZYzya7AZ4EFVfWbaaptug3TF/OBs9qQ2BzYP8nyqvratFQ4fYb9G7m9qu4F7k1yIbAbsK4FxTB9cRjw4Wp21C9Jcj2wM/Dj6SlxZExpvTmqu568/MdKE/ZFkm2Bs4E3r4PfFgdN2BdVtX1VzauqecBXgHetgyEBw/2NfB14YZLZSZ5Ic/Xmq6a5zukwTF/cRLNlRZKnAH8CXDetVY6GKa03R3KLovq7/MdaZ8i+eD/wZOCE9pv08qpa566YOWRfPCYM0xdVdVWSbwOXA48An62qcU+bXJsN+bk4FjgtyRU0u1+OqqrbZ6zoniQ5E9gb2DzJUuADwHqweutNL+EhSeo0qrueJEkjwqCQJHUyKCRJnQwKSVIng0KS1MmgkMbRXnn20iQ/ba86uukanv8NSTZvn9+zJuctrWkGhTS++6tq96p6Js1F1t490wVJM8WgkCb2I9oLpyXZIcm324vsfT/Jzu34pyT5P+39Di5L8rx2/NfatouTHDGDyyBN2Uj+MlsaFUlm0Vz64ZR21MnAwqq6NslewAk0F5r7NHBBVb26fc1Gbfu3VdUdSTYALk7y1XX4WlxaRxkU0vg2SHIpMA+4hOZqoxvR3CTqywNXpl2//Xcf4FCAqnoYWNaO/6skr26fbwPsCBgUWqsYFNL47q+q3ZPMAb5Jc4ziNOCuqtp9mBkk2RvYF3huVd2X5HvAE/ooVuqTxyikDlW1DPgr4H8C9wPXJ/nv8Oj9h1fcn/y7wDvb8bOSbALMAe5sQ2JnmltPSmsdg0KaQFX9hOY+zAcBhwCHJ7kMWMzKW26+B3hJe3XSS4BnAN8GZie5nObqpRdNd+3SmuDVYyVJndyikCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUqf/DwdteuDIzubEAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "roc auc score: 0.6677571254906322\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA7lklEQVR4nO3deZxV8//A8de7tC8ihJKiRKWiRL5kp+xrkuWXLWuRryUS2fcvIZEsUU0ICSVbKUKFtCop1Si0oUW0vH9/vM+Y25i5c2c5c+7yfj4e9zF3Ofec9z0zc973fD6f8/6IquKcc84VpFzUATjnnEtuniicc87F5YnCOedcXJ4onHPOxeWJwjnnXFyeKJxzzsXlicIViYjMEpHDo44jWYjILSIyKKJtvygid0ex7dImIueKyPvFfK//TYbME0UKE5EfReRPEVkrIj8HB47qYW5TVZup6vgwt5FDRCqJyH0isjj4nN+LyA0iImWx/XziOVxEsmOfU9V7VfWSkLYnItJDRGaKyDoRyRaR10Rk3zC2V1wi0ldEhpRkHao6VFWPTWBb/0qOZfk3mak8UaS+k1S1OtAK2A+4Odpwik5EtingpdeAo4DjgRrA+UA3oF8IMYiIJNv/Qz/gGqAHsD2wFzASOKG0NxTndxC6KLftEqSqfkvRG/AjcHTM4weBd2MeHwRMAn4DvgUOj3lte+AFYCmwGhgZ89qJwLTgfZOAFnm3CewK/AlsH/PafsAKoELw+CJgTrD+scDuMcsqcBXwPbAwn892FLAB2C3P8wcCm4FGwePxwH3AZOB34K08McXbB+OBe4DPgs/SCLgwiHkNsAC4LFi2WrDMFmBtcNsV6AsMCZZpEHyu/wMWB/uid8z2qgCDg/0xB7gRyC7gd9s4+Jxt4/z+XwT6A+8G8X4J7Bnzej9gCfAH8BVwaMxrfYERwJDg9UuAtsDnwb5aBjwJVIx5TzPgA2AV8AtwC9AB+BvYGOyTb4NltwWeC9bzE3A3UD54rWuwzx8N1nV38NynwesSvPZr8DudDjTHviRsDLa3Fng77/8BUD6I64dgn3xFnr8hvxXjWBN1AH4rwS9v63+QesAMoF/wuC6wEvs2Xg44Jni8Y/D6u8ArwHZABeCw4Pn9g3/QA4N/uv8LtlMpn21+DFwaE89DwNPB/VOB+cA+wDbArcCkmGU1OOhsD1TJ57PdD3xSwOdeRO4BfHxwIGqOHcxfJ/fAXdg+GI8d0JsFMVbAvq3vGRysDgPWA/sHyx9OngM7+SeKZ7Gk0BL4C9gn9jMF+7wedgAsKFFcDiwq5Pf/InagbRvEPxQYHvP6eUDt4LX/Aj8DlWPi3hj8nsoF8bbGEus2wWeZA1wbLF8DO+j/F6gcPD4w7z6I2fZI4Jngd7ITlshzfmddgU1A92BbVdg6URyHHeBrBb+HfYBdYj7z3XH+D27A/g+aBO9tCdSO+n811W+RB+C3Evzy7B9kLfbNSYGPgFrBazcBL+dZfix24N8F+2a8XT7rHADclee5ueQmkth/ykuAj4P7gn17bR88HgNcHLOOcthBd/fgsQJHxvlsg2IPenle+4Lgmzp2sL8/5rWm2DfO8vH2Qcx77yxkH48ErgnuH05iiaJezOuTgc7B/QXAcTGvXZJ3fTGv9Qa+KCS2F4FBMY+PB76Ls/xqoGVM3BMKWf+1wJvB/XOAbwpY7p99EDyugyXIKjHPnQOMC+53BRbnWUdXchPFkcA8LGmVy+czx0sUc4FTSvq/5betb8nWJuuK7lRVrYEdxPYGdgie3x04S0R+y7kBh2BJYjdglaquzmd9uwP/zfO+3bBmlrxGAO1EZFegPXaQnBiznn4x61iFJZO6Me9fEudzrQhizc8uwev5rWcRdmawA/H3Qb4xiEhHEflCRFYFyx9P7j5N1M8x99cDOQMMds2zvXiffyUFf/5EtoWI/FdE5ojI78Fn2ZatP0vez76XiLwTDIz4A7g3ZvndsOacROyO/Q6Wxez3Z7Azi3y3HUtVP8aavfoDv4jIQBGpmeC2ixKnS5AnijShqp9g37YeDp5agn2brhVzq6aq9wevbS8itfJZ1RLgnjzvq6qqWfls8zfgfaAT0AXI0uBrXbCey/Ksp4qqTopdRZyP9CFwoIjsFvukiLTFDgYfxzwdu0x9rEllRSH74F8xiEglrOnqYaCOqtYCRmMJrrB4E7EMa3LKL+68PgLqiUib4mxIRA7Fzqg6YWeOtbD2/tgRY3k/zwDgO6CxqtbE2vpzll+CNcnlJ+96lmBnFDvE7Peaqtosznu2XqHq46raGmsW3AtrUir0fYXE6YrJE0V6eQw4RkRaYZ2UJ4nIcSJSXkQqB8M766nqMqxp6CkR2U5EKohI+2AdzwKXi8iBwUigaiJygojUKGCbw4ALgDOC+zmeBm4WkWYAIrKtiJyV6AdR1Q+xg+XrItIs+AwHYe3wA1T1+5jFzxORpiJSFbgTGKGqm+PtgwI2WxGoBCwHNolIRyB2yOYvQG0R2TbRz5HHq9g+2U5E6gJXF7Rg8PmeArKCmCsG8XcWkV4JbKsG1g+wHNhGRG4DCvtWXgPr2F4rInsDV8S89g6ws4hcGwxbriEiBwav/QI0yBk1Fvx9vQ88IiI1RaSciOwpIoclEDcickDw91cBWIcNatgcs6094rx9EHCXiDQO/n5biEjtRLbrCuaJIo2o6nLgJaCPqi4BTsG+FS7HvmndQO7v/Hzsm/d3WOf1tcE6pgKXYqf+q7EO6a5xNjsKG6Hzi6p+GxPLm8ADwPCgGWMm0LGIH+kMYBzwHtYXMwQbSdM9z3IvY2dTP2MdrT2CGArbB1tR1TXBe1/FPnuX4PPlvP4dkAUsCJpU8muOi+dOIBtYiJ0xjcC+eRekB7lNML9hTSqnAW8nsK2x2JeBeVhz3AbiN3UBXI995jXYF4ZXcl4I9s0xwEnYfv4eOCJ4+bXg50oR+Tq4fwGWeGdj+3IEiTWlgSW0Z4P3LcKa4XLOlJ8Dmgb7f2Q+7/0f9vt7H0t6z2Gd5a4EJLelwLnUIyLjsY7USK6OLgkRuQLr6E7om7ZzUfEzCufKiIjsIiL/CZpimmBDTd+MOi7nChNaohCR50XkVxGZWcDrIiKPi8h8EZkuIvuHFYtzSaIiNvpnDdYZ/xbWD+FcUgut6SnoHF0LvKSqzfN5/Xisrfl47OKufqp6YN7lnHPORSu0MwpVnYCNnS/IKVgSUVX9AqglIol2djnnnCsjURbjqsvWozCyg+eW5V1QRLphdV6oVq1a67333rtMAnTOuVS1bh2sWgUVVixjpy0/8w1bVqjqjsVZV5SJIr9S0fm2g6nqQGAgQJs2bXTq1KlhxuWccylp9mzIyoJhw2DBAqViRaFv21Gcte37NB7bf1Fx1xtloshm6ytT62GVTJ1zziVo0SIYPtwSxLffwvaymiG7XM9Op+3Bns/3platk4GTQfoXextRDo8dBVwQjH46CPg9uKLTOedcHMuXw1NPwaGHQoMG0KsXVKkCb1/0Jr/u2JSOvwymdYuN1KpVOtsL7YxCRLKwQnU7iM0KdjtWKAxVfRqroXM8duXvemweAOecc/lYswZGjrRmpQ8+gM2boWlTuPtuOO+YX9j94e7w/GvQqhWMeRf2L70rDkJLFKp6TiGvKzZxjXPOuXz89ReMGWPJ4e23YcMG2H13uOEGOOcc2HdfEAGmLoF334V77rEXK1Qo1Th8CkLnnEsimzfDuHHW5/D66/D777DjjnDxxZYc2rWDcuWwzon+b8PVV0ObNrB4MdQOp/6hJwrnnIuYKkyebMnhlVfg55+hRg047TTo0gWOOgq2yTlab9kC/QdYxwTAGWfALruEliTAE4VzzkVm9mxrVsrKggULoGJFOOEESw4nnGAd1FuZOxcuuQQ+/RSOOw6eecaSRMg8UTjnXBnKO5y1XDk7Y7j1VjuDKHCk0vr1cMgh1jb14otwwQVBB0X4PFE451zIli+H116zs4fPPrPnDjoI+vWDTp1g553jvHnePGjcGKpWhZdftlFNcd9Q+rzMuHPOhWDNGjuud+xorUNXXQWrV9vApB9+gM8/hx494hzzN2yA3r1tDOzQofZchw5lniTAzyicc67UJDyctTCffWbDnObOhQsvtA6LCHmicM65Ekh4OGui7roLbr8d6teHsWPh2GMLf0/IPFE451wRFWk4a1FWKmJ9EN27WxtV9ephhF9kniiccy5BRR7OmohVq6BnT2jUCPr0gZNOslsS8UThnHNxFHs4ayJGjLBe7lWrLEkkKU8UzjmXR4mGsyZi2TIrvfHGG9C6Nbz/PrRsWeK4w+KJwjnnKLg66z33QOfOsMcepbixpUuto/qBB+C664rRoVG2kjs655wL0YYNNpw1K6uEw1kT8eOPtpHu3e0sYskS2G67Ulp5uDxROOcySrzhrF26WBNTkYazJrLB/v3hlltsxWedZW1XKZIkwBOFcy4D5AxnHTYMXn21lIazJmLOHCviN2mSXVX9zDORXFldUp4onHNpK5ThrIlavx7at7ey4C+9BOedV2ZF/EqbJwrnXFoJdThrIr77Dpo0sSJ+Q4faaKY6dULeaLi8KKBzLuUtXw5PPWVVuBs0sDl9qlSx4aw//WSjTy+8MOQk8eefcNNN0KxZbhG/Y49N+SQBfkbhnEtRZTqctTATJlhfxPff288TTyzDjYfPE4VzLmWU6XDWRN1xB/TtCw0bwocfWjtXmvFE4ZxLaoUNZ23XLqI+4pwifm3aWK2mu+6CatUiCCR8niicc0knsuGsiVixwhJD48Zw2202fCri+SLC5onCOZc08hvOeuKJ1qwU+nDWwqhaAairr7ap6m6/PcJgypYnCudcpHKGsw4bBtOnRzCcNRFLl8KVV8Jbb1lT04cfQosWUUdVZjxROOfKXEHVWR9/PLfCRVL5+Wf4+GN46CG49tqkL+JX2jLr0zrnIpPfcNZmzSIazpqIBQtg1ChLDPvvD4sXJ8npTdnzROGcC0284axduthw1qSzebOd2vTuDRUqWBbbeeeMTRLgicI5V8pyhrMOG2bz8iTNcNZEzJplgX75pfWeP/10EraDlT1PFM65EitoOOvpp9uIpUiHsyZq/Xo47DDLYsOG2ZlE0ma0spXsvzrnXBLLO5y1UiX7Ip4Uw1kTNXs27LOPFfEbPtyK+O24Y9RRJRUvCuicK5JFi2wGz5YtrTP6vvtgzz3hhRfgl1/s6ukzz0yBJLF+vXWW7LsvDBlizx19tCeJfPgZhXOuUCk3nLUw48fDpZfC/Plw2WVw8slRR5TUPFE45/L1xx82nDUrK0WGsybq9tvhzjvtNOjjj+GII6KOKOl5onDO/SNnOOuwYfDOOykynDVROUX82raF//7XkkXVqlFHlRJCTRQi0gHoB5QHBqnq/Xle3xYYAtQPYnlYVV8IMybn3NYKGs56ySXWKZ3Uw1kTsXw5XHONzTp3++0ZUcSvtIWWKESkPNAfOAbIBqaIyChVnR2z2FXAbFU9SUR2BOaKyFBV/TusuJxzWw9nfeUV64ROueGshVG1drMePawd7Y47oo4oZYX5p9AWmK+qCwBEZDhwChCbKBSoISICVAdWAZtCjMm5jDZrlh078w5n7dIFjj8+BUYqJSo7G664wtrPDjwQnnvOOlhcsYSZKOoCS2IeZwMH5lnmSWAUsBSoAZytqlvyrkhEugHdAOrXrx9KsM6lq4Kqs/bpY9VZt9026ghDsHy5TU/6v//ZGUX58lFHlNLCTBT5tWpqnsfHAdOAI4E9gQ9EZKKq/rHVm1QHAgMB2rRpk3cdzrk84g1n7dQJ6tSJNr5QzJ9vBaV69oT99oMlS6BmzaijSgthJopsYLeYx/WwM4dYFwL3q6oC80VkIbA3MDnEuJxLS2k7nLUwmzbBY4/ZKVKlStaOVqeOJ4lSFGaimAI0FpGGwE9AZ6BLnmUWA0cBE0WkDtAEWBBiTM6llbQezpqIGTOsiN+UKXbR3FNPpenpUrRCSxSquklErgbGYsNjn1fVWSJyefD608BdwIsiMgNrqrpJVVeEFZNz6SDth7Mmav16u1iuXDnrhOnUKUM+eNkLdQCcqo4GRud57umY+0uBY8OMwbl0kBHDWRM1c6a1qVWtajujZUvYYYeoo0prmfKn5VxKypjhrIlYt876IR57DAYPhvPPtwzpQueJwrkkk5HDWQvz0UdWxG/hQrjySjjllKgjyiieKJxLAr/+asNZs7IyaDhrovr0gbvvhsaN4ZNPoH37qCPKOJ4onItIxg5nTdSWLXY6dfDBcOON0LdvhrW1JQ9PFM6VoYwfzpqIX3+1q6mbNLH6TB072s1FxhOFcyGLHc76+ut2JpGRw1kLowpDh1ql17VrrQy4SwqeKJwLgSp8+aU1K+UdztqlCxx5ZAYNZ03EkiVw+eUwerRlzkGDoGnTqKNyAf9Tda4U+XDWYlq50nrx+/WDq67yIn5JxhOFcyX04482nDUry4ezFsm8eTBqFFx/PbRqZWcVNWpEHZXLhycK54ohv+Gs7dr5cNaEbNoEjzxis81VqWIXztWp40kiiXmicC5BOcNZhw2DDz/04azF8u23cNFF8PXXdrrVv79n1RTgicK5OAoaznrjjTZiyYezFsH69blFqUaMgDPOiDoilyBPFM7lsWmTDWfNyvr3cNYuXeyKaR/OWgTTp1tGrVrV2utatoTtt486KlcEniicw4ezhmLtWujdG554Al58ES64wMqCu5Tjf/ouo82aZc1Kw4f7cNZS9cEH0K2bDQm7+mrrj3ApyxOFyzj5DWc9+mgfzlpqeveGe++1EhwTJ8Ihh0QdkSuhhBOFiFRT1XVhBuNcWHw4axnIKeJ3yCFw881w221QuXLUUblSUGiiEJGDgUFAdaC+iLQELlPVK8MOzrmSyG84a/Pm9mW3c2do2DDqCNPEzz9b81LTplafyYv4pZ1EzigeBY4DRgGo6rci4gXhXVLy4axlSNVmmrvuOhv6etBBUUfkQpJQ05OqLpGtxwNuDicc54rOh7NGYNEi66x+/31raho0yPokXFpKJFEsCZqfVEQqAj2AOeGG5Vx8Ppw1Yr/9BlOmwJNPwhVXWN+ES1uJ/CtdDvQD6gLZwPuA90+4SPhw1gjNnWtF/G64wS6aW7wYqlePOipXBhJJFE1U9dzYJ0TkP8Bn4YTk3NZ8OGvENm6Ehx+22eaqVYP/+z/YaSdPEhkkkUTxBLB/As85V2pyhrMOGwaTJtlzPpw1At98AxdfbD/PPNOamnbaKeqoXBkrMFGISDvgYGBHEbku5qWagM8q4kqdD2dNMuvXwzHHQIUKNkrg9NOjjshFJN4ZRUXs2oltgNhC8X8AZ4YZlMscGzbY7JdZWbnDWRs08OGskfrmG5tIqGpVq/LasiVst13UUbkIFZgoVPUT4BMReVFVF5VhTC7N5TecdaedfDhr5NassSuq+/e36yMuuAAOPzzqqFwSSKSPYr2IPAQ0A/65Hl9VjwwtKpd2coazDhsGr77qw1mTznvvwWWX2XSk11zjzUxuK4n8aw4FXgFOxIbK/h+wPMygXPrIGc6alQULF/pw1qR0881w//2wzz5WCKtdu6gjckkmkURRW1WfE5FrYpqjPgk7MJe6coazDhsGM2bkDme97TYfzppUNm+G8uWteWmbbeDWWy2TO5dHIoliY/BzmYicACwF6oUXkktFPpw1hSxbBlddZRN+33UXHHec3ZwrQCKJ4m4R2Rb4L3b9RE3g2jCDcqnBh7OmGFWbae6662x4mc8T4RJUaKJQ1XeCu78DR8A/V2a7DOTDWVPUjz/CpZdaRj/0UCvit9deUUflUkS8C+7KA52wGk/vqepMETkRuAWoAuxXNiG6qPlw1jTw++/w9dfw1FM2usmL+LkiiHdG8RywGzAZeFxEFgHtgF6qOjKRlYtIB6ygYHlgkKren88yhwOPARWAFap6WOLhuzB9+SUMHerDWVPW7NlWxK9Xr9wiftWqRR2VS0Hx/tXbAC1UdYuIVAZWAI1U9edEVhyckfQHjsGqzk4RkVGqOjtmmVrAU0AHVV0sIl5EJkk8+SR0726DYE480ZqVfDhrivj7b3jwQeuorlEDLrrITgE9Sbhiipco/lbVLQCqukFE5iWaJAJtgfmqugBARIYDpwCzY5bpAryhqouD7fxapOhdKObMsUrSHTtac5MPZ00hU6daEb/p021EQb9+XsTPlVi8RLG3iEwP7guwZ/BYAFXVFoWsuy6wJOZxNnBgnmX2AiqIyHisnlQ/VX0p74pEpBvQDaB+/fqFbNaVxMaNcP759uXz+ec9SaSUdetsmGvlyvDWW3DyyVFH5NJEvESxTwnXnV/3puaz/dbAUVgH+eci8oWqztvqTaoDgYEAbdq0ybsOV4ruvhu++so6rXfeOepoXEK+/tqK+FWrBm++CS1aQK1aUUfl0kiBQx9UdVG8WwLrzsY6w3PUwy7Wy7vMe6q6TlVXABOAlkX9EK50TJ4M99xjteC81E8K+OMPuPJKaN0ahgyx59q39yThSl2YY+SmAI1FpGEw13ZnYFSeZd4CDhWRbUSkKtY05fNxR2D9emty2nVXu5raJbnRo+3K6meesQvozjgj6ohcGgttgKOqbhKRq4Gx2PDY51V1lohcHrz+tKrOEZH3gOnAFmwI7cywYnIFu+kmmDcPPvrI+yWS3k032aimpk1tvogD83b9OVe6EkoUIlIFqK+qc4uyclUdDYzO89zTeR4/BDxUlPW60vXBBzYc9ppr7PoIl4RUYcsWK+J31FHWYX3LLV7Ez5WJQpueROQkYBrwXvC4lYjkbUJyKWr1arjwQqswfd99UUfj8vXTT3DqqXD77fb42GPhjjs8Sbgyk0gfRV/smojfAFR1GtAgrIBc2br6arvq+uWX/WK6pKMKzz5rTUzvvw877BB1RC5DJdL0tElVfxcv5pN2Xn3VKr/ecYcNnHFJZOFCu3Bu3DibL+LZZ6FRo6ijchkqkUQxU0S6AOVFpDHQA5gUblgubEuXwhVXQNu21tTtkszatXZ19TPPWPVFL+LnIpTIX193bL7sv4BhWLnxa0OMyYVM1Y49f/4JL73kxf2SxsyZNpkHWL32xYuhWzdPEi5yiRwimqhqb6B32MG4sjFwIIwZA088AU2aRB2N4++/bSTBPffY2ORLLrH6TFWrRh2Zc0BiZxT/E5HvROQuEWkWekQuVPPn2/VZxxxjF/W6iE2ZYh1EffvCWWdZaXAv4ueSTCIz3B0hIjtjkxgNFJGawCuqenfo0blStWmTleeoWNEK/nmLRsTWrYMOHWy42ahRcNJJUUfkXL4SOlSo6s+q+jhwOXZNxW1hBuXC8dBD8Pnn0L8/1KsXdTQZbOpUu3iuWjWr8jprlicJl9QSueBuHxHpKyIzgSexEU9+mEkx06bZ9VqdOtkkRC4Cv/9u05AecEBuEb9DDvGaKS7pJdKZ/QKQBRyrqnmrv7oUsGEDnHeeXa/11FM+v3Uk3n4bLr8cfv4Zrr8ezjwz6oicS1gifRQHlUUgLjx9+ljrxujRULt21NFkoBtugIcftiGvI0faGYVzKaTARCEir6pqJxGZwdYTDiU6w51LAp98Ao88Yl9mO3aMOpoMogqbN9tFKsceCzVrWtXXihWjjsy5IhPV/CeME5FdVHWZiOye3+sJTl5U6tq0aaNTp06NYtMp548/oGVLKzg6bRpUrx51RBkiO9sue2/Rwq6NcC4JiMhXqtqmOO+NN8PdsuDulfnMbucj8FNAz552ce/LL3uSKBNbtljJjaZN4eOPfS5ZlzYSGR57TD7PeSNGknvrLbtWolcvaNcu6mgywIIFNpnH5ZdbAa0ZM6B796ijcq5UxOujuAI7c9hDRKbHvFQD+CzswFzx/forXHoptGqVO4WBC9m6dXZV9aBBcNFFPrTMpZV4o56GAWOA+4BeMc+vUdVVoUblik3Vhur//ru1fnjfaYhmzLBTt1tvtRFNixb5pB4uLcVrelJV/RG4ClgTc0NEtg8/NFccgwfbCMx77oHmzaOOJk399Rfcdhvsvz88/ridwoEnCZe2CjujOBH4ChseG3surcAeIcblimHRIujRA9q3t45sF4IvvrAJhWbPhvPPh0cf9YtTXNorMFGo6onBz4ZlF44rri1boGtXa3oaPNiGxLpStm4dnHCC1WgaPdovTHEZI5FaT/8RkWrB/fNE5H8iUj/80FxR9OsH48fbzwYNoo4mzXz5ZW4Rv7fftsvcPUm4DJLI8NgBwHoRaQncCCwCXg41Klcks2bBzTfDySfDhRdGHU0a+e03m0TooINyi/gdfDDUqBFpWM6VtUQSxSa1y7dPAfqpaj9siKxLAn//bU3lNWvCs8/6qMxSM3KkXTj34otWeuOss6KOyLnIJFI9do2I3AycDxwqIuWBCuGG5RJ1113wzTfw5ps+MVqpue4666Ru2dKamlq3jjoi5yKVSKI4G+gCXKSqPwf9Ew+FG5ZLxBdfwL33Wif2qadGHU2Kiy3id/zxNpLpxhuhgn8ncq7AooBbLSRSB8ipjTxZVX8NNao4vCigWbfOrrz++2+YPt3nvimRxYut9MZ++3kRP5e2QikKGLPyTsBk4Cxs3uwvRcRnXYnYjTfCDz/YUFhPEsW0ZYvN5NSsmdVj33XXqCNyLikl0vTUGzgg5yxCRHYEPgRGhBmYK9jYsXZ8u+46OPzwqKNJUfPnW02miRPhmGNg4EAfV+xcARJJFOXyNDWtJLHRUi4Eq1bZ8a1pU28lKZENG2DePHjhBfi///PhYs7FkUiieE9ExmLzZoN1bo8OLyQXz1VXWWmhd96BypWjjibFTJtmRfxuv90KYf34o+9E5xJQ6JmBqt4APAO0AFoCA1X1prADc/82fLjd+va1fleXoA0boHdvaNMGBgzILeLnScK5hMSbj6Ix8DCwJzADuF5VfyqrwNzWfvoJrrzSLhK+ydN04iZNsiJ+331nTUz/+x9s78WPnSuKeGcUzwPvAGdgFWSfKJOI3L+o2rHur7/gpZdsqL9LwLp1cNJJsH49vPeeXWXtScK5Iot3yKmhqs8G9+eKyNdlEZD7twEDbKRT//7QuHHU0aSAzz+HAw+0In7vvGP9EV6fyblii3dGUVlE9hOR/UVkf6BKnseFEpEOIjJXROaLSK84yx0gIpv9+ox/+/57uP56OO44uOKKqKNJcqtX25Cwgw+Gl4O6le3aeZJwroTinVEsA/4X8/jnmMcKHBlvxUFNqP7AMUA2MEVERqnq7HyWewAYW7TQ09+mTVbwr3JleO45H8EZ1xtv2JCw5cutlO7ZZ0cdkXNpI97ERUeUcN1tgfmqugBARIZjFWhn51muO/A6uSVCXOCBB2wqhKwsqFs36miSWM+e8NhjVtNk9GgfEuZcKQuzW7QusCTmcTZwYOwCIlIXOA07OykwUYhIN6AbQP36mTFn0tdf2zDYzp3t5vKILeJ34olWOvf6672In3MhCPMK6/waSvJWIHwMuElVN8dbkaoOVNU2qtpmxx13LK34ktaGDdbktNNO1oHt8vjxR+jQAfr0scdHHWXNTZ4knAtFmGcU2cBuMY/rAUvzLNMGGC7W+L4DcLyIbFLVkSHGlfR694bZs21Ep4/mjLFli2XOm2+2DpvTTos6IucyQqGJQuwofi6wh6reGcxHsbOqTi7krVOAxiLSEPgJ6IzNa/EPVW0Ys50XgXcyPUmMH29z5lx5pY10coHvv7d5Xj/7zM4mnn4adt896qicywiJND09BbQDzgker8FGM8WlqpuAq7HRTHOAV1V1lohcLiKXFzPetPb773bxcKNG8OCDUUeTZP7+2+qqv/SSdVh7knCuzCTS9HSgqu4vIt8AqOpqEamYyMpVdTR5Cgiq6tMFLNs1kXWms2uvhexs+9JcrVrU0SSBb76xIn59+9qcET/+CJUqRR2VcxknkTOKjcG1Dgr/zEexJdSoMtDIkVZh4pZbrJ5TRtuwwfohDjgAnnnGro0ATxLORSSRRPE48Cawk4jcA3wK3BtqVBnm11+hWzcb/p8zkCdjffoptGwJ998PF1xgvfoZMNLNuWRWaNOTqg4Vka+Ao7Ahr6eq6pzQI8sQqnDppfDHHzBkCFRMqFEvTa1dC6ecAjVrwvvv28xzzrnIJTLqqT6wHng79jlVXRxmYJnihRdg1Cirft20adTRROTTT60+U/Xq8O67VsSvevWoo3LOBRJpenoXKzf+LvARsAAYE2ZQmWLhQrjmGpv3+pproo4mAitXWvPSoYfmFvE76CBPEs4lmUSanvaNfRxUjr0stIgyxObN0LWrXTf24otQLpNmIVeFESPg6qttEvA+fbxOiXNJrMhXZqvq1yLiBfxK6NFHYcIEa3rKuEsCevaEfv2gdWvri2jZMuqInHNxJNJHcV3Mw3LA/sDy0CLKADNnWpmOU0+1C+wygqrVTa9QAU4+GXbdFa67zqfrcy4FJNLgUSPmVgnrqzglzKDS2d9/w3nnQa1aMHBghswxsXAhHHts7tjfI4+EG2/0JOFcioj7nxpcaFddVW8oo3jS3h13wLff2gXHaX95wObN8OSTdhVh+fJw1llRR+ScK4YCE4WIbKOqmxKd9tQVbtIku47soous9SWtzZtnvfWffw4dO9oV1rvtVujbnHPJJ94ZxWSsP2KaiIwCXgPW5byoqm+EHFtaWbvWRoLWr28d2Wlv0yZYtMiuIuzSJUPa2JxLT4k0Em8PrMRmoVPs6mwFPFEUwQ03wIIFVka8Zs2oownJ1KnWpnbXXXb14IIFXp/JuTQQL1HsFIx4mklugsiRd6Y6F8eYMTZ9wvXXQ/v2UUcTgj//hNtvh0cegZ13hh49rAPGk4RzaSHeqKfyQPXgViPmfs7NJWDlSuuTaN7cvminnU8+gRYt4KGH4OKLYdasDOildy6zxDujWKaqd5ZZJGlIFa64wpLFmDFQuXLUEZWytWvh9NNtrO9HH9mwV+dc2omXKLz3sYSysuC11+Dee6FVq6ijKUUTJ8J//mM1mcaMsUmFfKYl59JWvKano8osijSUnQ1XXQXt2llHdlpYscKuFmzfPreIX9u2niScS3MFnlGo6qqyDCSdbNli/RIbN9oUzyl/AbIqvPoqdO8Oq1dbx7UX8XMuY6T6ISwpPfUUfPCBjXRq1CjqaErBNdfAE0/Y1KQffQT77lv4e5xzacMTRSmbO9fKGHXsaNObpixVOyWqWBFOO81K3F57rZXicM5llEyaBSF0mzbZ1ddVqsBzz6Xwxcg//ABHHQW33mqPjzgC/vtfTxLOZShPFKXo3nth8mRrctpll6ijKYbNm21O1n33ha++giZNoo7IOZcEvOmplEydahfUdemSokVSv/vOJseYPBlOOgkGDIC6daOOyjmXBDxRlII//4Tzz4c6dayqdkrasgWWLrWLP84+O4XbzZxzpc0TRSm45Rb7Qv7++7DddlFHUwSTJ1sRv3vusSJ+P/xgndfOORfD+yhK6OOP4bHH4Oqr4Zhjoo4mQevXW4XCdu1g8GBYHsxs60nCOZcPTxQl8NtvNjfPXnvBAw9EHU2Cxo2zzupHHoFLL/Uifs65QnnTUwlcc40160+aBFWrRh1NAtautZ72WrUsYRx+eNQROedSgJ9RFNMbb1h5jt69rdxRUhs/3jqrc4r4TZ/uScI5lzBPFMXw88921XXr1rnXpCWl5cvhnHPsgrkhQ+y5Aw5IkdMf51yy8KanIlK1pv1166yAaoUKUUeUD1Ub5tqjB6xZYxd4eBE/51wxeaIooueeg3fesZFO++wTdTQF6N4d+veHgw6ygJs2jToi51wK80RRBAsWQM+eNpFb9+5RR5PHli1WbKpiRTjzTCtb272712dyzpVYqH0UItJBROaKyHwR6ZXP6+eKyPTgNklEWoYZT0ls3mwVLsqVgxdesJ9J4/vvLXv17m2PDz/cK70650pNaIc7ESkP9Ac6Ak2Bc0QkbxvIQuAwVW0B3AUMDCueknrkEfj0UyvRUb9+1NEENm2Chx+GFi1g2rQkbgtzzqWyMJue2gLzVXUBgIgMB04BZucsoKqTYpb/AqgXYjzFNn069OkDp59uM4EmhTlzrKb51Klwyik2W9Kuu0YdlXMuDYXZgFIXWBLzODt4riAXA2Pye0FEuonIVBGZujyn3EQZ+esvK/i33XbwzDNJVivvl1/glVfgzTc9STjnQhPmGUV+h1TNd0GRI7BEcUh+r6vqQIJmqTZt2uS7jrDcfrudUbz9NuywQ1luOR9ffGFF/O67z5qZfvghScfnOufSSZhnFNnAbjGP6wFL8y4kIi2AQcApqroyxHiK7NNP4cEH4ZJL4MQTIwxk3TobbnXwwTB0aG4RP08SzrkyEGaimAI0FpGGIlIR6AyMil1AROoDbwDnq+q8EGMpsjVrbJRTgwY26VtkPvwQmje3CzeuvNKL+DnnylxoTU+quklErgbGAuWB51V1lohcHrz+NHAbUBt4Sqzxf5OqtgkrpqK4/npYuBA++QRq1IgoiLVr7Yrq7beHCRPg0EMjCsQ5l8lEtUyb/EusTZs2OnXq1FC38e671tR0440RlQ//+GM47DC7DuKrr+zK6ipVIgjEOZcuROSr4n4RT6bLxpLCihVw8cU2ZcOdd5bxxn/5BTp1gqOOyi3i17q1JwnnXKS8hEcMVbj8cli1CsaOhUqVynDDQ4bY1dRr19rUpF26lNHGnXMuPk8UMYYOhddfh/vvh5ZlWUzkqqtgwACbmvS55/wKa+dcUvFEEViyxOa9/s9/rCM7dFu2wMaNdtpy9tmWHK680uszOeeSjvdRYMfsCy+00kmDB5fBsXruXOuszinid9hhXunVOZe0PFFghf4++ggefRT23DPEDW3cmNuuNXOm9Zg751ySy/imp+++g5tughNOsCuwQzNrlhWN+uYbqy7Yvz/svHOIG3TOudKR0Yli40Y7dlerBoMGhVzwr3x5G041YgSccUaIG3LOudKV0YninnusSveIESF9uZ80yYr4PfAA7L03zJ8P22T0LnfOpaCM7aOYMgXuvtvmlyj1L/hr10KPHnDIIVYGfMUKe96ThHMuBWVkoli/3pqcdtkFnniilFf+/vtWxO/JJ2287cyZSVCf3Dnnii8jv+L26mUjVD/8EGrVKsUVr10L554LtWvDxIl2UYZzzqW4jDuj+PBDO4vo0cNKKpWKDz6AzZuhenU7o5g2zZOEcy5tZFSi+O03u7Bu773tcoYSW7bMOjiOPdbqfwDstx9UrlwKK3fOueSQUU1P3bvbsf3zz0tYkFXVLuHu2RP+/NOyjhfxc86lqYxJFCNGWIHWvn3hgANKuLIrroBnnrFRTYMGQZMmpRGic2ln48aNZGdns2HDhqhDyRiVK1emXr16VCjFqZIzIlEsWwaXXWYJ4pZbirmS2CJ+XbpAixZWk7xcRrXeOVck2dnZ1KhRgwYNGiChXtHqAFSVlStXkp2dTcOGDUttvWl/lFO10hzr18NLL0GxkuycOTYNaU6Wad/eKr16knAurg0bNlC7dm1PEmVERKhdu3apn8Gl/ZHu2Wdh9Gh48EHrxC6SjRvh3nuhVSsrCrXffmGE6Fxa8yRRtsLY32nd9PTDD3DddXD00TY3UJHMmmWXbU+bBmedZWNq69QJI0znnEtqaXtGsXkzXHCBVc14/vlitBJtsw38/ju88Qa8+qonCedS2JtvvomI8N133/3z3Pjx4znxxBO3Wq5r166MGDECsI74Xr160bhxY5o3b07btm0ZM2ZMiWO57777aNSoEU2aNGHs2LEFLvfEE0/QpEkTmjVrxo033gjA0KFDadWq1T+3cuXKMW3atBLHVJi0PaN46CGryTdkCOy2W4JvmjjRivg9/LCNZJo3z+szOZcGsrKyOOSQQxg+fDh9+/ZN6D19+vRh2bJlzJw5k0qVKvHLL7/wySeflCiO2bNnM3z4cGbNmsXSpUs5+uijmTdvHuXzTFo2btw43nrrLaZPn06lSpX49ddfATj33HM599xzAZgxYwannHIKrVq1KlFMiUjLo+C338Jtt8GZZyZ4ecOaNVbX46mnoGFDu7/DDp4knCtF115rLbmlqVUreOyx+MusXbuWzz77jHHjxnHyyScnlCjWr1/Ps88+y8KFC6lUqRIAderUoVOnTiWK96233qJz585UqlSJhg0b0qhRIyZPnky7du22Wm7AgAH06tXrn23vtNNO/1pXVlYW55xzToniSVTaNT399ZcV/KtdG55+OoE5JsaMgWbNYMAA+0ueMcOL+DmXRkaOHEmHDh3Ya6+92H777fn6668Lfc/8+fOpX78+NWvWLHTZnj17btUclHO7P5/yDz/99BO7xTRx1KtXj59++ulfy82bN4+JEydy4IEHcthhhzFlypR/LfPKK6+UWaJIu6/MffrYsf7ddy1ZxLVmjXVk7LSTtVMddFCZxOhcJirsm39YsrKyuPbaawHo3LkzWVlZ7L///gWODirqqKFHH3004WVVNaHtbdq0idWrV/PFF18wZcoUOnXqxIIFC/5Z9ssvv6Rq1ao0b968SLEWV1oliokTrXuhWzc4/vgCFlKFsWPhmGOgRg2rErj33nYhnXMuraxcuZKPP/6YmTNnIiJs3rwZEeHBBx+kdu3arF69eqvlV61axQ477ECjRo1YvHgxa9asoUaNGnG30bNnT8aNG/ev5zt37kyvXr22eq5evXosWbLkn8fZ2dnsuuuu/3pvvXr1OP300xER2rZtS7ly5VixYgU77rgjAMOHDy+zswnAMlwq3Vq3bq35+eMP1YYNVffcU3XNmnwXUV26VPXUU1VBdfDgAhZyzpWW2bNnR7r9p59+Wrt167bVc+3bt9cJEybohg0btEGDBv/E+OOPP2r9+vX1t99+U1XVG264Qbt27ap//fWXqqouXbpUX3755RLFM3PmTG3RooVu2LBBFyxYoA0bNtRNmzb9a7kBAwZonz59VFV17ty5Wq9ePd2yZYuqqm7evFnr1q2rP/zwQ4HbyW+/A1O1mMfdtOmj6NkTFi2yWn3Vq+d5UdXGyO6zD7z3nl1950X8nEt7WVlZnHbaaVs9d8YZZzBs2DAqVarEkCFDuPDCC2nVqhVnnnkmgwYNYttttwXg7rvvZscdd6Rp06Y0b96cU0899Z9v9MXVrFkzOnXqRNOmTenQoQP9+/f/Z8TTJZdcwtSpUwG46KKLWLBgAc2bN6dz584MHjz4n2anCRMmUK9ePfbYY48SxVIUovm0mSWzNm3aaM7OzPH223DyyTZY6b778nnTZZfBwIFWemPQIGjcuGyCdS7DzZkzh3322SfqMDJOfvtdRL5S1TbFWV/K91EsX261nFq2hDvuiHlh82YrwVG5sl1hvd9+1nnh9Zmcc65IUvqoqWonC7/9Bi+/DBUrBi/MmmUzzOUU8Tv0UK/06pxzxZTSR86XX4Y334S774Z99wX+/hvuusvOHubPL4WJJ5xzJZVqzdupLoz9nbJNT4sX24x1hx5qhf+YMQPOPdd+du4Mjz8OJex4cs6VTOXKlVm5cqWXGi8jGsxHUbmUp2NOyUSxZQt07Wo/Bw+G8uWxdqf1661W08knRx2icw67HiA7O5vly5dHHUrGyJnhrjSlZKJ4/HEYNw7eueETGj45Ch55xIr4zZ0bZA3nXDKoUKFCqc605qIRah+FiHQQkbkiMl9EeuXzuojI48Hr00Vk/8LWuWED3HPTH4ze/QpOeOhwGDkSVqywFz1JOOdcqQstUYhIeaA/0BFoCpwjIk3zLNYRaBzcugEDClvv8vm/M21TMzosGWidE17EzznnQhXmGUVbYL6qLlDVv4HhwCl5ljkFeCm4wvwLoJaI7BJvpbv89SM16m6LTJpkTU5Vq4YTvXPOOSDcPoq6wJKYx9nAgQksUxdYFruQiHTDzjgA/tp2yayZXukVgB2AFVEHkSR8X+TyfZHL90WuJsV9Y5iJIr+xcHkH+CayDKo6EBgIICJTi3sZerrxfZHL90Uu3xe5fF/kEpGphS+VvzCbnrKB2ElI6wFLi7GMc865CIWZKKYAjUWkoYhUBDoDo/IsMwq4IBj9dBDwu6ouy7si55xz0Qmt6UlVN4nI1cBYoDzwvKrOEpHLg9efBkYDxwPzgfXAhQmsemBIIaci3xe5fF/k8n2Ry/dFrmLvi5QrM+6cc65spXRRQOecc+HzROGccy6upE0UYZT/SFUJ7Itzg30wXUQmiUjLKOIsC4Xti5jlDhCRzSJyZlnGV5YS2RcicriITBORWSLySVnHWFYS+B/ZVkTeFpFvg32RSH9oyhGR50XkVxGZWcDrxTtuFney7TBvWOf3D8AeQEXgW6BpnmWOB8Zg12IcBHwZddwR7ouDge2C+x0zeV/ELPcxNljizKjjjvDvohYwG6gfPN4p6rgj3Be3AA8E93cEVgEVo449hH3RHtgfmFnA68U6bibrGUUo5T9SVKH7QlUnqerq4OEX2PUo6SiRvwuA7sDrwK9lGVwZS2RfdAHeUNXFAKqarvsjkX2hQA2xSTGqY4liU9mGGT5VnYB9toIU67iZrImioNIeRV0mHRT1c16MfWNIR4XuCxGpC5wGPF2GcUUhkb+LvYDtRGS8iHwlIheUWXRlK5F98SSwD3ZB7wzgGlXdUjbhJZViHTeTdT6KUiv/kQYS/pwicgSWKA4JNaLoJLIvHgNuUtXNaT6jWiL7YhugNXAUUAX4XES+UNV5YQdXxhLZF8cB04AjgT2BD0Rkoqr+EXJsyaZYx81kTRRe/iNXQp9TRFoAg4COqrqyjGIra4nsizbA8CBJ7AAcLyKbVHVkmURYdhL9H1mhquuAdSIyAWgJpFuiSGRfXAjcr9ZQP19EFgJ7A5PLJsSkUazjZrI2PXn5j1yF7gsRqQ+8AZyfht8WYxW6L1S1oao2UNUGwAjgyjRMEpDY/8hbwKEiso2IVMWqN88p4zjLQiL7YjF2ZoWI1MEqqS4o0yiTQ7GOm0l5RqHhlf9IOQnui9uA2sBTwTfpTZqGFTMT3BcZIZF9oapzROQ9YDqwBRikqvkOm0xlCf5d3AW8KCIzsOaXm1Q17cqPi0gWcDiwg4hkA7cDFaBkx00v4eGccy6uZG16cs45lyQ8UTjnnIvLE4Vzzrm4PFE455yLyxOFc865uDxRuKQUVH6dFnNrEGfZtaWwvRdFZGGwra9FpF0x1jFIRJoG92/J89qkksYYrCdnv8wMqqHWKmT5ViJyfGls22UuHx7rkpKIrFXV6qW9bJx1vAi8o6ojRORY4GFVbVGC9ZU4psLWKyKDgXmqek+c5bsCbVT16tKOxWUOP6NwKUFEqovIR8G3/Rki8q+qsSKyi4hMiPnGfWjw/LEi8nnw3tdEpLAD+ASgUfDe64J1zRSRa4PnqonIu8HcBjNF5Ozg+fEi0kZE7geqBHEMDV5bG/x8JfYbfnAmc4aIlBeRh0Rkitg8AZclsFs+JyjoJiJtxeYi+Sb42SS4SvlO4OwglrOD2J8PtvNNfvvRuX+Jun663/yW3w3YjBVxmwa8iVURqBm8tgN2ZWnOGfHa4Od/gd7B/fJAjWDZCUC14PmbgNvy2d6LBHNXAGcBX2IF9WYA1bDS1LOA/YAzgGdj3rtt8HM89u39n5hilsmJ8TRgcHC/IlbJswrQDbg1eL4SMBVomE+ca2M+32tAh+BxTWCb4P7RwOvB/a7AkzHvvxc4L7hfC6v7VC3q37ffkvuWlCU8nAP+VNVWOQ9EpAJwr4i0x8pR1AXqAD/HvGcK8Hyw7EhVnSYihwFNgc+C8iYVsW/i+XlIRG4FlmNVeI8C3lQrqoeIvAEcCrwHPCwiD2DNVROL8LnGAI+LSCWgAzBBVf8MmrtaSO6MfNsCjYGFed5fRUSmAQ2Ar4APYpYfLCKNsWqgFQrY/rHAySJyffC4MlCf9KwB5UqJJwqXKs7FZiZrraobReRH7CD3D1WdECSSE4CXReQhYDXwgaqek8A2blDVETkPROTo/BZS1Xki0hqrmXOfiLyvqncm8iFUdYOIjMfKXp8NZOVsDuiuqmMLWcWfqtpKRLYF3gGuAh7HahmNU9XTgo7/8QW8X4AzVHVuIvE6B95H4VLHtsCvQZI4Atg97wIisnuwzLPAc9iUkF8A/xGRnD6HqiKyV4LbnACcGrynGtZsNFFEdgXWq+oQ4OFgO3ltDM5s8jMcK8Z2KFbIjuDnFTnvEZG9gm3mS1V/B3oA1wfv2Rb4KXi5a8yia7AmuBxjge4SnF6JyH4FbcO5HJ4oXKoYCrQRkanY2cV3+SxzODBNRL7B+hH6qepy7MCZJSLTscSxdyIbVNWvsb6LyVifxSBV/QbYF5gcNAH1Bu7O5+0Dgek5ndl5vI/Nbfyh2tSdYHOJzAa+FpGZwDMUcsYfxPItVlb7Qezs5jOs/yLHOKBpTmc2duZRIYhtZvDYubh8eKxzzrm4/IzCOedcXJ4onHPOxeWJwjnnXFyeKJxzzsXlicI551xcniicc87F5YnCOedcXP8PjSn6oGp6JU0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.66      0.73      0.69      5257\n",
      "           1       0.68      0.61      0.64      5055\n",
      "\n",
      "    accuracy                           0.67     10312\n",
      "   macro avg       0.67      0.67      0.67     10312\n",
      "weighted avg       0.67      0.67      0.67     10312\n",
      "\n",
      "\n",
      "Confusion Matrix: \n",
      " [[3825 1432]\n",
      " [1982 3073]]\n",
      "\n",
      "End of validation Result\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/bettyau/opt/anaconda3/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function plot_confusion_matrix is deprecated; Function `plot_confusion_matrix` is deprecated in 1.0 and will be removed in 1.2. Use one of the class methods: ConfusionMatrixDisplay.from_predictions or ConfusionMatrixDisplay.from_estimator.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAT8AAAEGCAYAAAAT05LOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAb60lEQVR4nO3debRV5Z3m8e9zByYVZZbJgliURq1SIyFapiw06UCSSjDVsQsz6Oq2FtFop6qTrixNupNWQzpd6VRSmmiFGEtSKaVwZYAMTk00mloaxFk0CHFA4CpeFEVRuMOv/9jvxQPee+7ZcM895579fNba6+79nj28hws/3mG/76uIwMysaJpqnQEzs1pw8DOzQnLwM7NCcvAzs0Jy8DOzQmqpdQZKjR/bHDOmt9Y6G5bDEw+PqnUWLIc3eI3dsUsHco95px8U217squjc+x7edUtEzD+Q51VLXQW/GdNbWX3L9Fpnw3KYN+WEWmfBcvhtrDrge7S/2MVvb5lW0bmtk38//oAfWCV1FfzMbCgIuqK71pk4YA5+ZpZLAN0M/cERDn5mlls3LvmZWcEEQYervWZWNAF0udprZkXkNj8zK5wAuhpgNigHPzPLbei3+Dn4mVlOQbjNz8yKJwI6hn7sc/Azs7xEFwc0PLguOPiZWS4BdLvkZ2ZF5JKfmRVO9pKzg5+ZFUwAHTH050F28DOzXALR1QCTwDv4mVlu3eFqr5kVjNv8zKygRJfb/MysaLKZnB38zKxgIsTuaK51Ng6Yg5+Z5dbtNj8zK5qsw8PVXjMrHHd4mFkBNUqHx9D/BmY26LpCFW39kfS0pEckPShpTUobK+k2SevTzzEl518iaYOkdZLmlaSflO6zQdIVkvp9uIOfmeUSiI5oqWir0OkRcUJEzE7HFwOrImIWsCodI+kYYCFwLDAfuEpST7fz1cAiYFba5vf3UAc/M8ulp8Ojkm0/LQCWpv2lwJkl6csiYldEPAVsAOZImgyMjoi7IyKAH5Rc0ycHPzPLJaisyltJtZcslt4q6T5Ji1LapIhoA0g/J6b0qcCzJdduSmlT0/6+6WW5w8PMcsvR4TG+py0vWRIRS0qOT42ILZImArdJ+l2Ze/UWTaNMelkOfmaWSwR5XnVpL2nL6+VesSX93CrpJ8Ac4HlJkyOiLVVpt6bTNwHTSy6fBmxJ6dN6SS/L1V4zyyXr8GiuaCtH0kGSDunZB94HPAqsBM5Np50LrEj7K4GFkoZLmknWsbE6VY13SDo59fKeU3JNn1zyM7PcBmiExyTgJ+mtlBbg+oi4WdK9wHJJ5wEbgbMAImKtpOXAY0AncGFEdKV7XQBcB4wEbkpbWQ5+ZpZLoAGZzDQingSO7yV9G/CePq5ZDCzuJX0NcFye5zv4mVluHttrZoWTrdvr4GdmhSNPY29mxZMtXenJTM2sYCLkaq+ZFZPn8zOzwsnm83Obn5kVjmdyNrMCyl51ccnPzAqmZ2zvUOfgZ2a5NcIaHg5+ZpZLNqWVq71mVkBu8zOzwslmdXG118wKJhve5uBXWOfMOYaRB3fR1ATNLcG3b36C3z86kisunsbuN5pobgku+t+bOPrEnQAsu3IiN98wjuam4IKvbGb23B3sfLWJz505a88929taOeM/vsQFl22u1dcqvNlzX+H8y7fQ3BTcdMNYln97Uq2zVIdc8uuXpPnAPwLNwDUR8bVqPm+w/f2NGzh0XNee42u+MplPfPY53nnGDlavOoTvf2UKX//RBp55Yjh3rBjDktt/x4vPt3LxXx3J93/zOKMO7ubq/7duz/UXzvsj3v2B7TX4JgbQ1BRc+NXNXLLwbbS3tXLlL9dzzy2HsnH9iFpnre40wgiPqoXvtJjwd4D3A8cAZ6dFhxuWBK/tyN5/eu2VZsZO6gDg7lsOZe6Clxg2PDj8iN1MmbGLdQ+M2uvazU8OY3t7C8e967VBz7dljjpxJ1ueHsZzG4fT2dHEHSsO45R5L9c6W3Wnp7d3gJaurJlqlvzmABvSVNVIWka26PBjVXzm4FHwhbOPBMEHP7mND3xiG+dftpkvnH0k37tsChHwzZXrgaw6+/aTdu65dPzkDrY917rX7W7/6Rj+/MPbUX3/fWlo4w7v4IUtw/Yct7e1cvQ7dpa5orhc7S2vtwWG37XvSWmh4kUAR0wdOk2Q31yxnnGHd7K9vYWLFx7J9D98g7t+fhifunQzf/bBl/n1ysP4h88ewf9Z/vveVxDdJ8j9esUYPn/lM4OSd+tdb//xRL+rvxbPQK3hUWvVDN8VLSQcEUsiYnZEzJ4wbugMmRl3eCcAh43v5NT5L/O7B0Zx241jefcHsmrSaR/azhMPZlXb8VM6eGHLmyW99rZWxqUqMcDv146gqwtm/cnrg/gNbF/tba1MmLJ7z3FvJXTL/hF3RlNFWz2rZu76WmB4yHtjZxM7X23as3/frw9hxtFvMG5SBw/ffTAAD/7mYKbM3AXAye97hTtWjGH3LvHcxmFsfmo4R534ZnXqjp+OYe6C7YP+PWxv6x4cxdSZu5k0fRctrd3MXbCde249tNbZqkvd0VTRVs+qWc+8F5iVFhfeDCwEPlbF5w2al15o4dLzZgLQ1Qmnf2Q77zx9ByNHPcvVX5pKV5cYNrybv/16VuufcdQbnPah7SyaezTNzcFFX91Ec0kh986fHcbl//JkLb6KlejuEt/54lS+ev2TNDXDrcvG8swT7ul9i2iMaq+iio0akj4AfIvsVZdr05qbfZp9/IhYfcv0cqdYnZk35YRaZ8Fy+G2s4pV48YAi15ijJ8YZ1360onN/fOrV90XE7AN5XrVUtYchIn4J/LKazzCzwdcIJb+h071qZnXBk5maWSEForO7vjszKuHgZ2a5NcLwNgc/M8snGqPaO/TLrmY2qHra/CrZKiGpWdIDkn6ejsdKuk3S+vRzTMm5l0jaIGmdpHkl6SdJeiR9doXU/0BRBz8zy20ggx/wN8DjJccXA6siYhawKh2TJkZZCBwLzAeuShOoAFxNNkx2Vtrm9/dQBz8zyyUQXd1NFW39kTQN+CBwTUnyAmBp2l8KnFmSviwidkXEU8AGYI6kycDoiLg7sheXf1ByTZ/c5mdmueXo8BgvaU3J8ZKIWFJy/C3g88AhJWmTIqINICLaJE1M6VOBe0rO25TSOtL+vullOfiZWS6Rr8Ojva8RHpL+AtgaEfdJmlvBvfqaLKWiSVT25eBnZrnFwPT2ngp8OA2DHQGMlvRD4HlJk1OpbzKwNZ3f12Qpm9L+vulluc3PzHKqrLOjv9JhRFwSEdMiYgZZR8avIuITwErg3HTaucCKtL8SWChpeJowZRawOlWRd0g6OfXynlNyTZ9c8jOz3Aao5NeXrwHLJZ0HbATOyp4ZayUtJ5sNvhO4MCJ6FtG5ALgOGAnclLayHPzMLJcI6Ooe2OAXEXcAd6T9bcB7+jhvMfCW2aEiYg1wXJ5nOviZWW4e3mZmhRNUvdo7KBz8zCynxpjJ2cHPzHJrhFXtHPzMLDdXe82scLLe3qH/irCDn5nl5mqvmRWSq71mVjiBHPzMrJgaoNbr4GdmOQXEAA9vqwUHPzPLzdVeMyukhu7tlXQlZar2EfGZquTIzOpaEcb2rinzmZkVVQCNHPwiYmnpsaSDIuK16mfJzOpdI1R7+x2jIukUSY+R1tWUdLykq6qeMzOrUyK6K9vqWSUD9L4FzAO2AUTEQ8BpVcyTmdW7qHCrYxX19kbEs9m6IHt09XWumTW4aPwOjx7PSvpTICQNAz5DqgKbWUHVeamuEpVUe88HLiRbAX0zcEI6NrPCUoVb/eq35BcR7cDHByEvZjZUdNc6Aweukt7et0n6maQXJG2VtELS2wYjc2ZWh3re86tkq2OVVHuvB5YDk4EpwI3ADdXMlJnVt4jKtnpWSfBTRPxLRHSm7Yc0RHOnme23Rn7VRdLYtHu7pIuBZWRf56+AXwxC3sysXtV5lbYS5To87iMLdj3f8lMlnwVwebUyZWb1TXVeqqtEubG9MwczI2Y2RISgzoeuVaKi9eckHSfpP0k6p2erdsbMrI4NQJufpBGSVkt6SNJaSZem9LGSbpO0Pv0cU3LNJZI2SFonaV5J+kmSHkmfXaF9hqT1ppJXXb4MXJm204G/Bz7c33Vm1sAGpsNjF3BGRBxPNnhivqSTgYuBVRExC1iVjpF0DLAQOBaYD1wlqTnd62pgETArbfP7e3glJb+PAu8BnouI/wwcDwyv4Doza1QDEPwi82o6bE1bAAuAnin1lgJnpv0FwLKI2BURTwEbgDmSJgOjI+LuiAjgByXX9KmS4Pd6RHQDnZJGA1sBv+RsVlT5XnIeL2lNybao9FaSmiU9SBZXbouI3wKTIqINIP2cmE6fCjxbcvmmlDY17e+bXlYlExuskXQY8D2yHuBXgdUVXGdmDSpHb297RMzu68OI6AJOSDHmJ5KOK/fY3m5RJr2sSsb2fjrt/pOkm8mKlw/3d52ZNbABftUlIrZLuoOsre55SZMjoi1Vabem0zYB00sumwZsSenTekkvq89qr6R37LsBY4GWtG9mBaWobCt7D2lCKvEhaSTwXuB3wErg3HTaucCKtL8SWChpuKSZZB0bq1PVeIekk1Mv7zkl1/SpXMnvG2U+C+CM/m6e1xMPj2LelBMG+rZWRUetaa11FiyHhz4xQO/nDcwIj8nA0tRj2wQsj4ifS7obWC7pPGAjcBZARKyVtBx4DOgELkzVZoALgOuAkcBNaSur3EvOp+/3VzKzxjVA43ZT89mJvaRvI3vDpLdrFgOLe0lfA5RrL3wLL1puZvk18vA2M7O+qAEmM3XwM7P8GqDkV8nwNkn6hKQvpeMjJM2pftbMrB5V2tNb7zO/VDLC4yrgFODsdLwD+E7VcmRm9a8BprGvpNr7roh4h6QHACLipbSEpZkVVZ2X6ipRSfDrSO/hBGQvJtIQazeZ2f6q9yptJSoJflcAPwEmSlpMNsvL/6hqrsysfkVBensj4l8l3Uf20qGAMyPi8arnzMzqVxFKfpKOAHYCPytNi4iN1cyYmdWxIgQ/spXaeqaNGQHMBNaRzaZqZgVUiDa/iPjj0uM0o8un+jjdzGxIyD3CIyLul/TOamTGzIaIIpT8JH225LAJeAfwQtVyZGb1rSi9vcAhJfudZG2AP6pOdsxsSGj0kl96ufngiPi7QcqPmdU50eAdHpJaIqLTU9ab2Vs0cvAjW6HtHcCDklYCNwKv9XwYET+uct7MrB4NgRlbKlFJm99YYBvZmh097/sF4OBnVlQN3uExMfX0Pspb18ZsgLhvZvur0Ut+zcDB7OeCwGbWwBogApQLfm0Rcdmg5cTMhoYBWr2t1soFv/qehtXMaqbRq729rptpZtbQJb+IeHEwM2JmQ0dRhreZmb2pAG1+ZmZvIRqjQ8DBz8zyc8nPzIqoEXp7K1m03Mxsb1HhVoak6ZJul/S4pLWS/ialj5V0m6T16eeYkmsukbRB0jpJ80rST5L0SPrsCkn91swd/MwsnzSZaSVbPzqBz0XE24GTgQslHQNcDKyKiFnAqnRM+mwh2fpB84Gr0rR7AFcDi4BZaZvf38Md/MwsvwEo+UVEW0Tcn/Z3AI8DU4EFwNJ02lLgzLS/AFgWEbsi4ilgAzBH0mRgdETcHREB/KDkmj65zc/McsvR5jde0pqS4yURseQt95NmACcCvwUmRUQbZAFS0sR02lTgnpLLNqW0jrS/b3pZDn5mll/lwa89ImaXO0HSwWRLY/xtRLxSprmur0lW9mvyFVd7zSw3RWVbv/eRWskC37+WTJD8fKrKkn5uTembgOkll08DtqT0ab2kl+XgZ2b5BNlkppVsZaQe2e8Dj0fEP5R8tBI4N+2fC6woSV8oabikmWQdG6tTFXmHpJPTPc8puaZPrvaaWS4DuIDRqcAngUckPZjSvgB8DVgu6TxgI3AWQESslbQceIysp/jCiOhK110AXAeMBG5KW1kOfmaW3wAEv4j4DX2PlOt1VqmIWAws7iV9DXBcnuc7+JlZboqhP8TDwc/M8vGsLmZWVI0wttfBz8xy82SmZlZMLvmZWeFU+AJzvXPwM7P8HPzMrGgG8CXnmnLwM7Pc1D30o5+Dn5nl4/f8rD+z577C+ZdvobkpuOmGsSz/9qRaZ6nQoit45pOdtEwU077VQvt3u3j5p900p0nSx3+6mYPfnc31se2fu3h5RTc0waS/a+agU7L0jYs66WwPmkZk10z7dgstYxthLbN8/KpLGZKuBf4C2BoRucbcNYKmpuDCr27mkoVvo72tlSt/uZ57bjmUjetH1DprhfXSDd0Mmym6X3szbczHmhj7yea9ztv1ZLDj1m5mLG+h8wXY9OlOZv5YqDkLclO+0syIYwo+IVIDlPyq+Ru8jgrm0W9UR524ky1PD+O5jcPp7GjijhWHccq8l2udrcLqeD547d+DQ8/s/6/8q7/u5pD3NdE0TAybKlqnizfWNsC/9gE0UPP51VLVgl9E3Am8WK3717txh3fwwpZhe47b21oZP7mjhjkqtq3f6GLCZ5rZd5Lgl5Z389TCDtou7aTrlexfa+dWaJn05omtE7O0Hm2XdvH0xzpov6aLaIAB/rkFEFHZVsdqXnaXtEjSGklrOthV6+wMmN5m4q7zvwsN69W7umkZK0a8fe9fymEfbeJtP21hxvUttIwXW7/Z1ccd2DPx0uSvNDPz31o54nstvP5A8MovivlLHaDV22qq5sEvIpZExOyImN3K8FpnZ8C0t7UyYcruPcfjJ3ew7bnWGuaouF5/KHj1zm5+/6EOtnyxi533Blv+Zyct47J2PDWJwz7StKdq2zIROp9/M6h1bIWWCdl+68QsCjYdJEbPbypkdbjnPT9Xe61X6x4cxdSZu5k0fRctrd3MXbCde249tNbZKqQJFzVz5C9bOfJnrUxZ3Myod4opl7fQ2f7mv84dt3cz/MgssB18WhM7bu2me3ewe3PQ8Www4lgRnUHn9uya6Axevaub4UfW5CvVVqVV3jqv6vhVlyrp7hLf+eJUvnr9kzQ1w63LxvLME+7prScv/GMXbzwRIGidLA7/YtbrO/xIcch7m3j6rE5ohomfb0bNovv1YNNFnUQn0A2j5jRx6EeKWX6o91JdJar5qssNwFyydTs3AV+OiO9X63n16N5fjebeX42udTasxKjZTYyanQWsyZf3/dd/3HnNjDtv71dgmkaKGT900wXQEK+6VC34RcTZ1bq3mdWWS35mVjwBdA396OfgZ2a5ueRnZsVU5z25lXDwM7PcXPIzs+LxlFZmVkQC5A4PMysiuc3PzAqnQaq9xRybY2YHYODG9kq6VtJWSY+WpI2VdJuk9ennmJLPLpG0QdI6SfNK0k+S9Ej67Aqpt3mV9ubgZ2a5DeCsLtfx1kmPLwZWRcQsYFU6RtIxwELg2HTNVZJ6xiBeDSwCZqWt34mUHfzMLL8BKvn1MenxAmBp2l8KnFmSviwidkXEU8AGYI6kycDoiLg7stllf1ByTZ/c5mdm+USu3t7xktaUHC+JiCX9XDMpItoAIqJN0sSUPhW4p+S8TSmtI+3vm16Wg5+Z5Vd5h0d7RMweoKf21o4XZdLLcrXXzHJTREXbfno+VWVJP3tWUNkETC85bxqwJaVP6yW9LAc/M8uvujM5rwTOTfvnAitK0hdKGi5pJlnHxupURd4h6eTUy3tOyTV9crXXzPIJYIAWJ+pt0mPga8BySecBG4GzACJiraTlwGNAJ3BhRPSsOnUBWc/xSOCmtJXl4GdmuYgDqtLupcykx+/p4/zFwOJe0tcAx+V5toOfmeXXXefrUlbAwc/M8hnAam8tOfiZWW6e2MDMisnBz8yKp/4XJK+Eg5+Z5ePV28ysqNzmZ2bF5OBnZoUTQLeDn5kVjjs8zKyoHPzMrHAC6Br6Qzwc/Mwsp4Bw8DOzInK118wKx729ZlZYLvmZWSE5+JlZ4URAV1f/59U5Bz8zy88lPzMrJAc/MyuecG+vmRVQQPglZzMrJA9vM7PCifDSlWZWUO7wMLMiCpf8zKx4PJmpmRWRJzYwsyIKIBpgeFtTrTNgZkNMpMlMK9n6IWm+pHWSNki6eBByv4dLfmaWWwxAtVdSM/Ad4D8Am4B7Ja2MiMcO+OYVcMnPzPIbmJLfHGBDRDwZEbuBZcCCquc9UdRRr42kF4Bnap2PKhgPtNc6E5ZLo/7O/iAiJhzIDSTdTPbnU4kRwBslx0siYkm6z0eB+RHx1+n4k8C7IuKiA8lfpeqq2nugv5R6JWlNRMyudT6scv6d9S0i5g/QrdTb7Qfo3v1ytdfMamUTML3keBqwZbAe7uBnZrVyLzBL0kxJw4CFwMrBenhdVXsb2JJaZ8By8++syiKiU9JFwC1AM3BtRKwdrOfXVYeHmdlgcbXXzArJwc/MCsnBr4pqOXTH9o+kayVtlfRorfNi1eXgVyUlQ3feDxwDnC3pmNrmyipwHTBQ77FZHXPwq56aDt2x/RMRdwIv1jofVn0OftUzFXi25HhTSjOzOuDgVz01HbpjZuU5+FVPTYfumFl5Dn7VU9OhO2ZWnoNflUREJ9AzdOdxYPlgDt2x/SPpBuBu4ChJmySdV+s8WXV4eJuZFZJLfmZWSA5+ZlZIDn5mVkgOfmZWSA5+ZlZIDn5DiKQuSQ9KelTSjZJGHcC9rkurZyHpmnKTLkiaK+lP9+MZT0t6yypffaXvc86rOZ/1vyT997x5tOJy8BtaXo+IEyLiOGA3cH7ph2kmmdwi4q/7WSh6LpA7+JnVMwe/oesu4A9Tqex2SdcDj0hqlvR1SfdKeljSpwCU+bakxyT9ApjYcyNJd0ianfbnS7pf0kOSVkmaQRZk/1sqdf6ZpAmSfpSeca+kU9O14yTdKukBSd+l9/HNe5H0U0n3SVoradE+n30j5WWVpAkp7UhJN6dr7pJ09ID8aVrheAGjIUhSC9k8gTenpDnAcRHxVAogL0fEOyUNB/5d0q3AicBRwB8Dk4DHgGv3ue8E4HvAaeleYyPiRUn/BLwaEf83nXc98M2I+I2kI8hGsbwd+DLwm4i4TNIHgb2CWR/+S3rGSOBeST+KiG3AQcD9EfE5SV9K976IbGGh8yNivaR3AVcBZ+zHH6MVnIPf0DJS0oNp/y7g+2TV0dUR8VRKfx/wJz3tecChwCzgNOCGiOgCtkj6VS/3Pxm4s+deEdHXvHbvBY6R9hTsRks6JD3jL9O1v5D0UgXf6TOSPpL2p6e8bgO6gX9L6T8Efizp4PR9byx59vAKnmH2Fg5+Q8vrEXFCaUIKAq+VJgH/NSJu2ee8D9D/lFqq4BzImktOiYjXe8lLxeMlJc0lC6SnRMROSXcAI/o4PdJzt+/7Z2C2P9zm13huAS6Q1Aog6Y8kHQTcCSxMbYKTgdN7ufZu4M8lzUzXjk3pO4BDSs67lawKSjrvhLR7J/DxlPZ+YEw/eT0UeCkFvqPJSp49moCe0uvHyKrTrwBPSTorPUOSju/nGWa9cvBrPNeQtefdnxbh+S5ZCf8nwHrgEeBq4Nf7XhgRL5C10/1Y0kO8We38GfCRng4P4DPA7NSh8hhv9jpfCpwm6X6y6vfGfvJ6M9Ai6WHgcuCeks9eA46VdB9Zm95lKf3jwHkpf2vx0gC2nzyri5kVkkt+ZlZIDn5mVkgOfmZWSA5+ZlZIDn5mVkgOfmZWSA5+ZlZI/x9MfCFr8yiWQAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# classfication report\n",
    "clf_score(model1, X_train_sc, y_train, X_test_sc, y_test, train=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "669bf295",
   "metadata": {},
   "source": [
    "### 5.4.2 ANN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "5b1378e0",
   "metadata": {},
   "outputs": [],
   "source": [
    "mlp = MLPClassifier(hidden_layer_sizes = (5),max_iter = 1000, random_state = 5) \n",
    "model2 = mlp.fit(X_train_sc, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "b3763948",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.6938518231186966\n"
     ]
    }
   ],
   "source": [
    "# cross-validation\n",
    "y_test_pred = model2.predict(X_test_sc)\n",
    "accuracy_ANN = metrics.accuracy_score(y_test, y_test_pred)\n",
    "print(accuracy_ANN) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "ceb08f4a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation Result:\n",
      "\n",
      "accuracy score: 0.6939\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAawUlEQVR4nO3de7QddX338ffHBBG5BBSwEi5BCiJaoBjBa0VEBYoP3h4FURQvNCqtq0/rA6urXii2VXzU6gIEKogWhKJSRYvwWKmgD1IJSy4GCkSuEVERCHIRCHyfP2ZCtseTOfucZM4+Ce/XWntlz8xvz/7OL/vsz9z2TKoKSZJW5gmjLkCSNLMZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GxRouyduT/GDUdaxOSQ5O8n+HaHdCkg9OR03TIclNSfZun38kyWmjrkkCg2Ikkqyb5OQkNyf5TZIfJ9l31HUNo/0yeyDJvUl+keQLSTZYne9RVadX1SuHaLegqo5ene+9XJJKcl+7nD9L8qkks/p4r7VdklOTLEuyxZjxH0nycNvHdye5OMkLpjD/N7d/S/cl+XqSp0zQ/v1JbmzbX5Nkh3b8y5Jc1dby6yT/lmTuZOtZGxkUozEbuBV4KTAH+CBwVpJ5oyxqEl5dVRsAuwHPA/52bIMks6e9qtVvl3Y5Xwq8CXjHiOtZrabj/yjJ+sDrgaXAweM0+de2jzcDfgCcnSSTmP+zgROBtwJPA+4Hju9o/y7gncCfAhsA+wN3tJOvBl5VVRsDWwDXA58btpa1mUExAlV1X1V9pKpuqqpHq+pbwI3Ac1f2miRbJTk7ya/atZ1jV9LuM0luTXJPksuSvGRg2u5JFrbTfpHkU+34JyU5rZ3v3UkuTfK0IZbjZ8C3gee086kk70tyPc0fGUn2T3L5wBrjzhMt0+DutDQ+neSXSZYmuTLJ8vc7NclHB+b37iSLk9yZ5JzBNdi2tgVJrk9yV5Ljhv1CqqrFwP8Ddh2Y31SWa7skF7Tj7khyepKNh6lhrCQHtO9/T5KfJtmnHf/Y7qt2+LFdWEnmtf3wziS3ABckOS/J4WPmfUWS17XPd0zynbZPr03yxkmW+nrgbuDvgLetrFFVPQx8EfgD4KmTmP/BwDer6qKqupdmpet1STYc2zDJE4APA39ZVVdX46dVdWdbwy+q6raBlzwC/OEkallrGRQzQPulvAOwaCXTZwHfAm4G5gFzgTNXMrtLab7QngJ8GfhKkie10z4DfKaqNgK2A85qx7+NZstmK5o/0gXAA0PUvRWwH/DjgdGvAfYAdkqyG3AK8GftfE8Ezkmz623YZXol8Cc0/bMxzZr9r8epZS/gH4E3Ak9v5zt2fvvTbAHt0rZ71UTL2M57R+AlwOJ2eKrLlbbGLYBn0fT3R4apYUw9uwNfAj5A0yd/Atw0iVm8tH3/V9F8Rg4amPdOwDbAv7dbA99p22zetju+XYtfvsvnygne623AGTR9sGPbd+Mt07rA24ElVXVHkhe3Ibyyx4vblz4buGL5fKrqp8BDNJ+XsbZsH89pV6ZuTHJUGyDL69g6yd00n/+/Bo6ZYPkeH6rKxwgfwDrAfwAndrR5AfArYPY4094O/KDjtXfR7EIBuAg4Cth0TJt3ABcDOw9R703AvTRriTfTbOav104rYK+Btp8Djh7z+mtpvqiGWiZgL+A64PnAE8a0OxX4aPv8ZOCYgWkbAA8D8wZqe/HA9LOAIzuWs4B7gPva52cA667Kco3zHq8Bfjymb/dun38EOG0lrzsR+HTH/8/eA8OPzYcmuAp4xsD0Ddtl3KYd/nvglPb5m4Dvj/PeHx7ys7018Ciwazt8Ps2KymBtD7WfpV8CFwDPneTfz3eBBWPG/QzYc5y2L2yX/99pAnZe+9l69zhtnwIcATx/MvWsrQ+3KEaoXZP5F5o/lsMHxn87zQG+e5McTLPmeXNVLRtinn+V5gDd0nbNaA6waTv5nTRrWv/d7l7avx3/LzR/xGcmuS3JMUnW6Xib11TVxlW1TVW9t6oGtz5uHXi+DfBXg2uC7bJsMewyVdUFwLHAccAvkpyUZKNxmm5BE1zLX3cvzZbH4MHI2wee308TJiRZNNDfLxlos1vb5k00W0nrr8pyJdk8yZlpDo7fA5zGiv+bydgK+OkUXrfcY/9HVfUbmi/OA9tRBwKnt8+3AfYYs5wH0+weGsZbgWuq6vJ2+HTgzWM+W2e1n6XNq2qvqrpskstyLzD287AR8Jtx2i7/nB5TVXdX1U00wbff2IbV7I76IvCNrB3H21aJQTEi7f7xk2kOwL2+mn20AFTVvlW1Qfs4neYPe+uJPrDtl9wRNLtVNqnmoNxSml0eVNX1VXUQzW6EjwNfTbJ+VT1cVUdV1U40a137A4dMcdEGL0d8K/D37RfB8seTq+qMYZeprfuzVfVcmt0MO9DschnrNpovNuCxg6hPpVm7nGj+zx7o7++PmVZVdRbwQ+BDq7hc/0jTPztXs/vvLbT/N5N0K82uw/HcBzx5YHi8L/Wxl4w+AzgozRlH6wH/OfA+F45Zzg2q6j1D1nkI8Iwktye5HfgUTTBOeIZfkpcMhPd4j+WBvohmV+Ly1z0DWJdmS2Gsa2lWyoa9ZPZsmr+V8VZMHlcMitH5HM1+4lePWSMfz4+AnwMfS7J+moPPLxqn3YbAMtpdH0k+xMCHPMlbkmxWVY/SbO4DPJLmtMA/avev30Ozy+aRVVm41j8DC5Lskcb6Sf60PdA41DIleV77+nVovgR/u5LavgwcmmTXdn/3PwD/1a41rg4fAw5L8gersFwb0u62S3Pa5XiBN4yTaZb15UmekGRumuMoAJcDByZZJ8l84A1DzO9cmpD9O5qzkB5tx38L2CHJW9v5rdP+fzxrohm2obMdsDvNMbNdaU56+DIdB7WXq6rvD4T3eI/lgX468Oo2WNZvl+Hsdktp7DzvB/4V+N9JNkyyJfDudjlJ8rokz2z7dDOaYPtxu3XxuGZQjECSbWgOhO4K3D5mN9PvqapHgFfTnIFxC7CEZnfIWOfTnIV0Hc1umN/yu7uC9gEWJbmX5sD2gVX1W5q1zq/ShMQ1wIU0u0VWSVUtpPlDPJbmWMlimuMPk1mmjWi+mO9ql+nXwP8Z572+S3PGy9dovqi3Y8XulFVWVVfR9MsHVmG5jqLZnbWUZnfP2VOs5UfAocCn23ldyIqtqQ/SLPtd7ft9eYj5PdjWsvdg+/bL9pU0/Xgbza67j9OssS//YeS4J2DQhME3quqqqrp9+YPmc7d/Jvitw7CqahHNyRen0xzn2BB47/LpaX6UecLASw6nCevbaLYSv0xzYgI0uynPo9ltdRXN8ZXXro4613Sp8sZFkqSVc4tCktTJoJAkdTIoJEmdDApJUqc17ockm266ac2bN2/UZUjSGuWyyy67o6o2m8pr17igmDdvHgsXLhx1GZK0Rkly88StxueuJ0lSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUqbegSHJKmvsc/2Ql05Pks2nucXxlVnKLREnSaPW5RXEqzWWtV2ZfYPv2cRjN/RkkSTNMb0FRVRcBXTf8OAD4UnsHsUuAjZM8faL5PvTQ6qpQkjSMUR6jmMvv3lRnCb97f+PHJDksycIkC3/+87umpThJUmOUQTHevYLHvYtSVZ1UVfOrav6cOZv0XJYkadAog2IJsNXA8JY0tyeUJM0gowyKc4BD2rOfng8sraqfj7AeSdI4ert6bJIzgD2BTZMsAT4MrANQVScA5wL70dyY/n6am8VLkmaY3oKiqg6aYHoB7+vr/SVJq4e/zJYkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1KnXoEiyT5JrkyxOcuQ40+ck+WaSK5IsSnJon/VIkiavt6BIMgs4DtgX2Ak4KMlOY5q9D7i6qnYB9gQ+meSJfdUkSZq8PrcodgcWV9UNVfUQcCZwwJg2BWyYJMAGwJ3Ash5rkiRNUp9BMRe4dWB4STtu0LHAs4DbgKuA91fVo2NnlOSwJAuTLFy69K6+6pUkjaPPoMg442rM8KuAy4EtgF2BY5Ns9HsvqjqpquZX1fw5czZZ3XVKkjr0GRRLgK0Ghrek2XIYdChwdjUWAzcCO/ZYkyRpkvoMikuB7ZNs2x6gPhA4Z0ybW4CXAyR5GvBM4IYea5IkTdLsvmZcVcuSHA6cD8wCTqmqRUkWtNNPAI4GTk1yFc2uqiOq6o6+apIkTV5vQQFQVecC544Zd8LA89uAV/ZZgyRp1fjLbElSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1KnXS3j04dFH4brrRl2FtGZ6ylNg001HXYXWNGtcUABcdNGoK5DWPA8+2ATFQQeNuhKtada4oJg9G/74j0ddhbTmuflmuPPOUVehNZHHKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktRpqKBI8qIk30lyXZIbktyY5IYhXrdPkmuTLE5y5Era7Jnk8iSLklw42QWQJPVr9pDtTgb+ErgMeGSYFySZBRwHvAJYAlya5JyqunqgzcbA8cA+VXVLks0nUbskaRoMGxRLq+rbk5z37sDiqroBIMmZwAHA1QNt3gycXVW3AFTVLyf5HpKkng17jOI/k3wiyQuS7Lb8McFr5gK3DgwvaccN2gHYJMn3klyW5JAh65EkTZNhtyj2aP+dPzCugL06XpNxxtU47/9c4OXAesAPk1xSVdf9zoySw4DDADbffOshS5YkrQ5DBUVVvWwK814CbDUwvCVw2zht7qiq+4D7klwE7AL8TlBU1UnASQA77DB/bNhIkno07FlPc5J8KsnC9vHJJHMmeNmlwPZJtk3yROBA4Jwxbb4BvCTJ7CRPptlyuWayCyFJ6s+wxyhOAX4DvLF93AN8oesFVbUMOBw4n+bL/6yqWpRkQZIFbZtrgPOAK4EfAZ+vqp9MZUEkSf0Y9hjFdlX1+oHho5JcPtGLqupc4Nwx404YM/wJ4BND1iFJmmbDblE8kOTFyweSvAh4oJ+SJEkzybBbFO8BvtgelwhwJ/D2voqSJM0cw571dDmwS5KN2uF7+ixKkjRzdAZFkrdU1WlJ/teY8QBU1ad6rE2SNANMtEWxfvvvhn0XIkmamTqDoqpObP89anrKkSTNNMP+4O6YJBslWSfJd5PckeQtfRcnSRq9YU+PfWV7AHt/mstu7AB8oLeqJEkzxrBBsU77737AGVV1Z0/1SJJmmGF/R/HNJP9N8yO79ybZDPhtf2VJkmaKobYoqupI4AXA/Kp6GLiP5iZEkqS13ES/o9irqi5I8rqBcYNNzu6rMEnSzDDRrqeXAhcArx5nWmFQSNJab6LfUXy4/ffQ6SlHkjTTDPs7in9IsvHA8CZJPtpbVZKkGWPY02P3raq7lw9U1V00p8pKktZyw54eOyvJulX1IECS9YB1+ytLUh8efBCuu27idtKgYYPiNOC7Sb5AcxD7HcAXe6tK0mo3Zw7cfjtcdNGoK9FobLj+xG3GN+z9KI5JciWwN82Ni46uqvOn+qaSpt/GG8Mee4y6Co3OrFlTfeWwWxQA1wDLquo/kjw5yYZV9ZupvrEkac0w7FlP7wa+CpzYjpoLfL2nmiRJM8iwZz29D3gRcA9AVV0PbN5XUZKkmWPYoHiwqh5aPpBkNs1BbUnSWm7YoLgwyd8A6yV5BfAV4Jv9lSVJmimGDYojgF8BVwF/BpwL/G1fRUmSZo4Jz3pK8gTgyqp6DvDP/ZckSZpJJtyiqKpHgSuSbD0N9UiSZphhf0fxdGBRkh/R3LQIgKr6H71UJUmaMYYNiqN6rUKSNGNNdIe7JwELgD+kOZB9clUtm47CJEkzw0THKL4IzKcJiX2BT/ZekSRpRplo19NOVfVHAElOBn7Uf0mSpJlkoi2Kh5c/cZeTJD0+TRQUuyS5p338Bth5+fMk90w08yT7JLk2yeIkR3a0e16SR5K8YbILIEnqV+eup6qa8vXLk8wCjgNeASwBLk1yTlVdPU67jwPe30KSZqBhL+ExFbsDi6vqhvaCgmcCB4zT7s+BrwG/7LEWSdIU9RkUc4FbB4aXtOMek2Qu8FrghK4ZJTksycIkC5cu/dVqL1SStHJ9BkXGGTf20uT/BBxRVY90zaiqTqqq+VU1f86czVZXfZKkIUzmVqiTtQTYamB4S+C2MW3mA2cmAdgU2C/Jsqr6eo91SZImoc+guBTYPsm2wM+AA4E3Dzaoqm2XP09yKvAtQ0KSZpbegqKqliU5nOZsplnAKVW1KMmCdnrncQlJ0szQ5xYFVXUuzU2OBseNGxBV9fY+a5EkTU2fB7MlSWsBg0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdeg2KJPskuTbJ4iRHjjP94CRXto+Lk+zSZz2SpMnrLSiSzAKOA/YFdgIOSrLTmGY3Ai+tqp2Bo4GT+qpHkjQ1fW5R7A4srqobquoh4EzggMEGVXVxVd3VDl4CbNljPZKkKegzKOYCtw4ML2nHrcw7gW+PNyHJYUkWJlm4dOmvVmOJkqSJ9BkUGWdcjdsweRlNUBwx3vSqOqmq5lfV/DlzNluNJUqSJjK7x3kvAbYaGN4SuG1soyQ7A58H9q2qX/dYjyRpCvrcorgU2D7JtkmeCBwInDPYIMnWwNnAW6vquh5rkSRNUW9bFFW1LMnhwPnALOCUqlqUZEE7/QTgQ8BTgeOTACyrqvl91SRJmrw+dz1RVecC544Zd8LA83cB7+qzBknSqvGX2ZKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROvQZFkn2SXJtkcZIjx5meJJ9tp1+ZZLc+65EkTV5vQZFkFnAcsC+wE3BQkp3GNNsX2L59HAZ8rq96JElT0+cWxe7A4qq6oaoeAs4EDhjT5gDgS9W4BNg4ydN7rEmSNEmze5z3XODWgeElwB5DtJkL/HywUZLDaLY4gDw8f/4mN63eUtdUD86BdZeOuoqZwb5Ywb5Ywb5Y4Z4tp/rKPoMi44yrKbShqk4CTgJIsrDqrvmrXt6ar+mL++0L7ItB9sUK9sUKSRZO9bV97npaAmw1MLwlcNsU2kiSRqjPoLgU2D7JtkmeCBwInDOmzTnAIe3ZT88HllbVz8fOSJI0Or3teqqqZUkOB84HZgGnVNWiJAva6ScA5wL7AYuB+4FDh5j1ST2VvCayL1awL1awL1awL1aYcl+k6vcOCUiS9Bh/mS1J6mRQSJI6zdig8PIfKwzRFwe3fXBlkouT7DKKOqfDRH0x0O55SR5J8obprG86DdMXSfZMcnmSRUkunO4ap8sQfyNzknwzyRVtXwxzPHSNk+SUJL9M8pOVTJ/a92ZVzbgHzcHvnwLPAJ4IXAHsNKbNfsC3aX6L8Xzgv0Zd9wj74oXAJu3zfR/PfTHQ7gKakyXeMOq6R/i52Bi4Gti6Hd581HWPsC/+Bvh4+3wz4E7giaOuvYe++BNgN+AnK5k+pe/NmbpF4eU/VpiwL6rq4qq6qx28hOb3KGujYT4XAH8OfA345XQWN82G6Ys3A2dX1S0AVbW29scwfVHAhkkCbEATFMumt8z+VdVFNMu2MlP63pypQbGyS3tMts3aYLLL+U6aNYa10YR9kWQu8FrghGmsaxSG+VzsAGyS5HtJLktyyLRVN72G6YtjgWfR/KD3KuD9VfXo9JQ3o0zpe7PPS3isitV2+Y+1wNDLmeRlNEHx4l4rGp1h+uKfgCOq6pFm5XGtNUxfzAaeC7wcWA/4YZJLquq6voubZsP0xauAy4G9gO2A7yT5flXd03NtM82UvjdnalB4+Y8VhlrOJDsDnwf2rapfT1Nt022YvpgPnNmGxKbAfkmWVdXXp6XC6TPs38gdVXUfcF+Si4BdgLUtKIbpi0OBj1Wzo35xkhuBHYEfTU+JM8aUvjdn6q4nL/+xwoR9kWRr4GzgrWvh2uKgCfuiqratqnlVNQ/4KvDetTAkYLi/kW8AL0kyO8mTaa7efM001zkdhumLW2i2rEjyNOCZwA3TWuXMMKXvzRm5RVH9Xf5jjTNkX3wIeCpwfLsmvayq1rorZg7ZF48Lw/RFVV2T5DzgSuBR4PNVNe5pk2uyIT8XRwOnJrmKZvfLEVV1x8iK7kmSM4A9gU2TLAE+DKwDq/a96SU8JEmdZuquJ0nSDGFQSJI6GRSSpE4GhSSpk0EhSepkUEjjaK88e3mSn7RXHd14Nc//piSbts/vXZ3zllY3g0Ia3wNVtWtVPYfmImvvG3VB0qgYFNLEfkh74bQk2yU5r73I3veT7NiOf1qSf2vvd3BFkhe247/etl2U5LARLoM0ZTPyl9nSTJFkFs2lH05uR50ELKiq65PsARxPc6G5zwIXVtVr29ds0LZ/R1XdmWQ94NIkX1uLr8WltZRBIY1vvSSXA/OAy2iuNroBzU2ivjJwZdp123/3Ag4BqKpHgKXt+L9I8tr2+VbA9oBBoTWKQSGN74Gq2jXJHOBbNMcoTgXurqpdh5lBkj2BvYEXVNX9Sb4HPKmPYqU+eYxC6lBVS4G/AP4aeAC4Mcn/hMfuP7z8/uTfBd7Tjp+VZCNgDnBXGxI70tx6UlrjGBTSBKrqxzT3YT4QOBh4Z5IrgEWsuOXm+4GXtVcnvQx4NnAeMDvJlTRXL71kumuXVgevHitJ6uQWhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjr9f0uvjiISROlyAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "roc auc score: 0.692417363726044\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAABAA0lEQVR4nO3dd3hU5fLA8e+INKmKiFcRQQUEpIgRBKUoKmIBBQvq1WvFrogNRWxYsIsNC3qx/fCqIFhoFhAUUbDRQQSEUJSmEhAhZH5/zIlZYrLZlM3J7s7nefYhu3t2z+wJObPnLfOKquKcc87lZ5ewA3DOOVe2eaJwzjkXlScK55xzUXmicM45F5UnCuecc1F5onDOOReVJwpXKCIyV0Q6hx1HWSEit4nIsJD2PVxE7g1j3yVNRM4VkYlFfK3/n4wzTxQJTESWicifIpIhImuCE0fVeO5TVZup6uR47iObiFQUkQdEZHnwOX8UkZtEREpj/3nE01lE0iMfU9X7VfWSOO1PRORaEZkjIptFJF1E3haR5vHYX1GJyF0i8npx3kNV31DV42PY1z+SY2n+n0xVnigS3ymqWhVoBRwK3BpuOIUnIrvm89TbQBfgRKAacB7QBxgShxhERMra38MQ4DrgWmAPoBEwGjippHcU5XcQd2Hu28VIVf2WoDdgGXBsxP2HgA8j7h8BTAN+A34AOkc8twfwX2AVsBEYHfHcycD3weumAS1y7xPYB/gT2CPiuUOBdUD54P5FwPzg/ScA+0dsq8BVwI/A0jw+WxdgK7BfrsfbAjuAg4L7k4EHgK+B34ExuWKKdgwmA/cBXwSf5SDgwiDmTcAS4LJg2yrBNllARnDbB7gLeD3Ypn7wuf4DLA+OxYCI/VUGXgmOx3zgZiA9n99tw+Bztony+x8OPAN8GMT7FXBgxPNDgBXAH8A3QIeI5+4C3gFeD56/BGgDfBkcq9XA00CFiNc0Az4CNgC/ALcBJwDbgO3BMfkh2LYG8FLwPiuBe4FywXMXBMf88eC97g0e+zx4XoLnfg1+p7OAQ7AvCduD/WUA7+f+OwDKBXH9FByTb8j1f8hvRTjXhB2A34rxy9v5D6QuMBsYEtzfF1iPfRvfBTguuF87eP5D4H/A7kB5oFPweOvgD7Rt8Ef3n2A/FfPY56fApRHxPAw8F/x8KrAYaALsCtwOTIvYVoOTzh5A5Tw+22Dgs3w+98/knMAnByeiQ7CT+UhyTtwFHYPJ2Am9WRBjeezb+oHByaoTsAVoHWzfmVwndvJOFC9iSaEl8BfQJPIzBce8LnYCzC9RXA78XMDvfzh2om0TxP8G8GbE8/8GagXP3QCsASpFxL09+D3tEsR7GJZYdw0+y3ygb7B9NeykfwNQKbjfNvcxiNj3aOD54HeyF5bIs39nFwCZwDXBviqzc6Loip3gawa/hybAvyI+871R/g5uwv4OGgevbQnUCvtvNdFvoQfgt2L88uwPJAP75qTAJ0DN4LlbgNdybT8BO/H/C/tmvHse7zkUGJTrsYXkJJLIP8pLgE+DnwX79toxuD8OuDjiPXbBTrr7B/cVOCbKZxsWedLL9dx0gm/q2Ml+cMRzTbFvnOWiHYOI195TwDEeDVwX/NyZ2BJF3YjnvwZ6Bz8vAbpGPHdJ7veLeG4AML2A2IYDwyLunwgsiLL9RqBlRNxTCnj/vsC7wc9nA9/ls93fxyC4XwdLkJUjHjsbmBT8fAGwPNd7XEBOojgGWIQlrV3y+MzREsVCoEdx/7b8tvOtrLXJusI7VVWrYSexg4E9g8f3B84Qkd+yb8BRWJLYD9igqhvzeL/9gRtyvW4/rJklt3eAdiKyD9ARO0lOjXifIRHvsQFLJvtGvH5FlM+1Log1L/8Kns/rfX7Grgz2JPoxyDMGEekmItNFZEOw/YnkHNNYrYn4eQuQPcBgn1z7i/b515P/549lX4jIDSIyX0R+Dz5LDXb+LLk/eyMR+SAYGPEHcH/E9vthzTmx2B/7HayOOO7PY1cWee47kqp+ijV7PQP8IiIviEj1GPddmDhdjDxRJAlV/Qz7tvVI8NAK7Nt0zYhbFVUdHDy3h4jUzOOtVgD35Xrdbqo6Io99/gZMBM4EzgFGaPC1Lnify3K9T2VVnRb5FlE+0sdAWxHZL/JBEWmDnQw+jXg4cpt6WJPKugKOwT9iEJGKWNPVI0AdVa0JjMUSXEHxxmI11uSUV9y5fQLUFZG0ouxIRDpgV1RnYleONbH2/sgRY7k/z1BgAdBQVatjbf3Z26/AmuTykvt9VmBXFHtGHPfqqtosymt2fkPVJ1X1MKxZsBHWpFTg6wqI0xWRJ4rk8gRwnIi0wjopTxGRriJSTkQqBcM766rqaqxp6FkR2V1EyotIx+A9XgQuF5G2wUigKiJykohUy2ef/wecD/QKfs72HHCriDQDEJEaInJGrB9EVT/GTpYjRaRZ8BmOwNrhh6rqjxGb/1tEmorIbsA9wDuquiPaMchntxWAisBaIFNEugGRQzZ/AWqJSI1YP0cub2HHZHcR2Re4Or8Ng8/3LDAiiLlCEH9vEekfw76qYf0Aa4FdReQOoKBv5dWwju0METkYuCLiuQ+AvUWkbzBsuZqItA2e+wWonz1qLPj/NRF4VESqi8guInKgiHSKIW5E5PDg/195YDM2qGFHxL4OiPLyYcAgEWkY/P9tISK1Ytmvy58niiSiqmuBV4GBqroC6IF9K1yLfdO6iZzf+XnYN+8FWOd13+A9ZgKXYpf+G7EO6Qui7PY9bITOL6r6Q0Qs7wIPAm8GzRhzgG6F/Ei9gEnAeKwv5nVsJM01ubZ7DbuaWoN1tF4bxFDQMdiJqm4KXvsW9tnPCT5f9vMLgBHAkqBJJa/muGjuAdKBpdgV0zvYN+/8XEtOE8xvWJPKacD7MexrAvZlYBHWHLeV6E1dADdin3kT9oXhf9lPBMfmOOAU7Dj/CBwdPP128O96Efk2+Pl8LPHOw47lO8TWlAaW0F4MXvcz1gyXfaX8EtA0OP6j83jtY9jvbyKW9F7COstdMUhOS4FziUdEJmMdqaHMji4OEbkC6+iO6Zu2c2HxKwrnSomI/EtEjgyaYhpjQ03fDTsu5woSt0QhIi+LyK8iMief50VEnhSRxSIyS0RaxysW58qICtjon01YZ/wYrB/CuTItbk1PQedoBvCqqh6Sx/MnYm3NJ2KTu4aoatvc2znnnAtX3K4oVHUKNnY+Pz2wJKKqOh2oKSKxdnY555wrJWEW49qXnUdhpAePrc69oYj0weq8UKVKlcMOPvjgUgnQOecSkSr88Qds3AgVN6ymjq7hO7LWqWrtorxfmIkir1LRebaDqeoLwAsAaWlpOnPmzHjG5ZxzCScjA8aNg5Ej4cMPISNDqVFDGNDpPXrsNpEDxz7zc1HfO8xEkc7OM1PrYpVMnXPOxeC33+D99y05TJgAW7fCQbU2Mm6fG6nT7gD2f2EAFSp0B7qDPFPk/YQ5PPY94Pxg9NMRwO/BjE7nnHP5WLsWXnwRTjgBateG88+HmTPh0kth9qB3WVS+KUf99AoN62+nQoWS2WfcrihEZARWqG5PsVXB7sQKhaGqz2E1dE7EZv5uwdYBcM45l8vKlTBqlF05TJ0KWVlwwAFw/fXQqxccXu8XdrnuGnjqbWjVytqeWpfcjIO4JQpVPbuA5xVbuMY551wuS5ZYYhg1CqZPt8eaNoUBAyw5tGgBfy8KPHOFJYf77oObboLy5Us0Fl+C0Dnnyoh583KSw/ff22OtW9v5v2dP2GnA588/WwfF1VdDWhosXw614lP/0BOFc86FRBW++y4nOSxYYI+3bw+PPgqnnQYNGuR6UVYWDB0K/YMiwr16wb/+FbckAZ4onHOuVGVlWVNSdnJYtgzKlYNOneCaa+DUU2Gf/OoSL1wIl1wCn38OXbvC889bkogzTxTOORdnmZkwZYolhnffhVWrrBvhuONg4EDo3h32LGgdxS1b4KijYMcOGD7chjtJXtPRSp4nCueci4O//oJPPrHkMGYMrFsHlStDt27WWnTSSVAjliWwFi2Chg1ht93gtddsVNPee8c7/J14onDOuRKyZQuMH2/J4f33rYxGtWpwyimWHLp2hSpVYnyzrVth0CB48EG7gvj3v23yRAg8UTjnXDH88YeNTB050kpobNli/cqnn24jlY49FipWLOSbfvEFXHyx9UlceKFdfoTIE4VzzhXS+vXw3nuWHD76CLZtsz7lCy6w5NCpE+xa1LProEFw551Qr57V5Tj++IJfE2eeKJxzLgarV8Po0ZYcJk+2PuX997dpDD17Qrt2sEtxiiKpWud0q1Y2/Om++6Bq1ZIJvpg8UTjnXD5+/jmndMa0aXYub9wYbrnFkkPr1iUw8GjDBqvFcdBBNgTqlFPsVoZ4onDOuQiLFlliGDkSvvnGHmvZEu66yzqkmzYtwVGp77wDV11lyWLgwBJ605LnicI5l9JUYfbsnOQwd6493ratDTjq2dO+7Jeo1autzWrUKDjsMJg40bJRGeWJwjmXclRhxoyc2dGLF9tVQocOMGSIlc7Yb7+C36fIVq2yjuoHH4R+/YrR8106ynZ0zjlXQnbssFGn2ckhPd3Oz126WMHVHj2gTp04BrBsmU2uuOYau4pYsQJ23z2OOyw5niicc0lr+3aYNMmSw+jR8OuvNqeha1cbVHTKKaVwrt6xA555Bm67zYZFnXGGzaxOkCQBniicc0lm61Zr8h850uY6/PabzYY+6STrjD7xxFIcdTp/vhXxmzbNZlU//3ypl98oCZ4onHMJLyMDxo615DB2rN2vWdOK7fXqZcX3Klcu5aC2bIGOHa1c7KuvWgmOUiriV9I8UTjnEtLGjdbkP3Kk9Qv/9RfstRecc44lh86dKbE1owtlwQKbbLHbbvDGGzaaKa6dH/HnicI5lzB+/TVndvSnn1r57rp14bLLLDkceaSt7RCKP/+0yRaPPAKvvGJXEGWg/EZJ8EThnCvT0tNzZkd//rm15Bx4oI0q7dXLVgEtVumMkjBlivVF/Pij/XvyySEHVLI8UTjnypyffsoZxvrVV/ZYs2Zw++2WHJo3L0PN/XffbVcSDRrAxx/beNsk44nCORc6VZg3Lyc5/PCDPX7YYXD//TY7unHjcGP8h+wifmlpVqtp0KBCLDaRWDxROOdCoQrffpuTHBYutPNu+/bw2GM2O7p+/bCjzMO6dZYYGjaEO+6wcbchrxcRb54onHOlJisLvvzSEsOoUTZZuVw5G6F03XVw6qm2rkOZpApvv201mjZutDUjUoQnCudcXGVmwmefWWJ4912rh1ehgs1tuOMOmx29555hR1mAVavgyitt8eu0NOuLaNEi7KhKjScK51yJ++svO5eOGmXn1vXrbVpBt27W33DSSVCjRthRFsKaNTYe9+GHoW/fMl/Er6Sl1qd1zsXN5s028W3kSPjgA1tLunp1u2Lo2dMqWOy2W9hRFsKSJVYDpG9fW6Fo+XKb7p2CPFE454rs99/hww8tOYwbZ3POatWyunc9e9pI0YoVw46ykHbsgCefhAEDoHx56N3b6jOlaJIATxTOuUJat86+aI8cac1L27ZZB/SFF9och44dE7hlZu5cuPhim7xx0knw3HMJWcSvpCXqr9M5V4pWr7aO6JEjrWN6xw4bunr11ZYcjjiiDMyOLq4tW6BTJxuj+3//Z1cSZWZWX7g8UTjn8rRsWU7pjC+/tNGhjRvDLbdYcjj00CQ5j86bB02aWAfKm29aEb/atcOOqkzxROGc+9vChTlrR3/7rT3WqpVVqejVC5o2DTW8krVli82FeOwxGD4czjsPjj027KjKJE8UzqUwVZg1Kyc5zJtnj7dtCw89ZB3SBx4YboxxMXkyXHqpLZZ92WW2cIXLlycK51JMVhbMmJFTOuOnn6x/oUMHG+xz2mlWujtp3Xkn3HOPZcBPP4Wjjw47ojLPE4VzKWDHDivRPXKkdUqnp9vIpC5drM+hRw9b9CepZRfxa9MGbrjBkkVCTewIT1wThYicAAwBygHDVHVwrudrAK8D9YJYHlHV/8YzJudSxbZtMGmSJYfRo2HtWqhUCbp2tYqsJ58Mu+8edpSlYO1aKyTVuLFdTaRAEb+SFrdEISLlgGeA44B0YIaIvKeq8yI2uwqYp6qniEhtYKGIvKGq2+IVl3PJ7M8/YeJESw7vvw+//QZVq9p5sVcvK6FRtWrYUZYSVRgxAq691qaJ33132BElrHheUbQBFqvqEgAReRPoAUQmCgWqiYgAVYENQGYcY3Iu6WzaBGPHWnIYO9ZKaey+uzUn9eplxfcqVQo7ylKWng5XXGG1RNq2hZdespWPXJHEM1HsC6yIuJ8OtM21zdPAe8AqoBpwlqpm5X4jEekD9AGoV69eXIJ1LpFs3JgzO3riRCvCt9detkxzr15Wtrt8+bCjDNHatbY86WOP2RVFaAtpJ4d4Joq8puJorvtdge+BY4ADgY9EZKqq/rHTi1RfAF4ASEtLy/0ezqWEX36xvoZRo2ywTmYm7LcfXH65JYf27VP8fLh4sbW3XX+9zQZcscKqErpii2eiSAf2i7hfF7tyiHQhMFhVFVgsIkuBg4Gv4xiXcwljxYqc2dGff27N7gcdZIN2evWypRGSYnZ0cWRmwhNPwMCBVoHwnHOgTh1PEiUonoliBtBQRBoAK4HewDm5tlkOdAGmikgdoDGwJI4xOVfmLV6cM8fh6+Ar0yGH2CI/vXrZzymfHLLNnm1F/GbMsElzzz5rScKVqLglClXNFJGrgQnY8NiXVXWuiFwePP8cMAgYLiKzsaaqW1R1Xbxicq4sUrWipdnJYdYsezwtDR54wGZHN2oUboxl0pYtNllul12sRtOZZ3oGjROxVp/EkZaWpjNnzgw7DOeKRRW++SanWWnRIjvHHXmkXTWcdhrsv3/YUZZRc+bYCCYR+OQTK+JX5tdSDZ+IfKOqaUV5rc/Mdq6UZGXBtGmWHEaNgp9/ts7no4+2/tcePWxdB5ePzZutH+KJJ+CVV6yIX5cuYUeVEjxROBdH27fb+g2jRlnpjDVroEIFOP54myTcvbutCOcK8MknVsRv6VK48krLqq7UeKJwroT99Rd89JElhzFjYMMGKyl04onW33DSST4gp1AGDoR774WGDS3rduwYdkQpxxOFcyVg82YYP976Gz74wGZL16gBp5xiyaFrV68/V2hZWdZR3b493Hwz3HUXVK4cdlQpyROFc0X0+++WFEaOtCTx55/Wp3rWWZYcunSxZiZXSL/+arOpGze2+kzdutnNhcYThXOFsHZtTumMjz+2Poh99oGLLrLRSh06WPluVwSq8MYbVuk1I8PKgLsywf9LO1eAVausI3rkSGsiz8qC+vXtS2+vXlZzbpddwo4ywa1YYbVIxo6Fdu1g2LAkW3c1sXmicC4PS5fmzHH48kt77OCD4dZbLTm0auVzu0rU+vXwxRcwZAhcdVWKF60qezxROBdYsCBn7ejvvrPHDj0UBg2y5NCkSbjxJZ1Fi6wd78YbLfOuWAHVqoUdlcuDJwqXslThhx9yksP8+fb4EUfAww9bh/QBB4QbY1LKzIRHH7WJJJUr28S5OnU8SZRhnihcSsnKskJ72XWVliyx/oWOHW0e12mnwb77hh1lEvvhB+v5//ZbO9jPPONF/BKAJwqX9HbsgKlTLTm8+y6sXGmL+nTpYn0OPXpA7dphR5kCtmyxg77rrvDOO9ae5xKCJwqXlLZts8V9Ro602dFr19pyoCecAIMHw8knQ82aYUeZImbNgubNbcbh229bEb899gg7KlcInihc0vjzT5gwwZLD++/bhLiqVS0p9Oplc7aqVAk7yhSSkQEDBsBTT8Hw4XD++VYB0SUcTxQuoW3aBB9+aMlh7Fhr3dh9d2v+7tULjj3WriRcKfvoI+jTB5Ytg6uvtl+IS1ieKFzC2bAhZ3b0Rx9ZEb46dewLa69e0KmT9UG4kAwYAPffbyU4pk6Fo44KOyJXTDEnChGpoqqb4xmMc/lZswZGj7aRSpMm2QjLevXgiitsGGv79j5HK3TZRfyOOspGCdxxh1/OJYkCE4WItAeGAVWBeiLSErhMVa+Md3AutS1fnrPIz+ef27yHhg1tflbPnrZUqM+OLgPWrLHmpaZNrT6TF/FLOrFcUTwOdAXeA1DVH0TEC8K7uPjxx5w5DjNm2GPNm9vcrJ494ZBDPDmUGaq20ly/ftY5dMQRYUfk4iSmpidVXSE7/3XuiE84LtWo2hLI2clh9mx7/PDDbRhrz552FeHKmJ9/ts7qiROtqWnYMOuTcEkplkSxImh+UhGpAFwLzI9vWC6ZqcLMmTlF93780a4SjjoKHn/ckkO9emFH6aL67Te75Hv6aeso8vK5SS2WRHE5MATYF0gHJgLeP+EKZccOmDYtp89h+XLrfD7mGGu5OPVU2HvvsKN0US1caMPNbrrJJs0tX24TVVzSiyVRNFbVcyMfEJEjgS/iE5JLFtu3w+TJlhjefRd++QUqVoTjj7eFy7p39wm6CWH7dnjkEfulVakC//kP7LWXJ4kUEkuieApoHcNjzrF1q81tGDXKvnxu2GDnlhNPtCalk07yIqEJ5bvv4OKL7d/TT7empr32CjsqV8ryTRQi0g5oD9QWkX4RT1UHfMS6+1tGhq0ZPXKkzZLetAlq1LArhp49oWtXqybtEsyWLXDccTZ7ceRI+2W6lBTtiqICNndiVyDyO+AfwOnxDMqVfb/9Bh98YOeP8ePtSqJ2bejd284nxxwDFSqEHaUrku++s4WEdtvNqry2bGl1UVzKyjdRqOpnwGciMlxVfy7FmFwZtXatVWIdORI++cSarvfZBy65xEpnHHWUVZB2CWrTJptR/cwzNj/i/POhc+ewo3JlQCx/1ltE5GGgGfD3fHxVPSZuUbkyY+VK64geORKmTLEqDQ0awHXXWXJo08ZHRiaF8ePhsstsOdLrrvNmJreTWBLFG8D/gJOxobL/AdbGMygXriVLcuY4TJ9ujzVpArfdZsmhZUufHZ1Ubr3VZjc2aQJffAHt2oUdkStjYkkUtVT1JRG5LqI56rN4B+ZK1/z5OWtHf/+9Pda6Ndx7r325bNIk1PBcPOzYYZNZOne2NsPbb7fxy87lEkui2B78u1pETgJWAXXjF5IrDaqWELKTw4IF9ni7djZkvmdPa2JySWj1arjqKmjWDAYNsmFpXbuGHZUrw2JJFPeKSA3gBmz+RHWgbzyDcvGRlQVffZVTV2npUutf6NQpZ22ZffYJO0oXN6q20ly/fjZMzdeJcDEqMFGo6gfBj78DR8PfM7Ndgtixw9aSee01WLXKhsUfe6w91qMH7Lln2BG6uFu2DC69FD7+GDp0sCJ+jRqFHZVLENEm3JUDzsRqPI1X1TkicjJwG1AZOLR0QnTF9eKL8OCDtnb0Qw/Z7OiaNcOOypWq33+Hb7+FZ5+10U0+VM0Vgqhq3k+IDAf2A74G2gI/A+2A/qo6OqY3FzkBKyhYDhimqoPz2KYz8ARQHlinqp2ivWdaWprOnDkzlt07YONGK9N9yCG2MpyPVkoh8+ZZHZX+/e3+5s1WT8WlJBH5RlXTivLaaE1PaUALVc0SkUrAOuAgVV0TY1DlgGeA47CqszNE5D1VnRexTU3gWeAEVV0uIl5EpoTdeacliyef9CSRMrZts0vHQYOssNZFF1l9Jk8SroiiXX9uU9UsAFXdCiyKNUkE2gCLVXWJqm4D3gR65NrmHGCUqi4P9vNrId7fFWDOHGtpuPxyaNEi7GhcqZg501Z9GjjQhq7Nm+dF/FyxRbuiOFhEZgU/C3BgcF8AVdWCTj37Aisi7qdjTViRGgHlRWQyVk9qiKq+mvuNRKQP0Aegnq9oExNV6NsXqle3ZYxdCti82Ya5VqpktVa6dw87IpckoiWK4k6xyquhI3eHyK7AYUAXrIP8SxGZrqqLdnqR6gvAC2B9FMWMKyW8+67VY3r6aahVK+xoXFx9+60V8atSxX7xLVr4aAVXovJtelLVn6PdYnjvdKwzPFtdbLJe7m3Gq+pmVV0HTAFaFvZDuJ39+SfccAM0b24DXFyS+uMPuPJKOOwweP11e6xjR08SrsTFc4zcDKChiDQI1truDbyXa5sxQAcR2VVEdsOapnw97mJ69FEbNj9kiFdzTVpjx9rM6ueftwl0vXqFHZFLYnE7jahqpohcDUzAhse+rKpzReTy4PnnVHW+iIwHZgFZ2BDaOfGKKRWkp8MDD9h54+ijw47GxcUtt9iopqZNbb2Itrm7/pwrWTElChGpDNRT1YWFeXNVHQuMzfXYc7nuPww8XJj3dfm7+WYr1fHII2FH4kqUqv1iy5WDLl2sw/q227yInysVBTY9icgpwPfA+OB+KxHJ3YTkyoDPP4cRIyxZ1K8fdjSuxKxcCaeeapNiAI4/Hu6+25OEKzWx9FHchc2J+A1AVb8H6scrIFc0O3bANdfAfvtZy4RLAqpWf6VpU5g40YtyudDE0vSUqaq/i0/rLdNeesnKhr/5pi117BLc0qVw8cVWd6VzZ0sYBx0UdlQuRcWSKOaIyDlAORFpCFwLTItvWK4wNm60SrAdO8KZZ4YdjSsRGRkwa5aNarrkEi/i50IVy/++a7D1sv8C/g8rN943jjG5Qrr7btiwwYbD+oVfApszB+6/335u3hyWL4c+fTxJuNDF8j+wsaoOUNXDg9vtQe0nVwbMm2ezr/v0scm5LgFt22bZvnVrePxx+DUoeeZtiK6MiCVRPCYiC0RkkIg0i3tELmaqcN11ViB00KCwo3FFMmOGzay+6y444wwv4ufKpFhWuDtaRPbGFjF6QUSqA/9T1XvjHp2LaswYW7DsySd9QExC2rwZTjgBKle2dSNOOSXsiJzLU74LF+W5sUhz4GbgLFWtELeoovCFi8zWrTZqcrfdbLSTl+pIIDNnWjPTLrvY5JfmzaFGjbCjckmuOAsXxTLhromI3CUic4CnsRFPdYuyM1dyHnvMRlB6PacE8vvvVqXx8MNzivgddZQnCVfmxXKK+S8wAjheVXNXf3UhSE+H++6zdWm6dAk7GheT99+3FaTWrIEbb4TTTw87IudiFksfxRGlEYiL3S232Exsr+eUIG66yX5ZzZvD6NF2ReFcAsk3UYjIW6p6pojMZucFh2Jd4c7FwRdfwP/9H9x+OzRoEHY0Ll+qls133dVqM1Wvbhm+Qihde84VS76d2SLyL1VdLSL75/V8jIsXlbhU7szesQPatLFh9gsW2IJmrgxKT4crrrCV5u67L+xonAPi1JmtqquDH6/MY3W7K4uyM1c8//2vrXr50EOeJMqkrCwrudG0KXz6Key9d9gROVciYplwd1wej3Ur6UBcdL/9ZssPHHUU9O4ddjTuH5YsgWOOsQ7rNm1g9mwr5+tcEojWR3EFduVwgIjMiniqGvBFvANzO7v7bli3DiZM8HpOZdLmzTaretgwuOgi/yW5pBJt1NP/AeOAB4D+EY9vUtUNcY3K7WT+fKvndOmlcOihYUfj/jZ7tk2Pv/12G9H08882y9q5JBOt6UlVdRlwFbAp4oaI7BH/0BzY4Jm+fa1P4l4vmlI2/PUX3HGHza5+8smcIn6eJFySKuiK4mTgG2x4bOS1tAIHxDEuF3j/fVvc7IknoHbtsKNxTJ9uCwrNmwfnnWfVXmvVCjsq5+KqULWeyoJUGh67dSs0awaVKlk9p/Llw44oxW3eDPXq2eXd889DNx/T4RJHcYbHFjgzW0SOBL5X1c0i8m+gNfCEqi4vyg5d7B5/3AbTTJzoSSJUX31ls6mrVLFLvObNrba7cykiluGxQ4EtItISqxz7M/BaXKNyrFxpc7VOPRWOy2uAsou/336zZUiPOCKniF/79p4kXMqJJVFkqrVP9QCGqOoQbIisi6P+/SEzEx59NOxIUtTo0TZxbvhwK71xxhlhR+RcaGJJFJtE5FbgPOBDESkHeENIHH35pX2BveEGOMCHDJS+fv3gtNNspbmvvoLBg31Ek0tpsZQZPws4B7hIVdeISD3g4fiGlbqysuDaa2GffeDWW8OOJoVEFvE78UQbyXTzzd455BwxXFGo6hrgDaCGiJwMbFXVV+MeWYoaPtwWQHv4YahaNexoUsTy5XDSSXDnnXb/2GNhwABPEs4FYlnh7kzga+AMbN3sr0TEV12Jg99/t6uII4+Es88OO5oUkJUFzz5rY5A/+8wu45xz/xBL09MA4HBV/RVARGoDHwPvxDOwVHTPPbB2LYwb56WC4m7xYqvJNHWqDSt74QWoXz/sqJwrk2JJFLtkJ4nAemLrBHeFsGCBVYO4+GKrDOHibOtWWLTIarf/5z+emZ2LIpZEMV5EJmDrZoN1bo+NX0ipRxWuv97mc/k6N3H0/fdWxO/OO+GQQ2DZMpv27pyLKpbO7JuA54EWQEvgBVW9Jd6BpZIPP4Tx4+38tddeYUeThLZutc7ptDQYOjSniJ8nCediEm09iobAI8CBwGzgRlVdWVqBpYq//rLqsE2awNVXhx1NEpo2zdrzFiywJqbHHoM9vPixc4URrenpZeBVYApwCvAU0LM0gkolTzwBP/1kCxL5aMwStnkznHKKjTMePx66dg07IucSUrREUU1VXwx+Xigi35ZGQKlk9WpbY6J7dzj++LCjSSJffglt21qnzwcfWH+E12dyrsii9VFUEpFDRaS1iLQGKue6XyAROUFEForIYhHpH2W7w0VkR6rNz+jfH7Zts9YQVwI2brQhr+3bw2tB3cp27TxJOFdM0a4oVgORp7A1EfcVOCbaGwc1oZ4BjgPSgRki8p6qzstjuweBCYULPbFNnw6vvmrJ4sADw44mCYwaBVddZRNRbr0Vzjor7IicSxr5JgpVPbqY790GWKyqSwBE5E2sAu28XNtdA4wEDi/m/hJGZD2nAQPCjiYJXH+9dfa0agVjx/rC4s6VsFjmURTVvsCKiPvpQNvIDURkX+A07Ook30QhIn2APgD16tUr8UBL2yuvwIwZ1jri9ZyKKLKI38kn27jiG2/0EQHOxUE8Z1jnNdU197qrTwC3qOqOaG+kqi+oapqqptVO8IWj//jDWkbatYNzzw07mgS1bBmccAIMHGj3u3Sxg+pJwrm4iOcVRTqwX8T9usCqXNukAW+KlU/YEzhRRDJVdXQc4wrVoEE23+uDD7xqRKFlZcEzz1hSELE1I5xzcRfLmtkCnAscoKr3BOtR7K2qXxfw0hlAQxFpAKwEemPrWvxNVRtE7Gc48EEyJ4mFC2HIELjwQpsk7Arhxx/twH3xhV1NPPcc7L9/2FE5lxJiaXp6FmgHZBe+3oSNZopKVTOBq7HRTPOBt1R1rohcLiKXFzHehHb99bZQ2v33hx1JAtq2zWYmvvqqdVh7knCu1MTS9NRWVVuLyHcAqrpRRCrE8uaqOpZcBQRV9bl8tr0glvdMVB9+aOXDH30U6tQJO5oE8d13VsTvrrtszYhly6BixbCjci7lxHJFsT2Y66Dw93oUWXGNKsls22ZXE40bez2nmGzdav0Qhx8Ozz9vcyPAk4RzIYklUTwJvAvsJSL3AZ8D3nhSCEOGWBP7E09AhZiuxVLY559Dy5YweDCcfz7MmwcJPtLNuURXYNOTqr4hIt8AXbAhr6eq6vy4R5YkVq+2letOPtn6YF0UGRnQowdUrw4TJ9rKc8650MUy6qkesAV4P/IxVV0ez8CSxa23Wilxr+cUxeefW32mqlWtM+eQQ3wmonNlSCxNTx8CHwT/fgIsAcbFM6hk8dVXNgu7Xz9o2DDsaMqg9eutealDh5wifkcc4UnCuTImlqan5pH3g8qxl8UtoiSRXc/pX//yek7/oArvvGM9+xs22Azr3r3Djso5l49Cz8xW1W9FJGUK+BXVa6/B11/bFYVXuc7l+uuth/+ww6wvomXLsCNyzkURSx9Fv4i7uwCtgbVxiygJbNpk5cPbtoV//zvsaMoIVcjMtHpM3btb6dx+/ayon3OuTIvlrzTy+3Am1lcxMj7hJId774U1a2yu2C7xLLuYKJYuhT597Api8GA45hi7OecSQtREEUy0q6qqN5VSPAlv0SJ4/HErS9SmTdjRhGzHDnj6abjtNihXDs44I+yInHNFkG+iEJFdVTUz1mVPnenXDypV8npOLFoEF1xg61d362YzrPfbr8CXOefKnmhXFF9j/RHfi8h7wNvA5uwnVXVUnGNLOOPG2TSAhx+GvfcOO5qQZWbCzz/D66/DOed4TXXnElgsfRR7AOuxVegUm52tgCeKCNu2Qd++0KiRDYtNSTNnWsfMoEHQtCksWeL1mZxLAtESxV7BiKc55CSIbLlXqkt5Tz1lrS0ffpiC9Zz+/BPuvNNK4+69t2XK2rU9STiXJKKNySkHVA1u1SJ+zr65wJo1cPfdcOKJdkspn30GLVpYe9vFF8PcuV7Ez7kkE+2KYrWq3lNqkSSw226zytiPPx52JKUsIwN69oSaNeGTT3zIq3NJKtoVhfc+xmDGDPjvf3P6J1LC1KlWo6RqVevBnzXLk4RzSSxaouhSalEkqOx6TnXqwO23hx1NKVi3zqaad+yYU8SvTRuoUiXcuJxzcZVv05OqbijNQBLRG2/A9Ol2RVG9etjRxJEqvPUWXHMNbNxoHddexM+5lOGFdopo0ya4+Wb7Qn3++WFHE2fXXWfDug4/3Poimjcv+DXOuaThiaKI7rvPRjuNHp2k9ZxUYft2G+t72mmw//7WEVOuXNiROedKWTKe4uJu8WIb4fSf/1iF2KTz00/QpUtOx8vRR8MNN3iScC5FeaIogn797Iv2Aw+EHUkJ27HD1mxt3hy++QYaNw47IudcGeBNT4U0fjy8/z48+KCtXpc0FiywS6Svv4ZTToGhQ2HffcOOyjlXBniiKITsek4HHWT9u0klKwtWrYIRI+Css7yIn3Pub54oCuHpp2HhQruiSIoyRl9/bUX87rvPivj99FMKFqpyzhXE+yhi9MsvVs+pWzc46aSwoymmLVvgxhuhXTtb1HttsLKtJwnnXB48UcRowAA7vz7+eIK3ykyaZJ3Vjz4Kl17qRfyccwXypqcYzJwJL79so50SeiBQRoYtR1qzpiWMzp3Djsg5lwD8iqIAqjnLKwwcGHY0RTR58j+L+HmScM7FyBNFAd54w5Z9fuABqFEj7GgKae1aOPtsmzD3+uv22OGHw267hRuXcy6heNNTFBkZcMstkJYGF1wQdjSFoGrDXK+91opSDRrkRfycc0XmiSKK+++3qQUjRyZYPadrroFnnoEjjoCXXrKhr845V0SeKPLx0082MOi88+x8W+ZlZUFmpg1xPf10mxV4zTVen8k5V2xx/Z4sIieIyEIRWSwi/fN4/lwRmRXcpolIy3jGUxj9+kH58jB4cNiRxODHH22FuQED7H7nzl7p1TlXYuKWKESkHPAM0A1oCpwtIrnbQJYCnVS1BTAIeCFe8RTGxInw3ntWPHWffcKOJorMTHjkEWjRAr7/Hpo0CTsi51wSimfTUxtgsaouARCRN4EewLzsDVR1WsT204G6cYwnJtu325fxAw+E668PO5oo5s+3FZNmzoQePeDZZ8t4VnPOJap4Jop9gRUR99OBaKs3XAyMy+sJEekD9AGoV69eScWXp2eesXPwe+8lQD2nX36B//3PJtEl9HRx51xZFs8+irzOXJrnhiJHY4nilryeV9UXVDVNVdNqx7HcxNq1cNdd0LUrnHxy3HZTdNOnw6232s9NmliP+5lnepJwzsVVPBNFOrBfxP26wKrcG4lIC2AY0ENV18cxngINGACbN8MTT5Sxc+/mzdYO1r69zQDMLuJXvny4cTnnUkI8E8UMoKGINBCRCkBv4L3IDUSkHjAKOE9VF8UxlgJ98w0MG2YjSg8+OMxIcvn4YzjkEMteV17pRfycc6Uubn0UqpopIlcDE4BywMuqOldELg+efw64A6gFPCv2FT5TVdPiFVP+sdpCRHvuCXfcUdp7jyIjw2ZU77EHTJkCHTqEHZFzLgXFdcKdqo4FxuZ67LmIny8BLolnDLEYMQK++MKuKGrWDDsa4NNPoVMnK+I3YYLNrK5cOeyonHMpKpEKU8TF5s1w881w2GFw4YUhB/PLL9Y53aVLThG/ww7zJOGcC1XKl/B44AFYuRLeeivEek6qlhj69rXmpvvug3POCSkY55zbWUoniiVLbGLzuefagKLQXHUVDB1qS5O+9JLPsHbOlSkpnShuuAF23RUefDCEnWdl2TTwihXhrLMsOVx5pddncs6VOSnbR/HxxzB6NNx2G+y7bynvfOFC66zOLuLXqZNXenXOlVkpmSi2b7fhsAccYFViS3XHgwdDy5YwZw40b16KO3fOuaJJyaanoUNh3jy7oqhUqZR2OneuLW7x3XfQs6cVldp771LauXPOFV3KJYq1a21S3XHHQffupbjjcuVgwwZ45x3o1asUd+ycc8WTck1Pt99uI1BLpZ7TtGm26DZYXZDFiz1JOOcSTkoliu++gxdfhKuvjvMy0hkZcO21cNRRVgZ83Tp7fNeUu4BzziWBlEkUqnburlXLSonHzcSJVsTv6actI82ZY0WknHMuQaXMV9z//Q8+/xxeeCGO9ZwyMmz2Xq1aMHUqHHlknHbknHOlJyWuKDZvhhtvhEMPhYsuisMOPvoIduywIn4TJ9r61Z4knHNJIiUSxeDBVs/pySdLeE7b6tXWOX388bagEFg2KrUxt845F39JnyiWLoWHH4azz7a+5RKhCsOHW4/4hx9aJvIifs65JJX0fRQ33mhXEQ89VIJvesUV8PzzlnmGDYPGjUvwzZ1LHtu3byc9PZ2tW7eGHUrKqFSpEnXr1qV8CS6VnNSJ4pNPYNQouPdeqFu3mG8WWcTvnHOgRQu4/PIQa5M7V/alp6dTrVo16tevj5SpheiTk6qyfv160tPTadCgQYm9b9Ke5TIzrZ5TgwZWJbZY5s+3ZUhvu83ud+xolV49STgX1datW6lVq5YniVIiItSqVavEr+CS9kw3dKiVV3r00WL0LW/fDvffD61awYIF1lHtnCsUTxKlKx7HOymbntats3pOXbrAqacW8U3mzoV//9uGup5xBjz1FNSpU4JROudcYkjKK4qBA2HTJhgypBj1nHbdFX7/3To53nrLk4RzCezdd99FRFiwYMHfj02ePJmTTz55p+0uuOAC3nnnHcA64vv370/Dhg055JBDaNOmDePGjSt2LA888AAHHXQQjRs3ZsKECflu99RTT9G4cWOaNWvGzTffDMC2bdu48MILad68OS1btmTy5MnFjicWSXdF8cMPNvv66quhWbNCvnjqVBgzxtZHbdwYFi3y+kzOJYERI0Zw1FFH8eabb3JXjDV8Bg4cyOrVq5kzZw4VK1bkl19+4bPPPitWHPPmzePNN99k7ty5rFq1imOPPZZFixZRLtcEr0mTJjFmzBhmzZpFxYoV+fXXXwF48cUXAZg9eza//vor3bp1Y8aMGewS5/7SpDoLZtdz2n33QtZz2rQJ+veHZ5+13u/+/a0+kycJ50pM377WkluSWrWyStDRZGRk8MUXXzBp0iS6d+8eU6LYsmULL774IkuXLqVixYoA1KlThzPPPLNY8Y4ZM4bevXtTsWJFGjRowEEHHcTXX39Nu3btdtpu6NCh9O/f/+9977XXXoAlmi5duvz9WM2aNZk5cyZt2rQpVlwFSaqmp7fegilT4L77LFnEZNw4u/QYOtT+J8+e7UX8nEsio0eP5oQTTqBRo0bssccefPvttwW+ZvHixdSrV4/q1asXuO31119Pq1at/nEbPHjwP7ZduXIl++2339/369aty8qVK/+x3aJFi5g6dSpt27alU6dOzJgxA4CWLVsyZswYMjMzWbp0Kd988w0rVqwoMMbiSpqvzFu2wE032TeMSy6J8UWbNsH558Nee9naEUccEc8QnUtpBX3zj5cRI0bQt29fAHr37s2IESNo3bp1vqODCjtq6PHHH495W1WNaX+ZmZls3LiR6dOnM2PGDM4880yWLFnCRRddxPz580lLS2P//fenffv27FoKLR9JkygefBBWrLCSS1HrOanChAm2xF21avDxx7aoUHCJ55xLHuvXr+fTTz9lzpw5iAg7duxARHjooYeoVasWGzdu3Gn7DRs2sOeee3LQQQexfPlyNm3aRLVq1aLu4/rrr2fSpEn/eLx37970799/p8fq1q270xVAeno6++yzzz9eW7duXXr27ImI0KZNG3bZZRfWrVtH7dq1d0pM7du3p2HDhjEdi2JR1YS6HXbYYZrbsmWqlSqpnnXWP57a2apVqqeeqgqqr7xSwMbOueKaN29eqPt/7rnntE+fPjs91rFjR50yZYpu3bpV69ev/3eMy5Yt03r16ulvv/2mqqo33XSTXnDBBfrXX3+pquqqVav0tddeK1Y8c+bM0RYtWujWrVt1yZIl2qBBA83MzPzHdkOHDtWBAweqqurChQu1bt26mpWVpZs3b9aMjAxVVZ04caJ26NAhz/3kddyBmVrE827oJ/7C3vJKFKefrlq5sury5XkeM9WsLNWXXlKtUcMyykMPqW7fns/GzrmSEnai6NSpk44bN26nx4YMGaKXX365qqp+/vnn2rZtW23ZsqWmpaXpxIkT/97ur7/+0ptuukkPPPBAbdasmbZp00bHjx9f7JjuvfdePeCAA7RRo0Y6duzYvx+/+OKLdcaMGX/v+9xzz9VmzZrpoYceqp988omqqi5dulQbNWqkBx98sHbp0kWXLVuW5z5KOlGI5tFmVpalpaXpzJkz/74/aRIccwzcc4/Nn8jTZZfZmNmOHa2IX2lcqjnnmD9/Pk2aNAk7jJST13EXkW9UNa0o75fQfRSZmTYctn59qxK7kx07rARHpUo2w/rQQ6FPH6/P5JxzhZTQZ83nn7clqR95BCpXjnhi7lxbYS67iF+HDl7p1Tnniihhz5zr11tT09FHQ8+ewYPbtsGgQXb1sHgxHH54qDE65/IeEuriJx7HO2Gbnu64A/74I6Ke0+zZcO659m/v3rbuae3aYYfpXEqrVKkS69ev91LjpUSD9SgqlfByzAmZKH74AZ57zpaEaN48eLBCBZt1N2YMdO8eanzOOVO3bl3S09NZu3Zt2KGkjOwV7kpSQo56qlp1JnPmwJL/fkb1ye/ZohNgHdhRZ9s551xqKs6op7j2UYjICSKyUEQWi0j/PJ4XEXkyeH6WiLQu6D03boRvP/uDz5peQfXunWH0aFuAAjxJOOdcHMQtUYhIOeAZoBvQFDhbRJrm2qwb0DC49QGGFvS+vy//nUXlm9H0ixegXz8v4uecc3EWzyuKNsBiVV2iqtuAN4EeubbpAbwaTBycDtQUkX9Fe9O6mcuotm8NZNo0a3Labbf4RO+ccw6Ib2f2vkBk/dt0oG0M2+wLrI7cSET6YFccAH9VXTZ3jld6BWBPYF3YQZQRfixy+LHI4cciR+OivjCeiSKvsXC5e85j2QZVfQF4AUBEZha1QybZ+LHI4ccihx+LHH4scojIzIK3yls8m57Sgf0i7tcFVhVhG+eccyGKZ6KYATQUkQYiUgHoDbyXa5v3gPOD0U9HAL+r6urcb+Sccy48cWt6UtVMEbkamACUA15W1bkicnnw/HPAWOBEYDGwBbgwhrd+IU4hJyI/Fjn8WOTwY5HDj0WOIh+LhJtw55xzrnQlbFFA55xzpcMThXPOuajKbKKIR/mPRBXDsTg3OAazRGSaiLQMI87SUNCxiNjucBHZISKnl2Z8pSmWYyEinUXkexGZKyKflXaMpSWGv5EaIvK+iPwQHItY+kMTjoi8LCK/isicfJ4v2nmzqGuoxvOGdX7/BBwAVAB+AJrm2uZEYBw2F+MI4Kuw4w7xWLQHdg9+7pbKxyJiu0+xwRKnhx13iP8vagLzgHrB/b3CjjvEY3Eb8GDwc21gA1Ah7NjjcCw6Aq2BOfk8X6TzZlm9oohL+Y8EVeCxUNVpqroxuDsdm4+SjGL5fwFwDTAS+LU0gytlsRyLc4BRqrocQFWT9XjEciwUqCa2KEZVLFFklm6Y8aeqU7DPlp8inTfLaqLIr7RHYbdJBoX9nBdj3xiSUYHHQkT2BU4DnivFuMIQy/+LRsDuIjJZRL4RkfNLLbrSFcuxeBpogk3onQ1cp6pZpRNemVKk82ZZXbioxMp/JIGYP6eIHI0liqPiGlF4YjkWTwC3qOqOJF9RLZZjsStwGNAFqAx8KSLTVXVRvIMrZbEci67A98AxwIHARyIyVVX/iHNsZU2RzptlNVF4+Y8cMX1OEWkBDAO6qer6UoqttMVyLNKAN4MksSdwoohkquroUomw9MT6N7JOVTcDm0VkCtASSLZEEcuxuBAYrNZQv1hElgIHA1+XTohlRpHOm2W16cnLf+Qo8FiISD1gFHBeEn5bjFTgsVDVBqpaX1XrA+8AVyZhkoDY/kbGAB1EZFcR2Q2r3jy/lOMsDbEci+XYlRUiUgerpLqkVKMsG4p03iyTVxQav/IfCSfGY3EHUAt4NvgmnalJWDEzxmOREmI5Fqo6X0TGA7OALGCYquY5bDKRxfj/YhAwXERmY80vt6hq0pUfF5ERQGdgTxFJB+4EykPxzptewsM551xUZbXpyTnnXBnhicI551xUniicc85F5YnCOedcVJ4onHPOReWJwpVJQeXX7yNu9aNsm1EC+xsuIkuDfX0rIu2K8B7DRKRp8PNtuZ6bVtwYg/fJPi5zgmqoNQvYvpWInFgS+3apy4fHujJJRDJUtWpJbxvlPYYDH6jqOyJyPPCIqrYoxvsVO6aC3ldEXgEWqep9Uba/AEhT1atLOhaXOvyKwiUEEakqIp8E3/Zni8g/qsaKyL9EZErEN+4OwePHi8iXwWvfFpGCTuBTgIOC1/YL3muOiPQNHqsiIh8GaxvMEZGzgscni0iaiAwGKgdxvBE8lxH8+7/Ib/jBlUwvESknIg+LyAyxdQIui+GwfElQ0E1E2oitRfJd8G/jYJbyPcBZQSxnBbG/HOznu7yOo3P/EHb9dL/5La8bsAMr4vY98C5WRaB68Nye2MzS7CvijODfG4ABwc/lgGrBtlOAKsHjtwB35LG/4QRrVwBnAF9hBfVmA1Ww0tRzgUOBXsCLEa+tEfw7Gfv2/ndMEdtkx3ga8ErwcwWskmdloA9we/B4RWAm0CCPODMiPt/bwAnB/erArsHPxwIjg58vAJ6OeP39wL+Dn2tidZ+qhP379lvZvpXJEh7OAX+qaqvsOyJSHrhfRDpi5Sj2BeoAayJeMwN4Odh2tKp+LyKdgKbAF0F5kwrYN/G8PCwitwNrsSq8XYB31YrqISKjgA7AeOAREXkQa66aWojPNQ54UkQqAicAU1T1z6C5q4XkrMhXA2gILM31+soi8j1QH/gG+Chi+1dEpCFWDbR8Pvs/HuguIjcG9ysB9UjOGlCuhHiicIniXGxlssNUdbuILMNOcn9T1SlBIjkJeE1EHgY2Ah+p6tkx7OMmVX0n+46IHJvXRqq6SEQOw2rmPCAiE1X1nlg+hKpuFZHJWNnrs4AR2bsDrlHVCQW8xZ+q2kpEagAfAFcBT2K1jCap6mlBx//kfF4vQC9VXRhLvM6B91G4xFED+DVIEkcD++feQET2D7Z5EXgJWxJyOnCkiGT3OewmIo1i3OcU4NTgNVWwZqOpIrIPsEVVXwceCfaT2/bgyiYvb2LF2DpghewI/r0i+zUi0ijYZ55U9XfgWuDG4DU1gJXB0xdEbLoJa4LLNgG4RoLLKxE5NL99OJfNE4VLFG8AaSIyE7u6WJDHNp2B70XkO6wfYYiqrsVOnCNEZBaWOA6OZYeq+i3Wd/E11mcxTFW/A5oDXwdNQAOAe/N4+QvArOzO7FwmYmsbf6y2dCfYWiLzgG9FZA7wPAVc8Qex/ICV1X4Iu7r5Auu/yDYJaJrdmY1deZQPYpsT3HcuKh8e65xzLiq/onDOOReVJwrnnHNReaJwzjkXlScK55xzUXmicM45F5UnCuecc1F5onDOORfV/wM8VLZzxwZbfAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.68      0.77      0.72      5257\n",
      "           1       0.72      0.62      0.66      5055\n",
      "\n",
      "    accuracy                           0.69     10312\n",
      "   macro avg       0.70      0.69      0.69     10312\n",
      "weighted avg       0.70      0.69      0.69     10312\n",
      "\n",
      "\n",
      "Confusion Matrix: \n",
      " [[4025 1232]\n",
      " [1925 3130]]\n",
      "\n",
      "End of validation Result\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/bettyau/opt/anaconda3/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function plot_confusion_matrix is deprecated; Function `plot_confusion_matrix` is deprecated in 1.0 and will be removed in 1.2. Use one of the class methods: ConfusionMatrixDisplay.from_predictions or ConfusionMatrixDisplay.from_estimator.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAT8AAAEGCAYAAAAT05LOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcQ0lEQVR4nO3de5hdVZnn8e/vVFUq95ALCSEJEDSKEeUWI5FnbEAlsdsZ0JHp4IU8LYrQ2OjY2g2t7Y0nDqN2a6OARmUIo5AJCkNUINBRGnECJEEkJgiJoCEkEBIuCblV1al3/ti7wiGpOnV2UqfOqbN/n+fZT+29zr6sk0rerLXXXu9WRGBmljeFWlfAzKwWHPzMLJcc/Mwslxz8zCyXHPzMLJeaa12BUuPGNMUxU1pqXQ3L4PFHhta6CpbBHnbSFnt1KOeYfcaw2PZ8saJ9Vz2yd2lEzDmU61VLXQW/Y6a08ODSKbWuhmUw+8gTa10Fy+CBWHbI59j6fJEHlk6uaN+WiX8cd8gXrJK6Cn5mNhAExeisdSUOmYOfmWUSQCcDf3KEg5+ZZdaJW35mljNB0O5ur5nlTQBFd3vNLI98z8/McieAYgNkg3LwM7PMBv4dPwc/M8soCN/zM7P8iYD2gR/7HPzMLCtR5JCmB9cFBz8zyySATrf8zCyP3PIzs9xJHnJ28DOznAmgPQZ+HmQHPzPLJBDFBkgC7+BnZpl1hru9ZpYzvudnZjklir7nZ2Z5k2RydvAzs5yJEG3RVOtqHDIHPzPLrNP3/Mwsb5IBD3d7zSx3POBhZjnkAQ8zy62iH3I2s7wJRHsM/NAx8L+BmfUrD3iYWS4FcrfXzPKpEQY8Bv43MLN+FQHFKFS09EbSnyStlvSwpJVp2RhJd0tal/4cXbL/5ZLWS3pM0uyS8lPS86yXdJWkXpumDn5mlkky4NFU0VKhMyLixIiYkW5fBiyLiGnAsnQbSdOBucAbgTnANZK6LnItcCEwLV3m9HZRBz8zy6xIoaLlIJ0NLEzXFwLnlJQvioi9EfEksB6YKWkiMDIilkdEADeUHNMjBz8zyyQQnVHZAoyTtLJkufCA08FdklaVfDYhIjYDpD/Hp+WTgKdKjt2Ylk1K1/cvL8sDHmaWWYZW3daS7mx3TouITZLGA3dL+kOZfbu7jxdlysty8DOzTJL39vZNpzEiNqU/t0i6FZgJPCtpYkRsTru0W9LdNwJTSg6fDGxKyyd3U16Wu71mlpEoVriUPYs0TNKIrnXgLOD3wBJgXrrbPOC2dH0JMFdSq6SpJAMbD6Zd4x2STk1Hec8vOaZHbvmZWSbJqyv7JJnpBODW9KmUZuDGiLhT0gpgsaQLgA3AuQARsUbSYmAt0AFcEhHF9FwXA9cDQ4A70qUsBz8zyyRCfdLtjYgngBO6Kd8GvKOHY+YD87spXwkcn+X6Dn5mlpnz+ZlZ7iT5/Dy318xyx5mczSyHkkdd3PIzs5zpmts70Dn4mVlmjZDSysHPzDJJUlq522tmOeR7fmaWO0lWF3d7zSxnkultDn65df7M6QwZXqRQgKbm4Dt3Ps4f1wzm25dNYffOAhMmt/GPV/+ZYSM6eeapQXzsL45j8rF7ATjulJ188n8m6cfWPTKEb3zqKPbuKTDzzO1cfMXT9J6A26plxunbueiKTTQVgjtuGsPi70yodZXqkFt+vZI0B/g3oAn4QURcWc3r9bev3byeUWOL+7a/9Zmj+NgXnubNs3ay9KYx/OTa8cz7h2cAmHj0Xq7998cOOMdVl03mk197ijecsovPf+hYVv5qBG85c0e/fQd7RaEQXPLVp7l87rFs3dzCt29fx/1LR7Fh3eBaV63uNMIMj6qF7zS3/tXAu4HpwHlpDv6GtfGPrbzp1J0AnPT2Hdz3i8PK7r/t2WZ27Whi+oxdSPDO9z/P/7tzVD/U1Lrz+pN2selPg3hmQysd7QXuue0wZs1+qdbVqjtdo72VLPWsmm3XmcD6iHgiItqARSQ5+BuDgn867zVcMvt13P6jsQAc/fo9LF86EoBf//wwntvUsm/3ZzYM4m/f9To+877XsvqBYQBse6aFcRPb9+0z7sh2tj7TgtXG2CPaeW7ToH3bWze/+vdjr+iMQkVLPatmt7e7fPtv3X+nNG//hQBHTRo4tyC/eds6xh7RwYtbm7ls7muY8to9fPpfN3DtP0/ix988gllnvUTzoCST9pjx7fxoxVpGjimy7pEhfOlvprLgnj8Q3STaru//Kxtbd/dau/sd5V3XOzwGumpGm4ry6kfEAmABwIwTBg+Yv2pjj+gA4LBxHZw25yX+8NuhnHvxc/yPRU8ASRf4gWVJK3BQazCoNbk3OO3NuznymDaefqKVcRPb2br5lZbe1k0tjD3CLY1a2bq5hcOPbNu3PW5iO9vcEj9AAB113qqrRDW/QU/59ge8PbsK7Hq5sG991X+M4Jjj9vDi1uT/ks5OuPHfJvCeD28D4MVtTRTTcZHNfx7E008O4oij2hg7oYOhwzt5dNVQIuDffzLG95hq6LGHhzJpahsTpuyluaWT089+kfvv8j3Y7rjbW94KYFqaa/9pkpcNf6CK1+s3LzzXzJcvmApAsQPOeO+LvOWMHdz6g3H87PpxAJz27pc4a+7zAKy+fzg3fP0ImpqhqRBceuVGRo5OouHfXfkU3/jUUbTtKTDjjO0e6a2hzqK4+nOT+OqNT1BogrsWjeHPj3uk9wDRGN1eRRVvakj6S+BbJI+6XJemoO7RjBMGx4NLp5TbxerM7CNPrHUVLIMHYhnb4/lDilyjjxsfZ173/or2veW0a1f18urKmqnqCENE3A7cXs1rmFn/a4SW38AZXjWzuuBkpmaWS4Ho6KzvwYxKOPiZWWaNML3Nwc/Msgl3e80sh3zPz8xyy8HPzHInEMUGGPAY+N/AzPpdJ6poqYSkJkm/lfTzdHuMpLslrUt/ji7Z93JJ6yU9Jml2Sfkpklann10l9Z4S2MHPzDKJdMCjkqVCnwQeLdm+DFgWEdOAZek2aT7QucAbgTnANWneUIBrSbJDTUuXOb1d1MHPzDKLUEVLbyRNBv4K+EFJ8dnAwnR9IXBOSfmiiNgbEU8C64GZkiYCIyNieSTzdW8oOaZHvudnZhn1aWKDbwH/AIwoKZsQEZsBImKzpPFp+STg/pL9NqZl7en6/uVlueVnZpllaPmNk7SyZLmw6xyS3gNsiYhVFV62pxyhFeUO3Z9bfmaWSQQUOytu+W0tk9XlNOC/pNmfBgMjJf0IeFbSxLTVNxHYku7fU47Qjen6/uVlueVnZpn1xWhvRFweEZMj4hiSgYxfRsSHgCXAvHS3ecBt6foSYK6k1jRP6DTgwbSLvEPSqeko7/klx/TILT8zyySgosGMQ3AlsFjSBcAG4FyAiFgjaTGwFugALomIrnfHXgxcDwwB7kiXshz8zCyjvs/kHBH3APek69uAd/Sw33zggKTIEbESOD7LNR38zCyzRnirnYOfmWVW5W5vv3DwM7NMktHegT9W6uBnZpm522tmueRur5nlTlDZvN165+BnZpk1QK/Xwc/MMgqIyqe31S0HPzPLzN1eM8ulhh7tlfRtynTtI+LSqtTIzOpaP8zt7RflWn4r+60WZjZwBNDIwS8iFpZuSxoWETurXyUzq3eN0O3tdY6KpFmS1pK+YETSCZKuqXrNzKxOieisbKlnlUzQ+xYwG9gGEBG/A95exTqZWb2LCpc6VtFob0Q8td9rMIs97WtmDS4af8Cjy1OS3gaEpEHApbz6HZtmljd13qqrRCXd3ouAS0heBfc0cGK6bWa5pQqX+tVryy8itgIf7Ie6mNlA0VnrChy6SkZ7j5X0M0nPSdoi6TZJx/ZH5cysDnU951fJUscq6fbeCCwGJgJHAjcDN1WzUmZW3yIqW+pZJcFPEfG/I6IjXX5EQ9zuNLOD1siPukgak67+StJlwCKSr/PXwC/6oW5mVq/qvEtbiXIDHqtIgl3Xt/x4yWcBXFGtSplZfVOdt+oqUW5u79T+rIiZDRAhqPOpa5WoaIaHpOOB6cDgrrKIuKFalTKzOtfILb8ukr4InE4S/G4H3g3cBzj4meVVAwS/SkZ73w+8A3gmIv4GOAForWqtzKy+NcBobyXBb3dEdAIdkkYCWwA/5GyWV330kLOkwZIelPQ7SWskfTktHyPpbknr0p+jS465XNJ6SY9Jml1Sfoqk1elnV2m/TCzdqST4rZR0GPB9khHgh4AHKzjOzBqUorKlF3uBMyPiBJKcAXMknQpcBiyLiGnAsnQbSdOBucAbgTnANZKa0nNdC1wITEuXOb1dvJK5vX+brn5X0p3AyIh4pNevZWaNqw+6tBERwMvpZku6BHA2yTgDwELgHuAf0/JFEbEXeFLSemCmpD+RxKXlAJJuAM4B7ih3/XIPOZ9c7rOIeKj8VzOzRpXhOb9xkkrfB7QgIhbsO0/SclsFvBa4OiIekDQhIjYDRMRmSePT3ScB95eca2Na1p6u719eVrmW37+U+SyAM3s7eVaPPzKU2Uee2NentSoadd/YWlfBMmj6SFPvO1Wi8hkeWyNiRo+niSgCJ6a31m5NH6vrSXcXjTLlZZV7yPmM3g42sxyqwkhuRLwo6R6Se3XPSpqYtvomkgyyQtKim1Jy2GRgU1o+uZvysioZ8DAze7U+eNRF0uFpiw9JQ4B3An8AlgDz0t3mAbel60uAuZJaJU0lGdh4MO0i75B0ajrKe37JMT2qaIaHmVkp9U0y04nAwvS+XwFYHBE/l7QcWCzpAmADcC5ARKyRtBhYC3QAl6TdZoCLgeuBISQDHWUHO8DBz8wORt+M9j4CnNRN+TaSiRXdHTMfmN9N+Uqg3P3CA1SSyVmSPiTpC+n2UZJmZrmImTWOSp/xq/fML5Xc87sGmAWcl27vAK6uWo3MrP41QBr7Srq9b42IkyX9FiAiXkhfYWlmeVXnrbpKVBL82tMbkgHJCA0N8e4mMztY9d6lrUQlwe8q4FZgvKT5JFlePl/VWplZ/Yo+G+2tqUrm9v5Y0iqS0RcB50TEo1WvmZnVrzy0/CQdBewCflZaFhEbqlkxM6tjeQh+JG9q65o/NxiYCjxGklbGzHIoF/f8IuJNpdtptpeP97C7mdmAkHmGR0Q8JOkt1aiMmQ0QeWj5Sfp0yWYBOBl4rmo1MrP6lpfRXmBEyXoHyT3An1anOmY2IDR6yy99uHl4RHy2n+pjZnVONPiAh6TmiOgol87ezHKqkYMfyRvaTgYelrQEuBnY2fVhRNxS5bqZWT0aABlbKlHJPb8xwDaSd3Z0Pe8XgIOfWV41+IDH+HSk9/cc+JKQBoj7ZnawGr3l1wQM5yDfjGRmDawBIkC54Lc5Ir7SbzUxs4GhCm9vq4Vywa++07CaWc00ere32xeImJk1dMsvIp7vz4qY2cCRl+ltZmavyME9PzOzA4jGGBBw8DOz7NzyM7M8avTRXjOz7jn4mVnuNEgy00KtK2BmA1BUuJQhaYqkX0l6VNIaSZ9My8dIulvSuvTn6JJjLpe0XtJjkmaXlJ8iaXX62VWSeh2TcfAzs8wUlS296AD+PiLeAJwKXCJpOnAZsCwipgHL0m3Sz+aSvDlyDnBNmnAZ4FrgQmBauszp7eIOfmaWXR+0/CJic0Q8lK7vAB4FJgFnAwvT3RYC56TrZwOLImJvRDwJrAdmSpoIjIyI5RERwA0lx/TI9/zMLLMMo73jJK0s2V4QEQsOOJ90DHAS8AAwISI2QxIgJY1Pd5sE3F9y2Ma0rD1d37+8LAc/M8smyJLMdGtEzCi3g6ThJC9F+1REbC9zu66n9HoHlXbP3V4zy6TrBUZ9cM8PSS0kge/HJa/GeDbtypL+3JKWbwSmlBw+GdiUlk/uprwsBz8zy65vRnsF/BB4NCL+teSjJcC8dH0ecFtJ+VxJrZKmkgxsPJh2kXdIOjU95/klx/TI3V4zy0zRJ085nwZ8GFgt6eG07J+AK4HFki4ANgDnAkTEGkmLgbUkI8WXREQxPe5i4HpgCHBHupTl4Gdm2fRRVpeIuI+ecyR0m080IuYD87spXwkcn+X6Dn5mlpnn9ppZLjXC9DYHPzPLzi0/M8udCh9jqXcOfmaWnYOfmeVN10POA52Dn5llps6BH/0c/MwsG7+9zXoz4/TtXHTFJpoKwR03jWHxdybUukq5FsXg5Y++ROHwAsO+NpI9399F+31tICiMLjDkc8MpjEtmfBbXd7D76zuJnQEFGP79URCw65930Pl0EQqi5bQWBl88rMbfqjb8qEsZkq4D3gNsiYhMT143gkIhuOSrT3P53GPZurmFb9++jvuXjmLDusG1rlputd28h6ajm4hdSbOl9QODGfyxoQDsvXk3e//XLoZ8djjREey64mWGfn44TdOa6XypM/mX0g6t5w2h+eQWoj3Y+cnttC9vo2XWoBp+qxppgJZfNRMbXE8F2VQb1etP2sWmPw3imQ2tdLQXuOe2w5g1+6VaVyu3OrcUaV/exqD//Mp/Phr2yl//2MO+iVYdK9ppek0TTdOStkFhVAE1CQ0WzSe3JMe2iKbXNRPPNUAT6CD0VVaXWqpa8IuIe4Hnq3X+ejf2iHae2/RKi2Dr5hbGTWyvYY3ybfdVuxhy8bADZpLu+d4utr/vBdrv2kvrBUkrsPOpIgh2fno7Oz7yInt/vPuA88WOTtp/00bzKS39Uf36EkBEZUsdq3lKK0kXSlopaWU7e2tdnT7TXT7GOv+70LDaf9NG4TDRdNyBd3kGf3woI28ZTctZrbTdsicp7ICORzoY8oXhDL9mFO33ttGx8pX/uKIj2PWll2k9dzCFSU0HnDMP1FnZUs9qHvwiYkFEzIiIGS201ro6fWbr5hYOP7Jt3/a4ie1seyaHrYQ6UFzdTvtv2tn+/hfY9aUddKxqZ9dXdrxqn0HvGkT7PcnvS+MLNJ/YQuGwQtLVndVC8fGOffvu/tpOClOaaP1vQ/r1e9SLvkxmWks1D36N6rGHhzJpahsTpuyluaWT089+kfvvGlXrauXS4IuGMfLW0Yz8yWiGfmkEzae0MPQLIyg+Vdy3T/t9bRSOTlpxLTNbKP6xg9gTREfQ8dsOCsckn+1ZsIvYGQy+dGhNvktdqLTLW+ddHT/qUiWdRXH15ybx1RufoNAEdy0aw58f90hvPdnz3V10bihCAQoTCgz5bPLYikYWaP3rIbz80ZdA0DyrhZa3DaJzS5G9N+ymcHQTL38kGbxq/a+DXzWIkhf13qqrRDUfdbkJOJ3k7U0bgS9GxA+rdb16tOKXI1nxy5G1roaVaD65Zd+I7bD5I3rcb9DsVgbNfvVtmML4JkbdN7aq9RswHPx6FhHnVevcZlZbbvmZWf4EUBz40c/Bz8wyc8vPzPKpzkdyK+HgZ2aZueVnZvnjlFZmlkcC5AEPM8sj+Z6fmeWOu71mlk/1P2+3Ek5sYGaZ9VVWF0nXSdoi6fclZWMk3S1pXfpzdMlnl0taL+kxSbNLyk+RtDr97Cqpu6Ryr+bgZ2bZ9V1Wl+s5MOP7ZcCyiJgGLEu3kTQdmAu8MT3mGkldCRWvBS4EpqVLr1nkHfzMLJtIRnsrWXo9VfcZ388GFqbrC4FzSsoXRcTeiHgSWA/MlDQRGBkRyyMigBtKjumR7/mZWXaV3/IbJ2llyfaCiFjQyzETImIzQERsljQ+LZ8E3F+y38a0rD1d37+8LAc/M8ssw6MuWyNiRl9dtpuyKFNelru9ZpZddTM5P5t2ZUl/bknLNwJTSvabDGxKyyd3U16Wg5+ZZRNAZ4XLwVkCzEvX5wG3lZTPldQqaSrJwMaDaRd5h6RT01He80uO6ZG7vWaWiYg+m+HRXcZ34EpgsaQLgA3AuQARsUbSYmAt0AFcEhFdL2K5mGTkeAhwR7qU5eBnZtl19s17KctkfH9HD/vPB+Z3U74SOD7LtR38zCybrm7vAOfgZ2aZObGBmeWTg5+Z5U9jJDZw8DOzbPz2NjPLK9/zM7N8cvAzs9wJoNPBz8xyxwMeZpZXDn5mljsBFAf+FA8HPzPLKCAc/Mwsj9ztNbPc8WivmeWWW35mlksOfmaWOxFQLPa+X51z8DOz7NzyM7NccvAzs/wJj/aaWQ4FhB9yNrNc8vQ2M8udiD57dWUtOfiZWXYe8DCzPAq3/Mwsf5zM1MzyyIkNzCyPAogGmN5WqHUFzGyAiTSZaSVLLyTNkfSYpPWSLuuH2u/jlp+ZZRZ90O2V1ARcDbwL2AiskLQkItYe8skr4JafmWXXNy2/mcD6iHgiItqARcDZVa97SlFHozaSngP+XOt6VME4YGutK2GZNOrv7OiIOPxQTiDpTpI/n0oMBvaUbC+IiAXped4PzImIj6bbHwbeGhGfOJT6Vaquur2H+kupV5JWRsSMWtfDKuffWc8iYk4fnUrdnb6Pzt0rd3vNrFY2AlNKticDm/rr4g5+ZlYrK4BpkqZKGgTMBZb018XrqtvbwBbUugKWmX9nVRYRHZI+ASwFmoDrImJNf12/rgY8zMz6i7u9ZpZLDn5mlksOflVUy6k7dnAkXSdpi6Tf17ouVl0OflVSMnXn3cB04DxJ02tbK6vA9UBfPcdmdczBr3pqOnXHDk5E3As8X+t6WPU5+FXPJOCpku2NaZmZ1QEHv+qp6dQdMyvPwa96ajp1x8zKc/CrnppO3TGz8hz8qiQiOoCuqTuPAov7c+qOHRxJNwHLgddL2ijpglrXyarD09vMLJfc8jOzXHLwM7NccvAzs1xy8DOzXHLwM7NccvAbQCQVJT0s6feSbpY09BDOdX369iwk/aBc0gVJp0t620Fc40+SDnjLV0/l++3zcsZrfUnSZ7LW0fLLwW9g2R0RJ0bE8UAbcFHph2kmmcwi4qO9vCj6dCBz8DOrZw5+A9evgdemrbJfSboRWC2pSdLXJa2Q9IikjwMo8R1JayX9AhjfdSJJ90iaka7PkfSQpN9JWibpGJIg+9/TVud/knS4pJ+m11gh6bT02LGS7pL0W0nfo/v5za8i6f9KWiVpjaQL9/vsX9K6LJN0eFr2Gkl3psf8WtJxffKnabnjFxgNQJKaSfIE3pkWzQSOj4gn0wDyUkS8RVIr8BtJdwEnAa8H3gRMANYC1+133sOB7wNvT881JiKel/Rd4OWI+Ea6343ANyPiPklHkcxieQPwReC+iPiKpL8CXhXMevCR9BpDgBWSfhoR24BhwEMR8feSvpCe+xMkLxa6KCLWSXorcA1w5kH8MVrOOfgNLEMkPZyu/xr4IUl39MGIeDItPwt4c9f9PGAUMA14O3BTRBSBTZJ+2c35TwXu7TpXRPSU1+6dwHRpX8NupKQR6TXelx77C0kvVPCdLpX03nR9SlrXbUAn8H/S8h8Bt0gann7fm0uu3VrBNcwO4OA3sOyOiBNLC9IgsLO0CPi7iFi6335/Se8ptVTBPpDcLpkVEbu7qUvF8yUlnU4SSGdFxC5J9wCDe9g90uu+uP+fgdnB8D2/xrMUuFhSC4Ck10kaBtwLzE3vCU4Ezujm2OXAX0iamh47Ji3fAYwo2e8uki4o6X4npqv3Ah9My94NjO6lrqOAF9LAdxxJy7NLAehqvX6ApDu9HXhS0rnpNSTphF6uYdYtB7/G8wOS+3kPpS/h+R5JC/9WYB2wGrgW+I/9D4yI50ju090i6Xe80u38GfDergEP4FJgRjqgspZXRp2/DLxd0kMk3e8NvdT1TqBZ0iPAFcD9JZ/tBN4oaRXJPb2vpOUfBC5I67cGvxrADpKzuphZLrnlZ2a55OBnZrnk4GdmueTgZ2a55OBnZrnk4GdmueTgZ2a59P8BnygZt0KbWxEAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# classfication report\n",
    "clf_score(model2, X_train_sc, y_train, X_test_sc, y_test, train=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0ceeb736",
   "metadata": {},
   "source": [
    "In ANN there is no clear heuristic way to perform feature importance.\n",
    "\n",
    "There are two very popular alternative approaches:\n",
    "\n",
    "1. CW - Connection weight algorithm\n",
    "2. Garson's algorithm\n",
    "\n",
    "Both provide a calculated relative score of each feature's importance. There are many resources debating which is better, including more algorithms of the same family. We will explore these two approaches in the second half of the course. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc926ca9",
   "metadata": {},
   "source": [
    "### 5.4.3 Random Forest"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "aaf5ca8a",
   "metadata": {},
   "outputs": [],
   "source": [
    "randomforest = RandomForestClassifier(random_state = 5,oob_score = True) \n",
    "model3 = randomforest.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "c1fa6ffd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7523821069220512"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# cross-validation\n",
    "model3.oob_score_ "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "dc239355",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>predictor</th>\n",
       "      <th>feature importance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>timeOnIce</td>\n",
       "      <td>0.164544</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>shortHandedTimeOnIce</td>\n",
       "      <td>0.107938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>evenTimeOnIce</td>\n",
       "      <td>0.102960</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>powerPlayTimeOnIce</td>\n",
       "      <td>0.094662</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>goals</td>\n",
       "      <td>0.079308</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pim</td>\n",
       "      <td>0.072245</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>faceOffWinPercentage</td>\n",
       "      <td>0.043572</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>penalties</td>\n",
       "      <td>0.039125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>hits</td>\n",
       "      <td>0.033681</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>shots</td>\n",
       "      <td>0.032469</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>powerPlayGoals</td>\n",
       "      <td>0.029372</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>blockedShots</td>\n",
       "      <td>0.029076</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>missedShots</td>\n",
       "      <td>0.027490</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>giveaways</td>\n",
       "      <td>0.027040</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>takeaways</td>\n",
       "      <td>0.024540</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>goalieReplacement_No</td>\n",
       "      <td>0.020668</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>goalieReplacement_Yes</td>\n",
       "      <td>0.018583</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>startRinkSide_right</td>\n",
       "      <td>0.010247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>startRinkSide_left</td>\n",
       "      <td>0.010165</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>settledIn_OT</td>\n",
       "      <td>0.009382</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>settledIn_REG</td>\n",
       "      <td>0.008185</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>hoa_home</td>\n",
       "      <td>0.007403</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>hoa_away</td>\n",
       "      <td>0.007345</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                predictor  feature importance\n",
       "11              timeOnIce            0.164544\n",
       "13   shortHandedTimeOnIce            0.107938\n",
       "12          evenTimeOnIce            0.102960\n",
       "14     powerPlayTimeOnIce            0.094662\n",
       "4                   goals            0.079308\n",
       "0                     pim            0.072245\n",
       "2    faceOffWinPercentage            0.043572\n",
       "10              penalties            0.039125\n",
       "6                    hits            0.033681\n",
       "3                   shots            0.032469\n",
       "1          powerPlayGoals            0.029372\n",
       "7            blockedShots            0.029076\n",
       "9             missedShots            0.027490\n",
       "8               giveaways            0.027040\n",
       "5               takeaways            0.024540\n",
       "21   goalieReplacement_No            0.020668\n",
       "22  goalieReplacement_Yes            0.018583\n",
       "20    startRinkSide_right            0.010247\n",
       "19     startRinkSide_left            0.010165\n",
       "17           settledIn_OT            0.009382\n",
       "18          settledIn_REG            0.008185\n",
       "16               hoa_home            0.007403\n",
       "15               hoa_away            0.007345"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get importance\n",
    "importance = model3.feature_importances_\n",
    "\n",
    "# summarize feature importance\n",
    "pd.DataFrame(list(zip(X_train.columns, importance)), \n",
    "             columns = ['predictor','feature importance']).sort_values(\"feature importance\", ascending = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "dfb1c0f2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAASG0lEQVR4nO3df4hd533n8fdnxxHdJDVuyXRjJO1KWUTcYdmNzaC6mxLazXaR7FJlIQUZEhfDohVYtb0ktGr+yf6zkD/SNDUID9pYbU28FbtOuiuSoWpJE3YLsdH4B04UVeygdaup5GiKiR1qiKz6u3/cE3x7daV7RprRSM+8XzDonufHuc89XD7z6JnzI1WFJKld/2i9ByBJWlsGvSQ1zqCXpMYZ9JLUOINekhp323oPYJz3ve99tW3btvUehiTdMp5//vm/rarpcXU3ZdBv27aNhYWF9R6GJN0ykvzVlepcupGkxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMbdlFfGSjeDbQe/3qvdK5+7f41HIl0fZ/SS1LheQZ9kV5LTSRaTHBxTf1eSbyf5UZJPj9TdkeSZJH+Z5FSSn1+twUuSJpu4dJNkCjgE/DKwBJxIcqyqvjfU7DXgEeBjY3bxe8CfVNXHk2wC3n3do5Yk9dZnRr8TWKyqM1V1ETgK7BluUFUXquoE8NZweZLbgY8AT3btLlbVD1Zj4JKkfvoE/Wbg7ND2UlfWxweAZeD3k7yY5EtJ3jOuYZJ9SRaSLCwvL/fcvSRpkj5BnzFl1XP/twH3AE9U1d3A3wGXrfEDVNXhqpqtqtnp6bH3zpckXYM+Qb8EbB3a3gKc67n/JWCpqp7rtp9hEPySpBukT9CfAHYk2d79MXUvcKzPzqvqVeBskg92RR8FvneVLpKkVTbxrJuqupTkAHAcmAKOVNXJJPu7+rkk7wcWgNuBt5M8BsxU1RvAbwBPd78kzgAPrc1HkSSN0+vK2KqaB+ZHyuaGXr/KYElnXN+XgNlrH6Ik6Xp4ZawkNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXG9gj7JriSnkywmuezh3knuSvLtJD9K8ukx9VNJXkzytdUYtCSpv4lBn2QKOATsBmaAB5LMjDR7DXgE+PwVdvMocOo6xilJukZ9ZvQ7gcWqOlNVF4GjwJ7hBlV1oapOAG+Ndk6yBbgf+NIqjFeStEJ9gn4zcHZoe6kr6+uLwG8Cb6+gjyRplfQJ+owpqz47T/IrwIWqer5H231JFpIsLC8v99m9JKmHPkG/BGwd2t4CnOu5/w8Dv5rkFQZLPv8myZfHNayqw1U1W1Wz09PTPXcvSZqkT9CfAHYk2Z5kE7AXONZn51X121W1paq2df3+vKo+cc2jlSSt2G2TGlTVpSQHgOPAFHCkqk4m2d/VzyV5P7AA3A68neQxYKaq3li7oUuS+pgY9ABVNQ/Mj5TNDb1+lcGSztX28S3gWyseoSTpunhlrCQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDWuV9An2ZXkdJLFJAfH1N+V5NtJfpTk00PlW5N8M8mpJCeTPLqag5ckTTbxUYJJpoBDwC8DS8CJJMeq6ntDzV4DHgE+NtL9EvCpqnohyU8Czyf5s5G+kqQ11GdGvxNYrKozVXUROArsGW5QVReq6gTw1kj5+ap6oXv9Q+AUsHlVRi5J6qVP0G8Gzg5tL3ENYZ1kG3A38NwV6vclWUiysLy8vNLdS5KuoE/QZ0xZreRNkrwX+ArwWFW9Ma5NVR2uqtmqmp2enl7J7iVJV9En6JeArUPbW4Bzfd8gybsYhPzTVfXVlQ1PknS9+gT9CWBHku1JNgF7gWN9dp4kwJPAqar6wrUPU5J0rSaedVNVl5IcAI4DU8CRqjqZZH9XP5fk/cACcDvwdpLHgBngXwKfBL6T5KVul5+pqvlV/ySSpLEmBj1AF8zzI2VzQ69fZbCkM+ovGL/GL0m6QbwyVpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGtfrylhJa2fbwa/3bvvK5+5fw5GoVc7oJalxBr0kNc6gl6TGGfSS1DiDXpIa51k30iryDBrdjHrN6JPsSnI6yWKSg2Pq70ry7SQ/SvLplfSVJK2tiUGfZAo4BOxm8HjAB5LMjDR7DXgE+Pw19JUkraE+M/qdwGJVnamqi8BRYM9wg6q6UFUngLdW2leStLb6BP1m4OzQ9lJX1kfvvkn2JVlIsrC8vNxz95KkSfoE/biHe1fP/ffuW1WHq2q2qmanp6d77l6SNEmfoF8Ctg5tbwHO9dz/9fSVJK2CPkF/AtiRZHuSTcBe4FjP/V9PX0nSKph4Hn1VXUpyADgOTAFHqupkkv1d/VyS9wMLwO3A20keA2aq6o1xfdfos0iSxuh1wVRVzQPzI2VzQ69fZbAs06uvJOnG8RYIktQ4b4Eg3aL63m7BWy3IGb0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJalyvoE+yK8npJItJDo6pT5LHu/qXk9wzVPefkpxM8t0kf5TkJ1bzA0iSrm5i0CeZAg4Bu4EZ4IEkMyPNdgM7up99wBNd383AI8BsVf0LBo8T3Ltqo5ckTdRnRr8TWKyqM1V1ETgK7Blpswd4qgaeBe5IcmdXdxvwj5PcBrwbOLdKY5ck9dAn6DcDZ4e2l7qyiW2q6m+AzwN/DZwHXq+qPx33Jkn2JVlIsrC8vNx3/JKkCfo8SjBjyqpPmyQ/xWC2vx34AfA/knyiqr58WeOqw8BhgNnZ2dH9b2h9HxkHPjZO0uX6zOiXgK1D21u4fPnlSm3+LfD/qmq5qt4Cvgr862sfriRppfoE/QlgR5LtSTYx+GPqsZE2x4AHu7Nv7mWwRHOewZLNvUnenSTAR4FTqzh+SdIEE5duqupSkgPAcQZnzRypqpNJ9nf1c8A8cB+wCLwJPNTVPZfkGeAF4BLwIt3yjCTpxuizRk9VzTMI8+GyuaHXBTx8hb6fBT57HWOUJF0Hr4yVpMYZ9JLUOINekhpn0EtS43r9MfZW0vfiIi8skrRROKOXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIa1yvok+xKcjrJYpKDY+qT5PGu/uUk9wzV3ZHkmSR/meRUkp9fzQ8gSbq6iUGfZAo4BOwGZoAHksyMNNsN7Oh+9gFPDNX9HvAnVXUX8K/wmbGSdEP1mdHvBBar6kxVXQSOAntG2uwBnqqBZ4E7ktyZ5HbgI8CTAFV1sap+sHrDlyRN0ifoNwNnh7aXurI+bT4ALAO/n+TFJF9K8p5xb5JkX5KFJAvLy8u9P4Ak6er6BH3GlFXPNrcB9wBPVNXdwN8Bl63xA1TV4aqararZ6enpHsOSJPXRJ+iXgK1D21uAcz3bLAFLVfVcV/4Mg+CXJN0gfYL+BLAjyfYkm4C9wLGRNseAB7uzb+4FXq+q81X1KnA2yQe7dh8Fvrdag5ckTTbxUYJVdSnJAeA4MAUcqaqTSfZ39XPAPHAfsAi8CTw0tIvfAJ7ufkmcGamTJK2xXs+Mrap5BmE+XDY39LqAh6/Q9yVg9tqHKEm6Hl4ZK0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuN63Y9el9t28Ou9277yufvXcCSSdHW9ZvRJdiU5nWQxyWUP9+4eIfh4V/9ykntG6qeSvJjka6s1cElSPxODPskUcAjYDcwADySZGWm2G9jR/ewDnhipfxQ4dd2jlSStWJ8Z/U5gsarOVNVF4CiwZ6TNHuCpGngWuCPJnQBJtgD3A19axXFLknrqE/SbgbND20tdWd82XwR+E3j7am+SZF+ShSQLy8vLPYYlSeqjT9BnTFn1aZPkV4ALVfX8pDepqsNVNVtVs9PT0z2GJUnqo89ZN0vA1qHtLcC5nm0+DvxqkvuAnwBuT/LlqvrEtQ9ZWhnPkNJG12dGfwLYkWR7kk3AXuDYSJtjwIPd2Tf3Aq9X1fmq+u2q2lJV27p+f27IS9KNNXFGX1WXkhwAjgNTwJGqOplkf1c/B8wD9wGLwJvAQ2s3ZEnSSvS6YKqq5hmE+XDZ3NDrAh6esI9vAd9a8QglSdfFWyBIUuMMeklqnPe60T/gGSpSe5zRS1LjnNHfYM6YJd1ozuglqXEGvSQ1zqCXpMa5Ro/r5tfL4yfd3JzRS1LjnNHrluL/HqSVM+i1bvqGtoEtXR+XbiSpcQa9JDXOpZuGuTQiCZzRS1LzegV9kl1JTidZTHJwTH2SPN7Vv5zknq58a5JvJjmV5GSSR1f7A0iSrm5i0CeZAg4Bu4EZ4IEkMyPNdgM7up99wBNd+SXgU1X1s8C9wMNj+kqS1lCfGf1OYLGqzlTVReAosGekzR7gqRp4FrgjyZ3dA8JfAKiqHwKngM2rOH5J0gR9/hi7GTg7tL0E/FyPNpuB8z8uSLINuBt47loGKknr7Va9YK/PjD5jymolbZK8F/gK8FhVvTH2TZJ9SRaSLCwvL/cYliSpjz4z+iVg69D2FuBc3zZJ3sUg5J+uqq9e6U2q6jBwGGB2dnb0F4kk3ZJuhv8F9JnRnwB2JNmeZBOwFzg20uYY8GB39s29wOtVdT5JgCeBU1X1hVUduSSpl4kz+qq6lOQAcByYAo5U1ckk+7v6OWAeuA9YBN4EHuq6fxj4JPCdJC91ZZ+pqvlV/RSSpCvqdWVsF8zzI2VzQ68LeHhMv79g/Pq9JOkG8cpYSWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIa5xOmJN3SrvUWAxvpCWwGvaQ1sZGC9GZn0Eu6adwMNwBrkUEv6aoM31ufQS9tIIb2xuRZN5LUOINekhpn0EtS4wx6SWqcQS9JjesV9El2JTmdZDHJwTH1SfJ4V/9yknv69pUkra2JQZ9kCjgE7AZmgAeSzIw02w3s6H72AU+soK8kaQ31mdHvBBar6kxVXQSOAntG2uwBnqqBZ4E7ktzZs68kaQ1l8FzvqzRIPg7sqqr/0G1/Evi5qjow1OZrwOe6h4GT5BvAbwHbJvUd2sc+Bv8bAPggcPr6Pto/8D7gb1dxf7cyj8U7PBYDHod33MrH4p9V1fS4ij5XxmZM2ehvhyu16dN3UFh1GDjcYzwrlmShqmbXYt+3Go/FOzwWAx6Hd7R6LPoE/RKwdWh7C3CuZ5tNPfpKktZQnzX6E8COJNuTbAL2AsdG2hwDHuzOvrkXeL2qzvfsK0laQxNn9FV1KckB4DgwBRypqpNJ9nf1c8A8cB+wCLwJPHS1vmvySa5uTZaEblEei3d4LAY8Du9o8lhM/GOsJOnW5pWxktQ4g16SGtd80HsLhnckeSXJd5K8lGRhvcdzoyQ5kuRCku8Olf10kj9L8n+7f39qPcd4o1zhWPznJH/TfS9eSnLfeo7xRkiyNck3k5xKcjLJo115k9+LpoPeWzCM9UtV9aEWzxW+ij8Ado2UHQS+UVU7gG902xvBH3D5sQD43e578aGqmr/BY1oPl4BPVdXPAvcCD3fZ0OT3oumgx1swCKiq/w28NlK8B/jD7vUfAh+7kWNaL1c4FhtOVZ2vqhe61z8ETgGbafR70XrQbwbODm0vdWUbVQF/muT57pYTG9k/6a71oPv3Z9Z5POvtQHfn2SOtLFf0lWQbcDfwHI1+L1oP+t63YNggPlxV9zBYyno4yUfWe0C6KTwB/HPgQ8B54HfWdTQ3UJL3Al8BHquqN9Z7PGul9aDvc/uGDaOqznX/XgD+mMHS1kb1/e4Oq3T/Xljn8aybqvp+Vf19Vb0N/Fc2yPciybsYhPzTVfXVrrjJ70XrQe8tGDpJ3pPkJ3/8Gvh3wHev3qtpx4Bf717/OvC/1nEs6+rHwdb592yA70WSAE8Cp6rqC0NVTX4vmr8ytjtV7Iu8cwuG/7K+I1ofST7AYBYPg1tf/LeNciyS/BHwiwxuQft94LPA/wT+O/BPgb8Gfq2qmv8j5RWOxS8yWLYp4BXgP/54nbpVSX4B+D/Ad4C3u+LPMFinb+570XzQS9JG1/rSjSRteAa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJatz/B/Pav7doOJwsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot feature importance\n",
    "plt.bar([x for x in range(len(importance))], importance)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "eb78fa8b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation Result:\n",
      "\n",
      "accuracy score: 0.7638\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAaKElEQVR4nO3de7QkZX3u8e/jDCJyGVDA6HAZIBBEBYIjeEMRUYGjB29REEXQSEYlcZ3jccHKirdgEi9Ro0sQiCAaEA4KR9EgHCMKepDAsOTiSICR64ioCA6CCAz8zh9VwzTbPbV775navWfm+1mr13RVvfX2r97p3U9XVXd1qgpJklbmcaMuQJI0sxkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgbFGi7J4Ul+OOo6Vqckhyb5v0O0OyHJ+6ejpumQ5OYk+7X3P5TktFHXJIFBMRJJ1k9ycpJbkvwuyY+THDDquobRvpjdn+TeJL9M8sUkG63Ox6iq06vq5UO0W1BVx67Ox14uSSW5r93Onyf5VJJZfTzW2i7JqUmWJXnamPkfSvJQO8a/TXJJkudNof83tX9L9yX5epInraTdNu1jDd4qyXsn29e6xqAYjdnAbcCLgTnA+4GzkswbZVGT8Kqq2gjYA3gO8HdjGySZPe1VrX67tdv5YuCNwNtGXM9qNR3/R0k2BF4HLAUOHafJ/27HeAvgh8A5STKJ/p8BnAi8BXgK8Hvg+PHaVtWtVbXR8hvwLOAR4OzJ9rWuMShGoKruq6oPVdXNVfVIVX0LuAl49srWSbJ1knOS/DrJb5J8biXtPpPktiT3JLkiyd4Dy/ZMsrBd9sskn2rnPyHJaW2/v01yeZKnDLEdPwe+DTyz7aeSvDvJDcAN7bxXJrly4B3jrhNt0+DhtDQ+neRXSZYmuTrJ8sc7NclHBvp7R5LFSe5Kcu7gO9i2tgVJbkhyd5Ljhn1BqqrFwP8Ddh/obyrbtUOSC9t5dyY5Pcmmw9QwVpKD2se/J8nPkuzfzn/08FU7/eghrCTz2nF4e5JbgQuTnJ/kqDF9X5Xkte39nZN8px3T65K8YZKlvg74LfD3wFtX1qiqHgK+BPwJ8ORJ9H8o8M2quriq7qV50/XaJBsPse5hwMVVdfNq6GutZlDMAO2L8k7AopUsnwV8C7gFmAfMBc5cSXeX07ygPQn4CvDVJE9ol30G+ExVbQLsAJzVzn8rzZ7N1jR/pAuA+4eoe2vgQODHA7NfDewF7JJkD+AU4K/afk8Ezk1z6G3YbXo58CKa8dmU5p39b8apZV/gn4A3AE9t+x3b3ytp9oB2a9u9YqJtbPveGdgbWNxOT3W70tb4NODpNOP9oWFqGFPPnsCXgffRjMmLgJsn0cWL28d/Bc1z5JCBvncBtgX+vd0b+E7bZsu23fHtO+/lh2munuCx3gqcQTMGO7djN942rQ8cDiypqjuTvLAN4ZXdXtiu+gzgquX9VNXPgAdpni8TOYwmnJZblb7WblXlbYQ3YD3gP4ATO9o8D/g1MHucZYcDP+xY926aQygAFwMfBjYf0+ZtwCXArkPUezNwL827xFtods03aJcVsO9A288Dx45Z/zqaF6qhtgnYF7geeC7wuDHtTgU+0t4/Gfj4wLKNgIeAeQO1vXBg+VnAMR3bWcA9wH3t/TOA9Vdlu8Z5jFcDPx4ztvu19z8EnLaS9U4EPt3x/7PfwPSj/dAEVwHbDyzfuN3GbdvpfwBOae+/EfjBOI/9wSGf29vQHNrZvZ2+gOaNymBtD7bPpV8BFwLPnuTfz3eBBWPm/RzYZ4L19m6fxxutal/rws09ihFK8jjg32j+WI4amP/trDjZdijNO89bqmrZEH2+N8m17WGa39LsKWzeLn47zbuj/2oPL72ynf9vNH/EZya5PcnHk6zX8TCvrqpNq2rbqnpXVQ3ufdw2cH9b4L2D7wTbbXnasNtUVRcCnwOOA36Z5KQkm4zT9Gk0wbV8vXtp9jzmDrS5Y+D+72nChCSLBsZ774E2e7Rt3kizl7ThqmxXki2TnJnm5Pg9wGms+L+ZjK2Bn01hveUe/T+qqt8B/w4c3M46GDi9vb8tsNeY7TyU5vDQMN4CXFtVV7bTpwNvGvPcOqt9Lm1ZVftW1RWT3JZ7gbHPh02A302w3luBs9vnyar2tdYzKEakPT5+Ms1Js9dVc4wWgKo6oFacdDud5g97m0xw8rF9kTua5rDKZlW1Kc1JxLT93lBVh9AcRvgY8LUkG1bVQ1X14araBXg+zSGaw6a4aYOXI74N+If2hWD57YlVdcaw29TW/dmqejbNoYGdaA65jHU7zQsb8OhJ1CfTvCOcqP9nDIz3D8Ysq6o6C/gR8IFV3K5/ohmfXas5/Pdm2v+bSbqN5tDheO4DnjgwPd6L+thLRp8BHJLmE0cbAN8beJyLxmznRlX1ziHrPAzYPskdSe4APkUTjBN+wi/J3vnjTyjdO06gL6I5lLh8ve2B9Wn2QlfW9wbAX/DYw05T6mtdYVCMzudpjhO/asw78vFcBvwC+GiSDdOcfH7BOO02BpbRHvpI8gEG3iEleXOSLarqEZrdfYCHk7wkybPa4+v30ByyeXhVNq71r8CCJHulsWGS/9aeHBxqm5I8p11/PZoXwT+spLavAEck2b093v2PwH/WihOVq+qjwJFJ/mQVtmtj2sN2SeYyfuAN42SabX1pksclmdueRwG4Ejg4yXpJ5gOvH6K/82hC9u9pPoX0SDv/W8BOSd7S9rde+//x9Ik6bENnB2BPmnNmu9N86OErdJzUXq6qfjAQ3uPdlgf66cCr2mDZsN2Gc9o9pZV5Dc3z/3tj5k+lr3WCQTECSbalORG6O3DHmMNMf6SqHgZeBfwpcCuwhOZwyFgX0HwK6XqawzB/4LGHgvYHFiW5l+bE9sFV9Qead51fowmJa4GLaA6LrJKqWgi8g+bQ0d00J4MPn+Q2bULzwnx3u02/Af55nMf6Ls2nVM6meaHegRWHU1ZZVV1DMy7vW4Xt+jDN4aylNId7zpliLZcBRwCfbvu6iBV7U++n2fa728f7yhD9PdDWst9g+/YF8uU043g7zaG7j9G8y17+xchxP4BBEwbfqKprquqO5Tea590rs5q+n1BVi2g+fHE6zXmOjYF3LV+e5kuZJ4xT25er6jF7VhP1tS7LmLGSJOkx3KOQJHUyKCRJnQwKSVIng0KS1GmNu3Db5ptvXvPmzRt1GZK0RrniiivurKotprLuGhcU8+bNY+HChaMuQ5LWKElumbjV+Dz0JEnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI69RYUSU5J8zvHP1nJ8iT5bJrfOL46K/mJREnSaPW5R3EqzWWtV+YAYMf2diTN7zNIkmaY3oKiqi4G7upochDtNeGr6lJg0yRPnajfBx9cXRVKkoYxynMUc3nsj+os4bG/b/yoJEcmWZhk4S9+cfe0FCdJaowyKMb7reBxf0Wpqk6qqvlVNX/OnM16LkuSNGiUQbEE2Hpgeiuan1uUJM0gowyKc4HD2k8/PRdYWlW/GGE9kqRx9Hb12CRnAPsAmydZAnwQWA+gqk4AzgMOpPlh+t/T/Fi8JGmG6S0oquqQCZYX8O6+Hl+StHr4zWxJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ16DYok+ye5LsniJMeMs3xOkm8muSrJoiRH9FmPJGnyeguKJLOA44ADgF2AQ5LsMqbZu4GfVtVuwD7AJ5M8vq+aJEmT1+cexZ7A4qq6saoeBM4EDhrTpoCNkwTYCLgLWNZjTZKkSeozKOYCtw1ML2nnDfoc8HTgduAa4D1V9cjYjpIcmWRhkoVLl97dV72SpHH0GRQZZ16NmX4FcCXwNGB34HNJNvmjlapOqqr5VTV/zpzNVnedkqQOfQbFEmDrgemtaPYcBh0BnFONxcBNwM491iRJmqQ+g+JyYMck27UnqA8Gzh3T5lbgpQBJngL8GXBjjzVJkiZpdl8dV9WyJEcBFwCzgFOqalGSBe3yE4BjgVOTXENzqOroqrqzr5okSZOXqrGnDWa27befX+efv3DUZUhaTZ70JNh881FXsfZLckVVzZ/Kur3tUfTp4otHXYGk1eGBB5qgOOSQUVeiLmtcUMyeDX/+56OuQtLqcMstcNddo65CE/FaT5KkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSp01BBkeQFSb6T5PokNya5KcmNQ6y3f5LrkixOcsxK2uyT5Moki5JcNNkNkCT1a/aQ7U4G/gdwBfDwMCskmQUcB7wMWAJcnuTcqvrpQJtNgeOB/avq1iRbTqJ2SdI0GDYollbVtyfZ957A4qq6ESDJmcBBwE8H2rwJOKeqbgWoql9N8jEkST0b9hzF95J8Isnzkuyx/DbBOnOB2waml7TzBu0EbJbk+0muSHLYkPVIkqbJsHsUe7X/zh+YV8C+HetknHk1zuM/G3gpsAHwoySXVtX1j+koORI4EmDLLbcZsmRJ0uowVFBU1Uum0PcSYOuB6a2A28dpc2dV3Qfcl+RiYDfgMUFRVScBJwHstNP8sWEjSerRsJ96mpPkU0kWtrdPJpkzwWqXAzsm2S7J44GDgXPHtPkGsHeS2UmeSLPncu1kN0KS1J9hz1GcAvwOeEN7uwf4YtcKVbUMOAq4gObF/6yqWpRkQZIFbZtrgfOBq4HLgC9U1U+msiGSpH4Me45ih6p63cD0h5NcOdFKVXUecN6YeSeMmf4E8Ikh65AkTbNh9yjuT/LC5RNJXgDc309JkqSZZNg9incCX2rPSwS4Czi8r6IkSTPHsJ96uhLYLckm7fQ9fRYlSZo5OoMiyZur6rQk/3PMfACq6lM91iZJmgEm2qPYsP13474LkSTNTJ1BUVUntv9+eHrKkSTNNMN+4e7jSTZJsl6S7ya5M8mb+y5OkjR6w3489uXtCexX0lx2Yyfgfb1VJUmaMYYNivXafw8Ezqiqu3qqR5I0wwz7PYpvJvkvmi/ZvSvJFsAf+itLkjRTDLVHUVXHAM8D5lfVQ8B9ND9CJElay030PYp9q+rCJK8dmDfY5Jy+CpMkzQwTHXp6MXAh8KpxlhUGhSSt9Sb6HsUH23+PmJ5yJEkzzbDfo/jHJJsOTG+W5CO9VSVJmjGG/XjsAVX12+UTVXU3zUdlJUlruWE/HjsryfpV9QBAkg2A9fsrS9K64oEH4PrrR12FugwbFKcB303yRZqT2G8DvtRbVZLWCXPmwB13wMUXj7qSdcHGG07cZnzD/h7Fx5NcDexH88NFx1bVBVN9UEkC2HRT2GuvUVexrpg1a6prDrtHAXAtsKyq/iPJE5NsXFW/m+oDS5LWDMN+6ukdwNeAE9tZc4Gv91STJGkGGfZTT+8GXgDcA1BVNwBb9lWUJGnmGDYoHqiqB5dPJJlNc1JbkrSWGzYoLkryt8AGSV4GfBX4Zn9lSZJmimGD4mjg18A1wF8B5wF/11dRkqSZY8JPPSV5HHB1VT0T+Nf+S5IkzSQT7lFU1SPAVUm2mYZ6JEkzzLDfo3gqsCjJZTQ/WgRAVf33XqqSJM0YwwbFh3utQpI0Y030C3dPABYAf0pzIvvkqlo2HYVJkmaGic5RfAmYTxMSBwCf7L0iSdKMMtGhp12q6lkASU4GLuu/JEnSTDLRHsVDy+94yEmS1k0TBcVuSe5pb78Ddl1+P8k9E3WeZP8k1yVZnOSYjnbPSfJwktdPdgMkSf3qPPRUVVO+fnmSWcBxwMuAJcDlSc6tqp+O0+5jgL9vIUkz0LCX8JiKPYHFVXVje0HBM4GDxmn318DZwK96rEWSNEV9BsVc4LaB6SXtvEclmQu8Bjihq6MkRyZZmGTh0qW/Xu2FSpJWrs+gyDjzxl6a/F+Ao6vq4a6OquqkqppfVfPnzNliddUnSRrCZH4KdbKWAFsPTG8F3D6mzXzgzCQAmwMHJllWVV/vsS5J0iT0GRSXAzsm2Q74OXAw8KbBBlW13fL7SU4FvmVISNLM0ltQVNWyJEfRfJppFnBKVS1KsqBd3nleQpI0M/S5R0FVnUfzI0eD88YNiKo6vM9aJElT0+fJbEnSWsCgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUqdegyLJ/kmuS7I4yTHjLD80ydXt7ZIku/VZjyRp8noLiiSzgOOAA4BdgEOS7DKm2U3Ai6tqV+BY4KS+6pEkTU2fexR7Aour6saqehA4EzhosEFVXVJVd7eTlwJb9ViPJGkK+gyKucBtA9NL2nkr83bg2+MtSHJkkoVJFi5d+uvVWKIkaSJ9BkXGmVfjNkxeQhMUR4+3vKpOqqr5VTV/zpwtVmOJkqSJzO6x7yXA1gPTWwG3j22UZFfgC8ABVfWbHuuRJE1Bn3sUlwM7JtkuyeOBg4FzBxsk2QY4B3hLVV3fYy2SpCnqbY+iqpYlOQq4AJgFnFJVi5IsaJefAHwAeDJwfBKAZVU1v6+aJEmT1+ehJ6rqPOC8MfNOGLj/l8Bf9lmDJGnV+M1sSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUqdegyLJ/kmuS7I4yTHjLE+Sz7bLr06yR5/1SJImr7egSDILOA44ANgFOCTJLmOaHQDs2N6OBD7fVz2SpKnpc49iT2BxVd1YVQ8CZwIHjWlzEPDlalwKbJrkqT3WJEmapNk99j0XuG1gegmw1xBt5gK/GGyU5EiaPQ4gD82fv9nNq7fUNdUDc2D9paOuYmZwLFZwLFZwLFa4Z6uprtlnUGSceTWFNlTVScBJAEkWVt09f9XLW/M1Y/F7xwLHYpBjsYJjsUKShVNdt89DT0uArQemtwJun0IbSdII9RkUlwM7JtkuyeOBg4Fzx7Q5Fzis/fTTc4GlVfWLsR1Jkkant0NPVbUsyVHABcAs4JSqWpRkQbv8BOA84EBgMfB74Ighuj6pp5LXRI7FCo7FCo7FCo7FClMei1T90SkBSZIe5TezJUmdDApJUqcZGxRe/mOFIcbi0HYMrk5ySZLdRlHndJhoLAbaPSfJw0leP531TadhxiLJPkmuTLIoyUXTXeN0GeJvZE6Sbya5qh2LYc6HrnGSnJLkV0l+spLlU3vdrKoZd6M5+f0zYHvg8cBVwC5j2hwIfJvmuxjPBf5z1HWPcCyeD2zW3j9gXR6LgXYX0nxY4vWjrnuEz4tNgZ8C27TTW4667hGOxd8CH2vvbwHcBTx+1LX3MBYvAvYAfrKS5VN63ZypexRe/mOFCceiqi6pqrvbyUtpvo+yNhrmeQHw18DZwK+ms7hpNsxYvAk4p6puBaiqtXU8hhmLAjZOEmAjmqBYNr1l9q+qLqbZtpWZ0uvmTA2KlV3aY7Jt1gaT3c6307xjWBtNOBZJ5gKvAU6YxrpGYZjnxU7AZkm+n+SKJIdNW3XTa5ix+BzwdJov9F4DvKeqHpme8maUKb1u9nkJj1Wx2i7/sRYYejuTvIQmKF7Ya0WjM8xY/AtwdFU93Lx5XGsNMxazgWcDLwU2AH6U5NKqur7v4qbZMGPxCuBKYF9gB+A7SX5QVff0XNtMM6XXzZkaFF7+Y4WhtjPJrsAXgAOq6jfTVNt0G2Ys5gNntiGxOXBgkmVV9fVpqXD6DPs3cmdV3Qfcl+RiYDdgbQuKYcbiCOCj1RyoX5zkJmBn4LLpKXHGmNLr5kw99OTlP1aYcCySbAOcA7xlLXy3OGjCsaiq7apqXlXNA74GvGstDAkY7m/kG8DeSWYneSLN1ZuvneY6p8MwY3ErzZ4VSZ4C/Blw47RWOTNM6XVzRu5RVH+X/1jjDDkWHwCeDBzfvpNeVlVr3RUzhxyLdcIwY1FV1yY5H7gaeAT4QlWN+7HJNdmQz4tjgVOTXENz+OXoqrpzZEX3JMkZwD7A5kmWAB8E1oNVe930Eh6SpE4z9dCTJGmGMCgkSZ0MCklSJ4NCktTJoJAkdTIopHG0V569MslP2quObrqa+785yebt/XtXZ9/S6mZQSOO7v6p2r6pn0lxk7d2jLkgaFYNCmtiPaC+clmSHJOe3F9n7QZKd2/lPSfJ/2t87uCrJ89v5X2/bLkpy5Ai3QZqyGfnNbGmmSDKL5tIPJ7ezTgIWVNUNSfYCjqe50NxngYuq6jXtOhu17d9WVXcl2QC4PMnZa/G1uLSWMiik8W2Q5EpgHnAFzdVGN6L5kaivDlyZdv32332BwwCq6mFgaTv/b5K8pr2/NbAjYFBojWJQSOO7v6p2TzIH+BbNOYpTgd9W1e7DdJBkH2A/4HlV9fsk3wee0EexUp88RyF1qKqlwN8A/wu4H7gpyV/Ao78/vPz3yb8LvLOdPyvJJsAc4O42JHam+elJaY1jUEgTqKof0/wO88HAocDbk1wFLGLFT26+B3hJe3XSK4BnAOcDs5NcTXP10kunu3ZpdfDqsZKkTu5RSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqdP/B25niV9iV5PaAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "roc auc score: 0.7631589889943737\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA84klEQVR4nO3dd3hUZfbA8e+RDlIUsIEIKoqAECAguIIoFsS+NkRU+KmIKHbXXnYta18DgoioqCisstgL2LArRZAuIiJEUGlKEyHk/P44N2QSk8mQZHKnnM/zzJOZuXfuPXOTzJn7vu89r6gqzjnnXHF2CjsA55xzic0ThXPOuag8UTjnnIvKE4VzzrmoPFE455yLyhOFc865qDxRuB0iInNFpHvYcSQKEblJREaFtO/RInJXGPsubyJyjohMKuVr/W8yzjxRJDERWSIif4jIBhH5Ofjg2Dme+1TVVqo6OZ77yCMi1UTk3yKyNHif34nIdSIiFbH/IuLpLiLZkc+p6j2qemGc9icicrmIzBGRjSKSLSIvicjB8dhfaYnIHSIypizbUNXnVfWYGPb1l+RYkX+T6coTRfI7UVV3BjKAdsCN4Yaz40SkcjGLXgJ6AL2A2sC5wAAgKw4xiIgk2v9DFnAFcDmwK3AA8ApwfHnvKMrvIO7C3LeLkar6LUlvwBLgqIjH9wNvRjzuDHwO/AZ8A3SPWLYr8DSwHFgLvBKx7ARgZvC6z4E2hfcJ7AX8AewasawdsAqoEjz+P2B+sP2JwD4R6ypwKfAd8EMR760HsBnYu9DzhwDbgP2Dx5OBfwNTgN+BVwvFFO0YTAbuBj4L3sv+QP8g5vXAYuDiYN1awTq5wIbgthdwBzAmWKdp8L7OB5YGx+LmiP3VAJ4Jjsd84B9AdjG/2+bB++wU5fc/GhgGvBnE+xWwX8TyLGAZsA6YDnSNWHYHMB4YEyy/EOgEfBEcqxXAo0DViNe0At4F1gC/ADcBPYEtwNbgmHwTrFsXeDLYzk/AXUClYFm/4Jj/J9jWXcFznwbLJVj2a/A7nQW0xr4kbA32twF4vfD/AVApiOv74JhMp9DfkN9K8VkTdgB+K8Mvr+A/SGNgNpAVPG4ErMa+je8EHB08bhgsfxP4L7ALUAU4PHi+ffAPekjwT3d+sJ9qRezzA+CiiHgeAEYE908BFgEHAZWBW4DPI9bV4ENnV6BGEe/tXuCjYt73j+R/gE8OPohaYx/m/yP/g7ukYzAZ+0BvFcRYBfu2vl/wYXU4sAloH6zfnUIf7BSdKJ7AkkJb4E/goMj3FBzzxtgHYHGJYiDwYwm//9HYB22nIP7ngXERy/sC9YNl1wA/A9Uj4t4a/J52CuLtgCXWysF7mQ9cGaxfG/vQvwaoHjw+pPAxiNj3K8Djwe9kNyyR5/3O+gE5wOBgXzUomCiOxT7g6wW/h4OAPSPe811R/g+uw/4PDgxe2xaoH/b/arLfQg/Ab2X45dk/yAbsm5MC7wP1gmXXA88VWn8i9sG/J/bNeJcitvkYcGeh574lP5FE/lNeCHwQ3Bfs22u34PHbwAUR29gJ+9DdJ3iswJFR3tuoyA+9Qsu+JPimjn3Y3xuxrCX2jbNStGMQ8dp/lXCMXwGuCO53J7ZE0Thi+RSgd3B/MXBsxLILC28vYtnNwJclxDYaGBXxuBewIMr6a4G2EXF/XML2rwReDu6fDcwoZr3txyB4vDuWIGtEPHc28GFwvx+wtNA2+pGfKI4EFmJJa6ci3nO0RPEtcHJZ/7f8VvCWaG2ybsedoqq1sQ+xFkCD4Pl9gDNE5Le8G3AYliT2Btao6toitrcPcE2h1+2NNbMUNh7oIiJ7Ad2wD8lPIraTFbGNNVgyaRTx+mVR3teqINai7BksL2o7P2JnBg2IfgyKjEFEjhORL0VkTbB+L/KPaax+jri/CcgbYLBXof1Fe/+rKf79x7IvROQaEZkvIr8H76UuBd9L4fd+gIi8EQyMWAfcE7H+3lhzTiz2wX4HKyKO++PYmUWR+46kqh9gzV7DgF9EZKSI1Ilx3zsSp4uRJ4oUoaofYd+2HgyeWoZ9m64XcaulqvcGy3YVkXpFbGoZcHeh19VU1bFF7PM3YBJwJtAHGKvB17pgOxcX2k4NVf08chNR3tJ7wCEisnfkkyLSCfsw+CDi6ch1mmBNKqtKOAZ/iUFEqmFNVw8Cu6tqPeAtLMGVFG8sVmBNTkXFXdj7QGMRySzNjkSkK3ZGdSZ25lgPa++PHDFW+P08BiwAmqtqHaytP2/9ZViTXFEKb2cZdkbRIOK411HVVlFeU3CDqkNUtQPWLHgA1qRU4utKiNOVkieK1PIIcLSIZGCdlCeKyLEiUklEqgfDOxur6gqsaWi4iOwiIlVEpFuwjSeAgSJySDASqJaIHC8itYvZ5wvAecBpwf08I4AbRaQVgIjUFZEzYn0jqvoe9mH5PxFpFbyHzlg7/GOq+l3E6n1FpKWI1AT+BYxX1W3RjkExu60KVANWAjkichwQOWTzF6C+iNSN9X0U8iJ2THYRkUbAZcWtGLy/4cDYIOaqQfy9ReSGGPZVG+sHWAlUFpHbgJK+ldfGOrY3iEgL4JKIZW8Ae4jIlcGw5doickiw7Begad6oseDvaxLwkIjUEZGdRGQ/ETk8hrgRkY7B318VYCM2qGFbxL72jfLyUcCdItI8+PttIyL1Y9mvK54nihSiqiuBZ4FbVXUZcDL2rXAl9k3rOvJ/5+di37wXYJ3XVwbbmAZchJ36r8U6pPtF2e1r2AidX1T1m4hYXgbuA8YFzRhzgON28C2dBnwIvIP1xYzBRtIMLrTec9jZ1M9YR+vlQQwlHYMCVHV98NoXsffeJ3h/ecsXAGOBxUGTSlHNcdH8C8gGfsDOmMZj37yLczn5TTC/YU0qpwKvx7CvidiXgYVYc9xmojd1AVyLvef12BeG/+YtCI7N0cCJ2HH+DjgiWPxS8HO1iHwd3D8PS7zzsGM5ntia0sAS2hPB637EmuHyzpSfBFoGx/+VIl77MPb7m4QlvSexznJXBpLfUuBc8hGRyVhHaihXR5eFiFyCdXTH9E3bubD4GYVzFURE9hSRvwVNMQdiQ01fDjsu50oSt0QhIk+JyK8iMqeY5SIiQ0RkkYjMEpH28YrFuQRRFRv9sx7rjH8V64dwLqHFrekp6BzdADyrqq2LWN4La2vuhV3claWqhxRezznnXLjidkahqh9jY+eLczKWRFRVvwTqiUisnV3OOecqSJjFuBpRcBRGdvDcisIrisgArM4LtWrV6tCiRYsKCdA555KJKvzxB2zYABs32s9dt6xgT35mBrmrVLVhabYbZqIoqlR0ke1gqjoSGAmQmZmp06ZNi2dczjmXFLKz4csv82/Tp8PmzbZszz2U7icI59R+jc7rJtHo5WE/lnY/YSaKbApemdoYq2TqnHOukD/+sEQQmRh++smWVasGHTrAoEHQtfVajpp4LbVa74vccjNwkt1kWKn3HWaieA24TETGYZ3ZvwdXdDrnXFpThcWLCyaFmTMhJ8eWN2sG3bpB587QpQu0bQtVqwIvv2zZYuVKaHFLucUTt0QhImOxQnUNxGYFux0rFIaqjsBq6PTCrvzdhM0D4JxzaWf9epgypWBiWBWUvaxVCzp1guuus8RwyCGw++6FNvDLLzB4MLz0EmRkwJtvQvvyu+IgbolCVc8uYbliE9c451zayM2FBQsKJoU5c+wsAqBFCzjhBDtT6NwZWraEyiV9Ui9bZsnh7rsto1SpUq4x+xSEzjkXR6tXw1df5SeFr76CdetsWb16lgxOO81+duoEu+wS44Z//BFefx0uuwwyM2HpUqgfn/qHniicc66c5OTA7NkFzxYWLrRlO+0EBx8MffpYUujcGZo3t+d3SG4uPPYY3BAUET7tNNhzz7glCfBE4ZxzpbZiRcGkMG0abNpky3bbzZqP+ve3pJCZCTvvHH17Jfr2W7jwQvj0Uzj2WHj8cUsSceaJwjnnYvDnnzBjRsHE8GNwZUKVKtCuHVx0Uf7Zwj77gBR1tVhpbdoEhx0G27bB6NFw3nnlvIPieaJwzrlCVC0JRCaFGTNgyxZb3qSJJYMrrrCf7dpB9epxCmbhQmujqlkTnnvORjXtsUecdlY0TxTOubS3caM1G0Umhp+D2chr1LBmoyuvzB+euteOTllVGps3w513wn332RlE377Qs2cF7PivPFE459KKqn1Jj0wKs2dbiw7Yl/ejj85vQjr44HIfbVqyzz6DCy6wPon+/eH44ys4gII8UTjnUtpvv/31Yra1a21Z7dp2hnDTTfnDUxs0CDVcO4u4/XZr35o4EY45puTXxJknCudcyti2DebOLZgU5s+3ZSLQqlX+NQudO9vFbZUqhRvzdqoWZEaGXWV9993lMEyqfHiicM4lrV9/LXgx25QpVlob7Mygc2c45xz72bEj1KkTbrxFWrMGrroK9t8fbr0VTjzRbgnEE4VzLils2QLffFPwbGHxYltWubIVxjv//Pyzhf32q7DRo6U3fjxceqkli1tvDTuaYnmicM4lpGhzLey1l13MdskllhTat7fRo0ljxQorvTFhgtUHnzTJMl2C8kThnAtdrHMt5JXVbtw43HjLbPly66i+7z64+uoYqv6FK7Gjc86lnLy5Fr74Ij8pfPNN/lwL++4Lhx+e34S0fa6FZLdkiRXxGzzYMt+yZTtQATBcniicc3G1bh1MnRrbXAudO1uNpJSybRsMG2ZjcHfaCc44w66sTpIkAZ4onHPlKG+uhcizhblz8+daOOggG9CTlxRatUqg4anxMH++FfH7/HO7qvrxxyu8/EZ58EThnCu1WOZaOP30Usy1kAo2bbL5SnNz4dlnrQRHwg/DKponCudcTPLmWog8W/juO1u2007Qpk05zLWQChYsgAMPtGFYzz9vnSx/mbs0uXiicM4VKdpcC7vvbsngggvsZ4cOCXMRcXj++APuuAMefBCeecbOIBKg/EZ58EThnNs+10Lk2cLSpbasShW7TiGucy0ku48/tr6I776znyecEHZE5coThXNpJta5Fq66yn5mZMRxroVU8M9/2plEs2bw3nvQo0fYEZU7TxTOpbgNG/4618Ivv9iyGjWsBlKFz7WQCvKK+GVmWla9804b75uCPFE4l0Jyc631IzIpzJplz4N1MB9zjF3d3LkztG4dwlwLyW7VKksMzZvDbbfZXBEhzxcRb54onEtia9cWnGvhq6/y51qoU8fOEG6+Of9soX79cONNaqrw0ktWo2ntWpszIk14onAuScQ610Le2UKLFmk6PDUeli+3YlOvvmpNTe+9Z+OB04QnCucS1K+/FkwKU6cm4VwLqeLnn+GDD+CBB6xDJ8GL+JW39Hq3ziWokuZayMiAfv3yh6fuu68PT427xYvhtdcsMbRvb+OF69ULO6pQeKJwLgTLlv11roU//7RlST/XQrLbtg2GDLHOnSpVoHdvq8+UpkkCPFE4F3clzbWQmWn9o3lnC0k/10IymzvXLjf/6isbyTRiRFIW8StvniicK0eq8P33BZNCWsy1kAo2bbJfjgi88IKdSXj7HuCJwrkyiTbXws47W8XUf/wjf3hqys21kArmzbP65zVrwrhxlr0bNgw7qoTiicK5GOXm2nDUyKSQ1nMtJLtNm+xaiIcfhtGj4dxz4aijwo4qIXmicK4YeXMt5BXKmzIlf66FXXaxZHDGGflzLaRxX2fymTzZqhwuWgQXXwwnnRR2RAnNE4VzwNatNtdC5NlCtLkWDjjAm6+T1u23w7/+BfvtZ9dGHHFE2BElPE8ULi3lzbWQd7YwbZqNTgKba6FLF59rIeXkFfHr1AmuucaShY87jklcE4WI9ASygErAKFW9t9DyusAYoEkQy4Oq+nQ8Y3LpZ/NmK6MdebZQeK6FAQPyS180aeJnCyll5Uq44gqbde7229OiiF95i1uiEJFKwDDgaCAbmCoir6nqvIjVLgXmqeqJItIQ+FZEnlfVLfGKy6W2yLkW8s4WZsywpiWwCXe6dPG5FtKCKowdC5dfbp1L//xn2BElrXieUXQCFqnqYgARGQecDEQmCgVqi4gAOwNrgJw4xuRSTCxzLVx1lSWHQw6BPfcMN15XQbKz7dL2N96wX/yTT9owNFcq8UwUjYBlEY+zgUMKrfMo8BqwHKgNnKWquYU3JCIDgAEATZo0iUuwLvFFzrWQd7Ywe3b+XAsHHADHHpvf4XzwwWlXu83lWbnSpid9+GE7o/BxymUSz3+jolp5tdDjY4GZwJHAfsC7IvKJqq4r8CLVkcBIgMzMzMLbcCkq1rkWunSx/kmfayHNLVoEr79up5Dt2llBLS+pWy7imSiygb0jHjfGzhwi9QfuVVUFFonID0ALYEoc43IJaNs2mDOnYBPSggW2TMRmYjv99PyzBZ9rwW2XkwOPPAK33mrFs/r0saFrniTKTTwTxVSguYg0A34CegN9Cq2zFOgBfCIiuwMHAovjGJNLEL/8YmcIeUlhyhTYuNGW5c210LevnS1kZvr/vCvG7Nk2jnnqVLtobvhwSxKuXMUtUahqjohcBkzEhsc+papzRWRgsHwEcCcwWkRmY01V16vqqnjF5MKxZQvMnFnwbOGHH2xZ3lwL/fv7XAtuB23aZBfL7bST1Wg680z/w4kTUU2uJv/MzEydNm1a2GG4YqjagJPi5lpo1Cj/eoW8uRZq1Ag3Zpdk5syxEUwi8P77VsSvQYOwo0p4IjJdVTNL81ofE+LKZNOmv861sDzoiape3a5q9rkWXLnYuNH6IR55BJ55xor49egRdlRpwROFi1lJcy3st5+1BOQlhTZtfK4FV07ef9+K+P3wAwwaBCefHHZEacUThSvWunUFh6d++aVVVAWfa8FVoFtvhbvugubN4aOPoFu3sCNKO54oHFDyXAstW9qXuLyzhZYt/RomF2e5udZRfeih9o3kjju8QysknijS1KpVfx2e6nMtuITw6692NfWBB1p9puOOs5sLjSeKNBA510Je6YtFi2xZpUrWl3DOOflnC82b+yhDFwJVeP55q/S6YYOVAXcJwRNFClq+vGATUlFzLVx4oSWFzEyoVSvceJ1j2TIYOBDeesv+QEeNsvZNlxA8USS5rVvzq6fmnS0sC0oxVq1q1ylcfHH+2YLPteAS0urV8NlnkJUFl17qHWAJxhNFkuvTB8aPt/v77GP9fp0725eyjAwrfeNcQlq4EF57Da691v5Yly2D2rXDjsoVwRNFEvv2W0sSgwbBLbf4XAsuSeTkwEMP2WxzNWrYhXO77+5JIoF5/c0kNnSoNS/ddpsnCZckvvnGLrq54Qbo1QvmzfMifknAzyiS1G+/wejR0Lu3/5+5JLFpk5XcqFzZToVPOy3siFyMPFEkqaeestI3V1wRdiTOlWDWLJtusGZNeOklK+K3665hR+V2gDc9JaFt2+DRR+Gww2xUk3MJacMG+yaTkQHPPWfPHXGEJ4kk5GcUSeiNN6w22n33hR2Jc8V4910YMACWLLHywaeeGnZErgz8jCIJZWXB3nv7/55LUDffDMccY2OzP/nERl34iKakFnOiEBG/fjcBzJoFH35o1yRV9vNBl0hyc+3nYYfBjTfatIaHHRZqSK58lJgoRORQEZkHzA8etxWR4XGPzBVp6FAben7RRWFH4lzg55/h9NOtuitYAb977rGZq1xKiOWM4j/AscBqAFX9BvCC8CFYtQrGjIG+fb0/0CUAVRuj3bKldZzVqRN2RC5OYmq8UNVlUrBA0Lb4hOOieeIJ2LzZKjA7F6off7TO6kmTrHlp1CgrC+5SUixnFMtE5FBARaSqiFxL0AzlKs7WrTBsmF2v1Lp12NG4tPfbbzB1qo3T/ugjTxIpLpYzioFAFtAIyAYmAYPiGZT7q5dfhp9+gsceCzsSl7a+/daK+F13nV00t3SpzYnrUl4sZxQHquo5qrq7qu6mqn2Bg+IdmCsoKwv23dfK4zhXobZuhX//25LDvffaDHTgSSKNxJIohsb4nIuTadPg889h8GAv0+8q2IwZVsTvppvgxBOtiN9uu4UdlatgxTY9iUgX4FCgoYhcHbGoDuAfVxVoyBD78ta/f9iRuLSyaRMcfTRUqQL/+x/8/e9hR+RCEq2Poiqwc7BO5GWV64DT4xmUy/fzzzBunM1SV7du2NG4tDBjhtVnqlnTqry2bQu77BJ2VC5ExSYKVf0I+EhERqvqjxUYk4swYoQ1EQ8eHHYkLuWtX29XVA8bBs88A+edB927hx2VSwCxjHraJCIPAK2A7ZdaquqRcYvKAfDnnzbKqVcvOOCAsKNxKe2dd+y0ddkyq/jqzUwuQiyd2c8DC4BmwD+BJcDUOMbkAi++aANMfM4JF1c33mhlN2rVgs8+g0ce8RFNroBYzijqq+qTInJFRHPUR/EOLN2p2pDYFi2sP9G5crdtmw2j697dKkzecotVfHWukFgSxdbg5woROR5YDjSOX0gObDjs9OkwfDgUrJ7iXBmtWGHlh1u1gjvvhGOPtZtzxYil6ekuEakLXANcC4wCroxnUM7OJurVs/5E58qFKjz9tBXxe/ttH8nkYlbiGYWqvhHc/R04AkBE/hbPoNLdsmUwYQJcdZU1GztXZkuWWG36996Drl2tiJ+PkHAxinbBXSXgTKzG0zuqOkdETgBuAmoA7SomxPQzfLh9+bv00rAjcSnj99/h66/tj+vii2Enn9zSxS7aX8uTwIVAfWCIiDwNPAjcr6oxJQkR6Ski34rIIhG5oZh1uovITBGZ653kdjHsyJFw8snQtGnY0bikNm+e1WaC/CJ+l1ziScLtsGhNT5lAG1XNFZHqwCpgf1X9OZYNB2ckw4CjsaqzU0XkNVWdF7FOPWA40FNVl4pI2heRef55WLPGh8S6MtiyBe6/3zqqa9eG//s/q8/k7ZiulKJ9tdiiqrkAqroZWBhrkgh0Ahap6mJV3QKMA04utE4fYIKqLg328+sObD/lqFpdp7ZtoZvPIehKY9o06NgRbr3VLprzIn6uHEQ7o2ghIrOC+wLsFzwWQFW1TQnbbgQsi3icDRxSaJ0DgCoiMhmrJ5Wlqs8W3pCIDAAGADRp0qSE3SavDz+EOXPgySd9SKwrhY0bbZhr9erw6qtw0klhR+RSRLREUdY5J4r6qNMi9t8B6IF1kH8hIl+q6sICL1IdCYwEyMzMLLyNlJGVBQ0aQJ8+YUfiksrXX1sRv1q1bIarNm1sbLVz5aTYpidV/THaLYZtZwN7RzxujF2sV3idd1R1o6quAj4G2u7om0gFixfD66/bgJTq1Ute3znWrYNBg6BDBxgzxp7r1s2ThCt38Rz+MBVoLiLNRKQq0Bt4rdA6rwJdRaSyiNTEmqbScj7uRx+1agqXXBJ2JC4pvPWWXVn9+ONw9dVw2mlhR+RSWCwlPEpFVXNE5DJgIjbR0VOqOldEBgbLR6jqfBF5B5gF5AKjVHVOvGJKVOvXW7/E6adDo0ZhR+MS3vXX26imli1tvohDCnf9OVe+YkoUIlIDaKKq3+7IxlX1LeCtQs+NKPT4AeCBHdluqnnmGWtF8CGxrliqkJtrp509elj75E03eRE/VyFKbHoSkROBmcA7weMMESnchORKKTcXhg6FTp2gc+ewo3EJ6aef4JRT4Pbb7fExx8A//+lJwlWYWPoo7sCuifgNQFVnAk3jFVC6mTgRFi70swlXBFV44glrYpo0yYbEOReCWJqeclT1d/GB/XGRlQV77mn9E85t98MPcMEFdnFN9+6WMPbfP+yoXJqK5Yxijoj0ASqJSHMRGQp8Hue40sKCBXZGccklULVq2NG4hLJhA8yaZaOa3n/fk4QLVSyJYjA2X/afwAtYufEr4xhT2hg61BLExReHHYlLCHPmwD332P2DD7YifgMGeBE/F7pY/gIPVNWbVbVjcLslqP3kyuC332y0U58+Xoon7W3ZYp3T7dvDf/5jE6UD1KwZblzOBWJJFA+LyAIRuVNEWsU9ojTx5JNWmufyy8OOxIVq6lS7svqOO+CMM7yIn0tIscxwd4SI7IFNYjRSROoA/1XVu+IeXYrats2uxO7aFdr59E/pa+NG6NkTatSA116DE08MOyLnihRT46eq/qyqQ4CB2DUVt8UzqFT3+us2M6UPiU1T06bZBTS1almV17lzPUm4hBbLBXcHicgdIjIHeBQb8dQ47pGlsKwsaNLEZrFzaeT3323kQseO+UX8DjsM6tYNNy7nShDLdRRPA2OBY1S1cPVXt4NmzYLJk+G++6By3CptuYTz+uswcCD8/DNce61fOOOSSix9FF5YohxlZVmT9IUXhh2JqzDXXQcPPmhDXl95xc4onEsixSYKEXlRVc8UkdkUnHAo1hnuXCGrVtmc2P36wa67hh2NiytVG7VQubLVZqpTx6q++pWVLglFO6PI62o9oSICSQcjR8Kff8LgwWFH4uIqO9sut2/TBu6+G44+2m7OJaloM9ytCO4OKmJ2u0EVE17q2LoVhg+Ho46y+WZcCsrNtZIbLVvCBx/AHnuEHZFz5SKW4bFFfRU6rrwDSXUTJli1aB8Sm6IWL4Yjj7QO606dYPZsP3V0KSNaH8Ul2JnDviIyK2JRbeCzeAeWarKyYL/9oFevsCNxcbFxo11VPWoU/N//gVdbdikkWh/FC8DbwL+BGyKeX6+qa+IaVYqZOhW++AIeecTru6WU2bPtgrlbbrERTT/+aEPanEsx0T62VFWXAJcC6yNuiIiP2dkBWVlQuzb07x92JK5c/Pkn3HabFfEbMiS/iJ8nCZeiSjqjOAGYjg2PjTyXVmDfOMaVMlasgBdftEEwdeqEHY0rsy+/tAmF5s2Dc8+1aq/164cdlXNxVWyiUNUTgp/NKi6c1DNiBOTkeL9mSti4EY4/3mo0vfUWHOdjOlx6iKXW099EpFZwv6+IPCwiTeIfWvL7809LFL16+QRlSe2rr/KL+L3+uhXx8yTh0kgsXauPAZtEpC3wD+BH4Lm4RpUixo2z5msfEpukfvvNaq107pxfxO/QQ63Dybk0EkuiyFFVBU4GslQ1Cxsi66JQtU7sli3tIjuXZF55xX55o0db6Y0zzgg7IudCE0v90vUiciNwLtBVRCoBVeIbVvL77DOYMcOannxIfZK5+mrrpG7b1pqaOnQIOyLnQhVLojgL6AP8n6r+HPRPPBDfsJJfVhbUqwd9+4YdiYtJZBG/Xr1sJNM//gFV/DuRcyU2Panqz8DzQF0ROQHYrKrPxj2yJLZ0Kbz8Mlx0kfV/ugS3dKmNZrr9dnt81FFw882eJJwLxDLq6UxgCnAGNm/2VyLis65EMXy4fUG99NKwI3FR5ebaL6tVK/joI9hrr7Ajci4hxdL0dDPQUVV/BRCRhsB7wPh4BpasNm2ycuKnngr77BN2NK5YixZZTaZPPrES4CNHQtOmYUflXEKKJVHslJckAquJbbRUWhozBtauhcsvDzsSF9XmzbBwITz9NJx/vo84cC6KWBLFOyIyEZs3G6xz+634hZS8VK30T0YGdO0adjTuL2bOtCJ+t98OrVvDkiVQvXrYUTmX8GLpzL4OeBxoA7QFRqrq9fEOLBl98IFdtHvFFf4FNaFs3myd05mZ8Nhj+UX8PEk4F5No81E0Bx4E9gNmA9eq6k8VFVgyysqChg2hd++wI3Hbff65FfFbsMCamB5+2Ccsd24HRTujeAp4AzgNqyA7tEIiSlLffw9vvAEXX+xfVBPGxo1w4ok2wuCdd+wqa08Szu2waH0UtVX1ieD+tyLydUUElKwefRQqVbJy4i5kX3wBhxxiF7G88Yb1R3h9JudKLdoZRXURaSci7UWkPVCj0OMSiUhPEflWRBaJyA1R1usoItuS9fqM9evhqafgzDN9KH6o1q61Ia+HHgrPBXUru3TxJOFcGUU7o1gBPBzx+OeIxwocGW3DQU2oYcDRQDYwVUReU9V5Rax3HzBxx0JPHKNHw7p1XiU2VBMm2BWOK1fCjTfCWWeFHZFzKSPaxEVHlHHbnYBFqroYQETGYRVo5xVabzDwP6BjGfcXitxcGDrUWjo6dQo7mjR11VU2IXlGhk0o1K5d2BE5l1JiuY6itBoByyIeZwOHRK4gIo2AU7Gzk2IThYgMAAYANGmSWHMmvfMOfPcdvPBC2JGkmcgifiecALvtBtde6/WZnIuDeF5hXdSVBFro8SPA9aq6LdqGVHWkqmaqambDhg3LK75ykZVl/RKnJ2XvSpJasgR69oRbb7XHPXpYc5MnCefiIp6JIhvYO+JxY2B5oXUygXEisgQ4HRguIqfEMaZyNX8+TJpkI538M6oC5LXztW5t10d4MS3nKkSJTU8iIsA5wL6q+q9gPoo9VHVKCS+dCjQXkWbAT0BvbF6L7VS1WcR+RgNvqOorO/QOQjRkCFSrZtdOuDj77jvo399mhOrZ02aE8kThXIWI5YxiONAFODt4vB4bzRSVquYAl2GjmeYDL6rqXBEZKCIDSxlvwli7Fp59Fvr0sauxXZxt2WJXNT77rHVYe5JwrsLE0pl9iKq2F5EZAKq6VkSqxrJxVX2LQgUEVXVEMev2i2WbieLJJ+2CX68SG0czZlgRvzvusDkjliyxUzjnXIWK5Yxia3Ctg8L2+Shy4xpVgsvJsSuxu3WzEZmunG3ebJ3THTvC44/btRHgScK5kMSSKIYALwO7icjdwKfAPXGNKsG9/jr8+KNfYBcXn34KbdvCvffCeefBvHnetudcyEpselLV50VkOtADG/J6iqrOj3tkCSwry5rITz457EhSzIYNdlDr1LHhZEcfHXZEzjliG/XUBNgEvB75nKoujWdgiWrmTJte+YEHrAigKweffmr1mXbeGd5804a/7rxz2FE55wKxND29iZUbfxN4H1gMvB3PoBLZkCFQs6ZNceDKaPVqa17q2jW/iF/nzp4knEswsTQ9HRz5OKgcm5ZXDqxcaaU6+veHXXYJO5okpgrjx8Nll8GaNXaFtc/25FzC2uFaT6r6tYgkZQG/sho5Ev7804fEltlVV1lHT4cO1hfRtm3YETnnooilj+LqiIc7Ae2BlXGLKEFt3QrDh8Mxx8BBB4UdTRJStXHFVarASSdZgayrr7aifs65hBZLH0XtiFs1rK8i7cb7jB8Py5f72USp/PCDZdi8In5HHgn/+IcnCeeSRNT/1OBCu51V9boKiidhDRkCzZvDcceFHUkS2bbNrky86SYbInbGGWFH5JwrhWIThYhUVtWcWKc9TWVTpsCXX1qy2Cme9XZTycKF0K+fzV993HF2hfXee5f4Mudc4ol2RjEF64+YKSKvAS8BG/MWquqEOMeWMLKybNrlfv3CjiSJ5OTY5etjxljlRClqehLnXDKIpZF4V2A1NgudYldnK5AWiWL5cnjxRZuOuXbtsKNJcNOmWRG/O++Eli1h8WKvz+RcCoiWKHYLRjzNIT9B5Ck8U13KGjHCmtoHDw47kgT2xx9w++3w0EOwxx7W49+woScJ51JEtBb3SsDOwa12xP28W8rbvNkSxQknwH77hR1NgvroI2jTxmqaXHABzJ3rRfycSzHRzihWqOq/KiySBDRunF2N7VVii7FhA/z971CvHrz/vg17dc6lnGiJIq17H1VtlFOrVv759xeffAJ/+5vVZHr7bTtItWqFHZVzLk6iNT31qLAoEtCnn9oEa5df7gN2tlu1Cvr2tRmb8or4derkScK5FFfsGYWqrqnIQBJNVhbsuqt9LqY9VRv6NXiwTRZ+++1exM+5NOI1FIrw44/w8stw3XVWUjztXXEFDB1qU5O+/z4cfHDJr3HOpQxPFEUYPtyamwYNCjuSEKlaJcSqVeHUU21Kvyuv9NmanEtDXpCikI0b4Ykn7LOxSZOwownJ999Djx5wyy32+Igj4JprPEk4l6Y8URQyZow1w6flkNht2+Dhh61pafp0OPDAsCNyziUAb3qKkDcktn17G/2ZVhYsgPPPtwqIJ54Ijz0GjRqFHZVzLgF4oojw/vswbx6MHp2GQ2Jzc62w1dixcNZZaXgAnHPF8UQRISsLdtstjUZ+TpliRfzuvtuK+H3/vXVeO+dcBO+jCCxaBG++CQMHpkEtu02b4NproUsXeOYZq1MCniScc0XyRBEYOtRm5hw4MOxI4uzDD62z+qGH4KKLvIifc65E3vQErFsHTz8NZ54Je+4ZdjRxtGGDTUdar54ljO7dw47IOZcE/IwC67xevz6Fh8ROnmyd1XlF/GbN8iThnItZ2ieK3FxrdurSxSpUpJSVK+Hss+2CuTFj7LmOHb0uiXNuh6R9onj7bevIvvzysCMpR6rwwgtw0EEwYYJNTZo2Q7mcc+Ut7fsosrLsurLTTgs7knI0eDAMGwadO8OTT9rQV+ecK6W0ThTz5sG779plBFWqhB1NGeXmQk6ODXE9/XTYf39LGF6fyTlXRnFtehKRniLyrYgsEpEbilh+jojMCm6fi0jbeMZT2JAhds3EgAEVudc4+O47m4bv5pvtcffuXunVOVdu4pYoRKQSMAw4DmgJnC0ihdtAfgAOV9U2wJ3AyHjFU9jatfDss3DOOdCgQUXttZzl5MCDD0KbNjBzpvVJOOdcOYtn01MnYJGqLgYQkXHAycC8vBVU9fOI9b8EGscxngJGjYI//kjiIbHz58N558G0aXDyyTaJxl57hR2Vcy4FxbPpqRGwLOJxdvBccS4A3i5qgYgMEJFpIjJtZV65iTLIyYFHH7UWmjZtyry58PzyC/z3vzYdnycJ51ycxDNRFFV+VItcUeQILFFcX9RyVR2pqpmqmtmwHMpNvPoqLF2ahGcTX34JN95o9w86yIr4nXmmV3p1zsVVPBNFNrB3xOPGwPLCK4lIG2AUcLKqro5jPNsNGQJNm9q0C0lh40a46io49FB4/vn8In5JP1TLOZcM4pkopgLNRaSZiFQFegOvRa4gIk2ACcC5qrowjrFsN3MmfPwxXHZZkgwKeu89aN0aHnnEJvH2In7OuQoWt85sVc0RkcuAiUAl4ClVnSsiA4PlI4DbgPrAcLHmkxxVzYxXTGAX2NWqBRdcEM+9lJMNG+yK6l13tezWtWvYETnn0pCoFtltkLAyMzN12rRppXrtr7/C3nvDhRfahcsJ64MP4PDD7ZRn+nS7srpGjbCjcs4lMRGZXtov4mlV62nkSNiyxS5YTki//GKd0z165Bfx69DBk4RzLlRpkyi2bLFLDY49Flq0CDuaQlThuefszCFvatI+fcKOyjnngDSq9TR+PKxYYTXyEs6ll8Jjj1mt8yef9CusnXMJJW0SxZAhcMABdkaREHJzYetWKzZ11lmWHAYNSpKhWM65dJIWTU9ffWW3wYNhp0R4x99+a53VeUX8Dj/cK7065xJWInxsxl1WFtSpA+efH3IgW7fCvfdC27YwZw4cfHDIATnnXMlSvunpp5/gpZfsC3vt2iEGMncunHsuzJgBf/+7jc/dY48QA3LOudikfKIYMQK2bbMrsUNVqRKsWWO96ik1nZ5zLtWldNPT5s3w+ONW02nffUMI4PPP4fqgzmGLFjY5tycJ51ySSelEMXas1c+r8CqxGzbA5ZfDYYdZGfBVq+z5yil/AuecS0EpmyhUrRO7dWs44ogK3PGkSbbTRx+19q45c5J4Cj3nnEvhPopPPoFvvrGyHRU2XcOGDTa3av36FsDf/lZBO3bOufhJ2TOKrCwrunrOORWws3fftR7znXe2M4qZMz1JOOdSRkomiiVL4JVXYMAAqFkzjjtascI6p485xiYUAmjXDqpXj+NOnXOuYqVkohg2zJqbBg2K0w5UYfRoK+L35pt2EZ0X8XPOpaiU66PYuBFGjbJr2vbeu+T1S+WSS2zc7WGH2c4OPDBOO3IuuW3dupXs7Gw2b94cdihpo3r16jRu3Jgq5ThVcsoliueeg99+i8OQ2Mgifn36QJs2MHBgghSPci4xZWdnU7t2bZo2bYpU2KiS9KWqrF69muzsbJo1a1Zu202pTzlVqxLboQMcemg5bnj+fJuG9Kab7HG3btau5UnCuag2b95M/fr1PUlUEBGhfv365X4Gl1KfdO+9Z5/pV1xRTkNit26Fe+6BjAxYsMA6qp1zO8STRMWKx/FOqaanrCzYfXebTbTM5s6Fvn1tqOsZZ8DQobZx55xLMylzRvHddzYAaeBA60Yos8qV4fffYcIEePFFTxLOJbGXX34ZEWHBggXbn5s8eTInnHBCgfX69evH+PHjAeuIv+GGG2jevDmtW7emU6dOvP3222WO5d///jf7778/Bx54IBMnTixynbPOOouMjAwyMjJo2rQpGRkZ25fNmjWLLl260KpVKw4++OAKGSiQMmcUQ4dClSqWKErtk09szuoHH7SRTAsXen0m51LA2LFjOeywwxg3bhx33HFHTK+59dZbWbFiBXPmzKFatWr88ssvfPTRR2WKY968eYwbN465c+eyfPlyjjrqKBYuXEilQpOW/fe//91+/5prrqFu3boA5OTk0LdvX5577jnatm3L6tWry3V0U3FS4lNw3Tp4+mno3buUUzysXw833ADDh0OzZna/QQNPEs6VoyuvtJbc8pSRAY88En2dDRs28Nlnn/Hhhx9y0kknxZQoNm3axBNPPMEPP/xAtaCJYvfdd+fMMrZrv/rqq/Tu3Ztq1arRrFkz9t9/f6ZMmUKXLl2KXF9VefHFF/nggw8AmDRpEm3atKFt27YA1K9fv0zxxColmp6efjq/YOsOe/ttaNUKHnvM/pJnz/Yifs6lkFdeeYWePXtywAEHsOuuu/L111+X+JpFixbRpEkT6tSpU+K6V1111fZmosjbvffe+5d1f/rpJ/aOuMCrcePG/PTTT8Vu+5NPPmH33XenefPmACxcuBAR4dhjj6V9+/bcf//9JcZXHpL+K/O2bdbsdOihkJm5gy9evx7OOw92283mjujcOS4xOudK/uYfL2PHjuXKK68EoHfv3owdO5b27dsXOzpoR0cN/ec//4l5XVXdof2NHTuWs88+e/vjnJwcPv30U6ZOnUrNmjXp0aMHHTp0oEePHjsU845K+kTx1lvw/fc2ijUmqjBxIhx9tM2N+t57NqlQufSAO+cSyerVq/nggw+YM2cOIsK2bdsQEe6//37q16/P2rVrC6y/Zs0aGjRowP7778/SpUtZv349tUuYQ/mqq67iww8//MvzvXv35oYbbijwXOPGjVm2bNn2x9nZ2ey1115FbjcnJ4cJEyYwffr0Aq8//PDDaRC0evTq1Yuvv/467okCVU2qW4cOHTTSUUepNm6sumWLlmz5ctVTTlEF1WeeieEFzrmymDdvXqj7HzFihA4YMKDAc926ddOPP/5YN2/erE2bNt0e45IlS7RJkyb622+/qarqddddp/369dM///xTVVWXL1+uzz33XJnimTNnjrZp00Y3b96sixcv1mbNmmlOTk6R67799tvarVu3As+tWbNG27Vrpxs3btStW7dqjx499I033vjLa4s67sA0LeXnblL3UcydaycEgwbZiKdiqcJTT8FBB8E778D993sRP+fSwNixYzn11FMLPHfaaafxwgsvUK1aNcaMGUP//v3JyMjg9NNPZ9SoUdtHGN111100bNiQli1b0rp1a0455RQaNmxYpnhatWrFmWeeScuWLenZsyfDhg3bPuLpwgsvZNq0advXHTduXIFmJ4BddtmFq6++mo4dO5KRkUH79u05/vjjyxRTLESLaDNLZJmZmZp3MC++GJ59FrKzba6gYl18sc1g1K2bFfELOoacc/E1f/58DjrooLDDSDtFHXcRma6qO9qTCyRxH8WaNVYAsG/fYpLEtm1WgqN6dVupXTuboMLrMznn3A5J2k/NUaPgjz+KGRI7d67NMJdXxK9rV6/06pxzpZSUn5w5OfDoo3DEEXDwwRELtmyBO++0s4dFi6Bjx9BidM6ZZGveTnbxON5J2fT0yiuwbJldP7Hd7Nk2Qfbs2XaJ9pAhUMaOJ+dc2VSvXp3Vq1d7qfEKosF8FNXLeTrmpEwUWVlWaaNAPa+qVWHTJqvVdNJJocXmnMvXuHFjsrOzWblyZdihpI28Ge7KU9Ilik2bYPp0ePhhqPTpR/Daa/DQQ1bE79tvoVBxLedceKpUqVKuM625cMS1j0JEeorItyKySERuKGK5iMiQYPksEWlf0jZ//RX2qLmOQbMvge7drR1q1Spb6EnCOefKXdwShYhUAoYBxwEtgbNFpGWh1Y4Dmge3AcBjJW03Z/XvzNupFdWeGQlXX+1F/JxzLs7ieUbRCVikqotVdQswDji50DonA88GV5h/CdQTkT2jbbQpS6i5Z10r4vfQQ1CzZnyid845B8S3j6IRsCzicTZwSAzrNAJWRK4kIgOwMw6AP6t/N3eOV3oFoAGwKuwgEoQfi3x+LPL5sch3YGlfGM9EUdRYuMIDfGNZB1UdCYwEEJFppb0MPdX4scjnxyKfH4t8fizyici0ktcqWjybnrKBvSMeNwaWl2Id55xzIYpnopgKNBeRZiJSFegNvFZondeA84LRT52B31V1ReENOeecC0/cmp5UNUdELgMmApWAp1R1rogMDJaPAN4CegGLgE1A/xg2PTJOIScjPxb5/Fjk82ORz49FvlIfi6QrM+6cc65iJWVRQOeccxXHE4VzzrmoEjZRxKP8R7KK4VicExyDWSLyuYi0DSPOilDSsYhYr6OIbBOR0ysyvooUy7EQke4iMlNE5orIRxUdY0WJ4X+kroi8LiLfBMcilv7QpCMiT4nIryIyp5jlpfvcLO1k2/G8YZ3f3wP7AlWBb4CWhdbpBbyNXYvRGfgq7LhDPBaHArsE949L52MRsd4H2GCJ08OOO8S/i3rAPKBJ8Hi3sOMO8VjcBNwX3G8IrAGqhh17HI5FN6A9MKeY5aX63EzUM4q4lP9IUiUeC1X9XFXXBg+/xK5HSUWx/F0ADAb+B/xakcFVsFiORR9ggqouBVDVVD0esRwLBWqLTYqxM5Yocio2zPhT1Y+x91acUn1uJmqiKK60x46ukwp29H1egH1jSEUlHgsRaQScCoyowLjCEMvfxQHALiIyWUSmi8h5FRZdxYrlWDwKHIRd0DsbuEJVcysmvIRSqs/NRJ2PotzKf6SAmN+niByBJYrD4hpReGI5Fo8A16vqthSfUS2WY1EZ6AD0AGoAX4jIl6q6MN7BVbBYjsWxwEzgSGA/4F0R+URV18U5tkRTqs/NRE0UXv4jX0zvU0TaAKOA41R1dQXFVtFiORaZwLggSTQAeolIjqq+UiERVpxY/0dWqepGYKOIfAy0BVItUcRyLPoD96o11C8SkR+AFsCUigkxYZTqczNRm568/Ee+Eo+FiDQBJgDnpuC3xUglHgtVbaaqTVW1KTAeGJSCSQJi+x95FegqIpVFpCZWvXl+BcdZEWI5FkuxMytEZHeskuriCo0yMZTqczMhzyg0fuU/kk6Mx+I2oD4wPPgmnaMpWDEzxmORFmI5Fqo6X0TeAWYBucAoVS1y2GQyi/Hv4k5gtIjMxppfrlfVlCs/LiJjge5AAxHJBm4HqkDZPje9hIdzzrmoErXpyTnnXILwROGccy4qTxTOOeei8kThnHMuKk8UzjnnovJE4RJSUPl1ZsStaZR1N5TD/kaLyA/Bvr4WkS6l2MYoEWkZ3L+p0LLPyxpjsJ284zInqIZar4T1M0SkV3ns26UvHx7rEpKIbFDVnct73SjbGA28oarjReQY4EFVbVOG7ZU5ppK2KyLPAAtV9e4o6/cDMlX1svKOxaUPP6NwSUFEdhaR94Nv+7NF5C9VY0VkTxH5OOIbd9fg+WNE5IvgtS+JSEkf4B8D+wevvTrY1hwRuTJ4rpaIvBnMbTBHRM4Knp8sIpkici9QI4jj+WDZhuDnfyO/4QdnMqeJSCUReUBEporNE3BxDIflC4KCbiLSSWwukhnBzwODq5T/BZwVxHJWEPtTwX5mFHUcnfuLsOun+81vRd2AbVgRt5nAy1gVgTrBsgbYlaV5Z8Qbgp/XADcH9ysBtYN1PwZqBc9fD9xWxP5GE8xdAZwBfIUV1JsN1MJKU88F2gGnAU9EvLZu8HMy9u19e0wR6+TFeCrwTHC/KlbJswYwALgleL4aMA1oVkScGyLe30tAz+BxHaBycP8o4H/B/X7AoxGvvwfoG9yvh9V9qhX279tviX1LyBIezgF/qGpG3gMRqQLcIyLdsHIUjYDdgZ8jXjMVeCpY9xVVnSkihwMtgc+C8iZVsW/iRXlARG4BVmJVeHsAL6sV1UNEJgBdgXeAB0XkPqy56pMdeF9vA0NEpBrQE/hYVf8ImrvaSP6MfHWB5sAPhV5fQ0RmAk2B6cC7Ees/IyLNsWqgVYrZ/zHASSJybfC4OtCE1KwB5cqJJwqXLM7BZibroKpbRWQJ9iG3nap+HCSS44HnROQBYC3wrqqeHcM+rlPV8XkPROSoolZS1YUi0gGrmfNvEZmkqv+K5U2o6mYRmYyVvT4LGJu3O2Cwqk4sYRN/qGqGiNQF3gAuBYZgtYw+VNVTg47/ycW8XoDTVPXbWOJ1DryPwiWPusCvQZI4Atin8Aoisk+wzhPAk9iUkF8CfxORvD6HmiJyQIz7/Bg4JXhNLazZ6BMR2QvYpKpjgAeD/RS2NTizKco4rBhbV6yQHcHPS/JeIyIHBPsskqr+DlwOXBu8pi7wU7C4X8Sq67EmuDwTgcESnF6JSLvi9uFcHk8ULlk8D2SKyDTs7GJBEet0B2aKyAysHyFLVVdiH5xjRWQWljhaxLJDVf0a67uYgvVZjFLVGcDBwJSgCehm4K4iXj4SmJXXmV3IJGxu4/fUpu4Em0tkHvC1iMwBHqeEM/4glm+wstr3Y2c3n2H9F3k+BFrmdWZjZx5VgtjmBI+di8qHxzrnnIvKzyicc85F5YnCOedcVJ4onHPOReWJwjnnXFSeKJxzzkXlicI551xUniicc85F9f83TsNPiAeZ2QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.76      0.79      0.77      5257\n",
      "           1       0.77      0.73      0.75      5055\n",
      "\n",
      "    accuracy                           0.76     10312\n",
      "   macro avg       0.76      0.76      0.76     10312\n",
      "weighted avg       0.76      0.76      0.76     10312\n",
      "\n",
      "\n",
      "Confusion Matrix: \n",
      " [[4176 1081]\n",
      " [1355 3700]]\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/bettyau/opt/anaconda3/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function plot_confusion_matrix is deprecated; Function `plot_confusion_matrix` is deprecated in 1.0 and will be removed in 1.2. Use one of the class methods: ConfusionMatrixDisplay.from_predictions or ConfusionMatrixDisplay.from_estimator.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "End of validation Result\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAT8AAAEGCAYAAAAT05LOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAblUlEQVR4nO3de5RU5Znv8e+vmzsCchFsAQdMiEqceEPE8SSRmAwkuoLmxHUwRlmJc0g85jKT25GZWZNJsvC4nGSSMV4mJHHEyajBkxgwibdBjckZlYsaFQyBiMEWlItRQBTo7uf8UW9jCd3VtaGrq7r277PWXr33W/vyVAMP77vf/b5bEYGZWd40VDsAM7NqcPIzs1xy8jOzXHLyM7NccvIzs1zqU+0Aio0a0RgTxvetdhiWwe+fHFTtECyDN3iNPbFbh3KOGdMHx7aXW8vad+WTu++JiJmHcr1KqankN2F8X5bdM77aYVgGM446qdohWAaPxtJDPsfWl1t59J5xZe3bt+kPow75ghVSU8nPzHqDoDXaqh3EIXPyM7NMAmij9w+OcPIzs8zacM3PzHImCPa62WtmeRNAq5u9ZpZHvudnZrkTQGsdzAbl5GdmmfX+O35OfmaWURC+52dm+RMBe3t/7nPyM7OsRCuHNDy4Jjj5mVkmAbS55mdmeeSan5nlTuEhZyc/M8uZAPZG758H2cnPzDIJRGsdTALf+7+BmfW4tlBZS1ckPSfpKUlPSFqRykZIuk/S2vRzeNH+8yStk7RG0oyi8lPTedZJukZSlxd38jOzTNrv+ZWzlGl6RJwUEVPS9hXA0oiYBCxN20iaDMwG3gnMBK6X1JiOuQGYC0xKS5dT5zv5mVlGojUayloO0ixgYVpfCJxXVH5bROyOiPXAOmCqpCZgaEQ8HBEB3Fx0TKec/Mwsk8JMzg1lLcAoSSuKlrkdnO5eSSuLPhsTEZsA0s/RqXws8HzRsc2pbGxa37+8JHd4mFkmEWJPNHa9Y8HWouZsR86MiI2SRgP3SfpdiX07akdHifKSXPMzs8zaUFlLVyJiY/q5GbgDmAq8lJqypJ+b0+7NQPHrHccBG1P5uA7KS3LyM7NMCh0eDWUtpUgaLGlI+zrwl8DTwBJgTtptDrA4rS8BZkvqL2kihY6NZalpvEPStNTLe0nRMZ1ys9fMMtKhdGYUGwPckZ5K6QPcEhF3S1oOLJJ0KbABuAAgIlZJWgSsBlqAyyOi/e3plwE3AQOBu9JSkpOfmWXS3uFxyOeJeBY4sYPybcDZnRwzH5jfQfkK4IQs13fyM7PMWst4gLnWOfmZWSaB2Bu9P3X0/m9gZj2qvcOjt3PyM7NMArnZa2b51B0dHtXm5GdmmUTQXY+6VJWTn5llUujwKHt4W81y8jOzzNzhYWa5E5Q3UWmtc/Izs8xc8zOz3Cm8t9fJz8xyJ9MU9TXLyc/MMim8utK9vWaWMxFys9fM8skPOZtZ7hTm8/M9PzPLnW6bybmqnPzMLJPCoy6u+ZlZznhsr5nllqe0MrPcKUxp5WavmeWQ7/mZWe4UZnVxs9fMcqYwvM3JL7cumTqZgYe10tAAjX2Ca+/+Pf/+zSO565YRDBtReIn8J+ZtZOrZO/jd44P4ly+PBwp/cS7+4ouc+cFXAfi3q47kP28fwc5XG1m87qlqfR1Lppy1nU9/YyONDcFdt45g0bVjqh1SDXLNr0uSZgL/AjQCP4iIqyp5vZ529e3rGDay9S1l5//PLVxw2Za3lE049nWuvXsNjX1g20t9uOz9xzLtA6/S2AemfWA7H/7EVj555vE9Gbp1oKEhuPzKF5g3+xi2burLd3+5lkfuGcaGtQOqHVrN8QiPEiQ1AtcBHwCageWSlkTE6kpds1YNGBT71vfubkBFf2+OP3VXFSKyjhx78i42PtePFzf0B+DBxYdzxoxXnfz2497erk0F1kXEswCSbgNmAfWR/BT87YVvA8E5F2/jQx/fBsCd/3YES//vCCa9axdzv7qRIYcXaoa/e2wQ3/rCeDY39+Mr391Ao2841JyRR+5ly8Z++7a3burLcaf4P6eOuNlb2ljg+aLtZuD0/XeSNBeYC3D02N6TEb69eC0jj2zhla19uGL22xj/9jc4d85WPvY3LyLBwquPZMHXjuKL3y78Co47ZRfff3ANG9b2558+fzSnTd9OvwHRxVWsJ6mDykz4j+gA9fIOj0qm745+Owf8VYqIBRExJSKmHDGy9wyZGXlkCwCHj2rhzJmv8rvHBzH8iBYaG6GhAT540cuseWLQAccdPWk3Awa18dwaN6VqzdZNfTniqD37tkc17WXbi32rGFFtCqAlGspaalklo2sGxhdtjwM2VvB6PeaNXQ3s2tmwb33lr4Yw4bg32PbSmzXX/7prGBOOfQOAFzf0o7WQK3mpuS/NfxjAmHF7DjivVdeaJwYxduIexozfTZ++bZw16xUeuXdYtcOqSW3RUNZSDkmNkh6X9PO0PULSfZLWpp/Di/adJ2mdpDWSZhSVnyrpqfTZNVJH9fi3qmQ7czkwSdJE4AVgNvCxCl6vx/xpSx++dulEAFpbYPr5r3Da9B1c/dmj+cOqgUgwZtwePnd1ocn79LLB/PjaifTpU+hR/OyVzft6iX/wjSYe+Nlwdr/ewEWnTmbmhS9z8ZderNp3y7O2VnHd343lyluepaER7r1tBH/8vWvoB4hub/Z+HngGGJq2rwCWRsRVkq5I2/9b0mQKeeSdwFHAf0p6R0S0AjdQuH32CPBLYCZwV6mLKip4U0PSh4DvUHjU5caImF9q/yknDohl94wvtYvVmBlHnVTtECyDR2Mp2+PlQ8pcw48bHe+78aNl7fvTM29YGRFTOvtc0jhgITAf+EJEnCtpDXBWRGyS1AQ8GBHHSpoHEBH/Jx17D/CPwHPAAxFxXCq/MB3/qVKxVbSHISJ+SSELm1kd6caa33eArwBDisrGRMQmgJQAR6fysRRqdu2aU9netL5/eUm1fUfSzGpO+2Sm5SzAKEkripa57eeRdC6wOSJWlnnpzjpRy+pc3V/vebbEzGpCIFrayq43bS3R7D0T+HC6PTYAGCrpR8BLkpqKmr2b0/6ddaI2p/X9y0tyzc/MMmtDZS2lRMS8iBgXERModGTcHxEfB5YAc9Juc4DFaX0JMFtS/9SROglYlprIOyRNS728lxQd0ynX/Mwsm6j4fH5XAYskXQpsAC4AiIhVkhZRGCXWAlyeenoBLgNuAgZS6OUt2dMLTn5mllElXmAUEQ8CD6b1bcDZnew3n0LP8P7lK4ATslzTyc/MMquH4W1OfmaWSSBay+/wqFlOfmaWmefzM7Pcicp3ePQIJz8zyyyc/Mwsf+pjPj8nPzPLzDU/M8udCGhtc/Izsxxyb6+Z5U7gZq+Z5ZI7PMwsp+rhrXZOfmaWmZu9ZpY7hd5ej+01sxxys9fMcsnNXjPLnUBOfmaWT3XQ6nXyM7OMAsLD28wsj9zsNbNcquveXknfpUTTPiI+V5GIzKym5WFs74oei8LMeo8A6jn5RcTC4m1JgyPitcqHZGa1rh6avV2OUZF0hqTVwDNp+0RJ11c8MjOrUSLayltqWTkD9L4DzAC2AUTEb4H3VDAmM6t1UeZSw8rq7Y2I56W3ZPHWyoRjZjUv6r/Do93zkv4CCEn9gM+RmsBmllM1XqsrRznN3k8DlwNjgReAk9K2meWWylxqV5c1v4jYClzUA7GYWW/RVu0ADl05vb3HSLpT0hZJmyUtlnRMTwRnZjWo/Tm/cpYaVk6z9xZgEdAEHAXcDtxayaDMrLZFlLeUImmApGWSfitplaSvpfIRku6TtDb9HF50zDxJ6yStkTSjqPxUSU+lz67Rfj20HSkn+Ski/j0iWtLyI+ridqeZHbTuedRlN/C+iDiRQl/CTEnTgCuApRExCViatpE0GZgNvBOYCVwvqTGd6wZgLjApLTO7uninyS9l3xHAA5KukDRB0p9J+grwiy6/lpnVr25o9kbBzrTZNy0BzALaR5gtBM5L67OA2yJid0SsB9YBUyU1AUMj4uGICODmomM6VarDY2UKpP0bfKo4buAbXZ3czOqTym/7jZJUPE/AgohYsO88hZrbSuDtwHUR8aikMRGxCSAiNkkanXYfCzxSdK7mVLY3re9fXlKpsb0TuzrYzHIoBOUPXdsaEVM6PVVEK3CSpMOBOySdUOJcHV00SpSXVNYIjxTQZGDAvjNH3FzOsWZWh7r5rn9EvCLpQQr36l6S1JRqfU3A5rRbMzC+6LBxwMZUPq6D8pLKedTlq8B30zIduBr4cJffxszqVzd0eEg6ItX4kDQQeD/wO2AJMCftNgdYnNaXALMl9Zc0kULHxrLURN4haVrq5b2k6JhOlVPz+yhwIvB4RHxC0hjgB2UcZ2b1qntqfk3AwnTfrwFYFBE/l/QwsEjSpcAG4AKAiFglaRGwGmgBLk/NZoDLgJuAgcBdaSmpnOT3ekS0SWqRNJRCFdQPOZvlVTdNZhoRTwInd1C+DTi7k2PmA/M7KF8BlLpfeIBykt+KVDX9PoVemZ3AsiwXMbP6kqG3t2aVM7b3f6XVf5V0N4XnaZ6sbFhmVtPqOflJOqXUZxHxWGVCMrNaV+81v2+V+CyA93VzLPz+yUHMOOqk7j6tVdAn1vyx2iFYBn/4yO7uOVGNT1pQjlIPOU/vyUDMrJfoBVPUl8MvLTez7Jz8zCyPVAeTmTr5mVl2dVDzK2d4myR9XNI/pO2jJU2tfGhmVosU5S+1rJzJTK8HzgAuTNs7gOsqFpGZ1b46mMa+nGbv6RFxiqTHASLiT+kVlmaWVzVeqytHOclvbxp4HFCYiYG6eHeTmR2sWm/SlqOc5HcNcAcwWtJ8CrO8/H1FozKz2hU56e2NiP+QtJLCLAsCzouIZyoemZnVrjzU/CQdDewC7iwui4gNlQzMzGpYHpIfhTe1tc+TPwCYCKyh8Po4M8uhXNzzi4g/L95Os718qpPdzcx6hcwjPCLiMUmnVSIYM+sl8lDzk/SFos0G4BRgS8UiMrPalpfeXmBI0XoLhXuAP6lMOGbWK9R7zS893HxYRHy5h+Ixsxon6rzDQ1KfiGgpNZ29meVUPSc/Cm9oOwV4QtIS4HbgtfYPI+KnFY7NzGpRL5ixpRzl3PMbAWyj8M6O9uf9AnDyM8urOu/wGJ16ep/mzaTXrg7yvpkdrHqv+TUCh/HWpNeuDr66mR20OsgApZLfpoj4eo9FYma9Qw7e3lbb07CaWdXUe7P37B6Lwsx6l3pOfhHxck8GYma9Rz0MbyvnBUZmZm+KDEsJksZLekDSM5JWSfp8Kh8h6T5Ja9PP4UXHzJO0TtIaSTOKyk+V9FT67BpJXd62c/Izs0yUYelCC/DFiDgemAZcLmkycAWwNCImAUvTNumz2RTmEp0JXJ+G4ALcAMwFJqVlZlcXd/Izs+y6oeYXEZsi4rG0vgN4BhgLzAIWpt0WAuel9VnAbRGxOyLWA+uAqZKagKER8XBEBHBz0TGdyjyfn5lZht7eUZJWFG0viIgFB5xPmgCcDDwKjImITVBIkJJGp93GAo8UHdacyvam9f3LS3LyM7Psyk9+WyNiSqkdJB1GYZq8v46I7SVu13U24OKgBmK42Wtm2aTJTMtZuiKpL4XE9x9Fk6W8lJqypJ+bU3kzML7o8HHAxlQ+roPykpz8zCy77untFfBD4JmI+Oeij5YAc9L6HGBxUflsSf0lTaTQsbEsNZF3SJqWznlJ0TGdcrPXzDLrphEeZwIXA09JeiKV/S1wFbBI0qXABuACgIhYJWkRsJpCT/HlEdGajrsMuAkYCNyVlpKc/Mwsu25IfhHxGzp/IqbDEWYRMR+Y30H5CuCELNd38jOzzOp9bK+Z2YGCup/M1MzsAHX/AiMzs045+ZlZHil6f/Zz8jOzbHIwk7OZWYd8z8/McqkeJjN18jOz7FzzM7PcCTd7zSyvnPzMLG/8kLOZ5Zbaen/2c/Izs2z8nJ91ZcpZ2/n0NzbS2BDcdesIFl07ptoh5VpbK9z535sYNKaFD3xvCw/89Si2r+8LwJ4dDfQb0sasxZto2wu/+fuRbFvdj2gRbz9vJ+/61HYAVn77cNb9bDB7tjdw8ePPV/PrVJUfdSlB0o3AucDmiMg0z1Y9aGgILr/yBebNPoatm/ry3V+u5ZF7hrFh7YBqh5Zbq28ewuFv28uenYUp5KZ/Z+u+z5ZdNZx+hxX+Ra+/exBte8T5d26i5XVxxzlHMfGc1xgyrpXx03dx/EU7+MmMo6ryHWpGHdT8KjmN/U2U8e7MenXsybvY+Fw/XtzQn5a9DTy4+HDOmPFqtcPKrddebKT5wYFM+ujOAz6LgPV3DWLiua8BIEHL66KtBVreEA19g36HFf61jz5pD4NGtx5wjrxRlLfUsoolv4h4CHi5UuevdSOP3MuWjf32bW/d1JdRTXurGFG+PXrlcKZ8+RXUcOC/yJdW9GfgyFaGTWgBYMKMXfQZGNz238Zx+/SxnPDJ7fQ/vA7aed0lKPyPUc5Sw6r+AiNJcyWtkLRiL7urHU636ejtezX+d6FuPf/AQAaOaGPUCXs6/PzZnw/mmFTrA9jyZH/UALN/3cxHl77A0zcOZcfzvj1erLve3lZNVf8TTS8wXgAwVCPqJj1s3dSXI4568x/bqKa9bHuxbxUjyq+XHuvPhvsH0vzQWFp3iz07xa++NJL3fnMbbS3wx/sG8eGfbtq3/7M/H8zYd79OQ18YOLKNMafsZutT/RgyvqWK36J21MtzflWv+dWrNU8MYuzEPYwZv5s+fds4a9YrPHLvsGqHlUtTvvgK/+OhF7jg/hd47z9voWnaG7z3m9sA2PhfAxh2zF4GH/nmfbzBTS1senQAEbB3l9j8234MO8a3LPYpt8lb402dqtf86lVbq7ju78Zy5S3P0tAI9942gj/+3j29tWb9LwdzzDmvvaXs+It28Jt5I/nZuU1EwKSPvMaI4wrJb/nVh/PszwfT8rr48XvG8o4LdnLyZ/PXkVUPNb9KPupyK3AWMEpSM/DViPhhpa5Xi5bfP5Tl9w+tdhhWpOn03TSdvmXf9ruv2nbAPn0HB9Ov2XpAOcBpX3mF077ySqXC6z2c/DoXERdW6txmVl2u+ZlZ/gTQ2vuzn5OfmWXmmp+Z5VON9+SWw8nPzDJzzc/M8sdTWplZHglQHXR4eISHmWWmiLKWLs8j3Shps6Sni8pGSLpP0tr0c3jRZ/MkrZO0RtKMovJTJT2VPrtG6mh0/Vs5+ZlZNpFh6dpNHDj13RXA0oiYBCxN20iaDMwG3pmOuV5SYzrmBmAuMCktXU6n5+RnZhl139jeTqa+mwUsTOsLgfOKym+LiN0RsR5YB0yV1AQMjYiHIyKAm4uO6ZTv+ZlZZhl6e0dJWlG0vSDN5FTKmIjYBBARmySNTuVjgUeK9mtOZXvT+v7lJTn5mVl25T/ntzUipnTTVTu6jxclykty8jOzbKLivb0vSWpKtb4mYHMqbwbGF+03DtiYysd1UF6S7/mZWXbd1+HRkSXAnLQ+B1hcVD5bUn9JEyl0bCxLTeQdkqalXt5Lio7plGt+ZpZZOY+xlHWeDqa+A64CFkm6FNgAXAAQEaskLQJWAy3A5RHRPgvtZRR6jgcCd6WlJCc/M8uum5Jfianvzu5k//nA/A7KVwCZXpHr5Gdm2QRQ4y8nKoeTn5llIsobvVHrnPzMLLu23l/1c/Izs2zc7DWzvHKz18zyycnPzPKn9l9IXg4nPzPLxm9vM7O88j0/M8snJz8zy50A2pz8zCx33OFhZnnl5GdmuRNAa+8f4uHkZ2YZBYSTn5nlkZu9ZpY77u01s9xyzc/McsnJz8xyJwJaW7ver8Y5+ZlZdq75mVkuOfmZWf6Ee3vNLIcCwg85m1kueXibmeVOhF9daWY55Q4PM8ujcM3PzPLHk5maWR55YgMzy6MAog6GtzVUOwAz62UiTWZaztIFSTMlrZG0TtIVPRD9Pq75mVlm0Q3NXkmNwHXAB4BmYLmkJRGx+pBPXgbX/Mwsu+6p+U0F1kXEsxGxB7gNmFXx2BNFDfXaSNoC/LHacVTAKGBrtYOwTOr1z+zPIuKIQzmBpLsp/H7KMQB4o2h7QUQsSOf5KDAzIv4qbV8MnB4RnzmU+MpVU83eQ/1DqVWSVkTElGrHYeXzn1nnImJmN51KHZ2+m87dJTd7zaxamoHxRdvjgI09dXEnPzOrluXAJEkTJfUDZgNLeuriNdXsrWMLqh2AZeY/swqLiBZJnwHuARqBGyNiVU9dv6Y6PMzMeoqbvWaWS05+ZpZLTn4VVM2hO3ZwJN0oabOkp6sdi1WWk1+FFA3d+SAwGbhQ0uTqRmVluAnorufYrIY5+VVOVYfu2MGJiIeAl6sdh1Wek1/ljAWeL9puTmVmVgOc/CqnqkN3zKw0J7/KqerQHTMrzcmvcqo6dMfMSnPyq5CIaAHah+48AyzqyaE7dnAk3Qo8DBwrqVnSpdWOySrDw9vMLJdc8zOzXHLyM7NccvIzs1xy8jOzXHLyM7NccvLrRSS1SnpC0tOSbpc06BDOdVN6exaSflBq0gVJZ0n6i4O4xnOSDnjLV2fl++2zM+O1/lHSl7LGaPnl5Ne7vB4RJ0XECcAe4NPFH6aZZDKLiL/q4kXRZwGZk59ZLXPy671+Dbw91coekHQL8JSkRkn/JGm5pCclfQpABddKWi3pF8Do9hNJelDSlLQ+U9Jjkn4raamkCRSS7N+kWue7JR0h6SfpGsslnZmOHSnpXkmPS/oeHY9vfgtJP5O0UtIqSXP3++xbKZalko5IZW+TdHc65teSjuuW36bljl9g1AtJ6kNhnsC7U9FU4ISIWJ8SyKsRcZqk/sD/k3QvcDJwLPDnwBhgNXDjfuc9Avg+8J50rhER8bKkfwV2RsQ30363AN+OiN9IOprCKJbjga8Cv4mIr0s6B3hLMuvEJ9M1BgLLJf0kIrYBg4HHIuKLkv4hnfszFF4s9OmIWCvpdOB64H0H8Wu0nHPy610GSnoirf8a+CGF5uiyiFifyv8SeFf7/TxgGDAJeA9wa0S0Ahsl3d/B+acBD7WfKyI6m9fu/cBkaV/FbqikIekaH0nH/kLSn8r4Tp+TdH5aH59i3Qa0AT9O5T8CfirpsPR9by+6dv8yrmF2ACe/3uX1iDipuCAlgdeKi4DPRsQ9++33IbqeUktl7AOF2yVnRMTrHcRS9nhJSWdRSKRnRMQuSQ8CAzrZPdJ1X9n/d2B2MHzPr/7cA1wmqS+ApHdIGgw8BMxO9wSbgOkdHPsw8F5JE9OxI1L5DmBI0X73UmiCkvY7Ka0+BFyUyj4IDO8i1mHAn1LiO45CzbNdA9Bee/0Yheb0dmC9pAvSNSTpxC6uYdYhJ7/68wMK9/MeSy/h+R6FGv4dwFrgKeAG4Ff7HxgRWyjcp/uppN/yZrPzTuD89g4P4HPAlNShspo3e52/BrxH0mMUmt8buoj1bqCPpCeBbwCPFH32GvBOSSsp3NP7eiq/CLg0xbcKvxrADpJndTGzXHLNz8xyycnPzHLJyc/McsnJz8xyycnPzHLJyc/McsnJz8xy6f8DrKKHaK1Yjv8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# classification report\n",
    "clf_score(model3, X_train, y_train, X_test, y_test, train=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff743252",
   "metadata": {},
   "source": [
    "### 5.4.4 Gradient Boosting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "3231232e",
   "metadata": {},
   "outputs": [],
   "source": [
    "gbt = GradientBoostingClassifier(random_state = 5)\n",
    "model4 = gbt.fit(X_train_sc, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "fb1b8710",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.7190651667959659\n"
     ]
    }
   ],
   "source": [
    "# cross-validation\n",
    "y_test_pred = model4.predict(X_test_sc)\n",
    "accuracy_gbt=metrics.accuracy_score(y_test, y_test_pred)\n",
    "print(accuracy_gbt)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "daab26e1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>predictor</th>\n",
       "      <th>feature importance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>timeOnIce</td>\n",
       "      <td>3.209036e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>goals</td>\n",
       "      <td>2.643736e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>powerPlayGoals</td>\n",
       "      <td>8.172163e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td>goalieReplacement_No</td>\n",
       "      <td>7.042770e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>pim</td>\n",
       "      <td>6.177498e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td>goalieReplacement_Yes</td>\n",
       "      <td>6.076237e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td>settledIn_OT</td>\n",
       "      <td>4.168470e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td>shortHandedTimeOnIce</td>\n",
       "      <td>3.423580e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td>powerPlayTimeOnIce</td>\n",
       "      <td>1.626676e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td>hoa_home</td>\n",
       "      <td>1.360430e-02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>hits</td>\n",
       "      <td>6.555537e-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td>settledIn_REG</td>\n",
       "      <td>5.828775e-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>penalties</td>\n",
       "      <td>5.296913e-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>hoa_away</td>\n",
       "      <td>4.793632e-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>shots</td>\n",
       "      <td>3.999431e-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>blockedShots</td>\n",
       "      <td>2.834474e-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>faceOffWinPercentage</td>\n",
       "      <td>1.964256e-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>evenTimeOnIce</td>\n",
       "      <td>1.823543e-03</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>missedShots</td>\n",
       "      <td>6.000243e-04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>takeaways</td>\n",
       "      <td>5.477219e-04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>startRinkSide_right</td>\n",
       "      <td>2.292851e-07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td>giveaways</td>\n",
       "      <td>0.000000e+00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td>startRinkSide_left</td>\n",
       "      <td>0.000000e+00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                predictor  feature importance\n",
       "11              timeOnIce        3.209036e-01\n",
       "4                   goals        2.643736e-01\n",
       "1          powerPlayGoals        8.172163e-02\n",
       "21   goalieReplacement_No        7.042770e-02\n",
       "0                     pim        6.177498e-02\n",
       "22  goalieReplacement_Yes        6.076237e-02\n",
       "17           settledIn_OT        4.168470e-02\n",
       "13   shortHandedTimeOnIce        3.423580e-02\n",
       "14     powerPlayTimeOnIce        1.626676e-02\n",
       "16               hoa_home        1.360430e-02\n",
       "6                    hits        6.555537e-03\n",
       "18          settledIn_REG        5.828775e-03\n",
       "10              penalties        5.296913e-03\n",
       "15               hoa_away        4.793632e-03\n",
       "3                   shots        3.999431e-03\n",
       "7            blockedShots        2.834474e-03\n",
       "2    faceOffWinPercentage        1.964256e-03\n",
       "12          evenTimeOnIce        1.823543e-03\n",
       "9             missedShots        6.000243e-04\n",
       "5               takeaways        5.477219e-04\n",
       "20    startRinkSide_right        2.292851e-07\n",
       "8               giveaways        0.000000e+00\n",
       "19     startRinkSide_left        0.000000e+00"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# get importance\n",
    "importance = model4.feature_importances_\n",
    "# summarize feature importance\n",
    "pd.DataFrame(list(zip(X_train.columns, importance)), \n",
    "             columns = ['predictor','feature importance']).sort_values(\"feature importance\", ascending = False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "7b876301",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAPO0lEQVR4nO3dXYhd132G8eftuLpxHdLGkw8kuVJT0VQXsWsGOeCQxBc2UnKhhLZUJiRpGqMKLNJAAxG9SAuh4EC/QYlQXZEE6opCo1bUk8jBFNziuGhUjG05UTooajWVE42TkLSkRFH978XZQifjI80eaY5Gs+b5wXD2Xh971tnavFqss8+eVBWSpHb91EoPQJI0Xga9JDXOoJekxhn0ktQ4g16SGnfLSg9glNtvv702bdq00sOQpFXjxIkTL1fV5Ki6mzLoN23axMzMzEoPQ5JWjST/caU6l24kqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxN+U3Y6WbwaZ9j/dqd+aR94x5JNL1cUYvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mN6xX0SbYnOZVkNsm+EfU7kzyX5NkkM0ne3revJGm8Fg36JBPAfmAHsBV4MMnWBc2eBO6sqruA3wIeXUJfSdIY9ZnRbwNmq+p0VV0ADgM7hxtU1f9UVXW7twLVt68kabz6BP164OzQ/lxX9hOSvC/J14HHGczqe/ft+u/uln1m5ufn+4xdktRDn6DPiLJ6VUHVkap6C/Be4FNL6dv1P1hVU1U1NTk52WNYkqQ++gT9HLBxaH8DcO5KjavqKeDNSW5fal9J0vLrE/THgS1JNidZB+wCjg43SPKLSdJt3w2sA77Tp68kabwW/cMjVXUxyV7gGDABHKqqk0n2dPUHgF8FPpjkx8D/Ar/RfTg7su+Y3oskaYRef2GqqqaB6QVlB4a2Pw18um9fSdKN4zdjJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXG9/sKUVtamfY/3bnvmkfeMcSSSViNn9JLUOINekhpn0EtS4wx6SWqcQS9JjesV9Em2JzmVZDbJvhH170/yXPfzdJI7h+rOJHk+ybNJZpZz8JKkxS16e2WSCWA/cD8wBxxPcrSqXhxq9k3gnVX1vSQ7gIPAPUP191XVy8s4bklST31m9NuA2ao6XVUXgMPAzuEGVfV0VX2v230G2LC8w5QkXas+Qb8eODu0P9eVXclHgC8N7RfwRJITSXZfqVOS3UlmkszMz8/3GJYkqY8+34zNiLIa2TC5j0HQv32o+N6qOpfk9cBXkny9qp561QGrDjJY8mFqamrk8SVJS9dnRj8HbBza3wCcW9goyVuBR4GdVfWdS+VVda57PQ8cYbAUJEm6QfoE/XFgS5LNSdYBu4Cjww2S3AF8EfhAVX1jqPzWJLdd2gYeAF5YrsFLkha36NJNVV1Mshc4BkwAh6rqZJI9Xf0B4JPA64DPJAG4WFVTwBuAI13ZLcBjVfXlsbwTSdJIvZ5eWVXTwPSCsgND2w8BD43odxq4c2G5JOnG8ZuxktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxvUK+iTbk5xKMptk34j69yd5rvt5OsmdfftKksZr0aBPMgHsB3YAW4EHk2xd0OybwDur6q3Ap4CDS+grSRqjPjP6bcBsVZ2uqgvAYWDncIOqerqqvtftPgNs6NtXkjRefYJ+PXB2aH+uK7uSjwBfWmrfJLuTzCSZmZ+f7zEsSVIffYI+I8pqZMPkPgZB/4ml9q2qg1U1VVVTk5OTPYYlSerjlh5t5oCNQ/sbgHMLGyV5K/AosKOqvrOUvpKk8ekzoz8ObEmyOck6YBdwdLhBkjuALwIfqKpvLKWvJGm8Fp3RV9XFJHuBY8AEcKiqTibZ09UfAD4JvA74TBKAi90yzMi+Y3ovkqQR+izdUFXTwPSCsgND2w8BD/XtK0m6cfxmrCQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mN6xX0SbYnOZVkNsm+EfVvSfLVJD9K8vEFdWeSPJ/k2SQzyzVwSVI/tyzWIMkEsB+4H5gDjic5WlUvDjX7LvBR4L1XOMx9VfXydY5VknQN+szotwGzVXW6qi4Ah4Gdww2q6nxVHQd+PIYxSpKuQ5+gXw+cHdqf68r6KuCJJCeS7L5SoyS7k8wkmZmfn1/C4SVJV9Mn6DOirJbwO+6tqruBHcDDSd4xqlFVHayqqaqampycXMLhJUlX0yfo54CNQ/sbgHN9f0FVnetezwNHGCwFSZJukD5BfxzYkmRzknXALuBon4MnuTXJbZe2gQeAF651sJKkpVv0rpuquphkL3AMmAAOVdXJJHu6+gNJ3gjMAK8BXknyMWArcDtwJMml3/VYVX15LO9EkjTSokEPUFXTwPSCsgND299isKSz0A+AO69ngJKk6+M3YyWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjDHpJapxBL0mNM+glqXEGvSQ1zqCXpMYZ9JLUOINekhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1DiDXpIaZ9BLUuMMeklqXK+gT7I9yakks0n2jah/S5KvJvlRko8vpa8kabwWDfokE8B+YAewFXgwydYFzb4LfBT4o2voK0kaoz4z+m3AbFWdrqoLwGFg53CDqjpfVceBHy+1ryRpvPoE/Xrg7ND+XFfWR+++SXYnmUkyMz8/3/PwkqTF9An6jCirnsfv3beqDlbVVFVNTU5O9jy8JGkxfYJ+Dtg4tL8BONfz+NfTV5K0DPoE/XFgS5LNSdYBu4CjPY9/PX0lScvglsUaVNXFJHuBY8AEcKiqTibZ09UfSPJGYAZ4DfBKko8BW6vqB6P6jum9SJJGWDToAapqGpheUHZgaPtbDJZlevWVJN04fjNWkhpn0EtS4wx6SWqcQS9JjTPoJalxBr0kNa7X7ZWt27Tv8d5tzzzynjGORJKWnzN6SWqcQS9JjTPoJalxBr0kNc6gl6TGGfSS1Dhvr5SknlbrrdjO6CWpcQa9JDXOoJekxhn0ktQ4g16SGtfcXTd9PxW/mT4Rl6Rxai7oJelmcjPckunSjSQ1zqCXpMb1Cvok25OcSjKbZN+I+iT5i67+uSR3D9WdSfJ8kmeTzCzn4CVJi1t0jT7JBLAfuB+YA44nOVpVLw412wFs6X7uAT7bvV5yX1W9vGyjliT11mdGvw2YrarTVXUBOAzsXNBmJ/CFGngGeG2SNy3zWCVJ16BP0K8Hzg7tz3VlfdsU8ESSE0l2X+tAJUnXps/tlRlRVktoc29VnUvyeuArSb5eVU+96pcM/hPYDXDHHXf0GJYkqY8+M/o5YOPQ/gbgXN82VXXp9TxwhMFS0KtU1cGqmqqqqcnJyX6jlyQtqk/QHwe2JNmcZB2wCzi6oM1R4IPd3TdvA75fVS8luTXJbQBJbgUeAF5YxvFLkhax6NJNVV1Mshc4BkwAh6rqZJI9Xf0BYBp4NzAL/BD4cNf9DcCRJJd+12NV9eVlfxeSpCvq9QiEqppmEObDZQeGtgt4eES/08Cd1zlGSdJ18Fk3ksbCBwzePHwEgiQ1zqCXpMYZ9JLUOINekhrnh7HSCrsZ/jCF2uaMXpIaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxhn0ktQ4g16SGmfQS1LjfASCtEr5vHf15YxekhrnjF7SVfnQtdXPGb0kNc6gl6TGGfSS1DiDXpIa54ex+gktfvDW4ntqlf9W49Er6JNsB/4cmAAerapHFtSnq3838EPgN6vq3/r0XWu8kLWSvP7WpkWDPskEsB+4H5gDjic5WlUvDjXbAWzpfu4BPgvc07OvVjnDQ7q59ZnRbwNmq+o0QJLDwE5gOKx3Al+oqgKeSfLaJG8CNvXouyqthnBr8ZuTN/t5v9nHp7Upg2y+SoPk14DtVfVQt/8B4J6q2jvU5h+BR6rqX7r9J4FPMAj6q/YdOsZuYHe3+0vAqet7az/hduDlZTzeaua5uMxzMeB5uGw1n4ufr6rJURV9ZvQZUbbwf4crtenTd1BYdRA42GM8S5ZkpqqmxnHs1cZzcZnnYsDzcFmr56JP0M8BG4f2NwDnerZZ16OvJGmM+txHfxzYkmRzknXALuDogjZHgQ9m4G3A96vqpZ59JUljtOiMvqouJtkLHGNwi+ShqjqZZE9XfwCYZnBr5SyD2ys/fLW+Y3knVzeWJaFVynNxmediwPNwWZPnYtEPYyVJq5uPQJCkxhn0ktS45oM+yfYkp5LMJtm30uNZSUnOJHk+ybNJZlZ6PDdKkkNJzid5Yajs55J8Jcm/d68/u5JjvFGucC7+IMl/ddfFs0nevZJjvBGSbEzyT0m+luRkkt/pypu8LpoO+qFHMOwAtgIPJtm6sqNacfdV1V0t3it8FZ8Dti8o2wc8WVVbgCe7/bXgc7z6XAD8aXdd3FVV0zd4TCvhIvC7VfXLwNuAh7tsaPK6aDroGXp8Q1VdAC49gkFrSFU9BXx3QfFO4PPd9ueB997IMa2UK5yLNaeqXrr04MWq+m/ga8B6Gr0uWg/69cDZof25rmytKuCJJCe6R06sZW/ovutB9/r6FR7PStub5LluaaeJ5Yq+kmwCfgX4Vxq9LloP+t6PYFgj7q2quxksZT2c5B0rPSDdFD4LvBm4C3gJ+OMVHc0NlORngL8DPlZVP1jp8YxL60Hf5/ENa0ZVnetezwNHGCxtrVXf7p6wSvd6foXHs2Kq6ttV9X9V9Qrwl6yR6yLJTzMI+b+uqi92xU1eF60HvY9g6CS5Ncltl7aBB4AXrt6raUeBD3XbHwL+YQXHsqIuBVvnfayB66L7Y0l/BXytqv5kqKrJ66L5b8Z2t4r9GZcfwfCHKzuilZHkFxjM4mHw6IvH1sq5SPI3wLsYPIL228DvA38P/C1wB/CfwK9XVfMfUl7hXLyLwbJNAWeA3760Tt2qJG8H/hl4HnilK/49Buv0zV0XzQe9JK11rS/dSNKaZ9BLUuMMeklqnEEvSY0z6CWpcQa9JDXOoJekxv0/MbvBTL1eQMsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# plot feature importance\n",
    "plt.bar([x for x in range(len(importance))], importance)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "02992bac",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation Result:\n",
      "\n",
      "accuracy score: 0.7191\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAax0lEQVR4nO3de7QddX338ffHBBG5BDRgNYBBCkW0QDGC14p4A4rF26MgiiJKo9K6+rQ+sLrqhWJbL49aXYCYCqIVoV6ookV5rFTQKpWw5GKgYOQaERGBIBeBwPf5YyZkezyZs89J5pydw/u11l7ZM/Pbs7/zyz77M7c9k6pCkqS1ecRMFyBJGm0GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBsYFL8sYk35vpOtanJIcm+X9DtDspybumo6bpkOTaJC9sn783yedmuiYJDIoZkWTjJCcnuS7Jr5P8KMn+M13XMNovs3uS3JnkF0k+nWSz9fkeVXVaVb14iHaLq+q49fneqyWpJHe1y/mzJB9JMqeP95rtkpyaZFWSJ4wZ/94k97d9fHuS7yd55hTm/9r2b+muJF9J8pgJ2r8jyTVt+yuS7Dwwbeskn2/ruS3JaZOtZzYyKGbGXOAG4HnAPOBdwBeSLJzJoibhpVW1GbAn8HTgb8c2SDJ32qta/3Zvl/N5wGuAN81wPevVdPwfJdkUeCWwEjh0nCb/2vbx1sD3gDOTZBLzfwrwSeD1wOOAu4ETO9q/GTgC+BNgM+BA4JaBJmcCNwFPBLYB/u+wtcxmBsUMqKq7quq9VXVtVT1YVV8HrgGetrbXJNkuyZlJfpnkV0mOX0u7jyW5IckdSS5K8tyBaXslWdpO+0WSj7TjH5Xkc+18b09yYZLHDbEcPwO+ATy1nU8leXuSnwA/accdmOTigTXG3SZapsHdaWl8NMnNSVYmuTTJ6vc7Ncn7Bub3liTLk9ya5KzBNdi2tsVJftKuKZ4w7BdSVS0H/gvYY2B+U1muHZOc2467JclpSbYcpoaxkhzUvv8dSX6aZL92/EO7r9rhh3ZhJVnY9sMRSa4Hzk3yzSRHjZn3JUle0T7fJcm32j69MsmrJ1nqK4Hbgb8D3rC2RlV1P/AZ4PeAx05i/ocCX6uq86vqTpqVrlck2XxswySPAN4D/GVVXV6Nn1bVre30FwPbAe+sqpVVdX9V/WgStcxaBsUIaL+UdwaWrWX6HODrwHXAQmABcMZaZnchzRfaY4DPA19M8qh22seAj1XVFsCOwBfa8W+g2bLZjuaPdDFwzxB1bwccAAz+Mb0M2BvYNcmewCnAn7Xz/SRwVppdb8Mu04uBP6bpny1p1ux/NU4t+wL/CLwaeHw737HzO5BmC2j3tt1LJlrGdt67AM8FlrfDU12utDU+AXgyTX+/d5gaxtSzF/BZ4J00ffLHwLWTmMXz2vd/Cc1n5JCBee9Kszb97+3WwLfaNtu07U5s1+JX7/K5dIL3egNwOk0f7NL23XjLtDHwRmBFVd2S5DltCK/t8Zz2pU8BLlk9n6r6KXAfzedlrG3bx1PblalrkhzbBgjAM4Argc+0YX5hkudNsHwPD1XlYwYfwEbAfwCf7GjzTOCXwNxxpr0R+F7Ha2+j2YUCcD5wLDB/TJs3Ad8Hdhui3muBO2nWEq+j2czfpJ1WwL4DbT8BHDfm9VfSfFENtUzAvsBVNH/EjxjT7lTgfe3zk4EPDkzbDLgfWDhQ23MGpn8BOKZjOQu4A7irfX46sPG6LNc47/Ey4Edj+vaF7fP3Ap9by+s+CXy04//nhQPDD82HJrgKeNLA9M3bZXxiO/z3wCnt89cA3x3nvd8z5Gd7e+BBYI92+ByaFZXB2u5rP0s3A+cCT5vk38+3gcVjxv0M2Gects9ql//faQJ2YfvZeks7fUk7/Qiav8uD29rmT6am2fhwi2IGtWsy/0Lzx3LUwPhvpDnAd2eSQ2nWPK+rqlVDzPOv0hygW5nkdpothfnt5CNo1rT+p11bOrAd/y80f8RnJLkxyQeTbNTxNi+rqi2r6olV9baqGtz6uGHg+ROBvxpcE2yX5QnDLlNVnQscD5wA/CLJkiRbjNP0CTTBtfp1d9JseSwYaHPTwPO7acKEJMsG+vu5A232bNu8hmYradN1Wa4k2yQ5I83B8TuAz7Hm/2YytgN+OoXXrfbQ/1FV/Zrmi/PgdtTBwOoDuE8E9h6znIfS7B4axuuBK6rq4nb4NOC1Yz5bX2g/S9tU1b5VddEkl+VOYOznYQvg1+O0Xf05/WBV3V5V19IE3wED06+tqpOr2e10Bk1fPXuSNc06BsUMafePn0xzAO6V1eyjBaCq9q+qzdrHaTQf1u0zwcHH9kvuaJrdKltV1ZY0BxHTzvcnVXUIzW6EDwBfSrJp+0dxbFXtSrPWdSBw2BQXbfByxDcAf99+Eax+PLqqTh92mdq6P15VT6PZzbAzzS6XsW6k+WIDHjqI+liatcuJ5v+Ugf7+7phpVVVfAH4AvHsdl+sfafpnt2p2/72O9v9mkm6g2XU4nruARw8Mj/elPvaS0acDh6Q542gT4D8H3ue8Mcu5WVW9dcg6DwOelOSmJDcBH6EJxgnP8Evy3IHwHu+xOtCX0exKXP26JwEb02wpjHUlzUrZ2i6ZfWnHtIc1g2LmfIJmP/FLx6yRj+eHwM+B9yfZNM3B5/HWcjYHVtHu+kjybgbWtpK8LsnWVfUgzSY1wANJnp/kD9v963fQ7LJ5YF0WrvXPwOIke6exaZI/aQ80DrVMSZ7evn4jmi/B36ylts8DhyfZo93f/Q/Af7drjevD+4Ejk/zeOizX5rS77ZIsYPzAG8bJNMv6giSPSLIgzXEUgIuBg5NslGQR8Koh5nc2Tcj+Hc1ZSA+2478O7Jzk9e38Nmr/P5480Qzb0NkR2IvmmNkeNCc9fJ6Og9qrVdV3B8J7vMfqQD8NeGkbLJu2y3Bmu6U0dp53A/8K/J8kmyfZFnhLu5wA/wZsleQNSeYkeRXNFul/TVTvbGdQzIAkT6Q5ELoHcNOY3Uy/o6oeAF4K/D5wPbCCZnfIWOfQnIV0Fc1umN/w27uC9gOWJbmT5sD2wVX1G5q1zi/RhMQVwHk0u0XWSVUtpflDPJ7mWMlymuMPk1mmLWi+mG9rl+lXjHPKYlV9m+aMly/TfFHvyJrdKeusqi6j6Zd3rsNyHUuzO2slze6eM6dYyw+Bw4GPtvM6jzVbU++iWfbb2vf7/BDzu7et5YWD7dsv2xfT9OONNLvuPkCzxr76h5HjnoBBEwZfrarLquqm1Q+az92BmeC3DsOqqmU0J1+cRnOcY3Pgbaunp/lR5kkDLzmKJqxvpNlK/DzNiQlUc/bTnwJ/TdOvxwAHVdXg6bMPS6lyS0uStHZuUUiSOhkUkqROBoUkqZNBIUnqtMFduG3+/Pm1cOHCmS5DkjYoF1100S1VtfVUXrvBBcXChQtZunTpTJchSRuUJNdN3Gp87nqSJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ16C4okp6S5z/GP1zI9ST6e5h7Hl2Ytt0iUJM2sPrcoTqW5rPXa7A/s1D6OpLk/gyRpxPQWFFV1PnBrR5ODgM+2dxC7ANgyyeMnmu99962vCiVJw5jJYxQL+O2b6qzgt+9v/JAkRyZZmmTpz39+27QUJ0lqzGRQjHev4HHvolRVS6pqUVUtmjdvq57LkiQNmsmgWAFsNzC8Lc3tCSVJI2Qmg+Is4LD27KdnACur6uczWI8kaRy9XT02yenAPsD8JCuA9wAbAVTVScDZwAE0N6a/m+Zm8ZKkEdNbUFTVIRNML+Dtfb2/JGn98JfZkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI69RoUSfZLcmWS5UmOGWf6vCRfS3JJkmVJDu+zHknS5PUWFEnmACcA+wO7Aock2XVMs7cDl1fV7sA+wIeTPLKvmiRJk9fnFsVewPKqurqq7gPOAA4a06aAzZME2Ay4FVjVY02SpEnqMygWADcMDK9oxw06HngycCNwGfCOqnpw7IySHJlkaZKlK1fe1le9kqRx9BkUGWdcjRl+CXAx8ARgD+D4JFv8zouqllTVoqpaNG/eVuu7TklShz6DYgWw3cDwtjRbDoMOB86sxnLgGmCXHmuSJE1Sn0FxIbBTkh3aA9QHA2eNaXM98AKAJI8D/gC4useaJEmTNLevGVfVqiRHAecAc4BTqmpZksXt9JOA44BTk1xGs6vq6Kq6pa+aJEmT11tQAFTV2cDZY8adNPD8RuDFfdYgSVo3/jJbktTJoJAkdTIoJEmdDApJUqdeD2b34cEH4aqrZroKafZ5zGNg/vyZrkKjaIMLCoDzz5/pCqTZ5d57m6A45JCZrkSjaIMLirlz4Y/+aKarkGaX666DW2+d6So0qjxGIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqNFRQJHl2km8luSrJ1UmuSXL1EK/bL8mVSZYnOWYtbfZJcnGSZUnOm+wCSJL6NXfIdicDfwlcBDwwzAuSzAFOAF4ErAAuTHJWVV0+0GZL4ERgv6q6Psk2k6hdkjQNhg2KlVX1jUnOey9geVVdDZDkDOAg4PKBNq8Fzqyq6wGq6uZJvockqWfDHqP4zyQfSvLMJHuufkzwmgXADQPDK9pxg3YGtkrynSQXJTlsyHokSdNk2C2Kvdt/Fw2MK2DfjtdknHE1zvs/DXgBsAnwgyQXVNVVvzWj5EjgSIBtttl+yJIlSevDUEFRVc+fwrxXANsNDG8L3DhOm1uq6i7griTnA7sDvxUUVbUEWAKw886LxoaNJKlHw571NC/JR5IsbR8fTjJvgpddCOyUZIckjwQOBs4a0+arwHOTzE3yaJotlysmuxCSpP4Me4ziFODXwKvbxx3Ap7teUFWrgKOAc2i+/L9QVcuSLE6yuG1zBfBN4FLgh8CnqurHU1kQSVI/hj1GsWNVvXJg+NgkF0/0oqo6Gzh7zLiTxgx/CPjQkHVIkqbZsFsU9yR5zuqBJM8G7umnJEnSKBl2i+KtwGfa4xIBbgXe2FdRkqTRMexZTxcDuyfZoh2+o8+iJEmjozMokryuqj6X5H+PGQ9AVX2kx9okSSNgoi2KTdt/N++7EEnSaOoMiqr6ZPvvsdNTjiRp1Az7g7sPJtkiyUZJvp3kliSv67s4SdLMG/b02Be3B7APpLnsxs7AO3urSpI0MoYNio3afw8ATq+qW3uqR5I0Yob9HcXXkvwPzY/s3pZka+A3/ZUlSRoVQ21RVNUxwDOBRVV1P3AXzU2IJEmz3ES/o9i3qs5N8oqBcYNNzuyrMEnSaJho19PzgHOBl44zrTAoJGnWm+h3FO9p/z18esqRJI2aYX9H8Q9JthwY3irJ+3qrSpI0MoY9PXb/qrp99UBV3UZzqqwkaZYb9vTYOUk2rqp7AZJsAmzcX1mSptu998JVV03cTg8/wwbF54BvJ/k0zUHsNwGf6a0qSdNq3jy46SY4//yZrkT92XzTiduMb9j7UXwwyaXAC2luXHRcVZ0z1TeVNFq23BL23numq1C/5syZ6iuH3aIAuAJYVVX/keTRSTavql9P9Y0lSRuGYc96egvwJeCT7agFwFd6qkmSNEKGPevp7cCzgTsAquonwDZ9FSVJGh3DBsW9VXXf6oEkc2kOakuSZrlhg+K8JH8DbJLkRcAXga/1V5YkaVQMGxRHA78ELgP+DDgb+Nu+ipIkjY4Jz3pK8gjg0qp6KvDP/ZckSRolE25RVNWDwCVJtp+GeiRJI2bY31E8HliW5Ic0Ny0CoKr+tJeqJEkjY9igOLbXKiRJI2uiO9w9ClgM/D7NgeyTq2rVdBQmSRoNEx2j+AywiCYk9gc+3HtFkqSRMtGup12r6g8BkpwM/LD/kiRJo2SiLYr7Vz9xl5MkPTxNFBS7J7mjffwa2G318yR3TDTzJPsluTLJ8iTHdLR7epIHkrxqsgsgSepX566nqpry9cuTzAFOAF4ErAAuTHJWVV0+TrsPAN7fQpJG0LCX8JiKvYDlVXV1e0HBM4CDxmn358CXgZt7rEWSNEV9BsUC4IaB4RXtuIckWQC8HDipa0ZJjkyyNMnSlSt/ud4LlSStXZ9BkXHGjb00+T8BR1fVA10zqqolVbWoqhbNm7f1+qpPkjSEydwKdbJWANsNDG8L3DimzSLgjCQA84EDkqyqqq/0WJckaRL6DIoLgZ2S7AD8DDgYeO1gg6raYfXzJKcCXzckJGm09BYUVbUqyVE0ZzPNAU6pqmVJFrfTO49LSJJGQ59bFFTV2TQ3ORocN25AVNUb+6xFkjQ1fR7MliTNAgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOvUaFEn2S3JlkuVJjhln+qFJLm0f30+ye5/1SJImr7egSDIHOAHYH9gVOCTJrmOaXQM8r6p2A44DlvRVjyRpavrcotgLWF5VV1fVfcAZwEGDDarq+1V1Wzt4AbBtj/VIkqagz6BYANwwMLyiHbc2RwDfGG9CkiOTLE2ydOXKX67HEiVJE+kzKDLOuBq3YfJ8mqA4erzpVbWkqhZV1aJ587ZejyVKkiYyt8d5rwC2GxjeFrhxbKMkuwGfAvavql/1WI8kaQr63KK4ENgpyQ5JHgkcDJw12CDJ9sCZwOur6qoea5EkTVFvWxRVtSrJUcA5wBzglKpalmRxO/0k4N3AY4ETkwCsqqpFfdUkSZq8Pnc9UVVnA2ePGXfSwPM3A2/uswZJ0rrxl9mSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTr0GRZL9klyZZHmSY8aZniQfb6dfmmTPPuuRJE1eb0GRZA5wArA/sCtwSJJdxzTbH9ipfRwJfKKveiRJU9PnFsVewPKqurqq7gPOAA4a0+Yg4LPVuADYMsnje6xJkjRJc3uc9wLghoHhFcDeQ7RZAPx8sFGSI2m2OIDcv2jRVteu31I3VPfOg41XznQVo8G+WMO+WMO+WOOObaf6yj6DIuOMqym0oaqWAEsAkiytum3Rupe34Wv64m77AvtikH2xhn2xRpKlU31tn7ueVgDbDQxvC9w4hTaSpBnUZ1BcCOyUZIckjwQOBs4a0+Ys4LD27KdnACur6udjZyRJmjm97XqqqlVJjgLOAeYAp1TVsiSL2+knAWcDBwDLgbuBw4eY9ZKeSt4Q2Rdr2Bdr2Bdr2BdrTLkvUvU7hwQkSXqIv8yWJHUyKCRJnUY2KLz8xxpD9MWhbR9cmuT7SXafiTqnw0R9MdDu6UkeSPKq6axvOg3TF0n2SXJxkmVJzpvuGqfLEH8j85J8LcklbV8Mczx0g5PklCQ3J/nxWqZP7XuzqkbuQXPw+6fAk4BHApcAu45pcwDwDZrfYjwD+O+ZrnsG++JZwFbt8/0fzn0x0O5cmpMlXjXTdc/g52JL4HJg+3Z4m5muewb74m+AD7TPtwZuBR4507X30Bd/DOwJ/Hgt06f0vTmqWxRe/mONCfuiqr5fVbe1gxfQ/B5lNhrmcwHw58CXgZuns7hpNkxfvBY4s6quB6iq2dofw/RFAZsnCbAZTVCsmt4y+1dV59Ms29pM6XtzVINibZf2mGyb2WCyy3kEzRrDbDRhXyRZALwcOGka65oJw3wudga2SvKdJBclOWzaqptew/TF8cCTaX7Qexnwjqp6cHrKGylT+t7s8xIe62K9Xf5jFhh6OZM8nyYontNrRTNnmL74J+DoqnqgWXmctYbpi7nA04AXAJsAP0hyQVVd1Xdx02yYvngJcDGwL7Aj8K0k362qO3qubdRM6XtzVIPCy3+sMdRyJtkN+BSwf1X9appqm27D9MUi4Iw2JOYDByRZVVVfmZYKp8+wfyO3VNVdwF1Jzgd2B2ZbUAzTF4cD769mR/3yJNcAuwA/nJ4SR8aUvjdHddeTl/9YY8K+SLI9cCbw+lm4tjhowr6oqh2qamFVLQS+BLxtFoYEDPc38lXguUnmJnk0zdWbr5jmOqfDMH1xPc2WFUkeB/wBcPW0VjkapvS9OZJbFNXf5T82OEP2xbuBxwIntmvSq6pq1l0xc8i+eFgYpi+q6ook3wQuBR4EPlVV4542uSEb8nNxHHBqkstodr8cXVW3zFjRPUlyOrAPMD/JCuA9wEawbt+bXsJDktRpVHc9SZJGhEEhSepkUEiSOhkUkqROBoUkqZNBIY2jvfLsxUl+3F51dMv1PP9rk8xvn9+5PuctrW8GhTS+e6pqj6p6Ks1F1t4+0wVJM8WgkCb2A9oLpyXZMck324vsfTfJLu34xyX5t/Z+B5ckeVY7/itt22VJjpzBZZCmbCR/mS2NiiRzaC79cHI7agmwuKp+kmRv4ESaC819HDivql7evmaztv2bqurWJJsAFyb58iy+FpdmKYNCGt8mSS4GFgIX0VxtdDOam0R9ceDKtBu3/+4LHAZQVQ8AK9vxf5Hk5e3z7YCdAINCGxSDQhrfPVW1R5J5wNdpjlGcCtxeVXsMM4Mk+wAvBJ5ZVXcn+Q7wqD6KlfrkMQqpQ1WtBP4C+GvgHuCaJP8LHrr/8Or7k38beGs7fk6SLYB5wG1tSOxCc+tJaYNjUEgTqKof0dyH+WDgUOCIJJcAy1hzy813AM9vr056EfAU4JvA3CSX0ly99ILprl1aH7x6rCSpk1sUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6vT/AUdikqrEtvoKAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "roc auc score: 0.7175871764029196\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA9HklEQVR4nO3dd3gU5fbA8e+RDlKkiAURFERAmiBgw4KFYv3ZkKte28WuWLB37L2hiKjYLqjYsCCoiKDoBRQvXeQCQqQXkSI15/fHmZAlJpslyWS2nM/z7EN2d3bm7ITs2bfMeUVVcc455wqyU9QBOOecS26eKJxzzsXlicI551xcniicc87F5YnCOedcXJ4onHPOxeWJwu0QEZkmIkdGHUeyEJFbRWRgRMceJCL3RXHskiYi/xCRkUV8rf+fDJknihQmIvNE5C8RWSsii4MPjp3DPKaqNlfV0WEeI4eIVBCRB0VkfvA+fxWRPiIipXH8fOI5UkSyYh9T1QdU9eKQjicicrWITBWRdSKSJSLvikiLMI5XVCJyt4i8WZx9qOpbqnpcAsf6W3Iszf+TmcoTReo7UVV3BloDbYBbog1nx4lI2QKeehfoDHQDqgLnAr2Ap0OIQUQk2f4engauAa4GagL7AR8C3Uv6QHF+B6GL8tguQarqtxS9AfOAY2LuPwJ8GnO/IzAO+AP4L3BkzHM1gVeBhcAq4MOY504Afg5eNw5omfeYwB7AX0DNmOfaAMuBcsH9C4EZwf5HAHvHbKvAFcCvwNx83ltnYAOwV57HOwBbgUbB/dHAg8B4YDXwUZ6Y4p2D0cD9wHfBe2kEXBDEvAaYA1wSbFsl2CYbWBvc9gDuBt4MtmkQvK9/AvODc3FbzPEqAa8F52MGcCOQVcDvtnHwPtvH+f0PAvoBnwbx/gfYN+b5p4EFwJ/Aj8DhMc/dDQwF3gyevxhoD3wfnKtFwHNA+ZjXNAe+AFYCS4BbgS7AJmBzcE7+G2xbHXg52M/vwH1AmeC584Nz/mSwr/uCx74NnpfguaXB73QycAD2JWFzcLy1wMd5/w6AMkFc/wvOyY/k+T/ktyJ81kQdgN+K8cvb/g+kHjAFeDq4vyewAvs2vhNwbHC/TvD8p8DbwC5AOeCI4PEDgz/QDsEf3T+D41TI55ijgH/FxPMo0D/4+RRgNtAUKAvcDoyL2VaDD52aQKV83ttDwDcFvO/fyP0AHx18EB2AfZi/R+4Hd2HnYDT2gd48iLEc9m193+DD6ghgPXBgsP2R5PlgJ/9E8RKWFFoBG4Gmse8pOOf1sA/AghLFpcBvhfz+B2EftO2D+N8ChsQ8fw5QK3juemAxUDEm7s3B72mnIN62WGItG7yXGUDvYPuq2If+9UDF4H6HvOcg5tgfAi8Gv5NdsUSe8zs7H9gCXBUcqxLbJ4rjsQ/4GsHvoSmwe8x7vi/O30Ef7O+gSfDaVkCtqP9WU/0WeQB+K8Yvz/5A1mLfnBT4CqgRPHcT8Eae7UdgH/y7Y9+Md8lnny8AffM89gu5iST2j/JiYFTws2DfXjsF94cDF8XsYyfsQ3fv4L4CR8d5bwNjP/TyPPcDwTd17MP+oZjnmmHfOMvEOwcxr723kHP8IXBN8PORJJYo6sU8Px7oEfw8Bzg+5rmL8+4v5rnbgB8KiW0QMDDmfjdgZpztVwGtYuIeU8j+ewMfBD+fDUwqYLtt5yC4XxdLkJViHjsb+Dr4+Xxgfp59nE9uojgamIUlrZ3yec/xEsUvwMnF/dvy2/a3ZOuTdTvuFFWtin2I7Q/UDh7fGzhDRP7IuQGHYUliL2Clqq7KZ397A9fned1eWDdLXkOBg0VkD6AT9iE5NmY/T8fsYyWWTPaMef2COO9reRBrfnYPns9vP79hLYPaxD8H+cYgIl1F5AcRWRls343cc5qoxTE/rwdyJhjsked48d7/Cgp+/4kcCxG5XkRmiMjq4L1UZ/v3kve97ycinwQTI/4EHojZfi+sOycRe2O/g0Ux5/1FrGWR77FjqeoorNurH7BERAaISLUEj70jcboEeaJIE6r6DfZt67HgoQXYt+kaMbcqqvpQ8FxNEamRz64WAPfneV1lVR2czzH/AEYCZwI9gcEafK0L9nNJnv1UUtVxsbuI85a+BDqIyF6xD4pIe+zDYFTMw7Hb1Me6VJYXcg7+FoOIVMC6rh4D6qpqDeAzLMEVFm8iFmFdTvnFnddXQD0RaVeUA4nI4ViL6kys5VgD6++PnTGW9/28AMwEGqtqNayvP2f7BViXXH7y7mcB1qKoHXPeq6lq8ziv2X6Hqs+oalusW3A/rEup0NcVEqcrIk8U6eUp4FgRaY0NUp4oIseLSBkRqRhM76ynqouwrqHnRWQXESknIp2CfbwEXCoiHYKZQFVEpLuIVC3gmP8GzgNOC37O0R+4RUSaA4hIdRE5I9E3oqpfYh+W74lI8+A9dMT64V9Q1V9jNj9HRJqJSGXgXmCoqm6Ndw4KOGx5oAKwDNgiIl2B2CmbS4BaIlI90feRxzvYOdlFRPYErixow+D9PQ8MDmIuH8TfQ0RuTuBYVbFxgGVAWRG5EyjsW3lVbGB7rYjsD1wW89wnwG4i0juYtlxVRDoEzy0BGuTMGgv+f40EHheRaiKyk4jsKyJHJBA3InJQ8P+vHLAOm9SwNeZY+8R5+UCgr4g0Dv7/thSRWokc1xXME0UaUdVlwOvAHaq6ADgZ+1a4DPum1Yfc3/m52Dfvmdjgde9gHxOBf2FN/1XYgPT5cQ47DJuhs0RV/xsTywfAw8CQoBtjKtB1B9/SacDXwOfYWMyb2Eyaq/Js9wbWmlqMDbReHcRQ2DnYjqquCV77DvbeewbvL+f5mcBgYE7QpZJfd1w89wJZwFysxTQU++ZdkKvJ7YL5A+tSORX4OIFjjcC+DMzCuuM2EL+rC+AG7D2vwb4wvJ3zRHBujgVOxM7zr8BRwdPvBv+uEJGfgp/PwxLvdOxcDiWxrjSwhPZS8LrfsG64nJbyy0Cz4Px/mM9rn8B+fyOxpPcyNljuikFyewqcSz0iMhobSI3k6ujiEJHLsIHuhL5pOxcVb1E4V0pEZHcROTToimmCTTX9IOq4nCtMaIlCRF4RkaUiMrWA50VEnhGR2SIyWUQODCsW55JEeWz2zxpsMP4jbBzCuaQWWtdTMDi6FnhdVQ/I5/luWF9zN+zirqdVtUPe7ZxzzkUrtBaFqo7B5s4X5GQsiaiq/gDUEJFEB7ucc86VkiiLce3J9rMwsoLHFuXdUER6YXVeqFKlStv999+/VAJ0zrlUtGkTrF5tt8p/LmI3Xcwksperap2i7C/KRJFfqeh8+8FUdQAwAKBdu3Y6ceLEMONyzrmUsnUr/Oc/8OmndpsyBUBp0EC4+dBhdNlpJA0+6fdbUfcfZaLIYvsrU+thlUydc84VYuVKGDHCEsPnn8OKFVCmDHTpsIrXD7qB3Q7ehzpP3YbIScBJIP2KfKwoE8Uw4EoRGYINZq8Oruh0zjmXhypMnZrbahg3DrKzoXZt6NYNTjgBum/6gCp9Lodly6Db7fn32xRBaIlCRAZjhepqi60KdhdWKAxV7Y/V0OmGXfm7HlsHwDnnXGD9ehg1Kjc5LAhGddu0gVtvhe7d4aCDoMzyJXDVVfDuu9C6tW18YMldcRBaolDVswt5XrGFa5xzzgXmzctNDF9/DRs2QJUqcOyxcOed1nrYI2/xmAUL7AX33w99+kC5ciUaky9B6JxzEdq82bqRcpLD9On2eKNGcMkl1mro1AkqVMjzwt9+g48/hiuvhHbtYP58qBVO/UNPFM45V8qWLYPhwy0xjBhh01jLlbOEcPHFlhz226+AF2dnwwsvwM1BEeHTToPddw8tSYAnCuecC50qTJqU22oYP94e2203+5zv3h2OOQaqFVYI/pdfLJN8+y0cfzy8+KIliZB5onDOuRCsWQNffmmJ4bPPYNEiELHB57vvtuTQpg3slGh9jPXr4bDD7KKJQYPgvPNsh6XAE4VzzpWQ2bPhk08sOYwZY1dIV6tmX/67d4euXWHXXQvfz3ZmzYLGjaFyZXjjDZvVtNtuYYRfIE8UzjlXRJs2wdixuV1Ks2bZ402bwtVXW3I49NAiTkLasAH69oWHH7YWxDnnQJcuJRl+wjxROOfcDli82LqSPv0UvvjCupgqVICjjrJLGbp1g33iLdaaiO++g4susjGJCy6wjBMhTxTOORdHdjZMnJjbavjxR3u8Xj3o2dM+w48+2q51KBF9+8Jdd0H9+jYl6rjjCn9NyDxROOdcHqtXw8iRlhiGD4elS23QuWNHu6ate3do2bKEx5JVbYetW1vT5P77YeedS/AAReeJwjmX8VRh5szcVsO338KWLbDLLjYs0L27/RvKpQorV8K119oVdnfcASeeaLck4onCOZeRNmyA0aNzk8PcufZ4ixZwww2WHDp2hLJhfkoOHQpXXGHJ4o47QjxQ8XiicM5ljKys3MTw1Vd2aUKlStC5M9x4ow1E169fCoEsWmSlN95/H9q2tX6uVq1K4cBF44nCOZe2tm6FH37ITQ6TJ9vjDRrkTiY68khLFqVq4UIbqH74YbjuupCbLcWX3NE559wOWrnSFvLJWdBn5Upb0Oeww+CRRyw5NG1aahc155o3z4r4XXWVtSIWLLBBkBTgicI5l9JUbenPnFbD99/blNY6dYLFfLrbDNMaNSIKcOtW6NfPFpDYaSc44wy7sjpFkgR4onDOpaD1622MIaeOUs6CPgceCLfdlrugT8J1lMIyY4YV8Rs3zqZNvfhiqZffKAmeKJxzKSF2QZ9Ro2DjRrvM4Nhj7fq0rl3zWdAnSuvXW93w7Gx4/XUrwVHq/V0lwxOFcy4pFbSgT+PGcNll1mo4/PB8FvSJ2syZ0KSJFfF76y2bzVS3btRRFYsnCudc0ihoQZ8jjoB//cuSQ+PGUUdZgL/+svrhjz0Gr71mLYgkKL9REjxROOciU9iCPiecYAv6VK0adaSFGDPGxiJ+/dX+PeGEqCMqUZ4onHOlqsQX9InaPfdY4A0b2hvr3DnqiEqcJwrnXOh+/TW31fDNNzb+UOwFfaKWU8SvXTur1dS3bwmWkE0uniiccyVu0ybrjclJDr/+ao83bQrXXFPMBX2itny5JYbGjeHOO+3NRLxeRNg8UTjnSsSiRdsv6LN2be6CPjmrvTVsGHWUxaAK775rNZpWrbI5uRnCE4Vzrkiys2HChNxWw08/2eP16sE//hHCgj5RWrgQLr8cPvrIupq+/NIWpMgQniiccwn744/tF/RZtswGnQ8+GB54wJJDixYpe11ZwRYvtqv8Hn0UevdO+iJ+JS2z3q1zbofkLOjzySeWHL77zhb0qVkzd0Gf448PaUGfqM2ZA8OGWWI48ECYPz/CglHR8kThnNtOQQv6tGwJffpYcujQIY2/VG/dCs88Y0WjypWDHj3swo4MTRLgicI5hxXVyxmIjl3Q55hj4KabbEGfvfaKOspSMG0aXHQR/Oc/lhH790/JIn4lzROFcxko3oI+F16Yu6BPxYpRRlnK1q+3WiEi8O9/W0si7QZbisYThXMZImkX9Ina9On2xitXhiFDrIhfnTpRR5VUPFE4l6aSfkGfqK1fb9dCPPEEDBoE555rfW3ubzxROJdGUmZBn6iNHm3laGfPhksugZNOijqipOaJwrkUN3dubqvh66//vqBPt26w++5RR5lE7roL7r0X9t3Xro046qioI0p6niicSzGbN9v1DDnJYcYMezzpF/SJWk4Rv/bt4frrLVlUrhx1VCkh1EQhIl2Ap4EywEBVfSjP89WBN4H6QSyPqeqrYcbkXCpaujR3QZ+RI7df0KdXryRf0Cdqy5ZZJcImTaw1kQFF/EpaaIlCRMoA/YBjgSxggogMU9XpMZtdAUxX1RNFpA7wi4i8paqbworLuVSQnb39gj4TJtgX4t13h9NPt8+5lFjQJ0qqMHiwVST8809bN8IVSZgtivbAbFWdAyAiQ4CTgdhEoUBVERFgZ2AlsCXEmJxLWmvWWNXVnDpKOQv6tG9vn3Hdu0Pr1j4QnZCsLOuH++QTu4z85ZehefOoo0pZYSaKPYEFMfezgA55tnkOGAYsBKoCZ6lqdt4diUgvoBdA/fr1QwnWuSjkt6BP9eq5C/p06ZKCC/okg2XLbEGMJ56wFkWZMlFHlNLCTBT5Xbajee4fD/wMHA3sC3whImNV9c/tXqQ6ABgA0K5du7z7cC5lFLSgT7NmVnuue3c45JAUXdAnarNnw8cf26JCbdrY3OBq1aKOKi2EmSiygNjqMPWwlkOsC4CHVFWB2SIyF9gfGB9iXM6VqoIW9Dn6aBtj7dYtxRf0idqWLfDUU3DHHXZie/aEunU9SZSgMBPFBKCxiDQEfgd6AD3zbDMf6AyMFZG6QBNgTogxORe6jFrQJ2pTplgRvwkT7KK555+3JOFKVGiJQlW3iMiVwAhseuwrqjpNRC4Nnu8P9AUGicgUrKvqJlVdHlZMzoUlYxf0idL69Xax3E47WY2mM8/0ExySUK+jUNXPgM/yPNY/5ueFwHFhxuBcGFTtQrecVsO331pF1oxY0CdqU6faDKbKleHtt62IX+3aUUeV1vzKbOcStGGDlcjISQ7z5tnjLVvCjTdacujY0SfYhGbdOhuHeOopeO01K+LXuXPUUWUETxTOxbFgQW5i+Oor+Osv+yLbuTPcfHMGLegTta++siJ+c+fC5ZfDySdHHVFG8UThXIwtW7Zf0GfKFHu8YUMbM83IBX2idscdcN99VqPkm2+gU6eoI8o4nihcxluxIndBnxEjbEGfsmVtQZ9HH7XksP/+Pk5a6rKzbaD6kEOsb+/uu219VlfqPFG4jBO7oM8nn1gLImdBnxNPzF3Qp3r1qCPNUEuX2tXUTZpY7ZKuXe3mIuOJwmWEdets6YGcLqWsLHu8bVu4/XZLDu3aeR2lSKnCW2/ZVYhr11oZcJcUPFG4tDVnTu4V0bEL+hx3XO4XVV/QJ0ksWACXXmq/sIMPhoEDra6JSwqeKFzaiLegz+WX5y7oU758tHG6fKxYYb+8p5+GK67wOcZJxhOFS2m+oE8KmzULhg2DG26w+ukLFvgCG0nKE4VLKb6gTxrYsgUef9xWm6tUyS6cq1vXf2lJzBOFS3qxC/p89hksXvz3BX3atPHpqynhv/+FCy+0Somnngr9+nkRvxTgicIlpVmzclsNY8b8fUGfrl1tOqtLIevX2yXtZcvC0KFw2mlRR+QS5InCJYWNG7df0Gf2bHvcF/RJA5MnW+ncypXh3XetiF/NmlFH5XaAJwoXmYULc6evfvnl9gv65CSHBg2ijtIV2dq1cNtt8OyzMGgQnHeelQV3KccThSs1W7duv6DPpEn2+F57wTnn5C7oU7lytHG6EvDFFzbtbN48uPJKG49wKcsThQvVH39Y/aScBX2WL88t3/Pgg5YcDjjAB6LTym232WpNTZrA2LFWNMultIQThYhUUdV1YQbjUp8qTJ+e22r47rvcBX26ds1d0Me7qNNQThG/ww6DW26BO+/0MrtpotBEISKHAAOBnYH6ItIKuERVLw87OJca/voLRo/++4I+rVrBTTdZcujQwS+2TVuLF1v3UrNmVp/Ji/ilnURaFE8CxwPDAFT1vyLiBeEz3Pz5uQPRsQv6HHOML+iTMVRtpbnrrrOprx07Rh2RC0lCXU+qukC270TeGk44Lln5gj5uO7/9ZoPVI0daV9PAgTYm4dJSIoliQdD9pCJSHrgamBFuWC4ZxC7o8/nnsGqVL+jjAn/8YVPYnnsOLrvM67OnuUQSxaXA08CeQBYwEvDxiTSkatdG5bQachb02XVXOOkkX9An4/3yixXx69PHBqDmz7e67S7tJZIomqjqP2IfEJFDge/CCcmVpnXrbIwhp46SL+jj/mbzZnjsMSusVaUK/POf9u3Bk0TGSCRRPAscmMBjLkXMmZPbahg92hf0cXFMmmSDUJMmWXne556zJOEySoGJQkQOBg4B6ojIdTFPVQN8omMK2bwZvv02NznMnGmP77efL+jj4li/Ho491gpsvfce/N//RR2Ri0i8FkV57NqJskBsofg/gdPDDMoV35Il2y/o8+eflgiOOMJWnOzeHRo1ijpKl5QmTbKFhCpXtiqvrVrBLrtEHZWLUIGJQlW/Ab4RkUGq+lspxuSKIDvbSvzntBomTrTB6T32gDPPzF3Qx7uVXYHWrLErqvv1s+sjzjvP5jy7jJfIGMV6EXkUaA5smyWvqkeHFpVL2Jo1Ngnlo49yF/Tp0MEukO3e3b4Y+vRVV6jPP4dLLrHlSK+5xruZ3HYSSRRvAW8DJ2BTZf8JLAszKJe4Rx6BF1/MbTX4gj5uh91yCzz0EDRtasW5Dj446ohckkkkUdRS1ZdF5JqY7qhvwg7MFW7VKnjmGZuM8vbbUUfjUs7WrVaA68gj7UrK22+3BUGcyyORRLE5+HeRiHQHFgL1wgvJJeqpp2yQ+s47o47EpZRFi+CKK6B5c+jb18r5Hn981FG5JJbIZVT3iUh14HrgBqySbO8wg3KF++MPePpp60pu0SLqaFxKUIVXX7Uqr8OH+0wml7BCWxSq+knw42rgKNh2ZbaL0NNPw+rV3ppwCZo3D/71L1tz9vDDrYjffvtFHZVLEfEuuCsDnInVePpcVaeKyAnArUAloE3phOjyWr3aup1OOcWmuDtXqNWrbf7088/b7CavyeJ2QLz/LS8DFwO1gGdE5FXgMeARVU0oSYhIFxH5RURmi8jNBWxzpIj8LCLTfJA8Mc88Y11P3ppwcU2fbrOZILeIn1d6dUUQr+upHdBSVbNFpCKwHGikqosT2XHQIukHHItVnZ0gIsNUdXrMNjWA54EuqjpfRLyITCH+/BOefNKqubbxNp3Lz6ZNNm+6b1+oWhUuvNDqM1WpEnVkLkXF+2qxSVWzAVR1AzAr0SQRaA/MVtU5qroJGAKcnGebnsD7qjo/OM7SHdh/Rnr2WZsW660Jl6+JE+Ggg+COO2ymw/TpXsTPFVu8FsX+IjI5+FmAfYP7Aqiqtixk33sCC2LuZwEd8myzH1BOREZj9aSeVtXX8+5IRHoBvQDq169fyGHT15o18MQTcMIJVgbcue2sW2fTXCtWtEv1Tzop6ohcmoiXKJoWc9/5FY7QfI7fFuiMDZB/LyI/qOqs7V6kOgAYANCuXbu8+8gYzz0HK1fCXXdFHYlLKj/9ZLVaqlSBDz6Ali2hRo2oo3JppMCuJ1X9Ld4tgX1nAXvF3K+HXayXd5vPVXWdqi4HxgA+jycfa9fC449Dt262kJBz/Pmn1Ylv2xbefNMe69TJk4QrcWFOf5gANBaRhsFa2z2AYXm2+Qg4XETKikhlrGvK1+POR79+toa1tyYcYMsRNm9uhb6uuw5OOy3qiFwaS6SER5Go6hYRuRIYgS109IqqThORS4Pn+6vqDBH5HJgMZAMDVXVqWDGlqrVrbSXKLl2gffuoo3GRu+kmm9XUrJmtF9Eh79CfcyUroUQhIpWA+qr6y47sXFU/Az7L81j/PPcfBR7dkf1mmhdegOXLvTWR0VRt0ZEyZaBzZxuwvvVWL+LnSkWhXU8iciLwM/B5cL+1iOTtQnIhWbcOHn3U1rPu2DHqaFwkfv/dLsPP+aaQs7i5JwlXShIZo7gbuybiDwBV/RloEFZAbnsvvADLlnlrIiOpwksvWRfTyJFQu3bUEbkMlUjX0xZVXS2+TFqpW7/eWhPHHguHHBJ1NK5UzZ0LF10EX39t60W89JIvcu4ik0iimCoiPYEyItIYuBoYF25YDqB/f1i61FsTGWntWpg82WY1XXyx12dykUrkf99V2HrZG4F/Y+XGe4cYk8NaE488YuOWh3pR98wwdSo88ID93KKFFfHr1cuThItcIi2KJqp6G3Bb2MG4XAMGwJIl8O67UUfiQrdpEzz4INx/P1Svbi2IXXeFypWjjsw5ILEWxRMiMlNE+opI89Ajcvz1Fzz8MBx1lK0x49LYhAl2ZfXdd8MZZ3gRP5eUElnh7igR2Q1bxGiAiFQD3lbV+0KPLkO99BIsXgxDhkQdiQvVunV2FWWlSjBsGJx4YtQROZcvUU28xp6ItABuBM5S1fKhRRVHu3btdOLEiVEculRs2AD77guNG8Po0VFH40IxcSIceKCNPXz7rY1HVK8edVQuzYnIj6papEpxiVxw11RE7haRqcBz2IynekU5mCvcwIGwcKHPdEpLq1fbMqQHHZRbxO+wwzxJuKSXyGD2q8Bg4DhVzVv91ZWgjRtt5crDD7ep8y6NfPwxXHqp9SnecAOcfnrUETmXsETGKLxwRCl5+WWr1vDaa+DXN6aRPn2sqmOLFvDhh9aicC6FFJgoROQdVT1TRKaw/YJDia5w53bAxo02Q/LQQ+Hoo6OOxhWbKmzdCmXLWm2matWs6mv5SIb2nCuWeC2Ka4J/TyiNQDLdq69CVha88oq3JlJeVhZcdpmtNHf//VaD5dhjo47KuSKLt8LdouDHy/NZ3e7y0gkvM+Rcb3XwwXDMMVFH44osO9tKbjRrBqNGwW67RR2RcyUikQvu8vsq1LWkA8lkgwZZtYa77vLWRMqaM8f6DC+91FaXmjIFrroq6qicKxHxxiguw1oO+4jI5JinqgLfhR1Ypti0ycr7dOhgXdkuRa1bZ1dVDxwIF17oGd+llXhjFP8GhgMPAjfHPL5GVVeGGlUGef11+O03W3fCP1tSzJQp8NFHcPvtNqPpt9/sKmvn0ky8ridV1XnAFcCamBsiUjP80NLf5s021nnQQVbJwaWIjRvhzjvt6upnnrFa8OBJwqWtwloUJwA/YtNjY7/vKrBPiHFlhDfegHnz4LnnvDWRMn74wRYUmj4dzj0XnnwSatWKOirnQlVgolDVE4J/G5ZeOJkjpzXRrh106xZ1NC4h69ZB9+5QpQp89hl09TkdLjMkUuvpUBGpEvx8jog8ISL1ww8tvb31lk2UufNOb00kvf/8x6a+VqlipTimTfMk4TJKItNjXwDWi0grrHLsb8AboUaV5rZsgfvusy7uE/xyxuT1xx+2iFDHjrlF/A45BKpWjTQs50pbIkUBt6iqisjJwNOq+rKI/DPswNLZW2/B//5nZX+8NZGkPvwQLr/cBqpvuskWFXIuQyWSKNaIyC3AucDhIlIGKBduWOkrpzXRujWcdFLU0bh8XXedDVK3amVdTW3bRh2Rc5FKJFGcBfQELlTVxcH4xKPhhpW+Bg+G2bPh/fe9NZFUYov4detmM5luvBHK+Xci5xJa4U5E6gI5tZHHq+rSUKOKI5VXuNu61coAVawIkybZAmcuCcyfb6U32rSxqWjOpaGwV7g7ExgPnIGtm/0fEfFVV4pgyBCYNctqOnmSSALZ2fD889C8OXzzDeyxR9QROZeUEul6ug04KKcVISJ1gC+BoWEGlm62boW+fa3SwymnRB2NY/Zsq8k0dqyVAB8wABo0iDoq55JSIolipzxdTStIbFqti/HOO/DLL/Duu96aSAobNljz7tVX4Z//9AEj5+JIJFF8LiIjsHWzwQa3PwsvpPST05o44AD4v/+LOpoM9vPPVsTvrrvslzFvng0YOefiKvS7rar2AV4EWgKtgAGqelPYgaWToUNhxgy44w5vTURiwwa47Tarl/LCC7lF/DxJOJeQeOtRNAYeA/YFpgA3qOrvpRVYusjOttZEs2Zwuk8BKH3jxlkRv5kzrYvpiSegphc/dm5HxOt6egV4HRgDnAg8C3jHyQ567z0rDTR4sLcmSt26dXDiibDzzvD553D88VFH5FxKipcoqqrqS8HPv4jIT6URUDrJzoZ774X99/cKEKXq++9tycAqVeCTT2w8wuszOVdk8b7jVhSRNiJyoIgcCFTKc79QItJFRH4RkdkicnOc7Q4Ska3pdn3GBx/A1Kk2NlGmTNTRZIBVq2zK6yGH2GIfAAcf7EnCuWKK16JYBDwRc39xzH0Fjo6346AmVD/gWCALmCAiw1R1ej7bPQyM2LHQk1tOa6JJEzjrrKijyQDvvw9XXAHLlsEtt/hJd64ExVu46Khi7rs9MFtV5wCIyBDgZGB6nu2uAt4jt0RIWvjoI5g82b7YemsiZNdeC089ZZUWP/vMSnE450pMItdRFNWewIKY+1lAh9gNRGRP4FSsdVJgohCRXkAvgPr1k3/NJFVrTTRuDD16RB1Nmoot4nfCCbDrrnDDDV7Ez7kQhDkPJ79LXfNWIHwKuElVt8bbkaoOUNV2qtquTp06JRVfaIYNs2u7br/dPsdcCZs3D7p0scEfgM6drbvJk4RzoQgzUWQBe8XcrwcszLNNO2CIiMwDTgeeF5FTQowpdKpwzz3QqBH07Bl1NGkmOxuefdZmMY0bB3vvHXVEzmWEQr/viogA/wD2UdV7g/UodlPV8YW8dALQWEQaAr8DPbB1LbZR1YYxxxkEfKKqH+7QO0gyn3xiJcRffdVbEyXq11/hggvgu++sNdG/vycK50pJIi2K54GDgbOD+2uw2UxxqeoW4EpsNtMM4B1VnSYil4rIpUWMN6nltCb22QfOOSfqaNLMpk22fuzrr9uAtScJ50pNIt95O6jqgSIyCUBVV4lI+UR2rqqfkaeAoKr2L2Db8xPZZzL77DP48Ud4+WVvTZSISZNs+tjdd9uaEfPmQYUKUUflXMZJpEWxObjWQWHbehTZoUaVgnJaEw0bwrnnRh1NituwwQanDzoIXnzRro0ATxLORSSRRPEM8AGwq4jcD3wLPBBqVClo+HCYMAFuvdUn3xTLt99Cq1bw0ENw3nkwfTqkwEw359JZoR0kqvqWiPwIdMamvJ6iqjNCjyyF5LQm9t7bPttcEa1dCyefDNWqwciRtvKccy5yicx6qg+sBz6OfUxV54cZWCoZMQLGj7dekvIJjd647Xz7rdVn2nln+PRTm/66885RR+WcCyTS9fQp8Enw71fAHGB4mEGlkpzWRP36cP75UUeTYlassCbY4YfnFvHr2NGThHNJJpGupxax94PKsZeEFlGK+eIL+OEHWzjNWxMJUrVl/668ElautCusvdaJc0lrhydxqupPIpJWBfyKKqc1Ua+eXQvmEnTttfD009C2rY1FtGoVdUTOuTgSGaO4LubuTsCBwLLQIkohX31llSSef95nbhZKFbZssSlhJ50Ee+wB113nF5w4lwISGaOoGnOrgI1VnBxmUKkgtjVx4YVRR5Pk5s6F447LLeJ39NFw442eJJxLEXH/UoML7XZW1T6lFE/K+Pprm6zz3HPemijQ1q12gm691Rbl8PVgnUtJBSYKESmrqlsSXfY009xzj/WeXHRR1JEkqVmzbBrY999D1642d3ivvQp9mXMu+cRrUYzHxiN+FpFhwLvAupwnVfX9kGNLWqNHw5gx8MwzULFi1NEkqS1b4Lff4M03rd665Lc8iXMuFSTSSVwTWIGtQqfY1dkKZGyiuOce2H13+Ne/oo4kyUycaEX8+vaFZs1gzhzvl3MuDcRLFLsGM56mkpsgcuRdqS5jjBljLYqnnvLWxDZ//QV33QWPPw677QZXX231mTxJOJcW4s16KgPsHNyqxvycc8tI99xjn4W9ekUdSZL45hto2RIefdQGbKZN8yJ+zqWZeC2KRap6b6lFkgK+/RZGjYInnoBKlaKOJgmsXQv/939Qo4ZdVHL00VFH5JwLQbxE4aOPedxzD9StC5dkegGTsWPh0EOtJtPw4baoUJUqUUflnAtJvK6nzqUWRQoYNw6+/BL69IHKlaOOJiLLl9sar5065Rbxa9/ek4Rzaa7AFoWqrizNQJLdPfdY1/ulabnadyFU4Z134KqrYNUqG7j2In7OZQyvoZCAH36w2nWPPJKhX56vuQaefdaWJv3qK2jRovDXOOfShieKBNxzD9SuDZdfHnUkpUgVNm+22umnnmrL9/XubaU4nHMZJZGigBlt/Hj4/HO44YYMak3873/QuTPcfrvdP+oouP56TxLOZShPFIW45x6oVQuuuCLqSErB1q0297dFC/jxR2jSJOqInHNJwLue4pgwAT77DB54IANW55w5E/75T2tCnXiiLdm3555RR+WcSwKeKOK45x6oWdNW7Ex72dmwcCEMHgxnneVF/Jxz23jXUwEmToRPP7VF2KpWjTqakIwfD7fdZj83a2ZjEz16eJJwzm3HE0UB7r0XdtnFLh1IO+vX2+j8wQfDa6/BsmBl2/Llo43LOZeUPFHk46ef4OOP4dproVq1qKMpYV9/bYPVjz9uddK9iJ9zrhA+RpGPe++1OndXXx11JCVs7VpbjrRGDUsYRx4ZdUTOuRTgLYo8fv7Z1t7p3RuqV486mhIyerQNVucU8Zs82ZOEcy5hnijyuPdeSxDXXBN1JCVg2TI4+2y7YO7NN+2xgw7K4KqGzrmi8EQRY/Jk+OADSxI1akQdTTGowr//DU2bwvvv29KkXsTPOVdEPkYR4957bfC6d++oIymmq66Cfv2gY0d4+WWb+uqcc0XkiSIwZQq89x7ccYdNi0052dmwZYtNcT39dGjUyBKG12dyzhVTqF1PItJFRH4RkdkicnM+z/9DRCYHt3Ei0irMeOLp29curEvJ1sSvv9oypDkXzx15pFd6dc6VmNAShYiUAfoBXYFmwNkikrcPZC5whKq2BPoCA8KKJ55p02DoUJsOW7NmFBEU0ZYt8Nhj0LKlTddq2jTqiJxzaSjMrqf2wGxVnQMgIkOAk4HpORuo6riY7X8A6oUYT4H69rUS4tdeG8XRi2jGDDjvPKs1cvLJ8PzzsMceUUflnEtDYXY97QksiLmfFTxWkIuA4fk9ISK9RGSiiExcllNuooRMn567ymetWiW66/AtWQJvv21TtTxJOOdCEmaiyK+ynOa7ochRWKK4Kb/nVXWAqrZT1XZ1SrjcxH332WUF111XorsNxw8/wC232M9Nm1oRvzPP9CJ+zrlQhZkosoC9Yu7XAxbm3UhEWgIDgZNVdUWI8fzNzJkwZIiVEa9duzSPvIPWrbN+sUMOgbfeyi3iV65ctHE55zJCmIliAtBYRBqKSHmgBzAsdgMRqQ+8D5yrqrNCjCVf990HlSrZKp9J68sv4YAD4KmnbNFuL+LnnCtloQ1mq+oWEbkSGAGUAV5R1WkicmnwfH/gTqAW8LxY98kWVW0XVkyxZs2yNXquvz6JP3fXrrUrqmvWhDFj4PDDo47IOZeBRDXfYYOk1a5dO504cWKx93PeeXaB3dy5sOuuJRBYSRo1Co44wq6D+PFHu7K6UqWoo3LOpTAR+bGoX8QzstbTr79aV/9llyVZkliyxAanO3fOLeLXtq0nCedcpDIyUdx/P1SoAH36RB1JQBXeeMNaDh99ZAH27Bl1VM45B2Rgopg9276sX3op1K0bdTSBK66wvrAmTewK61tv9RlNzrmkkXFFAe+/3z6Db7wx4kCys2HzZmvanHWWXRdx+eVen8k5l3QyqkXxv/9ZD88ll8Buu0UYyC+/2GB1ThG/I47wSq/OuaSVUYnigQegbNkIWxObN8NDD0GrVjB1KrRoEVEgzjmXuIzpepo7F15/3WY6RVIWado0OPdcmDQJ/u//bGGhSJs1zjmXmIxJFA88ADvtBDflW02qFJQpAytXWj3z006LKAjnnNtxGdH1NG8eDBoE//oX7Bmvfm1JGzcuNzPtv79NufIk4ZxLMRmRKB580FoTN/9tjb2QrF1rqyAddpiVAV++3B4vmzENOOdcGkn7RDF/Prz6Klx0EdQrjWWRRo60In7PPWdlaadOTfLStM45F1/af8V98EH7t1RaE2vXwj/+YSsgjR0Lhx5aCgd1zrlwpXWLYsECePlluPBCqF8/xAN98QVs3Qo772wtip9/9iThnEsbaZ0oHnrI/s1ZFK7ELVpkg9PHHWdVBgHatIGKFUM6oHPOlb60TRRZWTBwIFxwAey9dwnvXNWmUTVrBp9+ahnJi/g559JU2o5RPPywlVMKpTVx2WXw4os2q2ngQCvm55z7m82bN5OVlcWGDRuiDiVjVKxYkXr16lGuBAuLpmWiWLgQXnoJzj8fGjQooZ3GFvHr2RNatrQStDulbaPMuWLLysqiatWqNGjQgGAVSxciVWXFihVkZWXRsGHDEttvWn7KPfywjS3femsJ7XDGDFuGNGeHnTpZpVdPEs7FtWHDBmrVquVJopSICLVq1SrxFlzafdItWgQDBtjyDsVOqJs3W+2P1q1h5kwbqHbO7RBPEqUrjPOddl1Pjzxin+85FbyLbNo0OOccm+p6xhnw7LNJtNKRc86VnrRqUSxeDP37W5HWffYp5s7KloXVq+H99+GddzxJOJfCPvjgA0SEmTNnbnts9OjRnHDCCdttd/755zN06FDABuJvvvlmGjduzAEHHED79u0ZPnx4sWN58MEHadSoEU2aNGHEiBH5bnPWWWfRunVrWrduTYMGDWjdujUAX3zxBW3btqVFixa0bduWUaNGFTueRKRVi+LRR4vZmhg71tasfuwxm8k0a5bXZ3IuDQwePJjDDjuMIUOGcPfddyf0mjvuuINFixYxdepUKlSowJIlS/jmm2+KFcf06dMZMmQI06ZNY+HChRxzzDHMmjWLMnkWLXv77be3/Xz99ddTvXp1AGrXrs3HH3/MHnvswdSpUzn++OP5/fffixVTItLmU3DJEnjhBaug0ajRDr54zRqr8fH88zawcfPNVp/Jk4RzJaZ3b+vJLUmtW8NTT8XfZu3atXz33Xd8/fXXnHTSSQklivXr1/PSSy8xd+5cKlSoAEDdunU588wzixXvRx99RI8ePahQoQINGzakUaNGjB8/noMPPjjf7VWVd955Z1vLoU3MOGnz5s3ZsGEDGzdu3BZjWNKm6+mxx2DjRrj99h184fDh0Ly5ZZnevWHKFC/i51wa+fDDD+nSpQv77bcfNWvW5Keffir0NbNnz6Z+/fpUq1at0G2vvfbabd1EsbeHckpDxPj999/Za6+9tt2vV69e3BbB2LFjqVu3Lo0bN/7bc++99x5t2rQJPUlAmrQoli61xkDPnpDP+SzYmjU2PWrXXW3tiI4dQ4vRuUxX2Df/sAwePJjevXsD0KNHDwYPHsyBBx5Y4OygHZ019OSTTya8raru0PEGDx7M2Wef/bfHp02bxk033cTIkSMTPnZxpEWieOwx2LAhwdaEKowYAcceC1Wrwpdf2qJCpZCVnXOla8WKFYwaNYqpU6ciImzduhUR4ZFHHqFWrVqsWrVqu+1XrlxJ7dq1adSoEfPnz2fNmjVUrVo17jGuvfZavv7667893qNHD27OU7a6Xr16LFiwYNv9rKws9ihgbeYtW7bw/vvv8+OPP273eFZWFqeeeiqvv/46++67b9zYSoyqptStbdu2GmvpUtXKlVV79tTCLVyoesopqqD62msJvMA5VxzTp0+P9Pj9+/fXXr16bfdYp06ddMyYMbphwwZt0KDBthjnzZun9evX1z/++ENVVfv06aPnn3++bty4UVVVFy5cqG+88Uax4pk6daq2bNlSN2zYoHPmzNGGDRvqli1b8t12+PDh2qlTp+0eW7VqlbZs2VKHDh0a9zj5nXdgohbxczflxygefxz++quQ1oQqvPIKNG0Kn39uF1t4ET/n0t7gwYM59dRTt3vstNNO49///jcVKlTgzTff5IILLqB169acfvrpDBw4cNsMo/vuu486derQrFkzDjjgAE455RTq1KlTrHiaN2/OmWeeSbNmzejSpQv9+vXbNuPp4osvZuLEidu2HTJkyN+6nZ577jlmz55N3759t42FLF26tFgxJUI0nz6zZNauXTvNOZnLl1stpxNPhMGD47zokkvscu1OnayI3w4NZDjnimrGjBk0bdo06jAyTn7nXUR+VNV2RdlfSo9RPPEErF8Pd9yRz5Nbt9pFFRUr2hXWbdpAr15en8k553ZQyn5qrlhhVTXOOMOWhdjOtGm2wlxOEb/DD/dKr845V0Qp+8n55JO2RPV2rYlNm6BvX2s9zJ4NBx0UWXzOOZNq3dupLozznZJdTytXwjPPwOmnwwEHBA9OmWKXZU+ZAj162AbFHHhyzhVPxYoVWbFihZcaLyUarEdRsYSXY07JRPHUU3at3J13xjxYvrwNWHz0EZx0UlShOedi1KtXj6ysLJYtWxZ1KBkjZ4W7kpRys57atGmnc+ZM5Jhj4L2rv4Fhw2yOLNgAdp7iWs4554o36ynUMQoR6SIiv4jIbBG5OZ/nRUSeCZ6fLCIHFrbPJUtA//yTF8tcBkceCR9+aPNkwZOEc86FILREISJlgH5AV6AZcLaI5J2f1BVoHNx6AS8Utt8NS1Yzp1Jzar83AK67zov4OedcyMJsUbQHZqvqHFXdBAwBTs6zzcnA68EV5j8ANURk93g7rZ89jyp7VLcifo8/DpUrhxO9c845INzB7D2BBTH3s4AOCWyzJ7AodiMR6YW1OAA2Vv7ftKle6RWA2sDyqINIEn4ucvm5yOXnIleTor4wzESR31y4vCPniWyDqg4ABgCIyMSiDsikGz8Xufxc5PJzkcvPRS4RmVj4VvkLs+spC9gr5n49YGERtnHOORehMBPFBKCxiDQUkfJAD2BYnm2GAecFs586AqtVdVHeHTnnnItOaF1PqrpFRK4ERgBlgFdUdZqIXBo83x/4DOgGzAbWAxcksOsBIYWcivxc5PJzkcvPRS4/F7mKfC5S7oI755xzpStliwI655wrHZ4onHPOxZW0iSKM8h+pKoFz8Y/gHEwWkXEi0iqKOEtDYeciZruDRGSriJxemvGVpkTOhYgcKSI/i8g0EfmmtGMsLQn8jVQXkY9F5L/BuUhkPDTliMgrIrJURKYW8HzRPjeLuth2mDds8Pt/wD5AeeC/QLM823QDhmPXYnQE/hN13BGei0OAXYKfu2byuYjZbhQ2WeL0qOOO8P9FDWA6UD+4v2vUcUd4Lm4FHg5+rgOsBMpHHXsI56ITcCAwtYDni/S5mawtilDKf6SoQs+Fqo5T1VXB3R+w61HSUSL/LwCuAt4Dwl91PjqJnIuewPuqOh9AVdP1fCRyLhSoKrYoxs5YothSumGGT1XHYO+tIEX63EzWRFFQaY8d3SYd7Oj7vAj7xpCOCj0XIrIncCrQvxTjikIi/y/2A3YRkdEi8qOInFdq0ZWuRM7Fc0BT7ILeKcA1qppdOuEllSJ9bibrwkUlVv4jDST8PkXkKCxRHBZqRNFJ5Fw8BdykqlvTfEW1RM5FWaAt0BmoBHwvIj+o6qywgytliZyL44GfgaOBfYEvRGSsqv4ZcmzJpkifm8maKLz8R66E3qeItAQGAl1VdUUpxVbaEjkX7YAhQZKoDXQTkS2q+mGpRFh6Ev0bWa6q64B1IjIGaAWkW6JI5FxcADyk1lE/W0TmAvsD40snxKRRpM/NZO168vIfuQo9FyJSH3gfODcNvy3GKvRcqGpDVW2gqg2AocDlaZgkILG/kY+Aw0WkrIhUxqo3zyjlOEtDIudiPtayQkTqYpVU55RqlMmhSJ+bSdmi0PDKf6ScBM/FnUAt4Pngm/QWTcOKmQmei4yQyLlQ1Rki8jkwGcgGBqpqvtMmU1mC/y/6AoNEZArW/XKTqqZd+XERGQwcCdQWkSzgLqAcFO9z00t4OOeciytZu56cc84lCU8Uzjnn4vJE4ZxzLi5PFM455+LyROGccy4uTxQuKQWVX3+OuTWIs+3aEjjeIBGZGxzrJxE5uAj7GCgizYKfb83z3LjixhjsJ+e8TA2qodYoZPvWItKtJI7tMpdPj3VJSUTWqurOJb1tnH0MAj5R1aEichzwmKq2LMb+ih1TYfsVkdeAWap6f5ztzwfaqeqVJR2LyxzeonApQUR2FpGvgm/7U0Tkb1VjRWR3ERkT84378ODx40Tk++C174pIYR/gY4BGwWuvC/Y1VUR6B49VEZFPg7UNporIWcHjo0WknYg8BFQK4ngreG5t8O/bsd/wg5bMaSJSRkQeFZEJYusEXJLAafmeoKCbiLQXW4tkUvBvk+Aq5XuBs4JYzgpifyU4zqT8zqNzfxN1/XS/+S2/G7AVK+L2M/ABVkWgWvBcbezK0pwW8drg3+uB24KfywBVg23HAFWCx28C7szneIMI1q4AzgD+gxXUmwJUwUpTTwPaAKcBL8W8tnrw72js2/u2mGK2yYnxVOC14OfyWCXPSkAv4Pbg8QrARKBhPnGujXl/7wJdgvvVgLLBz8cA7wU/nw88F/P6B4Bzgp9rYHWfqkT9+/Zbct+SsoSHc8Bfqto6546IlAMeEJFOWDmKPYG6wOKY10wAXgm2/VBVfxaRI4BmwHdBeZPy2Dfx/DwqIrcDy7AqvJ2BD9SK6iEi7wOHA58Dj4nIw1h31dgdeF/DgWdEpALQBRijqn8F3V0tJXdFvupAY2BuntdXEpGfgQbAj8AXMdu/JiKNsWqg5Qo4/nHASSJyQ3C/IlCf9KwB5UqIJwqXKv6BrUzWVlU3i8g87ENuG1UdEySS7sAbIvIosAr4QlXPTuAYfVR1aM4dETkmv41UdZaItMVq5jwoIiNV9d5E3oSqbhCR0VjZ67OAwTmHA65S1RGF7OIvVW0tItWBT4ArgGewWkZfq+qpwcD/6AJeL8BpqvpLIvE6Bz5G4VJHdWBpkCSOAvbOu4GI7B1s8xLwMrYk5A/AoSKSM+ZQWUT2S/CYY4BTgtdUwbqNxorIHsB6VX0TeCw4Tl6bg5ZNfoZgxdgOxwrZEfx7Wc5rRGS/4Jj5UtXVwNXADcFrqgO/B0+fH7PpGqwLLscI4CoJmlci0qagYziXwxOFSxVvAe1EZCLWupiZzzZHAj+LyCRsHOFpVV2GfXAOFpHJWOLYP5EDqupP2NjFeGzMYqCqTgJaAOODLqDbgPvyefkAYHLOYHYeI7G1jb9UW7oTbC2R6cBPIjIVeJFCWvxBLP/Fymo/grVuvsPGL3J8DTTLGczGWh7lgtimBvedi8unxzrnnIvLWxTOOefi8kThnHMuLk8Uzjnn4vJE4ZxzLi5PFM455+LyROGccy4uTxTOOefi+n8EyePNtHRodwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.70      0.79      0.74      5257\n",
      "           1       0.75      0.64      0.69      5055\n",
      "\n",
      "    accuracy                           0.72     10312\n",
      "   macro avg       0.72      0.72      0.72     10312\n",
      "weighted avg       0.72      0.72      0.72     10312\n",
      "\n",
      "\n",
      "Confusion Matrix: \n",
      " [[4169 1088]\n",
      " [1809 3246]]\n",
      "\n",
      "End of validation Result\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/bettyau/opt/anaconda3/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function plot_confusion_matrix is deprecated; Function `plot_confusion_matrix` is deprecated in 1.0 and will be removed in 1.2. Use one of the class methods: ConfusionMatrixDisplay.from_predictions or ConfusionMatrixDisplay.from_estimator.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAT8AAAEGCAYAAAAT05LOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAcCUlEQVR4nO3dfZRdVZnn8e+vKpVXCHkpEkISIdgRDNiARgRZQwd8SWxdDbqkO+hIlo2NMLHtGWVa0BltpeMwM/2iYKCNSCe0QjooNFGBwIoi4oAQAhITDImgISQQEgiEvFXVrWf+uLvCTVJ1656kbt1b9/w+a51V5+x7XvatJE/2Pvvs5ygiMDPLm6ZaV8DMrBYc/Mwslxz8zCyXHPzMLJcc/MwslwbVugKlWsc0x/GTW2pdDcvg6SeH17oKlsEedtIWe3U455h57ojY9nKhon0fe3LvsoiYdTjXq5a6Cn7HT27hkWWTa10Ny2DmsafVugqWwa9i+WGfY+vLBX61bFJF+7ZM+F3rYV+wSuoq+JnZQBAUorPWlThsDn5mlkkAnQz8yREOfmaWWSdu+ZlZzgRBu7u9ZpY3ARTc7TWzPPI9PzPLnQAKDZANysHPzDIb+Hf8HPzMLKMgfM/PzPInAtoHfuxz8DOzrESBw5oeXBcc/MwskwA63fIzszxyy8/Mcqf4kLODn5nlTADtMfDzIDv4mVkmgSg0QBJ4Bz8zy6wz3O01s5zxPT8zyylR8D0/M8ubYiZnBz8zy5kI0RbNta7GYXPwM7PMOn3Pz8zypjjg4W6vmeWOBzzMLIc84GFmuVXwQ85mljeBaI+BHzoG/jcws37lAQ8zy6VA7vaaWT55wMPMcieChnjUZeB/AzPrV8UBj+aKlt5I+r2kVZKekLQilY2RdJ+kdenn6JL9r5K0XtJaSTNLyt+RzrNe0rWSeu2XO/iZWWYFmipaKnRuRJwWEdPT9pXA8oiYCixP20iaBswGTgZmAddL6oqwNwCXAlPTMqu3izr4mVkmgeiMypZDdD6wKK0vAi4oKV8cEXsj4llgPXCGpAnAyIh4KCICuLnkmB45+JlZZhlafq2SVpQslx5wqgDulfRYyWfjI2IzQPo5LpVPBJ4rOXZjKpuY1g8sL8sDHmaWSfG9vRW3m7aWdGe7c3ZEbJI0DrhP0m/L7NtdUzLKlJfl4GdmGanP0thHxKb0c4ukO4AzgBclTYiIzalLuyXtvhGYXHL4JGBTKp/UTXlZ7vaaWSbFV1ce/mivpBGSjuxaB94P/AZYCsxJu80B7kzrS4HZkoZImkJxYOOR1DXeIenMNMp7cckxPXLLz8wyiVCWbm8544E70lMpg4BbIuIeSY8CSyRdAmwALixeN1ZLWgKsATqAuRFRSOe6HFgIDAPuTktZDn5mlllfPOQcEc8Ap3ZTvg14Tw/HzAPmdVO+Ajgly/Ud/Mwsk2I+P8/tNbPccSZnM8uh4qMubvmZWc50ze0d6Bz8zCwzp7Qys9wpprRyt9fMcsj3/Mwsd4pZXdztNbOcKU5vc/DLrYvPmMawIwo0NUHzoOBb9zzN71YP5borJ7N7ZxPjJ7Xxhfl/YMSRnfz09tHcdv24fcc++9RQ5i97mjefspuf3TGKxdeNR4Ix49v5wnV/4KixhTJXtmqaPuM1Lrt6E81Nwd23jmHJt8bXukp1yC2/XkmaBXwTaAZujIhrqnm9/vZ/blu/X6D6xhVv4q++/Dx/fNZOlt06hh/cMI45f/sC533kFc77yCtAMfD93Sen8OZTdlPogBu+PJHv3P9bjhpb4MarJ7D0X4/mE1e8UKuvlGtNTcHcrz/PVbNPYOvmFq67ax0PLzuKDeuG1rpqdacRZnhULXyn9NLzgQ8A04CLUhrqhrXxd0N425k7ATj9nB08+JNRB+3zs/8YzYwLioEwAgixZ3cTEbDz9WbGHtPejzW2UieevotNvx/MCxuG0NHexP13juKsma/Wulp1p2u0t5KlnlWz7XoGsD4inomINmAxxTTUjUHBFy96M3NnvoW7vjcWgONO3MNDy0YC8Isfj+KlTS0HHfbA0lGce8F2AAa1wF9f8xyXnXcSHzv9ZDY8PZSZF23rt69g+xt7TDsvbRq8b3vr5hZaJ/g/o+50RlNFSz2rZu16Sjm9H0mXdqW4fmnbwLnX9c93rmP+vU8z7/vPsHRhK6seHsHn/mkDP1rYytyZb2H3600MGrx/MtnfrhzOkGGdHH/SHgA62uHHN7cy/9613PL4aqa8dTf/fp3vMdVKd+/7il7zAedPP7zDo19UM/hVlFo6IhZExPSImH702IEzZWbsMR0AjGrt4OxZr/Lbx4fzpql7+V+Ln2H+sqeZccF2Jhy3d79j7r9z1L4uL8DvVg8D4Njj25DgT/5sO2tWjOi/L2H72bq5haOPbdu33TqhnW0vHNx6z7sAOqKpoqWeVbN2PaWcHvD27Gpi1+tN+9Yf+/mRHH/SHrZvLY4fdXbCLd8cz4c+8UYXtrOz2BWecf72fWWtx7Sz4emhbN9WDPorHziSyVP39N8Xsf2sfWI4E6e0MX7yXga1dDLj/O08fO9Rta5WXWqEbm81R3sfBaamdNPPU3zf5seqeL1+88pLg/jqJVMAKHTAuR/ezjvP3cEdN7byo4WtAJz9gVd5/+yX9x2z6uEjaJ3QzoTj3mhZjD2mg49/7gWu+PBUBrUE4ya2ccU3NvTvl7F9Ogti/pcm8vVbnqGpGe5dPIY/PO2R3oMMgC5tJRRVvKkh6U+Bb1B81OWmlIW1R9NPHRqPLJtcbherMzOPPa3WVbAMfhXLeS1ePqzINfqkcXHeTR+taN/bz77hsV7e3lYzVX3OLyLuAu6q5jXMrP81QsvPMzzMLBMnMzWzXApER2d9D2ZUwsHPzDJrhOltDn5mlk2422tmOeR7fmaWWw5+ZpY7gSh4wMPM8qgRBjwGfvg2s34VacCjr7K6SGqW9LikH6ftMZLuk7Qu/Rxdsu9VktZLWitpZkn5OyStSp9dK3WXo2d/Dn5mllmEKloq9DfAUyXbVwLLI2IqsDxtk5IhzwZOBmYB16ekyQA3AJcCU9Myq7eLOviZWUZ9l89P0iTgg8CNJcXnA4vS+iLggpLyxRGxNyKeBdYDZ0iaAIyMiIeimKzg5pJjeuR7fmaWWYZWXaukFSXbCyJiQcn2N4C/BY4sKRsfEZuL14nNkrre/jUReLhkv64Eye1p/cDyshz8zCyTCCh0Vhz8tvaU1UXSh4AtEfGYpBkVnKunBMkVJU4+kIOfmWXWR6O9ZwN/llLfDQVGSvoe8KKkCanVNwHYkvbvKUHyxrR+YHlZvudnZpkEfTPgERFXRcSkiDie4kDGTyPiPwNLgTlptznAnWl9KTBb0pCUJHkq8EjqIu+QdGYa5b245JgeueVnZhlVPZPzNcASSZcAG4ALASJitaQlwBqgA5gbEV1vPbscWAgMA+5OS1kOfmaWWV8ngI+I+4H70/o24D097DcPOCgjfESsAE7Jck0HPzPLLMNob91y8DOzTIqjvQN/uMDBz8wya4SXuTv4mVlm7vaaWe4Emebt1i0HPzPLrAF6vQ5+ZpZRQFQ+va1uOfiZWWbu9ppZLjX0aK+k6yjTtY+Iz1alRmZW17rm9g505Vp+K8p8ZmZ5FUAjB7+IWFS6LWlEROysfpXMrN41Qre31zkqks6StIaUY1/SqZKur3rNzKxOieisbKlnlUzQ+wYwE9gGEBG/Bs6pYp3MrN5FhUsdq2i0NyKeO+BNcIWe9jWzBheNP+DR5TlJ7wZC0mDgs+z/mjkzy5s6b9VVopJu72XAXIpvQ3oeOC1tm1luqcKlfvXa8ouIrcDH+6EuZjZQdNa6AoevktHeEyT9SNJLkrZIulPSCf1ROTOrQ13P+VWy1LFKur23AEuACcCxwG3ArdWslJnVt4jKlnpWSfBTRPxbRHSk5Xs0xO1OMztkjfyoi6QxafVnkq4EFlP8On8B/KQf6mZm9arOu7SVKDfg8RjFYNf1LT9d8lkAV1erUmZW31TnrbpKlJvbO6U/K2JmA0QI6nzqWiUqmuEh6RRgGjC0qywibq5WpcyszjVyy6+LpK8AMygGv7uADwAPAg5+ZnnVAMGvktHejwLvAV6IiE8CpwJDqlorM6tvjTzaW2J3RHRK6pA0EtgC+CFns7xqkGSmlbT8VkgaBXyH4gjwSuCRalbKzOqborKl7DmkoZIekfRrSaslfTWVj5F0n6R16efokmOukrRe0lpJM0vK3yFpVfrsWh2Qhqo7vQa/iPgvEbE9Iv4FeB8wJ3V/zSyv+qbbuxc4LyJOpZgwZZakM4ErgeURMRVYnraRNA2YDZwMzAKul9ScznUDcCkwNS2zert4uYec317us4hY2etXM7OG1BfP+UVEAK+nzZa0BHA+xUFWgEXA/cAXUvniiNgLPCtpPXCGpN8DIyPiIQBJNwMXAHeXu365e37/WK7ewHnlTnwonn5yODOPPa2vT2tVNOKBo2tdBcug6VN99Lbayu/5tUoqfRnagohY0LWRWm6PAX8EzI+IX0kaHxGbASJis6RxafeJwMMl59qYytrT+oHlZZV7yPnc3g42sxzKNpK7NSKm93iqiAJwWhpXuCM9U9yT7iJulCkvq5IBDzOz/fXxoy4RsZ1i93YW8KKkCQDp55a020Zgcslhk4BNqXxSN+VlOfiZWWbqrGwpew7p6NTiQ9Iw4L3Ab4GlwJy02xzgzrS+FJgtaYikKRQHNh5JXeQdks5Mo7wXlxzToz66AWBmudI3DzBPABal+35NwJKI+LGkh4Alki4BNgAXAkTEaklLgDVABzA3dZsBLgcWAsMoDnSUHeyAyqa3iWIa+xMi4muS3gQcExF+1s8shyp5hq8SEfEkcHo35dsozirr7ph5wLxuylcA5e4XHqSSbu/1wFnARWl7BzA/y0XMrME0QBr7Srq974qIt0t6HCAiXkmvsDSzvKrzebuVqCT4tac+eUDxJiUN8e4mMztUDZ3MtMS1wB3AOEnzKGZ5+R9VrZWZ1a/ofSR3IKjkvb3fl/QYxRuQAi6IiKeqXjMzq195aPml0d1dwI9KyyJiQzUrZmZ1LA/Bj+Kb2rqmkAwFpgBrKWZWMLMcysU9v4h4W+l2yvby6R52NzMbEDLP8IiIlZLeWY3KmNkAkYeWn6TPlWw2AW8HXqpajcysvuVltBc4smS9g+I9wB9WpzpmNiA0essvPdx8RET8936qj5nVOdHgAx6SBkVER7l09maWU40c/Ci+oe3twBOSlgK3ATu7PoyI26tcNzOrR32U1aXWKrnnNwbYRvGdHV3P+wXg4GeWVw0+4DEujfT+hoPz5DdA3DezQ9XoLb9m4AgO8eUgZtbAGiAClAt+myPia/1WEzMbGDK+nKhelQt+9Z2G1cxqptG7vd3m0Dcza+iWX0S83J8VMbOBIy/T28zM3pCDe35mZgcRjTEg4OBnZtm55WdmedToo71mZt1z8DOz3GmQZKZNta6AmQ1AUeFShqTJkn4m6SlJqyX9TSofI+k+SevSz9Elx1wlab2ktZJmlpS/Q9Kq9Nm1knodk3HwM7PMFJUtvegAPh8RbwXOBOZKmgZcCSyPiKnA8rRN+mw2xTdHzgKuTwmXAW4ALgWmpmVWbxd38DOz7Pqg5RcRmyNiZVrfATwFTATOBxal3RYBF6T184HFEbE3Ip4F1gNnSJoAjIyIhyIigJtLjumR7/mZWWYZRntbJa0o2V4QEQsOOp90PHA68CtgfERshmKAlDQu7TYReLjksI2prD2tH1heloOfmWUTZElmujUippfbQdIRFF+K9l8j4rUyt+t6Sq93SGn33O01s0y6XmDUB/f8kNRCMfB9v+TVGC+mrizp55ZUvhGYXHL4JGBTKp/UTXlZDn5mll3fjPYK+C7wVET8U8lHS4E5aX0OcGdJ+WxJQyRNoTiw8UjqIu+QdGY658Ulx/TI3V4zy0zRJ085nw18Algl6YlU9kXgGmCJpEuADcCFABGxWtISYA3FkeK5EVFIx10OLASGAXenpSwHPzPLpo+yukTEg/ScI6HbfKIRMQ+Y1035CuCULNd38DOzzDy318xyqRGmtzn4mVl2bvmZWe5U+BhLvXPwM7PsHPzMLG+6HnIe6Bz8zCwzdQ786OfgZ2bZ+O1t1pvpM17jsqs30dwU3H3rGJZ8a3ytq5RrUQj2XLodtTYx9H8fRduNO+l4cC9qAkY1MeSLR9LU2kxhTTtt/7AjHQQtnxzBoHOG7HeuPVe+SufmAsMXjen/L1IH/KhLGZJuAj4EbImITE9eN4KmpmDu15/nqtknsHVzC9fdtY6Hlx3FhnVDa1213Or4wW50XDPsLDZbWi4axuBPjQCg/Qe7aF+4iyFXHEnTCYMYumA0GiQ6txbY/Zev0PzuwWhQcTJCx8/3wvBGeHnjYWiAll81ExsspIJsqo3qxNN3sen3g3lhwxA62pu4/85RnDXz1VpXK7c6txToeKiNlg++8Z+PRrzx1z/2sG+ilYZqX6Cjjf0mYMWuoH3JLgZfPLz6la5jfZXVpZaq1vKLiAdSgsJcGntMOy9tGrxve+vmFk56+64a1ijf2q57ncGXj4Bd+/+LbPvOTjru2QNHiGHfHLWvvLCmnb3X7CBeLDDkSyP3BcO27+6k5S+Gw5Act/wC6JvEBjVV85RWki6VtELSinb21ro6faa7fIwN8PdlQOr4f3vR6CaaT2w56LPBfzWC4T8cy6D3DaX99t37ypuntTD85jEM+/Zo2r+3i9gbFNZ1EM8XDrr/l0fqrGypZzUPfhGxICKmR8T0FhrnL9XWzS0cfWzbvu3WCe1se+Hgf3xWfZ2r2in8so1df76NvV99jcLKNvZc/dp++wx675DivbwDNB0/CIaJzmc76FzdTufaDnb9+Tb2fGY78VyB3Z/d3k/fon70ZTLTWvJob5WsfWI4E6e0MX7yXra90MKM87dzzdzjal2tXBr86SMY/OkjACg83kb74t0M/Z8j6Xyug6bJxX8ChV+20fSm4ovAOjcV0Lim4oDHCwViQ4GmY5ppPqmFlguGFffZXGDPla8y7NpRNflONRXREN0YB78q6SyI+V+ayNdveYamZrh38Rj+8LRHeutJ27d30vlcAQRNxzQz+PMpQK5qp/37u9AgQGLw545Ao2reSaor9d6qq0Q1H3W5FZhB8e1NG4GvRMR3q3W9evToT0fy6E9H1roaVqL59ME0n14ciBr690d1u0/LzKG0zCz/H1XThObcPuMHNMSjLtUc7b2oWuc2s9pyy8/M8ieAwsCPfg5+ZpaZW35mlk8e7TWzPHLLz8zyxymtzCyPBMgDHmaWR/I9PzPLHXd7zSyfGmNurycsmllmfZXVRdJNkrZI+k1J2RhJ90lal36OLvnsKknrJa2VNLOk/B2SVqXPrpW6Syq3Pwc/M8uuK7NLb0vvFnJwxvcrgeURMRVYnraRNA2YDZycjrleUnM65gbgUmBqWnrNIu/gZ2bZRHG0t5Kl11NFPAC8fEDx+cCitL4IuKCkfHFE7I2IZ4H1wBmSJgAjI+KhiAjg5pJjeuR7fmaWXeW3/FolrSjZXhARC3o5ZnxEbAaIiM2SxqXyicDDJfttTGXtaf3A8rIc/MwsswyPumyNiOl9ddluyqJMeVnu9ppZdn13z687L6auLOnnllS+EZhcst8kYFMqn9RNeVkOfmaWTQCdFS6HZikwJ63PAe4sKZ8taYikKRQHNh5JXeQdks5Mo7wXlxzTI3d7zSwTEX02w6O7jO/ANcASSZcAG4ALASJitaQlwBqgA5gbEYV0qsspjhwPA+5OS1kOfmaWXWffvJeyTMb39/Sw/zxgXjflK4BTslzbwc/Msunq9g5wDn5mlpkTG5hZPjn4mVn+NEZiAwc/M8vGb28zs7zyPT8zyycHPzPLnQA6HfzMLHc84GFmeeXgZ2a5E0Bh4E/xcPAzs4wCwsHPzPLI3V4zyx2P9ppZbrnlZ2a55OBnZrkTAYVC7/vVOQc/M8vOLT8zyyUHPzPLn/Bor5nlUED4IWczyyVPbzOz3Inos1dX1pKDn5ll5wEPM8ujcMvPzPLHyUzNLI+c2MDM8iiAaIDpbU21roCZDTCRkplWsvRC0ixJayWtl3RlP9R+H7f8zCyz6INur6RmYD7wPmAj8KikpRGx5rBPXgG3/Mwsu75p+Z0BrI+IZyKiDVgMnF/1uieKOhq1kfQS8Ida16MKWoGtta6EZdKof2bHRcTRh3MCSfdQ/P1UYiiwp2R7QUQsSOf5KDArIj6Vtj8BvCsiPnM49atUXXV7D/cPpV5JWhER02tdD6uc/8x6FhGz+uhU6u70fXTuXrnba2a1shGYXLI9CdjUXxd38DOzWnkUmCppiqTBwGxgaX9dvK66vQ1sQa0rYJn5z6zKIqJD0meAZUAzcFNErO6v69fVgIeZWX9xt9fMcsnBz8xyycGvimo5dccOjaSbJG2R9Jta18Wqy8GvSkqm7nwAmAZcJGlabWtlFVgI9NVzbFbHHPyqp6ZTd+zQRMQDwMu1rodVn4Nf9UwEnivZ3pjKzKwOOPhVT02n7phZeQ5+1VPTqTtmVp6DX/XUdOqOmZXn4FclEdEBdE3deQpY0p9Td+zQSLoVeAg4UdJGSZfUuk5WHZ7eZma55JafmeWSg5+Z5ZKDn5nlkoOfmeWSg5+Z5ZKD3wAiqSDpCUm/kXSbpOGHca6F6e1ZSLqxXNIFSTMkvfsQrvF7SQe95aun8gP2eT3jtf5O0hVZ62j55eA3sOyOiNMi4hSgDbis9MOUSSaziPhULy+KngFkDn5m9czBb+D6BfBHqVX2M0m3AKskNUv6v5IelfSkpE8DqOhbktZI+gkwrutEku6XND2tz5K0UtKvJS2XdDzFIPvfUqvzP0k6WtIP0zUelXR2OnaspHslPS7p23Q/v3k/kv5D0mOSVku69IDP/jHVZbmko1PZmyXdk475haST+uS3abnjFxgNQJIGUcwTeE8qOgM4JSKeTQHk1Yh4p6QhwC8l3QucDpwIvA0YD6wBbjrgvEcD3wHOSecaExEvS/oX4PWI+Ie03y3AP0fEg5LeRHEWy1uBrwAPRsTXJH0Q2C+Y9eAv0zWGAY9K+mFEbANGACsj4vOSvpzO/RmKLxa6LCLWSXoXcD1w3iH8Gi3nHPwGlmGSnkjrvwC+S7E7+khEPJvK3w/8cdf9POAoYCpwDnBrRBSATZJ+2s35zwQe6DpXRPSU1+69wDRpX8NupKQj0zU+ko79iaRXKvhOn5X04bQ+OdV1G9AJ/Hsq/x5wu6Qj0ve9reTaQyq4htlBHPwGlt0RcVppQQoCO0uLgL+OiGUH7Pen9J5SSxXsA8XbJWdFxO5u6lLxfElJMygG0rMiYpek+4GhPewe6brbD/wdmB0K3/NrPMuAyyW1AEh6i6QRwAPA7HRPcAJwbjfHPgT8iaQp6dgxqXwHcGTJfvdS7IKS9jstrT4AfDyVfQAY3UtdjwJeSYHvJIotzy5NQFfr9WMUu9OvAc9KujBdQ5JO7eUaZt1y8Gs8N1K8n7cyvYTn2xRb+HcA64BVwA3Azw88MCJeonif7nZJv+aNbuePgA93DXgAnwWmpwGVNbwx6vxV4BxJKyl2vzf0Utd7gEGSngSuBh4u+WwncLKkxyje0/taKv84cEmq32r8agA7RM7qYma55JafmeWSg5+Z5ZKDn5nlkoOfmeWSg5+Z5ZKDn5nlkoOfmeXS/wdefSQScgurbwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# classification report\n",
    "clf_score(model4, X_train_sc, y_train, X_test_sc, y_test, train=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fb9361c9",
   "metadata": {},
   "source": [
    "## 5.5 Hyperparamter tuning"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1e03e0c4",
   "metadata": {},
   "source": [
    "### 5.5.1 Hyperparameter tuning with Grid Search"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "908dc144",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'max_features': 0.4, 'min_samples_leaf': 1, 'n_estimators': 200, 'random_state': 5}\n"
     ]
    }
   ],
   "source": [
    "clf_rf_grid = ensemble.RandomForestClassifier()\n",
    "params = {\n",
    "    \"max_features\": [0.4, \"auto\"],\n",
    "    \"n_estimators\": [10, 100, 200],\n",
    "    \"min_samples_leaf\": [1, 0.1],\n",
    "    \"random_state\": [5],\n",
    "}\n",
    "cv = model_selection.GridSearchCV(clf_rf_grid, params,\n",
    "                                  n_jobs=-1).fit(X_train, y_train)\n",
    "print(cv.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "91bb5003",
   "metadata": {},
   "source": [
    "### 5.5.2 Use best hyperparameters with RF"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "51e08ed2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.7697827773467805"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rf_best = ensemble.RandomForestClassifier(\n",
    "    **{\n",
    "        \"max_features\": 0.4,\n",
    "        \"min_samples_leaf\": 1,\n",
    "        \"n_estimators\": 200,\n",
    "        \"random_state\": 5,\n",
    "    })\n",
    "rf_best.fit(X_train, y_train)\n",
    "rf_best.score(X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "3e31eade",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation Result:\n",
      "\n",
      "accuracy score: 0.7698\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAaKElEQVR4nO3dfbQddX3v8ffHBBF5SNCA1fAQpFBEBYoRfEIRUYGrF5+qIIqglUaldd3rdcHqqg8U2/pQtboEgQqiBeGicBUtyrWioBcphCUPRgpEHiMgIpDwJBD43j9mQjbHkzn7nGTOOQnv11p7Zc/Mb//2d37ZZ3/2zOw9k6pCkqRVedJUFyBJmt4MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDYi2X5JAkP5vqOtakJAcl+b9DtDsuyUcmo6bJkOSGJHu39z+e5JSprkkCg2JKJFk/yYlJbkxyT5JfJNl3qusaRvtm9kCSe5P8NslXk2y0Jp+jqk6tqtcM0W5BVR29Jp97hSSV5L52PX+T5HNJZvTxXOu6JCcnWZ7kWSPmfzzJw+0Y353kwiQvnkD/b2//lu5L8u0kT1tFu63a5xq8VZIPjbevJxqDYmrMBG4GXgHMAj4CnJFk3lQWNQ6vr6qNgF2BFwJ/N7JBkpmTXtWat3O7nq8A3ga8e4rrWaMm4/8oyYbAm4GlwEGjNPnf7RhvBvwMOCtJxtH/c4HjgXcCzwDuB44drW1V3VRVG624Ac8HHgXOHG9fTzQGxRSoqvuq6uNVdUNVPVpV3wOuB16wqsck2TLJWUl+l+T3Sb60inZfSHJzkmVJLk2yx8Cy3ZIsbJf9Nsnn2vlPSXJK2+/dSS5J8owh1uM3wPeB57X9VJIPJLkWuLad97oklw18YtxprHUa3J2WxueT3J5kaZIrkqx4vpOTfGKgv/cmWZzkziRnD36CbWtbkOTaJHclOWbYN6SqWgz8P2CXgf4msl7bJjmvnXdHklOTzB6mhpGS7N8+/7Ikv06yTzv/sd1X7fRju7CSzGvH4T1JbgLOS/KDJIeP6PvyJG9q7++Q5IftmF6d5K3jLPXNwN3A3wPvWlWjqnoY+BrwJ8DTx9H/QcB3q+qCqrqX5kPXm5JsPMRjDwYuqKob1kBf6zSDYhpo35S3BxatYvkM4HvAjcA8YC5w+iq6u4TmDe1pwDeAbyZ5SrvsC8AXqmoTYFvgjHb+u2i2bLak+SNdADwwRN1bAvsBvxiY/QZgd2DHJLsCJwF/1fZ7PHB2ml1vw67Ta4CX04zPbJpP9r8fpZa9gH8C3go8s+13ZH+vo9kC2rlt99qx1rHtewdgD2BxOz3R9Upb47OA59CM98eHqWFEPbsBXwc+TDMmLwduGEcXr2if/7U0r5EDB/reEdga+Pd2a+CHbZvN23bHtp+8V+ymuWKM53oXcBrNGOzQjt1o67Q+cAiwpKruSPKyNoRXdXtZ+9DnApev6Keqfg08RPN6GcvBNOG0wur0tW6rKm9TeAPWA/4DOL6jzYuB3wEzR1l2CPCzjsfeRbMLBeAC4Chgzog27wYuBHYaot4bgHtpPiXeSLNpvkG7rIC9Btp+GTh6xOOvpnmjGmqdgL2Aa4AXAU8a0e5k4BPt/ROBTw8s2wh4GJg3UNvLBpafARzZsZ4FLAPua++fBqy/Ous1ynO8AfjFiLHdu73/ceCUVTzueODzHf8/ew9MP9YPTXAV8OyB5Ru367h1O/0PwEnt/bcBPx3luT825Gt7K5pdO7u00+fSfFAZrO2h9rV0O3Ae8IJx/v38CFgwYt5vgD3HeNwe7et4o9Xt64lwc4tiCiV5EvBvNH8shw/M/35WHmw7iOaT541VtXyIPj+U5Kp2N83dNFsKc9rF76H5dPRf7e6l17Xz/43mj/j0JLck+XSS9Tqe5g1VNbuqtq6q91fV4NbHzQP3twY+NPhJsF2XZw27TlV1HvAl4Bjgt0lOSLLJKE2fRRNcKx53L82Wx9yBNrcN3L+fJkxIsmhgvPcYaLNr2+ZtNFtJG67OeiXZPMnpaQ6OLwNOYeX/zXhsCfx6Ao9b4bH/o6q6B/h34IB21gHAqe39rYHdR6znQTS7h4bxTuCqqrqsnT4VePuI19YZ7Wtp86raq6ouHee63AuMfD1sAtwzxuPeBZzZvk5Wt691nkExRdr94yfSHDR7czX7aAGoqn1r5UG3U2n+sLfKGAcf2ze5I2h2q2xaVbNpDiKm7ffaqjqQZjfCp4BvJdmwqh6uqqOqakfgJTS7aA6e4KoNno74ZuAf2jeCFbenVtVpw65TW/cXq+oFNLsGtqfZ5TLSLTRvbMBjB1GfTvOJcKz+nzsw3j8dsayq6gzg58BHV3O9/olmfHaqZvffO2j/b8bpZppdh6O5D3jqwPRob+ojTxl9GnBgmm8cbQD8eOB5zh+xnhtV1fuGrPNg4NlJbktyG/A5mmAc8xt+SfbIH39D6d5RAn0Rza7EFY97NrA+zVboqvreAPgLHr/baUJ9PVEYFFPnyzT7iV8/4hP5aC4GbgU+mWTDNAefXzpKu42B5bS7PpJ8lIFPSEnekWSzqnqUZnMf4JEkr0zy/Hb/+jKaXTaPrM7Ktf4VWJBk9zQ2TPLf2oODQ61Tkhe2j1+P5k3wD6uo7RvAoUl2afd3/yPwn7XyQOXq+iRwWJI/WY312ph2t12SuYweeMM4kWZdX5XkSUnmtsdRAC4DDkiyXpL5wFuG6O8cmpD9e5pvIT3azv8esH2Sd7b9rdf+fzxnrA7b0NkW2I3mmNkuNF96+AYdB7VXqKqfDoT3aLcVgX4q8Po2WDZs1+GsdktpVd5I8/r/8Yj5E+nrCcGgmAJJtqY5ELoLcNuI3Ux/pKoeAV4P/ClwE7CEZnfISOfSfAvpGprdMH/g8buC9gEWJbmX5sD2AVX1B5pPnd+iCYmrgPNpdouslqpaCLyXZtfRXTQHgw8Z5zptQvPGfFe7Tr8H/nmU5/oRzbdUzqR5o96WlbtTVltVXUkzLh9ejfU6imZ31lKa3T1nTbCWi4FDgc+3fZ3Pyq2pj9Cs+13t831jiP4ebGvZe7B9+wb5GppxvIVm192naD5lr/hh5KhfwKAJg+9U1ZVVdduKG83r7nVZQ79PqKpFNF++OJXmOMfGwPtXLE/zo8zjRqnt61X1uC2rsfp6IsuIsZIk6XHcopAkdTIoJEmdDApJUieDQpLUaa07cducOXNq3rx5U12GJK1VLr300juqarOJPHatC4p58+axcOHCqS5DktYqSW4cu9Xo3PUkSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjr1FhRJTkpzneNfrmJ5knwxzTWOr8gqLpEoSZpafW5RnExzWutV2RfYrr0dRnN9BknSNNNbUFTVBcCdHU32pz0nfFVdBMxO8syx+n3ooTVVoSRpGFN5jGIuj7+ozhIef33jxyQ5LMnCJAtvvfWuSSlOktSYyqAY7VrBo15FqapOqKr5VTV/1qxNey5LkjRoKoNiCbDlwPQWNJdblCRNI1MZFGcDB7fffnoRsLSqbp3CeiRJo+jt7LFJTgP2BOYkWQJ8DFgPoKqOA84B9qO5MP39NBeLlyRNM70FRVUdOMbyAj7Q1/NLktYMf5ktSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKlTr0GRZJ8kVydZnOTIUZbPSvLdJJcnWZTk0D7rkSSNX29BkWQGcAywL7AjcGCSHUc0+wDwq6raGdgT+GySJ/dVkyRp/PrcotgNWFxV11XVQ8DpwP4j2hSwcZIAGwF3Ast7rEmSNE59BsVc4OaB6SXtvEFfAp4D3AJcCXywqh4d2VGSw5IsTLJw6dK7+qpXkjSKPoMio8yrEdOvBS4DngXsAnwpySZ/9KCqE6pqflXNnzVr0zVdpySpQ59BsQTYcmB6C5oth0GHAmdVYzFwPbBDjzVJksapz6C4BNguyTbtAeoDgLNHtLkJeBVAkmcAfwZc12NNkqRxmtlXx1W1PMnhwLnADOCkqlqUZEG7/DjgaODkJFfS7Ko6oqru6Or30Ufhmmv6qlrSVHva02DOnKmuQoN6CwqAqjoHOGfEvOMG7t8CvGa8/V5wwerXJmn6efDBJigOPHCqK9GgXoOiDzNnwp//+VRXIakPN94Id9451VVoJE/hIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE4GhSSpk0EhSepkUEiSOhkUkqROBoUkqZNBIUnqZFBIkjoZFJKkTgaFJKmTQSFJ6mRQSJI6GRSSpE5DBUWSlyb5YZJrklyX5Pok1w3xuH2SXJ1kcZIjV9FmzySXJVmU5PzxroAkqV8zh2x3IvA/gEuBR4Z5QJIZwDHAq4ElwCVJzq6qXw20mQ0cC+xTVTcl2XwctUuSJsGwQbG0qr4/zr53AxZX1XUASU4H9gd+NdDm7cBZVXUTQFXdPs7nkCT1bNhjFD9O8pkkL06y64rbGI+ZC9w8ML2knTdoe2DTJD9JcmmSg4esR5I0SYbdoti9/Xf+wLwC9up4TEaZV6M8/wuAVwEbAD9PclFVXfO4jpLDgMMANt98qyFLliStCUMFRVW9cgJ9LwG2HJjeArhllDZ3VNV9wH1JLgB2Bh4XFFV1AnACwPbbzx8ZNpKkHg37radZST6XZGF7+2ySWWM87BJguyTbJHkycABw9og23wH2SDIzyVNptlyuGu9KSJL6M+wxipOAe4C3trdlwFe7HlBVy4HDgXNp3vzPqKpFSRYkWdC2uQr4AXAFcDHwlar65URWRJLUj2GPUWxbVW8emD4qyWVjPaiqzgHOGTHvuBHTnwE+M2QdkqRJNuwWxQNJXrZiIslLgQf6KUmSNJ0Mu0XxPuBr7XGJAHcCh/RVlCRp+hj2W0+XATsn2aSdXtZnUZKk6aMzKJK8o6pOSfI/R8wHoKo+12NtkqRpYKwtig3bfzfuuxBJ0vTUGRRVdXz771GTU44kaboZ9gd3n06ySZL1kvwoyR1J3tF3cZKkqTfs12Nf0x7Afh3NaTe2Bz7cW1WSpGlj2KBYr/13P+C0qrqzp3okSdPMsL+j+G6S/6L5kd37k2wG/KG/siRJ08VQWxRVdSTwYmB+VT0M3EdzESJJ0jpurN9R7FVV5yV508C8wSZn9VWYJGl6GGvX0yuA84DXj7KsMCgkaZ031u8oPtb+e+jklCNJmm6G/R3FPyaZPTC9aZJP9FaVJGnaGPbrsftW1d0rJqrqLpqvykqS1nHDfj12RpL1q+pBgCQbAOv3V5akJ6oHH4RrrpnqKjRo2KA4BfhRkq/SHMR+N/C13qqS9IQ0axbcdhtccMFUV7Iu2njDsduMbtjrUXw6yRXA3jQXLjq6qs6d6JNK0mhmz4bdd5/qKtZVM2ZM9JHDblEAXAUsr6r/SPLUJBtX1T0TfWJJ0tph2G89vRf4FnB8O2su8O2eapIkTSPDfuvpA8BLgWUAVXUtsHlfRUmSpo9hg+LBqnpoxUSSmTQHtSVJ67hhg+L8JH8LbJDk1cA3ge/2V5YkaboYNiiOAH4HXAn8FXAO8Hd9FSVJmj7G/NZTkicBV1TV84B/7b8kSdJ0MuYWRVU9ClyeZKtJqEeSNM0M+zuKZwKLklxMc9EiAKrqv/dSlSRp2hg2KI7qtQpJ0rQ11hXungIsAP6U5kD2iVW1fDIKkyRND2Mdo/gaMJ8mJPYFPtt7RZKkaWWsXU87VtXzAZKcCFzcf0mSpOlkrC2Kh1fccZeTJD0xjRUUOydZ1t7uAXZacT/JsrE6T7JPkquTLE5yZEe7FyZ5JMlbxrsCkqR+de56qqoJn788yQzgGODVwBLgkiRnV9WvRmn3KcDrW0jSNDTsKTwmYjdgcVVd155Q8HRg/1Ha/TVwJnB7j7VIkiaoz6CYC9w8ML2knfeYJHOBNwLHdXWU5LAkC5MsXLr0d2u8UEnSqvUZFBll3shTk/8LcERVPdLVUVWdUFXzq2r+rFmbran6JElDGM+lUMdrCbDlwPQWwC0j2swHTk8CMAfYL8nyqvp2j3VJksahz6C4BNguyTbAb4ADgLcPNqiqbVbcT3Iy8D1DQpKml96CoqqWJzmc5ttMM4CTqmpRkgXt8s7jEpKk6aHPLQqq6hyaixwNzhs1IKrqkD5rkSRNTJ8HsyVJ6wCDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ16DYok+yS5OsniJEeOsvygJFe0twuT7NxnPZKk8estKJLMAI4B9gV2BA5MsuOIZtcDr6iqnYCjgRP6qkeSNDF9blHsBiyuquuq6iHgdGD/wQZVdWFV3dVOXgRs0WM9kqQJ6DMo5gI3D0wvaeetynuA74+2IMlhSRYmWbh06e/WYImSpLH0GRQZZV6N2jB5JU1QHDHa8qo6oarmV9X8WbM2W4MlSpLGMrPHvpcAWw5MbwHcMrJRkp2ArwD7VtXve6xHkjQBfW5RXAJsl2SbJE8GDgDOHmyQZCvgLOCdVXVNj7VIkiaoty2Kqlqe5HDgXGAGcFJVLUqyoF1+HPBR4OnAsUkAllfV/L5qkiSNX5+7nqiqc4BzRsw7buD+XwJ/2WcNkqTV4y+zJUmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ0MCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHUyKCRJnQwKSVIng0KS1MmgkCR1MigkSZ16DYok+yS5OsniJEeOsjxJvtguvyLJrn3WI0kav96CIskM4BhgX2BH4MAkO45oti+wXXs7DPhyX/VIkiamzy2K3YDFVXVdVT0EnA7sP6LN/sDXq3ERMDvJM3usSZI0TjN77HsucPPA9BJg9yHazAVuHWyU5DCaLQ4gD8+fv+kNa7bUtdWDs2D9pVNdxfTgWKzkWKzkWKy0bIuJPrLPoMgo82oCbaiqE4ATAJIsrLpr/uqXt/ZrxuJ+xwLHYpBjsZJjsVKShRN9bJ+7npYAWw5MbwHcMoE2kqQp1GdQXAJsl2SbJE8GDgDOHtHmbODg9ttPLwKWVtWtIzuSJE2d3nY9VdXyJIcD5wIzgJOqalGSBe3y44BzgP2AxcD9wKFDdH1CTyWvjRyLlRyLlRyLlRyLlSY8Fqn6o0MCkiQ9xl9mS5I6GRSSpE7TNig8/cdKQ4zFQe0YXJHkwiQ7T0Wdk2GssRho98IkjyR5y2TWN5mGGYskeya5LMmiJOdPdo2TZYi/kVlJvpvk8nYshjkeutZJclKS25P8chXLJ/a+WVXT7kZz8PvXwLOBJwOXAzuOaLMf8H2a32K8CPjPqa57CsfiJcCm7f19n8hjMdDuPJovS7xlquuewtfFbOBXwFbt9OZTXfcUjsXfAp9q728G3Ak8eapr72EsXg7sCvxyFcsn9L45XbcoPP3HSmOORVVdWFV3tZMX0fweZV00zOsC4K+BM4HbJ7O4STbMWLwdOKuqbgKoqnV1PIYZiwI2ThJgI5qgWD65Zfavqi6gWbdVmdD75nQNilWd2mO8bdYF413P99B8YlgXjTkWSeYCbwSOm8S6psIwr4vtgU2T/CTJpUkOnrTqJtcwY/El4Dk0P+i9EvhgVT06OeVNKxN63+zzFB6rY42d/mMdMPR6JnklTVC8rNeKps4wY/EvwBFV9Ujz4XGdNcxYzAReALwK2AD4eZKLquqavoubZMOMxWuBy4C9gG2BHyb5aVUt67m26WZC75vTNSg8/cdKQ61nkp2ArwD7VtXvJ6m2yTbMWMwHTm9DYg6wX5LlVfXtSalw8gz7N3JHVd0H3JfkAmBnYF0LimHG4lDgk9XsqF+c5HpgB+DiySlx2pjQ++Z03fXk6T9WGnMskmwFnAW8cx38tDhozLGoqm2qal5VzQO+Bbx/HQwJGO5v5DvAHklmJnkqzdmbr5rkOifDMGNxE82WFUmeAfwZcN2kVjk9TOh9c1puUVR/p/9Y6ww5Fh8Fng4c236SXl5V69wZM4cciyeEYcaiqq5K8gPgCuBR4CtVNerXJtdmQ74ujgZOTnIlze6XI6rqjikruidJTgP2BOYkWQJ8DFgPVu9901N4SJI6TdddT5KkacKgkCR1MigkSZ0MCklSJ4NCktTJoJBG0Z559rIkv2zPOjp7Dfd/Q5I57f1712Tf0ppmUEije6Cqdqmq59GcZO0DU12QNFUMCmlsP6c9cVqSbZP8oD3J3k+T7NDOf0aS/9Ne7+DyJC9p53+7bbsoyWFTuA7ShE3LX2ZL00WSGTSnfjixnXUCsKCqrk2yO3AszYnmvgicX1VvbB+zUdv+3VV1Z5INgEuSnLkOn4tL6yiDQhrdBkkuA+YBl9KcbXQjmotEfXPgzLTrt//uBRwMUFWPAEvb+X+T5I3t/S2B7QCDQmsVg0Ia3QNVtUuSWcD3aI5RnAzcXVW7DNNBkj2BvYEXV9X9SX4CPKWPYqU+eYxC6lBVS4G/Af4X8ABwfZK/gMeuP7zi+uQ/At7Xzp+RZBNgFnBXGxI70Fx6UlrrGBTSGKrqFzTXYT4AOAh4T5LLgUWsvOTmB4FXtmcnvRR4LvADYGaSK2jOXnrRZNcurQmePVaS1MktCklSJ4NCktTJoJAkdTIoJEmdDApJUieDQpLUyaCQJHX6/5tzjoYPfLmDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "roc auc score: 0.7692991324082609\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAEWCAYAAAB42tAoAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAA9vUlEQVR4nO3dd3hUZfbA8e+hd1CsNEFBFJAawIptdZEVsYuiLjYUe1376tr7T40oIipWcGUVsSCsomJZFVCkg4iUCNJVAiIkOb8/zo0zCclkCJm5M5PzeZ55MjP3zr1nbpI5c9/3vecVVcU555wrTZWwA3DOOZfaPFE455yLyROFc865mDxROOeci8kThXPOuZg8UTjnnIvJE4XbJiIyS0QOCzuOVCEiN4nI8JD2PUJE7gpj3xVNRAaIyIRyvtb/JhPME0UaE5FFIvK7iOSKyM/BB0e9RO5TVdur6seJ3EchEakpIveKyJLgfX4vIteJiCRj/yXEc5iI5EQ/p6r3qOr5CdqfiMjlIjJTRDaISI6IvC4i+yVif+UlIreLyMvbsw1VfUVVj45jX1slx2T+TVZWnijSX19VrQd0BroAN4YbzrYTkWqlLHodOBLoA9QHzgIGAY8lIAYRkVT7f3gMuAK4HNgR2BsYA/ytoncU43eQcGHu28VJVf2WpjdgEfCXqMcPAO9GPd4f+AL4BfgOOCxq2Y7A88AyYB0wJmrZscC04HVfAB2L7xNoAvwO7Bi1rAuwGqgePD4XmBNsfzywR9S6ClwCfA/8WMJ7OxLYBDQv9nxPIB9oHTz+GLgX+Br4FXirWEyxjsHHwN3A58F7aQ2cE8S8HlgIXBisWzdYpwDIDW5NgNuBl4N1Wgbv6+/AkuBY3By1v9rAC8HxmAP8A8gp5XfbJnifPWL8/kcAQ4B3g3i/AvaKWv4YsBT4DZgKHBK17HZgNPBysPx8oAfwv+BYLQeeAGpEvaY98F9gLbACuAnoDWwGtgTH5Ltg3YbAs8F2fgLuAqoGywYGx/z/gm3dFTz3WbBcgmUrg9/pdKAD9iVhS7C/XODt4v8HQNUgrh+CYzKVYn9DfivHZ03YAfhtO355Rf9BmgEzgMeCx02BNdi38SrAUcHjnYPl7wKvATsA1YFDg+e7Bv+gPYN/ur8H+6lZwj4nAhdExfMgMDS4fzywANgXqAbcAnwRta4GHzo7ArVLeG/3AZ+U8r4XE/kA/zj4IOqAfZj/h8gHd1nH4GPsA719EGN17Nv6XsGH1aHARqBrsP5hFPtgp+RE8QyWFDoBfwD7Rr+n4Jg3wz4AS0sUFwGLy/j9j8A+aHsE8b8CjIpafibQOFh2DfAzUCsq7i3B76lKEG83LLFWC97LHODKYP362If+NUCt4HHP4scgat9jgKeD38kuWCIv/J0NBPKAy4J91aZoovgr9gHfKPg97AvsHvWe74rxf3Ad9n/QNnhtJ6Bx2P+r6X4LPQC/bccvz/5BcrFvTgp8CDQKll0PvFRs/fHYB//u2DfjHUrY5lPAncWem0ckkUT/U54PTAzuC/bttVfweBxwXtQ2qmAfunsEjxU4IsZ7Gx79oVds2ZcE39SxD/v7opa1w75xVo11DKJee0cZx3gMcEVw/zDiSxTNopZ/DfQP7i8E/hq17Pzi24tadjPwZRmxjQCGRz3uA8yNsf46oFNU3JPK2P6VwJvB/dOBb0tZ789jEDzeFUuQtaOeOx34KLg/EFhSbBsDiSSKI4D5WNKqUsJ7jpUo5gH9tvd/y29Fb6nWJuu23fGqWh/7ENsH2Cl4fg/gFBH5pfAGHIwliebAWlVdV8L29gCuKfa65lgzS3GjgQNEpAnQC/uQ/DRqO49FbWMtlkyaRr1+aYz3tTqItSS7B8tL2s5i7MxgJ2IfgxJjEJFjRORLEVkbrN+HyDGN189R9zcChQMMmhTbX6z3v4bS3388+0JErhGROSLya/BeGlL0vRR/73uLyDvBwIjfgHui1m+ONefEYw/sd7A86rg/jZ1ZlLjvaKo6EWv2GgKsEJFhItIgzn1vS5wuTp4oMoSqfoJ923ooeGop9m26UdStrqreFyzbUUQalbCppcDdxV5XR1VHlrDPX4AJwKnAGcBIDb7WBdu5sNh2aqvqF9GbiPGWPgB6ikjz6CdFpAf2YTAx6unodVpgTSqryzgGW8UgIjWxpquHgF1VtRHwHpbgyoo3HsuxJqeS4i7uQ6CZiGSVZ0cicgh2RnUqdubYCGvvjx4xVvz9PAXMBdqoagOsrb9w/aVYk1xJim9nKXZGsVPUcW+gqu1jvKboBlUfV9VuWLPg3liTUpmvKyNOV06eKDLLo8BRItIZ66TsKyJ/FZGqIlIrGN7ZTFWXY01DT4rIDiJSXUR6Bdt4BrhIRHoGI4HqisjfRKR+Kft8FTgbOCm4X2gocKOItAcQkYYickq8b0RVP8A+LP8jIu2D97A/1g7/lKp+H7X6mSLSTkTqAHcAo1U1P9YxKGW3NYCawCogT0SOAaKHbK4AGotIw3jfRzH/xo7JDiLSFLi0tBWD9/ckMDKIuUYQf38RuSGOfdXH+gFWAdVE5J9AWd/K62Md27kisg8wOGrZO8BuInJlMGy5voj0DJatAFoWjhoL/r4mAA+LSAMRqSIie4nIoXHEjYh0D/7+qgMbsEEN+VH72jPGy4cDd4pIm+Dvt6OINI5nv650nigyiKquAl4EblXVpUA/7FvhKuyb1nVEfudnYd+852Kd11cG25gCXICd+q/DOqQHxtjtWGyEzgpV/S4qljeB+4FRQTPGTOCYbXxLJwEfAe9jfTEvYyNpLiu23kvY2dTPWEfr5UEMZR2DIlR1ffDaf2Pv/Yzg/RUunwuMBBYGTSolNcfFcgeQA/yInTGNxr55l+ZyIk0wv2BNKicAb8exr/HYl4H5WHPcJmI3dQFci73n9dgXhtcKFwTH5iigL3acvwcODxa/HvxcIyLfBPfPxhLvbOxYjia+pjSwhPZM8LrFWDNc4Znys0C74PiPKeG1j2C/vwlY0nsW6yx320EiLQXOpR8R+RjrSA3l6ujtISKDsY7uuL5pOxcWP6NwLklEZHcROShoimmLDTV9M+y4nCtLwhKFiDwnIitFZGYpy0VEHheRBSIyXUS6JioW51JEDWz0z3qsM/4trB/CuZSWsKanoHM0F3hRVTuUsLwP1tbcB7u46zFV7Vl8Peecc+FK2BmFqk7Cxs6Xph+WRFRVvwQaiUi8nV3OOeeSJMxiXE0pOgojJ3huefEVRWQQVueFunXrdttnn32SEqBzzqWT/HzYuBE2bIjcGm9Zzu78zLcUrFbVncuz3TATRUmloktsB1PVYcAwgKysLJ0yZUoi43LOuZS3aRN89x18/TVMnmw/582LLG+9l9Knj3Ba7bF0XzeBJm8MWVzefYWZKHIoemVqM6ySqXPOuSj5+TBnTiQhTJ4M06fDli22fLfdoEcPOOssOGCfdRz05rXU3HdPuPlm4Di7yZBy7z/MRDEWuFRERmGd2b8GV3Q651ylpQqLFhVNClOnWjMSQIMG0L07XHONJYfu3aFpUxAB3nwTLr4YVq2CW26psJgSlihEZCRWqG4nsVnBbsMKhaGqQ7EaOn2wK383YvMAOOdcpbJypSWD6MSwOih5WbMmdO4M555rCaFHD2jTBqoUH4a0YgVcdhm8/rq94N13oWvFXXGQsEShqqeXsVyxiWucc65SWL8evvmmaL/C4qDnoEoVaNcO+vaNnCnstx/UqBHHhpcuteRw991w3XVQvXqFxu1TEDrnXAJs3mz9CNFnCrNnW9MSQKtW0LOnnQh0724nAPW2Zcb7xYvh7bfh0kshKwuWLIHGial/6InCOee2U0EBzJ9f9Exh2jRLFgA772xnCaecYj+zsuy5cu/sqafghqCI8Eknwe67JyxJgCcK55zbJqqQk1M0KUydCr/9Zsvr1YNu3eCKKyL9Ci1aBJ3N22vePDj/fPjsM/jrX+Hppy1JJJgnCueci2Ht2qLNR19/bX3HYF0BnTrBgAGRfoV99oGqVRMQyMaNcPDBNlZ2xAg4++wKyj5l80ThnHOBjRutszk6MfwQTKwqAm3b2hf5wjOFTp1sZFJCzZ9vQ53q1IGXXrJRTbvtluCdFuWJwjlXKW3ZArNmFT1TmDXLvrADNG9uyeCCCywxdOsGDcs7t2F5bNoEd94J999vZxBnngm9eycxgAhPFM65jKcKCxZEksLkyXbmsGmTLd9xR0sG/frZz+7dk/6lvajPP4fzzrM+iXPOgb/9LcRgPFE45zLQsmVFm48mT4ZffrFltWvb2cHgwZF+hT33TFpzf9nuvBNuu816wMePh6OPLvs1CeaJwjmX1n75BaZMKZoYfvrJllWtahetnXpqpF+hXTuoloqffKqWrTp3tosr7r57Gy+sSJxUPFzOOVeiTZvs+oTofoX58yPL27SBQw+NnCl07mx9wClt7Vq46ipo3RpuvdUuze7bN+yoivBE4ZxLSfn5diVz8YqpeXm2fPfdLSGcfXbkIrYddgg35m02ejRccokli1tvDTuaUnmicM6FThV+/HHriqkbN9ryhg3tDOG664pWTE1by5db6Y033rAOkwkTbKxtivJE4ZxLuhUrtq6YumaNLatZE7p0sQuQC/sVWrcuoWJqOlu2zDqq778frr46RTtNIlI7Oudc2lu/3s4OovsVliyxZVWqQPv2Niw1umJqBRc/TQ2LFlkRv8sus7OIpUvTpq3ME4VzrsL88cfWFVPnzClaMfWAA+Dyyy0xdOmSMgN7Eic/H4YMgZtussx4yil2kUaaJAnwROGcK6eCApg7t2gT0nffRSqm7rKLnSGcdlrkIraddgo35qSbM8fa0L74wq6qfvrpkK/kKx9PFM65MqlaS0nxiqnr19vyevVs1NGVV0b6FZo3T6GL2MKwcSP06mUZ9cUXrQRHmh4QTxTOua2sWbN1xdSVK21Z9ep2fcJZZ0X6Fdq2TVDF1HQ0d64dkDp14JVXbDTTrruGHdV28UThXCW3YcPWFVMXLrRlIlY2+5hjImcKHTsmoWJqOvr9d7j9dnjoIXjhBTuDSIHyGxXBE4VzlciWLTBz5tYVUwsKbHmLFpYQLrwwUjG1QYNwY04LkyZZX8T339vPY48NO6IK5YnCuQxVUFC0Ymrh9JzRFVN79IDjj480IaV5C0k4/vUvO5No1Qo++ACOPDLsiCqcJwrnMsRPP21dMfXXX21ZnTrQtStcfHGkCalVq7TtW00NhUX8srKsVtOdd0LdumFHlRCeKJxLQ+vWbV0xddkyW1atml201r9/ZFhqylZMTUerV1tiaNMG/vlPmysi5PkiEs3/dJxLcb//vnXF1O+/jyzfe284/PCiFVNr1w4r2gymCq+/bjWa1q2zOSMqCU8UzqWQvLytK6bOmBGpmNqkiSWEgQPtZ7duaXWBb/patsza7d56y5qaPvjAhn9VEp4onAuJqg1DjU4K33wTqZjaqJF9Jv3jH5EmpLSumJrOfv4ZJk6EBx+0qworWTte5Xq3zoVoxYqizUeTJ9s0BGDXJXTtaiMrC5uQMq5iarpZuBDGjrXE0LWrVTJs1CjsqELhicK5BPjtt60rpi5dasuqVIEOHeCEEyIjkDp0yNCKqekoPx8efxxuvtl+Kf37W32mSpokwBOFc9vtjz+sGF70mcLcuZGKqXvuCQcdFEkKXbpk7CjK9DdrFpx3Hnz1lY1kGjo0LYv4VTRPFM5tg/z8kiumbtliy3fZxZLB6adbYsjKqoQVU9PVxo024bYIvPqqnUn4hSaAJwrnSqVqzdLFK6bm5try+vUj11oV9itU+oqp6Wj2bNh3X7sqcdQoK+K3885hR5VSPFE4F1i9euuKqatW2bIaNezz4+9/jzQhtW3rnc1pbeNGuxbikUdgxAgrh/uXv4QdVUryROEqpdzcrSum/vijLROxL5h9+kTOFLxiaob5+GO44AIrhnXhhXDccWFHlNI8UbiMt2WLXbQWfaYwe3akYuoee1gyGDw4UjG1fv1wY3YJdNttcMcdsNdedm3E4YeHHVHK80ThMkpBgZW3iD5T+PZbG5kE0LixnSWceGLkbGGXXcKN2SVJYRG/Hj3gmmssWdSpE3ZUaSGhiUJEegOPAVWB4ap6X7HlDYGXgRZBLA+p6vOJjMllDtWtK6ZOmVK0Ymq3blaap7BfoWVL72yudFatgiuusE6l226rFEX8KlrCEoWIVAWGAEcBOcBkERmrqrOjVrsEmK2qfUVkZ2CeiLyiqpsTFZdLX+vWFR2WOnkyLF9uy6pVs36E/v0jZwr77lvpKi24aKowciRcfrldAfmvf4UdUdpK5L9RD2CBqi4EEJFRQD8gOlEoUF9EBKgHrAXyEhiTSxO//25NRtH9CgsWRJbvvbfND1N4ptCpk1dMdVFycqzT6Z13oGdPePZZaN8+7KjSViITRVNgadTjHKBnsXWeAMYCy4D6wGmqWlB8QyIyCBgE0KJFi4QE68KTl2cXxBavmJqfb8ubNrVkcO65kYqplbiagovHqlU2Pekjj9gZRdWqYUeU1hKZKEpqCdZij/8KTAOOAPYC/isin6rqb0VepDoMGAaQlZVVfBsujajCDz9sXTH1999teaNGdpZwww2RiqlNmoQasksXCxbA22/bFZBdulhxLZ/wu0IkMlHkAM2jHjfDzhyinQPcp6oKLBCRH4F9gK8TGJdLop9/Ltp8NGVKpGJqrVpWlHPQoKIVU72z2W2TvDx49FG49Va72OWMM2zyb08SFSaRiWIy0EZEWgE/Af2BM4qtswQ4EvhURHYF2gILExiTS6Bff926YmpOji0rrJh64omRfoX27b1iqttOM2ZYEb/Jk+2iuSeftCThKlTCEoWq5onIpcB4bHjsc6o6S0QuCpYPBe4ERojIDKyp6npVXZ2omFzF2bSp5IqphfbaCw4+OHKm4BVTXYXbuNEulqtSxWo0nXqqn44miKimV5N/VlaWTpkyJewwKpX8fJgzp+jQ1OnTIxVTd901khB69LBCeY0bhxuzy2AzZ9rpqAh8+KENefMSvWUSkamqmlWe1/ooc1eEKixevHXF1A0bbHlhxdSrr44khmbN/IucS4ING6wf4tFH4YUXrIjfkUeGHVWl4Imiklu1auuKqauDxr8aNaBzZzjnnEhS2Htvr5jqQvDhh1bE78cf4eKLoV+/sCOqVDxRVCK5uXZ2EJ0YFi2yZSLQrh0ce2zRiqk1aoQasnN2FnHXXdCmDXzyCfTqFXZElY4nigy1ebP1I0T3K8yZU7Riao8e9uWsRw8bpuoVU11KKSiw09cDD4R//ANuv90vvw+JJ4oMUFAA8+cXbT6aNs2SBVg/X/fucPLJkYvYvGKqS1krV9rV1G3bWn2mY46xmwuNJ4o098ordlbwW3Ate926VuLisssiTUheMdWlBVX7g77iCmsnveOOsCNyAU8UaSwvz0pdtGxp/1s9eljFVC9r49LO0qVw0UXw3ntwwAEwfLh1mrmU4IkijY0ZY1c+DxniMzm6NLdmDXz+OTz2GFxyiX/bSTGeKNJYdradTfgcLC4tzZ8PY8fCtdfaOOylS31ERYryEfFp6rvvrIqyf/lyaScvD+6/38Zf3303rFhhz3uSSFmeKNJUdraNFDz33LAjcW4bfPedTSR0ww3Qpw/Mnu1F/NKANz2loTVrbHDIWWfBjjuGHY1zcdq40UpuVKsGo0fDSSeFHZGLkyeKNPTss1a99bLLwo7EuThMnw777Qd16sDrr1sRP/+Gk1a86SnN5Odbyf1DD7X/PedSVm6ujdvu3BleesmeO/xwTxJpyM8o0szbb1t114cfDjsS52L4739t6sJFi+DSS+GEE8KOyG0HP6NIM9nZ0Ly5F890Kezmm+Hoo21a0k8/tT9aH9GU1uJOFCLi85OFbNYsmDjRSnZU83NBl2oKK04efDDceKMVHDv44FBDchWjzEQhIgeKyGxgTvC4k4g8mfDI3FaeeMK+pJ1/ftiROBfl55+t4uTtt9vjY46Be+6BWrVCDctVnHjOKP4P+CuwBkBVvwO8IHyS/fILvPginHGGz/roUoQqjBhhNZneeQcaNAg7IpcgcTVgqOpSKVp+ND8x4bjSPP+8DUP3IbEuJSxebJ3VEyZY89Lw4VYW3GWkeM4olorIgYCKSA0RuZagGcolR0GBFf476CDo0iXsaJzDTnEnT7b20E8+8SSR4eJJFBcBlwBNgRygM3BxAmNyxYwbBz/84GcTLmTz5sGDD9r9Tp1gyRIrNuaTqGe8eH7DbVV1gKruqqq7qOqZwL6JDsxFZGdDkyZw4olhR+IqpS1b4N57LTncd5/NQAdQr164cbmkiSdRZMf5nEuAefNg/Hib06V69bCjcZXOt99aEb+bboK+fa2In8+jW+mU2pktIgcABwI7i8jVUYsaAF7YOkmGDIEaNazf0Lmk2rgRjjrKvqH85z9+SluJxRr1VAOoF6wTfVnlb8DJiQzKmd9+s9GHp57qlZhdEn37rdVnqlPHqrx26gQ77BB2VC5EpSYKVf0E+ERERqjq4iTG5AIvvADr13sntkuS9evtiuohQ+yP7+yz4bDDwo7KpYB4rqPYKCIPAu2BPy+1VNUjEhaVo6DARh726GE35xLq/ffhwgttOtIrrvBmJldEPJ3ZrwBzgVbAv4BFwOQExuSw4pvz5/vZhEuCG2+0sht168Lnn8Ojj/qIJldEPGcUjVX1WRG5Iqo56pNEB1bZZWfb4JJTTgk7Epex8vNtwvXDDrMqk7fcYsXEnCsmnjOKLcHP5SLyNxHpAjRLYEyV3g8/wHvvWUuA/9+6Crd8uTUtFRbx++tf4c47/Y/NlSqeRHGXiDQErgGuBYYDVyYyqMpuyBD7onfRRWFH4jKKqhUNa9fOLvf3kUwuTmU2PanqO8HdX4HDAUTkoEQGVZnl5sJzz9m8802ahB2NyxiLFsEFF8AHH8Ahh1gRv733DjsqlyZiXXBXFTgVq/H0vqrOFJFjgZuA2oCXp0uAl1+GX3+Fyy8POxKXUX79Fb75xiZcv/BCr8/ktkmsv5ZngfOBxsDjIvI88BDwgKrGlSREpLeIzBORBSJyQynrHCYi00RkVmXvJFe1IbFdu8IBB4QdjUt7s2dbbSaIFPEbPNiThNtmsZqesoCOqlogIrWA1UBrVf05ng0HZyRDgKOwqrOTRWSsqs6OWqcR8CTQW1WXiEilLiLz0Uc23enzz0PR6T+c2wabN8MDD1gHdf36cO65NoSurs9m7Mon1leLzapaAKCqm4D58SaJQA9ggaouVNXNwCigX7F1zgDeUNUlwX5WbsP2M052ts1e179/2JG4tDVlCnTvDrfeaiObvIifqwCxzij2EZHpwX0B9goeC6Cq2rGMbTcFlkY9zgF6Fltnb6C6iHyM1ZN6TFVfLL4hERkEDAJo0aJFGbtNT4sXw9ixcP31PtWwK6cNG2yoa61a8NZbcNxxYUfkMkSsRLG9c06U1HiiJey/G3Ak1kH+PxH5UlXnF3mR6jBgGEBWVlbxbWSEJ5+05qbBg8OOxKWdb76xIn5168Kbb0LHjtCoUdhRuQxSatOTqi6OdYtj2zlA86jHzYBlJazzvqpuUNXVwCSg07a+iXT3++82WvH446F58zJXd8789htcfDF062bD5QB69fIk4SpcIoc/TAbaiEgrEakB9AfGFlvnLeAQEakmInWwpqlKNx/3q6/C2rVe18ltg/feg/bt4emn4eqr7cIb5xIknlpP5aKqeSJyKTAem+joOVWdJSIXBcuHquocEXkfmA4UAMNVdWaiYkpFqtaJvd9+9mXQuTJdf72NamrXzuaL6Fm868+5ihVXohCR2kALVZ23LRtX1feA94o9N7TY4weBB7dlu5nks8/gu+9g2DAfEutiULXa81WrwpFHWof1TTd5fSaXFGU2PYlIX2Aa8H7wuLOIFG9CcuX0+ONWcmfAgLAjcSnrp5+sA+u22+zx0UfDv/7lScIlTTx9FLdj10T8AqCq04CWiQqoMlm61AapnHeezTrpXBGq8Mwz1sQ0YYJdZONcCOJpespT1V/F20Uq3NCh1ppw8cVhR+JSzo8/2jeIjz6y+SKeeQZatw47KldJxZMoZorIGUBVEWkDXA58kdiwMt+mTdYv0bcvtGoVdjQu5eTmwvTpNqrp/PO9PpMLVTx/fZdh82X/AbyKlRu/MoExVQqvvQarV/uQWBdl5ky45x67v99+VsRv0CBPEi50ohr7QmcR6aKq3yYpnjJlZWXplClTwg5ju6haOZ6NG60IoLfqVXKbN8O998Ldd0PDhvZH4fWZXAUTkamqmlWe18bzVeUREZkrIneKSPvy7MQV9eWXMHUqXHqpJ4lKb/Jku7L69tttgnQv4udSUDwz3B0uIrthkxgNE5EGwGuqelfCo8tQ2dnQoAGcfXbYkbhQbdgAvXtD7dpWEbJv37Ajcq5EcTV+qurPqvo4cBF2TcU/ExlUJlu+HF5/3aYIqFcv7GhcKKZMseFudetalddZszxJuJQWzwV3+4rI7SIyE3gCG/HULOGRZainn4b8fLjkkrAjcUn36682DWn37pEifgcfbP0SzqWweIbHPg+MBI5W1eLVX9022LzZEsUxx/iQ+Ern7bfhoovg55/h2mvh5JPDjsi5uMXTR7F/MgKpDEaPts8JHxJbyVx3HTz0kA15HTPGziicSyOlJgoR+beqnioiMyg64VC8M9y5YrKzoU0bK9XjMpyqtTFWq2a/8AYNrOprjRphR+bcNot1RnFF8PPYZASS6aZMsWGxjz3m109lvJwcm6qwY0e7NuKoo+zmXJqKNcPd8uDuxSXMbufVibZRdraNcho4MOxIXMIUFFgnVLt2MHEi7LZb2BE5VyHi+W5b0lehYyo6kEy2ciWMGgV//7u1QLgMtHAhHHGEdVj36AEzZnhnlMsYsfooBmNnDnuKyPSoRfWBzxMdWCZ55hkb8XTppWFH4hJmwwa7qnr4cLtIxi+5dxkkVh/Fq8A44F7ghqjn16vq2oRGlUG2bIEnn7Qm6n32CTsaV6FmzLAL5m65xUY0LV5sV1k7l2FiNT2pqi4CLgHWR90QkR0TH1pmePNNWLbMWyEyyh9/wD//CV272hSFK1fa854kXIYq64ziWGAqNjw2+lxagT0TGFfGyM62+Sb69Ak7ElchvvzSJhSaPRvOOgv+7/+gceOwo3IuoUpNFKp6bPDTp9Upp2nT4LPP7FqrqlXDjsZttw0b4G9/sxpN771nl9g7VwnEU+vpIBGpG9w/U0QeEZEWiQ8t/WVn21zY554bdiRuu3z1VaSI39tvWxE/TxKuEolneOxTwEYR6QT8A1gMvJTQqDLAmjXw6qtw5pmwww5hR+PK5ZdfbBrS/fePFPE78ECoXz/UsJxLtngSRZ7aNHj9gMdU9TFsiKyLYfhwmxfbh8SmqTFj7MK5ESOs9MYpp4QdkXOhiad67HoRuRE4CzhERKoC1RMbVnrLy7MhsYcfbqMmXZq5+mrrpO7UyZqaunULOyLnQhVPojgNOAM4V1V/DvonHkxsWOnt7bdhyRJ49NGwI3Fxiy7i16ePjWT6xz+gun8nck6sVamMlUR2BQprI3+tqisTGlUMWVlZOmXKlLB2H5cjjoAffrBbtXhSsQvXkiVWeqNLFyvi51wGEpGpqppVntfGM+rpVOBr4BRs3uyvRMRnXSnFzJnw0Udw8cWeJFJeQYG1EbZvD598Ak2ahB2Rcykpno+ym4HuhWcRIrIz8AEwOpGBpasnnoBatWywjEthCxbYuOVPP7X6KsOGQcuWYUflXEqKJ1FUKdbUtIb4RktVOuvWwUsvwRln+MW6KW/TJpg/H55/3sr6ehE/50oVT6J4X0TGY/Nmg3Vuv5e4kNLX88/Dxo1e1yllTZtmRfxuuw06dIBFi+z0zzkXU5lnBqp6HfA00BHoBAxT1esTHVi6yc+HIUPg4IOhc+ewo3FFbNoEN98MWVnw1FORIn6eJJyLS6z5KNoADwF7ATOAa1X1p2QFlm7GjbO5a+69N+xIXBFffGFF/ObOtSamRx6BHb34sXPbItYZxXPAO8BJWAXZ7KRElKays6FpUzjhhLAjcX/asAH69rX2wPfft6usPUk4t81i9VHUV9VngvvzROSbZASUjubOhQkT4M47/fqslPC//0HPnlbE7513rD/C6zM5V26xzihqiUgXEekqIl2B2sUel0lEeovIPBFZICI3xFivu4jkp+v1GU88ATVqwKBBYUdSya1bZ0NeDzzQhp8BHHCAJwnntlOsM4rlwCNRj3+OeqzAEbE2HNSEGgIcBeQAk0VkrKrOLmG9+4Hx2xZ6avjtN3jhBTjtNNhll7CjqcTeeAMuuQRWrYIbb7RfiHOuQsSauOjw7dx2D2CBqi4EEJFRWAXa2cXWuwz4D5ESIWllxAjIzfUhsaG66iorrNW5s00o1KVL2BE5l1ESWWSiKbA06nEO0DN6BRFpCpyAnZ2UmihEZBAwCKBFi9SZM6mgwJqdevaE7mmZ5tJYdBG/Y4+107lrr/VOIucSIJFXWJd0qWvxCoSPAteran6sDanqMFXNUtWsnXfeuaLi224TJsD33/vZRNItWgS9e8Ott9rjI4+05iZPEs4lRCITRQ7QPOpxM2BZsXWygFEisgg4GXhSRI5PYEwVKjsbdtvN57RJmoICO+gdOtj1EXvsEXZEzlUK8VSPlWCu7H8Gj1uISI84tj0ZaCMirUSkBtAfGBu9gqq2UtWWqtoSKzJ4saqO2dY3EYYFC+wiuwsvtBFPLsG+/x569YLLL4dDDrEyvRddFHZUzlUK8ZxRPAkcAJwePF6PjWaKSVXzgEux0UxzgH+r6iwRuUhE0v4/fMgQqFrVEoVLgs2bbYKPF1+0Dms/m3AuaeLpzO6pql1F5FsAVV0XnCGUSVXfo1gBQVUdWsq6A+PZZirIzYXnnrMmp913DzuaDPbtt1bE7/bbbc6IRYugZs2wo3Ku0onnjGJLcK2Dwp/zURQkNKoU99JLdv2Ed2InyKZN1jndvTs8/bRdGwGeJJwLSTyJ4nHgTWAXEbkb+Ay4J6FRpTBVGxLbrRvsv3/Y0WSgzz6DTp3gvvvg7LNh9mxIoZFuzlVGZTY9qeorIjIVOBIb8nq8qs5JeGQpauJE++waMcLnuqlwubnQrx80aGBjj486KuyInHPEkShEpAWwEXg7+jlVXZLIwFJVdjbstJNXiKhQn31m9Znq1YN337Xhr/XqhR2Vcy4QT9PTu1i58XeBD4GFwLhEBpWqFi2Ct9+24n8+500FWLPGmpcOOSRSxG///T1JOJdi4ml62i/6cVA5tlIOCn3ySWtuGjw47EjSnCqMHg2XXgpr19oV1v37hx2Vc64U21zrSVW/EZFKV9lo40YYPtwmJmrWLOxo0txVV8Fjj9mIgAkTrPPaOZey4umjuDrqYRWgK7AqYRGlqFdesekOfEhsOalCXp7VYzruOGjSBK6+2or6OedSWjx9FPWjbjWxvop+iQwq1ahaJ3bHjtac7rbRjz/C0UdHivgdcQT84x+eJJxLEzH/U4ML7eqp6nVJiiclTZoEM2bAM8/4kNhtkp9vF53cdJPVO/Hqic6lpVIThYhUU9W8eKc9zWTZ2bDDDnDGGWFHkkbmz4eBA23+6mOOsSusmzcv82XOudQT64zia6w/YpqIjAVeBzYULlTVNxIcW0pYuhTGjLHm9Dp1wo4mjeTlweLF8PLLlmH9VMy5tBVPI/GOwBpsFjrFrs5WoFIkiqeesj6Kiy8OO5I0MGWKFfG7805o1w4WLvT6TM5lgFiJYpdgxNNMIgmiUPGZ6jLSpk3WL3HccdCyZdjRpLDff4fbboOHH7aZnC6/3OozeZJwLiPEGvVUFagX3OpH3S+8ZbxRo2D1ah8SG9Mnn9hwsAcfhPPOg1mzvIifcxkm1hnFclW9I2mRpJjCIbHt28Phh4cdTYrKzYUTT4RGjeDDD23Yq3Mu48RKFJW69/F//4NvvrE+Cu+HLebTT+Ggg6wm07hxlk3r1g07KudcgsRqejoyaVGkoOxsaNgQzjwz7EhSyOrVdkB69YoU8evRw5OEcxmu1EShqmuTGUgqWbbMatade64XMgWsHe6112wk02uvWce1F/FzrtLwGgolePppu6j4kkvCjiRFXHGFnWJ17259EfvtV/ZrnHMZwxNFMZs3W6Lo0wf22ivsaEKkClu2QI0aVjJ3jz3gyiutFIdzrlKJpyhgpfL667BiRSUfEvvDD3DkkXDLLfb48MPhmms8SThXSXmiKCY7G/beu5JO15yfD488Yk1LU6dC27ZhR+ScSwHe9BTl66/hq6/g8cehSmVLoXPnwt//bgehb18bF9y0adhROedSgCeKKNnZNsrp738PO5IQFBTYcK+RI+G00/ziEefcnyrb9+ZSrVhhIz8HDoQGDcKOJkm+/hpuvtnut2tnfRP9+3uScM4V4YkiMGyYDfK59NKwI0mCjRvh2mvhgAPghRdgVTCzbY0a4cblnEtJniiwBDF0qM3WmfH9tx99ZJ3VDz8MF1zgRfycc2XyPgrgjTesef7pp8OOJMFyc2060kaNLGEcdljYETnn0oCfUWCd2HvuaTN2ZqSPP7bO6sIiftOne5JwzsWt0ieKb7+Fzz+3vomMu55s1So4/XS7YO7ll+257t19Tlfn3Dap9E1P2dn2uXnOOWFHUoFUbZjr5ZfD+vU2NakX8XPOlVOlPqNYvRpefRXOPtua7TPGZZfBgAHQpo2dMt1yi49ocs6VW6U+oxg+HP74I0OGxBYUQF6eJYSTT4bWrS1hZFx7mnMu2RJ6RiEivUVknogsEJEbSlg+QESmB7cvRKRTIuOJlpcHTz5ps3e2b5+svSbI99/bGym8eO6ww7zSq3OuwiQsUYhIVWAIcAzQDjhdRNoVW+1H4FBV7QjcCQxLVDzFjR0LS5emeZXYvDx46CHo2BGmTYN99w07IudcBkpk01MPYIGqLgQQkVFAP2B24Qqq+kXU+l8CzRIYTxHZ2TbFQt++ydpjBZszxzpXpkyBfv3s9KhJk7Cjcs5loEQ2PTUFlkY9zgmeK815wLiSFojIIBGZIiJTVhWWm9gOM2bYpQUXX5zmrTOFBarefNOThHMuYRKZKEqqLKclrihyOJYori9puaoOU9UsVc3auQLKTTzxBNSqBeedt92bSq4vv4Qbb7T7++5rRfxOPdWL+DnnEiqRiSIHaB71uBmwrPhKItIRGA70U9U1CYwHgHXr7NqzAQOgceNE762CbNgAV10FBx4Ir7wSKeJXvXq4cTnnKoVEJorJQBsRaSUiNYD+wNjoFUSkBfAGcJaqzk9gLH967jkrnpo2ndgffAAdOsCjj1pbmRfxc84lWcI6s1U1T0QuBcYDVYHnVHWWiFwULB8K/BNoDDwp1nySp6pZiYopP9+anQ45BDolbSDudsjNtSuqd9wRJk2ywJ1zLskSesGdqr4HvFfsuaFR988Hzk9kDNHefRcWLYIHHkjWHstp4kQ49FAr4jd+vE0qVLt22FE55yqpSlXCIzvbpoE+/viwIynFihXWOX3kkZEift26eZJwzoWq0iSKOXOsuX/w4BTsA1aFl16yM4e33oK774Yzzgg7KuecAypRracnnrAySBdcEHYkJbjkEnjqKZua9Nln/Qpr51xKqRSJ4tdfbWro/v1hl13CjiZQUGBzsNasCaedZskh7a8AdM5lokrR9DRihF2KcPnlYUcSmDfPOqsLi/gdeqhXenXOpayMTxQFBdbsdMAB1i8cqi1b4L77bGzuzJmw334hB+Scc2XL+Kan8eNhwQK4446QA5k1C846yyYSOvFEGDIEdtst5KCcc65sGZ8osrPt8/ikk0IOpGpVWLsWRo9OgWCccy5+Gd309P33MG4cXHRRSDOBfvEFXB/UOdxnHzu18SThnEszGZ0ohgyxayYuvDDJO87NtZ7zgw+2MuCrV9vz1TL+BM45l4EyNlHk5sLzz8MppyS5K2DCBCvi98QTNhn3zJmw005JDMA55ypWxn7FffFF+O23JFeJzc2N1C//9FM46KAk7tw55xIjI88oVO0LfVYW9OyZhB3+979WmrZePTujmDbNk4RzLmNkZKL48EOr7XTZZQme/G35cuucPvpom1AIoEsXmz7POecyREYmiuxsm9vntNMStANVu9y7XTurXX7ffV7EzzmXsTKuj2LhQnj7bbjpJiujlBCDB8PTT9uopuHDoW3bBO3IufS2ZcsWcnJy2LRpU9ihVBq1atWiWbNmVK/AMtkZlyiefBKqVLFrJypUdBG/M86Ajh1tJ1Uy8qTMuQqRk5ND/fr1admyJZLQdmAHoKqsWbOGnJwcWrVqVWHbzahPuQ0brEr3iSdCs2YVuOE5c2wa0ptusse9elmlV08SzsW0adMmGjdu7EkiSUSExo0bV/gZXEZ90r3yCvzySwUOid2yBe65Bzp3hrlzraPaObdNPEkkVyKOd8Y0PalaJ3anTtZ1sN1mzYIzz7ShrqecYhvfddcK2LBzzqWXjDmj+OQTuwi6wobEVqtmMx698Qb8+9+eJJxLY2+++SYiwty5c/987uOPP+bYY48tst7AgQMZPXo0YB3xN9xwA23atKFDhw706NGDcePGbXcs9957L61bt6Zt27aMHz++xHVOO+00OnfuTOfOnWnZsiWdO3cG4JVXXvnz+c6dO1OlShWmTZu23TGVJWPOKLKzYccdt3OU6qef2pzVDz1kI5nmz/f6TM5lgJEjR3LwwQczatQobr/99rhec+utt7J8+XJmzpxJzZo1WbFiBZ988sl2xTF79mxGjRrFrFmzWLZsGX/5y1+YP38+VYtNWvbaa6/9ef+aa66hYcOGAAwYMIABAwYAMGPGDPr16/dnEkmkjPgUXLIExoyB666D2rXLsYH16+GGG2zIVKtWdn+nnTxJOFeBrrzSWnIrUufO8OijsdfJzc3l888/56OPPuK4446LK1Fs3LiRZ555hh9//JGawTj7XXfdlVNPPXW74n3rrbfo378/NWvWpFWrVrRu3Zqvv/6aAw44oMT1VZV///vfTJw4catlI0eO5PTTT9+ueOKVEU1PTz1lPwcPLseLx42D9u1tI1deCTNmeBE/5zLImDFj6N27N3vvvTc77rgj33zzTZmvWbBgAS1atKBBgwZlrnvVVVcVaQ4qvN13331brfvTTz/RvHnzPx83a9aMn376qdRtf/rpp+y66660adNmq2WvvfZa0hJF2n9l/v13eOYZ6NcP9thjG1+8fj2cfTbssovNHbH//gmJ0TlX9jf/RBk5ciRXXnklAP3792fkyJF07dq11NFB2zpq6P/+7//iXldVt2l/pZ01fPXVV9SpU4cOHTrEve/tkfaJYtQoWLNmG4bEqtr8qEcdBfXrwwcf2KRCCbuM2zkXljVr1jBx4kRmzpyJiJCfn4+I8MADD9C4cWPWrVtXZP21a9ey00470bp1a5YsWcL69eupX79+zH1cddVVfPTRR1s9379/f2644YYizzVr1oylS5f++TgnJ4cmTZqUuN28vDzeeOMNpk6dutWyUaNGJe1sArAMl063bt26aaGCAtUuXVQ7dLD7ZVq2TPX441VB9YUX4niBc257zJ49O9T9Dx06VAcNGlTkuV69eumkSZN006ZN2rJlyz9jXLRokbZo0UJ/+eUXVVW97rrrdODAgfrHH3+oquqyZcv0pZde2q54Zs6cqR07dtRNmzbpwoULtVWrVpqXl1fiuuPGjdNevXpt9Xx+fr42bdpUf/jhh1L3U9JxB6ZoOT9307qP4osv4NtvbX6gmGeLqvDcc7DvvvD++/DAA17Ez7lKYOTIkZxwwglFnjvppJN49dVXqVmzJi+//DLnnHMOnTt35uSTT2b48OF/jjC666672HnnnWnXrh0dOnTg+OOPZ+edd96ueNq3b8+pp55Ku3bt6N27N0OGDPlzxNP555/PlClT/ly3tLOGSZMm0axZM/bcc8/timVbiJbQZpbKsrKytPBg9u9vrUg5OVC3bowXXXghDBtmpTeGD4cSOoaccxVvzpw57LvvvmGHUemUdNxFZKqqZpVne2nbR7FsGfznPzY1dYlJIj/fSnDUqmVXWHfpAoMGeX0m55zbRmn7qTl0qOWCSy4pYeGsWTbDXGERv0MO8UqvzjlXTmn5yfnHHzYdxN/+BkWa6TZvhjvvtLOHBQuge/fQYnTOmXRr3k53iTjeadn09PrrsHJlsSGxM2bAgAH2s39/ePxxm+bOOReaWrVqsWbNGi81niQazEdRq4KnY07LRPH441aK6S9/iXqyRg3YuNFqNR13XGixOecimjVrRk5ODqtWrQo7lEqjcIa7ipR2iWLDBpsaIjsbqnz6CYwdCw8/bJlj3jwoVlzLORee6tWrV+hMay4cCe2jEJHeIjJPRBaIyA0lLBcReTxYPl1Eupa1zZUroUm93xg0bTAcdphVA1y92hZ6knDOuQqXsEQhIlWBIcAxQDvgdBFpV2y1Y4A2wW0Q8FRZ281f+yszaU+N54fB1Vd7ET/nnEuwRJ5R9AAWqOpCVd0MjAL6FVunH/BicIX5l0AjEdk91kZbsog6uze0y7Iffhjq1ElM9M4554DE9lE0BZZGPc4BesaxTlNgefRKIjIIO+MA+KPW97NmeqVXAHYCVocdRIrwYxHhxyLCj0VE2/K+MJGJoqSxcMUH+MazDqo6DBgGICJTynsZeqbxYxHhxyLCj0WEH4sIEZlS9lolS2TTUw7QPOpxM2BZOdZxzjkXokQmislAGxFpJSI1gP7A2GLrjAXODkY/7Q/8qqrLi2/IOedceBLW9KSqeSJyKTAeqAo8p6qzROSiYPlQ4D2gD7AA2AicE8emhyUo5HTkxyLCj0WEH4sIPxYR5T4WaVdm3DnnXHKlZVFA55xzyeOJwjnnXEwpmygSUf4jXcVxLAYEx2C6iHwhIp3CiDMZyjoWUet1F5F8ETk5mfElUzzHQkQOE5FpIjJLRD5JdozJEsf/SEMReVtEvguORTz9oWlHRJ4TkZUiMrOU5eX73CzvZNuJvGGd3z8AewI1gO+AdsXW6QOMw67F2B/4Kuy4QzwWBwI7BPePqczHImq9idhgiZPDjjvEv4tGwGygRfB4l7DjDvFY3ATcH9zfGVgL1Ag79gQci15AV2BmKcvL9bmZqmcUCSn/kabKPBaq+oWqrgsefoldj5KJ4vm7ALgM+A+wMpnBJVk8x+IM4A1VXQKgqpl6POI5FgrUF5sUox6WKPKSG2biqeok7L2Vplyfm6maKEor7bGt62SCbX2f52HfGDJRmcdCRJoCJwBDkxhXGOL5u9gb2EFEPhaRqSJydtKiS654jsUTwL7YBb0zgCtUtSA54aWUcn1upup8FBVW/iMDxP0+ReRwLFEcnNCIwhPPsXgUuF5V8zN8RrV4jkU1oBtwJFAb+J+IfKmq8xMdXJLFcyz+CkwDjgD2Av4rIp+q6m8Jji3VlOtzM1UThZf/iIjrfYpIR2A4cIyqrklSbMkWz7HIAkYFSWInoI+I5KnqmKREmDzx/o+sVtUNwAYRmQR0AjItUcRzLM4B7lNrqF8gIj8C+wBfJyfElFGuz81UbXry8h8RZR4LEWkBvAGclYHfFqOVeSxUtZWqtlTVlsBo4OIMTBIQ3//IW8AhIlJNROpg1ZvnJDnOZIjnWCzBzqwQkV2xSqoLkxplaijX52ZKnlFo4sp/pJ04j8U/gcbAk8E36TzNwIqZcR6LSiGeY6Gqc0TkfWA6UAAMV9USh02mszj/Lu4ERojIDKz55XpVzbjy4yIyEjgM2ElEcoDbgOqwfZ+bXsLDOedcTKna9OSccy5FeKJwzjkXkycK55xzMXmicM45F5MnCuecczF5onApKaj8Oi3q1jLGurkVsL8RIvJjsK9vROSAcmxjuIi0C+7fVGzZF9sbY7CdwuMyM6iG2qiM9TuLSJ+K2LervHx4rEtJIpKrqvUqet0Y2xgBvKOqo0XkaOAhVe24Hdvb7pjK2q6IvADMV9W7Y6w/EMhS1UsrOhZXefgZhUsLIlJPRD4Mvu3PEJGtqsaKyO4iMinqG/chwfNHi8j/gte+LiJlfYBPAloHr7062NZMEbkyeK6uiLwbzG0wU0ROC57/WESyROQ+oHYQxyvBstzg52vR3/CDM5mTRKSqiDwoIpPF5gm4MI7D8j+Cgm4i0kNsLpJvg59tg6uU7wBOC2I5LYj9uWA/35Z0HJ3bStj10/3mt5JuQD5WxG0a8CZWRaBBsGwn7MrSwjPi3ODnNcDNwf2qQP1g3UlA3eD564F/lrC/EQRzVwCnAF9hBfVmAHWx0tSzgC7AScAzUa9tGPz8GPv2/mdMUesUxngC8EJwvwZWybM2MAi4JXi+JjAFaFVCnLlR7+91oHfwuAFQLbj/F+A/wf2BwBNRr78HODO43wir+1Q37N+331L7lpIlPJwDflfVzoUPRKQ6cI+I9MLKUTQFdgV+jnrNZOC5YN0xqjpNRA4F2gGfB+VNamDfxEvyoIjcAqzCqvAeCbypVlQPEXkDOAR4H3hIRO7Hmqs+3Yb3NQ54XERqAr2BSar6e9Dc1VEiM/I1BNoAPxZ7fW0RmQa0BKYC/41a/wURaYNVA61eyv6PBo4TkWuDx7WAFmRmDShXQTxRuHQxAJuZrJuqbhGRRdiH3J9UdVKQSP4GvCQiDwLrgP+q6ulx7OM6VR1d+EBE/lLSSqo6X0S6YTVz7hWRCap6RzxvQlU3icjHWNnr04CRhbsDLlPV8WVs4ndV7SwiDYF3gEuAx7FaRh+p6glBx//HpbxegJNUdV488ToH3kfh0kdDYGWQJA4H9ii+gojsEazzDPAsNiXkl8BBIlLY51BHRPaOc5+TgOOD19TFmo0+FZEmwEZVfRl4KNhPcVuCM5uSjMKKsR2CFbIj+Dm48DUisnewzxKp6q/A5cC1wWsaAj8FiwdGrboea4IrNB64TILTKxHpUto+nCvkicKli1eALBGZgp1dzC1hncOAaSLyLdaP8JiqrsI+OEeKyHQscewTzw5V9Rus7+JrrM9iuKp+C+wHfB00Ad0M3FXCy4cB0ws7s4uZgM1t/IHa1J1gc4nMBr4RkZnA05Rxxh/E8h1WVvsB7Ozmc6z/otBHQLvCzmzszKN6ENvM4LFzMfnwWOecczH5GYVzzrmYPFE455yLyROFc865mDxROOeci8kThXPOuZg8UTjnnIvJE4VzzrmY/h+wlbZG9n21EAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Classification Report: \n",
      "               precision    recall  f1-score   support\n",
      "\n",
      "           0       0.76      0.79      0.78      5257\n",
      "           1       0.78      0.74      0.76      5055\n",
      "\n",
      "    accuracy                           0.77     10312\n",
      "   macro avg       0.77      0.77      0.77     10312\n",
      "weighted avg       0.77      0.77      0.77     10312\n",
      "\n",
      "\n",
      "Confusion Matrix: \n",
      " [[4174 1083]\n",
      " [1291 3764]]\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/bettyau/opt/anaconda3/lib/python3.9/site-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function plot_confusion_matrix is deprecated; Function `plot_confusion_matrix` is deprecated in 1.0 and will be removed in 1.2. Use one of the class methods: ConfusionMatrixDisplay.from_predictions or ConfusionMatrixDisplay.from_estimator.\n",
      "  warnings.warn(msg, category=FutureWarning)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "End of validation Result\n",
      "\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAT8AAAEGCAYAAAAT05LOAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAb5ElEQVR4nO3de5hV1Z3m8e9bxV1E5WpZYEBFE7TjjRDsXMRoB0wyjem0CeaiTw8ZE4fEdLo7EzXpTicOaTsz6e6YqIkxPmInSuMkGbGjosEQdR6UizEiGISIIoJcVUAE6vKbP86q8ghVp86BOnVOnf1+nmc9tfc6+7JOlf5Yl73XUkRgZpY1dZUugJlZJTj4mVkmOfiZWSY5+JlZJjn4mVkm9al0AfINH1ofY8f0rXQxrATPPjWo0kWwEuzldfbHPh3ONaaed0Rs39FS1LHLn9q3ICKmHc79yqWqgt/YMX1ZsmBMpYthJZh63BmVLoKV4PFYeNjX2LajhccXjC7q2L4Nfxx+2Dcsk6oKfmbWGwQt0VrpQhw2Bz8zK0kArfT+lyMc/MysZK245mdmGRMETW72mlnWBNDiZq+ZZZH7/MwscwJoqYHZoBz8zKxkvb/Hz8HPzEoUhPv8zCx7IqCp98c+Bz8zK5Vo4bBeD64KDn5mVpIAWl3zM7Mscs3PzDIn95Bz7w9+nszUzEoSQFPUFZW6Iul5SSskPSlpWcobKulBSWvSz2Pyjr9a0lpJqyVNzcs/O11nraTrJXUZnR38zKwkgWihrqhUpPMi4oyImJj2rwIWRsR4YGHaR9IEYAZwKjANuFFSfTrnJuByYHxKXU6g6uBnZiVrDRWVDtF0YE7angNclJc/NyL2RcQ6YC0wSVIDMCQiFkduIfLb887plIOfmZWkrc+vmAQMl7QsL13eweUekLQ877NREbEJIP0cmfIbgRfzzt2Q8hrT9oH5BXnAw8xKJFqK6M9LtuU1ZzvynojYKGkk8KCkPxS88cGiQH5BDn5mVpLcTM7d02iMiI3p5xZJvwQmAZslNUTEptSk3ZIO3wDkL/IzGtiY8kd3kF+Qm71mVpIIsT/qi0qFSDpC0pFt28AHgaeB+cBl6bDLgLvT9nxghqT+ksaRG9hYkprGuyRNTqO8l+ad0ynX/MysZK3d85zfKOCX6amUPsAdEXG/pKXAPEkzgfXAxQARsVLSPGAV0AzMioi2NTSvAG4DBgL3pVSQg5+ZlSQ34HH4jcaIeA44vYP87cD5nZwzG5jdQf4y4LRS7u/gZ2YlKmnAo2o5+JlZSbpzwKOSHPzMrGQth/4Ac9Vw8DOzkgSiKXp/6Oj938DMelR3DXhUmoOfmZUkkJu9ZpZNHvAws8yJwI+6mFn25AY8Cr+61hs4+JlZyTzgYWaZExzWRKVVw8HPzErmmp+ZZU5u3V4HPzPLHNXE0pUOfmZWktzSlR7tNbOMiZCbvWaWTX7I2cwyJzefn/v8zCxzPJOzmWVQ7lEX1/zMLGP8bq+ZZZantDKzzMlNaeVmr5llkPv8zCxzcrO6uNlrZhmTe73NwS+zLp00gYGDW6irg/o+wQ/uf7b9s7tuGsEt1zYyb8UKjhrWAsBzqwZw/VfH8PquOurq4Pv3Pku/AcFXPnYSOzb3od+AAOCf5v6Ro4c3V+Q7GUycspPPX7uR+rrgvjuHMu8HoypdpCrkml+XJE0DvgfUA7dExHXlvF9P+85da9uDW5stL/Xldw8fycjG/e15Lc3wnS++ja9c/wInnrqXnTvqqe8b7Z9/9YYXOPn0N3qs3Naxurpg1rdf4uoZJ7BtU1++f+8aHltwFOvXDKh00apOLbzhUbbwLakeuAG4EJgAXCJpQrnuVy1+9I+NzPz6RpT338by3x7JuHe8wYmn7gVgyNAW6nv/Y1I155Qz97Dx+X68vL4/zU11LLr7aM6Z+lqli1V12kZ7i0nVrJw1v0nA2oh4DkDSXGA6sKqM9+w5Cq655EQQfPgz2/nQp7ezeMEQhh/b1B7k2mx4bgASXHPJCby2vQ/nTn+Vj8/a0v75d798PHV18N4Pv8on/3rzWwKn9ZxhxzaxdWO/9v1tm/ry9rP2VLBE1cvN3sIagRfz9jcA7z7wIEmXA5cDHN/Ye7og//XuNQw7tplXt/XhqhknMuakvdx5/Sj+6c4/HnRsSzM8veQIvn/vs/Qf2MpVnziJ8e/cw5nv281Xf/ACwxua2LO7jms/O5Zf/59j+LOLX6nAN7KO/tGJODgv67p7DY/USlwGvBQRH5E0FPgPYCzwPPDxiHglHXs1MBNoAa6MiAUp/2zgNmAgcC/wpYjCf71yhu+OfjsHFSYibo6IiRExccSw3tMWHHZsblDi6OHNvGfaazy1eDAvr+/HFRe8nUsnTWDrpr7MmnoKO7b0YURDE+8853WOGtbCgEHBuz6wk7UrBgIwvKEJgEGDWznvo6+y+neDKvadsm7bpr6MOO7NvtrhDU1sf7lvBUtUnQJojrqiUpG+BDyTt38VsDAixgML0z6p22wGcCowDbgxBU6Am8hVosanNK2rm5Yz+G0AxuTtjwY2lvF+PWbvnjr27K5r317+2yM5+Yw9zFuxktuXrOL2JasY0dDEDQtWM3RkM2dP2cW6VQPYu0e0NMNTiwdz/Mn7aGmG17bn/nbNTfD4r4cw9u17C93aymj1k4NoHLefUWP20advK1Omv8pjDxxV6WJVpdaoKyp1RdJo4MPALXnZ04E5aXsOcFFe/tyI2BcR64C1wCRJDcCQiFicanu3553TqXK2M5cC4yWNA14iF7E/Wcb79ZhXtvbhmzPHAbkm7XkffZV3nber0+OPPLqFv/jcVr74oZORYNIHdvLuC3ayd08d13zyRFqaRUsLnPW+3Vz4qe099TXsAK0t4oavNfLtO56jrh4emDuUF571SO9Bolubvf8G/A/gyLy8URGxCSAiNkkamfIbgcfyjtuQ8prS9oH5BZUt+EVEs6QvAAvIPepya0SsLNf9elLD2/bzw1+vLnjM7UveOq5z/sde4fyPvbUvb8CgVm5Y8CxWPZY+NISlDw2pdDGqWomTmQ6XtCxv/+aIuBlA0keALRGxXNKUIq7VWVdaUV1sByrrCENE3Euu89HMakgJNb9tETGxk8/eA/y5pA8BA4Ahkn4KbJbUkGp9DUDboxGddaVtSNsH5hfU+8erzaxHtU1mWkwqeJ2IqyNidESMJdct9lBEfBqYD1yWDrsMuDttzwdmSOqfutPGA0tSE3mXpMmSBFyad06nes+zJWZWFQLR3FrWetN1wDxJM4H1wMUAEbFS0jxyzwo3A7Miou0Vqyt481GX+1IqyMHPzErW3a+3RcQiYFHa3g6c38lxs4HZHeQvA04r5Z4OfmZWmvB8fmaWQV7AyMwyy8HPzDInEC3lHfDoEQ5+ZlayWpjPz8HPzEoSHvAws6wKBz8zy57unc+vUhz8zKxkrvmZWeZEQEurg5+ZZZBHe80scwI3e80skzzgYWYZVQur2jn4mVnJ3Ow1s8zJjfb63V4zyyA3e80sk9zsNbPMCeTgZ2bZVAOtXgc/MytRQPj1NjPLIjd7zSyTanq0V9L3KdC0j4gry1IiM6tqWXi3d1mPlcLMeo8Aajn4RcSc/H1JR0TE6+UvkplVu1po9nb5joqkcyStAp5J+6dLurHsJTOzKiWitbhUzYp5Qe/fgKnAdoCI+D3w/jKWycyqXRSZqlhRo70R8aL0lijeUp7imFnVi9of8GjzoqQ/BUJSP+BKUhPYzDKqymt1xSim2ft5YBbQCLwEnJH2zSyzVGSqXl0Gv4jYFhGfiohRETEiIj4dEdt7onBmVqVai0wFSBogaYmk30taKembKX+opAclrUk/j8k752pJayWtljQ1L/9sSSvSZ9frgH66jhQz2nuCpHskbZW0RdLdkk7o6jwzq1Ftz/kVkwrbB3wgIk4n16KcJmkycBWwMCLGAwvTPpImADOAU4FpwI2S6tO1bgIuB8anNK2rmxfT7L0DmAc0AMcBdwF3FnGemdWoiOJS4WtERMTutNs3pQCmA23PGc8BLkrb04G5EbEvItYBa4FJkhqAIRGxOCICuD3vnE4VE/wUEf8eEc0p/ZSa6O40s0NW/KMuwyUty0uX519GUr2kJ4EtwIMR8TgwKiI2AaSfI9PhjcCLeadvSHmNafvA/IIKvds7NG3+RtJVwNz0dT4B/KqrC5tZDSv+UZdtETGx08tEtABnSDoa+KWk0wpcq6ObRoH8ggo96rL8gAt/7oALX9vVxc2sNqmb234R8aqkReT66jZLaoiITalJuyUdtgEYk3faaGBjyh/dQX5BnTZ7I2JcRJyQfh6YPOBhllUhaC0yFSBpRKrxIWkgcAHwB2A+cFk67DLg7rQ9H5ghqb+kceQGNpakpvEuSZPTKO+leed0qqg3PFJVdAIwoP37R9xezLlmVoO6p+bXAMxJI7Z1wLyI+E9Ji4F5kmYC64GLASJipaR5wCqgGZiVms0AVwC3AQOB+1IqqMvgJ+kbwBRywe9e4ELgUXIjKmaWRd0Q/CLiKeDMDvK3A+d3cs5sYHYH+cuAQv2FBylmtPcvU0Fejoi/Ak4H+pdyEzOrMRmZ2OCNiGiV1CxpCLnOR/f5mWVVrU9mmmdZ6pT8MbkR4N3AknIWysyqW3eP9lZCl8EvIv572vyhpPvJPUn9VHmLZWZVrZaDn6SzCn0WEU+Up0hmVu1qveb33QKfBfCBbi4Lzz41iKnHndHdl7Uy+vJaT+3Ym3xp+hvdc6Fa7vOLiPN6siBm1kv0gpHcYnjRcjMrnYOfmWWRupiotDdw8DOz0tVAza+YmZwl6dOS/iHtHy9pUvmLZmbVSFF8qmbFvN52I3AOcEna3wXcULYSmVn1655p7CuqmGbvuyPiLEm/A4iIV9ISlmaWVVVeqytGMcGvKU05E5Cbg4su12Uys1pW7U3aYhQT/K4HfgmMlDSb3CwvXy9rqcysekVGRnsj4meSlpOb1krARRHhx/rNsiwLNT9JxwN7gHvy8yJifTkLZmZVLAvBj9xKbW0LGQ0AxgGryS0cbGYZlIk+v4j4k/z9NNvL5zo53MysVyj5DY+IeELSu8pRGDPrJbJQ85P0N3m7dcBZwNaylcjMqltWRnuBI/O2m8n1Af68PMUxs16h1mt+6eHmwRHxlR4qj5lVOVHjAx6S+kREc6Hp7M0so2o5+JFboe0s4ElJ84G7gNfbPoyIX5S5bGZWjXrBjC3FKKbPbyiwndyaHW3P+wXg4GeWVTU+4DEyjfQ+zZtBr00NxH0zO1S1XvOrBwbz1qDXpga+upkdshqIAIWC36aI+FaPlcTMeocMrN5W3dOwmlnF1Hqz9/weK4WZ9S41EPw6XcMjInb0ZEHMrPdQa3Gp4DWkMZJ+I+kZSSslfSnlD5X0oKQ16ecxeedcLWmtpNWSpublny1pRfrsekldtlyLWcDIzOxNUUIqrBn424h4BzAZmCVpAnAVsDAixgML0z7psxnkptObBtyY3kIDuAm4HBif0rSubu7gZ2YlUQmpkIjYFBFPpO1dwDNAIzAdmJMOmwNclLanA3MjYl9ErAPWApMkNQBDImJxRARwe945nfKi5WZWuuL7/IZLWpa3f3NE3HzgQZLGAmcCjwOjImIT5AKkpJHpsEbgsbzTNqS8prR9YH5BDn5mVrISRnu3RcTEgteSBpObKeqvI2Jnge66zp45PqRnkd3sNbPSdU+fH5L6kgt8P8ubL2BzasqSfm5J+RuAMXmnjwY2pvzRHeQX5OBnZqWJbhvtFfAT4JmI+Je8j+YDl6Xty4C78/JnSOovaRy5gY0lqYm8S9LkdM1L887plJu9Zla67nnO7z3AZ4AVkp5MedcA1wHzJM0E1gMXA0TESknzgFXkRopnRURLOu8K4DZgIHBfSgU5+JlZybrjDY+IeJTOB4U7fMkiImYDszvIXwacVsr9HfzMrHQ18IaHg5+ZlazW3+01MztYUPOTmZqZHaTmFzAyM+uUg5+ZZZGi90c/Bz8zK00GZnI2M+uQ+/zMLJO6enWtN3DwM7PSueZnZpkTbvaaWVY5+JlZ1vghZzPLLLX2/ujn4GdmpfFzftaViVN28vlrN1JfF9x351Dm/WBUpYuUaa0tcMdF4xh8bBMX/XgDW1b1Z+HfN9CyX6g+OP+bL3Ps6Xvbj9+5sQ+3TzuRyVduZeJnd7B/dx3zLnlb++e7Xu7DO6bvZMrXN1fi61SUH3UpQNKtwEeALRFR0iSDtaCuLpj17Ze4esYJbNvUl+/fu4bHFhzF+jUDKl20zPrdbUMZetI+9u/Ord7wyD+PZPKVWxl37uusW3QEj/zzSC6+Y3378b+dPYqx79/dvt9vcCufvmdd+/7Ppo/lpA/u7LkvUE1qoOZXzjU8bqOIhYNr1Sln7mHj8/14eX1/mpvqWHT30Zwz9bVKFyuzdm3qw7pFgznt46+250m0B8J9u+o5YlRz+2drHxzMUWOaGDZ+X4fXe+X5vuzZ3ofGd71R1nJXK0VxqZqVLfhFxMPAjnJdv9oNO7aJrRv7te9v29SX4Q1NFSxRti36n6N431e3kL8q4rlf38wj143ix+89iYevG8l7/y63SFjTHrHsR8OY/MWtnV5v9T1HccqHd9L5Kos1LICI4lIVq/jqbZIul7RM0rImOv5Xtjfq6H+KKv9voWY999BgBg1rYdRpe9+S/9Qdx3Du1zbz3x5dy7nXbOaBq48DYPH3RnDmX+2g3xGd/8FW/+cQTvkvGW3y0j2rt1VaxQc80urtNwMM0dCaCQ/bNvVlxHH72/eHNzSx/eW+FSxRdm1cPpDnFg7m+d+eSPO+OvbvruO+vzmO5x4azJS/zw1WnPyhXfz6mgYANv1+IGvuP5JHvzOSfTvroQ769AvOuPQVALY+05/WFg4Kplnh5/ysoNVPDqJx3H5GjdnH9pf7MmX6q1w3621dn2jd7r1f2cp7v5Jrwr742CCW/2QoF/7LRuZMPYENjw9izOQ9vLh4EEePzf1j9Ym5L7Sfu/h7w+l7RGt74AP4wz1DOOUj2a319YYmbTEc/MqktUXc8LVGvn3Hc9TVwwNzh/LCsx7prSYXzN7EomtH0doi+vQPLpj9clHnPXvfED56y4tlLl11c82vAEl3AlOA4ZI2AN+IiJ+U637VaOlDQ1j60JBKF8PyjJm8hzGT9wDQOPENPnX38wWPP+dL2w7Km/mbP5ajaL2Lg1/nIuKScl3bzCrLNT8zy54AWnp/9HPwM7OSueZnZtnk0V4zyyLX/MwsezyllZllkQDVwIBHxd/tNbPeRxFFpS6vI90qaYukp/Pyhkp6UNKa9POYvM+ulrRW0mpJU/Pyz5a0In12vdT1lBMOfmZWmighde02Dp767ipgYUSMBxamfSRNAGYAp6ZzbpRUn865CbgcGJ9Sl9PpOfiZWYmKnM6qiJpfJ1PfTQfmpO05wEV5+XMjYl9ErAPWApMkNQBDImJxRARwe945nXKfn5mVrITR3uGSluXt35xmcipkVERsAoiITZJGpvxG4LG84zakvKa0fWB+QQ5+Zla64p/z2xYRE7vprh3140WB/IIc/MysNFH20d7NkhpSra8B2JLyNwBj8o4bDWxM+aM7yC/IfX5mVrruG/DoyHzgsrR9GXB3Xv4MSf0ljSM3sLEkNZF3SZqcRnkvzTunU675mVnJinmMpajrdDD1HXAdME/STGA9cDFARKyUNA9YBTQDsyKiJV3qCnIjxwOB+1IqyMHPzErXTcGvwNR353dy/Gxgdgf5y4CSlsh18DOz0gRQ5YsTFcPBz8xKIop7e6PaOfiZWelae3/Vz8HPzErjZq+ZZZWbvWaWTQ5+ZpY9XrTczLLIq7eZWVa5z8/MssnBz8wyJ4BWBz8zyxwPeJhZVjn4mVnmBNDS+1/xcPAzsxIFhIOfmWWRm71mljke7TWzzHLNz8wyycHPzDInAlpauj6uyjn4mVnpXPMzs0xy8DOz7AmP9ppZBgWEH3I2s0zy621mljkRXrrSzDLKAx5mlkXhmp+ZZY8nMzWzLPLEBmaWRQFEDbzeVlfpAphZLxNpMtNiUhckTZO0WtJaSVf1QOnbueZnZiWLbmj2SqoHbgD+DNgALJU0PyJWHfbFi+Can5mVrntqfpOAtRHxXETsB+YC08te9kRRRaM2krYCL1S6HGUwHNhW6UJYSWr1b/a2iBhxOBeQdD+5308xBgB78/Zvjoib03X+EpgWEZ9N+58B3h0RXzic8hWrqpq9h/tHqVaSlkXExEqXw4rnv1nnImJaN11KHV2+m67dJTd7zaxSNgBj8vZHAxt76uYOfmZWKUuB8ZLGSeoHzADm99TNq6rZW8NurnQBrGT+m5VZRDRL+gKwAKgHbo2IlT11/6oa8DAz6ylu9ppZJjn4mVkmOfiVUSVf3bFDI+lWSVskPV3pslh5OfiVSd6rOxcCE4BLJE2obKmsCLcB3fUcm1UxB7/yqeirO3ZoIuJhYEely2Hl5+BXPo3Ai3n7G1KemVUBB7/yqeirO2ZWmINf+VT01R0zK8zBr3wq+uqOmRXm4FcmEdEMtL268wwwrydf3bFDI+lOYDFwiqQNkmZWukxWHn69zcwyyTU/M8skBz8zyyQHPzPLJAc/M8skBz8zyyQHv15EUoukJyU9LekuSYMO41q3pdWzkHRLoUkXJE2R9KeHcI/nJR20yldn+Qccs7vEe/2jpL8rtYyWXQ5+vcsbEXFGRJwG7Ac+n/9hmkmmZBHx2S4Wip4ClBz8zKqZg1/v9QhwUqqV/UbSHcAKSfWS/pekpZKekvQ5AOX8QNIqSb8CRrZdSNIiSRPT9jRJT0j6vaSFksaSC7JfTrXO90kaIenn6R5LJb0nnTtM0gOSfifpR3T8fvNbSPq/kpZLWinp8gM++24qy0JJI1LeiZLuT+c8Iunt3fLbtMzxAka9kKQ+5OYJvD9lTQJOi4h1KYC8FhHvktQf+H+SHgDOBE4B/gQYBawCbj3guiOAHwPvT9caGhE7JP0Q2B0R/zsddwfwrxHxqKTjyb3F8g7gG8CjEfEtSR8G3hLMOvFf0z0GAksl/TwitgNHAE9ExN9K+od07S+QW1jo8xGxRtK7gRuBDxzCr9EyzsGvdxko6cm0/QjwE3LN0SURsS7lfxB4Z1t/HnAUMB54P3BnRLQAGyU91MH1JwMPt10rIjqb1+4CYILUXrEbIunIdI+/SOf+StIrRXynKyV9NG2PSWXdDrQC/5Hyfwr8QtLg9H3vyrt3/yLuYXYQB7/e5Y2IOCM/IwWB1/OzgC9GxIIDjvsQXU+ppSKOgVx3yTkR8UYHZSn6fUlJU8gF0nMiYo+kRcCATg6PdN9XD/wdmB0K9/nVngXAFZL6Akg6WdIRwMPAjNQn2ACc18G5i4FzJY1L5w5N+buAI/OOe4BcE5R03Blp82HgUynvQuCYLsp6FPBKCnxvJ1fzbFMHtNVeP0muOb0TWCfp4nQPSTq9i3uYdcjBr/bcQq4/74m0CM+PyNXwfwmsAVYANwG/PfDEiNhKrp/uF5J+z5vNznuAj7YNeABXAhPTgMoq3hx1/ibwfklPkGt+r++irPcDfSQ9BVwLPJb32evAqZKWk+vT+1bK/xQwM5VvJV4awA6RZ3Uxs0xyzc/MMsnBz8wyycHPzDLJwc/MMsnBz8wyycHPzDLJwc/MMun/AxGv0qNA1zhLAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# classification report\n",
    "clf_score(rf_best, X_train, y_train, X_test, y_test, train=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "73463026",
   "metadata": {},
   "source": [
    "## 5.6 Modeling results"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54659459",
   "metadata": {},
   "source": [
    "We display the visualization of validation result, roc auc score, classification report and confusion matrix for 4 models respectively by using the function ```clf_score```. The visualization is more intuitive than numbers to show the results of cross-validation. Based on the output above, we can conclude that ```Random Forest``` generates the highest number of correct predictions regarding the game's outcome. The most contributing factors are outlined below: \n",
    "\n",
    "* ```timeOnIce```\n",
    "* ```shortHandedTimeOnIce```\n",
    "* ```evenTimeOnIce```\n",
    "* ```powerPlayTimeOnIce```\n",
    "* ```goals```\n",
    "* ```pim```\n",
    "\n",
    "Based on cross validation of the Random Forest model, the accuracy score improves from 0.7638 to 0.7698 after the hyperparameters have been tuned. The detailed process is showed in Section 5.5. \n",
    "\n",
    "In Section 5.5, you can see that we choose to tune the hyperparameters:\n",
    "\n",
    "* ```max_features```\n",
    "* ```n_estimators```\n",
    "* ```min_samples_leaf```\n",
    "* ```random_state```"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4f51d39a",
   "metadata": {},
   "source": [
    "## 5.7 Modeling insights and limitations"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e5d20966",
   "metadata": {},
   "source": [
    "The modeling output implies that decision-makers should focus on the top contributing factors such as time on ice, short-handed time on ice, even time on ice, power-play time on ice, number of goals and penalty infraction minutes \n",
    "when judging the outcome of the hockey game with the first-period statistics. By leveraging the Random Forest model, users can utilize hockey game statistics to determine the game’s outcomes as to which team will win or lose after the first period. The model generates accurate predictions on sports betting over randomly guessing the outcome. Also, the model performance validates the proof of value and achieves the business objective to develop a predictive model with an accuracy score higher than 52.4% in order to break even. \n",
    "\n",
    "The system limitation is that Sports analytics is generally difficult to model with the best models performing at accuracy levels ranging from 50-70%. However, our model performs very well at the accuracy level of around 76%, which potentially faces overfitting issues. Our team will carry over investigating this topic in the second half of the course."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6f44e932",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b068667b",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}