7/cnn_using_keras_98.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2DtA0B2J-PpW"
   },
   "source": [
    "# Introduction:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VEGXl-ob-PpY"
   },
   "source": [
    "In this project, we will be working on a digit recognition problem using the popular MNIST dataset. The MNIST dataset consists of a large number of images of handwritten digits ranging from 0 to 9. Our goal is to develop a machine learning model that can accurately classify these images and correctly identify the digit that is being represented.\n",
    "\n",
    "To accomplish this, we will be using a convolutional neural network (CNN), which is a type of deep learning algorithm that is particularly well-suited for image recognition tasks. We will be implementing our CNN using the Keras library in Python, which provides an easy-to-use interface for building and training deep learning models.\n",
    "\n",
    "The accuracy of our model will be evaluated using a test set of images that the model has never seen before. Our ultimate goal is to develop a model that can achieve high accuracy on the test set, demonstrating its ability to generalize well to new, unseen data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UoUBjQJy-PpZ"
   },
   "source": [
    "<img src=\"https://www.upgrad.com/blog/wp-content/uploads/2020/12/1-4.png\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kQ4OV_2e-PpZ"
   },
   "source": [
    "# Table of Contents\n",
    "\n",
    "* &nbsp; **[Introduction](#Introduction)**\n",
    "    \n",
    "* &nbsp; **[Import](#Import)**\n",
    "\n",
    "* &nbsp; **[Load and Check Data](#Load-and-Check-Data)**\n",
    "   \n",
    "* &nbsp; **[Data Preparation](#Data-Preparation)**\n",
    "\n",
    "    * &nbsp; **[Normalization](#Normalization)**\n",
    "    \n",
    "    * &nbsp; **[Reshape](#Reshape)**\n",
    "    \n",
    "    * &nbsp; **[Label Encoding](#Label-Encoding)**\n",
    "    \n",
    "    * &nbsp; **[Train Test Split](#Train-Test-Split)**\n",
    "       \n",
    "\n",
    "* &nbsp; **[Data Augmentation](#Data-Augmentation)**\n",
    "    \n",
    "* &nbsp; **[CNN Building](#CNN-Building)**\n",
    "\n",
    "    * &nbsp; **[CNN Architecture](#CNN-Architecture)**\n",
    "    \n",
    "    * &nbsp; **[Set up Optimizer and Annealer](#Set-up-Optimizer-and-Annealer)**\n",
    "    \n",
    "    * &nbsp; **[Fit the Model](#Fit-the-Model)**\n",
    "\n",
    "* &nbsp; **[Prediction](#Prediction)**\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jwoTf1_J-PpZ"
   },
   "source": [
    "# Import:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "_cell_guid": "b1076dfc-b9ad-4769-8c92-a6c4dae69d19",
    "_uuid": "8f2839f25d086af736a60e9eeb907d3b93b6e0e5",
    "execution": {
     "iopub.execute_input": "2023-03-24T22:46:51.290405Z",
     "iopub.status.busy": "2023-03-24T22:46:51.289266Z",
     "iopub.status.idle": "2023-03-24T22:46:51.299621Z",
     "shell.execute_reply": "2023-03-24T22:46:51.298325Z",
     "shell.execute_reply.started": "2023-03-24T22:46:51.290362Z"
    },
    "id": "bsZbXD-s-Ppa"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "import plotly.express as px\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense , Dropout, Flatten, Conv2D, MaxPool2D, BatchNormalization\n",
    "from keras.optimizers import Adam ,RMSprop\n",
    "from sklearn.model_selection import train_test_split\n",
    "from keras.utils.np_utils import to_categorical\n",
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "from keras.callbacks import LearningRateScheduler\n",
    "\n",
    "import warnings\n",
    "# filter warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "# Input data files are available in the \"../input/\" directory."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yuPebwDK-Ppb"
   },
   "source": [
    "# Load and Check Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "Ep0hSvNW-vQ6",
    "outputId": "8a103f1e-14b5-437d-b375-6c02dafa5abc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Mounted at /content/drive\n"
     ]
    }
   ],
   "source": [
    "from google.colab import drive\n",
    "\n",
    "drive.mount('/content/drive')\n",
    "\n",
    "!cp /content/drive/MyDrive/Abdulla_OCR/Character/anotation_28x28/Test.csv .\n",
    "!cp /content/drive/MyDrive/Abdulla_OCR/Character/anotation_28x28/Train.csv .\n",
    "\n",
    "!rm -rf sample_data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 317
    },
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:40.545094Z",
     "iopub.status.busy": "2023-03-24T22:14:40.542295Z",
     "iopub.status.idle": "2023-03-24T22:14:43.963586Z",
     "shell.execute_reply": "2023-03-24T22:14:43.962394Z",
     "shell.execute_reply.started": "2023-03-24T22:14:40.545048Z"
    },
    "id": "T9B-JNQ0-Ppb",
    "outputId": "437d6fad-3949-455e-ea18-d2ffe6f2212b"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-d6926c82-3bd9-4e30-aed4-9e0f66b27b9c\">\n",
       "    <div class=\"colab-df-container\">\n",
       "      <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>pixel0</th>\n",
       "      <th>pixel1</th>\n",
       "      <th>pixel2</th>\n",
       "      <th>pixel3</th>\n",
       "      <th>pixel4</th>\n",
       "      <th>pixel5</th>\n",
       "      <th>pixel6</th>\n",
       "      <th>pixel7</th>\n",
       "      <th>pixel8</th>\n",
       "      <th>...</th>\n",
       "      <th>pixel774</th>\n",
       "      <th>pixel775</th>\n",
       "      <th>pixel776</th>\n",
       "      <th>pixel777</th>\n",
       "      <th>pixel778</th>\n",
       "      <th>pixel779</th>\n",
       "      <th>pixel780</th>\n",
       "      <th>pixel781</th>\n",
       "      <th>pixel782</th>\n",
       "      <th>pixel783</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 785 columns</p>\n",
       "</div>\n",
       "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-d6926c82-3bd9-4e30-aed4-9e0f66b27b9c')\"\n",
       "              title=\"Convert this dataframe to an interactive table.\"\n",
       "              style=\"display:none;\">\n",
       "        \n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "       width=\"24px\">\n",
       "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
       "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
       "  </svg>\n",
       "      </button>\n",
       "      \n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      flex-wrap:wrap;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "      <script>\n",
       "        const buttonEl =\n",
       "          document.querySelector('#df-d6926c82-3bd9-4e30-aed4-9e0f66b27b9c button.colab-df-convert');\n",
       "        buttonEl.style.display =\n",
       "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "        async function convertToInteractive(key) {\n",
       "          const element = document.querySelector('#df-d6926c82-3bd9-4e30-aed4-9e0f66b27b9c');\n",
       "          const dataTable =\n",
       "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                     [key], {});\n",
       "          if (!dataTable) return;\n",
       "\n",
       "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "            + ' to learn more about interactive tables.';\n",
       "          element.innerHTML = '';\n",
       "          dataTable['output_type'] = 'display_data';\n",
       "          await google.colab.output.renderOutput(dataTable, element);\n",
       "          const docLink = document.createElement('div');\n",
       "          docLink.innerHTML = docLinkHtml;\n",
       "          element.appendChild(docLink);\n",
       "        }\n",
       "      </script>\n",
       "    </div>\n",
       "  </div>\n",
       "  "
      ],
      "text/plain": [
       "   label  pixel0  pixel1  pixel2  pixel3  pixel4  pixel5  pixel6  pixel7  \\\n",
       "0      0       0       0       0       0       0       0       0       0   \n",
       "1      0       0       0       0       0       0       0       0       0   \n",
       "2      0       0       0       0       0       0       0       0       0   \n",
       "3      0       0       0       0       0       0       0       0       0   \n",
       "4      0       0       0       0       0       0       0       0       0   \n",
       "\n",
       "   pixel8  ...  pixel774  pixel775  pixel776  pixel777  pixel778  pixel779  \\\n",
       "0       0  ...         0         0         0         0         0         0   \n",
       "1       0  ...         0         0         0         0         0         0   \n",
       "2       0  ...         0         0         0         0         0         0   \n",
       "3       0  ...         0         0         0         0         0         0   \n",
       "4       0  ...         0         0         0         0         0         0   \n",
       "\n",
       "   pixel780  pixel781  pixel782  pixel783  \n",
       "0         0         0         0         0  \n",
       "1         0         0         0         0  \n",
       "2         0         0         0         0  \n",
       "3         0         0         4         0  \n",
       "4         0         0         0         0  \n",
       "\n",
       "[5 rows x 785 columns]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train shape: (209966, 785)\n"
     ]
    }
   ],
   "source": [
    "# Load train data\n",
    "train = pd.read_csv(\"Train.csv\")\n",
    "display(train.head())\n",
    "print(f\"train shape: {train.shape}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 317
    },
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:43.965823Z",
     "iopub.status.busy": "2023-03-24T22:14:43.96512Z",
     "iopub.status.idle": "2023-03-24T22:14:46.057273Z",
     "shell.execute_reply": "2023-03-24T22:14:46.05609Z",
     "shell.execute_reply.started": "2023-03-24T22:14:43.965782Z"
    },
    "id": "XJXH2u2H-Ppb",
    "outputId": "176534fa-da21-431d-e5f6-3736d2f9f339"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "  <div id=\"df-65b2f838-d6e1-4abc-bf1a-83ea39bb7705\">\n",
       "    <div class=\"colab-df-container\">\n",
       "      <div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pixel0</th>\n",
       "      <th>pixel1</th>\n",
       "      <th>pixel2</th>\n",
       "      <th>pixel3</th>\n",
       "      <th>pixel4</th>\n",
       "      <th>pixel5</th>\n",
       "      <th>pixel6</th>\n",
       "      <th>pixel7</th>\n",
       "      <th>pixel8</th>\n",
       "      <th>pixel9</th>\n",
       "      <th>...</th>\n",
       "      <th>pixel774</th>\n",
       "      <th>pixel775</th>\n",
       "      <th>pixel776</th>\n",
       "      <th>pixel777</th>\n",
       "      <th>pixel778</th>\n",
       "      <th>pixel779</th>\n",
       "      <th>pixel780</th>\n",
       "      <th>pixel781</th>\n",
       "      <th>pixel782</th>\n",
       "      <th>pixel783</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>...</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 784 columns</p>\n",
       "</div>\n",
       "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-65b2f838-d6e1-4abc-bf1a-83ea39bb7705')\"\n",
       "              title=\"Convert this dataframe to an interactive table.\"\n",
       "              style=\"display:none;\">\n",
       "        \n",
       "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
       "       width=\"24px\">\n",
       "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
       "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
       "  </svg>\n",
       "      </button>\n",
       "      \n",
       "  <style>\n",
       "    .colab-df-container {\n",
       "      display:flex;\n",
       "      flex-wrap:wrap;\n",
       "      gap: 12px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert {\n",
       "      background-color: #E8F0FE;\n",
       "      border: none;\n",
       "      border-radius: 50%;\n",
       "      cursor: pointer;\n",
       "      display: none;\n",
       "      fill: #1967D2;\n",
       "      height: 32px;\n",
       "      padding: 0 0 0 0;\n",
       "      width: 32px;\n",
       "    }\n",
       "\n",
       "    .colab-df-convert:hover {\n",
       "      background-color: #E2EBFA;\n",
       "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
       "      fill: #174EA6;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert {\n",
       "      background-color: #3B4455;\n",
       "      fill: #D2E3FC;\n",
       "    }\n",
       "\n",
       "    [theme=dark] .colab-df-convert:hover {\n",
       "      background-color: #434B5C;\n",
       "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
       "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
       "      fill: #FFFFFF;\n",
       "    }\n",
       "  </style>\n",
       "\n",
       "      <script>\n",
       "        const buttonEl =\n",
       "          document.querySelector('#df-65b2f838-d6e1-4abc-bf1a-83ea39bb7705 button.colab-df-convert');\n",
       "        buttonEl.style.display =\n",
       "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
       "\n",
       "        async function convertToInteractive(key) {\n",
       "          const element = document.querySelector('#df-65b2f838-d6e1-4abc-bf1a-83ea39bb7705');\n",
       "          const dataTable =\n",
       "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
       "                                                     [key], {});\n",
       "          if (!dataTable) return;\n",
       "\n",
       "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
       "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
       "            + ' to learn more about interactive tables.';\n",
       "          element.innerHTML = '';\n",
       "          dataTable['output_type'] = 'display_data';\n",
       "          await google.colab.output.renderOutput(dataTable, element);\n",
       "          const docLink = document.createElement('div');\n",
       "          docLink.innerHTML = docLinkHtml;\n",
       "          element.appendChild(docLink);\n",
       "        }\n",
       "      </script>\n",
       "    </div>\n",
       "  </div>\n",
       "  "
      ],
      "text/plain": [
       "   pixel0  pixel1  pixel2  pixel3  pixel4  pixel5  pixel6  pixel7  pixel8  \\\n",
       "0       0       0       0       0       0       0       0       0       0   \n",
       "1       0       0       0       0       0       0       0       0       0   \n",
       "2       0       0       0       0       0       0       0       0       0   \n",
       "3       0       0       0       0       0       0       0       0       0   \n",
       "4       0       0       0       0       0       0       0       0       0   \n",
       "\n",
       "   pixel9  ...  pixel774  pixel775  pixel776  pixel777  pixel778  pixel779  \\\n",
       "0       0  ...         0         0         0         0         0         0   \n",
       "1       0  ...         0         0         0         0         0         0   \n",
       "2       0  ...         0         0         0         0         0         0   \n",
       "3       0  ...         0         0         0         0         0         0   \n",
       "4       0  ...         0         0         0         0         0         0   \n",
       "\n",
       "   pixel780  pixel781  pixel782  pixel783  \n",
       "0         0         0         0         0  \n",
       "1         0         0         0         0  \n",
       "2         0         0         0         0  \n",
       "3         0         0         0         0  \n",
       "4         0         0         0         0  \n",
       "\n",
       "[5 rows x 784 columns]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "test shape: (34966, 784)\n"
     ]
    }
   ],
   "source": [
    "# Load test data\n",
    "test = pd.read_csv(\"Test.csv\")\n",
    "display(test.head())\n",
    "print(f\"test shape: {test.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "2nbvpANk-Ppc"
   },
   "source": [
    "Get target:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:46.061301Z",
     "iopub.status.busy": "2023-03-24T22:14:46.060445Z",
     "iopub.status.idle": "2023-03-24T22:14:46.149852Z",
     "shell.execute_reply": "2023-03-24T22:14:46.148357Z",
     "shell.execute_reply.started": "2023-03-24T22:14:46.061253Z"
    },
    "id": "v1tl9oq3-Ppc"
   },
   "outputs": [],
   "source": [
    "# Put labels into y_train variable\n",
    "y_train = train[\"label\"]\n",
    "\n",
    "# Drop \"label\" column\n",
    "X_train = train.drop(labels = [\"label\"],axis = 1) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "E0YDE-D8-Ppc"
   },
   "source": [
    "Plot some samples:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:46.154163Z",
     "iopub.status.busy": "2023-03-24T22:14:46.152675Z",
     "iopub.status.idle": "2023-03-24T22:14:46.161981Z",
     "shell.execute_reply": "2023-03-24T22:14:46.160839Z",
     "shell.execute_reply.started": "2023-03-24T22:14:46.154117Z"
    },
    "id": "QR3CyWa8-Ppc"
   },
   "outputs": [],
   "source": [
    "def plot_img(index):\n",
    "    # Reshape vector to 28x28 matrix\n",
    "    img = X_train.iloc[index].to_numpy()\n",
    "    img = img.reshape((28,28))\n",
    "\n",
    "    # Create figure\n",
    "    fig = px.imshow(img, color_continuous_scale='gray')\n",
    "\n",
    "    # Set Title and x/y axis labels\n",
    "    fig.update_layout(\n",
    "        showlegend = False,\n",
    "        #coloraxis_showscale=False,\n",
    "        title={\n",
    "            'text': f\"Visualization of Image No.{index}\",\n",
    "            'y':0.95,\n",
    "            'x':0.5\n",
    "        }\n",
    "    )\n",
    "    # Display\n",
    "    fig.show() # for Kaggle version\n",
    "    #fig.show(\"svg\") # for GitHub version"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:46.166104Z",
     "iopub.status.busy": "2023-03-24T22:14:46.165825Z",
     "iopub.status.idle": "2023-03-24T22:14:47.550178Z",
     "shell.execute_reply": "2023-03-24T22:14:47.548892Z",
     "shell.execute_reply.started": "2023-03-24T22:14:46.166078Z"
    },
    "id": "WpLke4Ag-Ppd",
    "outputId": "a9b854b4-7350-480f-bdff-585d2946cdba"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.18.2.min.js\"></script>                <div id=\"3c2519ca-2ff9-4ff6-9a7c-ff174222be33\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"3c2519ca-2ff9-4ff6-9a7c-ff174222be33\")) {                    Plotly.newPlot(                        \"3c2519ca-2ff9-4ff6-9a7c-ff174222be33\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"z\":[[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,42,142,49,1,2,0,0,0,0,1,0,3,0,0,0,0],[0,0,0,0,0,0,0,0,0,1,1,7,173,16,0,1,0,0,0,1,0,0,3,0,0,0,0,0],[0,0,0,0,0,0,0,0,3,0,0,12,175,32,2,0,0,5,0,3,0,1,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,1,2,58,178,22,2,1,0,1,0,0,0,0,3,0,0,0,0],[0,0,0,0,0,0,0,0,0,1,2,0,1,93,148,1,0,0,47,4,1,0,5,0,0,0,0,0],[0,0,0,0,0,0,0,0,2,0,0,5,0,107,135,1,3,0,2,173,31,0,0,3,0,0,0,0],[0,0,0,0,0,0,0,0,0,2,0,39,156,111,2,3,0,1,1,42,173,1,2,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,31,121,2,2,0,0,0,1,0,0,134,65,0,0,0,0,0,0],[1,0,2,0,0,0,1,0,0,0,3,2,0,0,0,2,0,1,0,0,75,123,0,3,0,0,0,0],[0,1,1,2,0,0,0,3,0,0,0,0,6,0,1,2,2,4,3,63,151,95,1,0,0,0,0,0],[2,0,0,2,1,0,4,0,1,1,4,7,48,78,100,131,154,165,154,115,59,0,3,0,0,0,0,0],[0,1,4,0,0,0,37,56,129,188,179,167,144,125,64,37,4,2,2,0,0,1,0,1,0,0,0,0],[0,1,0,5,0,2,83,89,37,2,1,1,1,0,0,0,0,0,1,0,2,2,0,1,0,0,0,0],[0,1,0,0,1,0,0,3,2,2,0,1,0,1,4,0,1,1,0,0,1,0,0,0,0,0,0,0],[0,0,1,0,2,6,0,1,0,0,3,0,1,0,0,1,0,0,2,2,0,2,0,1,0,0,0,0],[1,0,1,1,0,0,5,0,0,1,1,0,2,1,0,1,1,0,2,0,2,0,2,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,7,2],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,2,8,2],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\"},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\"},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(0, 0, 0)\"],[0.09090909090909091,\"rgb(16, 16, 16)\"],[0.18181818181818182,\"rgb(38, 38, 38)\"],[0.2727272727272727,\"rgb(59, 59, 59)\"],[0.36363636363636365,\"rgb(81, 80, 80)\"],[0.45454545454545453,\"rgb(102, 101, 101)\"],[0.5454545454545454,\"rgb(124, 123, 122)\"],[0.6363636363636364,\"rgb(146, 146, 145)\"],[0.7272727272727273,\"rgb(171, 171, 170)\"],[0.8181818181818182,\"rgb(197, 197, 195)\"],[0.9090909090909091,\"rgb(224, 224, 223)\"],[1.0,\"rgb(254, 254, 253)\"]]},\"margin\":{\"t\":60},\"title\":{\"text\":\"Visualization of Image No.3\",\"y\":0.95,\"x\":0.5},\"showlegend\":false},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('3c2519ca-2ff9-4ff6-9a7c-ff174222be33');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.18.2.min.js\"></script>                <div id=\"35d37f74-8198-4c99-b0bc-8a2ec5016290\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"35d37f74-8198-4c99-b0bc-8a2ec5016290\")) {                    Plotly.newPlot(                        \"35d37f74-8198-4c99-b0bc-8a2ec5016290\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"z\":[[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,2,0,0,1,1,0,4,0,0,1,0,1,0,3,0,2,0,0,0,3],[0,0,0,0,0,0,0,0,0,2,2,0,0,5,0,1,0,0,2,0,1,3,1,0,0,4,0,0],[0,0,0,0,0,0,0,0,1,0,2,0,1,1,2,0,1,0,2,0,0,0,65,42,0,0,0,2],[0,0,0,0,0,0,0,0,0,0,1,1,0,92,125,67,0,1,0,3,1,34,137,94,71,2,1,1],[0,0,0,0,0,0,0,0,0,0,1,0,20,171,159,95,2,3,0,0,2,20,144,0,0,1,0,0],[0,0,0,0,0,0,0,0,0,2,0,1,54,128,3,1,0,0,0,1,0,2,119,1,2,0,5,1],[0,0,0,0,0,0,0,0,2,0,1,1,0,143,115,26,0,2,4,0,0,1,110,8,0,0,1,0],[0,0,0,0,0,0,0,0,0,0,3,90,114,118,107,12,2,0,0,2,0,0,84,51,2,0,0,4],[0,3,0,0,2,0,0,0,2,0,0,2,2,1,1,1,1,0,0,3,0,0,117,61,0,0,0,0],[0,1,5,5,0,2,1,1,0,0,0,0,0,3,0,0,0,0,1,0,41,122,118,5,0,0,0,0],[0,2,1,54,5,1,0,0,3,2,0,1,0,2,3,18,59,95,155,125,101,30,0,2,0,0,0,0],[3,0,72,60,34,23,30,69,60,102,92,152,170,147,164,138,78,36,3,1,0,0,3,0,0,0,0,0],[0,0,14,81,102,136,133,104,87,83,35,19,4,0,3,0,0,2,0,2,0,0,1,0,0,0,0,0],[1,2,0,2,2,0,0,1,0,1,2,0,0,3,0,2,0,0,0,1,0,0,0,2,0,0,0,0],[0,0,1,0,0,1,3,0,2,0,0,1,2,0,0,0,0,0,1,0,2,0,3,0,0,0,0,0],[0,1,0,0,1,1,0,1,0,1,1,1,0,0,2,0,0,0,0,0,1,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\"},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\"},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(0, 0, 0)\"],[0.09090909090909091,\"rgb(16, 16, 16)\"],[0.18181818181818182,\"rgb(38, 38, 38)\"],[0.2727272727272727,\"rgb(59, 59, 59)\"],[0.36363636363636365,\"rgb(81, 80, 80)\"],[0.45454545454545453,\"rgb(102, 101, 101)\"],[0.5454545454545454,\"rgb(124, 123, 122)\"],[0.6363636363636364,\"rgb(146, 146, 145)\"],[0.7272727272727273,\"rgb(171, 171, 170)\"],[0.8181818181818182,\"rgb(197, 197, 195)\"],[0.9090909090909091,\"rgb(224, 224, 223)\"],[1.0,\"rgb(254, 254, 253)\"]]},\"margin\":{\"t\":60},\"title\":{\"text\":\"Visualization of Image No.5\",\"y\":0.95,\"x\":0.5},\"showlegend\":false},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('35d37f74-8198-4c99-b0bc-8a2ec5016290');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.18.2.min.js\"></script>                <div id=\"edff916d-5f7c-4835-be68-2025ebd38d1c\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"edff916d-5f7c-4835-be68-2025ebd38d1c\")) {                    Plotly.newPlot(                        \"edff916d-5f7c-4835-be68-2025ebd38d1c\",                        [{\"coloraxis\":\"coloraxis\",\"name\":\"0\",\"z\":[[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,5,0,0,4,1,0,0,1,1,0,0,2,0,2,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,8,0,0,0,2,0,0,0,0,0,2,0,0,4,0,0,0,0],[0,0,0,0,0,0,0,0,2,0,0,3,0,2,1,0,0,1,0,3,0,2,1,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,2,0,0,19,111,94,0,0,0,80,96,3,0,0,2,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,169,62,11,1,0,2,86,152,0,0,3,0,0,0,0,0],[0,0,0,0,0,0,0,0,3,0,0,1,117,158,54,17,1,0,13,218,9,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,1,2,0,3,58,148,182,2,0,2,148,98,3,0,4,0,0,0,0],[0,0,0,0,0,0,0,0,3,0,0,27,104,180,185,45,0,0,0,26,201,6,4,0,0,0,0,0],[1,0,1,0,3,0,0,1,0,0,0,41,70,22,3,0,0,0,2,0,158,49,0,1,0,0,0,0],[2,0,0,3,0,0,4,0,0,4,1,0,1,0,0,2,0,3,5,97,203,24,3,0,0,0,0,0],[0,4,0,0,0,2,0,2,3,0,0,3,0,4,3,2,42,120,197,142,16,0,2,0,0,0,0,0],[0,0,2,0,0,3,0,23,24,15,14,44,63,108,171,212,157,99,20,0,4,3,0,3,0,0,0,0],[1,0,0,1,4,0,1,62,172,181,180,181,146,89,43,0,1,0,0,0,1,1,0,0,0,0,0,0],[0,0,0,0,0,4,4,0,1,3,1,0,1,0,2,0,0,0,0,1,0,1,1,0,0,0,0,0],[0,1,0,0,0,0,2,1,1,0,0,0,2,0,0,3,0,1,0,1,0,1,0,3,0,0,0,0],[0,0,2,0,1,0,0,0,0,1,0,0,0,1,2,0,0,2,0,1,0,1,2,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]],\"type\":\"heatmap\",\"xaxis\":\"x\",\"yaxis\":\"y\",\"hovertemplate\":\"x: %{x}<br>y: %{y}<br>color: %{z}<extra></extra>\"}],                        {\"template\":{\"data\":{\"histogram2dcontour\":[{\"type\":\"histogram2dcontour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"choropleth\":[{\"type\":\"choropleth\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"histogram2d\":[{\"type\":\"histogram2d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmap\":[{\"type\":\"heatmap\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"heatmapgl\":[{\"type\":\"heatmapgl\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"contourcarpet\":[{\"type\":\"contourcarpet\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"contour\":[{\"type\":\"contour\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"surface\":[{\"type\":\"surface\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"},\"colorscale\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]]}],\"mesh3d\":[{\"type\":\"mesh3d\",\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"parcoords\":[{\"type\":\"parcoords\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolargl\":[{\"type\":\"scatterpolargl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"bar\":[{\"error_x\":{\"color\":\"#2a3f5f\"},\"error_y\":{\"color\":\"#2a3f5f\"},\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"scattergeo\":[{\"type\":\"scattergeo\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterpolar\":[{\"type\":\"scatterpolar\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"histogram\":[{\"marker\":{\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"histogram\"}],\"scattergl\":[{\"type\":\"scattergl\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatter3d\":[{\"type\":\"scatter3d\",\"line\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattermapbox\":[{\"type\":\"scattermapbox\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scatterternary\":[{\"type\":\"scatterternary\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"scattercarpet\":[{\"type\":\"scattercarpet\",\"marker\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}}}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"baxis\":{\"endlinecolor\":\"#2a3f5f\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"#2a3f5f\"},\"type\":\"carpet\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"#EBF0F8\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"#C8D4E3\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}],\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"#E5ECF6\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}]},\"layout\":{\"autotypenumbers\":\"strict\",\"colorway\":[\"#636efa\",\"#EF553B\",\"#00cc96\",\"#ab63fa\",\"#FFA15A\",\"#19d3f3\",\"#FF6692\",\"#B6E880\",\"#FF97FF\",\"#FECB52\"],\"font\":{\"color\":\"#2a3f5f\"},\"hovermode\":\"closest\",\"hoverlabel\":{\"align\":\"left\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"#E5ECF6\",\"polar\":{\"bgcolor\":\"#E5ECF6\",\"angularaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"radialaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"ternary\":{\"bgcolor\":\"#E5ECF6\",\"aaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"baxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"},\"caxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\"}},\"coloraxis\":{\"colorbar\":{\"outlinewidth\":0,\"ticks\":\"\"}},\"colorscale\":{\"sequential\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"sequentialminus\":[[0.0,\"#0d0887\"],[0.1111111111111111,\"#46039f\"],[0.2222222222222222,\"#7201a8\"],[0.3333333333333333,\"#9c179e\"],[0.4444444444444444,\"#bd3786\"],[0.5555555555555556,\"#d8576b\"],[0.6666666666666666,\"#ed7953\"],[0.7777777777777778,\"#fb9f3a\"],[0.8888888888888888,\"#fdca26\"],[1.0,\"#f0f921\"]],\"diverging\":[[0,\"#8e0152\"],[0.1,\"#c51b7d\"],[0.2,\"#de77ae\"],[0.3,\"#f1b6da\"],[0.4,\"#fde0ef\"],[0.5,\"#f7f7f7\"],[0.6,\"#e6f5d0\"],[0.7,\"#b8e186\"],[0.8,\"#7fbc41\"],[0.9,\"#4d9221\"],[1,\"#276419\"]]},\"xaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"yaxis\":{\"gridcolor\":\"white\",\"linecolor\":\"white\",\"ticks\":\"\",\"title\":{\"standoff\":15},\"zerolinecolor\":\"white\",\"automargin\":true,\"zerolinewidth\":2},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"yaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2},\"zaxis\":{\"backgroundcolor\":\"#E5ECF6\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"showbackground\":true,\"ticks\":\"\",\"zerolinecolor\":\"white\",\"gridwidth\":2}},\"shapedefaults\":{\"line\":{\"color\":\"#2a3f5f\"}},\"annotationdefaults\":{\"arrowcolor\":\"#2a3f5f\",\"arrowhead\":0,\"arrowwidth\":1},\"geo\":{\"bgcolor\":\"white\",\"landcolor\":\"#E5ECF6\",\"subunitcolor\":\"white\",\"showland\":true,\"showlakes\":true,\"lakecolor\":\"white\"},\"title\":{\"x\":0.05},\"mapbox\":{\"style\":\"light\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"scaleanchor\":\"y\",\"constrain\":\"domain\"},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"autorange\":\"reversed\",\"constrain\":\"domain\"},\"coloraxis\":{\"colorscale\":[[0.0,\"rgb(0, 0, 0)\"],[0.09090909090909091,\"rgb(16, 16, 16)\"],[0.18181818181818182,\"rgb(38, 38, 38)\"],[0.2727272727272727,\"rgb(59, 59, 59)\"],[0.36363636363636365,\"rgb(81, 80, 80)\"],[0.45454545454545453,\"rgb(102, 101, 101)\"],[0.5454545454545454,\"rgb(124, 123, 122)\"],[0.6363636363636364,\"rgb(146, 146, 145)\"],[0.7272727272727273,\"rgb(171, 171, 170)\"],[0.8181818181818182,\"rgb(197, 197, 195)\"],[0.9090909090909091,\"rgb(224, 224, 223)\"],[1.0,\"rgb(254, 254, 253)\"]]},\"margin\":{\"t\":60},\"title\":{\"text\":\"Visualization of Image No.700\",\"y\":0.95,\"x\":0.5},\"showlegend\":false},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('edff916d-5f7c-4835-be68-2025ebd38d1c');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Check some random values from train data\n",
    "for index in (3, 5, 700):\n",
    "    plot_img(index)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YfLGZEBo-Ppd"
   },
   "source": [
    "Check target distribution:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 542
    },
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:47.552951Z",
     "iopub.status.busy": "2023-03-24T22:14:47.551592Z",
     "iopub.status.idle": "2023-03-24T22:14:47.747411Z",
     "shell.execute_reply": "2023-03-24T22:14:47.746357Z",
     "shell.execute_reply.started": "2023-03-24T22:14:47.552908Z"
    },
    "id": "rbhamX9p-Ppd",
    "outputId": "2adeb607-c190-4462-fc0e-8332ddeda42b"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<html>\n",
       "<head><meta charset=\"utf-8\" /></head>\n",
       "<body>\n",
       "    <div>            <script src=\"https://cdnjs.cloudflare.com/ajax/libs/mathjax/2.7.5/MathJax.js?config=TeX-AMS-MML_SVG\"></script><script type=\"text/javascript\">if (window.MathJax && window.MathJax.Hub && window.MathJax.Hub.Config) {window.MathJax.Hub.Config({SVG: {font: \"STIX-Web\"}});}</script>                <script type=\"text/javascript\">window.PlotlyConfig = {MathJaxConfig: 'local'};</script>\n",
       "        <script src=\"https://cdn.plot.ly/plotly-2.18.2.min.js\"></script>                <div id=\"39f7b806-804b-4da8-a125-b7aa491bc90f\" class=\"plotly-graph-div\" style=\"height:525px; width:100%;\"></div>            <script type=\"text/javascript\">                                    window.PLOTLYENV=window.PLOTLYENV || {};                                    if (document.getElementById(\"39f7b806-804b-4da8-a125-b7aa491bc90f\")) {                    Plotly.newPlot(                        \"39f7b806-804b-4da8-a125-b7aa491bc90f\",                        [{\"alignmentgroup\":\"True\",\"hovertemplate\":\"variable=label<br>index=%{x}<br>value=%{y}<extra></extra>\",\"legendgroup\":\"label\",\"marker\":{\"color\":\"grey\",\"pattern\":{\"shape\":\"\"}},\"name\":\"label\",\"offsetgroup\":\"label\",\"orientation\":\"v\",\"showlegend\":true,\"textposition\":\"auto\",\"x\":[0,26,20,21,22,23,24,25,27,18,28,29,30,31,32,33,19,17,1,8,2,3,4,5,6,7,9,16,10,11,12,13,14,15,34],\"xaxis\":\"x\",\"y\":[6000,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999,5999],\"yaxis\":\"y\",\"type\":\"bar\"}],                        {\"template\":{\"data\":{\"barpolar\":[{\"marker\":{\"line\":{\"color\":\"white\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"barpolar\"}],\"bar\":[{\"error_x\":{\"color\":\"rgb(36,36,36)\"},\"error_y\":{\"color\":\"rgb(36,36,36)\"},\"marker\":{\"line\":{\"color\":\"white\",\"width\":0.5},\"pattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2}},\"type\":\"bar\"}],\"carpet\":[{\"aaxis\":{\"endlinecolor\":\"rgb(36,36,36)\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"rgb(36,36,36)\"},\"baxis\":{\"endlinecolor\":\"rgb(36,36,36)\",\"gridcolor\":\"white\",\"linecolor\":\"white\",\"minorgridcolor\":\"white\",\"startlinecolor\":\"rgb(36,36,36)\"},\"type\":\"carpet\"}],\"choropleth\":[{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"},\"type\":\"choropleth\"}],\"contourcarpet\":[{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"},\"type\":\"contourcarpet\"}],\"contour\":[{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"},\"colorscale\":[[0.0,\"#440154\"],[0.1111111111111111,\"#482878\"],[0.2222222222222222,\"#3e4989\"],[0.3333333333333333,\"#31688e\"],[0.4444444444444444,\"#26828e\"],[0.5555555555555556,\"#1f9e89\"],[0.6666666666666666,\"#35b779\"],[0.7777777777777778,\"#6ece58\"],[0.8888888888888888,\"#b5de2b\"],[1.0,\"#fde725\"]],\"type\":\"contour\"}],\"heatmapgl\":[{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"},\"colorscale\":[[0.0,\"#440154\"],[0.1111111111111111,\"#482878\"],[0.2222222222222222,\"#3e4989\"],[0.3333333333333333,\"#31688e\"],[0.4444444444444444,\"#26828e\"],[0.5555555555555556,\"#1f9e89\"],[0.6666666666666666,\"#35b779\"],[0.7777777777777778,\"#6ece58\"],[0.8888888888888888,\"#b5de2b\"],[1.0,\"#fde725\"]],\"type\":\"heatmapgl\"}],\"heatmap\":[{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"},\"colorscale\":[[0.0,\"#440154\"],[0.1111111111111111,\"#482878\"],[0.2222222222222222,\"#3e4989\"],[0.3333333333333333,\"#31688e\"],[0.4444444444444444,\"#26828e\"],[0.5555555555555556,\"#1f9e89\"],[0.6666666666666666,\"#35b779\"],[0.7777777777777778,\"#6ece58\"],[0.8888888888888888,\"#b5de2b\"],[1.0,\"#fde725\"]],\"type\":\"heatmap\"}],\"histogram2dcontour\":[{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"},\"colorscale\":[[0.0,\"#440154\"],[0.1111111111111111,\"#482878\"],[0.2222222222222222,\"#3e4989\"],[0.3333333333333333,\"#31688e\"],[0.4444444444444444,\"#26828e\"],[0.5555555555555556,\"#1f9e89\"],[0.6666666666666666,\"#35b779\"],[0.7777777777777778,\"#6ece58\"],[0.8888888888888888,\"#b5de2b\"],[1.0,\"#fde725\"]],\"type\":\"histogram2dcontour\"}],\"histogram2d\":[{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"},\"colorscale\":[[0.0,\"#440154\"],[0.1111111111111111,\"#482878\"],[0.2222222222222222,\"#3e4989\"],[0.3333333333333333,\"#31688e\"],[0.4444444444444444,\"#26828e\"],[0.5555555555555556,\"#1f9e89\"],[0.6666666666666666,\"#35b779\"],[0.7777777777777778,\"#6ece58\"],[0.8888888888888888,\"#b5de2b\"],[1.0,\"#fde725\"]],\"type\":\"histogram2d\"}],\"histogram\":[{\"marker\":{\"line\":{\"color\":\"white\",\"width\":0.6}},\"type\":\"histogram\"}],\"mesh3d\":[{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"},\"type\":\"mesh3d\"}],\"parcoords\":[{\"line\":{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"}},\"type\":\"parcoords\"}],\"pie\":[{\"automargin\":true,\"type\":\"pie\"}],\"scatter3d\":[{\"line\":{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"}},\"marker\":{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"}},\"type\":\"scatter3d\"}],\"scattercarpet\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"}},\"type\":\"scattercarpet\"}],\"scattergeo\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"}},\"type\":\"scattergeo\"}],\"scattergl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"}},\"type\":\"scattergl\"}],\"scattermapbox\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"}},\"type\":\"scattermapbox\"}],\"scatterpolargl\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"}},\"type\":\"scatterpolargl\"}],\"scatterpolar\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"}},\"type\":\"scatterpolar\"}],\"scatter\":[{\"fillpattern\":{\"fillmode\":\"overlay\",\"size\":10,\"solidity\":0.2},\"type\":\"scatter\"}],\"scatterternary\":[{\"marker\":{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"}},\"type\":\"scatterternary\"}],\"surface\":[{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"},\"colorscale\":[[0.0,\"#440154\"],[0.1111111111111111,\"#482878\"],[0.2222222222222222,\"#3e4989\"],[0.3333333333333333,\"#31688e\"],[0.4444444444444444,\"#26828e\"],[0.5555555555555556,\"#1f9e89\"],[0.6666666666666666,\"#35b779\"],[0.7777777777777778,\"#6ece58\"],[0.8888888888888888,\"#b5de2b\"],[1.0,\"#fde725\"]],\"type\":\"surface\"}],\"table\":[{\"cells\":{\"fill\":{\"color\":\"rgb(237,237,237)\"},\"line\":{\"color\":\"white\"}},\"header\":{\"fill\":{\"color\":\"rgb(217,217,217)\"},\"line\":{\"color\":\"white\"}},\"type\":\"table\"}]},\"layout\":{\"annotationdefaults\":{\"arrowhead\":0,\"arrowwidth\":1},\"autotypenumbers\":\"strict\",\"coloraxis\":{\"colorbar\":{\"outlinewidth\":1,\"tickcolor\":\"rgb(36,36,36)\",\"ticks\":\"outside\"}},\"colorscale\":{\"diverging\":[[0.0,\"rgb(103,0,31)\"],[0.1,\"rgb(178,24,43)\"],[0.2,\"rgb(214,96,77)\"],[0.3,\"rgb(244,165,130)\"],[0.4,\"rgb(253,219,199)\"],[0.5,\"rgb(247,247,247)\"],[0.6,\"rgb(209,229,240)\"],[0.7,\"rgb(146,197,222)\"],[0.8,\"rgb(67,147,195)\"],[0.9,\"rgb(33,102,172)\"],[1.0,\"rgb(5,48,97)\"]],\"sequential\":[[0.0,\"#440154\"],[0.1111111111111111,\"#482878\"],[0.2222222222222222,\"#3e4989\"],[0.3333333333333333,\"#31688e\"],[0.4444444444444444,\"#26828e\"],[0.5555555555555556,\"#1f9e89\"],[0.6666666666666666,\"#35b779\"],[0.7777777777777778,\"#6ece58\"],[0.8888888888888888,\"#b5de2b\"],[1.0,\"#fde725\"]],\"sequentialminus\":[[0.0,\"#440154\"],[0.1111111111111111,\"#482878\"],[0.2222222222222222,\"#3e4989\"],[0.3333333333333333,\"#31688e\"],[0.4444444444444444,\"#26828e\"],[0.5555555555555556,\"#1f9e89\"],[0.6666666666666666,\"#35b779\"],[0.7777777777777778,\"#6ece58\"],[0.8888888888888888,\"#b5de2b\"],[1.0,\"#fde725\"]]},\"colorway\":[\"#1F77B4\",\"#FF7F0E\",\"#2CA02C\",\"#D62728\",\"#9467BD\",\"#8C564B\",\"#E377C2\",\"#7F7F7F\",\"#BCBD22\",\"#17BECF\"],\"font\":{\"color\":\"rgb(36,36,36)\"},\"geo\":{\"bgcolor\":\"white\",\"lakecolor\":\"white\",\"landcolor\":\"white\",\"showlakes\":true,\"showland\":true,\"subunitcolor\":\"white\"},\"hoverlabel\":{\"align\":\"left\"},\"hovermode\":\"closest\",\"mapbox\":{\"style\":\"light\"},\"paper_bgcolor\":\"white\",\"plot_bgcolor\":\"white\",\"polar\":{\"angularaxis\":{\"gridcolor\":\"rgb(232,232,232)\",\"linecolor\":\"rgb(36,36,36)\",\"showgrid\":false,\"showline\":true,\"ticks\":\"outside\"},\"bgcolor\":\"white\",\"radialaxis\":{\"gridcolor\":\"rgb(232,232,232)\",\"linecolor\":\"rgb(36,36,36)\",\"showgrid\":false,\"showline\":true,\"ticks\":\"outside\"}},\"scene\":{\"xaxis\":{\"backgroundcolor\":\"white\",\"gridcolor\":\"rgb(232,232,232)\",\"gridwidth\":2,\"linecolor\":\"rgb(36,36,36)\",\"showbackground\":true,\"showgrid\":false,\"showline\":true,\"ticks\":\"outside\",\"zeroline\":false,\"zerolinecolor\":\"rgb(36,36,36)\"},\"yaxis\":{\"backgroundcolor\":\"white\",\"gridcolor\":\"rgb(232,232,232)\",\"gridwidth\":2,\"linecolor\":\"rgb(36,36,36)\",\"showbackground\":true,\"showgrid\":false,\"showline\":true,\"ticks\":\"outside\",\"zeroline\":false,\"zerolinecolor\":\"rgb(36,36,36)\"},\"zaxis\":{\"backgroundcolor\":\"white\",\"gridcolor\":\"rgb(232,232,232)\",\"gridwidth\":2,\"linecolor\":\"rgb(36,36,36)\",\"showbackground\":true,\"showgrid\":false,\"showline\":true,\"ticks\":\"outside\",\"zeroline\":false,\"zerolinecolor\":\"rgb(36,36,36)\"}},\"shapedefaults\":{\"fillcolor\":\"black\",\"line\":{\"width\":0},\"opacity\":0.3},\"ternary\":{\"aaxis\":{\"gridcolor\":\"rgb(232,232,232)\",\"linecolor\":\"rgb(36,36,36)\",\"showgrid\":false,\"showline\":true,\"ticks\":\"outside\"},\"baxis\":{\"gridcolor\":\"rgb(232,232,232)\",\"linecolor\":\"rgb(36,36,36)\",\"showgrid\":false,\"showline\":true,\"ticks\":\"outside\"},\"bgcolor\":\"white\",\"caxis\":{\"gridcolor\":\"rgb(232,232,232)\",\"linecolor\":\"rgb(36,36,36)\",\"showgrid\":false,\"showline\":true,\"ticks\":\"outside\"}},\"title\":{\"x\":0.05},\"xaxis\":{\"automargin\":true,\"gridcolor\":\"rgb(232,232,232)\",\"linecolor\":\"rgb(36,36,36)\",\"showgrid\":false,\"showline\":true,\"ticks\":\"outside\",\"title\":{\"standoff\":15},\"zeroline\":false,\"zerolinecolor\":\"rgb(36,36,36)\"},\"yaxis\":{\"automargin\":true,\"gridcolor\":\"rgb(232,232,232)\",\"linecolor\":\"rgb(36,36,36)\",\"showgrid\":false,\"showline\":true,\"ticks\":\"outside\",\"title\":{\"standoff\":15},\"zeroline\":false,\"zerolinecolor\":\"rgb(36,36,36)\"}}},\"xaxis\":{\"anchor\":\"y\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"Digit\"},\"tickvals\":[0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34]},\"yaxis\":{\"anchor\":\"x\",\"domain\":[0.0,1.0],\"title\":{\"text\":\"Frequency\"}},\"legend\":{\"title\":{\"text\":\"variable\"},\"tracegroupgap\":0},\"margin\":{\"t\":60},\"barmode\":\"relative\",\"font\":{\"size\":14},\"title\":{\"text\":\"Target Distribution\",\"y\":0.95,\"x\":0.5},\"showlegend\":false},                        {\"responsive\": true}                    ).then(function(){\n",
       "                            \n",
       "var gd = document.getElementById('39f7b806-804b-4da8-a125-b7aa491bc90f');\n",
       "var x = new MutationObserver(function (mutations, observer) {{\n",
       "        var display = window.getComputedStyle(gd).display;\n",
       "        if (!display || display === 'none') {{\n",
       "            console.log([gd, 'removed!']);\n",
       "            Plotly.purge(gd);\n",
       "            observer.disconnect();\n",
       "        }}\n",
       "}});\n",
       "\n",
       "// Listen for the removal of the full notebook cells\n",
       "var notebookContainer = gd.closest('#notebook-container');\n",
       "if (notebookContainer) {{\n",
       "    x.observe(notebookContainer, {childList: true});\n",
       "}}\n",
       "\n",
       "// Listen for the clearing of the current output cell\n",
       "var outputEl = gd.closest('.output');\n",
       "if (outputEl) {{\n",
       "    x.observe(outputEl, {childList: true});\n",
       "}}\n",
       "\n",
       "                        })                };                            </script>        </div>\n",
       "</body>\n",
       "</html>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Create figure\n",
    "fig = px.bar(y_train.value_counts(),\n",
    "             template='simple_white',\n",
    "             color_discrete_sequence = ['grey']\n",
    "            )\n",
    "\n",
    "# Set Title and x/y axis labels\n",
    "fig.update_layout(\n",
    "    xaxis = dict(\n",
    "        tickvals = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34]\n",
    "    ),\n",
    "    xaxis_title=\"Digit\",\n",
    "    yaxis_title=\"Frequency\",\n",
    "    showlegend = False,\n",
    "    font = dict(\n",
    "            size = 14\n",
    "            ),     \n",
    "    title={\n",
    "        'text': \"Target Distribution\",\n",
    "        'y':0.95,\n",
    "        'x':0.5\n",
    "        }\n",
    "    )\n",
    "\n",
    "# Display\n",
    "fig.show() # for Kaggle version\n",
    "#fig.show(\"svg\") # for GitHub version"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "c3AjpAeQ-Ppd"
   },
   "source": [
    "# Data Preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nLoJ1ZHO-Ppe"
   },
   "source": [
    "## Normalization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "FlEvFbYp-Ppe"
   },
   "source": [
    "Normalization:\n",
    "\n",
    "* In order to reduce the impact of illumination differences on image analysis, we perform grayscale normalization.\n",
    "* By normalizing the grayscale values of the images, we can ensure that the CNN is able to more accurately analyze and classify the images.\n",
    "* Normalization can also help to improve the consistency of the image data, making it easier to train the model and achieve better performance."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "86-qe2vj-Ppe"
   },
   "source": [
    "CNN speed improvement through normalization:\n",
    "\n",
    "* In addition to improving the accuracy of the CNN, normalization can also have a positive impact on the speed of the model.\n",
    "* By reducing the variation in grayscale values across different images, the CNN can process the data more quickly and efficiently."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:47.749795Z",
     "iopub.status.busy": "2023-03-24T22:14:47.749317Z",
     "iopub.status.idle": "2023-03-24T22:14:47.939599Z",
     "shell.execute_reply": "2023-03-24T22:14:47.938373Z",
     "shell.execute_reply.started": "2023-03-24T22:14:47.749754Z"
    },
    "id": "IN7ybHXm-Ppe",
    "outputId": "b507afc1-4fa0-4a3a-a8c7-4df262f33534"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train shape: (209966, 784)\n",
      "test shape: (34966, 784)\n"
     ]
    }
   ],
   "source": [
    "# Normalize the data\n",
    "X_train = X_train / 255.0\n",
    "test = test / 255.0\n",
    "\n",
    "print(f\"X_train shape: {X_train.shape}\")\n",
    "print(f\"test shape: {test.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Kgz1WRkW-Ppf"
   },
   "source": [
    "## Reshape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "kGUh5FNw-Ppf"
   },
   "source": [
    "Reshape:\n",
    "\n",
    "* In order to prepare our image data for use with a neural network, we need to reshape it into 3D matrices.\n",
    "* Our original images are 28x28 pixels in size, so we reshape them into 28x28x1 matrices to add an extra dimension for channels.\n",
    "* This is necessary because Keras, the deep learning library we are using, expects image data to be represented as 3D tensors with dimensions (width, height, channels).\n",
    "* Since our images are grayscale, we only need one channel to represent them, hence the \"1\" in the last dimension of the reshaped matrix."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:47.94193Z",
     "iopub.status.busy": "2023-03-24T22:14:47.941491Z",
     "iopub.status.idle": "2023-03-24T22:14:47.949607Z",
     "shell.execute_reply": "2023-03-24T22:14:47.948267Z",
     "shell.execute_reply.started": "2023-03-24T22:14:47.941886Z"
    },
    "id": "M-zt-e5r-Ppf",
    "outputId": "e54a182d-65f9-453c-b74e-319269c03733"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train shape: (209966, 28, 28, 1)\n",
      "test shape: (34966, 28, 28, 1)\n"
     ]
    }
   ],
   "source": [
    "# Reshape\n",
    "X_train = X_train.values.reshape(X_train.shape[0], 28, 28, 1)\n",
    "test = test.values.reshape(test.shape[0], 28, 28, 1)\n",
    "\n",
    "print(f\"X_train shape: {X_train.shape}\")\n",
    "print(f\"test shape: {test.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fVG81i_n-Ppf"
   },
   "source": [
    "## Label Encoding"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "q4GfaJ_--Ppf"
   },
   "source": [
    "Label Encoding:\n",
    "\n",
    "* In order to use categorical labels in our neural network, we need to encode them as numerical values.\n",
    "* One common method for doing this is label encoding, where each unique label is assigned a unique integer value.\n",
    "* However, since these numerical labels have an inherent ordering (e.g. 4 is \"greater\" than 2), they may not be suitable for use in certain types of models.\n",
    "* To address this issue, we can use one-hot encoding to represent each label as a binary vector where only the corresponding label index is set to 1, and all other indices are set to 0."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:47.955785Z",
     "iopub.status.busy": "2023-03-24T22:14:47.954522Z",
     "iopub.status.idle": "2023-03-24T22:14:47.962351Z",
     "shell.execute_reply": "2023-03-24T22:14:47.9611Z",
     "shell.execute_reply.started": "2023-03-24T22:14:47.95574Z"
    },
    "id": "liVrWSnQ-Ppg"
   },
   "outputs": [],
   "source": [
    "# One-hot encoding\n",
    "y_train = to_categorical(y_train, num_classes = 35)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7Z1OFEE5-Ppg"
   },
   "source": [
    "## Train Test Split"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r7EV4Fcl-Ppg"
   },
   "source": [
    "Train Test Split:\n",
    "\n",
    "* In order to evaluate the performance of our model and ensure that it can generalize well to new, unseen data, we need to split our dataset into separate training and testing sets.\n",
    "* This is typically done by randomly selecting a portion of the data to use for testing and setting aside the rest for training.\n",
    "* The size of the test and train sets can vary depending on the size of the dataset and other factors, but a common split is to use 80% of the data for training and 20% for testing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:47.964739Z",
     "iopub.status.busy": "2023-03-24T22:14:47.964071Z",
     "iopub.status.idle": "2023-03-24T22:14:48.355949Z",
     "shell.execute_reply": "2023-03-24T22:14:48.354745Z",
     "shell.execute_reply.started": "2023-03-24T22:14:47.96467Z"
    },
    "id": "pj5fM5eU-Ppg",
    "outputId": "d8a9a774-98ee-4e52-e708-c6d8d90ca72d"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "X_train shape: (167972, 28, 28, 1)\n",
      "y_train shape: (167972, 35)\n",
      "X_val shape: (41994, 28, 28, 1)\n",
      "y_val shape: (41994, 35)\n"
     ]
    }
   ],
   "source": [
    "X_train, X_val, y_train, y_val = train_test_split(X_train, y_train, test_size = 0.2, random_state=11)\n",
    "\n",
    "print(f\"X_train shape: {X_train.shape}\")\n",
    "print(f\"y_train shape: {y_train.shape}\")\n",
    "print(f\"X_val shape: {X_val.shape}\")\n",
    "print(f\"y_val shape: {y_val.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "At6Wt50b-Ppg"
   },
   "source": [
    "# Data Augmentation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "q8BsCqzR-Pph"
   },
   "source": [
    "Data augmentation is a commonly used technique to artificially expand the training dataset and reduce overfitting. Some of the common data augmentation techniques used in image recognition tasks are:\n",
    "\n",
    "* Rotation: rotating the image by a certain degree to simulate variations in writing angles.\n",
    "* Translation: shifting the image vertically or horizontally to simulate variations in position.\n",
    "* Zooming: zooming in or out of the image to simulate variations in writing size.\n",
    "* Shearing: distorting the image by shearing along the x or y-axis to simulate variations in writing style.\n",
    "\n",
    "By applying these transformations randomly to the existing training images, we can create new training images that are similar to the original ones but have small variations, making the model more robust and less likely to overfit to the training data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:48.358495Z",
     "iopub.status.busy": "2023-03-24T22:14:48.3575Z",
     "iopub.status.idle": "2023-03-24T22:14:48.445771Z",
     "shell.execute_reply": "2023-03-24T22:14:48.444617Z",
     "shell.execute_reply.started": "2023-03-24T22:14:48.358452Z"
    },
    "id": "B26uR38R-Pph"
   },
   "outputs": [],
   "source": [
    "datagen = ImageDataGenerator(\n",
    "    rotation_range=10,  \n",
    "    zoom_range = 0.10,  \n",
    "    width_shift_range=0.1, \n",
    "    height_shift_range=0.1\n",
    ")\n",
    "\n",
    "datagen.fit(X_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hLvXMS1t-Pph"
   },
   "source": [
    "# CNN Building"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "HXp5duhn-Pph"
   },
   "source": [
    "## CNN Architecture"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hwX40c_e-Pph"
   },
   "source": [
    "CNN Architecture:\n",
    "\n",
    "1. Convolutional Layer:\n",
    "    * The first layer in our CNN is a Conv2D layer, which applies a set of learnable filters to the input image.\n",
    "    * Each filter convolves over the input image, producing a feature map that captures different local features of the image.\n",
    "    * In this model, I have chosen to use 32 filters for the first two Conv2D layers and 64 filters for the last two layers.\n",
    "    * The size of the filters (kernel size) determines the size of the receptive field of the filter and the level of detail that it can capture.\n",
    "    \n",
    "    \n",
    "2. Batch Normalization Layer:\n",
    "    * Batch normalization is a technique used to normalize the inputs of each layer, improving the training speed and performance of the model.\n",
    "    * In this model, I have added a BatchNormalization layer after each Conv2D layer and before the activation function.\n",
    "    * This helps to stabilize the distribution of the inputs to each layer and reduces overfitting.\n",
    "     \n",
    "      \n",
    "3. Activation Function:\n",
    "    * In this model, I have used ReLU (Rectified Linear Unit) activation function in the Conv2D and Dense layers.\n",
    "    * The ReLU activation function adds non-linearity to the network, allowing it to learn more complex features.  \n",
    "    \n",
    "    \n",
    "4. Pooling Layer:\n",
    "    * The pooling layer is used to reduce the spatial dimensions of the feature maps produced by the Conv2D layer.\n",
    "    * In this model, I have used MaxPool2D layer, which selects the maximum value within a pool of adjacent pixels and downsamples the feature map.\n",
    "    * By downsampling the feature maps, we can reduce the computational cost and improve the model's ability to generalize by reducing overfitting.\n",
    "    * The size of the pooling window determines the level of downsampling applied to the feature maps.\n",
    "    \n",
    " \n",
    "5. Dropout Layer:\n",
    "    * Dropout is a regularization technique that randomly ignores a proportion of nodes in the layer during training, forcing the network to learn features in a distributed way.\n",
    "    * In this model, I have used Dropout layer to reduce overfitting and improve generalization.\n",
    "    * The proportion of nodes to drop can be set using a dropout rate hyperparameter.\n",
    "    \n",
    "    \n",
    "6. Flatten Layer:\n",
    "    * The Flatten layer is used to convert the final feature maps produced by the Conv2D and pooling layers into a one-dimensional vector.\n",
    "    * This is necessary to connect the CNN to the fully connected layers, which are artificial neural networks (ANN) classifiers.\n",
    "    \n",
    " \n",
    "7. Dense Layer:\n",
    "    * The fully connected layers are used to classify the input image into one of the predefined classes.\n",
    "    * In our model, we have used two Dense layers, one with 128 neurons, to extract high-level features from the flattened feature maps, and one with 10 neurons, to classify the image.\n",
    "    * The final Dense layer has 10 neurons, one for each possible class, and uses softmax activation to output a probability distribution over the classes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:48.44811Z",
     "iopub.status.busy": "2023-03-24T22:14:48.447678Z",
     "iopub.status.idle": "2023-03-24T22:14:51.452936Z",
     "shell.execute_reply": "2023-03-24T22:14:51.45165Z",
     "shell.execute_reply.started": "2023-03-24T22:14:48.448063Z"
    },
    "id": "bfDbg1u5-Ppi"
   },
   "outputs": [],
   "source": [
    "# Create CNN\n",
    "model = Sequential()\n",
    "        \n",
    "model.add(Conv2D(filters=32, kernel_size=(5,5), \n",
    "                 activation='relu', input_shape=(28,28,1)))\n",
    "model.add(BatchNormalization())\n",
    "model.add(Dropout(0.1))\n",
    "model.add(Conv2D(filters=32, kernel_size=(5,5),\n",
    "                 activation='relu'))\n",
    "model.add(BatchNormalization())\n",
    "model.add(MaxPool2D(pool_size=(2,2)))\n",
    "model.add(Dropout(0.1))\n",
    "\n",
    "\n",
    "model.add(Conv2D(filters=64, kernel_size=(3,3),\n",
    "                 activation='relu'))\n",
    "model.add(BatchNormalization())\n",
    "model.add(Dropout(0.2))\n",
    "model.add(Conv2D(filters=64, kernel_size = (3,3),\n",
    "                 activation=\"relu\"))\n",
    "model.add(BatchNormalization())\n",
    "model.add(MaxPool2D(pool_size=(2,2)))\n",
    "model.add(Dropout(0.2))\n",
    "\n",
    "\n",
    "model.add(Flatten())\n",
    "model.add(Dense(128, activation='relu'))\n",
    "model.add(Dropout(0.5))\n",
    "\n",
    "model.add(Dense(35, activation='softmax'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "5KSZnhga_bqq",
    "outputId": "1eae3057-bbb4-4e87-d069-6ed775ab03fd"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      " Layer (type)                Output Shape              Param #   \n",
      "=================================================================\n",
      " conv2d (Conv2D)             (None, 24, 24, 32)        832       \n",
      "                                                                 \n",
      " batch_normalization (BatchN  (None, 24, 24, 32)       128       \n",
      " ormalization)                                                   \n",
      "                                                                 \n",
      " dropout (Dropout)           (None, 24, 24, 32)        0         \n",
      "                                                                 \n",
      " conv2d_1 (Conv2D)           (None, 20, 20, 32)        25632     \n",
      "                                                                 \n",
      " batch_normalization_1 (Batc  (None, 20, 20, 32)       128       \n",
      " hNormalization)                                                 \n",
      "                                                                 \n",
      " max_pooling2d (MaxPooling2D  (None, 10, 10, 32)       0         \n",
      " )                                                               \n",
      "                                                                 \n",
      " dropout_1 (Dropout)         (None, 10, 10, 32)        0         \n",
      "                                                                 \n",
      " conv2d_2 (Conv2D)           (None, 8, 8, 64)          18496     \n",
      "                                                                 \n",
      " batch_normalization_2 (Batc  (None, 8, 8, 64)         256       \n",
      " hNormalization)                                                 \n",
      "                                                                 \n",
      " dropout_2 (Dropout)         (None, 8, 8, 64)          0         \n",
      "                                                                 \n",
      " conv2d_3 (Conv2D)           (None, 6, 6, 64)          36928     \n",
      "                                                                 \n",
      " batch_normalization_3 (Batc  (None, 6, 6, 64)         256       \n",
      " hNormalization)                                                 \n",
      "                                                                 \n",
      " max_pooling2d_1 (MaxPooling  (None, 3, 3, 64)         0         \n",
      " 2D)                                                             \n",
      "                                                                 \n",
      " dropout_3 (Dropout)         (None, 3, 3, 64)          0         \n",
      "                                                                 \n",
      " flatten (Flatten)           (None, 576)               0         \n",
      "                                                                 \n",
      " dense (Dense)               (None, 128)               73856     \n",
      "                                                                 \n",
      " dropout_4 (Dropout)         (None, 128)               0         \n",
      "                                                                 \n",
      " dense_1 (Dense)             (None, 35)                4515      \n",
      "                                                                 \n",
      "=================================================================\n",
      "Total params: 161,027\n",
      "Trainable params: 160,643\n",
      "Non-trainable params: 384\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "w4isRS-k-Ppi"
   },
   "source": [
    "## Set up Optimizer and Annealer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "dWXX-UhB-Ppi"
   },
   "source": [
    "Now that we have added layers to our model, we need to define a scoring function, a loss function, and an optimizer algorithm.\n",
    "* For this project, we will use the Adam optimizer and cross-entropy loss function.\n",
    "* To further improve our model's performance, we can also incorporate a learning rate annealer. This will gradually reduce the learning rate during training, allowing our model to converge more efficiently and reach better results.\n",
    "* We can choose an appropriate annealing method, such as reducing the learning rate by a factor after a certain number of epochs or when the validation loss plateaus. By using an annealer, we can fine-tune the learning process and achieve better accuracy with our model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:51.455029Z",
     "iopub.status.busy": "2023-03-24T22:14:51.454606Z",
     "iopub.status.idle": "2023-03-24T22:14:51.475164Z",
     "shell.execute_reply": "2023-03-24T22:14:51.474012Z",
     "shell.execute_reply.started": "2023-03-24T22:14:51.454984Z"
    },
    "id": "bT68djML-Ppi"
   },
   "outputs": [],
   "source": [
    "# Compile the model\n",
    "model.compile(optimizer=\"adam\", loss=\"categorical_crossentropy\", metrics=[\"accuracy\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:51.478623Z",
     "iopub.status.busy": "2023-03-24T22:14:51.477563Z",
     "iopub.status.idle": "2023-03-24T22:14:51.484091Z",
     "shell.execute_reply": "2023-03-24T22:14:51.482857Z",
     "shell.execute_reply.started": "2023-03-24T22:14:51.478574Z"
    },
    "id": "TPKok84O-Ppj"
   },
   "outputs": [],
   "source": [
    "# Set annealer\n",
    "annealer = LearningRateScheduler(lambda x: 1e-3 * 0.95 ** x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "j-xlNmNU-Ppj"
   },
   "source": [
    "## Fit the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "_kg_hide-output": true,
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "execution": {
     "iopub.execute_input": "2023-03-24T22:14:51.486175Z",
     "iopub.status.busy": "2023-03-24T22:14:51.485633Z",
     "iopub.status.idle": "2023-03-24T22:23:19.032777Z",
     "shell.execute_reply": "2023-03-24T22:23:19.031671Z",
     "shell.execute_reply.started": "2023-03-24T22:14:51.486134Z"
    },
    "id": "kV3Iv9DO-Ppj",
    "outputId": "937f472e-803a-46c6-f99c-71ed9d49ee3b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/200\n",
      "1312/1312 - 71s - loss: 1.5151 - accuracy: 0.5354 - val_loss: 0.3615 - val_accuracy: 0.8762 - lr: 0.0010 - 71s/epoch - 54ms/step\n",
      "Epoch 2/200\n",
      "1312/1312 - 54s - loss: 0.5252 - accuracy: 0.8304 - val_loss: 0.2263 - val_accuracy: 0.9274 - lr: 9.5000e-04 - 54s/epoch - 41ms/step\n",
      "Epoch 3/200\n",
      "1312/1312 - 54s - loss: 0.3816 - accuracy: 0.8816 - val_loss: 0.2062 - val_accuracy: 0.9362 - lr: 9.0250e-04 - 54s/epoch - 41ms/step\n",
      "Epoch 4/200\n",
      "1312/1312 - 54s - loss: 0.3189 - accuracy: 0.9032 - val_loss: 0.1554 - val_accuracy: 0.9515 - lr: 8.5737e-04 - 54s/epoch - 41ms/step\n",
      "Epoch 5/200\n",
      "1312/1312 - 54s - loss: 0.2818 - accuracy: 0.9145 - val_loss: 0.1337 - val_accuracy: 0.9609 - lr: 8.1451e-04 - 54s/epoch - 41ms/step\n",
      "Epoch 6/200\n",
      "1312/1312 - 54s - loss: 0.2591 - accuracy: 0.9229 - val_loss: 0.1082 - val_accuracy: 0.9675 - lr: 7.7378e-04 - 54s/epoch - 41ms/step\n",
      "Epoch 7/200\n",
      "1312/1312 - 54s - loss: 0.2396 - accuracy: 0.9285 - val_loss: 0.1074 - val_accuracy: 0.9697 - lr: 7.3509e-04 - 54s/epoch - 41ms/step\n",
      "Epoch 8/200\n",
      "1312/1312 - 54s - loss: 0.2257 - accuracy: 0.9333 - val_loss: 0.1165 - val_accuracy: 0.9671 - lr: 6.9834e-04 - 54s/epoch - 42ms/step\n",
      "Epoch 9/200\n",
      "1312/1312 - 53s - loss: 0.2197 - accuracy: 0.9357 - val_loss: 0.1040 - val_accuracy: 0.9708 - lr: 6.6342e-04 - 53s/epoch - 40ms/step\n",
      "Epoch 10/200\n",
      "1312/1312 - 52s - loss: 0.2069 - accuracy: 0.9391 - val_loss: 0.1236 - val_accuracy: 0.9663 - lr: 6.3025e-04 - 52s/epoch - 40ms/step\n",
      "Epoch 11/200\n",
      "1312/1312 - 52s - loss: 0.1985 - accuracy: 0.9419 - val_loss: 0.1076 - val_accuracy: 0.9703 - lr: 5.9874e-04 - 52s/epoch - 39ms/step\n",
      "Epoch 12/200\n",
      "1312/1312 - 52s - loss: 0.1918 - accuracy: 0.9441 - val_loss: 0.1085 - val_accuracy: 0.9696 - lr: 5.6880e-04 - 52s/epoch - 39ms/step\n",
      "Epoch 13/200\n",
      "1312/1312 - 53s - loss: 0.1852 - accuracy: 0.9456 - val_loss: 0.0976 - val_accuracy: 0.9730 - lr: 5.4036e-04 - 53s/epoch - 40ms/step\n",
      "Epoch 14/200\n",
      "1312/1312 - 52s - loss: 0.1809 - accuracy: 0.9472 - val_loss: 0.0882 - val_accuracy: 0.9748 - lr: 5.1334e-04 - 52s/epoch - 40ms/step\n",
      "Epoch 15/200\n",
      "1312/1312 - 52s - loss: 0.1768 - accuracy: 0.9491 - val_loss: 0.0883 - val_accuracy: 0.9763 - lr: 4.8767e-04 - 52s/epoch - 39ms/step\n",
      "Epoch 16/200\n",
      "1312/1312 - 54s - loss: 0.1708 - accuracy: 0.9497 - val_loss: 0.0846 - val_accuracy: 0.9771 - lr: 4.6329e-04 - 54s/epoch - 41ms/step\n",
      "Epoch 17/200\n",
      "1312/1312 - 54s - loss: 0.1698 - accuracy: 0.9508 - val_loss: 0.0819 - val_accuracy: 0.9778 - lr: 4.4013e-04 - 54s/epoch - 41ms/step\n",
      "Epoch 18/200\n",
      "1312/1312 - 55s - loss: 0.1669 - accuracy: 0.9509 - val_loss: 0.0811 - val_accuracy: 0.9777 - lr: 4.1812e-04 - 55s/epoch - 42ms/step\n",
      "Epoch 19/200\n",
      "1312/1312 - 54s - loss: 0.1609 - accuracy: 0.9531 - val_loss: 0.0839 - val_accuracy: 0.9776 - lr: 3.9721e-04 - 54s/epoch - 41ms/step\n",
      "Epoch 20/200\n",
      "1312/1312 - 53s - loss: 0.1579 - accuracy: 0.9541 - val_loss: 0.0789 - val_accuracy: 0.9793 - lr: 3.7735e-04 - 53s/epoch - 41ms/step\n",
      "Epoch 21/200\n",
      "1312/1312 - 53s - loss: 0.1560 - accuracy: 0.9541 - val_loss: 0.0877 - val_accuracy: 0.9768 - lr: 3.5849e-04 - 53s/epoch - 40ms/step\n",
      "Epoch 22/200\n",
      "1312/1312 - 54s - loss: 0.1542 - accuracy: 0.9556 - val_loss: 0.0887 - val_accuracy: 0.9756 - lr: 3.4056e-04 - 54s/epoch - 41ms/step\n",
      "Epoch 23/200\n",
      "1312/1312 - 54s - loss: 0.1513 - accuracy: 0.9558 - val_loss: 0.0868 - val_accuracy: 0.9768 - lr: 3.2353e-04 - 54s/epoch - 41ms/step\n",
      "Epoch 24/200\n",
      "1312/1312 - 55s - loss: 0.1507 - accuracy: 0.9562 - val_loss: 0.0758 - val_accuracy: 0.9791 - lr: 3.0736e-04 - 55s/epoch - 42ms/step\n",
      "Epoch 25/200\n",
      "1312/1312 - 52s - loss: 0.1470 - accuracy: 0.9572 - val_loss: 0.0775 - val_accuracy: 0.9792 - lr: 2.9199e-04 - 52s/epoch - 40ms/step\n",
      "Epoch 26/200\n",
      "1312/1312 - 53s - loss: 0.1457 - accuracy: 0.9576 - val_loss: 0.0774 - val_accuracy: 0.9794 - lr: 2.7739e-04 - 53s/epoch - 40ms/step\n",
      "Epoch 27/200\n",
      "1312/1312 - 53s - loss: 0.1434 - accuracy: 0.9587 - val_loss: 0.0742 - val_accuracy: 0.9799 - lr: 2.6352e-04 - 53s/epoch - 40ms/step\n",
      "Epoch 28/200\n",
      "1312/1312 - 52s - loss: 0.1418 - accuracy: 0.9584 - val_loss: 0.0727 - val_accuracy: 0.9808 - lr: 2.5034e-04 - 52s/epoch - 40ms/step\n",
      "Epoch 29/200\n",
      "1312/1312 - 53s - loss: 0.1393 - accuracy: 0.9594 - val_loss: 0.0768 - val_accuracy: 0.9793 - lr: 2.3783e-04 - 53s/epoch - 41ms/step\n",
      "Epoch 30/200\n",
      "1312/1312 - 53s - loss: 0.1410 - accuracy: 0.9586 - val_loss: 0.0750 - val_accuracy: 0.9807 - lr: 2.2594e-04 - 53s/epoch - 40ms/step\n",
      "Epoch 31/200\n",
      "1312/1312 - 53s - loss: 0.1379 - accuracy: 0.9599 - val_loss: 0.0720 - val_accuracy: 0.9805 - lr: 2.1464e-04 - 53s/epoch - 40ms/step\n",
      "Epoch 32/200\n",
      "1312/1312 - 55s - loss: 0.1372 - accuracy: 0.9601 - val_loss: 0.0727 - val_accuracy: 0.9807 - lr: 2.0391e-04 - 55s/epoch - 42ms/step\n",
      "Epoch 33/200\n",
      "1312/1312 - 53s - loss: 0.1346 - accuracy: 0.9607 - val_loss: 0.0707 - val_accuracy: 0.9810 - lr: 1.9371e-04 - 53s/epoch - 41ms/step\n",
      "Epoch 34/200\n",
      "1312/1312 - 53s - loss: 0.1328 - accuracy: 0.9614 - val_loss: 0.0734 - val_accuracy: 0.9805 - lr: 1.8403e-04 - 53s/epoch - 41ms/step\n",
      "Epoch 35/200\n",
      "1312/1312 - 53s - loss: 0.1341 - accuracy: 0.9609 - val_loss: 0.0737 - val_accuracy: 0.9802 - lr: 1.7482e-04 - 53s/epoch - 40ms/step\n",
      "Epoch 36/200\n",
      "1312/1312 - 52s - loss: 0.1325 - accuracy: 0.9616 - val_loss: 0.0739 - val_accuracy: 0.9799 - lr: 1.6608e-04 - 52s/epoch - 40ms/step\n",
      "Epoch 37/200\n",
      "1312/1312 - 53s - loss: 0.1310 - accuracy: 0.9619 - val_loss: 0.0727 - val_accuracy: 0.9806 - lr: 1.5778e-04 - 53s/epoch - 41ms/step\n",
      "Epoch 38/200\n",
      "1312/1312 - 52s - loss: 0.1315 - accuracy: 0.9614 - val_loss: 0.0694 - val_accuracy: 0.9816 - lr: 1.4989e-04 - 52s/epoch - 40ms/step\n",
      "Epoch 39/200\n",
      "1312/1312 - 52s - loss: 0.1303 - accuracy: 0.9615 - val_loss: 0.0696 - val_accuracy: 0.9815 - lr: 1.4240e-04 - 52s/epoch - 40ms/step\n",
      "Epoch 40/200\n",
      "1312/1312 - 52s - loss: 0.1295 - accuracy: 0.9623 - val_loss: 0.0699 - val_accuracy: 0.9819 - lr: 1.3528e-04 - 52s/epoch - 40ms/step\n",
      "Epoch 41/200\n",
      "1312/1312 - 53s - loss: 0.1274 - accuracy: 0.9632 - val_loss: 0.0685 - val_accuracy: 0.9817 - lr: 1.2851e-04 - 53s/epoch - 40ms/step\n",
      "Epoch 42/200\n",
      "1312/1312 - 52s - loss: 0.1273 - accuracy: 0.9631 - val_loss: 0.0692 - val_accuracy: 0.9816 - lr: 1.2209e-04 - 52s/epoch - 39ms/step\n",
      "Epoch 43/200\n",
      "1312/1312 - 52s - loss: 0.1256 - accuracy: 0.9632 - val_loss: 0.0715 - val_accuracy: 0.9811 - lr: 1.1598e-04 - 52s/epoch - 39ms/step\n",
      "Epoch 44/200\n",
      "1312/1312 - 52s - loss: 0.1259 - accuracy: 0.9631 - val_loss: 0.0692 - val_accuracy: 0.9820 - lr: 1.1018e-04 - 52s/epoch - 40ms/step\n",
      "Epoch 45/200\n",
      "1312/1312 - 53s - loss: 0.1285 - accuracy: 0.9628 - val_loss: 0.0689 - val_accuracy: 0.9819 - lr: 1.0467e-04 - 53s/epoch - 41ms/step\n",
      "Epoch 46/200\n",
      "1312/1312 - 54s - loss: 0.1257 - accuracy: 0.9631 - val_loss: 0.0693 - val_accuracy: 0.9821 - lr: 9.9440e-05 - 54s/epoch - 42ms/step\n",
      "Epoch 47/200\n",
      "1312/1312 - 55s - loss: 0.1238 - accuracy: 0.9636 - val_loss: 0.0693 - val_accuracy: 0.9820 - lr: 9.4468e-05 - 55s/epoch - 42ms/step\n",
      "Epoch 48/200\n",
      "1312/1312 - 55s - loss: 0.1240 - accuracy: 0.9638 - val_loss: 0.0696 - val_accuracy: 0.9820 - lr: 8.9745e-05 - 55s/epoch - 42ms/step\n",
      "Epoch 49/200\n",
      "1312/1312 - 53s - loss: 0.1229 - accuracy: 0.9642 - val_loss: 0.0687 - val_accuracy: 0.9823 - lr: 8.5258e-05 - 53s/epoch - 40ms/step\n",
      "Epoch 50/200\n",
      "1312/1312 - 55s - loss: 0.1238 - accuracy: 0.9641 - val_loss: 0.0683 - val_accuracy: 0.9822 - lr: 8.0995e-05 - 55s/epoch - 42ms/step\n",
      "Epoch 51/200\n",
      "1312/1312 - 53s - loss: 0.1210 - accuracy: 0.9641 - val_loss: 0.0677 - val_accuracy: 0.9823 - lr: 7.6945e-05 - 53s/epoch - 41ms/step\n",
      "Epoch 52/200\n",
      "1312/1312 - 55s - loss: 0.1219 - accuracy: 0.9643 - val_loss: 0.0682 - val_accuracy: 0.9821 - lr: 7.3098e-05 - 55s/epoch - 42ms/step\n",
      "Epoch 53/200\n",
      "1312/1312 - 54s - loss: 0.1235 - accuracy: 0.9641 - val_loss: 0.0668 - val_accuracy: 0.9829 - lr: 6.9443e-05 - 54s/epoch - 41ms/step\n",
      "Epoch 54/200\n",
      "1312/1312 - 56s - loss: 0.1205 - accuracy: 0.9648 - val_loss: 0.0676 - val_accuracy: 0.9827 - lr: 6.5971e-05 - 56s/epoch - 42ms/step\n",
      "Epoch 55/200\n",
      "1312/1312 - 52s - loss: 0.1199 - accuracy: 0.9650 - val_loss: 0.0679 - val_accuracy: 0.9825 - lr: 6.2672e-05 - 52s/epoch - 40ms/step\n",
      "Epoch 56/200\n",
      "1312/1312 - 55s - loss: 0.1221 - accuracy: 0.9651 - val_loss: 0.0683 - val_accuracy: 0.9827 - lr: 5.9539e-05 - 55s/epoch - 42ms/step\n",
      "Epoch 57/200\n",
      "1312/1312 - 55s - loss: 0.1189 - accuracy: 0.9654 - val_loss: 0.0661 - val_accuracy: 0.9827 - lr: 5.6562e-05 - 55s/epoch - 42ms/step\n",
      "Epoch 58/200\n",
      "1312/1312 - 54s - loss: 0.1187 - accuracy: 0.9652 - val_loss: 0.0675 - val_accuracy: 0.9826 - lr: 5.3734e-05 - 54s/epoch - 41ms/step\n",
      "Epoch 59/200\n",
      "1312/1312 - 55s - loss: 0.1219 - accuracy: 0.9647 - val_loss: 0.0675 - val_accuracy: 0.9824 - lr: 5.1047e-05 - 55s/epoch - 42ms/step\n",
      "Epoch 60/200\n",
      "1312/1312 - 56s - loss: 0.1194 - accuracy: 0.9649 - val_loss: 0.0674 - val_accuracy: 0.9828 - lr: 4.8495e-05 - 56s/epoch - 42ms/step\n",
      "Epoch 61/200\n",
      "1312/1312 - 56s - loss: 0.1201 - accuracy: 0.9649 - val_loss: 0.0682 - val_accuracy: 0.9826 - lr: 4.6070e-05 - 56s/epoch - 43ms/step\n",
      "Epoch 62/200\n",
      "1312/1312 - 54s - loss: 0.1195 - accuracy: 0.9647 - val_loss: 0.0672 - val_accuracy: 0.9827 - lr: 4.3766e-05 - 54s/epoch - 41ms/step\n",
      "Epoch 63/200\n",
      "1312/1312 - 54s - loss: 0.1176 - accuracy: 0.9653 - val_loss: 0.0673 - val_accuracy: 0.9826 - lr: 4.1578e-05 - 54s/epoch - 41ms/step\n",
      "Epoch 64/200\n",
      "1312/1312 - 54s - loss: 0.1176 - accuracy: 0.9654 - val_loss: 0.0675 - val_accuracy: 0.9825 - lr: 3.9499e-05 - 54s/epoch - 41ms/step\n",
      "Epoch 65/200\n",
      "1312/1312 - 54s - loss: 0.1172 - accuracy: 0.9651 - val_loss: 0.0672 - val_accuracy: 0.9828 - lr: 3.7524e-05 - 54s/epoch - 41ms/step\n",
      "Epoch 66/200\n",
      "1312/1312 - 57s - loss: 0.1175 - accuracy: 0.9658 - val_loss: 0.0670 - val_accuracy: 0.9824 - lr: 3.5648e-05 - 57s/epoch - 43ms/step\n",
      "Epoch 67/200\n",
      "1312/1312 - 56s - loss: 0.1185 - accuracy: 0.9655 - val_loss: 0.0676 - val_accuracy: 0.9823 - lr: 3.3866e-05 - 56s/epoch - 43ms/step\n",
      "Epoch 68/200\n",
      "1312/1312 - 54s - loss: 0.1179 - accuracy: 0.9660 - val_loss: 0.0654 - val_accuracy: 0.9832 - lr: 3.2172e-05 - 54s/epoch - 41ms/step\n",
      "Epoch 69/200\n",
      "1312/1312 - 56s - loss: 0.1182 - accuracy: 0.9656 - val_loss: 0.0667 - val_accuracy: 0.9832 - lr: 3.0564e-05 - 56s/epoch - 42ms/step\n",
      "Epoch 70/200\n",
      "1312/1312 - 56s - loss: 0.1190 - accuracy: 0.9653 - val_loss: 0.0668 - val_accuracy: 0.9830 - lr: 2.9035e-05 - 56s/epoch - 42ms/step\n",
      "Epoch 71/200\n",
      "1312/1312 - 54s - loss: 0.1189 - accuracy: 0.9653 - val_loss: 0.0671 - val_accuracy: 0.9827 - lr: 2.7584e-05 - 54s/epoch - 41ms/step\n",
      "Epoch 72/200\n",
      "1312/1312 - 53s - loss: 0.1168 - accuracy: 0.9661 - val_loss: 0.0670 - val_accuracy: 0.9827 - lr: 2.6205e-05 - 53s/epoch - 40ms/step\n",
      "Epoch 73/200\n",
      "1312/1312 - 53s - loss: 0.1161 - accuracy: 0.9662 - val_loss: 0.0667 - val_accuracy: 0.9827 - lr: 2.4894e-05 - 53s/epoch - 40ms/step\n",
      "Epoch 74/200\n",
      "1312/1312 - 55s - loss: 0.1149 - accuracy: 0.9663 - val_loss: 0.0670 - val_accuracy: 0.9828 - lr: 2.3650e-05 - 55s/epoch - 42ms/step\n",
      "Epoch 75/200\n",
      "1312/1312 - 53s - loss: 0.1163 - accuracy: 0.9662 - val_loss: 0.0668 - val_accuracy: 0.9827 - lr: 2.2467e-05 - 53s/epoch - 41ms/step\n",
      "Epoch 76/200\n",
      "1312/1312 - 54s - loss: 0.1162 - accuracy: 0.9661 - val_loss: 0.0671 - val_accuracy: 0.9826 - lr: 2.1344e-05 - 54s/epoch - 41ms/step\n",
      "Epoch 77/200\n",
      "1312/1312 - 54s - loss: 0.1178 - accuracy: 0.9659 - val_loss: 0.0668 - val_accuracy: 0.9827 - lr: 2.0277e-05 - 54s/epoch - 41ms/step\n",
      "Epoch 78/200\n",
      "1312/1312 - 53s - loss: 0.1180 - accuracy: 0.9654 - val_loss: 0.0673 - val_accuracy: 0.9826 - lr: 1.9263e-05 - 53s/epoch - 41ms/step\n",
      "Epoch 79/200\n",
      "1312/1312 - 54s - loss: 0.1172 - accuracy: 0.9659 - val_loss: 0.0668 - val_accuracy: 0.9826 - lr: 1.8300e-05 - 54s/epoch - 41ms/step\n",
      "Epoch 80/200\n",
      "1312/1312 - 56s - loss: 0.1165 - accuracy: 0.9660 - val_loss: 0.0668 - val_accuracy: 0.9830 - lr: 1.7385e-05 - 56s/epoch - 42ms/step\n",
      "Epoch 81/200\n",
      "1312/1312 - 53s - loss: 0.1147 - accuracy: 0.9667 - val_loss: 0.0665 - val_accuracy: 0.9829 - lr: 1.6515e-05 - 53s/epoch - 41ms/step\n",
      "Epoch 82/200\n",
      "1312/1312 - 55s - loss: 0.1181 - accuracy: 0.9657 - val_loss: 0.0658 - val_accuracy: 0.9830 - lr: 1.5690e-05 - 55s/epoch - 42ms/step\n",
      "Epoch 83/200\n",
      "1312/1312 - 55s - loss: 0.1175 - accuracy: 0.9659 - val_loss: 0.0666 - val_accuracy: 0.9826 - lr: 1.4905e-05 - 55s/epoch - 42ms/step\n",
      "Epoch 84/200\n",
      "1312/1312 - 55s - loss: 0.1172 - accuracy: 0.9658 - val_loss: 0.0666 - val_accuracy: 0.9828 - lr: 1.4160e-05 - 55s/epoch - 42ms/step\n",
      "Epoch 85/200\n",
      "1312/1312 - 55s - loss: 0.1152 - accuracy: 0.9667 - val_loss: 0.0664 - val_accuracy: 0.9829 - lr: 1.3452e-05 - 55s/epoch - 42ms/step\n",
      "Epoch 86/200\n",
      "1312/1312 - 53s - loss: 0.1156 - accuracy: 0.9668 - val_loss: 0.0659 - val_accuracy: 0.9829 - lr: 1.2779e-05 - 53s/epoch - 40ms/step\n",
      "Epoch 87/200\n",
      "1312/1312 - 54s - loss: 0.1144 - accuracy: 0.9663 - val_loss: 0.0661 - val_accuracy: 0.9830 - lr: 1.2140e-05 - 54s/epoch - 42ms/step\n",
      "Epoch 88/200\n",
      "1312/1312 - 54s - loss: 0.1167 - accuracy: 0.9656 - val_loss: 0.0658 - val_accuracy: 0.9828 - lr: 1.1533e-05 - 54s/epoch - 41ms/step\n",
      "Epoch 89/200\n",
      "1312/1312 - 52s - loss: 0.1161 - accuracy: 0.9665 - val_loss: 0.0665 - val_accuracy: 0.9829 - lr: 1.0957e-05 - 52s/epoch - 40ms/step\n",
      "Epoch 90/200\n",
      "1312/1312 - 52s - loss: 0.1140 - accuracy: 0.9659 - val_loss: 0.0660 - val_accuracy: 0.9831 - lr: 1.0409e-05 - 52s/epoch - 40ms/step\n",
      "Epoch 91/200\n",
      "1312/1312 - 54s - loss: 0.1170 - accuracy: 0.9661 - val_loss: 0.0659 - val_accuracy: 0.9829 - lr: 9.8884e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 92/200\n",
      "1312/1312 - 53s - loss: 0.1154 - accuracy: 0.9662 - val_loss: 0.0663 - val_accuracy: 0.9828 - lr: 9.3939e-06 - 53s/epoch - 40ms/step\n",
      "Epoch 93/200\n",
      "1312/1312 - 55s - loss: 0.1141 - accuracy: 0.9664 - val_loss: 0.0666 - val_accuracy: 0.9826 - lr: 8.9242e-06 - 55s/epoch - 42ms/step\n",
      "Epoch 94/200\n",
      "1312/1312 - 54s - loss: 0.1157 - accuracy: 0.9659 - val_loss: 0.0664 - val_accuracy: 0.9828 - lr: 8.4780e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 95/200\n",
      "1312/1312 - 51s - loss: 0.1164 - accuracy: 0.9660 - val_loss: 0.0659 - val_accuracy: 0.9829 - lr: 8.0541e-06 - 51s/epoch - 39ms/step\n",
      "Epoch 96/200\n",
      "1312/1312 - 53s - loss: 0.1174 - accuracy: 0.9661 - val_loss: 0.0659 - val_accuracy: 0.9829 - lr: 7.6514e-06 - 53s/epoch - 40ms/step\n",
      "Epoch 97/200\n",
      "1312/1312 - 55s - loss: 0.1165 - accuracy: 0.9658 - val_loss: 0.0659 - val_accuracy: 0.9827 - lr: 7.2689e-06 - 55s/epoch - 42ms/step\n",
      "Epoch 98/200\n",
      "1312/1312 - 53s - loss: 0.1162 - accuracy: 0.9664 - val_loss: 0.0658 - val_accuracy: 0.9827 - lr: 6.9054e-06 - 53s/epoch - 40ms/step\n",
      "Epoch 99/200\n",
      "1312/1312 - 53s - loss: 0.1140 - accuracy: 0.9667 - val_loss: 0.0658 - val_accuracy: 0.9828 - lr: 6.5601e-06 - 53s/epoch - 41ms/step\n",
      "Epoch 100/200\n",
      "1312/1312 - 55s - loss: 0.1149 - accuracy: 0.9665 - val_loss: 0.0664 - val_accuracy: 0.9827 - lr: 6.2321e-06 - 55s/epoch - 42ms/step\n",
      "Epoch 101/200\n",
      "1312/1312 - 54s - loss: 0.1146 - accuracy: 0.9662 - val_loss: 0.0662 - val_accuracy: 0.9828 - lr: 5.9205e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 102/200\n",
      "1312/1312 - 55s - loss: 0.1163 - accuracy: 0.9663 - val_loss: 0.0665 - val_accuracy: 0.9828 - lr: 5.6245e-06 - 55s/epoch - 42ms/step\n",
      "Epoch 103/200\n",
      "1312/1312 - 55s - loss: 0.1144 - accuracy: 0.9660 - val_loss: 0.0663 - val_accuracy: 0.9828 - lr: 5.3433e-06 - 55s/epoch - 42ms/step\n",
      "Epoch 104/200\n",
      "1312/1312 - 53s - loss: 0.1137 - accuracy: 0.9664 - val_loss: 0.0663 - val_accuracy: 0.9827 - lr: 5.0761e-06 - 53s/epoch - 40ms/step\n",
      "Epoch 105/200\n",
      "1312/1312 - 54s - loss: 0.1155 - accuracy: 0.9660 - val_loss: 0.0662 - val_accuracy: 0.9828 - lr: 4.8223e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 106/200\n",
      "1312/1312 - 56s - loss: 0.1147 - accuracy: 0.9662 - val_loss: 0.0662 - val_accuracy: 0.9828 - lr: 4.5812e-06 - 56s/epoch - 42ms/step\n",
      "Epoch 107/200\n",
      "1312/1312 - 54s - loss: 0.1165 - accuracy: 0.9664 - val_loss: 0.0663 - val_accuracy: 0.9829 - lr: 4.3521e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 108/200\n",
      "1312/1312 - 54s - loss: 0.1129 - accuracy: 0.9671 - val_loss: 0.0663 - val_accuracy: 0.9827 - lr: 4.1345e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 109/200\n",
      "1312/1312 - 53s - loss: 0.1153 - accuracy: 0.9665 - val_loss: 0.0663 - val_accuracy: 0.9827 - lr: 3.9278e-06 - 53s/epoch - 40ms/step\n",
      "Epoch 110/200\n",
      "1312/1312 - 54s - loss: 0.1156 - accuracy: 0.9663 - val_loss: 0.0660 - val_accuracy: 0.9827 - lr: 3.7314e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 111/200\n",
      "1312/1312 - 53s - loss: 0.1122 - accuracy: 0.9671 - val_loss: 0.0661 - val_accuracy: 0.9829 - lr: 3.5448e-06 - 53s/epoch - 41ms/step\n",
      "Epoch 112/200\n",
      "1312/1312 - 55s - loss: 0.1152 - accuracy: 0.9661 - val_loss: 0.0662 - val_accuracy: 0.9829 - lr: 3.3676e-06 - 55s/epoch - 42ms/step\n",
      "Epoch 113/200\n",
      "1312/1312 - 54s - loss: 0.1142 - accuracy: 0.9665 - val_loss: 0.0663 - val_accuracy: 0.9828 - lr: 3.1992e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 114/200\n",
      "1312/1312 - 53s - loss: 0.1153 - accuracy: 0.9665 - val_loss: 0.0663 - val_accuracy: 0.9829 - lr: 3.0393e-06 - 53s/epoch - 40ms/step\n",
      "Epoch 115/200\n",
      "1312/1312 - 52s - loss: 0.1154 - accuracy: 0.9658 - val_loss: 0.0666 - val_accuracy: 0.9828 - lr: 2.8873e-06 - 52s/epoch - 40ms/step\n",
      "Epoch 116/200\n",
      "1312/1312 - 54s - loss: 0.1137 - accuracy: 0.9668 - val_loss: 0.0665 - val_accuracy: 0.9828 - lr: 2.7429e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 117/200\n",
      "1312/1312 - 55s - loss: 0.1167 - accuracy: 0.9658 - val_loss: 0.0665 - val_accuracy: 0.9828 - lr: 2.6058e-06 - 55s/epoch - 42ms/step\n",
      "Epoch 118/200\n",
      "1312/1312 - 55s - loss: 0.1144 - accuracy: 0.9661 - val_loss: 0.0667 - val_accuracy: 0.9828 - lr: 2.4755e-06 - 55s/epoch - 42ms/step\n",
      "Epoch 119/200\n",
      "1312/1312 - 53s - loss: 0.1164 - accuracy: 0.9660 - val_loss: 0.0662 - val_accuracy: 0.9829 - lr: 2.3517e-06 - 53s/epoch - 41ms/step\n",
      "Epoch 120/200\n",
      "1312/1312 - 53s - loss: 0.1129 - accuracy: 0.9667 - val_loss: 0.0663 - val_accuracy: 0.9829 - lr: 2.2341e-06 - 53s/epoch - 41ms/step\n",
      "Epoch 121/200\n",
      "1312/1312 - 54s - loss: 0.1153 - accuracy: 0.9663 - val_loss: 0.0663 - val_accuracy: 0.9829 - lr: 2.1224e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 122/200\n",
      "1312/1312 - 54s - loss: 0.1160 - accuracy: 0.9662 - val_loss: 0.0665 - val_accuracy: 0.9828 - lr: 2.0163e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 123/200\n",
      "1312/1312 - 54s - loss: 0.1147 - accuracy: 0.9662 - val_loss: 0.0664 - val_accuracy: 0.9829 - lr: 1.9155e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 124/200\n",
      "1312/1312 - 54s - loss: 0.1130 - accuracy: 0.9671 - val_loss: 0.0666 - val_accuracy: 0.9828 - lr: 1.8197e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 125/200\n",
      "1312/1312 - 54s - loss: 0.1137 - accuracy: 0.9670 - val_loss: 0.0662 - val_accuracy: 0.9828 - lr: 1.7287e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 126/200\n",
      "1312/1312 - 56s - loss: 0.1142 - accuracy: 0.9665 - val_loss: 0.0665 - val_accuracy: 0.9829 - lr: 1.6423e-06 - 56s/epoch - 42ms/step\n",
      "Epoch 127/200\n",
      "1312/1312 - 55s - loss: 0.1146 - accuracy: 0.9661 - val_loss: 0.0664 - val_accuracy: 0.9829 - lr: 1.5602e-06 - 55s/epoch - 42ms/step\n",
      "Epoch 128/200\n",
      "1312/1312 - 55s - loss: 0.1141 - accuracy: 0.9666 - val_loss: 0.0663 - val_accuracy: 0.9830 - lr: 1.4822e-06 - 55s/epoch - 42ms/step\n",
      "Epoch 129/200\n",
      "1312/1312 - 55s - loss: 0.1146 - accuracy: 0.9665 - val_loss: 0.0664 - val_accuracy: 0.9829 - lr: 1.4081e-06 - 55s/epoch - 42ms/step\n",
      "Epoch 130/200\n",
      "1312/1312 - 54s - loss: 0.1134 - accuracy: 0.9665 - val_loss: 0.0663 - val_accuracy: 0.9830 - lr: 1.3377e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 131/200\n",
      "1312/1312 - 52s - loss: 0.1146 - accuracy: 0.9665 - val_loss: 0.0667 - val_accuracy: 0.9829 - lr: 1.2708e-06 - 52s/epoch - 40ms/step\n",
      "Epoch 132/200\n",
      "1312/1312 - 54s - loss: 0.1142 - accuracy: 0.9662 - val_loss: 0.0663 - val_accuracy: 0.9830 - lr: 1.2072e-06 - 54s/epoch - 41ms/step\n",
      "Epoch 133/200\n",
      "1312/1312 - 53s - loss: 0.1143 - accuracy: 0.9663 - val_loss: 0.0665 - val_accuracy: 0.9829 - lr: 1.1469e-06 - 53s/epoch - 41ms/step\n",
      "Epoch 134/200\n",
      "1312/1312 - 55s - loss: 0.1140 - accuracy: 0.9664 - val_loss: 0.0665 - val_accuracy: 0.9829 - lr: 1.0895e-06 - 55s/epoch - 42ms/step\n",
      "Epoch 135/200\n",
      "1312/1312 - 53s - loss: 0.1171 - accuracy: 0.9657 - val_loss: 0.0662 - val_accuracy: 0.9830 - lr: 1.0351e-06 - 53s/epoch - 40ms/step\n",
      "Epoch 136/200\n",
      "1312/1312 - 54s - loss: 0.1160 - accuracy: 0.9663 - val_loss: 0.0662 - val_accuracy: 0.9831 - lr: 9.8330e-07 - 54s/epoch - 41ms/step\n",
      "Epoch 137/200\n",
      "1312/1312 - 55s - loss: 0.1133 - accuracy: 0.9666 - val_loss: 0.0662 - val_accuracy: 0.9830 - lr: 9.3414e-07 - 55s/epoch - 42ms/step\n",
      "Epoch 138/200\n",
      "1312/1312 - 53s - loss: 0.1123 - accuracy: 0.9673 - val_loss: 0.0661 - val_accuracy: 0.9831 - lr: 8.8743e-07 - 53s/epoch - 41ms/step\n",
      "Epoch 139/200\n",
      "1312/1312 - 54s - loss: 0.1143 - accuracy: 0.9665 - val_loss: 0.0664 - val_accuracy: 0.9829 - lr: 8.4306e-07 - 54s/epoch - 41ms/step\n",
      "Epoch 140/200\n",
      "1312/1312 - 53s - loss: 0.1149 - accuracy: 0.9666 - val_loss: 0.0662 - val_accuracy: 0.9829 - lr: 8.0091e-07 - 53s/epoch - 40ms/step\n",
      "Epoch 141/200\n",
      "1312/1312 - 53s - loss: 0.1132 - accuracy: 0.9666 - val_loss: 0.0662 - val_accuracy: 0.9829 - lr: 7.6086e-07 - 53s/epoch - 40ms/step\n",
      "Epoch 142/200\n",
      "1312/1312 - 53s - loss: 0.1155 - accuracy: 0.9662 - val_loss: 0.0667 - val_accuracy: 0.9828 - lr: 7.2282e-07 - 53s/epoch - 40ms/step\n",
      "Epoch 143/200\n",
      "1312/1312 - 53s - loss: 0.1152 - accuracy: 0.9666 - val_loss: 0.0665 - val_accuracy: 0.9828 - lr: 6.8668e-07 - 53s/epoch - 41ms/step\n",
      "Epoch 144/200\n",
      "1312/1312 - 54s - loss: 0.1150 - accuracy: 0.9661 - val_loss: 0.0665 - val_accuracy: 0.9829 - lr: 6.5234e-07 - 54s/epoch - 41ms/step\n",
      "Epoch 145/200\n",
      "1312/1312 - 55s - loss: 0.1138 - accuracy: 0.9669 - val_loss: 0.0662 - val_accuracy: 0.9829 - lr: 6.1973e-07 - 55s/epoch - 42ms/step\n",
      "Epoch 146/200\n",
      "1312/1312 - 53s - loss: 0.1129 - accuracy: 0.9666 - val_loss: 0.0665 - val_accuracy: 0.9829 - lr: 5.8874e-07 - 53s/epoch - 41ms/step\n",
      "Epoch 147/200\n",
      "1312/1312 - 53s - loss: 0.1149 - accuracy: 0.9663 - val_loss: 0.0665 - val_accuracy: 0.9829 - lr: 5.5930e-07 - 53s/epoch - 41ms/step\n",
      "Epoch 148/200\n",
      "1312/1312 - 53s - loss: 0.1143 - accuracy: 0.9668 - val_loss: 0.0664 - val_accuracy: 0.9830 - lr: 5.3134e-07 - 53s/epoch - 40ms/step\n",
      "Epoch 149/200\n",
      "1312/1312 - 53s - loss: 0.1144 - accuracy: 0.9663 - val_loss: 0.0660 - val_accuracy: 0.9830 - lr: 5.0477e-07 - 53s/epoch - 41ms/step\n",
      "Epoch 150/200\n",
      "1312/1312 - 53s - loss: 0.1165 - accuracy: 0.9661 - val_loss: 0.0662 - val_accuracy: 0.9829 - lr: 4.7953e-07 - 53s/epoch - 40ms/step\n",
      "Epoch 151/200\n",
      "1312/1312 - 55s - loss: 0.1132 - accuracy: 0.9671 - val_loss: 0.0666 - val_accuracy: 0.9828 - lr: 4.5555e-07 - 55s/epoch - 42ms/step\n",
      "Epoch 152/200\n",
      "1312/1312 - 53s - loss: 0.1135 - accuracy: 0.9668 - val_loss: 0.0663 - val_accuracy: 0.9829 - lr: 4.3278e-07 - 53s/epoch - 40ms/step\n",
      "Epoch 153/200\n",
      "1312/1312 - 53s - loss: 0.1140 - accuracy: 0.9669 - val_loss: 0.0665 - val_accuracy: 0.9830 - lr: 4.1114e-07 - 53s/epoch - 41ms/step\n",
      "Epoch 154/200\n",
      "1312/1312 - 53s - loss: 0.1140 - accuracy: 0.9669 - val_loss: 0.0666 - val_accuracy: 0.9828 - lr: 3.9058e-07 - 53s/epoch - 40ms/step\n",
      "Epoch 155/200\n",
      "1312/1312 - 53s - loss: 0.1154 - accuracy: 0.9664 - val_loss: 0.0663 - val_accuracy: 0.9828 - lr: 3.7105e-07 - 53s/epoch - 40ms/step\n",
      "Epoch 156/200\n",
      "1312/1312 - 55s - loss: 0.1136 - accuracy: 0.9665 - val_loss: 0.0663 - val_accuracy: 0.9830 - lr: 3.5250e-07 - 55s/epoch - 42ms/step\n",
      "Epoch 157/200\n",
      "1312/1312 - 53s - loss: 0.1138 - accuracy: 0.9664 - val_loss: 0.0664 - val_accuracy: 0.9830 - lr: 3.3487e-07 - 53s/epoch - 40ms/step\n",
      "Epoch 158/200\n",
      "1312/1312 - 53s - loss: 0.1153 - accuracy: 0.9663 - val_loss: 0.0662 - val_accuracy: 0.9829 - lr: 3.1813e-07 - 53s/epoch - 40ms/step\n",
      "Epoch 159/200\n",
      "1312/1312 - 55s - loss: 0.1153 - accuracy: 0.9669 - val_loss: 0.0663 - val_accuracy: 0.9829 - lr: 3.0222e-07 - 55s/epoch - 42ms/step\n",
      "Epoch 160/200\n",
      "1312/1312 - 57s - loss: 0.1141 - accuracy: 0.9664 - val_loss: 0.0661 - val_accuracy: 0.9829 - lr: 2.8711e-07 - 57s/epoch - 43ms/step\n",
      "Epoch 161/200\n",
      "1312/1312 - 52s - loss: 0.1150 - accuracy: 0.9661 - val_loss: 0.0660 - val_accuracy: 0.9830 - lr: 2.7276e-07 - 52s/epoch - 40ms/step\n",
      "Epoch 162/200\n",
      "1312/1312 - 53s - loss: 0.1127 - accuracy: 0.9668 - val_loss: 0.0661 - val_accuracy: 0.9829 - lr: 2.5912e-07 - 53s/epoch - 40ms/step\n",
      "Epoch 163/200\n",
      "1312/1312 - 56s - loss: 0.1134 - accuracy: 0.9667 - val_loss: 0.0666 - val_accuracy: 0.9828 - lr: 2.4616e-07 - 56s/epoch - 42ms/step\n",
      "Epoch 164/200\n",
      "1312/1312 - 56s - loss: 0.1145 - accuracy: 0.9662 - val_loss: 0.0662 - val_accuracy: 0.9830 - lr: 2.3386e-07 - 56s/epoch - 43ms/step\n",
      "Epoch 165/200\n",
      "1312/1312 - 53s - loss: 0.1121 - accuracy: 0.9671 - val_loss: 0.0661 - val_accuracy: 0.9830 - lr: 2.2216e-07 - 53s/epoch - 40ms/step\n",
      "Epoch 166/200\n",
      "1312/1312 - 69s - loss: 0.1130 - accuracy: 0.9666 - val_loss: 0.0662 - val_accuracy: 0.9830 - lr: 2.1105e-07 - 69s/epoch - 52ms/step\n",
      "Epoch 167/200\n",
      "1312/1312 - 57s - loss: 0.1153 - accuracy: 0.9658 - val_loss: 0.0660 - val_accuracy: 0.9830 - lr: 2.0050e-07 - 57s/epoch - 43ms/step\n",
      "Epoch 168/200\n",
      "1312/1312 - 57s - loss: 0.1136 - accuracy: 0.9670 - val_loss: 0.0661 - val_accuracy: 0.9829 - lr: 1.9048e-07 - 57s/epoch - 43ms/step\n",
      "Epoch 169/200\n",
      "1312/1312 - 55s - loss: 0.1145 - accuracy: 0.9664 - val_loss: 0.0662 - val_accuracy: 0.9830 - lr: 1.8095e-07 - 55s/epoch - 42ms/step\n",
      "Epoch 170/200\n",
      "1312/1312 - 55s - loss: 0.1155 - accuracy: 0.9666 - val_loss: 0.0660 - val_accuracy: 0.9830 - lr: 1.7191e-07 - 55s/epoch - 42ms/step\n",
      "Epoch 171/200\n",
      "1312/1312 - 56s - loss: 0.1156 - accuracy: 0.9664 - val_loss: 0.0663 - val_accuracy: 0.9830 - lr: 1.6331e-07 - 56s/epoch - 42ms/step\n",
      "Epoch 172/200\n",
      "1312/1312 - 55s - loss: 0.1132 - accuracy: 0.9669 - val_loss: 0.0661 - val_accuracy: 0.9830 - lr: 1.5514e-07 - 55s/epoch - 42ms/step\n",
      "Epoch 173/200\n",
      "1312/1312 - 53s - loss: 0.1132 - accuracy: 0.9672 - val_loss: 0.0663 - val_accuracy: 0.9829 - lr: 1.4739e-07 - 53s/epoch - 41ms/step\n",
      "Epoch 174/200\n",
      "1312/1312 - 53s - loss: 0.1170 - accuracy: 0.9657 - val_loss: 0.0662 - val_accuracy: 0.9829 - lr: 1.4002e-07 - 53s/epoch - 40ms/step\n",
      "Epoch 175/200\n",
      "1312/1312 - 51s - loss: 0.1135 - accuracy: 0.9666 - val_loss: 0.0665 - val_accuracy: 0.9828 - lr: 1.3302e-07 - 51s/epoch - 39ms/step\n",
      "Epoch 176/200\n",
      "1312/1312 - 55s - loss: 0.1152 - accuracy: 0.9659 - val_loss: 0.0664 - val_accuracy: 0.9829 - lr: 1.2637e-07 - 55s/epoch - 42ms/step\n",
      "Epoch 177/200\n",
      "1312/1312 - 52s - loss: 0.1158 - accuracy: 0.9660 - val_loss: 0.0664 - val_accuracy: 0.9828 - lr: 1.2005e-07 - 52s/epoch - 40ms/step\n",
      "Epoch 178/200\n",
      "1312/1312 - 53s - loss: 0.1148 - accuracy: 0.9663 - val_loss: 0.0663 - val_accuracy: 0.9829 - lr: 1.1405e-07 - 53s/epoch - 41ms/step\n",
      "Epoch 179/200\n",
      "1312/1312 - 55s - loss: 0.1154 - accuracy: 0.9664 - val_loss: 0.0661 - val_accuracy: 0.9830 - lr: 1.0834e-07 - 55s/epoch - 42ms/step\n",
      "Epoch 180/200\n",
      "1312/1312 - 52s - loss: 0.1167 - accuracy: 0.9662 - val_loss: 0.0662 - val_accuracy: 0.9829 - lr: 1.0293e-07 - 52s/epoch - 40ms/step\n",
      "Epoch 181/200\n",
      "1312/1312 - 55s - loss: 0.1166 - accuracy: 0.9665 - val_loss: 0.0662 - val_accuracy: 0.9830 - lr: 9.7780e-08 - 55s/epoch - 42ms/step\n",
      "Epoch 182/200\n",
      "1312/1312 - 53s - loss: 0.1156 - accuracy: 0.9660 - val_loss: 0.0660 - val_accuracy: 0.9830 - lr: 9.2891e-08 - 53s/epoch - 41ms/step\n",
      "Epoch 183/200\n",
      "1312/1312 - 52s - loss: 0.1131 - accuracy: 0.9667 - val_loss: 0.0662 - val_accuracy: 0.9830 - lr: 8.8246e-08 - 52s/epoch - 40ms/step\n",
      "Epoch 184/200\n",
      "1312/1312 - 52s - loss: 0.1141 - accuracy: 0.9666 - val_loss: 0.0661 - val_accuracy: 0.9829 - lr: 8.3834e-08 - 52s/epoch - 40ms/step\n",
      "Epoch 185/200\n",
      "1312/1312 - 55s - loss: 0.1147 - accuracy: 0.9666 - val_loss: 0.0661 - val_accuracy: 0.9828 - lr: 7.9642e-08 - 55s/epoch - 42ms/step\n",
      "Epoch 186/200\n",
      "1312/1312 - 53s - loss: 0.1149 - accuracy: 0.9663 - val_loss: 0.0662 - val_accuracy: 0.9829 - lr: 7.5660e-08 - 53s/epoch - 40ms/step\n",
      "Epoch 187/200\n",
      "1312/1312 - 53s - loss: 0.1149 - accuracy: 0.9666 - val_loss: 0.0660 - val_accuracy: 0.9830 - lr: 7.1877e-08 - 53s/epoch - 40ms/step\n",
      "Epoch 188/200\n",
      "1312/1312 - 52s - loss: 0.1152 - accuracy: 0.9663 - val_loss: 0.0661 - val_accuracy: 0.9830 - lr: 6.8283e-08 - 52s/epoch - 40ms/step\n",
      "Epoch 189/200\n",
      "1312/1312 - 54s - loss: 0.1143 - accuracy: 0.9665 - val_loss: 0.0662 - val_accuracy: 0.9830 - lr: 6.4869e-08 - 54s/epoch - 41ms/step\n",
      "Epoch 190/200\n",
      "1312/1312 - 53s - loss: 0.1147 - accuracy: 0.9663 - val_loss: 0.0665 - val_accuracy: 0.9829 - lr: 6.1626e-08 - 53s/epoch - 41ms/step\n",
      "Epoch 191/200\n",
      "1312/1312 - 52s - loss: 0.1119 - accuracy: 0.9676 - val_loss: 0.0663 - val_accuracy: 0.9830 - lr: 5.8544e-08 - 52s/epoch - 40ms/step\n",
      "Epoch 192/200\n",
      "1312/1312 - 55s - loss: 0.1151 - accuracy: 0.9667 - val_loss: 0.0663 - val_accuracy: 0.9829 - lr: 5.5617e-08 - 55s/epoch - 42ms/step\n",
      "Epoch 193/200\n",
      "1312/1312 - 54s - loss: 0.1148 - accuracy: 0.9662 - val_loss: 0.0663 - val_accuracy: 0.9829 - lr: 5.2836e-08 - 54s/epoch - 41ms/step\n",
      "Epoch 194/200\n",
      "1312/1312 - 53s - loss: 0.1137 - accuracy: 0.9668 - val_loss: 0.0662 - val_accuracy: 0.9831 - lr: 5.0194e-08 - 53s/epoch - 40ms/step\n",
      "Epoch 195/200\n",
      "1312/1312 - 55s - loss: 0.1134 - accuracy: 0.9665 - val_loss: 0.0660 - val_accuracy: 0.9830 - lr: 4.7685e-08 - 55s/epoch - 42ms/step\n",
      "Epoch 196/200\n",
      "1312/1312 - 52s - loss: 0.1137 - accuracy: 0.9666 - val_loss: 0.0666 - val_accuracy: 0.9828 - lr: 4.5301e-08 - 52s/epoch - 40ms/step\n",
      "Epoch 197/200\n",
      "1312/1312 - 54s - loss: 0.1157 - accuracy: 0.9659 - val_loss: 0.0662 - val_accuracy: 0.9829 - lr: 4.3035e-08 - 54s/epoch - 41ms/step\n",
      "Epoch 198/200\n",
      "1312/1312 - 54s - loss: 0.1123 - accuracy: 0.9672 - val_loss: 0.0662 - val_accuracy: 0.9830 - lr: 4.0884e-08 - 54s/epoch - 41ms/step\n",
      "Epoch 199/200\n",
      "1312/1312 - 55s - loss: 0.1114 - accuracy: 0.9671 - val_loss: 0.0663 - val_accuracy: 0.9830 - lr: 3.8840e-08 - 55s/epoch - 42ms/step\n",
      "Epoch 200/200\n",
      "1312/1312 - 53s - loss: 0.1156 - accuracy: 0.9663 - val_loss: 0.0661 - val_accuracy: 0.9830 - lr: 3.6898e-08 - 53s/epoch - 40ms/step\n"
     ]
    }
   ],
   "source": [
    "# set epochs and batch_size\n",
    "epochs = 200\n",
    "batch_size = 128\n",
    "\n",
    "# Fit the model\n",
    "history = model.fit(datagen.flow(X_train,y_train, batch_size=batch_size), \n",
    "                    epochs = epochs, validation_data = (X_val,y_val),\n",
    "                    verbose = 2, steps_per_epoch=X_train.shape[0] // batch_size,\n",
    "                    callbacks=[annealer],)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "id": "kKbptKTQomvy"
   },
   "outputs": [],
   "source": [
    "model.save(\"model.h5\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "id": "tUToFs2GopoD"
   },
   "outputs": [],
   "source": [
    "!cp model.h5 /content/drive/MyDrive/Model/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-03-24T22:38:04.696339Z",
     "iopub.status.busy": "2023-03-24T22:38:04.695555Z",
     "iopub.status.idle": "2023-03-24T22:38:04.702926Z",
     "shell.execute_reply": "2023-03-24T22:38:04.701711Z",
     "shell.execute_reply.started": "2023-03-24T22:38:04.696297Z"
    },
    "id": "b2uOPGD5-Ppj"
   },
   "outputs": [],
   "source": [
    "history_df = pd.DataFrame(history.history)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 423
    },
    "execution": {
     "iopub.execute_input": "2023-03-24T22:49:43.736789Z",
     "iopub.status.busy": "2023-03-24T22:49:43.736253Z",
     "iopub.status.idle": "2023-03-24T22:49:44.122833Z",
     "shell.execute_reply": "2023-03-24T22:49:44.121573Z",
     "shell.execute_reply.started": "2023-03-24T22:49:43.736712Z"
    },
    "id": "fZDYmtmp-Ppj",
    "outputId": "73f558b6-2f1c-4aee-839b-ef08379dcba1"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABHAAAAKTCAYAAABxdv/MAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB2ZUlEQVR4nO3dd3hUdd7+8XsmhVSSkEIvoSR0CIhACFJEWLEtAq6uyiMPqyu2XVcX3F0bIIIFV122WLCzq66g+6ioKyIoRUSqFOklJIQUAiEJaTPz+2N+M2RIIDNDcs4E3q/rOtdkZs7M+U45Seaez+d7LA6HwyEAAAAAAAAELKvZAwAAAAAAAMC5EeAAAAAAAAAEOAIcAAAAAACAAEeAAwAAAAAAEOAIcAAAAAAAAAIcAQ4AAAAAAECAI8ABAAAAAAAIcI0iwHE4HCouLpbD4TB7KAAAAAAAAIZrFAFOSUmJ+vfvr5KSErOHAgAAAAAAYLhGEeAAAAAAAABczAhwAAAAAAAAAhwBDgAAAAAAQIAjwAEAAAAAAAhwBDgAAAAAAAABLtjsAQAAAAAAEAhsNpsqKyvNHgYuMCEhIQoKCjrv+yHAAQAAAABc1BwOh3JycnT8+HGzh4ILVGxsrFq0aCGLxeL3fRDgAAAAAAAuaq7wJikpSREREef1IRuozuFwqLS0VLm5uZKkli1b+n1fBDgAAAAAgIuWzWZzhzfx8fFmDwcXoPDwcElSbm6ukpKS/G6nYhJjAAAAAMBFyzXnTUREhMkjwYXM9f46nzmWCHAAAAAAABc92qbQkOrj/UWAAwAAAAAAEOAIcAAAAAAAgCRp5MiReuONN7xef+3atUpNTVVRUVHDDQqSmMQYAAAAAIBGJzU19ZzX33PPPbr33nt9vt8PPvjAPemuN9LS0rRy5UpFR0f7vC1frF27VpMmTdK6devUtGnTBt1WoCLAAQAAAACgkVm5cqX75yVLlujFF1/U559/7r6s+qTMDodDNptNwcF1RwDNmjXzaRyhoaFKTEz06TbwDy1UAAAAAAA0MomJie4lOjpaFovFfX7fvn3q16+fVqxYoeuvv169evXS+vXrdejQIU2dOlXp6elKS0vT+PHjtXr1ao/7PbOFKjU1Vf/+97919913q0+fPho9erS++uor9/VntlAtXrxYl1xyib799ltdeeWVSktL05QpU5Sbm+u+TVVVlZ544gldcsklGjhwoJ555hlNnz5dd911l9/Px4kTJzRt2jQNGDBAffr00a9+9SsdOHDAfX1WVpbuvPNODRgwQH379tVVV12lFStWuG/7wAMPaNCgQerdu7dGjx6tRYsW+T2WhkIFDgAAAAAAZ3A4HCotLTV0mxEREfV6NKx58+Zp+vTpatu2rZo2baqcnBwNGzZM999/v0JDQ/XRRx/pzjvv1Oeff65WrVqd9X7mz5+v3//+95o2bZrefvttPfjgg/r6668VGxtb6/plZWV67bXX9PTTT8tqter3v/+9nnrqKc2bN0+S9Morr+jjjz/WnDlz1LFjR7311ltaunSpBg4c6Pdjfeihh3Tw4EH9/e9/V1RUlJ555hndcccd+vTTTxUSEqKZM2eqsrJS77zzjiIiIrRnzx53ldILL7ygvXv36pVXXlFcXJwOHTqksrIyv8fSUAhwAAAAAACoxuFwKCMjo0Z1SkMbMmSIvv3223oLce677z4NGTLEfT42NlZdu3Z1n//tb3+rpUuXatmyZbrlllvOej/jxo3T1VdfLUn63e9+p7fffltbtmzRZZddVuv6lZWVmjFjhtq1aydJuvnmm/W3v/3Nff0777yjO+64Q1dccYUk6dFHH9U333zj9+M8cOCAli1bpn/961/q16+fJOnZZ5/V8OHDtXTpUl155ZXKzs7WmDFj3HMHtW3b1n377OxsdevWTb169ZIktWnTxu+xNCQCHAAAAAAAzlCflTBmcQUSLiUlJZo/f76WL1+uvLw82Ww2lZWVKTs7+5z3U33C5IiICEVFRenYsWNnXT88PNwd3khSUlKSCgoKJEknT55Ufn6+evfu7b4+KChIPXr0kN1u9+nxuezdu1fBwcHq06eP+7K4uDglJydr7969kqRJkybp8ccf18qVK5Wenq7Ro0e7w6ybbrpJ9913n7Zv364hQ4Zo1KhR7iAokBDgAAAAAABQjcVi0bffftvoW6jOPJrUU089pdWrV2v69Olq166dwsLCdN9996mysvKc9xMSEuJx3mKxnDNsOXOyZIvFIofD4ePo69fEiROVkZGh5cuXa9WqVXr55Zc1ffp03XrrrRo2bJi+/vprrVixQqtWrdJtt92mm2++WdOnTzd1zGciwAEAAAAA4AwWi0WRkZFmD6Nebdy4UePGjXO3LpWUlCgrK8vQMURHRyshIUE//vijBgwYIEmy2Wzavn27R3uXLzp16qSqqipt3rzZXTlTWFio/fv3q3Pnzu71WrZsqZtuukk33XST5s2bp/fff1+33nqrJOfRt8aNG6dx48bp3Xff1dNPP02AAwAAAAAAjNe+fXt9+eWXGjlypCwWi55//nm/25bOxy233KKXXnpJ7dq1U8eOHfXOO+/oxIkTXlUf7dq1yyNYs1gs6tq1qy6//HI98sgjmjFjhqKiovTss8+qefPmuvzyyyVJs2fP1mWXXaYOHTqoqKhIa9euVadOnSQ5JzHu0aOHunTpooqKCi1fvtx9XSAhwAEAAAAA4CLw0EMP6Y9//KNuvPFGxcXF6fbbb1dJSYnh47j99tuVn5+v6dOnKygoSDfccIMyMjIUFBRU521vvvlmj/NBQUHavn275syZo9mzZ+vOO+9UZWWlLrnkEr388svu9i+73a6ZM2cqJydHUVFRGjp0qP7whz9IcraIPffcc8rKylJYWJj69++v5557rv4f+HmyOMxuRPNCcXGx+vfvr/Xr1ysqKsrs4QAAAAAALhBlZWXav3+/kpOTFRYWZvZwLkp2u11XXnmlrrzySv32t781ezgNoj7eZ1TgAAAAAAAAw2RlZWnVqlUaMGCAKioqtHDhQmVlZemaa64xe2gBjQAHAAAAAAAYxmq1avHixXrqqafkcDiUkpKi119/PSDnnQkkBDgGmTt3roqKivTkk0+aPRQAAAAAAEzTsmVLvfvuu2YPo9EhwDHIww8/LJvNpgceeEDx8fFmDwcAAAAAADQiVrMHcLFwzRVdUVFh8kgAAAAAAEBjQ4BjENfh0Gw2m8kjAQAAAAAAjQ0BjkGCg53dalVVVSaPBAAAAAAANDYEOAahAgcAAAAAAPiLAMcgBDgAAAAAAMBfBDgGoYUKAAAAABBobr31Vs2ePdt9fuTIkXrjjTfOeZvU1FQtXbr0vLddX/dzsSDAMQgVOAAAAACA+nLnnXdqypQptV73ww8/KDU1VT/99JPP9/vBBx/oF7/4xfkOz8Nf/vIXXXfddTUuX7lypS677LJ63daZFi9erEsuuaRBt2EUAhyDEOAAAAAAAOrLhAkTtHr1auXk5NS4btGiRerZs6e6du3q8/02a9ZM4eHh9THEOiUmJio0NNSQbV0ICHAM4mqhIsABAAAAAJyv4cOHq1mzZlq8eLHH5SUlJfr88881YcIEFRYW6ne/+52GDh2qPn366JprrtEnn3xyzvs9s4XqwIEDuvnmm9WrVy+NHTtWq1atqnGbZ555RmPGjFGfPn10+eWX6/nnn1dlZaUkZwXM/Pnz9dNPPyk1NVWpqanuMZ/ZQrVz505NmjRJvXv31sCBA/XII4+opKTEff1DDz2ku+66SwsWLFBGRoYGDhyoGTNmuLflj+zsbE2dOlVpaWnq16+ffvOb3yg/P999/U8//aRbb73Vff3111+vH3/8UZKUlZWlO++8UwMGDFDfvn111VVXacWKFX6PpS7BDXbP8OCqwGEOHAAAAABoBBwOqbTU2G1GREgWi1erBgcH67rrrtOHH36oqVOnyvL/b/f555/Lbrfr6quvVmlpqXr06KHbb79dUVFRWr58uaZNm6Z27dqpd+/edW7Dbrfr3nvvVXx8vP7973/r5MmTevLJJ2usFxkZqTlz5igpKUm7du3SI488osjISN1+++0aO3asdu/erW+//Vavv/66JCk6OrrGfZSWlmrKlClKS0vTBx98oIKCAj388MOaNWuW5s6d615v7dq1SkxM1JtvvqlDhw7p/vvvV7du3XTDDTd49byd+fjuuusuRURE6O2335bNZtOMGTN0//336+2335YkPfjgg+rWrZsef/xxBQUFaceOHQoJCZEkzZw5U5WVlXrnnXcUERGhPXv2KCIiwudxeIsAxyC0UAEAAABAI+FwSBkZ0urVxm53yBDp22+9DnHGjx+vBQsW6Pvvv9fAgQMlOSteRo8erejoaEVHR3vMk3Prrbdq5cqV+uyzz7wKcFavXq19+/bp1VdfVfPmzSVJ999/v26//XaP9e666y73z23atNH+/fv16aef6vbbb1dYWJgiIiIUFBSkxMTEs27rk08+UUVFhZ566il3CPLoo4/qzjvv1IMPPqiEhARJUkxMjB599FEFBQWpU6dOGjZsmNasWeNXgLNmzRrt2rVLX331lVq2bClJevrpp3XVVVdpy5Yt6t27t7KzszVlyhR16tRJktShQwf37bOzszVmzBilpqZKktq2bevzGHxBgGMQWqgAAAAAoBHxMkQxU6dOnZSWlqZFixZp4MCBOnjwoH744Qe99dZbkpyfP//xj3/o888/19GjR1VZWamKigqFhYV5df979+5VixYt3OGNJKWlpdVYb8mSJXrrrbeUmZmp0tJSVVVVKSoqyqfHsnfvXqWmpnpUsPTr1092u1379+93BzidO3d2F0hIznl0du3a5dO2qm+zRYsW7vDGdf9NmzbVvn371Lt3b02ePFkPP/yw/vOf/yg9PV0/+9nP1K5dO0nSpEmT9Pjjj2vlypVKT0/X6NGj/Zp3yFvMgWMQWqgAAAAAoJGwWJyVMMXFxi4+VN+4TJgwQf/9739VXFysxYsXq127drr00kslSQsWLNBbb72lX/3qV3rrrbf00UcfKSMj47zmjDnTxo0b9eCDD2rYsGH6xz/+oQ8//FB33nlnvW6jOldxhIvFYpHD4WiQbUnSvffeq08++UTDhw/Xd999p7Fjx+rLL7+UJE2cOFFLly7Vddddp127dmnChAnu1quGQIBjEFqoAAAAAKARsVikyEhjFz+qfq688kpZLBZ98skn+uijjzR+/Hj3fDgbNmzQ5Zdfruuuu05du3ZV27ZtdeDAAa/vu1OnTsrJyVFubq77sk2bNnmss3HjRrVq1UpTp05Vr1691KFDB2VnZ3usExISIrvdXue2du7cqdJq8w5t2LBBVqtVycnJXo/ZF67Hd+TIEfdle/bsUVFRkbtlSpKSk5N122236bXXXtPo0aO1aNEi93UtW7bUTTfdpPnz52vy5Ml6//33G2SsEgGOYWihAgAAAADUt8jISI0dO1bPPfec8vLyNG7cOPd17du31+rVq7Vhwwbt3btXjz76qMcRluqSnp6uDh066KGHHtJPP/2kH374QX/+85891mnfvr2OHDmiTz/9VIcOHdJbb73lcWQpSWrdurUOHz6sHTt26NixY6qoqKixrWuuuUahoaF66KGHtGvXLn333XeaNWuWrrvuOnf7lL9sNpt27Njhsezdu1fp6elKSUnRgw8+qG3btmnLli2aNm2aLr30UvXq1UtlZWWaOXOm1q5dq6ysLK1fv14//vijO9yZPXu2vv32W2VmZmrbtm1au3atR/BT35gDxyC0UAEAAAAAGsKECRP0wQcfaNiwYR7z1UydOlWZmZmaMmWKwsPDdcMNN2jUqFE6efKkV/drtVo1f/58/elPf9KECRPUunVrPfzww/rVr37lXufyyy/X//zP/2jmzJmqqKjQ8OHDNXXqVM2fP9+9zpgxY/Tll19q0qRJKioq0pw5c3T99dd7bCs8PFwLFizQ7NmzNWHCBIWHh2v06NF66KGHzvPZcR7h6uc//7nHZe3atdOXX36pv/3tb5o1a5ZuueUWWSwWDR06VI888oj78R8/flzTp09Xfn6+4uLiNHr0aN13332SnEexmjlzpnJychQVFaWhQ4fqD3/4w3mP92wsjoZsFqsnxcXF6t+/v9avX+/zREiBYtCgQVq7dq3+85//6NprrzV7OAAAAAAASWVlZdq/f7+Sk5O9ntwX8FV9vM9ooTIILVQAAAAAAMBfBDgGoYUKAAAAAAD4iwDHIByFCgAAAAAA+IsAxyC0UAEAAAAAAH8R4BiEFioAAAAACFyN4Pg+aMTq4/1FgGMQWqgAAAAAIPCEhIRIch5qGmgorveX6/3mj+D6GgzOjQAHAAAAAAJPUFCQYmNjlZubK0mKiIiQxWIxeVS4UDgcDpWWlio3N1exsbHubMAfBDgGcc2BQwsVAAAAAASWFi1aSJI7xAHqW2xsrPt95i8CHINQgQMAAAAAgclisahly5ZKSkpSZWWl2cPBBSYkJOS8Km9cCHAMQoADAAAAAIEtKCioXj5oAw2BSYwNQgsVAAAAAADwFwGOQajAAQAAAAAA/iLAMQgBDgAAAAAA8BcBjkFooQIAAAAAAP4iwDEIFTgAAAAAAMBfBDgGIcABAAAAAAD+IsAxiKuFigAHAAAAAAD4igDHIK4KHObAAQAAAAAAviLAMQgtVAAAAAAAwF8EOAahhQoAAAAAAPiLAMcgtFABAAAAAAB/EeAYhBYqAAAAAADgLwIcg9BCBQAAAAAA/OVzgLNu3TrdeeedysjIUGpqqpYuXer1bdevX6/u3bvruuuu83WzjR4tVAAAAAAAwF8+BzilpaVKTU3VY4895tPtioqKNH36dA0ePNjXTV4QaKECAAAAAAD+Cvb1BsOGDdOwYcN83tBjjz2mq6++WkFBQT5V7VwoaKECAAAAAAD+MmQOnEWLFikzM1P33HOPEZsLSLRQAQAAAAAAf/lcgeOrAwcOaN68eVq4cKG7CuViRAsVAAAAAADwV4NW4NhsNj3wwAO69957lZyc3JCbCngEOAAAAAAAwF8NWhJTUlKirVu3aseOHZo1a5YkyW63y+FwqHv37lqwYMFFM6mxq/qIFioAAAAAAOCrBg1woqKi9PHHH3tc9s9//lPfffedXnzxRbVp06YhNx9QqMABAAAAAAD+8jnAKSkp0aFDh9znDx8+rB07digmJkatWrXSvHnzdPToUT399NOyWq1KSUnxuH18fLyaNGlS4/ILHQEOAAAAAADwl88BztatWzVp0iT3+Tlz5kiSxo0bp7lz5yovL09HjhypvxFeIGihAgAAAAAA/rI4HA6H2YOoS3Fxsfr376/169crKirK7OH45b333tONN96o4cOH6+uvvzZ7OAAAAAAAoBFp0KNQ4TRaqAAAAAAAgL8IcAxCCxUAAAAAAPAXAY5BqMABAAAAAAD+IsAxCAEOAAAAAADwFwGOQVwtVAQ4AAAAAADAVwQ4BnFV4DAHDgAAAAAA8BUBjkFooQIAAAAAAP4iwDEILVQAAAAAAMBfBDgGoYUKAAAAAAD4iwDHILRQAQAAAAAAfxHgGIQWKgAAAAAA4C8CHIPQQgUAAAAAAPxFgGMQWqgAAAAAAIC/CHAMQgsVAAAAAADwFwGOQWihAgAAAAAA/iLAMQgtVAAAAAAAwF8EOAYhwAEAAAAAAP4iwDGIaw4cWqgAAAAAAICvCHAMQgUOAAAAAADwFwGOQVwBjsPhkN1uN3k0AAAAAACgMSHAMYirhUqiCgcAAAAAAPiGAMcgrgociQAHAAAAAAD4hgDHIAQ4AAAAAADAXwQ4BqneQsWRqAAAAAAAgC8IcAxCBQ4AAAAAAPAXAY5BCHAAAAAAAIC/CHAMYrFYZLU6n24CHAAAAAAA4AsCHAO5qnCYAwcAAAAAAPiCAMdArgCHChwAAAAAAOALAhwDuY5ERYADAAAAAAB8QYBjIFqoAAAAAACAPwhwDEQLFQAAAAAA8AcBjoFooQIAAAAAAP4gwDEQLVQAAAAAAMAfBDgGooUKAAAAAAD4gwDHQLRQAQAAAAAAfxDgGIgWKgAAAAAA4A8CHAPRQgUAAAAAAPxBgGMgAhwAAAAAAOAPAhwDuebAoYUKAAAAAAD4ggDHQFTgAAAAAAAAfxDgGIgABwAAAAAA+IMAx0C0UAEAAAAAAH8Q4BiIChwAAAAAAOAPAhwDEeAAAAAAAAB/EOAYiBYqAAAAAADgDwIcA1GBAwAAAAAA/EGAYyACHAAAAAAA4A8CHAPRQgUAAAAAAPxBgGMgKnAAAAAAAIA/CHAMRIADAAAAAAD8QYBjIFcLFQEOAAAAAADwBQGOgVwVOMyBAwAAAAAAfEGAYyBaqAAAAAAAgD8IcAxECxUAAAAAAPAHAY6BaKECAAAAAAD+IMAxEC1UAAAAAADAHwQ4BqKFCgAAAAAA+IMAx0C0UAEAAAAAAH8Q4BiIFioAAAAAAOAPAhwDEeAAAAAAAAB/EOAYyDUHDi1UAAAAAADAFwQ4BqICBwAAAAAA+IMAx0AEOAAAAAAAwB8EOAaihQoAAAAAAPiDAMdAVOAAAAAAAAB/+BzgrFu3TnfeeacyMjKUmpqqpUuXnnP9//73v5o8ebIGDRqkfv366Re/+IW+/fZbvwfcmBHgAAAAAAAAf/gc4JSWlio1NVWPPfaYV+uvW7dO6enpevnll7V48WINHDhQU6dO1fbt230ebGNHCxUAAAAAAPBHsK83GDZsmIYNG+b1+n/60588zv/ud7/TV199pWXLlql79+6+br5RowIHAAAAAAD4w/A5cOx2u0pKShQbG2v0pk1HgAMAAAAAAPxheICzYMEClZaW6sorrzR606ajhQoAAAAAAPjD5xaq8/Hxxx/rr3/9q/72t78pPj7eyE0HBCpwAAAAAACAPwwLcD799FM9/PDDeuGFF5Senm7UZgMKAQ4AAAAAAPCHIS1Un3zyif7whz9o3rx5Gj58uBGbDEiuFioCHAAAAAAA4AufK3BKSkp06NAh9/nDhw9rx44diomJUatWrTRv3jwdPXpUTz/9tCRn29RDDz2kP/7xj+rTp4/y8vIkSWFhYYqOjq6nh9E4uCpwmAMHAAAAAAD4wucAZ+vWrZo0aZL7/Jw5cyRJ48aN09y5c5WXl6cjR464r3///fdVVVWlmTNnaubMme7LXetfTGihAgAAAAAA/vA5wBk4cKB27tx51uvPDGXefvtt30d1gaKFCgAAAAAA+MPww4hfzGihAgAAAAAA/iDAMRAtVAAAAAAAwB8EOAaihQoAAAAAAPiDAMdAtFABAAAAAAB/EOAYiBYqAAAAAADgDwIcAxHgAAAAAAAAfxDgGMg1Bw4tVAAAAAAAwBcEOAaiAgcAAAAAAPiDAMdABDgAAAAAAMAfBDgGooUKAAAAAAD4gwDHQFTgAAAAAAAAfxDgGIgABwAAAAAA+IMAx0C0UAEAAAAAAH8Q4BiIChwAAAAAAOAPAhwDEeAAAAAAAAB/EOAYiBYqAAAAAADgDwIcA1GBAwAAAAAA/EGAYyACHAAAAAAA4A8CHAPRQgUAAAAAAPxBgGMgVwWOw+GQw+EweTQAAAAAAKCxIMAxkCvAkWijAgAAAAAA3iPAMZCrhUoiwAEAAAAAAN4jwDFQ9Qoc5sEBAAAAAADeIsAxEC1UAAAAAADAHwQ4BqKFCgAAAAAA+IMAx0C0UAEAAAAAAH8Q4BjIYrHIYrFIogIHAAAAAAB4jwDHYK4qHAIcAAAAAADgLQIcg7nmwaGFCgAAAAAAeIsAx2BU4AAAAAAAAF8R4BiMAAcAAAAAAPiKAMdgtFABAAAAAABfEeAYjAocAAAAAADgKwIcgxHgAAAAAAAAXxHgGIwWKgAAAAAA4CsCHINRgQMAAAAAAHxFgGMwAhwAAAAAAOArAhyD0UIFAAAAAAB8RYBjMCpwAAAAAACArwhwDEaAAwAAAAAAfEWAYzBaqAAAAAAAgK8IcAxGBQ4AAAAAAPAVAY7BCHAAAAAAAICvCHAM5mqhIsABAAAAAADeIsAxmKsChzlwAAAAAACAtwhwDEYLFQAAAAAA8BUBjsFooQIAAAAAAL4iwDEYLVQAAAAAAMBXBDgGo4UKAAAAAAD4igDHYAQ4AAAAAADAVwQ4BnPNgUMLFQAAAAAA8BYBjsGowAEAAAAAAL4iwDEYAQ4AAAAAAPAVAY7BaKECAAAAAAC+IsAxGBU4AAAAAADAVwQ4BiPAAQAAAAAAviLAMRgtVAAAAAAAwFcEOAajAgcAAAAAAPiKAMdgBDgAAAAAAMBXBDgGo4UKAAAAAAD4igDHYFTgAAAAAAAAXxHgGIwABwAAAAAA+IoAx2C0UAEAAAAAAF8R4BiMChwAAAAAAOArAhyDEeAAAAAAAABfEeAYzNVCRYADAAAAAAC85XOAs27dOt15553KyMhQamqqli5dWudt1q5dq3Hjxqlnz5664oortHjxYr8GeyFwVeAwBw4AAAAAAPCWzwFOaWmpUlNT9dhjj3m1fmZmpn79619r4MCB+s9//qP/+Z//0cMPP6xvv/3W58FeCGihAgAAAAAAvgr29QbDhg3TsGHDvF7/3XffVZs2bfTQQw9Jkjp16qT169frjTfe0NChQ33dfKNHCxUAAAAAAPBVg8+Bs2nTJg0ePNjjsoyMDG3atKmhNx2QaKECAAAAAAC+avAAJz8/XwkJCR6XJSQkqLi4WGVlZQ29+YBDCxUAAAAAAPAVR6EyGAEOAAAAAADwVYMHOAkJCcrPz/e4LD8/X1FRUQoLC2vozQcc1xw4tFABAAAAAABvNXiA07dvX3333Xcel61evVp9+/Zt6E0HJCpwAAAAAACAr3wOcEpKSrRjxw7t2LFDknT48GHt2LFD2dnZkqR58+Zp2rRp7vVvvPFGZWZm6umnn9bevXu1cOFCffbZZ7rtttvq5xE0MgQ4AAAAAADAVz4fRnzr1q2aNGmS+/ycOXMkSePGjdPcuXOVl5enI0eOuK9v27atXnrpJc2ZM0dvvfWWWrRooSeeeOKiPIS4RAsVAAAAAADwnc8BzsCBA7Vz586zXj937txab/PRRx/5uqkLEhU4AAAAAADAVxyFymAEOAAAAAAAwFcEOAajhQoAAAAAAPiKAMdgVOAAAAAAAABfEeAYjAAHAAAAAAD4igDHYLRQAQAAAAAAXxHgGIwKHAAAAAAA4CsCHIMR4AAAAAAAAF8R4BiMFioAAAAAAOArAhyDUYEDAAAAAAB8RYBjMAIcAAAAAADgKwIcg9FCBQAAAAAAfEWAYzAqcAAAAAAAgK8IcAxGgAMAAAAAAHxFgGMwVwsVAQ4AAAAAAPAWAY7BXBU4zIEDAAAAAAC8RYBjMFqoAAAAAACArwhwDEaAAwAAAAAAfEWAYzAOIw4AAAAAAHxFgGMwKnAAAAAAAICvCHAM5gpw7Ha7HA6HyaMBAAAAAACNAQGOwVwtVBJVOAAAAAAAwDsEOAZzVeBIBDgAAAAAAMA7BDgGI8ABAAAAAAC+IsAxWPUWKo5EBQAAAAAAvEGAYzAqcAAAAAAAgK8IcAxGgAMAAAAAAHxFgGMwq9Uqi8UiiRYqAAAAAADgHQIcE7iqcKjAAQAAAAAA3iDAMQEBDgAAAAAA8AUBjglcR6KihQoAAAAAAHiDAMcEVOAAAAAAAABfEOCYgAAHAAAAAAD4ggDHBLRQAQAAAAAAXxDgmIAKHAAAAAAA4AsCHBMQ4AAAAAAAAF8Q4JjA1UJFgAMAAAAAALxBgGMCVwUOc+AAAAAAAABvEOCYgBYqAAAAAADgCwIcExDgAAAAAAAAXxDgmIDDiAMAAAAAAF8Q4JiAChwAAAAAAOALAhwTEOAAAAAAAABfEOCYgBYqAAAAAADgCwIcE1CBAwAAAAAAfEGAYwICHAAAAAAA4AsCHBPQQgUAAAAAAHxBgGMCKnAAAAAAAIAvCHBMQIADAAAAAAB8QYBjAlqoAAAAAACALwhwTEAFDgAAAAAA8AUBjgkIcAAAAAAAgC8IcExACxUAAAAAAPAFAY4JqMABAAAAAAC+IMAxAQEOAAAAAADwBQGOCWihAgAAAAAAviDAMQEVOAAAAAAAwBcEOCYgwAEAAAAAAL4gwDGBq4WKAAcAAAAAAHiDAMcErgoc5sABAAAAAADeIMAxAS1UAAAAAADAFwQ4JiDAAQAAAAAAviDAMQGHEQcAAAAAAL4gwDEBFTgAAAAAAMAXBDgmIMABAAAAAAC+IMAxAS1UAAAAAADAFwQ4JqACBwAAAAAA+IIAxwQEOAAAAAAAwBd+BTgLFy7UyJEj1atXL02cOFFbtmw55/pvvPGGxowZo969e2vYsGF68sknVV5e7teALwS0UAEAAAAAAF/4HOAsWbJEc+bM0d13360PP/xQXbt21ZQpU1RQUFDr+h9//LHmzZune+65R0uWLNHs2bO1ZMkSPffcc+c9+MaKChwAAAAAAOALnwOc119/XTfccIPGjx+vzp07a8aMGQoLC9OiRYtqXX/jxo3q16+frrnmGrVp00YZGRm6+uqr66zauZAR4AAAAAAAAF/4FOBUVFRo27ZtSk9PP30HVqvS09O1cePGWm+Tlpambdu2uQObzMxMrVixQsOGDTuPYTdutFABAAAAAABfBPuycmFhoWw2m+Lj4z0uj4+P1759+2q9zTXXXKPCwkL98pe/lMPhUFVVlW688Ubdeeed/o+6kaMCBwAAAAAA+KLBj0K1du1avfTSS3rssce0ePFizZ8/XytWrNBf//rXht50wCLAAQAAAAAAvvCpAicuLk5BQUE1JiwuKChQQkJCrbd54YUXdO2112rixImSpNTUVJWWlurRRx/V1KlTZbVefEcyp4UKAAAAAAD4wqf0JDQ0VD169NCaNWvcl9ntdq1Zs0ZpaWm13qasrKxGSOOqQHE4HL6O94JABQ4AAAAAAPCFTxU4kjR58mRNnz5dPXv2VO/evfXmm2/q1KlTuv766yVJ06ZNU/PmzfXAAw9IkkaMGKHXX39d3bt3V+/evXXo0CG98MILGjFihDvIuNgQ4AAAAAAAAF/4HOCMHTtWx44d04svvqi8vDx169ZNr776qruF6siRIx4VN1OnTpXFYtHzzz+vo0ePqlmzZhoxYoTuv//++nsUjQwtVAAAAAAAwBcWRyPoYyouLlb//v21fv16RUVFmT2c8/bvf/9bN9xwgy677DKtWLHC7OEAAAAAAIAAd/HNIBwAaKECAAAAAAC+IMAxAS1UAAAAAADAFwQ4JqACBwAAAAAA+IIAxwQEOAAAAAAAwBcEOCYgwAEAAAAAAL4gwDEBc+AAAAAAAABfEOCYgAocAAAAAADgCwIcExDgAAAAAAAAXxDgmIAWKgAAAAAA4AsCHBNQgQMAAAAAAHxBgGMCAhwAAAAAAOALAhwT0EIFAAAAAAB8QYBjAipwAAAAAACALwhwTECAAwAAAAAAfEGAYwJaqAAAAAAAgC8IcExABQ4AAAAAAPAFAY4JCHAAAAAAAIAvCHBMQAsVAAAAAADwBQGOCVwVOHa7XQ6Hw+TRAAAAAACAQEeAYwJXgCM5QxwAAAAAAIBzIcAxgauFSqKNCgAAAAAA1I0AxwTVK3CYyBgAAAAAANSFAMcEBDgAAAAAAMAXBDgmoIUKAAAAAAD4ggDHBFTgAAAAAAAAXxDgmMBqPf20E+AAAAAAAIC6EOCYxFWFQ4ADAAAAAADqQoBjEtc8OMyBAwAAAAAA6kKAYxIqcAAAAAAAgLcIcExCgAMAAAAAALxFgGMSWqgAAAAAAIC3CHBMQgUOAAAAAADwFgGOSQhwAAAAAACAtwhwTEILFQAAAAAA8BYBjkmowAEAAAAAAN4iwDEJAQ4AAAAAAPAWAY5JaKECAAAAAADeIsAxCRU4AAAAAADAWwQ4JiHAAQAAAAAA3iLAMQktVAAAAAAAwFsEOCahAgcAAAAAAHiLAMckBDgAAAAAAMBbBDgmoYUKAAAAAAB4iwDHJFTgAAAAAAAAbxHgmIQABwAAAAAAeIsAxyS0UAEAAAAAAG8R4JiEChwAAAAAAOAtAhyTEOAAAAAAAABvEeCYxBXg0EIFAAAAAADqQoBjEtccOFTgAAAAAACAuhDgmIQWKgAAAAAA4C0CHJMQ4AAAAAAAAG8R4JiEw4gDAAAAAABvEeCYhAocAAAAAADgLQIckxDgAAAAAAAAbxHgmIQWKgAAAAAA4C0CHJNQgQMAAAAAALxFgGMSAhwAAAAAAOAtAhyT0EIFAAAAAAC8RYBjEipwAAAAAACAtwhwTEKAAwAAAAAAvEWAYxJaqAAAAAAAgLcIcExCBQ4AAAAAAPAWAY5JCHAAAAAAAIC3CHBMQgsVAAAAAADwFgGOSajAAQAAAAAA3vIrwFm4cKFGjhypXr16aeLEidqyZcs51y8qKtKMGTOUkZGhnj17asyYMVqxYoVfA75QEOAAAAAAAABvBft6gyVLlmjOnDmaMWOG+vTpozfffFNTpkzR559/rvj4+BrrV1RUaPLkyYqPj9cLL7yg5s2bKzs7W02bNq2XB9BY0UIFAAAAAAC85XOA8/rrr+uGG27Q+PHjJUkzZszQ8uXLtWjRIt1xxx011l+0aJFOnDihd999VyEhIZKkNm3anOewGz8qcAAAAAAAgLd8aqGqqKjQtm3blJ6efvoOrFalp6dr48aNtd5m2bJl6tu3r2bOnKn09HRdffXV+sc//nHRBxcEOAAAAAAAwFs+BTiFhYWy2Ww1WqXi4+OVn59f620yMzP1xRdfyGaz6eWXX9Zdd92l119/XX//+9/9H/UFwBXg0EIFAAAAAADq4nMLla8cDofi4+M1a9YsBQUFqWfPnjp69KgWLFige+65p6E3H7Bcc+BQgQMAAAAAAOriU4ATFxenoKAgFRQUeFxeUFCghISEWm+TmJio4OBgd8WJJHXs2FF5eXmqqKhQaGioH8Nu/GihAgAAAAAA3vKphSo0NFQ9evTQmjVr3JfZ7XatWbNGaWlptd6mX79+OnTokOx2u/uyAwcOKDEx8aINbyQCHAAAAAAA4D2fAhxJmjx5st5//319+OGH2rt3rx5//HGdOnVK119/vSRp2rRpmjdvnnv9m266ScePH9fs2bO1f/9+LV++XC+99JJuvvnm+nsUjRCHEQcAAAAAAN7yeQ6csWPH6tixY3rxxReVl5enbt266dVXX3W3UB05ckRW6+lcqGXLllqwYIHmzJmja6+9Vs2bN9ekSZN0++2319+jaISowAEAAAAAAN7yaxLjW265Rbfcckut17399ts1LktLS9P777/vz6YuWAQ4AAAAAADAWz63UKF+0EIFAAAAAAC8RYBjEipwAAAAAACAtwhwTEKAAwAAAAAAvEWAYxJaqAAAAAAAgLcIcExCBQ4AAAAAAPAWAY5JCHAAAAAAAIC3CHBMQgsVAAAAAADwFgGOSajAAQAAAAAA3iLAMQkBDgAAAAAA8BYBjklooQIAAAAAAN4iwDEJFTgAAAAAAMBbBDgmIcABAAAAAADeIsAxCS1UAAAAAADAWwQ4JqECBwAAAAAAeIsAxyQEOAAAAAAAwFsEOCZxBTi0UAEAAAAAgLoQ4JjENQcOFTgAAAAAAKAuBDgmoYUKAAAAAAB4iwDHJNUDHIfDYfJoAAAAAABAICPAMYmrhUqS7Ha7iSMBAAAAAACBjgDHJK4KHIk2KgAAAAAAcG4EOCapXoFTWVlp4kgAAAAAAECgI8AxSWRkpEJDQyVJBQUFJo8GAAAAAAAEMgIck1gsFiUlJUmSjh49avJoAAAAAABAICPAMVHz5s0lEeAAAAAAAIBzI8AxERU4AAAAAADAGwQ4JqICBwAAAAAAeIMAx0SuACc3N9fkkQAAAAAAgEBGgGMiKnAAAAAAAIA3CHBMRIADAAAAAAC8QYBjIgIcAAAAAADgDQIcExHgAAAAAAAAbxDgmMgV4BQUFKiystLk0QAAAAAAgEBFgGOi+Ph4Wa3OlyAvL8/k0QAAAAAAgEBFgGMiq9WqxMRESbRRAQAAAACAsyPAMZmrjSo3N9fkkQAAAAAAgEBFgGMyJjIGAAAAAAB1IcAxGQEOAAAAAACoCwGOyQhwAAAAAABAXQhwTEaAAwAAAAAA6kKAYzICHAAAAAAAUBcCHJMR4AAAAAAAgLoQ4JiMAAcAAAAAANSFAMdkSUlJkqS8vDzZ7XaTRwMAAAAAAAIRAY7JEhMTJUl2u10FBQUmjwYAAAAAAAQiAhyThYSEKD4+XhJtVAAAAAAAoHYEOAGAeXAAAAAAAMC5EOAEAAIcAAAAAABwLgQ4AYAABwAAAAAAnAsBTgAgwAEAAAAAAOdCgBMACHAAAAAAAMC5EOAEAFeAk5uba/JIAAAAAABAICLACQBJSUmSqMABAAAAAAC1I8AJALRQAQAAAACAcyHACQDVW6gcDofJowEAAAAAAIGGACcAuFqoKioqdPz4cXMHAwAAAAAAAg4BTgAICwtTTEyMJNqoAAAAAABATQQ4AYJ5cAAAAAAAwNkQ4AQIAhwAAAAAAHA2BDgBovpExgAAAAAAANUR4AQIKnAAAAAAAMDZEOAECAIcAAAAAABwNgQ4AcJ1KHECHAAAAAAAcCYCnABBBQ4AAAAAADgbApwAQYADAAAAAADOhgAnQFQPcBwOh8mjAQAAAAAAgcSvAGfhwoUaOXKkevXqpYkTJ2rLli1e3e7TTz9Vamqq7rrrLn82e0FzBTinTp1ScXGxyaMBAAAAAACBxOcAZ8mSJZozZ47uvvtuffjhh+rataumTJmigoKCc97u8OHDeuqpp3TJJZf4PdgLWWRkpCIjIyVJubm5Jo8GAAAAAAAEEp8DnNdff1033HCDxo8fr86dO2vGjBkKCwvTokWLznobm82mBx98UPfee6/atm17XgO+kDEPDgAAAAAAqI1PAU5FRYW2bdum9PT003dgtSo9PV0bN2486+3++te/Kj4+XhMnTvR/pBcBAhwAAAAAAFCbYF9WLiwslM1mU3x8vMfl8fHx2rdvX623+eGHH/TBBx/oo48+8nuQFwsCHAAAAAAAUJsGPQpVcXGxpk2bplmzZqlZs2YNuakLQlJSkiQCHAAAAAAA4MmnCpy4uDgFBQXVmLC4oKBACQkJNdbPzMxUVlaWpk6d6r7MbrdLkrp3767PP/9c7dq182fcFyQqcAAAAAAAQG18CnBCQ0PVo0cPrVmzRqNGjZLkDGTWrFmjW265pcb6HTt21Mcff+xx2fPPP6+SkhL96U9/UosWLc5j6BceAhwAAAAAAFAbnwIcSZo8ebKmT5+unj17qnfv3nrzzTd16tQpXX/99ZKkadOmqXnz5nrggQfUpEkTpaSkeNy+adOmklTjchDgAAAAAACA2vkc4IwdO1bHjh3Tiy++qLy8PHXr1k2vvvqqu4XqyJEjslobdGqdC5YrwMnNzTV5JAAAAAAAIJBYHA6Hw+xB1KW4uFj9+/fX+vXrFRUVZfZwGsyuXbuUmpqqpk2b6sSJE2YPBwAAAAAABAhKZQJImzZtZLVaVVRUpJycHLOHAwAAAAAAAgQBTgCJiIhQt27dJEk//PCDyaMBAAAAAACBggAnwFxyySWSpHXr1pk8EgAAAAAAECgIcALMgAEDJFGBAwAAAAAATiPACTCuCpwffvhBjWB+aQAAAAAAYAACnADTp08fBQcHKzc3V5mZmWYPBwAAAAAABAACHKOcOuVc6hAWFqZevXpJoo0KAAAAAAA4EeAYZdAgKSVFKi+vc1UmMgYAAAAAANUR4Bhl3z7p8GHp0KE6V60+Dw4AAAAAAAABjlFatHCe5uTUuWr1I1ExkTEAAAAAACDAMUrLls7TI0fqXLVnz55q0qSJjh8/rr179zbwwAAAAAAAQKAjwDGKDxU4ISEh6tu3ryTaqAAAAAAAAAGOcXwIcCTmwQEAAAAAAKcR4BjFhxYqiSNRAQAAAACA0whwjOJjBY5rIuMNGzbIZrM11KgAAAAAAEAjQIBjFB8DnK5duyoiIkLFxcXauXNnAw4MAAAAAAAEOgIco7gCHC9bqIKCgtSvXz9JzIMDAAAAAMDFjgDHKK45cPLyJC9bolxtVAQ4AAAAAABc3AhwjJKYKFmtkt3uDHG8wETGAAAAAABAIsAxTlCQM8SRfJ7IeNOmTaqsrGyokQEAAAAAgABHgGMkHw8l3qlTJ8XExKisrEzbtm1rwIEBAAAAAIBARoBjJB+PRGW1WtW/f39JzIMDAAAAAMDFjADHSD4GOBITGQMAAAAAAAIcY/nYQiUxkTEAAAAAACDAMZYfFTiXXnqpJGnz5s3Kzc1tiFEBAAAAAIAAR4BjJD8CnHbt2ql///6y2WxatGhRAw0MAAAAAAAEMgIcI/kR4EjSTTfdJEn617/+Vd8jAgAAAAAAjQABjpH8mANHkm644QZJ0rfffqvDhw/X96gAAAAAAECAI8AxkqsCp7jYuXipbdu2Gjp0qCTpvffea4iRAQAAAACAAEaAY6SoKCkiwvnz0aM+3fTGG2+UJL377rv1PSoAAAAAABDgCHCMZLH43UY1ceJEBQUF6YcfftDu3bsbYHAAAAAAACBQEeAYzc+JjBMTEzVq1ChJVOEAAAAAAHCxIcAxmp8BjnS6jepf//qXHA5HfY4KAAAAAAAEMAIco7kCHB9bqCRp3LhxCg0N1Y4dO/Tjjz/W88AAAAAAAECgIsAxmmsOHD8qcGJiYjR27FhJtFEBAAAAAHAxIcAx2nm0UEnSTTfdJMkZ4NBGBQAAAADAxYEAx2jnGeBcffXVioyM1P79+/X999/X48AAAAAAAECgIsAxmp+HEXeJiIjQddddJ8k5mTEAAAAAALjwEeAYzVWBk5sr2Wx+3YWrjWrhwoUqKiqqr5EBAAAAAIAARYBjtMREyWJxhjcFBX7dxZgxY5SSkqL8/HzNmTOnngcIAAAAAAACDQGO0UJCpIQE589+tlGFhITo2WeflST9+c9/1oEDB+ppcAAAAAAAIBAR4JjhPA4l7nL11Vdr5MiRKi8v10MPPVRPAwMAAAAAAIGIAMcM53kkKkmyWCyaN2+eLBaL3nvvPa1Zs6aeBgcAAAAAAAINAY4Z6iHAkaS+ffvqf//3fyVJ999/v+x2+/mODAAAAAAABCACHDOc56HEq5s1a5YiIyO1du1avffee+d9fwAAAAAAIPAQ4JihnipwJKlly5buOXCmT5+uU6dOnfd9AgAAAACAwEKAY4Z6DHAk6YEHHlDbtm2VmZmp5557rl7uEwAAAAAABA4CHDPUYwuVJIWHh2vOnDmSpNmzZ2vPnj31cr8AAAAAACAwEOCYoZ4rcCTpl7/8pUaOHKlTp05pypQpTGgMAAAAAMAFhADHDK4Ap6hIKi2tl7u0WCx65ZVXFBERoW+++Ub/+Mc/6uV+AQAAAACA+QhwzNC0qRQW5vy5HqtwOnbsqLlz50qSpk2bpgMHDtTbfQMAAAAAAPMQ4JjBYjk9D049BjiSdPfddysjI0MlJSW6/fbb5XA46vX+AQAAAACA8QhwzNIA8+BIktVq1WuvvaawsDAtXbpUCxYsqNf7BwAAAAAAxiPAMUsDBTiS1KVLFz3xxBOSnIcYP3z4cL1vAwAAAAAAGIcAxyz1fCjxM/32t7/VoEGDVFRUpFtvvVXl5eUNsh0AAAAAANDwCHDM0oAVOJIUFBSk1157TZGRkVq+fLluvfVW2Wy2BtkWAAAAAABoWAQ4ZmngAEeSunXrpo8++kghISH697//rbvvvptJjQEAAAAAaIQIcMziCnAaqIXKZdSoUfrnP/8pi8Wil156SY888kiDbg8AAAAAANQ/AhyzNNBhxGszYcIE/eMf/5AkzZ49W3/+858bfJsAAAAAAKD+EOCYxVWBc/SoVFHR4Ju74447NHv2bEnS7373O82ePVvFxcUNvl0AAAAAAHD+CHDM0qKFlJQkVVVJzzxjyCb/8Ic/6P7775ckPfzww2rXrp0eeeQR5ebmGrJ9AAAAAADgHwIcswQHS/PmOX+eNUvaubPBN2mxWPTss8/q1VdfVZcuXVRYWKgnnnhC7du311133aWsrKwGHwMAAAAAAPAdAY6Zbr5ZGjNGKi+X7rhDstsbfJNWq1VTpkzRjh07tGjRIl166aUqKyvT3//+dw0ePFh79+5t8DEAAAAAAADfEOCYyWKR/v53KSJC+uYbacECwzYdFBSk66+/Xt99952WL1+url27KjMzUyNGjCDEAQAAAAAgwBDgmC052dlCJUm//32DH1b8TBaLRcOGDdPXX3/tDnGGDx9OiAMAAAAAQAAhwAkE990nXXKJdOKEdO+9pgyhRYsW+vrrr9WtWzcdPnyYEAcAAAAAgADiV4CzcOFCjRw5Ur169dLEiRO1ZcuWs677/vvv65e//KUGDBigAQMG6Lbbbjvn+hel4GDp1VeloCBp0SLpo49MGUaLFi20bNkyjxBnpwGTKwMAAAAAgHPzOcBZsmSJ5syZo7vvvlsffvihunbtqilTpqigoKDW9deuXaurrrpKb731lt599121bNlS//u//6ujR4+e9+AvKH36OFuoJGdFjgETGtfmzBAnLS1Nf/7zn2Wz2UwZDwAAAAAAkCwOh8Phyw0mTpyoXr166dFHH5Uk2e12DRs2TLfeeqvuuOOOOm9vs9k0YMAAPfroo/r5z3/u1TaLi4vVv39/rV+/XlFRUb4Mt3E5dUpKSpKKi6WNG6W+fU0bytGjR3XTTTfp66+/liQNGjRIr732mrp162bamAAAAAAAuFj5VIFTUVGhbdu2KT09/fQdWK1KT0/Xxo0bvbqPU6dOqaqqSjExMb6N9GIQHi5lZDh/Xr7c1KE0b95cS5cu1UsvvaTo6Gh999136tu3r2bPnq3KykpTxwYAAAAAwMXGpwCnsLBQNptN8fHxHpfHx8crPz/fq/t49tlnlZSU5BECoZoRI5yn/7/yxUxWq1V33HGHtm3bprFjx6qiokIPP/ywunXrpoULF9JWBQAAAACAQQw9CtXLL7+sJUuWaP78+WrSpImRm248hg93nq5YIQVIQNK2bVt98sknevvtt5WUlKS9e/fqlltuUd++ffXRRx/Jxy48AAAAAADgI58CnLi4OAUFBdWYsLigoEAJCQnnvO2CBQv08ssva8GCBeratavvI71Y9OsnRUc7Dym+ebPZo3GzWCy65ZZbtHfvXs2ePVuxsbHaunWrxo0bp4EDB+rLL78kyAEAAAAAoIH4FOCEhoaqR48eWrNmjfsyu92uNWvWKC0t7ay3e+WVV/S3v/1Nr776qnr16uX/aC8GwcHSZZc5fw6ANqozRUVF6Y9//KP279+vP/3pT4qMjNS6des0evRojRgxQqtWrTJ7iAAAAAAAXHB8bqGaPHmy3n//fX344Yfau3evHn/8cZ06dUrXX3+9JGnatGmaN2+ee/2XX35ZL7zwgp588km1bt1aeXl5ysvLU0lJSf09iguNq40qAAMcl9jYWD3xxBPat2+ffvvb3yo0NFQrVqxQRkaGrrrqKq8ntQYAAAAAAHXz+TDikvTOO+9owYIFysvLU7du3fTwww+rT58+kqRbb71VrVu31ty5cyVJI0eOVFZWVo37uOeee3Tvvfd6tb2L5jDiLuvXS5dcIjVtKhUUOKtyAlxmZqZmzZql1157zT258aBBgzR+/HiNHz9eycnJJo8QAAAAAIDGy68Ax2gXXYBjs0nx8c55cL7/XhowwOwReW337t16/PHH9a9//ctjTpy0tDR3mMMcSAAAAAAA+MbQo1DBS0FBAT0Pzrl06dJFCxcu1OHDhzV//nyNGDFCVqtVGzdudB+CvEePHnr00Ue1efNmJj4GAAAAAMALBDiBasQI5+ny5aYOw1+tWrXS3XffrWXLliknJ0evvPKKfvaznykkJETbt2/XrFmz1LdvX6WkpOiFF15gTiQAAAAAAM6BFqpAtWmTlJYmRUVJx45JISFmj6heHD9+XB9//LEWLVqkL774QmVlZZKkZs2a6b777tM999yj+Ph4k0cJAAAAAEBgIcAJVHa7lJAgFRZKa9ZIgwaZPaJ6V1xcrH/+8596+umntXfvXklSRESEbrvtNvXo0UOJiYnupWXLlmrWrJnJIwYAAAAAwBwEOIFs3Djpo4+kOXOkhx4yezQNxmaz6YMPPtDcuXO1adOms67XqVMnDR48WOnp6Ro8eLB69uyp4EZwhC4AAAAAAM4XAU4ge/FF6Te/kUaPlr74wuzRNDiHw6EvvvhC//nPf3T06FHl5eW5l2PHjtVYPzY2Vr/+9a/1m9/8Ri1btjRhxAAAAAAAGIMAJ5Bt2SL16SNFREjHj18w8+D448SJE1q7dq1Wr16t1atXa+3atSoqKpIkhYaG6rbbbtODDz6oLl26mDxSAAAAAADqHwFOILPbpaQkqaBAWrVKSk83e0QBw2az6dNPP9VTTz2l1atXS5IsFouuvfZajR49WkOGDFHPnj0VFBRk8kgBAAAAADh/BDiBbsIEadEi6YknpD/9yezRBKSVK1dq7ty5+vTTTz0ub9q0qQYPHqyBAwcqJSVFKSkp6tKli2JjY80ZKAAAAAAAfiLACXTz50v33iuNGiV9+aXZowloW7du1aJFi7Rq1SqtWbNGxcXFta6XkJCgLl261FhSUlIuvvcXAAAAAKBRIMAJdNu3Sz16OOe/2bpVSkkxe0SNQlVVlX788UetWrVKGzdu1J49e7R7924dOXLkrLexWq3q06eP0tPTNWTIEA0ZMkRt27aVxWIxcOQAAAAAANREgNMYjB0rffaZdOWV0qefSgQKfisuLnaHOWcuubm5NdaPjo5WdHS0oqKiFBUVpcjISHXo0EE/+9nPNGbMGMXHx5vwKAAAAAAAFxsCnMZg1y6pZ0+pslL6v/+TrrnG7BFdkLKzs7Vq1Sr3snHjRtlstrOub7VaNXDgQI0dO1bp6elq06aNWrVqdXG+RwEAAAAADYoAp7F46CHpqaekjh2lbduksDCzR3TBKy0t1eHDh1VSUqLi4mKVlJTo5MmT2rBhg5YsWaItW7bUervo6Gi1bt1aycnJ6t27t3r16qXevXsrNTVVoaGhBj8KAAAAAMCFgACnsSgullJTpexsjkgVIA4fPqzPPvtMS5Ys0fbt25WVlaWSkpKzrh8SEqKOHTsqOTnZY+natau6du2q4OBgA0cPAAAAAGhMCHAak3/9S/rlL6WICOmnn6S2bc0eEc5w8uRJZWdnKysrSzt37tSPP/6oLVu26Mcff1RRUdFZbxcWFqZevXopLS1Nffv2lc1m0759+7R3717t27dPBw8eVJs2bXTZZZdp2LBhGjZsmFq1amXgIwMAAAAAmIkApzFxOKThw6VvvpFuuEF67z2zRwQvORwOZWZmas+ePdq/f7/279+vAwcOaN++fdq6datOnjzp83127txZycnJio6OVtOmTRUdHa2YmBilpKSof//+Sk1NVVBQUAM8GgAAAACA0QhwGpvNm6V+/SS7XVq2TBoxwuwR4TzZ7Xbt27dPGzdu1IYNG7RlyxaFhYUpOTlZHTt2VMeOHdW2bVvt3r1b33zzjVasWKFNmzbJbref834jIiLUt29f9e/fX927d1eXLl3UpUsXtWnTRlarVZJks9mUm5ur7Oxs5efnKzExUe3atVN8fDyHTwcAAACAAEKA0xjdc4/0179KvXpJmzZJ///DOC4eJ06c0Nq1a5Wbm6uioiKdPHlSRUVFOnbsmLZu3aqNGzeedT6eJk2aqH379jp58qSOHj1aaxAUERGhdu3a1Vjat2+vli1bymaz6dSpUyorK9OpU6dUVVWl+Ph4NW/eXElJSQpjkm0AAAAAqFcEOI3RsWNSp07S8ePONqobbjB7RAgwNptNu3bt0vr167Vhwwbt2rVLu3fv1r59+1RVVeWxrtVqVfPmzZWQkKC8vDzl5OSc9/abNm2qpKQkxcXFKS4uTrGxsYqLi1NiYqJSUlLcEzdHR0ef97aqO378uMrKytSiRYt6vV8AAAAAMBsBTmM1c6b02GNS9+7Sli0Sc53AC1VVVTp48KAOHjyomJgYtWrVSklJSR5z5ZSXlyszM1OHDh3yWA4ePKhDhw4pJydHISEhCgsLU3h4uMLCwhQUFKT8/Hzl5uaqsrLS6/G0adNGHTt2VGhoqIKDgxUUFKSgoCCFh4erffv2Hkfrat68uWw2m6qqqlRZWamqqiplZWVp3bp17mXXrl3u+x08eLAGDRqkwYMHKy0tjaogAAAAAI0aAU5jdeKE1KGDswrnX/+Sbryx/rexfbvUubMUGlr/940LksPh0PHjx5Wbm6vc3FwVFhbq+PHjKiwsVGFhoXJycvTTTz9px44dys3NbZAxWK3WGm1hVqtV7dq1U5cuXZSSkqIuXbooMTFRNpvNHQpVVVUpPDxcSUlJSkpKUvPmzZWYmCiLxaJjx455LAkJCerXr5/XoZDD4dCxY8d06NAhtWrVSs2bN2+Ihw4AAADgAkaA05jNmiU9+qjUtau0dWv9VuG8/LL0619Ll18uff65FBxcf/cNSDp27Jh27NihzMxMVVVVucMUm82mkydP6sCBAzpw4ID7qF3V5/QJCQlRcHCwYmNj1b9/f1166aUaMGCALrnkEoWFhemHH37QmjVr3Et+fn69jz8kJER9+/bVoEGDNGjQIMXExKigoEDHjh1TQUGBCgoKlJmZ6X4cxcXF7tt26dJFGRkZ7qVdu3Zq0qSJx8TRVVVV2rt3r7Zv365t27Zp79696tSpky677DJdeumldYZHDodDe/fu1ddff63NmzcrOTlZ/fv3V1pammJiYur9+QAAAADQsAhwGrOiImcVTmGhtHCh9Mtf1s/9Hjwo9ewpuT5w3n+/9Nxz9XPfgB8cDofKy8sVEhIiq9Xq0xGyHA6HcnNz3fMAuZbCwkKFhIQoKCjI3b5VWlqq3NxcHT16VLm5ue75giwWi+Li4tSsWTPFxsbq0KFDflUQJSYmKj8/X7X92g0KClJUVJQiIyMVFhamw4cPq6Kiotb7CQ0N1cCBA5WRkaHExESFhIS4Q63KykqtXr1aX3/9tQ4fPlzr7Tt37qyePXuqsrJSRUVFOnHihIqKilRaWqqIiAhFRUW5l5iYGHXq1Empqanq2rWrUlNTFR8f7/Njr66qqkoFBQXKy8tTXl6eTp065bHN6OhohYSEqLKyUpWVlaqoqFBlZaVKS0vd1Vyu6q6ysjJ3653ValVQUJDi4+PVtWtXdevWzbCwqqqqSuXl5e5qLlcoGRERoaZNm7qP/OYvh8OhI0eOKCQkRFFRUQoLC+NIcQAAABcZApzGbvZs6eGHpdRUadu2uqtwSkulN9+UMjKcR7E6k8MhXXml9MUXUseO0r59zsvfflu65Zb6Hz8QoBwOhwoLC+VwOBQbG+sxT5DD4dDBgwf13Xff6bvvvtP333+vyspKNWvWTM2aNVN8fLyaNWum1q1bq0OHDurQoYPat2+vsLAwHT9+XKtXr9bKlSu1cuVKff/99yovL691DBEREerWrZt69Oihjh07avv27VqxYoWOHj3q1WMICQnRoEGDNGDAAO3fv18bNmzQwYMHz/u5iY6OltVqlcPhcC9Wq9U9L1JERITCw8MVEhKiiooKVVRUqLy8XBUVFSouLtaxY8fOewzeatmypbp166a2bdsqJiZGTZs2dZ8mJCSoRYsW7iUiIkJ2u135+fnKyspyL2VlZbJare7w0GKxKCcnx10hduDAAWVlZdV6RDfJGQBGR0crJiZGsbGxHqcxMTGKi4tTu3bt1LlzZ3Xu3Flt2rSR1WpVVlaWvvrqKy1dulRfffWVsrOz3fdZPfBr0qSJQkNDFRISotDQUIWHh6tjx45KTU11Lx06dJDNZlNZWZnKy8tVVlYmu92upKQkNWvWzCMMcjgcyszM1I8//qgtW7YoKytLERERioyMdG8zPj5e3bp1U0pKikLPaLM9ePCgvv32W3377bfKz8/3mM/KtS+cawLzsrIyHTp0SHl5eSoqKvJYqgd/eXl5ys/Pl9VqVXx8vOLj45WQkKD4+HhFRUW5nw/X0rp1a/Xo0UPt27f3K1ArLy93v+aZmZlKTExUly5d1KlTJ6/bKSsrK5WTk6OIiIgav1fsdruys7PdIXNeXp47qD3b/TscDp04ccIdHrpCz7CwMLVp06ZeQr6qqioVFxe7g2fXaURExFnHdeLECa1Zs0arVq1SZWWlevbsqd69e6tr167u94vdbtehQ4e0c+dO7dq1S3FxcRo1atQ5J6GvqKhQYWFhjfdFdHS0WrVqpVatWik6OvqiCDcPHTqkb7/9VitXrtShQ4cUHx+vxMREJSUlKTExUW3btlX//v3VrFkzw8aUn5+vkydPuve56vtgUFCQKa9LaWmpfvrpJ23fvl0HDhxQjx49NHz4cMXFxdX7tqqqqlRSUqKmTZv6/FjtdrtHpbHr9q7nLxBVVla6vzzxlsPhcP8fUFlZKZvNJrvd7v772b59e58+6+3atUtLlizRDz/8oF69eumKK65Q3759zzqmyspKFRcXq6SkxL24/n+Lj49XbGxsjdva7XYVFxfr+PHjOnLkiMdSWFgoSe7/DSTna1b971F8fLxat26t5OTk8/4yx1sOh0M2m820/c4fx48fV1ZWljp37qwmTZr4dFubzaaioqIG2a8DEQFOY1dUJCUnO49MVVfIsmGDdPPN0k8/SZGR0iefSMOHe67zxhvS5MlSkybS5s3O+5w9WwoLk1atkvr1a8hHA1x0bDabiouL3f9QFBcXq7S0VK1bt671g6bD4dDu3bv1zTff6Pvvv3f/8+FaHA6H0tLSNGLECA0ePFgREREet8/Pz9fGjRu1c+dOhYeHewQaEREROnXqlHs8xcXFys/P1+7du7Vz507t3LlTmZmZ9fK4LRaLmjVrpsTEREVERLgf+8mTJ1VcXCy73S6LxVIjmHAd2cx1dLPw8HCP9jubzaYjR45ox44dOnLkiE9jio6OdgdN9SEoKEg2m83n2zVp0kSJiYk1Kqhqm9+pPoSEhLhDrJCQEG3btk0nTpzw6rbBwcFKSUlRjx49FBIS4v4wWZfIyEi1bNlSLVu2VKtWreRwOHTgwAEdPHjQ64DSX5GRkerevbt69OihoKAgj9bHY8eOyWKxqEmTJmrSpInCwsIUEhKiI0eOKCsrq9bqOYvF4g7gEhIS1LRpU/cSFhamzMxM7dmzR3v27NGBAwc8jgTYtGlTxcXFKTw8XAcPHtSpU6dq3H94eLiGDx+uMWPGaMCAAdqzZ482bdqkzZs3a9OmTWcNRNu2bavLL7/cvTRv3lzbt2/XqlWrtHr1aq1atUqZmZlq1qyZ+4NGQkKCQkJCdPToUfdSUFBw1ueyRYsW6tChgzucKyoq0sqVK7Vly5Zan6vg4GB169ZNFotFu3btUllZWY11evfurdGjR+uKK65QUFCQNm3a5H68O3bsqHEkxTNFRkaqVatWateunZKTk9WxY0f3+CwWi8eHtzM/yLnOl5WVeYSd5eXl7t9BrqA6NDRU5eXlKikpUWlpqUpLS1VRUVEjGI6NjdWxY8c8ntPjx48rJCTE433mCjlcYfHZlry8PK/3M8nZsjtw4EBdeuml6t27txISEtxfNjRp0kQ2m80jSNu1a5eys7N18uRJFRUVuU/DwsLUsWNH95KcnKzS0lJt2bLFHfbWte9W/30eGxurFi1aqHnz5u7ThIQEj9/xMTExys7Odv/92blzp/bs2SOLxeJRtRkZGSmLxSK73e4OBMrLy9373JnvRYvFon79+mnkyJG67LLLFBER4a70PNtpbZeVlJQoOztbWVlZOnz4sHJycmS32xUbG+sO5F2hvM1m8/hbffLkSWVmZroPGnH48OGzHgQiNjZWzZs3V/PmzZWUlKTw8HD3/IKu09DQUHdQ2qtXL/Xq1UvNmzev8ToeO3bMPU+ha4mMjFTv3r3Vp08f9e7d2yP8tdlsOnHihAoLC7Vz505t2bJFmzdv1pYtW7Rz504FBQWpffv27vdEcnKyHA6Hu5X8zNZyV3BzNhaLRV26dFFaWpr69u2rPn36KDIy0uMLo5MnT+qrr77SkiVLtGfPnhr3kZCQoCuuuEJDhgxRQUGB9u7d617qOtKq6wuB6OholZSU6OTJkyotLT3nbbwVHR2ttLQ09evXT/369VObNm08npuCggIdP35cJ0+edP8vdPLkSZWVldWYasA1f+PZzrve88HBwe5wyrVYrVb377hTp06prKxMbdq0cU8JMGDAADVt2lQ2m01btmzR8uXLtXz5cq1atUplZWXuUNH15VFsbKwSExM9loiICPfvtyZNmigkJMRdqexaSkpKtGfPHve+7apsDw8P19ChQ3X55Zdr1KhRZw3kqqqq9M033+iDDz7Q4sWLdfToUX311VcaOXJkvbxegYwA50IwZ470xz9KXbo4Jx4+c74am0165hnpkUekqipnlY7NJoWHS//5j3TFFc71srOlHj2cEyM/9ZQ0bZpzvWuvlZYskdq1k374QUpMNPwhAggMJSUlysrKkuT5jZPdbnf/M+BaKisrPf7Ih4aGKiIiQomJiYqPj/eoPqjO4XDIbref9XpvnThxwj1p9tGjR92tYq62sby8POXk5OjIkSMeHyQtFouSkpLUunVrtW7dWhEREe4x2e12ORwOJSQkuD8UJicnq3379mratKmCg4PdLXmSs2rjxIkTOnHihI4fP17raUFBgQ4cOKA9e/Zo//797n9uLRaLLrnkEo0aNUqXX3650tPT1aRJkxphV/UPFRUVFTp58qRH6LZz506PQCYsLMxdOXH8+PFan7vg4GB17dpVvXv3VnJyssrKyjxCxpycHG3btk0nT56scdugoCD169dPQ4cOVbt27XTo0CF3pdL+/fvPus3qIiMj1aJFC3e46Fri4uLc/yAmJCQoMTFRdrvd/c9vfn6+CgoKVFpa6vGclJeXa//+/frpp5/OK6CLjIxUx44d1aZNG+Xm5mr37t0qKiry6T7OFewFBQUpOTlZXbp0UUxMjFasWOF1EOlqowwJCVFpaWmNoCMiIqLePoh4o1OnTsrIyFBERIR+/PFH/fjjjzWCwdDQUHXu3FkpKSnKzMzU+vXrvbrv6Ohod/gcFRWloqIiZWdn+/xaNGau/SwjI0PdunVTYWGh8vLylJubq7y8PO3evbvWD7fVRUZGuj9U1Zfw8HD3kSIDRUJCgrp37662bdtq/fr1+umnn8weUkCLi4tT06ZN3dVuDSE0NFRhYWEe4aTNZnNXtHgrJCREl112mYYMGaJNmzZp2bJlHnMOnk1QUJAiIyMVGRmpkJAQFRYW1vr37Mwxt2jRwv3lQ8uWLd0VrNUDpvLyco9QpqCgQIcOHao1sA5EFotFXbt2VXZ2ttdf5tSXyMhIj0o0yRlgtm7d2iOEqqys1Keffuoxx2ViYqKWLVumnj17GjpmMxDgXAhOnnRW4RQUOCc2vuIKKSbGuZSUSFOmSN9841z3+uulF1+U7rjDGco0aSItXuxsmxo3zhnoXHKJtGbN6SDo+HHp0kul3bulESOkV15xzrtz7JhzqayUrr5aukjK1gBcWBwOh4qKipSTk6OwsDC1bNnStJJ1m82mzMxMZWVlqXv37vVSDuxwOFRSUuL+9rt6OXV5ebmOHj2qnJwc5eTkqLS0VN27d/dodTnX/R4+fFhbt27Vtm3bdOrUKQ0ePFiDBg0659/qkpISd/l5dna2jhw5IofD4W6vat++fY22rvpSVVWlPXv2aOvWrdqxY4f721bXN5RxcXGyWCzuqgvX0rx5cyUnJ7uPTFf9OXB9WN67d6+OHz/u0dpTUlKi1q1bu7+J79Kli1q2bKmqqiqPb9BLSkrUtm1bdejQQSEhIR73v3XrVn3xxRf64osvtG3bNqWkpKhPnz7q27ev+vbtq27dutWYBL20tFQrV67UV199pa+++kobNmyQw+FQZGSkBg4cqCFDhig9PV2pqak6ceKE8vPz3UtlZaX7SHyuJTY21iOwtVgsOn78eI3J5kNDQzVkyBANGTJELVu29HjuXa15W7ZskcVicbf2BVf70ikvL09fffWV/vvf/2rZsmUKDg52fwvvOnW1GNbG9d7KysryGNe+fft06NAhWa1W94c2V+VGbeddlTauJTQ0VBUVFe5vq0+dOqXy8nKFhYUpMjJSERERioiIUEhIiPLz893BcE5OjgoLC9WsWTP3c9miRQvFxcV5fBtdVlamiooKd0h8riU8PNz9TXld/xMXFBRo3bp1+v7777V27Vrt3r1bx44dU2FhoUc1X2hoqLp06aLU1FSlpKS4W05dQZmrGmHfvn0eS2hoqLvio3fv3urRo4d7THa7XVVVVe4QtXqgWlFR4a5KysnJcZ+6xlZ9nrPExESPOdi6dOmioKAgj0pR1wf26vOhBQcHKzk5Wd26dVPiGV88Zmdna9myZVq2bJnWrVsnh8PhUR1U/bS2y1ynYWFhatWqlVq3bq02bdqodevWio6O1sGDB91Vd3v27FF2drY7XHXdNiIiQm3btvVYqv9+cX08c83NV31+vlOnTrmrUF2nxcXF7qD0xx9/1NatW1VcXKzw8HCP1zEuLs5dyeNqtyssLNTmzZu1efNm/fTTT7UGzBEREerQoYO7Ssf1uttsNvd+5lqCg4M9fq+6lurnw8PDa/0df/ToUXfV3caNG7Vt2zZVVla6f/9YLBYFBwdr0KBBGjt2rEaOHOnRkltZWanvvvtO//3vf7Vhwwa1aNFCnTp1UqdOndSxY0d16NBBMTExtf59Ky8v17Fjx9ytgJGRkYqOjnYvZ/6e9UVVVZV++uknbdiwQRs2bND69euVn59f4zmKi4vz2GZ0dLTCwsLcXwydudR1eWlpaY1KKIfD4a4kdFWZ7t692z0twIEDB9zjjo6O1tChQzV8+HANGzZMCQkJHvtxWVmZ+zmr3trs+h3pqmquqKhQcHCwR1VOWFiYOnTo4G71TklJUXR0tLZt2+ZuH1+xYsU5g7WEhAT9/Oc/14QJEzRixIiAbTesbwQ4F4qnnpIeeujs10dGOoObyZMli0UqL3ceevyjj6SQEOlXv5L+/nfnz+vX15wfZ/t2aeDA0xMbnyk52dmS1b17vT0kAABw/o4dO6YjR44oNTXVIzDBxclut+vEiRM6duyYrFar2rVrd94VjwgsrsDP1/29vLxcO3bsUHl5uUdAdLF8MIaUk5OjDRs2KDExUWlpaab+zaisrNSOHTuUm5vrEUSVlZW5WyAvxr9pBDgXitJSZzizY4d04oRzKSpyTko8ZIhzbpvOnT1vU1npnDPn/fdPXzZjhvPQ5LX5v/9zhj5Wq9Ss2ellzx4pM9NZ8fPvf59uyQIAAAAAAPWCAOdCZrdLp045q2/OpqrKGfy8847Uu7e0bp10rpTd4XBW8FSXn+9sv1q50jm/zvz50p131s9jAAAAAAAAMuZYZjCH1Xru8EZyznPzxhvSl18658mpq0Sytt7PhARp6VLp1ludkx5PnSrdf7/zZwAAAAAAcN4IcOCsmhk1ytkC5a8mTaQ335SeeMJ5/vnnna1U2dn1MkQAAAAAAC5mBDioPxaL9Kc/OefUiYyUvv5a6tNH+vRTs0cGAAAAAECjRoCD+jdxovNIVn37OufHufpq6Xe/cx75CgAAAAAA+IxJjNFwysuladOchy+XnNU4Awc6f3Y4nEuTJs6jZF1xhXMuHSPs3eucsyc3V8rLO71YLM4jdaWkSF26OE87dnTOEwQAAAAAgIkIcNDw/u//nEe6Onbs7OtYLNKAAdLPfiaNGSP17+8Md+qTzSb9+c/Sww97Xw3Upo20eLFzbAAAAAAAmIQAB8bIypL++U+prMx53mJxLgUFzmqYH3/0XD80VEpLc1bsDBwo9evnrNCJiZFCQpzrOBzS/v3SmjWnl6IiacIEacoUZzWNy9690m23OQ91LkmDB0vdu0uJic4lKcl5SPXdu53Lrl3O5dQpKTxcWrjQeah0AAAAAABMQICDwJCVJX3xhfT559KyZc5g52wiIqTYWKmiwjnHztmMGCH96lfSiRPS738vlZRIUVHOKpwpU2o/JHp1J09Kv/iF9NlnznWfflp64IG6bwcAAAAAQD0jwEHgcTikffuktWudy/ffS1u3SsXFNdcNCXFW5wwa5KyqCQqSXnvNGQSd+dYeNkx64w2pQwfvx1JVJf3mN9Lf/uY8/+tfS/PnO+87M9M5zv37nSFRUJDn0ratc36f2FjfHn9WlmS1Si1aEBYBAAAAACQR4KAxqapyBiUnTkjHj0t2u9SzpxQWVnPdQ4ek1193hjl5edKTT0r33ecMRnzlcEgvvOA8kpbD4WzlKix0zqlTF4vFeTSuYcOcS79+UuvWzoCnuu3bpUWLnMvmzc7LoqKcbWBdujiXvn2dgVCrVr4/Bl/Y7c7qpspK52Ns2tS/5w0AAAAAUG8IcHBhc4URtYU8vvrPf6Rf/lIqLXWeb9JESk52HqmqWTNn2GGzObdZVSVt2+acT+dMQUHOyZHbt3eebtgg/fST5/UOh/N+apOc7AxyhgyRIiOd8/64luJiqXlzqWtXKTXVGfyEhTnbxzZskNatcy5btjjXraw8HdZUVDiXM7cbHOy8z5YtnVVBrVs7q50uv9w5fiOUlDgfH1VJAAAAAC5SBDiAL3JznZMbJyc7A426KlOOHJFWrHAu33zjDHQqK2uuFxrqPJT6+PHStddK0dHO1izXpMo//eRsJ9uypWZr2LlYLM5x5uScPRA6H126OIOcIUOcgVZlpedSVeV53mp1jqd1a2f407q1s8KnvPx0CHXihJSd7axEci179jgfd1yc1Lu385D0ffo4t9+kifP5CwlxnkrO+ysrO71UVdUcu83mvK68/PQSGuqssIqPd566Js7mUPIAAAAATEaAAxjJZnOGKQcPOpfMTOdcOWPHOoOCuhQVSd995zya1vffO0OZmBhnCNK0qXOC56wsZ+Czc6ez1cyldWvn4dAvucR5mPb4+NOhx9lOLRbnRNFHjjjHnZPjPKLX119LP/xQP6FQUJB37WhWa8OEUN4ICnIejSw83Pkcu36uvthszqOWlZWdPm3a1DOsat3aGUQdP+5cXC2BpaWeQdKZwVJZmTMAa99e6tXr9NKtm7OS6vDh08uRI86xuI705lokz/MOh/P95BpLYaFzHDExzoqy+HjnEhvrGVRWr4Cq62eLxRl+hYQ4T4ODna9hSYlzKS52noaEOB9Lz55Sjx6e80ZVVkpHjzofl6t1snq1mzenNpvzNQwN9Vwcjpqho2vMwcHO27jG7KpUqx5Guu4nJMS5VFQ4H09p6enT0FDneyYy0rlERJx+/l2L5Lx9WJgzkGzSxPlzdPTpfbtJk9O3Ky4+/ZoVFTkvt1rPvbjew64xhIeffp1cFX+u58z12Ku/ppWVzondi4qcp+Xlp/cH19KkyeltVb/vqirnc1NefrrSz/Wz69T1GlVfXM9d9f3uzPbT+mCzOZ9T1+JweL7+rt9RZwbTkvPxuvap6j+fef7M/eLM07rWqcuZ721XJafrta2+uF7nsy21/VtYVeX5O6m83PP3zLmeB9ftXfui67UOC3Mu4eHOU6vVc79wjeXMn8913bnWc72urn3M9VzZbDXfj7W9jt6ennmZVPNLhYqK09t3jSckxLvXq7bXOzjY+/dKfa5XWen8e1d9qf6+cN3P+S7Vx3O2y1zPpev3qOu1de231auNq59WVTmfv+rPZ0jI6ef+zPdu9fMOx+n/l6p/mVT9PXe2pfrfGtffkDN/757rtfDl9SktPf36OByn9znXqevorrXd99l+ruu66n/favv5XNfV9bNre9X/9lksnn+jKyo8v7w72/8oF9Jl3q5vs9XcD1zXV38+a/v5zKWu/z+q/z48der0/38lJc7zQUGn9x3X78Fz/T05299Y1++Ai+QLVwIc4ELlcDjn/9m/X2rXzln5Up9OnHBWFi1b5mzPslhO/xNS/R+S6pfZbM7qmqws51JYePr+LJbTH1jj450BhavSpk8fZ7CwY4e0adPpypzMzJr/kDkcpz8YuJbafqFbrZ4fmJs0OX1kM9dSVFS/zxm807q1s/rpyBHnezjw/0w1vJAQZ/hSXFx7RZk/mjQ5/YGkNq792OFwfuj0VfUP5PXF9YG4rg963l7n+lBzobFY2G8AABeP6Gjpyy+lgQPNHkmDI8ABYJ7SUmclQXS088NpoE2WXFHh/MBc/dvF6t9kVV+qV+m4gqMTJ05XxrhCq6AgZ4VJbKwzlIqJcVYXnFl9Uf3UVdmwZ4/044/OZcsWZxVXeLizwse1tGzp3TeAFoszLIuLOz2eiAjnmAsKpGPHnKcnTtT85svbn13zQbkW17c8UVGnl8hI53O6bZvzaHOZmTVfh6Ag5+OKi3P+7KryONvPtV3mqqKp/k179dDRtUinx+v69jYoqOZ6drtneFhZ6fz2x1XhEhnpfG0qK2tW5bief+n0aWWlZ/VVWZnzvXfyZO3vzeBg5/PhmmT8XBUVrtfBVel1PsLCnNsMDXWOsbT09Lxg3jqzAsL1/j6zaqqiwtiAJTjY+Z60Wj3ft1VVtb8HJM8qj9oqQVwVE2fuQ7XtM+da51xcz9n5OPPb1jNZrZ5hd1iYZ5XPuSphpNOVTK7FbvesVjx1ynlZXRU956pyqus66XQlUfXfBWdWX7hacWt7bc/2Gp/rsupfKrieu5CQ0/u8axyuyr6zVdC5fnZVDrqq2ap/g16but5H5/MxwPV3r3plqqtKzpsqFG+W6mM8W1WG6/exq1r1bGOtrdI4OPh0lWD159Nqrfm+PfO8q+qj+m3PrOA6W5DscJz+G+P6e9MQQkM9q4Utlpr7XqCoLWyv/vOZz9+ZlYOufbn6a3tmmO3N7+DGeFld61fnqlapvi9YLJ6/v2qrzHRVnVW/zNffHxbL6YpkV/X6mRW6/t53RIQzwElP9+12jdDFUWcEIDC5Wi8CVWios50oUFxyiXTjjafPu+bt8aaUurE4ccIZ5pw44QxtWrVyVuMEWrhnFFd7T1GRMwCKinIGN65WLH/uz1XGXFZ2uoTftVgsnoFbZaXzn6imTZ1Bq+vDbXWuCp2yspr//Eme5dFnaxE4F9eH/dJS57jq+qDny3Xh4c7n1PXYGuu+VD1QdLUO1NVSVz2waayPG+apHkQHkuphffXgNdD/hlT/nVldbR9ivf1g6wqazqV60HzmfZ/t57rWO1fwcrbrYIz63m+rh9d1BUCuLzh92f6Zf7PPDMerX+b6P+MiQAUOAAAAAABAgAvwOBoAAAAAAAAEOAAAAAAAAAGOAAcAAAAAACDAEeAAAAAAAAAEOAIcAAAAAACAAEeAAwAAAAAAEOAIcAAAAAAAAAIcAQ4AAAAAAECAI8ABAAAAAAAIcAQ4AAAAAAAAAY4ABwAAAAAAIMAR4AAAAAAAAAQ4AhwAAAAAAIAAR4ADAAAAAAAQ4AhwAAAAAAAAAhwBDgAAAAAAQIAjwAEAAAAAAAhwBDgAAAAAAAABjgAHAAAAAAAgwPkV4CxcuFAjR45Ur169NHHiRG3ZsuWc63/22Wf62c9+pl69eumaa67RihUr/BosAAAAAADAxcjnAGfJkiWaM2eO7r77bn344Yfq2rWrpkyZooKCglrX37Bhgx544AFNmDBBH330kS6//HLdfffd2rVr13kPHgAAAAAA4GLgc4Dz+uuv64YbbtD48ePVuXNnzZgxQ2FhYVq0aFGt67/11lsaOnSofvWrX6lTp0767W9/q+7du+udd94578EDAAAAAABcDHwKcCoqKrRt2zalp6efvgOrVenp6dq4cWOtt9m0aZMGDx7scVlGRoY2bdrk+2gBAAAAAAAuQj4FOIWFhbLZbIqPj/e4PD4+Xvn5+bXeJj8/XwkJCV6vDwAAAAAAAE/BZg/AGw6HQ5JUXFxs8kgAAAAAAADqX2RkpCwWy1mv9ynAiYuLU1BQUI0JiwsKCmpU2bgkJCTUqLY51/q1KSkpkSQNGzbMl+ECAAAAAAA0CuvXr1dUVNRZr/cpwAkNDVWPHj20Zs0ajRo1SpJkt9u1Zs0a3XLLLbXepm/fvvruu+902223uS9bvXq1+vbt6/V2k5KStGLFijrTKAAAAAAAgMYoMjLynNf73EI1efJkTZ8+XT179lTv3r315ptv6tSpU7r++uslSdOmTVPz5s31wAMPSJImTZqkW2+9Va+99pqGDRumJUuWaOvWrZo5c6bX27RarWrRooWvQwUAAAAAALgg+BzgjB07VseOHdOLL76ovLw8devWTa+++qq7JerIkSOyWk/PjdyvXz89++yzev755/Xcc8+pQ4cO+utf/6qUlJT6exQAAAAAAAAXMIvDNUMwAAAAAAAAApJPhxEHAAAAAACA8QhwAAAAAAAAAhwBDgAAAAAAQIAjwAEAAAAAAAhwBDgAAAAAAAABjgAHAAAAAAAgwBHgAAAAAAAABDgCHIMsXLhQI0eOVK9evTRx4kRt2bLF7CHhPL300ksaP3680tLSNHjwYN11113at2+fxzq33nqrUlNTPZZHH33UpBHjfPzlL3+p8Vr+7Gc/c19fXl6uGTNmaODAgUpLS9O9996r/Px8E0eM8zFy5Mgar3dqaqpmzJghiX27sVu3bp3uvPNOZWRkKDU1VUuXLvW43uFw6IUXXlBGRoZ69+6t2267TQcOHPBY5/jx43rggQfUr18/XXLJJfrjH/+okpISAx8FvHWu17uyslLPPPOMrrnmGvXt21cZGRmaNm2ajh496nEftf1OePnll41+KPBCXfv3Qw89VOO1nDJlisc67N+NR12vd21/y1NTU/Xqq6+612H/bhy8+ezlzf/j2dnZuuOOO9SnTx8NHjxYTz31lKqqqox8KOcl2OwBXAyWLFmiOXPmaMaMGerTp4/efPNNTZkyRZ9//rni4+PNHh789P333+vmm29Wr169ZLPZ9Nxzz2nKlCn69NNPFRER4V7vhhtu0H333ec+Hx4ebsZwUQ+6dOmi119/3X0+KCjI/fOTTz6pFStW6Pnnn1d0dLRmzZqle+65R++++64ZQ8V5+uCDD2Sz2dznd+/ercmTJ3uEduzbjVdpaalSU1M1fvx43XPPPTWuf+WVV/T2229r7ty5atOmjV544QVNmTJFS5YsUZMmTSRJDz74oPLy8vT666+rsrJSf/zjH/Xoo49q3rx5Rj8c1OFcr3dZWZm2b9+uqVOnqmvXrioqKtLs2bM1depULV682GPd++67TzfccIP7fGRkpCHjh2/q2r8laejQoZozZ477fGhoqMf17N+NR12v98qVKz3Of/PNN/rTn/6kMWPGeFzO/h34vPnsVdf/4zabTb/+9a+VkJCgd999V7m5uZo+fbpCQkL0u9/9zsyH5z0HGtyECRMcM2bMcJ+32WyOjIwMx0svvWTiqFDfCgoKHCkpKY7vv//efdktt9zieOKJJ0wcFerLiy++6Lj22mtrva6oqMjRo0cPx2effea+bM+ePY6UlBTHxo0bDRohGtITTzzhGDVqlMNutzscDvbtC0lKSorjyy+/dJ+32+2OIUOGOF599VX3ZUVFRY6ePXs6PvnkE4fDcXr/3rJli3udFStWOFJTUx05OTnGDR4+O/P1rs3mzZsdKSkpjqysLPdlI0aMcLz++usNPDrUt9pe7+nTpzumTp161tuwfzde3uzfU6dOdUyaNMnjMvbvxunMz17e/D++fPlyR9euXR15eXnudf75z386+vXr5ygvLzd0/P6ihaqBVVRUaNu2bUpPT3dfZrValZ6ero0bN5o4MtS3kydPSpJiYmI8Lv/44481cOBAXX311Zo3b55OnTplxvBQDw4ePKiMjAxdfvnleuCBB5SdnS1J2rp1qyorKz32806dOqlVq1batGmTSaNFfamoqND//d//afz48bJYLO7L2bcvTIcPH1ZeXp7H/hwdHa0+ffq4/25v3LhRTZs2Va9evdzrpKeny2q10iJ9ASguLpbFYlHTpk09Ln/llVc0cOBA/fznP9err77aqEru4en777/X4MGDNWbMGD322GMqLCx0X8f+feHKz8/XihUrNGHChBrXsX83Pmd+9vLm//FNmzYpJSVFCQkJ7nUyMjJUXFysPXv2GDf480ALVQMrLCyUzWar0SoVHx9fo2cPjZfdbteTTz6pfv36KSUlxX351VdfrVatWikpKUk7d+7Us88+q/3792v+/Pkmjhb+6N27t+bMmaPk5GTl5eXpr3/9q26++WZ9/PHHys/PV0hISI1/9uPj45WXl2fSiFFfli5dqpMnT2rcuHHuy9i3L1yufba2v9uuPvr8/Hw1a9bM4/rg4GDFxMSwzzdy5eXlevbZZ3XVVVcpKirKffmtt96q7t27KyYmRhs3btRzzz2nvLw8/eEPfzBxtPDH0KFDdcUVV6hNmzbKzMzUc889p9tvv13vvfeegoKC2L8vYB9++KEiIyM1evRoj8vZvxuf2j57efP/eH5+vkd4I8l9vrHs3wQ4QD2YMWOGdu/erX/+858el//iF79w/5yamqrExETddtttOnTokNq1a2f0MHEehg0b5v65a9eu6tOnj0aMGKHPPvtMYWFhJo4MDW3RokW67LLL1Lx5c/dl7NvAhaeyslK/+c1v5HA43BOWu0yePNn9c9euXRUSEqLHHntMDzzwQI35UxDYrrrqKvfPrglrR40a5a7KwYVr0aJFuuaaa9xzmbmwfzc+Z/vsdTGghaqBxcXFKSgoSAUFBR6XFxQU1Ej/0DjNnDlTy5cv15tvvqkWLVqcc90+ffpIcrbioHFr2rSpOnTooEOHDikhIUGVlZUqKiryWKegoECJiYkmjRD1ISsrS6tXr6613Lo69u0Lh2ufPdff7YSEBB07dszj+qqqKp04cYJ9vpGqrKzUb3/7W2VnZ+u1117zqL6pTZ8+fVRVVaXDhw8bNEI0lLZt2youLs79+5v9+8L0ww8/aP/+/Zo4cWKd67J/B7azffby5v/xhISEGkelcp1vLPs3AU4DCw0NVY8ePbRmzRr3ZXa7XWvWrFFaWpqJI8P5cjgcmjlzpr788ku9+eabatu2bZ232bFjh6TG8wsCZ1dSUqLMzEwlJiaqZ8+eCgkJ8djP9+3bp+zsbPXt29e8QeK8LV68WPHx8Ro+fPg512PfvnC0adNGiYmJHvtzcXGxNm/e7P67nZaWpqKiIm3dutW9znfffSe73a7evXsbPmacH1d4c/DgQb3xxhuKi4ur8zY7duyQ1WrlaKIXgJycHB0/ftz9+5v9+8L0wQcfqEePHuratWud67J/B6a6Pnt58/943759tWvXLo8vaVavXq2oqCh17tzZkMdxvmihMsDkyZM1ffp09ezZU71799abb76pU6dO6frrrzd7aDgPM2bM0CeffKK//e1vioyMdPdNRkdHKywsTIcOHdLHH3+sYcOGKTY2Vjt37tScOXM0YMAAr/54ILA89dRTGjFihFq1aqXc3Fz95S9/kdVq1dVXX63o6GiNHz9ec+fOVUxMjKKiovTEE08oLS2NAKcRs9vtWrx4sX7+858rOPj0n0v27cavpKREhw4dcp8/fPiwduzYoZiYGLVq1UqTJk3S3//+d7Vv3959GPGkpCSNGjVKknNSxKFDh+qRRx7RjBkzVFlZqVmzZumqq67yaLVDYDjX652YmKj77rtP27dv10svvSSbzeb+ex4TE6PQ0FBt3LhRmzdv1qBBgxQZGamNGzdqzpw5uvbaa2scuADmO9frHRMTo/nz52vMmDFKSEhQZmamnnnmGbVv315Dhw6VxP7d2NT1+1xyhvCff/65pk+fXuP27N+NR12fvbz5fzwjI0OdO3fWtGnT9Pvf/155eXl6/vnndfPNNzeadjmLw+FwmD2Ii8E777yjBQsWKC8vT926ddPDDz/sLrlH45Samlrr5XPmzNH111+vI0eO6Pe//712796t0tJStWzZUqNGjdJdd91VZ2k2As/999+vdevW6fjx42rWrJn69++v+++/3z3fSXl5uebOnatPP/1UFRUVysjI0GOPPUZFRiO2cuVKTZkyRZ9//rmSk5Pdl7NvN35r167VpEmTalw+btw4zZ07Vw6HQy+++KLef/99FRUVqX///nrsscc83gfHjx/XrFmztGzZMlmtVo0ePVoPP/ywIiMjjXwo8MK5Xu977rlHl19+ea23e+uttzRw4EBt27ZNM2bM0L59+1RRUaE2bdrouuuu0+TJkxvNP/wXk3O93o8//rjuvvtubd++XSdPnlRSUpKGDBmi3/zmNx5TG7B/Nx51/T6XpPfee09PPvmkVq5cqejoaI/12L8bj7o+e0ne/T+elZWlxx9/XN9//73Cw8M1btw4PfDAAx5f1gUyAhwAAAAAAIAAxxw4AAAAAAAAAY4ABwAAAAAAIMAR4AAAAAAAAAQ4AhwAAAAAAIAAR4ADAAAAAAAQ4AhwAAAAAAAAAhwBDgAAAAAAQIAjwAEAAAAAAAhwBDgAAAAAAAABjgAHAAAAAAAgwBHgAAAAAAAABLj/B+ROFg+fhXhsAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1400x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Minimum validation loss: 0.06536959111690521\n"
     ]
    }
   ],
   "source": [
    "plt.figure(figsize = (14, 8))\n",
    "sns.set_style('white')\n",
    "\n",
    "plt.plot(history.history['loss'], color='k', label=\"Training loss\")\n",
    "plt.plot(history.history['val_loss'], color='r', label=\"Validation loss\")\n",
    "\n",
    "plt.legend(['Training Loss', 'Validation Loss'])\n",
    "\n",
    "sns.despine()\n",
    "plt.show()\n",
    "\n",
    "print(\"Minimum validation loss: {}\".format(history_df['val_loss'].min()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 423
    },
    "execution": {
     "iopub.execute_input": "2023-03-24T22:51:37.478354Z",
     "iopub.status.busy": "2023-03-24T22:51:37.477678Z",
     "iopub.status.idle": "2023-03-24T22:51:37.842082Z",
     "shell.execute_reply": "2023-03-24T22:51:37.840732Z",
     "shell.execute_reply.started": "2023-03-24T22:51:37.478313Z"
    },
    "id": "Aw8dxq-F-Ppk",
    "outputId": "1869b29a-efd5-4249-a2d7-a709ff4564e8"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABHAAAAKTCAYAAABxdv/MAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAB3FElEQVR4nO3deXhTZf7+8TtJ94VCF/ayCFKWllJAESiiKKKgIJvjjIA6qCNu4zbgOKMO6oj714VxdFwQHR1nBNwQcXQUFMGFHQEVZSlQKG0plO5tkt8f+SU0XZPQnpPC+3Vd58rSk5MnaU6Sc+fzPI/F6XQ6BQAAAAAAgKBlNbsBAAAAAAAAaBgBDgAAAAAAQJAjwAEAAAAAAAhyBDgAAAAAAABBjgAHAAAAAAAgyBHgAAAAAAAABDkCHAAAAAAAgCDXIgIcp9OpoqIiOZ1Os5sCAAAAAABguBYR4BQXF2vQoEEqLi42uykAAAAAAACGaxEBDgAAAAAAwKmMAAcAAAAAACDIEeAAAAAAAAAEOb8DnO+++07XX3+9MjMzlZKSok8//bTR23zzzTeaOHGiUlNTNXr0aC1ZsiSgxgIAAAAAAJyK/A5wSkpKlJKSovvuu8+n9ffu3avf/e53GjJkiN577z1deeWV+vOf/6wvv/zS78YCAAAAAACcikL8vcHIkSM1cuRIn9d/66231LlzZ911112SpB49emjdunV69dVXNWLECH/vHgAAAAAA4JTT7GPgbNy4UUOHDvW6LjMzUxs3bmzuuwYAAAAAADgpNHuAk5eXp8TERK/rEhMTVVRUpLKysua+ewAAAAAAgBaPWagAAAAAAACCXLMHOImJicrLy/O6Li8vTzExMYqIiGjuuwcAAAAAAGjxmj3AGTBggL7++muv61avXq0BAwY0910DAAAAAACcFPwOcIqLi7V9+3Zt375dkrRv3z5t375d2dnZkqQnnnhCs2fP9qx/+eWXa+/evXr00Uf1yy+/6I033tBHH32kq666qmkeAQAAAAAAwEnO72nEv//+e82YMcNzed68eZKkiRMn6uGHH1Zubq4OHDjg+XtycrJeeOEFzZs3T6+99prat2+vBx98kCnEAQAAAAAAfGRxOp1OsxvRmKKiIg0aNEjr1q1TTEyM2c0BAAAAAAAwFLNQAQAAAAAABDkCHAAAAAAAgCBHgAMAAAAAABDkCHAAAAAAAACCHAEOAAAAAABAkCPAAQAAAAAACHIEOAAAAAAAAEGOAAcAAAAAACDIEeAAAAAAAAAEOQIcAAAAAACAIEeAAwAAAAAAEOQIcAAAAAAAAIJciNkNAICTntMp7d4tbdkiVVVJMTFSdHTt0+hoyVojV6+okIqLpZISqbJSstuPLw6HFBYmRURI4eGu04gI13X1KSqSDh6U8vOl0FApMvL4Eh7ual9FxfHFbnf9LTpaiopyna/ZxuqPs7hYystzLWVlUtu2UocOrsdosdTfrqoq1/Pz1VfS6tVSVlbtdUJDpVatvJc2baSkJNf9uE8jI13PVVWV67Sy0nXfISGubYSGus47HK42lpVJ5eWuU4vF9fyFh7tOay7h4a7bOp2u56e8/Phz5b6PmovN5nrOLBbX7Y4dk7Kzjy85Oa622GzHF6vV9dxXfxwOhxQfL7VvL7Vr51oSElzr1XwcNS+Xl3u/dmouVVWuU4vF9Rqq/roIDT2+vdJS79Pq551OKS7u+NK6tRQbW/v1Wdd5yfUab2ypqHDdj8Nx/DQ8vPbrIjT0+Guy+mmg11VVHf8/uE8tFtfi/t9ara7H4t5PIiNdlx2O2q/Hus7X/J84HMdfr2Fh9Z8PCTm+vsNx/Lyvp76sI9Xef2y247d3/y8sFu91QkNdz0v1dZxO1+Ot+Xp1309jQkK8Xzvh4a7tFRcff68sKXE9NzExriU21vUeZrHU/9pvbKn5/6+qqvs1UP18faeS9/tsRYVrezbb8ee2vvcSi8V1/zXff2oulZXer2H3Uv2y+/l0P0fu5ysiwvv9yGZzrVvztet+zVf/f4SE1H4+HI7j+5D7uay52O3H2+R+Xqufr365rv2x5nm73dWWut7Dq18ODa39XlvXqd1e+70xIqL2/9zhcL3HFxYeX4qLj7/23fuB1ep6r3B/tkZHu7ZX1/8p0MtWa+3HX1VVu31VVccfh/vzp+Z5q9W13Zr7jc1W+zl1OGq/Pt2Pv7H33rrWqfk9Jzy8dtvqWqq/Bhta3K/thvYn92dozefTZqv7NV3zOvf7Y/W2hYR4f164v4fV3Pecztrvl3Wdd7+u6vrfVb9c1/PpcNTepvv9oeZ7UM3LxcXSkSPS0aOu08JC123dnwXuJSbm+Ge0+/2mqqrxx1VeXvd7bc3TxERp5kzXc3qSI8ABYIyiIunw4eNhQGRkwwf0DXE6XQf4e/bUPqhwOl0HlNWXqirvA4r6DiqKi11tzM93nR4+7LptZKT3h6xU+4PGZjv+96go1wfinj3Sxo3Spk2uDzZfuJ+figrXQUhVlf/PT1iY98FsZKQrUDlwwPV/OFEREbU/mB0O1/Pm/tCvKSrKFeQkJh7/kuL+4p+bK33zjev5P5m5D2wqKsxuCQAAwMklKUm6/HKzW9HsCHCAk507kW9qhw5JX3whrVwpbd5cu5rDanUFBvv2uZa6Agz3r07uxX05Lk7q0sW1dO3qWiRXZcbq1a4qjezspn9MzSk0VOrb1/X4iopcYYX7tLj4+K9N7tCpJpvt+K891as03L8MVQ9OKiqOV8HUJTraFaRUVbnuq6TEFUJVvy/3L2pWa+02uUOr+oSHuz5Ew8Nd1SVFRa77+OUX11KfVq2koUOl4cOlPn3qrkaq/svh0aNSQYHrtZib61oOHXK1rWYVgFT7V9rqvx67f4WSav8KV17u/WtgXdy/UNvt9a9TPYyLi5M6dnQt7dsf/xWvevWFzVb7MeTnu57TnBxXJZU79KqrEqv6ZXelRs1f9qr/4l/9lz7366K01PVcuX9tbujU6XT9X6r/Enfs2PHXZ81f1UpLaz+v4eGu94Gaizv0DQvz/tXNYnFtr7DQ+7VR/bmuHhT7c776de5fG6u/riTvANhud7XF/by5K5Pcz3HNELmuipbqi7vSwP26dVdV1Dx1/3pe1y/nvp42tk5dVUh2u/evye4Ks5qVEE5n7V9LQ0O9X6PuyrbGuNtR89fZ0FDvz5OoKNfzU1Tkel0UFR0Pr+vaBxrbL6r/D93/L3d7a/4QUP20vr85nbUrQtyVVHVVp1T/Vd9dedlQVYl7e9WrV+qqaLFYjld5Vn+eyspqVyBJtV+77irG6vt3VVXt/cKXyiL36616FUl91SV17Y81z4eE1K4ora/CwpftWa3H37eqf27W/P9aLMcrDNw/okRFeVdiun9AcleMub8HuKtAG/uf+Xqd3V77sVqtrs+f6pUQ7qqZ6lV5dVXpWSy195O67sP9faV6lZP7M7Lm+2tdp9XPV692rf4ZUr2dDS3V98WG1mlsf3J//6pZ/VZV1XiVintfrP4adldmVv+x0V1lWnPfc1fGNlTF6q7cqa+6svrl6s+n+7RmNV14uHe1W0MVdO7v7a1bH39tSbU/v4qLvSu/iopcr436vrfUrCpv6L3W6XRVZF94YeOfIycBAhzAaEVF0rffSmvWSGvXug7gZsyQzjyzdkWK3S7973/SkiWuN/fqb5Bxca43x5pfAHJzvQ/wCgpc3S66dfNe2rVzHWS7l4SE+r9AV1RIW7dKGzZI333nCm22b/f/sYeGut7I3dyl7rm5/m8rJMT1ONyl2e4PCcm7Wsb9xalmmbX7S2X1g4rISNfzkJDges7i4133U7Oix/1hWv2D0273/hAuLXVVnAwYIKWnS71719+1yV01VD3QCQvzDrUa6hYlHS9Zdm+nZvl2YqKrPe3bu76w1XX7ysrjXz7q+rv7deauDKp+MCe5nrfERFebq7+W3d22Dhw4XqVT/YtDZKR01lmugKuu+/aX09l4dZcv61TnPjiv6wuq+8u9e7sNlVPb7a7nKTo68MdXnfvgozlC2ubmPhh3B2TuffVk4u/rDAAAoAEWp7OxnxXNV1RUpEGDBmndunWKiYkxuzlA46qqXAesu3dLu3YdP12/3jXOR119/fv2la66Spo+3XWg+89/Sm++6dqOUeLjXQfg7lAnOlratk36/nvv4MUtLU06+2xpyBDvSg13t6UOHaTOnaVOnVynrVq5DmDdwU31MQvc593dmLKyjneT2rPHdaB65pnSsGGuCo0zzjgl+rkCAAAAgESAg5OVwyF9+qk0f74rBHjkEWnMmBPbZmmpa3vr1tUuyS0p8a54KClpeFtduri6ipxxhquqZfHi+rukJCRIl13mqjY5evR414SjR12/vNfsfpSYeHyAU/cgp7m5rhCp+nLokKuLTW6uKzBp7K2gdWspI8O1ZGZKI0a47gsAAAAA0OwIcHByOXZMWrjQFbT8+KP332bMkJ580hVo+MPplD74QLr1VlcVja9sNldQ062b1L2767RfP1dXkY4dvdc9elT697+lV191da0KD5cmTJCmTXMFT411nzlRVVWuEMcd6OTmus4fPSr16uUKbbp2pSsAAAAAAJiEAAfBpbJS+uEH1/gq27YdP92719VFp77BwdzX7dzpCnEk1zgfV13lCh2efdYVxLRt6wp3pkzxLYzYsUP6/e+ljz5yXe7USbr5Zlc1SvWBzSIja09j27q1b4My1pSdfXyqPQAAAAAAxCDGMJPD4Zpeee1aV7ek9etdsxnVNw2xr1JSpJtukq688vhgrZdfLl1zjSsMuuwy6aKLpF//WrrgAlc3o+pKSqQVK1xVN6+84uomFRoq3XmndPfdrnClOdWszgEAAAAAnPKowIHx9u93dRV6+eW6uyS1auXqatSnj2tg3z59XF2Q3NPZNTTNZlyca6DbumZkKS+XHnrItVSfYnbQINe0c23aSB9/7Joau3qINGaM9Mwzrq5EAAAAAACYgAAHxqiqkpYulV56ydUdyT0LU2ysK3AZNOj4ctppzTvWyvbt0uuvS8uXuwYQrkuXLq4qnUmTpNGjGfsFAAAAAGAqAhw0r6IiVzek//s/18xHbmef7erSNHmyuVNBHzwo/fe/rjCnsFA67zxXNU7v3oQ2AAAAAICgQYCD5nHwoGuw4OeekwoKXNclJkq//a00cybdkQAAAAAA8AODGKNpOZ3S3/7mGvDXPY5Mjx7SHXe4BhU2s9oGAAAAAIAWigAHTaey0jXl9t//7ro8ZIj0hz9Il17qmuYbAAAAAAAEhAAHLvv2SR06BB60HDkiTZ0qffqpa+yYhx92hTeMIwMAAAAAwAmrY65lnFKcTun++6XkZGnUKKmsrP518/Ol++6TnnrKNejv7t2u2aR+/lkaOtQV3kRFSUuWSLNnE94AAAAAANBEqMA5mX3yifTYY1JGhnTvvVJ0tPffHQ7p9tulp592Xf7iC2nGDOmttyRrjWwvP98V8Gze7H19RIRr3ZISqXNn6YMPpAEDmu0hAQAAAABwKqIC52S0aZM0Zox0wQWuEOfRR6W0NOl//zu+TlWVa0Yod3hz441SaKj09tuu6pnqDh+Wzj/fFd60a+ea+rtfPykszFWxU1IinXGG9O23hDcAAAAAADQDApyTSVaWa6anjAzpv/91BTLXXOPqHrVrlyuEufZa1xTfU6ZICxe6xrx5/XXXlN+vvurazhNPSM8+6zrvDm82bpTatpU+/1xatEj6/nupuFjasUNas0Zatco1hg4AAAAAoMVxOBxav369fv75ZzmdTrObgzpYnC3gP1NUVKRBgwZp3bp1iomJMbs5wemHH6Qzz5SOHXNdvvxy6a9/lU47zXXdH//omt5bcoU2drsUHu6quLnkkuPbmTdPuvtu1/g1Cxa4gpx166SkJFd406+f8Y8NAE4hlZWV2rdvn5KTkxUScuI9nZ1Opw4dOqSysjJVVFR4FovFos6dOyshIUGWU2zMssrKShUWFio0NFTR0dGyBelMiU6nU/v27dOGDRu0ceNGZWVlqUuXLurVq5d69eql008/XbGxsfXePicnR+vXr9f69euVn5+voUOH6pxzzlFSUlKTt3P37t1yOBw67bTTmuT1VFFRIavVesL7wMGDB7VmzRqtXbtWZWVlCgkJ8SyhoaHq0KGDunfvru7du/u1z1VUVMhmswXta6cuhw8f1pYtW7R582bPUlVVpYEDB2rQoEEaPHiw0tLSFB4eblibnE6n9u7dq02bNsnhcKh169aKi4tT69at1bp1a7Vq1UrWmt36/z+73a4DBw5o3759io+PV9euXX1qu9Pp1K5du/T5559r/fr1io6OVmJiomdp06aNnE6nqqqqGlwsFouSk5PVvXt3de7cucleC3a7XXv27NFPP/2kwsJCde7cWV26dFGHDh18uo/y8nL98MMPys/PV2hoqMLCwhQWFqbQ0FC1adNGHTp0qPM5PXjwoNauXat169apqqpKycnJnqVz586Ki4urd992Op364Ycf9MknnygiIkJ9+vRRnz59lJiY6PfjLy8v1759+7Rnzx7t3btXOTk5ys3N1aFDh5Sbm6vCwkKde+65+u1vf6vu3bv7vf1gUFVVpcLCQoWHhys8PLzO9x2Hw6HKykqFhIQ0+/tMVVWVVq5cqcWLF+udd97RwYMHJUnt2rVTZmamMjMzNXz4cKWmpioyMrLObTidTh08eFA///yzwsLClJCQoISEBMXFxdV6vbn3oYiIiHrbtGvXLn3wwQdatWqVHA6H13t3RESEUlNTNXjwYKWnp9fbppMVAc7JwOmUzjvPFbAMHCg9/7yrS1NNX34pzZzpqpqJjZXef18655za25o1S3rhhePXJSa6tp2a2qwPAwDKy8uVnZ2tsrIyJSYmKj4+/oS+uBw8eFDffvutDh06JLvdrqqqKtntdtntdpWWlqq4uNizFBUVyWazKSoqymsJDQ2V1Wr1LO51WrVqpbi4OLVq1UqtWrXyfMGtj9PpVGFhoYqLi1VeXu5ZioqKtGXLFm3YsEHr16/Xli1bVF5erlatWum8887TBRdcoDFjxnh9UXU6nSoqKvK0OTQ01PNFvaysTN99953WrFmjr7/+Wl9//bXy8/PrbVdUVJS6dOmirl27qlu3burXr59SU1OVmprqOdA/cuSIvv32W8/2cnJydPrpp6tPnz7q3bu3+vTpow4dOnjadOzYMR07dsxzvrHrwsPD1a1bN8+BdPfu3dWmTRvZ7XY5HA7P/6yiokJlZWUqLS1VWVmZysrKFBERobZt23qW1q1bq6CgQN9//722bt3qOc3JydHRo0d19OhRlZaWej0HkZGRiomJUWxsrNq0aaOkpCSvA7qqqirPa6S4uFglJSWKjY1VUlKSkpKS1LZtW8XHx6ugoEAHDhzQwYMHdeDAAeXl5alNmzZq37692rdvrw4dOigpKUkVFRWebdVc3Nfn5+dr8+bNOnz4cIOvcfcBp/t12KpVK1VWVmrjxo3Kzs6u8zZpaWkaNWqU+vXrp9zcXB04cEDZ2dk6cOCACgoKah2oRkdHe70u0tLSZLfb9eWXX2rVqlX68ssvPffVqVMnjRo1Suedd57OPfdcderUSbm5ucrJydHBgweVk5OjiooKz4FLeHi4wsLCdPDgQW3fvt2z7Nq1S06nU0lJSZ7nrkOHDoqJiZHNZvN8ka/rvCRt2bJFq1ev1s6dOxt8/qqz2Wyeg9UOHTqoY8eO6tChgxITE5Wdna2ff/5ZP//8s3755Rfl5ORIkkJDQxUREaGIiAiFh4fXOsCt62u2zWZTQkKC57WTlJSk1q1bq7y8XCUlJSopKfG8xt2vffd+EB4erpSUFPXr10/9+vVT7969FR4erry8PP3888/asWOHduzYob179yovL89rOXr0aKPPQWhoqDp27Oj1flnfaUxMjDp37qxOnTp5Th0Ohw4cOOBZDh48qIiICHXp0sWzdO7cWdnZ2Vq/fr3WrVunvLy8etsTEhKixMREz3OVkJCgvLw87dmzR1lZWaqsrPSsa7FY1LFjR3Xv3l3dunVTfHy8YmNjPYvVatWaNWu0YsUKZWVl+fqy8ElISIiSk5PVsWNHVVVVed6fysvLVVVVpcjISEVGRno+VyIiIrwOSENCQlRcXKyffvpJv/zyiyoqKuq8D/dzXf09LykpSaWlpdq8ebM2bdqkH374QVVVVfW2NTQ0VMnJyerWrZu6du2qgoICrV27Vvv27WvwMbZr106DBw/2hH0ZGRn66aef9MEHH+iDDz7QL7/8Uus2iYmJ6tOnj0JCQjzv9+73f5vNpoiICEVGRioiIkKhoaGe9wlfWCwWnX/++brmmms0YcIE5ebmatWqVZ5l27Zt6t69u9LT09W/f3+lp6erT58+cjgcKi0t9SzFxcWez4YjR47o6NGjKiwsVHl5ued/WF5eLrvd7tnX3f9Pq9WqI0eO6PDhwyooKPC8fyYnJ6tr166eJSwsTD/88IPn/W3Hjh1er12r1aqIiAhZrVZVVlaqsrJSDofD8zhbt26txMREJSQkKDExUTabzdNOd9utVqsn+HQHoVarVaWlpZ73lNLSUlksFq/3LJvNptWrV3t9T4iNjVV5eXmdr8P27dvrtNNOU/fu3dWlSxfl5ORo+/bt2rZtW53vMTabTa1bt5bD4fA8p+7H1qFDB6WlpXk+Vzp37qwVK1bo/fff15YtW3x6HdhsNvXr108jRozQ/fffr/j4eJ9u15IR4JwM/vUv6Te/cQ0ovG2b1FAaXVrqWn/YMKl377rXqaqSLr1U+vBDKSHBFd6kpTVL0wHUVlVVpfLycs+HeGVlpSoqKlRYWKgjR47oyJEjKigoUGFhoeeXiPDwcM+XCvcvgtE1By4/AU6nU7m5uZ4Dg+pffkpLS+VwOJSYmOj5Mtm2bVtZrVZt27bN60A6JydHsbGxngPN2NhYOZ1OZWdna//+/bW+yFssFsXHx3sOlN0H1u7zrVq18voCbLPZtGfPHn399df65ptvtGfPniZ7DnwRHx+v0047Taeddpq6du2qwsJCZWVleZZj7irJRlitVs8XHLcuXbrIarV6vqzV/HtD3F/Y3L/ChoWFqaqqSocOHWrwdm3btlVcXJx27Njh832Zra7nriULCQlR3759NWDAAHXv3l1ZWVnasWOHfvrpp0b/fxaLRb1799bAgQMVFxenL774Qt9//32ztdNqtdb6wm+xWEwtw7dYLEpNTdVZZ52lNm3aeAVT7l/6d+3apd27d9d5sBLsrFaroqOjfX5v6datm/r37+9ZQkJCtG7dOn333Xdau3Zto4Fhc3C/xiMjIz0H0UeOHFFZQzOjVrtthw4dlJ+fr5KSEr/uc8iQIRo2bJjsdrtX0FVQUOCp/mpoqays1N69e7Vnz54mf+2Eh4fr9NNPV1xcnPbt26d9+/bJbrf7fPvWrVt7wqTqlZcFBQX1bsdisahPnz4aPHiwoqKitHfvXs9SUFDQ6H2GhYXpnHPOkc1m0/bt27V7926f21tTZGSkJ/Br3769V9hpsVj0r3/9S5988oln/fDwcJWXlwd8f5ASEhJ06aWXasqUKRo1apQcDofWrl2rr776SqtWrdLq1asbfX+wWq3q0qWL7Ha73/tkTTabTSNGjNCFF16o2NhYz/u23W7X0aNHtWHDBq1du9brc/Bf//qXLr/88oDvs6UgwGnpCgtdQcyBA67pwO+5p2m2W1LiGhvn/POlHj2aZpvAKaCiosLzC1NoaKji4uIUHR3t+VW2oqJC27Zt08aNG7Vp0yZt3rxZeXl5Xr9MNdWXkPbt26tHjx7q2rWr7Ha7p3KguLhYpaWlXh+G7l/roqOjPcGKu3R9586d2rFjhwoLC5ukXY0JDw9XZGSkjhw5csLbslgs6tu3r7p37+4V8rh/+YuOjlZMTIyio6MVHR0th8Ph+QXcvbifo+qVICUlJSosLPQs7l/gfGG1Wr2qDyIjI5WSkqKMjAwNHDhQGRkZ6tatmzZs2KCPP/5Y//3vf7VmzZo6f1Gt7+C4W7duOuusszR06FANHTpU6enpCgsLq7VeWVmZp1R9z549+vnnnz1hW83KhZ49e+qss87SkCFDlJycrB07dmj79u2eXxULCgoUGRmp2NhYTzVL9fM1T6ufLykp8RxE79q1S7t27dKxY8c8/yt35VNYWJjXL7YREREqLS3VoUOHdOjQIa/XTLdu3TwVI/369fNUSLm7Z7Rq1Up2u93za7C7Iig/P99zIJebm6v8/HyFhIR4XicxMTGKjIxUYWGhV1n/4cOH1bp1a0+lSPv27ZWYmKgjR454KhEOHjyovLw8hYeH13rt1VxatWrlqbKor1vIkSNHlJWV5fVaLCwslMPh8Byg1/zedOjQIa1cuVKfffaZsrKy1K5dO0+lSYcOHZSQkKDQ0FCv/eXw4cP6/vvvvRaHw6GhQ4dqxIgRGjFihM4880xZLBatXr1an332mT777DN99913stvtslgsSkpKUrt27dSuXTtFRkZ6VaGVl5crPj7eq6Krd+/estlsXhVNBw8erPO9q+Zlu92unj17aujQoRoyZEiDlXFu7sqR3bt3Kzs721ORdODAAeXm5qpDhw7q0aOHevTooZ49e6pr166efah6tUVdalblVFZWKj8/3/O6zc3N1ZEjR7wqNNyvcfc+4N4PioqKtG3bNm3dulVbt271OrBOTk7W6aefrtNPP13dunWrVUnWoUMHtWrVqt7nwN0V7tChQ7WqmtxtqH6+sLBQ+/bt0/79+z0hg81m87yW3PtBWVmZV5C9d+9excfHa9CgQRo0aJD69+9fZ1eKsrIyz37o3tfy8/MVHx+vbt26qVu3burYsaNsNpvnRwb3+8eePXt09OhRr2q/srIy9e/fX+ecc46GDx/eZD9yOBwOZWdna/fu3crJyVFYWJjnRxV3hUNZWZmnCqKkpMRTXVX99RsWFqbTTz9dvXr1UnJysle3E7vdroMHDyorK0v79u3zeu85dOiQrFarZ59PT09XcnJynd2dqqqqlJ2d7XnP37Nnj6KiojwVNfUdZ7krRd1drNauXavt27crISFB48aN0yWXXKLRo0d7deksLi7Wjz/+qB9//FGSPN8t3O/91Sth3PtPUlKSunTposTExEa7Yu7atUuvvPKKXnnlFWVnZ8tqtSojI8PT5ad///7avXu3Nm3a5Pm+9fPPPys0NNRTQePe56p33XNX1lavUqn+f3S3t7S0VHa7Xa1bt1Z8fLzi4+PVpk0bWSwWZWVleT3HZWVlSklJUd++fT3dyzp27OipKq1e5eOuqHUv5eXlns+l/Px85ebmyuFweD7P3O11Op2e8NO9OJ1Or8fq7mrkvk/3aa9evXT22Wc32IXU6XSqoKBAO3fu1K5du7Rz505lZWUpKSnJ85h69erltS+XlZV5qpNsNpvXfiFJP/30k7Zs2eJZdu/erTPOOEPjx4/X2LFjG62mcTqd2r9/v9auXav8/HxNmzbN0C6gZiHAaeluu0166impZ09pyxZXFQ6AOrnHAtm9e7dyc3O9DtyKioqUn5+vnJwcr6WiosLzZcy9WCwWz4F89QP6Y8eO1fkrnM1mU1xcnGJjY5Wdne1VNusLdxeZVq1aeZXHtmrVSg6Hw+vgoaioSLt37/apVN5f7v7+3bp18xzEug8yLBaL54u2+0tlRUWFevfu7TmA7tevn5KTk1VcXOx1sOl0OtWxY0d16tRJnTp1Unx8vCwWi+cgp/oX+Ornc3NzVVRU5PUFuLKyUklJSRoyZIjOOussDR48uMEDlqZUVFSkXbt26ZdfftHOnTu1Z88excXFebondenSRcnJyYqKivJ724WFhdqwYYPCw8O9vmS6v4y5f2WtrKyUxWLx6YC1McXFxdq2bZsKCgo0cODABscycDqdcjgcpo8HUl5erry8PM+BApqH+6tjYwdYx44dU3FxsRITE5tkPCd4czqdysnJ0eHDh9W9e/dTbhwImKu8vNzTxdhMVVVV2r59u7p168b7Pk4JBDgt2ebNrjFv7HZp+XLX1OFAC5CVleXpo7xjx45av6rbbDbPrzLuX6zcv065x0tw/4Lt7k7j/oUxNDTU6wA/NzdX+/fv1+7du7V79+5aY180l/DwcK8+zNW1bt1aAwYMUHp6utLT09WxY8dav0y5+4O7uyX46/Dhw54gYe/evQoLC1N0dLSioqIUHR2tyMhIhYaGev2iKskrXHEHUt26ddPpp5+uHj16NDjgHAAAAIDmw88hLZXTKd14oyu8mTyZ8AbNzt2f9ejRo+rcubPPv/Tl5eV5ylfXrl2rVatWae/evc3c2vpZLBZ16tRJ7du39wpMYmJi1KZNG0+Jf/v27dWuXTtFRETUKvV3Op1eZe02m83TdcS9vdDQUDmdTq/B8QoLC9WxY8d6S5ubkruc94y6BjQHAAAA0OIQ4LRUr70mrVolRUdL//d/ZrcGLYDT6dSePXu0efNm/fTTT4qPj/fM9tK5c2fPDAhbt271TDO6fft2z6wheXl5XtUknTt39vS179q1q9esKkVFRcrNzdWWLVvqnAXFZrNp4MCByszMVHp6uqcbkLtfsXvGhupjAYSEhKiioqLW7D3uLjXuU3cXGvdgd0lJSerQoYNnhpvk5OQ6xwJpDhaLxRMOderUyZD7BAAAAHByIsBpiQoLpT/8wXX+3nul5GRz24OgYrfblZWV5Rk47scff9TmzZu1ZcuWegehtdlsatu2rQ4ePNjobCFRUVEqKSnxDFj4+eefN9qmHj16KD09XQMGDNDw4cM1ZMiQJp0hCQAAAABOdgQ4LdHChVJurnT66dKtt5rdGhisqqpKK1eu1NKlS5WTk+M1CG9hYaF2795d70wYoaGh6tu3r3r37q2CggLPTA0VFRU6cOCAJNe0wWlpaerfv79n5hZ3t6KkpCTZbDbl5+fr559/1o4dO7Rjxw7t27ev1ow+rVu3Vt++fZWWlsagcgAAAABwgghwWhqnU3ruOdf53/9eMqgrCE5cbm6uSkpKlJiYqKioKM8YKFVVVVq/fr1WrlypFStWaPXq1YqNjdXgwYN1xhln6IwzzlB6ero2bNigRYsW6Z133lFeXl6D9+WeijIlJUUpKSlKTU1V//79lZKSotDQUK913dNfZmdnq2vXrmrXrl2jj8U9YPBZZ50V+BMCAAAAAPAZs1C1NP/7n3T++VJMjLR/v2TQ9Ljwn91u13fffadly5Zp2bJlWrdunedvERERSkxMVEJCgn755RcVFRX5te2EhARdeumlSk1N9Yyx4l7cUxabPZ0vAAAAAKDpUIHT0vztb67TGTMIb4KAw+HQtm3btGPHDh08eNAz4G92drbWrFlTq1ImPDxc5eXlKisr84whI7mmlT777LN1zjnnaMSIESouLtZ3332ntWvX6rvvvtPOnTuVlJSkSZMmacqUKTrnnHMUEsLuCwAAAACnCo4AW5K9e6X33nOdv+EGc9tyirLb7dqyZYtWrFihlStX6osvvtDhw4frXT8uLk5jxozR2LFjdeGFF6pt27YqLi72zJqUl5en9u3bq3///rUqZkaOHOk5f+zYMUVFRVFVAwAAAACnKAKcluSFFySHQzrnHKlfP7Nbc1LZtWuXNm3apJKSEpWUlKi0tFQlJSXKz8/X3r17PUt2draqqqq8bhsVFaW0tDR16NBB7du39yx9+/bV0KFDa1XKuLs6de/e3ef2MQgwAAAAAJzaCHBaivJy6cUXXedvvNHctpwkiouLtXjxYi1YsEArVqzw+XYxMTHKzMzUyJEjdc4552jQoEG1BgYGAAAAAKApEeC0FIsXS4cOSZ06SRMmmN2aFqH6+DSVlZWqqqpSZWWlKisr9e233+qtt97SsWPHJEkWi0WDBg1SXFycoqKiFBkZqaioKLVu3VrJycnq3LmzkpOTlZycrA4dOtCVCQAAAABgKAKclsI9ePHvfidR7VGniooKrVmzRl999ZVWrVqlNWvW6MiRIw3e5rTTTtNvf/tbzZgxQ8nJycY0FAAAAAAAPxHgtAQbN0qrV7uCm2uvNbs1QcfpdOrdd9/VnXfeqZ07d3r9LTo6Wv369VNkZKRCQkIUGhqq0NBQtWvXTldccYXOPvtsWa1Wk1oOAAAAAIBvCHBaAnf1zeTJUvv25rYlyGzatEm33XabPv/8c0lSYmKiRo0apeHDh2v48OFKT09num0AAAAAQIvHkW2wKyyU3njDdf4UHbw4Pz9fmzdv9oxjY7fbVVVVpY8++kgvvfSSnE6nwsPDdeedd+quu+5STEyM2U0GAAAAAKBJEeAEuxUrpNJSqWdPafhws1tjiKqqKn3zzTf6+OOPtXz5cq1du1ZOp7Pe9S+77DI9+uij6tq1q4GtBAAAAADAOAQ4we6zz1yno0dLFou5bWkG2dnZ2rp1q7Zv3+5ZNm7cqKNHj3qtd9pppyk2NlY2m00hISGy2WxKTEzUH/7wB40YMcKk1gMAAAAAYAwCnGDnDnBGjTK3HU3I6XTqv//9rx566CF98cUXda7Tpk0bjR49WmPGjNEFF1ygzp07G9xKAAAAAACCBwFOMDt0SNqyxXX+nHNMbUpTsNvteueddzRv3jytX79ekmSz2dSzZ0/16dNHffr0Ud++fdWvXz/1799fNpvN5BYDAAAAABAcCHCC2f+fWUnp6VJiorltOQEFBQV64403NH/+fP3444+SpKioKP3ud7/T7bffTnUNAAAAAACNIMAJZu7uU+edZ247AuB0OvXFF1/oxRdf1KJFi1ReXi5Jat26tW655RbdfPPNSmzBoRQAAAAAAEYiwAlm//uf67QFjX+Tk5OjhQsX6qWXXtKOHTs816elpenaa6/VlVdeqVatWpnYQgAAAAAAWh4CnGC1Z4/0yy+SzSYF+SxLdrtd//3vf/Xiiy/qgw8+UFVVlSQpJiZGv/71r3XNNdfojDPOkOUknEULAAAAAAAjEOAEK/f4N2ecIQVpxUpVVZVefvll/fWvf9XevXs915911lm65ppr9Ktf/UoxMTEmthAAAAAAgJMDAU6wCuLxb5xOpz744APNmTNHP/zwgyTXtN8zZszQzJkzlZaWZnILAQAAAAA4uRDgBCOnM2jHv/n222/1hz/8QV988YUkKTExUffee6+uvfZaRUREmNw6AAAAAABOTgQ4weinn6TsbCk8XBo61OzWSJJKSko0e/Zs/e1vf5MkRURE6Pbbb9fs2bMVFxdncusAAAAAADi5EeAEI3f3qeHDpchIc9siad26dZo2bZqnu9RVV12lBx54QJ07dza5ZQAAAAAAnBqsZjcAdXAHOCZ3n7Lb7XrooYd01lln6YcfflCHDh308ccfa8GCBYQ3AAAAAAAYiAqcYONwHJ+BysQAJzs7W7/61a+0atUqSdKUKVP0/PPPKyEhwbQ2AQAAAABwqiLACTabN0v5+VJMjDR4sElN2Kxx48Zp3759io2N1fz58zV9+nRZLBZT2gMAAAAAwKmOACfYuLtPjRwphYYafvcff/yxpk6dqmPHjql3795aunSpevToYXg7AAAAAADAcYyBE2xMnD78xRdf1Lhx43Ts2DGdc845Wr16NeENAAAAAABBgAAnmFRUSF984TpvYIBjt9v1xz/+Udddd53sdrumT5+ujz/+WG3atDGsDQAAAAAAoH4EOMHk3/+Wioqkjh2l/v0Nuctvv/1WQ4YM0cMPPyxJuu+++7Rw4UKFhYUZcv8AAAAAAKBxjIETLJxO6bHHXOdvvlmyNm+2dvjwYf3xj3/Uiy++KKfTqbi4OM2fP1/Tpk1r1vsFAAAAAAD+owInWHzyibRlixQdLf3ud812N06nU6+++qpSUlL0j3/8Q06nUzNmzNCPP/5IeAMAAAAAQJCiAidYPP646/Saa6RmHHvm8ccf1+zZsyVJ/fr103PPPaezzz672e4PAAAAAACcOCpwgsGmTa4KHJtNuvXWZrub999/X3PmzJEk3XPPPdqwYQPhDQAAAAAALQAVOMHgiSdcp1OnSt26NctdbN68WVdccYWcTqeuv/56zZ07VxaLpVnuCwAAAAAANC0qcMy2b5/0r3+5zt9xR7PcRU5Oji655BIVFRVp1KhReuaZZwhvAAAAAABoQQhwzPb001JVlXTOOdLgwU2++bKyMk2aNElZWVnq2bOn3n77bYWGhjb5/QAAAAAAgOZDgGOmo0elF15wnb/zzibfvNPp1HXXXafVq1crLi5OS5cuVXx8fJPfDwAAAAAAaF4EOGZ66SXp2DGpTx/poouafPMvvviiXn/9ddlsNr399ttKSUlp8vsAAAAAAADNjwDHLA6Hq/uU5Kq+sTbtv+Knn37SbbfdJkl6+OGHNXr06CbdPgAAAAAAMA4Bjlny8qS9eyWLRfrNb5p005WVlZo2bZpKSko0atQo3X777U26fQAAAAAAYCwCHLPk5blO27SRIiKadNMPPvigvvvuO7Vu3VoLFy6UtYmrewAAAAAAgLE4sjeLO8BJTGzSza5Zs0YPPvigJOn5559X586dm3T7AAAAAADAeAQ4ZmmGAOfYsWOaPn26HA6Hpk2bpl/96ldNtm0AAAAAAGAeAhyz5Oe7ThMSmmyTt912m3755Rd16dJF8+fPb7LtAgAAAAAAcxHgmKWJK3BWrVqll19+WRaLRa+//rri4uKaZLsAAAAAAMB8BDhmaeIA56GHHpIkXXPNNTr77LObZJsAAAAAACA4EOCYpQkDnI0bN+qjjz6S1WrVnDlzTnh7AAAAAAAguBDgmKUJA5yHH35YknTZZZepR48eJ7w9AAAAAAAQXAhwzNJEAc7PP/+st99+W5J01113nWirAAAAAABAECLAMUsTzUL16KOPyuFwaOzYsUpPT2+ChgEAAAAAgGBDgGOWJqjAyc7O1sKFCyVJf/zjH5uiVQAAAAAAIAgR4JihslI6etR1/gQCnCeffFIVFRXKzMxUZmZmEzUOAAAAAAAEGwIcM7i7T1mtUuvWAW3i8OHDev755yVRfQMAAAAAwMmOAMcM7u5T8fGSzRbQJubPn6/i4mKlp6froosuasLGAQAAAACAYEOAYwZ3gBPgAMYlJSV65plnJLlmnrJYLE3VMgAAAAAAEIQIcMzg7kIV4Pg37733nvLz89WtWzdNmTKlCRsGAAAAAACCEQGOGU5wBqo333xTkjRt2jSFhIQ0VasAAAAAAECQIsAxwwkEOPn5+Vq+fLkk6Te/+U1TtgoAAAAAAAQpAhwznECAs3jxYlVVVSk9PV19+vRp4oYBAAAAAIBgRIBjhhMIcP71r39Jkn796183ZYsAAAAAAEAQI8Axg3sQYz9nodq/f79WrlwpSbr88subulUAAAAAACBIEeCYIcAKnH//+99yOp0aPny4unbt2gwNAwAAAAAAwYgAxwwBBjh0nwIAAAAA4NREgGOGAAKcHTt2aO3atbLZbJo6dWozNQwAAAAAAAQjAhyjlZdLx465zvsR4Lirb84//3y1bdu2OVoGAAAAAACCFAGO0dwDGFutUlycTzdxOp168803JdF9CgAAAACAUxEBjtGqz0Bl9e3p37hxo3788UeFh4dr4sSJzdg4AAAAAAAQjAhwjBbA+Dfu7lMXX3yxWrVq1RytAgAAAAAAQYwAx2h+BjgOh8MT4PzmN79prlYBAAAAAIAgRoBjND8DnG3btmnfvn2Kjo7W2LFjm7FhAAAAAAAgWAUU4LzxxhsaNWqU0tLSNHXqVG3evLnedSsrKzV//nydf/75SktL0/jx4/XFF18E3OAWzx3gJCT4tPqmTZskSQMGDFBERERztQoAAAAAAAQxvwOcZcuWad68ebrxxhv1zjvvqHfv3po5c6by3YPz1vDUU0/p3//+t+655x4tW7ZMl19+uW666SZt27bthBvfIrmfJx8rcNwBTnp6enO1CAAAAAAABDm/A5wFCxbosssu0+TJk9WzZ0/NnTtXERERWrx4cZ3rv/fee7r++us1cuRIJScn6ze/+Y1GjhypV1555YQb3yL52YWKAAcAAAAAAPgV4FRUVGjr1q0aNmzY8Q1YrRo2bJg2bNhQ520qKysVFhbmdV14eLjWr18fQHNPAgQ4AAAAAADAT34FOAUFBbLb7UqoMX5LQkKC8tzBRA2ZmZl69dVXtXv3bjkcDn311Vf65JNPdOjQocBb3ZL5EeDk5OQoJydHFotFqampzdwwAAAAAAAQrJp9Fqo//elP6tq1qy666CKlpqbq/vvv16RJk2S1nqITYPkR4Lirb04//XRFR0c3Z6sAAAAAAEAQC/Fn5TZt2shms9UasDg/P1+J9QQS8fHxeu6551ReXq4jR46obdu2evzxx5WcnBx4q1syP2ahovsUAAAAAACQ/KzACQsLU79+/bRmzRrPdQ6HQ2vWrFFGRkaDtw0PD1e7du1UVVWl//73vzrvvPMCa3FLVlYmFRe7zvtRgUOAAwAAAADAqc2vChxJuvrqqzVnzhylpqaqf//+WrhwoUpLSzVp0iRJ0uzZs9WuXTvdcccdklwhRE5Ojvr06aOcnBw9++yzcjgcuuaaa5r2kbQE7solm02Ki2t0dQIcAAAAAAAgBRDgjB07VocPH9Yzzzyj3Nxc9enTRy+99JKnC9WBAwe8xrcpLy/XU089pb179yoqKkojR47Uo48+qlatWjXdo2gpqo9/Y7E0uGp5ebl++OEHSQQ4AAAAAACc6ixOp9NpdiMaU1RUpEGDBmndunWKiYkxuzmB+9//pPPPl/r1k77/vsFVN2zYoIEDB6pNmzbKz8+XpZHABwAAAAAAnLxO0amgTBLgAMaENwAAAAAAnNoIcIzkHgOHAYwBAAAAAIAfCHCMVH0MnEYQ4AAAAAAAADcCHCP5GOA4nU4CHAAAAAAA4EGAYyQfA5z9+/fr8OHDstls6tu3rwENAwAAAAAAwYwAx0g+Bjju6pvevXsrIiKiuVsFAAAAAACCHAGOkXychYruUwAAAAAAoDoCHCP5OAsVAQ4AAAAAAKiOAMdIfnahIsABAAAAAAASAY5xSkpci9RggFNSUqIdO3ZIIsABAAAAAAAuBDhGcXefCg2VYmPrXe3777+Xw+FQ27Zt1b59e4MaBwAAAAAAghkBjlGqD2BssdS7Gt2nAAAAAABATQQ4RmH8GwAAAAAAECACHKMwAxUAAAAAAAgQAY5RfKjAcTqd2rx5syQCHAAAAAAAcBwBjlF8CHAOHjyowsJCWa1W9e7d26CGAQAAAACAYEeAY5TqgxjXIzc3V5KUmJio0NBQI1oFAAAAAABaAAIco/hQgZP3/9dJaCDkAQAAAAAApx4CHKP4MIixO8BJbGSgYwAAAAAAcGohwDGKHxU4BDgAAAAAAKA6Ahyj+BDg5P//Kh0CHAAAAAAAUB0BjlGowAEAAAAAAAEKMbsBp4z0dOnwYalTp3pXIcABAAAAAAB1IcAxyqpVrtOQ+p9yZqECAAAAAAB1IcAxSgPBjRsVOAAAAAAAoC6MgRNECHAAAAAAAEBdCHCCCLNQAQAAAACAuhDgBInS0lIVFxdLIsABAAAAAADeCHCChLv6JiQkRK1atTK5NQAAAAAAIJgQ4ASJ6uPfWCwWk1sDAAAAAACCCQFOkGAKcQAAAAAAUB8CnCDBDFQAAAAAAKA+BDhBghmoAAAAAABAfQhwggQVOAAAAAAAoD4EOEGCAAcAAAAAANSHACdIEOAAAAAAAID6EOAECWahAgAAAAAA9SHACRIMYgwAAAAAAOpDgBMk6EIFAAAAAADqQ4ATJAhwAAAAAABAfQhwgkBJSYlKS0slEeAAAAAAAIDaCHCCgLv6JiwsTDExMSa3BgAAAAAABBsCnCBQfQYqi8VicmsAAAAAAECwIcAJAsxABQAAAAAAGkKAEwQYwBgAAAAAADSEACcIEOAAAAAAAICGEOAEAQIcAAAAAADQEAKcIECAAwAAAAAAGkKAEwSqz0IFAAAAAABQEwFOEGAWKgAAAAAA0BACnCBAFyoAAAAAANAQApwgQIADAAAAAAAaQoBjMqfTSYADAAAAAAAaRIBjsuLiYpWXl0siwAEAAAAAAHUjwDGZu/omPDxcUVFRJrcGAAAAAAAEIwIck1WfgcpisZjcGgAAAAAAEIwIcEzG+DcAAAAAAKAxBDgmI8ABAAAAAACNIcAxGQEOAAAAAABoDAGOyQhwAAAAAABAYwhwTOYOcBISEkxuCQAAAAAACFYEOCarPgsVAAAAAABAXQhwTEYXKgAAAAAA0BgCHJMR4AAAAAAAgMYQ4JiMAAcAAAAAADSGAMdETqeTAAcAAAAAADSKAMdEx44dU2VlpSRmoQIAAAAAAPUjwDGRewaqyMhIRUVFmdwaAAAAAAAQrAhwTET3KQAAAAAA4AsCHBMR4AAAAAAAAF8Q4JiIAAcAAAAAAPiCAMdEBDgAAAAAAMAXBDgmcgc4zEAFAAAAAAAaQoBjIvcsVFTgAAAAAACAhhDgmIguVAAAAAAAwBcEOCYiwAEAAAAAAL4gwDERAQ4AAAAAAPAFAY6JCHAAAAAAAIAvCHBM4nQ6GcQYAAAAAAD4hADHJMXFxaqqqpIktW7d2tzGAAAAAACAoEaAYxJ3eCNJoaGhJrYEAAAAAAAEOwIck9jtds95m81mYksAAAAAAECwI8AxSfUKHKuVfwMAAAAAAKgfyYFJ3BU4NptNFovF5NYAAAAAAIBgRoBjEncFTkhIiMktAQAAAAAAwY4AxyTVK3AAAAAAAAAaQoBjEipwAAAAAACArwhwTEIFDgAAAAAA8BUBjkncAQ4VOAAAAAAAoDEEOCZxd6GiAgcAAAAAADSGAMckdKECAAAAAAC+IsAxCYMYAwAAAAAAXxHgmIQKHAAAAAAA4CsCHJNQgQMAAAAAAHxFgGMSKnAAAAAAAICvCHBMwjTiAAAAAADAVwQ4JmEacQAAAAAA4CsCHJPQhQoAAAAAAPiKAMckDGIMAAAAAAB8RYBjEipwAAAAAACArwhwTEIFDgAAAAAA8BUBjkmowAEAAAAAAL4iwDEJFTgAAAAAAMBXBDgmoQIHAAAAAAD4igDHJO4AhwocAAAAAADQGAIck7i7UFGBAwAAAAAAGkOAYxK6UAEAAAAAAF8R4JiEQYwBAAAAAICvCHBMQgUOAAAAAADwFQGOSajAAQAAAAAAviLAMQkVOAAAAAAAwFcEOCahAgcAAAAAAPiKAMckVOAAAAAAAABfEeCYxB3gUIEDAAAAAAAaE1CA88Ybb2jUqFFKS0vT1KlTtXnz5gbXf/XVVzVmzBj1799fI0eO1EMPPaTy8vKAGnyycHehogIHAAAAAAA0xu8AZ9myZZo3b55uvPFGvfPOO+rdu7dmzpyp/Pz8Otf/4IMP9MQTT+imm27SsmXL9Ne//lXLli3Tk08+ecKNb8noQgUAAAAAAHzld4CzYMECXXbZZZo8ebJ69uypuXPnKiIiQosXL65z/Q0bNmjgwIG65JJL1LlzZ2VmZuriiy9utGrnZMcgxgAAAAAAwFd+BTgVFRXaunWrhg0bdnwDVquGDRumDRs21HmbjIwMbd261RPY7N27VytXrtTIkSNPoNktHxU4AAAAAADAV36VfxQUFMhutyshIcHr+oSEBO3cubPO21xyySUqKCjQb37zGzmdTlVVVenyyy/X9ddfH3irTwJU4AAAAAAAAF81+yxU33zzjV544QXdd999WrJkiebPn6+VK1fqb3/7W3PfdVCjAgcAAAAAAPjKr/KPNm3ayGaz1RqwOD8/X4mJiXXe5umnn9b48eM1depUSVJKSopKSkp07733atasWbJaT82ZzJlGHAAAAAAA+Mqv9CQsLEz9+vXTmjVrPNc5HA6tWbNGGRkZdd6mrKysVkjjrjpxOp3+tvekwTTiAAAAAADAV36Xf1x99dWaM2eOUlNT1b9/fy1cuFClpaWaNGmSJGn27Nlq166d7rjjDknSueeeqwULFqhv377q37+/srKy9PTTT+vcc889pcMLulABAAAAAABf+R3gjB07VocPH9Yzzzyj3Nxc9enTRy+99JKnC9WBAwe8Km5mzZoli8Wip556Sjk5OYqPj9e5556r2267rekeRQvEIMYAAAAAAMBXAaUH06ZN07Rp0+r82+uvv+59ByEhuummm3TTTTcFclcnLSpwAAAAAACAr07NEYSDABU4AAAAAADAVwQ4JqECBwAAAAAA+IoAxyRU4AAAAAAAAF8R4JiEChwAAAAAAOArAhyTuAMcKnAAAAAAAEBjCHBM4u5CRQUOAAAAAABoDAGOSehCBQAAAAAAfEWAYxIGMQYAAAAAAL4iwDEJFTgAAAAAAMBXBDgmoQIHAAAAAAD4igDHJFTgAAAAAAAAXxHgmIRpxAEAAAAAgK8IcEzCNOIAAAAAAMBXBDgmoQIHAAAAAAD4igDHJFTgAAAAAAAAXxHgmIRBjAEAAAAAgK8IcEzCNOIAAAAAAMBXBDgmoQIHAAAAAAD4igDHJFTgAAAAAAAAXxHgmIQKHAAAAAAA4CsCHJMwjTgAAAAAAPAVAY5JmEYcAAAAAAD4igDHJHShAgAAAAAAviLAMQmDGAMAAAAAAF8R4JiEChwAAAAAAOArAhwTOJ1OORwOSVTgAAAAAACAxhHgmMBdfSNRgQMAAAAAABpHgGMC9/g3EhU4AAAAAACgcQQ4JqACBwAAAAAA+IMAxwTVAxwqcAAAAAAAQGMIcExQvQsVFTgAAAAAAKAxBDgmoAsVAAAAAADwBwGOCdwVOFarVRaLxeTWAAAAAACAYEeAYwJ3BQ7VNwAAAAAAwBcEOCZwV+AwgDEAAAAAAPAFAY4JqMABAAAAAAD+IMAxgTvAoQIHAAAAAAD4ggDHBO4uVFTgAAAAAAAAXxDgmIAKHAAAAAAA4A8CHBNQgQMAAAAAAPxBgGMCBjEGAAAAAAD+IMAxAdOIAwAAAAAAfxDgmIAKHAAAAAAA4A8CHBNQgQMAAAAAAPxBgGMCKnAAAAAAAIA/CHBMwDTiAAAAAADAHwQ4JmAacQAAAAAA4A8CHBPQhQoAAAAAAPiDAMcEDGIMAAAAAAD8QYBjAipwAAAAAACAPwhwTEAFDgAAAAAA8AcBjgmowAEAAAAAAP4gwDEBFTgAAAAAAMAfBDgmoAIHAAAAAAD4gwDHBO4AhwocAAAAAADgCwIcE7i7UFGBAwAAAAAAfEGAYwK6UAEAAAAAAH8Q4JiAQYwBAAAAAIA/CHBMQAUOAAAAAADwBwGOCajAAQAAAAAA/iDAMQEVOAAAAAAAwB8EOCZgGnEAAAAAAOAPAhwTMI04AAAAAADwBwGOCajAAQAAAAAA/iDAMQEVOAAAAAAAwB8EOCZgEGMAAAAAAOAPAhwTMI04AAAAAADwBwGOCajAAQAAAAAA/iDAMQEVOAAAAAAAwB8EOCagAgcAAAAAAPiDAMcETCMOAAAAAAD8QYBjAqYRBwAAAAAA/iDAMQFdqAAAAAAAgD8IcEzAIMYAAAAAAMAfBDgmoAIHAAAAAAD4gwDHBFTgAAAAAAAAfxDgmIAKHAAAAAAA4A8CHBMwjTgAAAAAAPAHAY4JmEYcAAAAAAD4gwDHBFTgAAAAAAAAfxDgmIAKHAAAAAAA4A8CHBMwiDEAAAAAAPAHAY4JmEYcAAAAAAD4gwDHBFTgAAAAAAAAfxDgmIAKHAAAAAAA4A8CHBNQgQMAAAAAAPxBgGMCphEHAAAAAAD+IMAxAdOIAwAAAAAAfxDgmIAKHAAAAAAA4A8CHBNQgQMAAAAAAPxBgGMCBjEGAAAAAAD+IMAxAdOIAwAAAAAAfxDgmIAKHAAAAAAA4A8CHBNQgQMAAAAAAPxBgGMCKnAAAAAAAIA/CHBMwDTiAAAAAADAHwQ4JmAacQAAAAAA4A8CHBPQhQoAAAAAAPiDAMcEDGIMAAAAAAD8QYBjMKfTSQUOAAAAAADwCwGOwRwOh+c8FTgAAAAAAMAXBDgGc1ffSFTgAAAAAAAA3xDgGKx6gEMFDgAAAAAA8AUBjsHcAxhLVOAAAAAAAADfEOAYjAocAAAAAADgLwIcg1GBAwAAAAAA/EWAY7DqFThWK08/AAAAAABoHAmCwdwVOHSfAgAAAAAAviLAMZi7AofuUwAAAAAAwFcEOAajAgcAAAAAAPiLAMdgVOAAAAAAAAB/EeAYzB3gUIEDAAAAAAB8RYBjMHcXKipwAAAAAACArwhwDEYFDgAAAAAA8FdAAc4bb7yhUaNGKS0tTVOnTtXmzZvrXXf69OlKSUmptVx33XUBN7olowIHAAAAAAD4y+8ykGXLlmnevHmaO3eu0tPTtXDhQs2cOVPLly9XQkJCrfWfffZZVVZWei4fOXJEEyZM0IUXXnhiLW+hGMQYAAAAAAD4y+8KnAULFuiyyy7T5MmT1bNnT82dO1cRERFavHhxneu3bt1aSUlJnuWrr75SRETEKRvgMI04AAAAAADwl18BTkVFhbZu3aphw4Yd34DVqmHDhmnDhg0+bWPx4sUaN26coqKi/GvpSYIKHAAAAAAA4C+/ApyCggLZ7fZaXaUSEhKUl5fX6O03b96sn376SVOnTvWvlScRKnAAAAAAAIC/DJ2FatGiRerVq5f69+9v5N0GFSpwAAAAAACAv/wKcNq0aSObzab8/Hyv6/Pz85WYmNjgbUtKSvThhx9qypQp/rfyJMI04gAAAAAAwF9+BThhYWHq16+f1qxZ47nO4XBozZo1ysjIaPC2y5cvV0VFhcaPHx9YS08STCMOAAAAAAD85XcZyNVXX605c+YoNTVV/fv318KFC1VaWqpJkyZJkmbPnq127drpjjvu8LrdokWLdP7556tNmzZN0/IWii5UAAAAAADAX34HOGPHjtXhw4f1zDPPKDc3V3369NFLL73k6UJ14MABWa3ehT07d+7UunXr9MorrzRNq1swBjEGAAAAAAD+CihFmDZtmqZNm1bn315//fVa15122mn68ccfA7mrkw4VOAAAAAAAwF+GzkIFKnAAAAAAAID/CHAMRgUOAAAAAADwFwGOwZhGHAAAAAAA+IsAx2BMIw4AAAAAAPxFgGMwKnAAAAAAAIC/CHAMRgUOAAAAAADwFwGOwRjEGAAAAAAA+IsAx2BMIw4AAAAAAPxFgGMwKnAAAAAAAIC/CHAMRgUOAAAAAADwFwGOwajAAQAAAAAA/iLAMRjTiAMAAAAAAH8R4BiMacQBAAAAAIC/CHAMRgUOAAAAAADwFwGOwajAAQAAAAAA/iLAMRiDGAMAAAAAAH8R4BiMacQBAAAAAIC/CHAMRgUOAAAAAADwFwGOwRjEGAAAAAAA+IsAx2AMYgwAAAAAAPxFgGMwKnAAAAAAAIC/CHAMRgUOAAAAAADwFwGOwRjEGAAAAAAA+IsAx2BMIw4AAAAAAPxFgGMwKnAAAAAAAIC/CHAMRgUOAAAAAADwFwGOwajAAQAAAAAA/iLAMRjTiAMAAAAAAH8R4BiMacQBAAAAAIC/CHAMRgUOAAAAAADwFwGOwajAAQAAAAAA/iLAMRiDGAMAAAAAAH8R4BiMacQBAAAAAIC/CHAMRgUOAAAAAADwFwGOwajAAQAAAAAA/iLAMRgVOAAAAAAAwF8EOAZjGnEAAAAAAOAvAhyDMY04AAAAAADwFwGOwajAAQAAAAAA/iLAMRgVOAAAAAAAwF8EOAZjEGMAAAAAAOAvAhyDMY04AAAAAADwFwGOwajAAQAAAAAA/iLAMRiDGAMAAAAAAH8R4BiMQYwBAAAAAIC/CHAMRgUOAAAAAADwFwGOwajAAQAAAAAA/iLAMRiDGAMAAAAAAH8R4BiMacQBAAAAAIC/CHAM5HA45HQ6JVGBAwAAAAAAfEeAYyB39ymJChwAAAAAAOA7AhwDVQ9wqMABAAAAAAC+IsAxEBU4AAAAAAAgEAQ4BnIPYCxRgQMAAAAAAHxHgGMgKnAAAAAAAEAgCHAMVL0Cx2rlqQcAAAAAAL4hRTCQuwLHarXKYrGY3BoAAAAAANBSEOAYyF2BQ/cpAAAAAADgDwIcA7krcBjAGAAAAAAA+IMAx0BU4AAAAAAAgEAQ4BiIChwAAAAAABAIAhwDuQMcKnAAAAAAAIA/CHAM5O5CRQUOAAAAAADwBwGOgajAAQAAAAAAgSDAMRAVOAAAAAAAIBAEOAZiEGMAAAAAABAIAhwDMY04AAAAAAAIBAGOgajAAQAAAAAAgSDAMRCDGAMAAAAAgEAQ4BiIQYwBAAAAAEAgCHAMRAUOAAAAAAAIBAGOgajAAQAAAAAAgSDAMRCDGAMAAAAAgEAQ4BiIacQBAAAAAEAgCHAMRAUOAAAAAAAIBAGOgajAAQAAAAAAgSDAMRAVOAAAAAAAIBAEOAZiGnEAAAAAABAIAhwDMY04AAAAAAAIBAGOgajAAQAAAAAAgSDAMRAVOAAAAAAAIBAEOAZiEGMAAAAAABAIAhwDMY04AAAAAAAIBAGOgajAAQAAAAAAgSDAMRCDGAMAAAAAgEAQ4BiIQYwBAAAAAEAgCHAMRAUOAAAAAAAIBAGOgajAAQAAAAAAgSDAMRAVOAAAAAAAIBAEOAaiAgcAAAAAAASCAMdATCMOAAAAAAACQYBjIHcFDl2oAAAAAACAPwhwDEQFDgAAAAAACAQBjoEYxBgAAAAAAASCAMdADGIMAAAAAAACQYBjICpwAAAAAABAIAhwDEQFDgAAAAAACAQBjoEYxBgAAAAAAASCAMdATCMOAAAAAAACQYBjICpwAAAAAABAIAhwDEQFDgAAAAAACAQBjoGowAEAAAAAAIEgwDEQ04gDAAAAAIBAEOAYiGnEAQAAAABAIAhwDEQFDgAAAAAACAQBjoGowAEAAAAAAIEgwDEQgxgDAAAAAIBAEOAYiGnEAQAAAABAIAhwDEQFDgAAAAAACAQBjoEYxBgAAAAAAASCAMdADGIMAAAAAAACQYBjICpwAAAAAABAIAhwDEQFDgAAAAAACAQBjoGowAEAAAAAAIEIKMB54403NGrUKKWlpWnq1KnavHlzg+sXFhZq7ty5yszMVGpqqsaMGaOVK1cG1OCWjAocAAAAAAAQCL9LQZYtW6Z58+Zp7ty5Sk9P18KFCzVz5kwtX75cCQkJtdavqKjQ1VdfrYSEBD399NNq166dsrOz1apVqyZ5AC0J04gDAAAAAIBA+B3gLFiwQJdddpkmT54sSZo7d65WrFihxYsX67rrrqu1/uLFi3X06FG99dZbCg0NlSR17tz5BJvdMrkrcOhCBQAAAAAA/OFXF6qKigpt3bpVw4YNO74Bq1XDhg3Thg0b6rzNZ599pgEDBuj+++/XsGHDdPHFF+v555/3VKOcSqjAAQAAAAAAgfArwCkoKJDdbq/VVSohIUF5eXl13mbv3r36+OOPZbfb9Y9//EM33HCDFixYoL///e+Bt7qFYhBjAAAAAAAQiGZPEpxOpxISEvTAAw/IZrMpNTVVOTk5evnll3XTTTc1990HFQYxBgAAAAAAgfArwGnTpo1sNpvy8/O9rs/Pz1diYmKdt0lKSlJISIhXaHHaaacpNzdXFRUVCgsLC6DZLRMVOAAAAAAAIBB+daEKCwtTv379tGbNGs91DodDa9asUUZGRp23GThwoLKysuRwODzX7d69W0lJSadUeCNRgQMAAAAAAALjV4AjSVdffbX+85//6J133tEvv/yiv/zlLyotLdWkSZMkSbNnz9YTTzzhWf/Xv/61jhw5or/+9a/atWuXVqxYoRdeeEFXXHFF0z2KFoJBjAEAAAAAQCD87sszduxYHT58WM8884xyc3PVp08fvfTSS54uVAcOHJDVejwX6tChg15++WXNmzdP48ePV7t27TRjxgxde+21TfcoWgCn00kXKgAAAAAAEBCL0+l0mt2IxhQVFWnQoEFat26dYmJizG5OQOx2uye4ycvLqzWTFwAAAAAAQH387kKFwLjHv5GowAEAAAAAAP4hwDGIu/uUxBg4AAAAAADAPwQ4Bqke4FCBAwAAAAAA/EGAY5DqXaiowAEAAAAAAP4gwDEIXagAAAAAAECgCHAM4q7AsVgsXtOsAwAAAAAANIYkwSDuChyqbwAAAAAAgL8IcAzirsBhAGMAAAAAAOAvAhyDUIEDAAAAAAACRYBjEHeAQwUOAAAAAADwFwGOQdxdqKjAAQAAAAAA/iLAMQgVOAAAAAAAIFAEOAahAgcAAAAAAASKAMcgVOAAAAAAAIBAEeAYhAocAAAAAAAQKAIcgzCNOAAAAAAACBQBjkHcFTh0oQIAAAAAAP4iwDEIFTgAAAAAACBQBDgGYRBjAAAAAAAQKAIcgzCIMQAAAAAACBQBjkGowAEAAAAAAIEiwDEIFTgAAAAAACBQBDgGYRBjAAAAAAAQKAIcgzCNOAAAAAAACBQBjkGowAEAAAAAAIEiwDEIgxgDAAAAAIBAEeAYhEGMAQAAAABAoAhwDEIFDgAAAAAACBQBjkGowAEAAAAAAIEiwDEIFTgAAAAAACBQBDgGoQIHAAAAAAAEigDHIEwjDgAAAAAAAkWAYxB3BQ5dqAAAAAAAgL8IcAxCBQ4AAAAAAAgUAY5BGMQYAAAAAAAEigDHIAxiDAAAAAAAAkWAYxAqcAAAAAAAQKAIcAxCBQ4AAAAAAAgUAY5BqMABAAAAAACBIsAxCBU4AAAAAAAgUAQ4BmEacQAAAAAAECgCHIO4K3DoQgUAAAAAAPxFgGMQKnAAAAAAAECgCHAMwiDGAAAAAAAgUAQ4BmEQYwAAAAAAECgCHINQgQMAAAAAAAJFgGMQKnAAAAAAAECgKAcxCIMYAwAAAED97Ha7KisrzW4G0ORCQ0ObJAsgwDEI04gDAAAAQG1Op1MHDx7UkSNHzG4K0Gxat26t9u3by2KxBLwN0gSDUIEDAAAAALW5w5u2bdsqKirqhA5wgWDjdDpVUlKiQ4cOSZI6dOgQ8LYIcAzCIMYAAAAA4M1ut3vCm4SEBLObAzSLyMhISdKhQ4fUtm3bgAs7GMTYIAxiDAAAAADe3GPeREVFmdwSoHm5X+MnMs4TAY5BqMABAAAAgLrRbQonu6Z4jRPgGIQKHAAAAAAAECgCHINQgQMAAAAAAAJFgGMQKnAAAAAAAPUZNWqUXn31VZ/X/+abb5SSkqLCwsLmaxSCCuUgBmEacQAAAABo+VJSUhr8+0033aSbb77Z7+0uWrTIM1uRLzIyMrRq1SrFxsb6fV+BuvDCC7Vv3z59/vnnSkpKMux+4UKAYxB3BQ5dqAAAAACg5Vq1apXn/LJly/TMM89o+fLlnuuqz6jldDplt9t9Og6Mj4/3qx1hYWGGhihr165VeXm5xowZo3feeUfXXXedYfddl8rKSoWGhpraBqPRhcogVOAAAAAAgG+cTqeKi4sNW5xOp89tS0pK8iyxsbGyWCyeyzt37tTAgQO1cuVKTZo0SWlpaVq3bp2ysrI0a9YsDRs2TBkZGZo8ebJWr17ttd2aXahSUlL09ttv68Ybb1R6erouuOAC/e9///P8vWYXqiVLlmjw4MH68ssvddFFFykjI0MzZ87UoUOHPLepqqrSgw8+qMGDB2vIkCF67LHHNGfOHN1www2NPu7Fixfr4osv1oQJE7R48eJafz948KBuv/12nXnmmRowYIAmTZqkTZs2ef7+2WefafLkyUpLS9OQIUN04403ej3WTz/91Gt7gwcP1pIlSyRJ+/btU0pKipYtW6Zp06YpLS1NH3zwgQoKCnT77bdrxIgRSk9P1yWXXKKlS5d6bcfhcOjFF1/U6NGjlZqaqnPOOUd///vfJUkzZszQ/fff77X+4cOHlZqaqjVr1jT6nBiNAMcgDGIMAAAAAI1zOp3KzMxUTEyMYcuIESP8CnEa88QTT+iOO+7QsmXLlJKSopKSEo0cOVKvvvqq3nnnHY0YMULXX3+9srOzG9zO/PnzddFFF+n999/X2WefrTvvvFNHjhypd/2ysjK98sorevTRR/XPf/5TBw4c0COPPOL5+4svvqgPPvhA8+bN05tvvqmioqJawUldioqKtHz5co0fP17Dhw9XUVGR1q5d6/l7cXGxpk2bppycHD333HN67733dM0118jhcEiSVqxYoZtuukkjR47Uu+++q4ULF6p///6N3m9Njz/+uGbMmKFly5YpMzNTFRUV6tevn/7xj39o6dKluuyyyzR79mxt3rzZc5snnnhCL774om644QYtW7ZMjz/+uBITEyVJU6dO1dKlS1VRUeFZ//3331fbtm111lln+d2+5kaaYBAGMQYAAAAA31gsFrObcEJuueUWDR8+3HO5devW6t27t+fyrbfeqk8//VSfffaZpk2bVu92Jk6cqIsvvliSdPvtt+v111/X5s2bdfbZZ9e5fmVlpebOnasuXbpIkq644go999xznr//85//1HXXXafRo0dLku6991598cUXjT6eZcuWqWvXrjr99NMlSWPHjtWiRYs0ePBgSdLSpUt1+PBhLVq0SK1bt5Ykde3a1XP7559/XmPHjtUtt9ziua768+GrK6+8UhdccIHXdTNnzvScnz59ulatWqWPPvpI/fv3V1FRkV577TXde++9mjhxoiSpS5cunnZfcMEFeuCBB/Tpp59q7NixklyVTJMmTQrK1yABjkGowAEAAACAxlksFn355ZcqKSkx7D6joqKa9IA9LS3N63JxcbHmz5+vFStWKDc3V3a7XWVlZY1W4FQfMDkqKkoxMTE6fPhwvetHRkZ6whtJatu2rfLz8yVJx44dU15enlfli81mU79+/TyVMvVZvHixxo8f77k8fvx4TZ8+XX/+858VExOj7du3q2/fvp7wpqbt27dr6tSpDd6HL1JTU70u2+12Pf/881q+fLlycnJUWVmpiooKRURESJJ27typioqKeqtpwsPDNX78eC1evFhjx47V1q1btWPHDk8Xq2BDmmAQKnAAAAAAwDcWi0XR0dFmNyNgNWeTeuSRR7R69WrNmTNHXbp0UUREhG655RZVVlY2uJ2ag/RaLJYGw5aaBQMWi+WEu4b9/PPP2rhxozZv3qzHH3/cc73dbteyZct02WWXeQKT+jT297ra6T6Grq76ANGS9PLLL+u1117T3XffrZSUFEVGRuqhhx7yPK/h4eEN3q/k6kZ16aWX6uDBg1qyZInOOussderUqdHbmYExcAxCBQ4AAAAAnJo2bNigiRMnavTo0UpJSVFiYqL2799vaBtiY2OVmJioLVu2eK6z2+3atm1bg7dbtGiRzjjjDL333nt69913PcvVV1+tRYsWSXJVCm3fvr3e8Xl69erV4KDA8fHxXoMt7969W6WlpY0+pvXr1+u8887ThAkT1Lt3byUnJ2v37t2ev3fr1k0RERH6+uuv691GSkqKUlNT9Z///EdLly7V5MmTG71fs5AmGIQKHAAAAAA4NXXt2lWffPKJRo0aJYvFoqeeeqrRbkvNYdq0aXrhhRfUpUsXnXbaafrnP/+po0eP1tt9rLKyUu+9955uueUW9erVy+tvU6dO1YIFC7Rjxw6NGzdOzz//vG688Ubdfvvtatu2rbZt26a2bdsqIyNDN910k6666ip16dJF48aNU1VVlVauXOmZivyss87SG2+8oYyMDNntdj3++OM+TRHetWtXffzxx1q/fr3i4uK0YMEC5eXlqUePHpJcFTjXXnutHnvsMYWGhmrgwIE6fPiwduzY4dWla+rUqbr//vsVFRXlGR8oGFGBYxCmEQcAAACAU9Ndd92lVq1a6fLLL9f111+vESNGqF+/foa349prr9XFF1+sOXPm6PLLL1dUVJQyMzPr7Wr02Wef6ciRI3WGGj169FCPHj20aNEihYWF6ZVXXlFCQoKuu+46XXLJJfrHP/7hOf4dMmSInn76aX322WeaMGGCrrzySq9KoDlz5qhDhw664oordOedd+q3v/1to92uJGnWrFnq27evZs6cqenTpysxMVHnn3++1zo33HCDrr76aj3zzDMaO3asbrvttlrjCI0bN04hISEaN26cT92uzGJxNuVcac2kqKhIgwYN0rp16xQTE2N2cwLStm1b5ebmasuWLbUGXgIAAACAU1FZWZl27dql7t27+3TAjqblcDh00UUX6aKLLtKtt95qdnNMs2/fPo0ePVqLFi1qtmCtKV7rdKEyCBU4AAAAAAAz7d+/X1999ZXOOOMMVVRU6I033tD+/ft1ySWXmN00U1RWVurIkSN66qmnlJ6ebkpVlD8IcAzCIMYAAAAAADNZrVYtWbJEjzzyiJxOp3r16qUFCxZ4xow51axfv14zZsxQt27d9Mwzz5jdnEaRJhiEQYwBAAAAAGbq0KGD3nrrLbObETSGDBmiH3/80exm+IxBjA1CBQ4AAAAAAAgUAY5BqMABAAAAAACBIsAxCIMYAwAAAACAQNGfxyAdOnRQUVGR4uLizG4KAAAAAABoYQhwDPLNN9+ovLxckZGRZjcFAAAAAAC0MAQ4BuncubPZTQAAAAAAAC0UY+AAAAAAAGCw6dOn669//avn8qhRo/Tqq682eJuUlBR9+umnJ3zfTbUdGIsABwAAAAAAH11//fWaOXNmnX9bu3atUlJS9MMPP/i93UWLFulXv/rViTbPy7PPPqsJEybUun7VqlU6++yzm/S+6lNWVqYzzzxTQ4YMUUVFhSH3ebIiwAEAAAAAwEdTpkzR6tWrdfDgwVp/W7x4sVJTU9W7d2+/txsfH2/YmKlJSUkKCwsz5L4+/vhj9ezZU6eddprpVT9Op1NVVVWmtuFEEOAAAAAAAIKL0ykVFxu3OJ0+N+2cc85RfHy8lixZ4nV9cXGxli9frilTpqigoEC33367RowYofT0dF1yySVaunRpg9ut2YVq9+7duuKKK5SWlqaxY8fqq6++qnWbxx57TGPGjFF6errOO+88PfXUU6qsrJQkLVmyRPPnz9cPP/yglJQUpaSkeNpcswvVjz/+qBkzZqh///4aMmSI7rnnHhUXF3v+ftddd+mGG27Qyy+/rMzMTA0ZMkRz58713FdDFi1apPHjx2v8+PFatGhRrb/v2LFDv/vd7zRw4EBlZGToN7/5jbKysrxuP27cOKWmpiozM1P333+/JGnfvn1KSUnR9u3bPesWFhYqJSVF33zzjSTXZEIpKSlauXKlJk2apLS0NK1bt05ZWVmaNWuWhg0bpoyMDE2ePFmrV6/2aldFRYUee+wxjRw5UqmpqRo9erTefvttOZ1OjR49Wi+//LLX+tu3b1dKSor27NnT6HMSKAYxBgAAAAAED6dTysyUahxQN6vhw6Uvv5QslkZXDQkJ0YQJE/TOO+9o1qxZsvz/2yxfvlwOh0MXX3yxSkpK1K9fP1177bWKiYnRihUrNHv2bHXp0kX9+/dv9D4cDoduvvlmJSQk6O2339axY8f00EMP1VovOjpa8+bNU9u2bfXTTz/pnnvuUXR0tK699lqNHTtWO3bs0JdffqkFCxZIkmJjY2tto6SkRDNnzlRGRoYWLVqk/Px8/fnPf9YDDzyghx9+2LPeN998o6SkJC1cuFBZWVm67bbb1KdPH1122WX1Po6srCxt3LhR8+fPl9Pp1Lx587R//3516tRJkpSTk6Np06bpzDPP1MKFCxUTE6P169d7qmTefPNNPfzww7rjjjt09tln69ixY1q/fn2jz19NTzzxhObMmaPk5GS1atVKBw8e1MiRI3XbbbcpLCxM7777rq6//notX75cHTt2lCTNnj1bGzdu1J///Gf17t1b+/btU0FBgSwWiyZPnqwlS5Z4daVbvHixzjjjDHXt2tXv9vmKAAcAAAAAEFx8CFLMNHnyZL388sv69ttvNWTIEEmuipcLLrhAsbGxio2N9Tq4nz59ulatWqWPPvrIpwBn9erV2rlzp1566SW1a9dOknTbbbfp2muv9Vrvhhtu8Jzv3Lmzdu3apQ8//FDXXnutIiIiFBUVJZvNpqSkpHrva+nSpaqoqNAjjzyiqKgoSdK9996r66+/XnfeeacSExMlSXFxcbr33ntls9nUo0cPjRw5UmvWrGkwwFm8eLHOPvtsxcXFSZIyMzO1ZMkS3XzzzZKkN954QzExMXryyScVGhoqSerevbvn9n//+9919dVX68orr/Rc58vzV9Mtt9yi4cOHey63bt3aq5vbrbfeqk8//VSfffaZpk2bpl27dumjjz7SggULNGzYMElScnKyZ/2JEyfqmWee0ebNm9W/f39VVlZq6dKlmjNnjt9t8wcBDgAAAAAgeFgsrmqYkhLj7jMqyq/QqEePHsrIyNDixYs1ZMgQ7dmzR2vXrtVrr70mSbLb7Xr++ee1fPly5eTkqLKyUhUVFYqIiPBp+7/88ovat2/vCW8kKSMjo9Z6y5Yt02uvvaa9e/eqpKREVVVViomJ8flxuO8rJSXFE95I0sCBA+VwOLRr1y5PgNOzZ0/ZbDbPOklJSfrpp5/q3a7dbtc777yjP/3pT57rxo8fr0cffVQ33nijrFartm/frsGDB3vCm+ry8/N16NAhDR061K/HU5e0tDSvy8XFxZo/f75WrFih3Nxc2e12lZWVKTs7W5KrO5TNZtMZZ5xR5/batWunkSNHatGiRerfv78+//xzVVRU6MILLzzhtjaEAAcAAAAAEFwsFik62uxWNGjKlCl68MEHde+992rJkiXq0qWLzjzzTEnSyy+/rNdee0133323UlJSFBkZqYceesinMWN8tWHDBt155526+eablZmZqdjYWH344Yee7lJNLSTEOz6wWCxyNjB20KpVq5STk6PbbrvN63q73a41a9Zo+PDhDQZa4eHhDbbHanUN6Vu9DfUNUFxzcOhHHnlEq1ev1pw5c9SlSxdFRETolltu8fx/fAnapk6dqtmzZ+vuu+/WkiVLNHbs2GYfhJpBjAEAAAAA8NNFF10ki8WipUuX6t1339XkyZM94+GsX79e5513niZMmKDevXsrOTlZu3fv9nnbPXr00MGDB3Xo0CHPdRs3bvRaZ8OGDerYsaNmzZqltLQ0devWzVNB4hYaGiqHw9Hoff34448qqVbxtH79elmtVq/uTP5yDz787rvvei3jxo3zDGackpKitWvX1hlsxcTEqFOnTlqzZk2d24+Pj5ck5ebmeq6rPqBxQzZs2KCJEydq9OjRSklJUWJiovbv3+/5e69eveRwOPTdd9/Vu42RI0cqMjJS//rXv/Tll19q8uTJPt33iSDAAQAAAADAT9HR0Ro7dqyefPJJ5ebmauLEiZ6/de3aVatXr9b69ev1yy+/6N5771VeXp7P2x42bJi6deumu+66Sz/88IPWrl2r//u///Nap2vXrjpw4IA+/PBDZWVl6bXXXqs1TXenTp20b98+bd++XYcPH1ZFRUWt+7rkkksUFhamu+66Sz/99JO+/vprPfDAA5owYYKn+5S/Dh8+rM8//1yXXnqpevXq5bVMmDBBn376qY4cOaIrrrhCRUVFuv3227Vlyxbt3r1b7777rnbu3ClJuvnmm7VgwQK99tpr2r17t7Zu3arXX39dkqtKZsCAAfrHP/6hX375Rd9++62eeuopn9rXtWtXffLJJ9q+fbt++OEH3XHHHV5BV+fOnTVx4kTdfffd+vTTT7V371598803WrZsmWcdm82mSZMm6YknnlDXrl3r7OLW1AhwAAAAAAAIwJQpU3T06FFlZmZ6jVcza9Ys9e3bVzNnztT06dOVmJio888/3+ftWq1WzZ8/X2VlZZoyZYr+9Kc/1eqKdN555+nKK6/U/fffrwkTJmjDhg2aNWuW1zpjxozRiBEjNGPGDA0dOrTOqcwjIyP18ssv68iRI5oyZYp+//vfa+jQobrnnnv8fDaOe/fddxUZGVnn+DVDhw5VRESE3n//fbVp00YLFy5USUmJpk+frkmTJuntt9/2jInjDlHefPNNXXzxxfrd737nNU33Qw89JLvdrkmTJumhhx7Srbfe6lP77rrrLrVq1UqXX365rr/+eo0YMUL9+vXzWucvf/mLxowZo7/85S+66KKLdM8996i0tNRrnSlTpqiyslKTJk3y8xkKjMXZUKe1IFFUVKRBgwZp3bp1fg/IBAAAAAAITmVlZdq1a5e6d+/u8wC/QLBYu3atrrrqKq1YsaLRaqWmeK0ziDEAAAAAAICPKioqdPjwYT377LMaM2ZMwF3N/EUXKgAAAAAAAB8tXbpU5557ro4dO6bZs2cbdr9U4AAAAAAAAPho0qRJho17Ux0VOAAAAAAAAEGOAAcAAAAAYKrqUzgDJ6OmeI3ThQoAAAAAYIqwsDBZrVZlZ2crKSlJYWFhslgsZjcLaDJOp1MVFRXKzc2V1WpVWFhYwNsiwAEAAAAAmMJqtap79+46cOCAsrOzzW4O0GyioqLUpUsXWa2Bd4QiwAEAAAAAmCYsLExdunRRVVWV7Ha72c0BmpzNZlNISMgJV5cR4AAAAAAATGWxWBQaGqrQ0FCzmwIELQYxBgAAAAAACHIEOAAAAAAAAEGOAAcAAAAAACDItYgxcJxOpySpqKjI5JYAAAAAAAA0vejo6AYHOm4RAU5xcbEkaeTIkSa3BAAAAAAAoOmtW7dOMTEx9f7d4nSXtwQxh8OhQ4cONZpGAQAAAAAAtESNZR4tIsABAAAAAAA4lTGIMQAAAAAAQJAjwAEAAAAAAAhyBDgAAAAAAABBjgAHAAAAAAAgyBHgAAAAAAAABDkCHAAAAAAAgCBHgAMAAAAAABDkCHAAAAAAAACCHAEOAAAAAABAkCPAMcgbb7yhUaNGKS0tTVOnTtXmzZvNbhJO0AsvvKDJkycrIyNDQ4cO1Q033KCdO3d6rTN9+nSlpKR4Lffee69JLcaJePbZZ2v9Ly+88ELP38vLyzV37lwNGTJEGRkZuvnmm5WXl2dii3EiRo0aVev/nZKSorlz50pi327pvvvuO11//fXKzMxUSkqKPv30U6+/O51OPf3008rMzFT//v111VVXaffu3V7rHDlyRHfccYcGDhyowYMH6+6771ZxcbGBjwK+auj/XVlZqccee0yXXHKJBgwYoMzMTM2ePVs5OTle26jrPeEf//iH0Q8FPmhs/77rrrtq/S9nzpzptQ77d8vR2P+7rs/ylJQUvfTSS5512L9bBl+OvXz5Pp6dna3rrrtO6enpGjp0qB555BFVVVUZ+VBOSIjZDTgVLFu2TPPmzdPcuXOVnp6uhQsXaubMmVq+fLkSEhLMbh4C9O233+qKK65QWlqa7Ha7nnzySc2cOVMffvihoqKiPOtddtlluuWWWzyXIyMjzWgumsDpp5+uBQsWeC7bbDbP+YceekgrV67UU089pdjYWD3wwAO66aab9NZbb5nRVJygRYsWyW63ey7v2LFDV199tVdox77dcpWUlCglJUWTJ0/WTTfdVOvvL774ol5//XU9/PDD6ty5s55++mnNnDlTy5YtU3h4uCTpzjvvVG5urhYsWKDKykrdfffduvfee/XEE08Y/XDQiIb+32VlZdq2bZtmzZql3r17q7CwUH/96181a9YsLVmyxGvdW265RZdddpnncnR0tCHth38a278lacSIEZo3b57nclhYmNff2b9bjsb+36tWrfK6/MUXX+hPf/qTxowZ43U9+3fw8+XYq7Hv43a7Xb/73e+UmJiot956S4cOHdKcOXMUGhqq22+/3cyH5zsnmt2UKVOcc+fO9Vy22+3OzMxM5wsvvGBiq9DU8vPznb169XJ+++23nuumTZvmfPDBB01sFZrKM8884xw/fnydfyssLHT269fP+dFHH3mu+/nnn529evVybtiwwaAWojk9+OCDzvPPP9/pcDicTif79smkV69ezk8++cRz2eFwOIcPH+586aWXPNcVFhY6U1NTnUuXLnU6ncf3782bN3vWWblypTMlJcV58OBB4xoPv9X8f9dl06ZNzl69ejn379/vue7cc891LliwoJlbh6ZW1/97zpw5zlmzZtV7G/bvlsuX/XvWrFnOGTNmeF3H/t0y1Tz28uX7+IoVK5y9e/d25ubmetZ58803nQMHDnSWl5cb2v5A0YWqmVVUVGjr1q0aNmyY5zqr1aphw4Zpw4YNJrYMTe3YsWOSpLi4OK/rP/jgAw0ZMkQXX3yxnnjiCZWWlprRPDSBPXv2KDMzU+edd57uuOMOZWdnS5K+//57VVZWeu3nPXr0UMeOHbVx40aTWoumUlFRoffff1+TJ0+WxWLxXM++fXLat2+fcnNzvfbn2NhYpaenez63N2zYoFatWiktLc2zzrBhw2S1WukifRIoKiqSxWJRq1atvK5/8cUXNWTIEF166aV66aWXWlTJPbx9++23Gjp0qMaMGaP77rtPBQUFnr+xf5+88vLytHLlSk2ZMqXW39i/W56ax16+fB/fuHGjevXqpcTERM86mZmZKioq0s8//2xc408AXaiaWUFBgex2e62uUgkJCbX67KHlcjgceuihhzRw4ED16tXLc/3FF1+sjh07qm3btvrxxx/1+OOPa9euXZo/f76JrUUg+vfvr3nz5ql79+7Kzc3V3/72N11xxRX64IMPlJeXp9DQ0Fpf9hMSEpSbm2tSi9FUPv30Ux07dkwTJ070XMe+ffJy77N1fW67+9Hn5eUpPj7e6+8hISGKi4tjn2/hysvL9fjjj2vcuHGKiYnxXD99+nT17dtXcXFx2rBhg5588knl5ubqj3/8o4mtRSBGjBih0aNHq3Pnztq7d6+efPJJXXvttfr3v/8tm83G/n0Se+eddxQdHa0LLrjA63r275anrmMvX76P5+XleYU3kjyXW8r+TYADNIG5c+dqx44devPNN72u/9WvfuU5n5KSoqSkJF111VXKyspSly5djG4mTsDIkSM953v37q309HSde+65+uijjxQREWFiy9DcFi9erLPPPlvt2rXzXMe+DZx8Kisr9fvf/15Op9MzYLnb1Vdf7Tnfu3dvhYaG6r777tMdd9xRa/wUBLdx48Z5zrsHrD3//PM9VTk4eS1evFiXXHKJZywzN/bvlqe+Y69TAV2omlmbNm1ks9mUn5/vdX1+fn6t9A8t0/33368VK1Zo4cKFat++fYPrpqenS3J1xUHL1qpVK3Xr1k1ZWVlKTExUZWWlCgsLvdbJz89XUlKSSS1EU9i/f79Wr15dZ7l1dezbJw/3PtvQ53ZiYqIOHz7s9feqqiodPXqUfb6Fqqys1K233qrs7Gy98sorXtU3dUlPT1dVVZX27dtnUAvRXJKTk9WmTRvP+zf798lp7dq12rVrl6ZOndrouuzfwa2+Yy9fvo8nJibWmpXKfbml7N8EOM0sLCxM/fr105o1azzXORwOrVmzRhkZGSa2DCfK6XTq/vvv1yeffKKFCxcqOTm50dts375dUst5g0D9iouLtXfvXiUlJSk1NVWhoaFe+/nOnTuVnZ2tAQMGmNdInLAlS5YoISFB55xzToPrsW+fPDp37qykpCSv/bmoqEibNm3yfG5nZGSosLBQ33//vWedr7/+Wg6HQ/379ze8zTgx7vBmz549evXVV9WmTZtGb7N9+3ZZrVZmEz0JHDx4UEeOHPG8f7N/n5wWLVqkfv36qXfv3o2uy/4dnBo79vLl+/iAAQP0008/ef1Is3r1asXExKhnz56GPI4TRRcqA1x99dWaM2eOUlNT1b9/fy1cuFClpaWaNGmS2U3DCZg7d66WLl2q5557TtHR0Z5+k7GxsYqIiFBWVpY++OADjRw5Uq1bt9aPP/6oefPm6YwzzvDpwwPB5ZFHHtG5556rjh076tChQ3r22WdltVp18cUXKzY2VpMnT9bDDz+suLg4xcTE6MEHH1RGRgYBTgvmcDi0ZMkSXXrppQoJOf5xyb7d8hUXFysrK8tzed++fdq+fbvi4uLUsWNHzZgxQ3//+9/VtWtXzzTibdu21fnnny/JNSjiiBEjdM8992ju3LmqrKzUAw88oHHjxnl1tUNwaOj/nZSUpFtuuUXbtm3TCy+8ILvd7vk8j4uLU1hYmDZs2KBNmzbprLPOUnR0tDZs2KB58+Zp/PjxtSYugPka+n/HxcVp/vz5GjNmjBITE7V371499thj6tq1q0aMGCGJ/bulaez9XHKF8MuXL9ecOXNq3Z79u+Vo7NjLl+/jmZmZ6tmzp2bPnq0//OEPys3N1VNPPaUrrriixXSXszidTqfZjTgV/POf/9TLL7+s3Nxc9enTR3/+8589JfdomVJSUuq8ft68eZo0aZIOHDigP/zhD9qxY4dKSkrUoUMHnX/++brhhhsaLc1G8Lntttv03Xff6ciRI4qPj9egQYN02223ecY7KS8v18MPP6wPP/xQFRUVyszM1H333UdFRgu2atUqzZw5U8uXL1f37t0917Nvt3zffPONZsyYUev6iRMn6uGHH5bT6dQzzzyj//znPyosLNSgQYN03333eb0Ojhw5ogceeECfffaZrFarLrjgAv35z39WdHS0kQ8FPmjo/33TTTfpvPPOq/N2r732moYMGaKtW7dq7ty52rlzpyoqKtS5c2dNmDBBV199dYv5wn8qaej//Ze//EU33nijtm3bpmPHjqlt27YaPny4fv/733sNbcD+3XI09n4uSf/+97/10EMPadWqVYqNjfVaj/275Wjs2Evy7fv4/v379Ze//EXffvutIiMjNXHiRN1xxx1eP9YFMwIcAAAAAACAIMcYOAAAAAAAAEGOAAcAAAAAACDIEeAAAAAAAAAEOQIcAAAAAACAIEeAAwAAAAAAEOQIcAAAAAAAAIIcAQ4AAAAAAECQI8ABAAAAAAAIcgQ4AAAAAAAAQY4ABwAAAAAAIMgR4AAAAAAAAAS5/wfvAnhThB6PYQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1400x800 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Maximum validation accuracy: 0.983211874961853\n"
     ]
    }
   ],
   "source": [
    "plt.figure(figsize = (14, 8))\n",
    "sns.set_style('white')\n",
    "\n",
    "plt.plot(history.history['accuracy'], color='k', label=\"Training accuracy\")\n",
    "plt.plot(history.history['val_accuracy'], color='r', label=\"Validation accuracy\")\n",
    "\n",
    "plt.legend(['Training Accuracy', 'Validation Accuracy'])\n",
    "\n",
    "sns.despine()\n",
    "plt.show()\n",
    "\n",
    "print(\"Maximum validation accuracy: {}\".format(history_df['val_accuracy'].max()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "r5qFLw0V-Ppk"
   },
   "source": [
    "# Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "execution": {
     "iopub.execute_input": "2023-03-24T22:53:57.172659Z",
     "iopub.status.busy": "2023-03-24T22:53:57.172116Z",
     "iopub.status.idle": "2023-03-24T22:54:00.365992Z",
     "shell.execute_reply": "2023-03-24T22:54:00.364632Z",
     "shell.execute_reply.started": "2023-03-24T22:53:57.172621Z"
    },
    "id": "57kcTEx1-Ppk",
    "outputId": "71d9490b-c317-4f6b-ae27-21d536a26c83"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1093/1093 [==============================] - 4s 4ms/step\n"
     ]
    }
   ],
   "source": [
    "# predict results\n",
    "results = model.predict(test)\n",
    "\n",
    "# select the indix with the maximum probability\n",
    "results = np.argmax(results,axis = 1)\n",
    "\n",
    "results = pd.Series(results,name=\"Label\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "execution": {
     "iopub.execute_input": "2023-03-24T22:55:40.64284Z",
     "iopub.status.busy": "2023-03-24T22:55:40.642198Z",
     "iopub.status.idle": "2023-03-24T22:55:40.686404Z",
     "shell.execute_reply": "2023-03-24T22:55:40.68532Z",
     "shell.execute_reply.started": "2023-03-24T22:55:40.642795Z"
    },
    "id": "GnCgyzEf-Ppk"
   },
   "outputs": [],
   "source": [
    "submissions=pd.DataFrame({\"ImageId\": list(range(1,len(results)+1)),\n",
    "                         \"Label\": results})\n",
    "submissions.to_csv(\"submission.csv\", index=False, header=True)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "provenance": []
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}