RNN.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "RNN.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# Recurrent Neutral Network\n",
        "- Able to send *feedback signals* to each other.\n",
        "- Accept fixed-sized vector as an input and produce a fix-sized vector as output with sequences.\n",
        "\n",
        "# Vanishing Gradient Problem\n",
        "- Back-propagation: moving backward in network and calculating gradient of loss\n",
        "- *Earlier layers* are important for learning and detecting simple patterns\n",
        "- Solutions: \n",
        "  - Do not use Sigmoid and Tanh as an Activation Function\n",
        "  - We use **ReLU activation function** to improve accuracy\n",
        "\n",
        "# LSTM & GRU\n",
        "- Long short term memery and gated recurrent unit\n",
        "- If the sequences are so long, it is difficult for RNN carry information from earlier time steps to later ones.\n",
        "- Gates can regulate the flow of information."
      ],
      "metadata": {
        "id": "pxgZpzqMTeZn"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Spam Detection with RNN\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "pL8GOv0_rmFd"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 1)Data Pre-processing"
      ],
      "metadata": {
        "id": "B1tQ5QAesKnu"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import tensorflow as tf\n",
        "print(tf.__version__)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "BXlJvKwQrlmo",
        "outputId": "706e29b5-cc0a-45f6-d516-b14196f18abc"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "2.8.2\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "import matplotlib.pyplot as plt\n",
        "from sklearn.model_selection import train_test_split\n",
        "from tensorflow.keras.preprocessing.text import Tokenizer\n",
        "from tensorflow.keras.preprocessing.sequence import pad_sequences\n",
        "from tensorflow.keras.layers import Dense,Input,GlobalMaxPooling1D\n",
        "from tensorflow.keras.layers import LSTM, Embedding\n",
        "from tensorflow.keras.models import Model"
      ],
      "metadata": {
        "id": "_dLLuUJEsIxW"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df=pd.read_csv('/content/25153382-spam.csv',encoding='ISO-8859-1')"
      ],
      "metadata": {
        "id": "T7TaI_UPsI13"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "I6RIKkhqsI5i",
        "outputId": "114fabf0-ab97-476d-be26-21d3e5a0698b"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     v1                                                 v2 Unnamed: 2  \\\n",
              "0   ham  Go until jurong point, crazy.. Available only ...        NaN   \n",
              "1   ham                      Ok lar... Joking wif u oni...        NaN   \n",
              "2  spam  Free entry in 2 a wkly comp to win FA Cup fina...        NaN   \n",
              "3   ham  U dun say so early hor... U c already then say...        NaN   \n",
              "4   ham  Nah I don't think he goes to usf, he lives aro...        NaN   \n",
              "\n",
              "  Unnamed: 3 Unnamed: 4  \n",
              "0        NaN        NaN  \n",
              "1        NaN        NaN  \n",
              "2        NaN        NaN  \n",
              "3        NaN        NaN  \n",
              "4        NaN        NaN  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-a632c59e-aa5c-47c2-806c-a628774d5928\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>v1</th>\n",
              "      <th>v2</th>\n",
              "      <th>Unnamed: 2</th>\n",
              "      <th>Unnamed: 3</th>\n",
              "      <th>Unnamed: 4</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>ham</td>\n",
              "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>ham</td>\n",
              "      <td>Ok lar... Joking wif u oni...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>spam</td>\n",
              "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>ham</td>\n",
              "      <td>U dun say so early hor... U c already then say...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>ham</td>\n",
              "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-a632c59e-aa5c-47c2-806c-a628774d5928')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-a632c59e-aa5c-47c2-806c-a628774d5928 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-a632c59e-aa5c-47c2-806c-a628774d5928');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#drop garbage columns,we only keep v1 and v2 columns\n",
        "df=df.drop([\"Unnamed: 2\",\"Unnamed: 3\",\"Unnamed: 4\"],axis=1)"
      ],
      "metadata": {
        "id": "X0RWnBLwsdIo"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "3VcajtaqsdQM",
        "outputId": "543a3684-3c63-49c8-fc2a-cf1711a6a074"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "     v1                                                 v2\n",
              "0   ham  Go until jurong point, crazy.. Available only ...\n",
              "1   ham                      Ok lar... Joking wif u oni...\n",
              "2  spam  Free entry in 2 a wkly comp to win FA Cup fina...\n",
              "3   ham  U dun say so early hor... U c already then say...\n",
              "4   ham  Nah I don't think he goes to usf, he lives aro..."
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-77f34c24-f781-45ae-a594-fc9fbc10b140\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>v1</th>\n",
              "      <th>v2</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>ham</td>\n",
              "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>ham</td>\n",
              "      <td>Ok lar... Joking wif u oni...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>spam</td>\n",
              "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>ham</td>\n",
              "      <td>U dun say so early hor... U c already then say...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>ham</td>\n",
              "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-77f34c24-f781-45ae-a594-fc9fbc10b140')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-77f34c24-f781-45ae-a594-fc9fbc10b140 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-77f34c24-f781-45ae-a594-fc9fbc10b140');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#rename columns\n",
        "df.columns=['labels','data']"
      ],
      "metadata": {
        "id": "oINsj_onsqrF"
      },
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df.head()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "id": "yKUaLWGwsucv",
        "outputId": "dcd8387e-62b9-4b08-f8fa-1c21d1a038c9"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "  labels                                               data\n",
              "0    ham  Go until jurong point, crazy.. Available only ...\n",
              "1    ham                      Ok lar... Joking wif u oni...\n",
              "2   spam  Free entry in 2 a wkly comp to win FA Cup fina...\n",
              "3    ham  U dun say so early hor... U c already then say...\n",
              "4    ham  Nah I don't think he goes to usf, he lives aro..."
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-e223f39f-23da-407a-9c58-bc4218ce7ea2\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>labels</th>\n",
              "      <th>data</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>ham</td>\n",
              "      <td>Go until jurong point, crazy.. Available only ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>ham</td>\n",
              "      <td>Ok lar... Joking wif u oni...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>spam</td>\n",
              "      <td>Free entry in 2 a wkly comp to win FA Cup fina...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>ham</td>\n",
              "      <td>U dun say so early hor... U c already then say...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>ham</td>\n",
              "      <td>Nah I don't think he goes to usf, he lives aro...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-e223f39f-23da-407a-9c58-bc4218ce7ea2')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-e223f39f-23da-407a-9c58-bc4218ce7ea2 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-e223f39f-23da-407a-9c58-bc4218ce7ea2');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#create binary labels (0,1)-->create a new column as b_labels\n",
        "df['b_labels']=df['labels'].map({'ham':0,'spam':1}) #use label encoder with the dictionary\n",
        "y=df['b_labels'].values"
      ],
      "metadata": {
        "id": "nSjqE1vXsypR"
      },
      "execution_count": 9,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#split the data\n",
        "x_train,x_test,y_train,y_test=train_test_split(df['data'],y,test_size=0.33)"
      ],
      "metadata": {
        "id": "Yj3-MYo0syru"
      },
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#Convert sentence to sequences\n",
        "max_vocab_size=20000\n",
        "#Define a tokenizer object\n",
        "tokenizer=Tokenizer(num_words=max_vocab_size)\n",
        "#map our tokenizer objects to our sentences we created\n",
        "tokenizer.fit_on_texts(x_train)\n",
        "#handle the words appear in our training and testing datasets from text->sequence\n",
        "sequences_train=tokenizer.texts_to_sequences(x_train)\n",
        "sequences_test=tokenizer.texts_to_sequences(x_test)"
      ],
      "metadata": {
        "id": "4IHYLxmqs5tF"
      },
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#define our word index and print it out\n",
        "word2index=tokenizer.word_index\n",
        "V=len(word2index)\n",
        "print('Total number of unique tokens are:%s' % V)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "4_hq5qDcs5zU",
        "outputId": "e9a7c279-b792-43a6-f700-d9c007406321"
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Total number of unique tokens are:7189\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#Pad train sentences to get N*T matrix, N: # of samples; T:# of time steps\n",
        "data_train=pad_sequences(sequences_train)\n",
        "print('Shape of data train tensor:',data_train.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2WxupsWWtAsq",
        "outputId": "242f1789-47f7-4d2b-8ace-bc70f929e618"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Shape of data train tensor: (3733, 189)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#set the value of T to get sequence length, just the same as the first two checking results\n",
        "T=data_train.shape[1]"
      ],
      "metadata": {
        "id": "JnwtptB-tA11"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "print(T)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "6IAfALBftA8W",
        "outputId": "a6e4a815-6b46-4e25-e67e-182aba310231"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "189\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#Pad test sentences\n",
        "data_test=pad_sequences(sequences_test,maxlen=T)\n",
        "#maxlen=T, to truncate longer sentences in test set\n",
        "print('Shape of data test tensor:', data_test.shape)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "pj5PJfzbtJ3T",
        "outputId": "48552ab1-2264-4bf0-e84a-632ff0d43a20"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Shape of data test tensor: (1839, 189)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# 2)Build the Model"
      ],
      "metadata": {
        "id": "Slw1T2K2tYoZ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Choose the embedding dimensionality\n",
        "D=20  #the hyperparameter, we choose any word vector size that we want\n",
        "\n",
        "#Hidden state vectorization\n",
        "M=15\n",
        "\n",
        "#Input layer,every single record has been represented to be the 189 record\n",
        "i=Input(shape=(T,))\n",
        "\n",
        "#Embedding layer, return sequences of word vectors, V is unique token\n",
        "x= Embedding(V+1,D)(i) #it will return a N*T*D array, we want size of embedding to (V+1) *D, because first word index start from 1, not from 0\n",
        "\n",
        "#LSTM layer\n",
        "x=LSTM(M,return_sequences=True)(x) #we are not care about encoding decoding issue\n",
        "x=GlobalMaxPooling1D()(x) \n",
        "\n",
        "#Dense Layer\n",
        "x= Dense(1,activation='sigmoid')(x)\n",
        "\n",
        "model=Model(i,x)"
      ],
      "metadata": {
        "id": "_VSWTY3etb9o"
      },
      "execution_count": 17,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#compile the model,we deal with binary classification problem\n",
        "model.compile(optimizer='adam',loss='binary_crossentropy',metrics=['accuracy'])"
      ],
      "metadata": {
        "id": "gi7JyOX_tcBk"
      },
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "#train the model\n",
        "r=model.fit(x=data_train,y=y_train,epochs=10,validation_data=(data_test,y_test)) #10次迭代"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "W8EFJNzvtcK8",
        "outputId": "2c631492-a1c5-4420-a9a9-aa9a16675eae"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "117/117 [==============================] - 19s 125ms/step - loss: 0.4794 - accuracy: 0.8663 - val_loss: 0.4249 - val_accuracy: 0.8488\n",
            "Epoch 2/10\n",
            "117/117 [==============================] - 13s 113ms/step - loss: 0.3792 - accuracy: 0.8744 - val_loss: 0.4261 - val_accuracy: 0.8488\n",
            "Epoch 3/10\n",
            "117/117 [==============================] - 10s 87ms/step - loss: 0.3780 - accuracy: 0.8744 - val_loss: 0.4283 - val_accuracy: 0.8488\n",
            "Epoch 4/10\n",
            "117/117 [==============================] - 10s 87ms/step - loss: 0.3781 - accuracy: 0.8744 - val_loss: 0.4264 - val_accuracy: 0.8488\n",
            "Epoch 5/10\n",
            "117/117 [==============================] - 11s 98ms/step - loss: 0.3773 - accuracy: 0.8744 - val_loss: 0.4237 - val_accuracy: 0.8488\n",
            "Epoch 6/10\n",
            "117/117 [==============================] - 10s 87ms/step - loss: 0.3728 - accuracy: 0.8744 - val_loss: 0.4160 - val_accuracy: 0.8488\n",
            "Epoch 7/10\n",
            "117/117 [==============================] - 10s 87ms/step - loss: 0.3210 - accuracy: 0.8744 - val_loss: 0.2786 - val_accuracy: 0.8488\n",
            "Epoch 8/10\n",
            "117/117 [==============================] - 10s 88ms/step - loss: 0.2014 - accuracy: 0.8754 - val_loss: 0.1929 - val_accuracy: 0.8912\n",
            "Epoch 9/10\n",
            "117/117 [==============================] - 16s 135ms/step - loss: 0.1361 - accuracy: 0.9740 - val_loss: 0.1494 - val_accuracy: 0.9690\n",
            "Epoch 10/10\n",
            "117/117 [==============================] - 10s 88ms/step - loss: 0.1020 - accuracy: 0.9863 - val_loss: 0.1243 - val_accuracy: 0.9799\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#Evaluation: loss per iteration\n",
        "import matplotlib.pyplot as plt\n",
        "plt.plot(r.history['loss'],label='loss')\n",
        "plt.plot(r.history['val_loss'],label='validation loss')\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "ZZWFMqC0usWk",
        "outputId": "a3a6930a-677e-40cb-edb9-32118aab0352"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3dd3wVVf7/8dfn3jQSCDXUBBIg9BYIiFQRpFhQxAKKCrZVwYa64rqu5ev+7MiqKIJiVxbBgquAuIsCSgsQegvNJLQESChJSDu/P+YCAVJukptMcvN5Ph555N6ZM5NPrvKeyZmZc8QYg1JKKe/lsLsApZRSZUuDXimlvJwGvVJKeTkNeqWU8nIa9Eop5eV87C7gQvXq1TPh4eF2l6GUUpXKmjVrko0xIfmtq3BBHx4eTkxMjN1lKKVUpSIi+wpap103Sinl5TTolVLKy2nQK6WUl6twffRKqfKXlZVFQkICGRkZdpeiihAQEEBoaCi+vr5ub6NBr5QiISGBGjVqEB4ejojYXY4qgDGGI0eOkJCQQEREhNvbadeNUoqMjAzq1q2rIV/BiQh169Yt9l9eGvRKKQAN+UqiJP+dvCbo0zNzeHn+NuKPptldilJKVSheE/TH0jL5bPle/vbtRnSMfaUqn+rVq9tdgtfymqBvXKsaTw5rw9KdyXyzNtHucpRSqsLwmqAHGHNJM6Kb1eaF/2wh6cRpu8tRSpWAMYYnnniCDh060LFjR/79738DcODAAfr160eXLl3o0KEDS5cuJScnh7Fjx55t++abb9pcfcXkVbdXOhzCyyM7ceW/lvLcD5uZektXu0tSqtJ5/ofNbNl/3KP7bNc4mGevae9W22+++YbY2FjWr19PcnIy3bt3p1+/fnz55ZcMGTKEp59+mpycHNLS0oiNjSUxMZFNmzYBkJKS4tG6vYVXndEDtKxfnYcGtuTHDQdYtOWQ3eUopYpp2bJljB49GqfTSYMGDejfvz+rV6+me/fufPTRRzz33HNs3LiRGjVq0Lx5c3bv3s2DDz7IggULCA4Otrv8CsmrzujP+Ev/FvxnwwH+/t1GLmleh+AA958gU6qqc/fMu7z169ePJUuW8OOPPzJ27FgmTpzI7bffzvr161m4cCHTpk1j9uzZzJw50+5SKxyvO6MH8HU6ePWGTiSdOM3L87fZXY5Sqhj69u3Lv//9b3JyckhKSmLJkiX06NGDffv20aBBA+655x7uvvtu1q5dS3JyMrm5uYwcOZIXX3yRtWvX2l1+heSVZ/QAnUJrcVefCGYs3cPwzo3p2byu3SUppdwwYsQIli9fTufOnRERXn31VRo2bMgnn3zCa6+9hq+vL9WrV+fTTz8lMTGRcePGkZubC8BLL71kc/UVk1S0e86jo6ONpyYeSc/MYciUJTgdwvyH+xLg6/TIfpXyNlu3bqVt27Z2l6HclN9/LxFZY4yJzq+9V3bdnFHNz8lL13dkT/Ip/vXfnXaXo5RStvDqoAfo3bIeN0WHMn3JbjYlptpdjlJKlTuvD3qAp69sR50gPyZ9s4HsnFy7y1FKqXLlVtCLyFAR2S4icSIyqZB2I0XEiEi06324iKSLSKzra5qnCi+OmoG+vDC8PZsSj/PBsj12lKCUUrYpMuhFxAlMBYYB7YDRItIun3Y1gIeBlRes2mWM6eL6us8DNZfIsI6NGNK+AW8u2sGe5FN2laGUUuXOnTP6HkCcMWa3MSYTmAVcm0+7/wNeASrsXGQvXNsBPx8Hk+ZuIDe3Yt1tpJRSZcWdoG8CxOd5n+BadpaIdAXCjDE/5rN9hIisE5HfRKRvyUstvQbBATx9ZVtW7jnKrNXxRW+glKqwzgxrvH//fm644YZ821x22WUUdbv2lClTSEs7N4/FlVde6ZExc5577jlef/31Uu/HE0p9MVZEHMBk4LF8Vh8AmhpjooCJwJcictFgFCJyr4jEiEhMUlJSaUsq1M3dw7i0eV1e+mkrB1Mr7B8fSik3NW7cmDlz5pR4+wuD/qeffqJWrVqeKK3CcCfoE4GwPO9DXcvOqAF0AH4Vkb1AT2CeiEQbY04bY44AGGPWALuAVhf+AGPMdGNMtDEmOiQkpGS/iZtEhJeu70hWbi7PfL9JJylRqgKYNGkSU6dOPfv+zNnwyZMnGThwIF27dqVjx458//33F227d+9eOnToAEB6ejqjRo2ibdu2jBgxgvT09LPt7r//fqKjo2nfvj3PPvssAG+99Rb79+9nwIABDBgwAIDw8HCSk5MBmDx5Mh06dKBDhw5MmTLl7M9r27Yt99xzD+3bt2fw4MHn/Zz8xMbG0rNnTzp16sSIESM4duzY2Z/frl07OnXqxKhRowD47bff6NKlC126dCEqKooTJ06U6DPNy50hEFYDkSISgRXwo4Bbzqw0xqQC9c68F5FfgceNMTEiEgIcNcbkiEhzIBLYXeqqSym8XhATr2jF//tpGz9tPMhVnRrZXZJSFcf8SXBwo2f32bAjDHu5wNU333wzjzzyCOPHjwdg9uzZLFy4kICAAL799luCg4NJTk6mZ8+eDB8+vMB5U9977z0CAwPZunUrGzZsoGvXc0OV//Of/6ROnTrk5OQwcOBANmzYwEMPPcTkyZNZvHgx9erVO29fa9as4aOPPmLlypUYY7jkkkvo378/tWvXZufOnXz11VfMmDGDm266iblz5zJmzJgCf7/bb7+dt99+m/79+/OPf/yD559/nilTpvDyyy+zZ88e/P39z3YXvf7660ydOpXevXtz8uRJAgIC3P6YC1LkGb0xJhuYACwEtgKzjTGbReQFERlexOb9gA0iEgvMAe4zxhwtbdGecGfvCDo2qcmz8zaRkpZpdzlKVWlRUVEcPnyY/fv3s379emrXrk1YWBjGGP72t7/RqVMnBg0aRGJiIocOFTz8+JIlS84GbqdOnejUqdPZdbNnz6Zr165ERUWxefNmtmzZUmhNy5YtY8SIEQQFBVG9enWuv/56li5dCkBERARdunQBoFu3buzdu7fA/aSmppKSkkL//v0BuOOOO1iyZMnZGm+99VY+//xzfHys8+7evXszceJE3nrrLVJSUs4uLw239mCM+Qn46YJl/yig7WV5Xs8F5paivjLj43TwyshODH9nGS/+uJXXb+xsd0lKVQyFnHmXpRtvvJE5c+Zw8OBBbr75ZgC++OILkpKSWLNmDb6+voSHh5ORUfxra3v27OH1119n9erV1K5dm7Fjx5ZoP2f4+/uffe10OovsuinIjz/+yJIlS/jhhx/45z//ycaNG5k0aRJXXXUVP/30E71792bhwoW0adOmxLVCFXkytiDtGgfzl/7NmbMmgaU7y/YisFKqcDfffDOzZs1izpw53HjjjYB1Nly/fn18fX1ZvHgx+/btK3QfZ2aiAti0aRMbNmwA4Pjx4wQFBVGzZk0OHTrE/Pnzz25To0aNfPvB+/bty3fffUdaWhqnTp3i22+/pW/f4t84WLNmTWrXrn32r4HPPvuM/v37k5ubS3x8PAMGDOCVV14hNTWVkydPsmvXLjp27MiTTz5J9+7d2bat9EOte+0wxe568PJI5m86yFPfbOTnR/sR6FflPxKlbNG+fXtOnDhBkyZNaNTIum526623cs0119CxY0eio6OLPLO9//77GTduHG3btqVt27Z069YNgM6dOxMVFUWbNm0ICwujd+/eZ7e59957GTp0KI0bN2bx4sVnl3ft2pWxY8fSo0cPAO6++26ioqIK7aYpyCeffMJ9991HWloazZs356OPPiInJ4cxY8aQmpqKMYaHHnqIWrVq8cwzz7B48WIcDgft27dn2LBhxf55F/LqYYrdtWrPUW56fzl39YngmasveuhXKa+nwxRXLjpMcQn0iKjDmJ5N+ej3PcTG6+TCSinvokHv8uTQNjQIDuDJORvIzNYRLpVS3kM7pF1qBPjy4nUduOuTGKb9touHBkaW3w/PzYXsdMjK8OD3DMjNBqcv+ASA08/67uPv+sq7zC//Nk7/PO3PvA/IZ5k/FHBfs6o8jDEF3p+uKo6SdLd7Z9Dn5lghl5MFuVnW+5wsa1luFuRkn3udm+16n8VA/2z+2jKe2MXr2B/YisY1fFz7ysqzv+zCX595n50BWelFfHcFc04p7uN3+oNvAPhUu/i7wweyM+H0Ccg+fe4r54LXnuC88GCR571vNfANtL77BVmv/YLOLb/odaDrdeDF6x06HWRZCAgI4MiRI9StW1fDvgIzxnDkyJFiP0TlPUF/Mgkmt7WClpJfYH4ArE9lYTE2EqcVqk5f67vDxwomn4Dzw9c/uOBQ9vHPs42b330CwFHK3jdjrANNdoZ1UMjOyPP+zAEhI582pws+eJx972qflQYZqXDigPU6M831/RTF/m915vf3DTp3IMh7UMj3AOFqG1gPaoZCrTAIqFm6z83LhIaGkpCQQFmPNaVKLyAggNDQ0GJt4z1B7xcEvSa4gtYXnD55XvtaZ4IO34sDOZ/X/9t5lNcW7eYvl0VyXbeIwvfl8Cl92NpJ5Fw3THkzxjoonAn9rLSLDwRZ6ZB16tyyfNenQdpRyEo4f31WWsE/2z8YaoZZoV8z1HpdMxRqNbVeV29Quf+7FpOvry8RERF2l6HKiBcFfSAMes4juxoQavh472r+9vtRons0JrR2oEf2qy4gYv1F4xsAgXU8v//cXFcXmeugcCoJUuMhJd76nppgvf5zufUXR14OX6jZxHUAuPCA4HrtW/oxSJQqD3offQESjqUx+M0ldA+vw8fjumu/pbfLOG4Ff2oCpP7pOhgknDswnDjARd1MQfXPdQXld0CoVlsvUqtyU9h99N5zRu9hobUD+euQ1jz3wxa+i01kRFTx+sRUJRMQDAHtoEEBD8zlZMHxxHN/BaTGnzsIHNoCOxZafz3k5Vf9gm6hMKjVDNpeY09XmaqyNOgLcdul4cxbv58XfthCv8gQ6lbXf5xVltMXaodbX/kxBk4ln38AOPMXQWo8JK6BdNfArVFj4Nqp+e9HqTKgXTdF2HnoBFe9tYyhHRry1ugou8tRldnpk/Dz07DuC3g41jrLV8pDdAiEUohsUIPxA1oyb/1+/ret4HGwlSqSf3Xo+xiYXFj+rt3VqCpEg94N91/WgtYNavD0t5s4kZFldzmqMqvVFDreCGs+tm4JVaocaNC7wc/HwcsjO3LweAavLthudzmqsuvziPVswKrpdleiqggNejdFNa3NuF4RfLZiH6v36pmYKoX6baH1lbBymuvpYKXKlltBLyJDRWS7iMSJyKRC2o0UESMi0XmWPeXabruIDPFE0XZ5fEgrQmtX48m5G8jIyrG7HFWZ9XkU0o/Bmk/srkRVAUUGvYg4ganAMKAdMFpELrrZWERqAA8DK/MsaweMAtoDQ4F3XfurlAL9fPh/IzqyO+kUb/9vp93lqMosrAc06w3L37HGA1KqDLlzRt8DiDPG7DbGZAKzgGvzafd/wCtA3qdGrgVmGWNOG2P2AHGu/VVa/VqFMLJrKO//tpst+4/bXY6qzPpMtB7C2vi13ZUoL+dO0DcB4vO8T3AtO0tEugJhxpgfi7uta/t7RSRGRGIqw+h5z1zdllqBvjw5dwPZOTpJiSqhlgOhQUf4fYo1Lo9SZaTUF2NFxAFMBh4r6T6MMdONMdHGmOiQkJDSllTmagX68dzw9mxMTGXm73vsLkdVViLWHTjJO2D7hedISnmOO0GfCITleR/qWnZGDaAD8KuI7AV6AvNcF2SL2rbSuqpjIwa1bcDkRTvYd0TvnFAl1O46qB0By960hlFQqgy4E/SrgUgRiRARP6yLq/POrDTGpBpj6hljwo0x4cAKYLgxJsbVbpSI+ItIBBAJrPL4b2EDEeHF6zrg63Dw1DcbSzS9l1I4faD3Q9ZYOHuX2l2N8lJFBr0xJhuYgDXn0lZgtjFms4i8ICLDi9h2MzAb2AIsAMYbY7zmvsSGNQN46sq2/LHrCLNj4oveQKn8dL7Fmuhk6WS7K1FeSgc1K6XcXMPoGSvYcuA4/53Yn/rBOhmFKoFlU+CXZ+HeX6GxDp6nik8HNStDDofw8shOZGbn8o/vN9tdjqqsou8E/5pW4CvlYRr0HhBRL4hHBrViweaDLNh0wO5yVGUUEAzd74It30NynN3VKC+jQe8h9/SNoH3jYJ75fjOpaTrCpSqBnvdbM0/98S+7K1FeRoPeQ3ycDl4Z2YmjpzJ58cctnDqdzensHHJzK9Y1EFWBVa9vzT4V+xUc3293NcqL6FSCHtShSU3u6ducab/t4us1CWeXO8Q6EPg6BKdD8HU68HEKPg4Hvk7Bx+nAJ89yX4f1/WzbfLe5eHsf13a+znOv87rwuvuFF+LzOyRdtM1F6907kDnE+n0cDsEpgtNxbtnZL8m7/txrh4OLljkdkmf7PPvPs88L9x/g48DHWcHPbXo9CDEfwfKpMOSfdlejvIQGvYdNvKIVzUOCOHYqk+xcQ1ZOLtk5hqxc63tOPsuyc3PJyjFk5+Se3SYz23qd7WqT5VqX9/WZ/ZzZXhWtRoAPtQJ9qVXNj1qBvtSs5pvPez/XMl9qupb5+5TTWHy1w6HDSGtikr6PQWCd8vm5yqtp0HuYn4+Dm6LDim5YBs4eRHLPHTTkgjYi5y+5eP3F+5ULW134tvDVGKzbUHNyDTnGkJuL67u1LDvXkGtc6/O8tr5z3rK8251dn2fZmdf57TMtM4eUtCxS07NIScskJT2LxGPppLjeF9bLVs3Xmf+BIc/rMweGs+8Dfanm67zoMy9Sn0dg42xY/SH0f6J42yqVDw16L2J1UVTaUaBtlZtrOJmZTWpaFilpWaSkZ7q+Z5Gadu61daDIZFfSSdcBI4vMQga283M6XOHvOkAE+nHLJU0Z0Lp+wcU0aA+RQ2Dle3DpePALLIPfWFUlGvRKYT0PERzgS3CAL2HF6C0xxpCRlXvuwOA6EFx4YDizbnNiKnd+vJqJg1oxfkBLHI4Czvb7PAofDYV1n8Elf/HML6mqLA16pUpBRKjm56SaXzUa1axWZPuMrBye+mYjbyzaweb9x3n9ps5U98/nn2GzS6HppfDH29bDVE7fMqheVRUV/BYEpbxLgK+TyTd15u9XtWXR1kOMmPo7e5ILGP20z6OQGg8b55RvkcrraNArVc5EhLv7NuezO3uQfPI0w99ZxuLthy9uGDkY6rfXiUlUqWnQK2WTXi3rMW9CH8JqB3Lnx6uZujju/OcSRKyz+qRtsGOBfYWqSk+DXikbhdUJZO79vbimU2NeW7id8V+u5dTp7HMN2o+AWk1h2WSdmESVmAa9Ujar5ufkX6O68PSVbVmw6SDXv/vHuVnLnD7Q6yFIWA37fre3UFVpadArVQGICPf0a84nd/bg0IkMhr/zO7/tSLJWRo2BoBBrukGlSkCDXqkKpG9kCPPG96FRzQDGfbSKab/twvgEWCNbxv0CBzbYXaKqhNwKehEZKiLbRSRORCbls/4+EdkoIrEiskxE2rmWh4tIumt5rIhM8/QvoJS3aVo3kG8e6MWwjo14ef42HvxqHWmdx4JfDT2rVyVS5ANTIuIEpgJXAAnAahGZZ4zZkqfZl8aYaa72w4HJwFDXul3GmC6eLVsp7xbo58M7o6Po0Lgmry7cRtzhk8zueAfBa9+FI3+Hui3sLlFVIu6c0fcA4owxu40xmcAs4Nq8DYwxx/O8DSL/EW+VUsUgItx/WQs+HteD/SnpXL+2E7niYz0tq1QxuBP0TYD4PO8TXMvOIyLjRWQX8CrwUJ5VESKyTkR+E5G++f0AEblXRGJEJCYpKakY5Svl/fq3CmHehD44ghsyK6svOWs/xxzXKSuV+zx2MdYYM9UY0wJ4Evi7a/EBoKkxJgqYCHwpIsH5bDvdGBNtjIkOCQnxVElKeY3wekF8+0BvtjYfC7nZ/Pfj50nPzLG7LFVJuBP0iUDeAdZDXcsKMgu4DsAYc9oYc8T1eg2wC2hVslKVqtqC/H14Yew17Ko/mEuOfMftU38m/mia3WWpSsCdoF8NRIpIhIj4AaOAeXkbiEhknrdXATtdy0NcF3MRkeZAJLDbE4UrVRWJCK2u/zs1JJ0+qd8x/J1l/BGXbHdZqoIrMuiNMdnABGAhsBWYbYzZLCIvuO6wAZggIptFJBari+YO1/J+wAbX8jnAfcaYox7/LZSqShp1gpZXMKHaIhoFwW0zV/Hhsj1uz9+rqh6paP9zREdHm5iYGLvLUKpi2/s7fHwlGYNf5eFd3Vi4+RAjoprw0vUdCfDVWcaqIhFZY4yJzm+dPhmrVGXUrBeE9iBg1Tu8N7ozj13Riu9iE7lh2h8kpqTbXZ2qYDTolaqMzgxhnPInji3f8uDASD64PZp9yWkMf3sZK3YfsbtCVYFo0CtVWbUaCiFtrWERjGFg2wZ8N6E3tQJ9ufWDlXz8u/bbK4sGvVKVlcMBfR6Bw1tgx0IAWoRU57vxvRnQuj7P/bCFx7/eQEaW3m9f1WnQK1WZdRgJNZueN9hZjQBfpt/WjUcGRTJ3bQI3vb+c/dpvX6Vp0CtVmTl9odeDEL8C9i0/u9jhEB4Z1Irpt3Vjd9Iphr+zjJXab19ladArVdlFjYHAuvkOYTy4fUO+G9+L4ACr3/7T5Xu1374K0qBXqrLzC4RL7oedC+HgpotWt6xfg+8m9KZfqxD+8f1mnpyr/fZVjQa9Ut6gx93gVx1+n5Lv6uAAXz64PZoHL2/J7JgEbp6+gmOnMsu5SGUXDXqlvEG12hA9DjbNhaN78m3icAiPDW7NtDFd2ZyYyhuLtpdzkcouGvRKeYue48FR9MQkQzs0YnSPpny1Kp7dSSfLqThlJw16pbxFcCPoPBrWfQ4nDxfa9MGBLfH3cfDGzzvKqThlJw16pbxJ74chJxNWvFdos/o1Ari7b3N+3HiA2PiUcipO2UWDXilvUrcFtLsWVn8AGamFNr23X3PqBvnx8vytesull9OgV8rb9HkUTh+HmJmFNqvu78NDAyNZsfsov+7QuZq9mQa9Ut6mcRdocTksfxeyMgptOrpHU5rWCeSV+dvIydWzem+lQa+UN+rzKJw6DLFfFNrMz8fB40Nas+3gCb6PLWwqaFWZuRX0IjJURLaLSJyITMpn/X0islFEYkVkmYi0y7PuKdd220VkiCeLV0oVILwvNImGP96CnOxCm17dsREdm9TkjZ936BOzXqrIoHdN7j0VGAa0A0bnDXKXL40xHY0xXYBXgcmubdthTSbeHhgKvHtmsnClVBk6MzHJsb2w5btCmzocwqRhbUhMSefzFfvKpz5Vrtw5o+8BxBljdhtjMoFZwLV5Gxhjjud5GwSc6ey7FphljDltjNkDxLn2p5Qqa62vhHqtYNkUKOKumt4t69E3sh7vLI4jNT2rnApU5cWdoG8CxOd5n+Badh4RGS8iu7DO6B8q5rb3ikiMiMQkJenVf6U8wuGA3o/AoY0Q90uRzZ8c2oaUtCze/21XORSnypPHLsYaY6YaY1oATwJ/L+a2040x0caY6JCQEE+VpJTqeCMEh+Y7hPGFOjSpybVdGjPz9z0cTC38bh1VubgT9IlAWJ73oa5lBZkFXFfCbZVSnuTjB70mwL7f4c+VRTZ/fHBrcnINU37RoRG8iTtBvxqIFJEIEfHDurg6L28DEYnM8/YqYKfr9TxglIj4i0gEEAmsKn3ZSim3db0dqtVx66w+rE4gY3o2Y3ZMPHGHT5RDcao8FBn0xphsYAKwENgKzDbGbBaRF0RkuKvZBBHZLCKxwETgDte2m4HZwBZgATDeGKP3bylVnvyC4JL7YMd8OLSlyOYTBrQk0M+HVxfoMMbeQiraGBfR0dEmJibG7jKU8i5pR+HNDtD2Grj+/SKbv/3fnbyxaAdz7ruU6PA65VCgKi0RWWOMic5vnT4Zq1RVEFgHuo2FjV/DsaLvlb+rbwQhNfx5ef42HfDMC2jQK1VVXDoexAHL3ymyaaCfD48MiiRm3zF+2Vr42Paq4tOgV6qqqNkEOt8Maz+Fk0U/r3JTdBjN6wXx6oJtZOfklkOBqqxo0CtVlfR+BLJPw8ppRTb1dTp4Ykhrdh4+ydy1CeVQnCorGvRKVSX1Iq0LsqtnQMbxIpsP7dCQLmG1eHPRTtIz9Ya5ykqDXqmqps+j1uxTi54psqmINeDZweMZfPzH3rKvTZUJDXqlqpomXa0unDUfw4avi2zes3ldLm9Tn3d/jSMlLbPs61Mep0GvVFV0+d8hrCf85xFI3llk878Obc3J09lMXRxXDsUpT9OgV6oqcvrCDTPBxx9m3wGZaYU2b9MwmJFdQ/nkj30kHCu8rap4NOiVqqpqNoER0+HwZpj/1yKbP3pFKxB4c1HRfwGoikWDXqmqLHIQ9H0M1n0G62cV2rRJrWqM7RXON+sS2Haw6Dt2VMWhQa9UVXfZ36BZb/jPo5BU+EBmD1zWghr+Prwyf1s5Fac8QYNeqarO6QMjPwTfQFd//akCm9YK9OOBAS1ZvD2J5buOlGORqjQ06JVSENwIRs6ApG3w0xOFNh3bK5yGwQG8vEAHPKssNOiVUpYWl0O/JyD2C1j3RYHNAnydTLyiFevjU1iw6WA5FqhKSoNeKXXOZZMgvC/8+Bgc3lpgs5HdQmnVoDqvLtxOlg54VuFp0CulznE4YeQH4F/D6q8/fTLfZk6H8NchbdiTfIp/r44v5yJVcbkV9CIyVES2i0iciEzKZ/1EEdkiIhtE5L8i0izPuhwRiXV9zbtwW6VUBVOjodVfn7zDOrMvoB9+YNv6dA+vzZRfdnLqdHY5F6mKo8igFxEnMBUYBrQDRotIuwuarQOijTGdgDnAq3nWpRtjuri+hqOUqviaX2Z142yYZd1jn48zA54lnzzNzGV7yrU8VTzunNH3AOKMMbuNMZnALODavA2MMYuNMWeei14BhHq2TKVUuev3hBX4Pz0Bhzbn26RbszoMad+A95fs5sjJ0+VannKfO0HfBMjbCZfgWlaQu4D5ed4HiEiMiKwQkevy20BE7nW1iUlKKnrmG6VUOXA44foZEFDT1V9/It9mTwxpQ1pmNm//Twc8q6g8ejFWRMYA0cBreRY3c81MfgswRURaXLidMUlRM0oAABKxSURBVGa6MSbaGBMdEhLiyZKUUqVRvb71MNXRXdaTs/n017esX52bu4fxxcp9/HlEBzyriNwJ+kQgLM/7UNey84jIIOBpYLgx5uzfcMaYRNf33cCvQFQp6lVKlbeIvtYwCRu/tsawz8cjg1rhdAhvLCp8CAVlD3eCfjUQKSIRIuIHjALOu3tGRKKA97FC/nCe5bVFxN/1uh7QG9jiqeKVUuWk72PWA1Xzn4QDGy5a3SA4gLv6RPB97H42JabaUKAqTJFBb4zJBiYAC4GtwGxjzGYReUFEztxF8xpQHfj6gtso2wIxIrIeWAy8bIzRoFeqsnE4rP76wDrw9dh855v9S/8W1Ar05ZUFOuBZRSMVbayK6OhoExMTY3cZSqn87PsDPr4K2l1nTVwict7qD5bu5sUft/LZXT3oG6nX28qTiKxxXQ+9iD4Zq5RyX7Ne1jSEm7+BmA8vWn3bpc1oUqsaryzYRm5uxTqJrMo06JVSxdP7UWh5BSx4CvbHnrfK38fJ40NasSnxOP/ZeMCmAtWFNOiVUsXjcMCI9yEoxNVff/7F12s7N6Fto2BeX7idzGwd8Kwi0KBXShVfUF2rjz7lT5j34Hn31zscwpNDW/Pn0TS+XLnPxiLVGRr0SqmSadoTBv4DtnwPq2act6p/qxAubV6Xt/4Xx4mMLJsKVGdo0CulSq7XQxA5BBb+DRLXnl18ZsCzo6cymbFUBzyzmwa9UqrkHA4YMQ2qN7D669NTzq7qHFaLqzo14oOluzl8IsO+GpUGvVKqlALrwI0fw/FE+H78ef31jw9uTWZ2Lm/9d6d99SkNeqWUB4R1h0HPw7b/wMppZxdH1AtidI+mfLUqnt1J+c9WpcqeBr1SyjMuHQ+tr4Sfn4GENWcXPzQwEn8fB2/8vMPG4qo2DXqllGeIwHXvQo1GVn992lEAQmr4c0/f5vy48QCx8SmF70OVCQ16pZTnVKtt9defOHBef/09/ZpTr7ofL/20lYo2vlZVoEGvlPKs0G4w+P9g+0+wfCoA1f19eGhgJCv3HOXXHTqLXHnToFdKed4l90Gbq+GXZyF+NQCjujelWd1AXpm/jRwd8KxcadArpTxPBK6dCsFNzvbX+/k4eHxwa7YdPMF36y6apE6VIQ16pVTZqFbL6q8/dRi+vQ9yc7mqYyM6NqnJ5EU7yMjKsbvCKkODXilVdpp0hcH/hJ0LYfnbOBzCU8PakJiSzucrdMCz8uJW0IvIUBHZLiJxIjIpn/UTRWSLiGwQkf+KSLM86+4QkZ2urzs8WbxSqhLocQ+0uxZ+eR7+XEGvlvXo1yqEdxbHkZquA56VhyKDXkScwFRgGNAOGC0i7S5otg6INsZ0AuYAr7q2rQM8C1wC9ACeFZHanitfKVXhicDwt6FWU/h6HJw6wpNDW5OSlsW033bZXV2V4M4ZfQ8gzhiz2xiTCcwCrs3bwBiz2BiT5nq7Agh1vR4CLDLGHDXGHAMWAUM9U7pSqtIIqGn116clw7f30r5hDa7r0piZy/awKTG1yM1V6bgT9E2A+DzvE1zLCnIXML+E2yqlvFXjLjD0JYj7BX6fwqRhbalX3Z8xH65ky/7jdlfn1Tx6MVZExgDRwGvF3O5eEYkRkZikJH2YQimvFX0XtL8e/vciDVPWMuvengT6Orn1gxVsPaBhX1bcCfpEICzP+1DXsvOIyCDgaWC4MeZ0cbY1xkw3xkQbY6JDQkLcrV0pVdmIwDX/gtrhMOdOwvxO8dW9PQnwdXLrByvZfvCE3RV6JXeCfjUQKSIRIuIHjALm5W0gIlHA+1ghfzjPqoXAYBGp7boIO9i1TClVVQUEu/rrj8I399As2MlX9/TE1yncMmMFOw9p2HtakUFvjMkGJmAF9FZgtjFms4i8ICLDXc1eA6oDX4tIrIjMc217FPg/rIPFauAF1zKlVFXWqBNc+RrsXgzT+xN+ejtf3dMTp0MYPWMlcYd17HpPkoo2klx0dLSJiYmxuwylVHnY+QvMexBOHoI+jxLX7gFGfbgOEZh1b09ahFS3u8JKQ0TWGGOi81unT8YqpewTOQgeWA6dR8HS12n57TV8MyIIYwyjp6/QWak8RINeKWWvarWsCUtGz4K0ZJrOvZqFXf5AcrIYPWMFe5NP2V1hpadBr5SqGFoPgwdWQPvrqRszmd/qvEiz7L2MnrGCfUc07EtDg14pVXEE1oGRM+DmzwlIP8S/mcQtmV8z5v3fiT+aVvT2Kl8a9EqpiqftNfDASqTt1TxovmJa5iQmTftaw76ENOiVUhVTUF3rfvsbPqK1/1FmZj7GvHefJPGoXqAtLg16pVTF1uF6fB5cRUazgYzP/pRj7wzg0O6NdldVqWjQK6Uqvur1qTl2Fvsu+xdhuYnU+vRyji+eArk6S5U7NOiVUpWDCM0uG8ufoxaz3HQg+LdnyfxwGBzRMe2LokGvlKpUOrZpTY1xc3kq9wEyEzdh3usNK6dDbq7dpVVYGvRKqUqnW3gdRt75BFfnvs4q0w7mPwGfDodjOg9tfjTolVKVUnR4HV4dN5RxWU/wRsAEcvevg/d6QcxMqGBjeNlNg14pVWn1iKjDzLE9+OBUX27z/xeZjbrCfx6Fz0ZASnzRO6giNOiVUpVaz+Z1mTm2O2tSghie8jinrngd4ldZZ/drP9OzezTolVJe4NIWdfnwju7sOZLGyNWtSR37GzTqDPMmwBc3wvH9dpdoKw16pZRX6N2yHh/cEc3u5FOMnnOQlBvnwLDXYN/vMLUnxH5VZc/uNeiVUl6jb2QIM26PJi7pJGNmria14zi4bxk0aAff3QezboETh+wus9xp0CulvEr/ViG8f1s3dhw8yW0zV5Ia2BTG/ghDXoJd/4N3L4GNc6rU2b1bQS8iQ0Vku4jEicikfNb3E5G1IpItIjdcsC7HNY/s2blklVKqLA1oXZ9pt3Vl64Hj3D5zFcczc+HSB6yz+7otYe5dMPs2OJlkd6nlosigFxEnMBUYBrQDRotIuwua/QmMBb7MZxfpxpgurq/h+axXSimPu7xNA967tRtb9qdyx8xVnMjIgnqRcOdCuOIF2PGzdXa/+Tu7Sy1z7pzR9wDijDG7jTGZwCzg2rwNjDF7jTEbAH0GWSlVYQxq14Cpt3RlY0IqYz9azcnT2eBwQu+H4S9LoFYz+PoOmHUr7Fnqtd057gR9EyDvkwcJrmXuChCRGBFZISLX5ddARO51tYlJSqoaf0oppcrH4PYNeeeWKGLjUxj30SpOnc62VtRvA3ctgsufsUL+k6vh7a6wdDKcOGhv0R5WHhdjmxljooFbgCki0uLCBsaY6caYaGNMdEhISDmUpJSqSoZ2aMTbo6NY+2cK4z5eTVqmK+ydPtDvcXhsG4yYDjUaw3+fh8nt4KtbYPt8yMm2t3gP8HGjTSIQlud9qGuZW4wxia7vu0XkVyAK0HFFlVLl6sqOjcjJNTw8ax13fryaj8b2oJqf01rpFwidb7a+kuNg3WcQ+yVs/xGqN4SoWyFqDNRpbu8vUULunNGvBiJFJEJE/IBRgFt3z4hIbRHxd72uB/QGtpS0WKWUKo1rOjfmzZu7sGrPUe76ZDXpmflMXFKvJVzxPEzcAjd/YT1hu+xNeCsKPrnGujUzK6P8iy8FMW5cfBCRK4EpgBOYaYz5p4i8AMQYY+aJSHfgW6A2kAEcNMa0F5FewPtYF2kdwBRjzIeF/azo6GgTExNTql9KKaUK8926RCbOjqVXC+tp2gBfZ+EbHN8PsV9YY+ek7IOAWtDpZuh6OzTsUD5FF0FE1ri6yS9e507QlycNeqVUefhmbQKPfb2ePi3rMeN2N8IerMlN9i6BtZ/C1h8gJxMad7UCv8NICAgu+8ILoEGvlFL5+Domnr/O3UDPiLq8OKIDLUKqu79x2lHY8G8r9A9vAd9AaH+9FfphPUCk7ArPhwa9UkoVYO6aBP7x/SYysnO5oWsoDw+KpHGtau7vwBhIXGMF/qa5kHkS6rWyAr/zaAiqV3bF56FBr5RShThy8jRTF+/i8xX7QOD2ns14YEBL6gT5FW9Hp0/C5m+t0E9YBQ5faHOlFfrNB1gPa5URDXqllHJDYko6//plB3PWJBDo58PdfSO4u29zqvu7cyf6BQ5vtS7erv8K0o9CzTDrFs0ut0KtsKK3LyYNeqWUKoa4wyd44+cdzN90kDpBfowf0JJbL2nq3gXbC2Wfhu0/WWf5uxZby1pcbp3lt74SfIr5V0MBNOiVUqoE1sen8NrC7SyLS6ZJrWo8PCiS66Oa4OMs4aACx/ZZt2mu+xyOJ0JgXasfv+vtENK6VLVq0CulVCn8HpfMqwu2sT4hlRYhQTwxpDVD2jdESnpnTW6ONTb+2k+sYRZysyHsEuh6B3S5pUR37GjQK6VUKRljWLj5EK//vJ24wyfpHFqTvw5tQ++Wpbyr5uRhWD/L6toJbgR3/FCi3WjQK6WUh+TkGr5Zm8CUX3aSmJJO75Z1+euQNnQOq1W6HRsD6ccgsE6JNtegV0opDzudncMXK/5k6uI4jpzKZGj7hjw+pBUt69ewpR4NeqWUKiMnT2fz4dI9zFi6m7TMbEZ2DeWRK1rRpDgPXXmABr1SSpWxo6cyee/XOD5Zvg8M3NqzKeMHtKRedf9y+fka9EopVU72p6Tz1n93Mjsmnmq+Tu7q25x7+kZQI8C3TH+uBr1SSpWzXUknmfzzDn7ceIDagb6MH9CSMT2bleyhKzdo0CullE02JqTy6sJtLN2ZTKOaATwyKJKRXUNL/tBVAQoL+vKYM1YppaqsjqE1+eyuS/jynktoEBzAk3M3MnjKEn7aeIDyOtHWoFdKqXLQq0U9vn2gF9Nv64ZThAe+WMvwd35nyY6kMg98t4JeRIaKyHYRiRORSfms7ycia0UkW0RuuGDdHSKy0/V1h6cKV0qpykZEGNy+IQse6ccbN3bmWFomt89cxS0zVrLuz2Nl93OLOpKIiBPYAVwBJGBNFj7aGLMlT5twIBh4HJhnjJnjWl4HiAGiAQOsAboZYwr8jbSPXilVVZzOzuGrlX/yzuI4kk9mclXHRrxzS1SJxtAprI/enUGWewBxxpjdrp3NAq4Fzga9MWava13uBdsOARYZY4661i8ChgJfFfN3UEopr+Pv42Rs7whujA5j5rI9ZGTnlHygtEK4E/RNgPg87xOAS9zcf37bNrmwkYjcC9wL0LRpUzd3rZRS3iHI34cHB0aW2f4rxMVYY8x0Y0y0MSY6JCTE7nKUUsqruBP0iUDeea9CXcvcUZptlVJKeYA7Qb8aiBSRCBHxA0YB89zc/0JgsIjUFpHawGDXMqWUUuWkyKA3xmQDE7ACeisw2xizWUReEJHhACLSXUQSgBuB90Vks2vbo8D/YR0sVgMvnLkwq5RSqnzoEAhKKeUFdAgEpZSqwjTolVLKy2nQK6WUl6twffQikgTsK8Uu6gHJHiqnstPP4nz6eZxPP49zvOGzaGaMyfdBpAoX9KUlIjEFXZCoavSzOJ9+HufTz+Mcb/8stOtGKaW8nAa9Ukp5OW8M+ul2F1CB6GdxPv08zqefxzle/Vl4XR+9Ukqp83njGb1SSqk8NOiVUsrLeU3QFzWvbVUiImEislhEtojIZhF52O6a7CYiThFZJyL/sbsWu4lILRGZIyLbRGSriFxqd012EpFHXf9ONonIVyISYHdNnuYVQe+a13YqMAxoB4wWkXb2VmWrbOAxY0w7oCcwvop/HgAPY42+quBfwAJjTBugM1X4cxGRJsBDQLQxpgPgxBqK3at4RdCTZ15bY0wmcGZe2yrJGHPAGLPW9foE1j/ki6ZwrCpEJBS4CvjA7lrsJiI1gX7AhwDGmExjTIq9VdnOB6gmIj5AILDf5no8zluC3q25aasiEQkHooCV9lZiqynAX4ELJ6+viiKAJOAjV1fWByISZHdRdjHGJAKvA38CB4BUY8zP9lbled4S9CofIlIdmAs8Yow5bnc9dhCRq4HDxpg1dtdSQfgAXYH3jDFRwCmgyl7Tcs18dy3WAbAxECQiY+ytyvO8Jeh1btoLiIgvVsh/YYz5xu56bNQbGC4ie7G69C4Xkc/tLclWCUCCMebMX3hzsIK/qhoE7DHGJBljsoBvgF421+Rx3hL0pZnX1uuIiGD1wW41xky2ux47GWOeMsaEGmPCsf6/+J8xxuvO2NxljDkIxItIa9eigcAWG0uy259ATxEJdP27GYgXXpz2sbsATzDGZIvImXltncBMY8xmm8uyU2/gNmCjiMS6lv3NGPOTjTWpiuNB4AvXSdFuYJzN9djGGLNSROYAa7HuVluHFw6HoEMgKKWUl/OWrhullFIF0KBXSikvp0GvlFJeToNeKaW8nAa9Ukp5OQ16pZTychr0Sinl5f4/XmCU/QYUTO4AAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#Accuracy score\n",
        "plt.plot(r.history['accuracy'],label='accuracy')\n",
        "plt.plot(r.history['val_accuracy'],label='validation accuracy')\n",
        "plt.legend()\n",
        "plt.show() "
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        },
        "id": "PRJVLA2SuuZA",
        "outputId": "7f9219d7-e909-4a7a-a86f-5c6e3f3721f0"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deXzU5bn38c+VhBACAQIEMAskKmtCMBACorKIKFhFxSJaN6jLsW6ntT4eXKoerI+eij7U6ukp7XGhWi0vrGtZFAGxihLWsIQAQoAsQmBCIATIMtfzxwxxiIEMZMJvMrnerxcvZn7LzDUDfLlz/+7ffYuqYowxJnSFOV2AMcaYpmVBb4wxIc6C3hhjQpwFvTHGhDgLemOMCXERThdQV5cuXTQ5OdnpMowxpllZtWrVPlWNq29f0AV9cnIyK1eudLoMY4xpVkRk58n2WdeNMcaEOAt6Y4wJcRb0xhgT4oKuj74+VVVVFBQUcPToUadLMUEiKiqKxMREWrVq5XQpxgS9ZhH0BQUFxMTEkJycjIg4XY5xmKqyf/9+CgoKSElJcbocY4Jes+i6OXr0KJ07d7aQNwCICJ07d7af8IzxU7MIesBC3pzA/j4Y479mE/TGGBOq3G5l3vpi3lmxq0lev1n00RtjTChSVb7YUsKMT/PYUHiQQT06cuOQpID/xGpBH2Sqq6uJiLA/FmNCXXa+ixcW5LEi30VSpza8dMNArrkgoUm6Ja3r5jRce+21DB48mNTUVGbNmgXAggULGDRoEAMHDmTMmDEAlJeXM3XqVAYMGEB6ejrvvfceAO3atat9rblz5zJlyhQApkyZwj333MPQoUN55JFHWLFiBRdeeCEZGRkMHz6cvLw8AGpqanj44YdJS0sjPT2dP/zhDyxevJhrr7229nU/++wzrrvuurPxdRhjzsCGwjKmvL6CSf+znPz9h3nm2jQ+f2gUEwclEh7WNNeeml3T8T8/3simooMBfc3+8e156urUBo977bXX6NSpE0eOHGHIkCFcc8013HXXXSxbtoyUlBRcLhcAzzzzDB06dGD9+vUAlJaWNvjaBQUFfP3114SHh3Pw4EG+/PJLIiIiWLRoEY899hjvvfces2bNIj8/n7Vr1xIREYHL5SI2NpZ7772XkpIS4uLieP311/n5z3/euC/EGBNw35WU89KnW/jn+mI6tGnFtPF9uf3CZNpEhjf5eze7oHfSyy+/zPvvvw/A7t27mTVrFiNGjKgdy92pUycAFi1axLvvvlt7XmxsbIOvPWnSJMLDPX/gZWVl3H777WzduhURoaqqqvZ177nnntqunePvd+utt/LWW28xdepUli9fzuzZswP0iY0xjVVQWsHLn29l7qoColqF8+Cl53PniHNpH3X2bvZrdkHvT8u7KSxdupRFixaxfPlyoqOjGTVqFBdccAGbN2/2+zV8+97qjgFv27Zt7ePf/OY3jB49mvfff5/8/HxGjRp1ytedOnUqV199NVFRUUyaNMn6+I0JAiWHjvHqkm387dtdIDD1ohR+Meo8urRrfdZr8auPXkTGiUieiGwTkWn17O8pIp+LSI6ILBWRRJ99vxORjSKSKyIvSzMdAF1WVkZsbCzR0dFs3ryZb775hqNHj7Js2TJ27NgBUNt1M3bsWF599dXac4933XTr1o3c3FzcbnftTwYne6+EhAQA3njjjdrtY8eO5U9/+hPV1dUnvF98fDzx8fH89re/ZerUqYH70MaY01Z2pIoXFm5mxO+W8NdvdnL94ASWPjyK31zV35GQBz+CXkTCgVeB8UB/4CYR6V/nsBnAbFVNB6YDz3nPHQ5cBKQDacAQYGTAqj+Lxo0bR3V1Nf369WPatGkMGzaMuLg4Zs2axcSJExk4cCCTJ08G4IknnqC0tJS0tDQGDhzIkiVLAHj++ee56qqrGD58OOecc85J3+uRRx7h0UcfJSMjozbUAe6880569OhBeno6AwcO5G9/+1vtvptvvpmkpCT69evXRN+AMeZUKiqr+e+l27jkvxbz6pLvuKx/NxY9NJLnJqYT37GNo7WJqp76AJELgadV9Qrv80cBVPU5n2M2AuNUdbe3xV6mqu29574CXAwIsAy4VVVzT/Z+mZmZWnfhkdzcXAuwBtx///1kZGRwxx13OF3KWWN/L0wwOFZdw7srdvOHxdvYV36MMX278uvL+9A/vv3pvVD5XigrgIRBZ1SHiKxS1cz69vnTmZsA7PZ5XgAMrXPMOmAi8HvgOiBGRDqr6nIRWQIU4wn6V+oLeRG5G7gboEePHn6UZHwNHjyYtm3b8uKLLzpdijEtRnWNm/fXFDJz0VYKDxxhaEon/nTrIAb37HR6L7T/O/j6D7D2b9ApBe79BoL0hqmHgVdEZAqeVnshUCMi5wP9gON99p+JyCWq+qXvyao6C5gFnhZ9gGpqMVatWuV0Cca0GG63smDj97z4aR7flRwmPbEDz18/gIvP73J6NzsVroJ/zYTcjyE8Ei64CS58IOAhD/4FfSGQ5PM80butlqoW4WnRIyLtgOtV9YCI3AV8o6rl3n3zgQuBE4LeGGOCXd3pCnp1bcf/3DKYK1K7+R/wqrBtEXz1e8j/Elp3gIt/BUPvgZhuTVa7P0GfDfQSkRQ8AX8j8DPfA0SkC+BSVTfwKPCad9cu4C4ReQ5P181IYGaAajfGmLPCd7qCxNg2vDhpINdmJPh/J2tNFWz4hyfg926EmHi4/LcweAq0jmnS2sGPoFfVahG5H1gIhAOvqepGEZkOrFTVj4BRwHMioni6bu7znj4XuBRYDyiwQFU/DvzHMMaYwNtQWMaMT/NYmldCXExrnrkmlclDehAZ4efsMcfKYc1fYfmrULYb4vrCtX+EtJ9CRGTTFu/Drz56VZ0HzKuz7Umfx3PxhHrd82qAf2tkjcYYc1Z9V1LOS59t4Z85ZzhdQXkJrJjl+XX0APQYDlfOgF6XQ9jZn2LMJjVrIscnMCsqKuKnP/1pvceMGjWKukNJ65o5cyYVFRW1z6+88koOHDgQuEKNMbUKSit4ZO46xr70BUs27+XBS89n2SOjuWfkef6FvGs7fPIQzEyDZS9A8sVwx2fw8/nQZ5wjIQ/NcAqE5iY+Pp65c3/0w47fZs6cyS233EJ0dDQA8+bNa+CM4KKqqCphDv0FN8YfjZ6uoGiNp/9904cQFgEDb/SMoInr3bSF+8n+9flh2rRpJ0xp8PTTTzNjxgzKy8sZM2YMgwYNYsCAAXz44Yc/Ojc/P5+0tDQAjhw5wo033ki/fv247rrrOHLkSO1xv/jFL8jMzCQ1NZWnnnoK8EyiVlRUxOjRoxk9ejQAycnJ7Nu3D4CXXnqJtLQ00tLSmDlzZu379evXj7vuuovU1FQuv/zyE97nuI8//pihQ4eSkZHBZZddxp49e4CTT7Fc33TMx7+H49LS0sjPzyc/P58+ffpw2223kZaWxu7du+v9fADZ2dkMHz6cgQMHkpWVxaFDhxgxYgRr166tPebiiy9m3bp1fv95GXM6lubtZeQLnukKJg46jekKVGHb5/DmBJg1yvN4+IPw7zkw4Q9BE/LQHFv086fB9+sD+5rdB8D450+6e/Lkyfzyl7/kvvs815jnzJnDwoULiYqK4v3336d9+/bs27ePYcOGMWHChJMOtfrjH/9IdHQ0ubm55OTkMGjQD3fAPfvss3Tq1ImamhrGjBlDTk4ODz74IC+99BJLliyhS5cuJ7zWqlWreP311/n2229RVYYOHcrIkSOJjY1l69atvPPOO/z5z3/mhhtu4L333uOWW2454fyLL76Yb775BhHhL3/5C7/73e948cUX651iuaSkpN7pmE9l69atvPnmmwwbNuykn69v375MnjyZv//97wwZMoSDBw/Spk0b7rjjDt544w1mzpzJli1bOHr0KAMHDmzwPY05E299s4v2Ua34211DOTeuXcMn1FTDpg/gq5meLGrXHcZO94ygierQ5PWeieYX9A7IyMhg7969FBUVUVJSQmxsLElJSVRVVfHYY4+xbNkywsLCKCwsZM+ePXTv3r3e11m2bBkPPvggAOnp6aSnp9fumzNnDrNmzaK6upri4mI2bdp0wv66/vWvf3HdddfVzno5ceJEvvzySyZMmEBKSgoXXHAB4LlrNj8//0fnFxQUMHnyZIqLi6msrKydarm+KZY//vjjeqdjPpWePXvWhvzJPp+IcM455zBkyBAA2rf33DI+adIknnnmGV544QVee+212gVajAk0t1tZudPF2H7dGg75ysOw5i1Y/goc2AVdesOEVyD9BohwZrIyfzW/oD9Fy7spTZo0iblz5/L999/XTl729ttvU1JSwqpVq2jVqhXJyck/mn7YHzt27GDGjBlkZ2cTGxvLlClTzuh1jmvd+oe/dOHh4fV23TzwwAM89NBDTJgwgaVLl/L000+f9vtERETgdrtrn/vW7Dvt8ul+vujoaMaOHcuHH37InDlz7M5f02S2lZRzoKKKISmnaLwc3v/DCJojLkgaCuP+C3o7d3H1dDWPKoPA5MmTeffdd5k7dy6TJk0CPNMJd+3alVatWrFkyRJ27tx5ytcYMWJE7YyTGzZsICcnB4CDBw/Stm1bOnTowJ49e5g/f37tOTExMRw6dOhHr3XJJZfwwQcfUFFRweHDh3n//fe55JJL/P48vlMhv/nmm7Xb65tiediwYfVOx5ycnMzq1asBWL16de3+uk72+fr06UNxcTHZ2dkAHDp0qHa2zjvvvJMHH3yQIUOG+LVwizFnYsUOz9/lrOR6gr40H+b9H/h/qfDF89BjGPx8IdzxKfS9stmEPDTHFr1DUlNTOXToEAkJCbVTDN98881cffXVDBgwgMzMTPr27XvK1/jFL37B1KlT6devH/369WPw4MEADBw4kIyMDPr27UtSUhIXXXRR7Tl3330348aNIz4+vna6Y4BBgwYxZcoUsrKyAE8wZmRk1NtNU5+nn36aSZMmERsby6WXXlob0k888QT33XcfaWlphIeH89RTTzFx4sTa6Zjdbjddu3bls88+4/rrr2f27NmkpqYydOhQeveu/+LTyT5fZGQkf//733nggQc4cuQIbdq0YdGiRbRr147BgwfTvn17m1/fNKnsfBddY1rTs3P0DxuL13lG0Gx8HyQc0ifDRQ9CXB/nCm2kBqcpPttsmmIDnvsPRo0axebNm086NNP+XpjGGv7c52T0jOXVmzJgxxeeSca2L4HIGMicAsPuhfbxTpfpl8ZOU2zMWTV79mwef/xxXnrpJRt/b5pMQWkFRWVH+Y/O+2HWSE9Lvl03uOxpGDwV2nR0usSAsaA3Qee2227jtttuc7oME+Ky8z3986OL/9fTH3/1y55umlZRzhbWBJpN0Kvq6c31bEJasHU5muZnxY5SOke5iSlc5hkiOfh2p0tqMs3i5+KoqCj2799v/7gN4An5/fv3ExUVei0vc/Zk57v4WdddSNVh6HOl0+U0qWbRok9MTKSgoICSkhKnSzFBIioqisTExIYPNKYe+8uPsW1vOZefuwZaRUPKCKdLalLNIuhbtWpVe1emMcY0VnZ+KaD0LvsKzrs0JPvlfTWLrhtjjAmk7HwXAyN20/pwkecO1xBnQW+MaXGy8138rONGQKD3FU6X0+Qs6I0xLcrhY9VsLDrISF0JiZnQrqvTJTU5C3pjTIuyelcpnd0uuh/ObRHdNmBBb4xpYbJ3uLgs3DMZX6gPqzzOr6AXkXEikici20RkWj37e4rI5yKSIyJLRSTRZ18PEflURHJFZJOIJAeufGOMOT0r8l1cG70eOvaAri1jrqQGg15EwoFXgfFAf+AmEelf57AZwGxVTQemA8/57JsNvKCq/YAsYG8gCjfGmNNVWe0md9ceMqrXelrzLeRue39a9FnANlXdrqqVwLvANXWO6Q8s9j5ecny/9z+ECFX9DEBVy1W1IiCVG2PMaVpfWMYQdw6ttLLF9M+Df0GfAOz2eV7g3eZrHTDR+/g6IEZEOgO9gQMi8g8RWSMiL3h/QjiBiNwtIitFZKXd/WqMaSrZ+S4uC1uFOzIGel7U8AkhIlAXYx8GRorIGmAkUAjU4Lnz9hLv/iHAucCUuier6ixVzVTVzLi4uACVZIwxJ8revo/LW60lrNdlEBHpdDlnjT9BXwgk+TxP9G6rpapFqjpRVTOAx73bDuBp/a/1dvtUAx8AgwJSuTHGnAa3Wzm6M5tOeqDFjLY5zp+gzwZ6iUiKiEQCNwIf+R4gIl1E5PhrPQq85nNuRxE53ky/FNjU+LKNMeb05O05xLDqbNwSDudf5nQ5Z1WDQe9tid8PLARygTmqulFEpovIBO9ho4A8EdkCdAOe9Z5bg6fb5nMRWQ8I8OeAfwpjjGmAp39+NZXnDIHoehYDD2F+zV6pqvOAeXW2PenzeC4w9yTnfgakN6JGY4xptK1bNnFb2C409S6nSznr7M5YY0zIU1Xa71oEgLSw/nmwoDfGtAC7XUcYWrmCsrbJ0OV8p8s56yzojTEhb/XWXQwL20TN+aE/JXF9LOiNMSGvfONCIqWGjhfUvam/ZbCgN8aEvK7FiykPiyGsx1CnS3GEBb0xJqSVlB1mSNVKiuJGQHizWCY74CzojTEhbdvqxcRKORH9W95om+Ms6I0xIc2dO49KDScx8yqnS3GMBb0xJqT12LeMzVEDiWzb0elSHGNBb4wJWeVFuSS5C9gXf6nTpTjKgt4YE7K+X/EBADHpVztcibMs6I0xISty+6dsdieRmprmdCmOsqA3xoSmChcJB9eS02440ZEtc1jlcRb0xpiQVJX3KeG4OdxzrNOlOK5l/zdnjAlZh9Z9RI12ICG15awNezLWojfGhJ7qStruXsrnNRkMSenidDWOs6A3xoSenV/RuuYwm2IuIrZty1kE/GSs68YYE3LcefOp1FbI+aOdLiUoWIveGBNaVKneNI9/udPIODfe6WqCggW9MSa07M0lsnw3n7sHMSSlZS0CfjJ+Bb2IjBORPBHZJiLT6tnfU0Q+F5EcEVkqIol19rcXkQIReSVQhRtjTL22zAdgU7vhJHRs43AxwaHBoBeRcOBVYDzQH7hJRPrXOWwGMFtV04HpwHN19j8DLGt8ucYYc2qaN5+NnEdKynlOlxI0/GnRZwHbVHW7qlYC7wJ11+PqDyz2Pl7iu19EBgPdgE8bX64xxpxC+V4oWMnCqgyyUjo7XU3Q8CfoE4DdPs8LvNt8rQMmeh9fB8SISGcRCQNeBB4+1RuIyN0islJEVpaUlPhXuTHG1LVlIYKyyD2IrJRYp6sJGoG6GPswMFJE1gAjgUKgBrgXmKeqBac6WVVnqWqmqmbGxcUFqCRjTIuzZQGlrbryfZtenBfXzulqgoY/4+gLgSSf54nebbVUtQhvi15E2gHXq+oBEbkQuERE7gXaAZEiUq6qP7qga4wxjVJ1FL5bzFIdSWZyJ0TE6YqChj9Bnw30EpEUPAF/I/Az3wNEpAvgUlU38CjwGoCq3uxzzBQg00LeGNMkdiyDqgo+qEznEhtWeYIGu25UtRq4H1gI5AJzVHWjiEwXkQnew0YBeSKyBc+F12ebqF5jjKlf3jyqI6JZ7u7PkGQLel9+TYGgqvOAeXW2PenzeC4wt4HXeAN447QrNMaYhqjClgXktR1CRGUUqfHtna4oqNidscaY5q94LRwqZn5VBoN6xBIRbtHmy74NY0zzl7cARXintK9129TDgt4Y0/zlzaOsSwb7tT1DbPz8j1jQG2Oat7JC+D6HNW0upFW4kJFkQV+XBb0xpnnbsgCADyoGMCChA20iwx0uKPhY0Btjmre8+bhjU5j3fXublvgkLOiNMc3XsXLYsYw93UdRVQNZdiG2Xhb0xpjma/sSqDnG8oihiEBmTwv6+tiascaY5itvAbTuwEelPejTrYYO0a2crigoWYveGNM8uWtgywLc519G9q5DNn7+FCzojTHNU+EqqNhHQdeRHK6ssQuxp2BBb4xpnvLmQVgES2sGAnYh9lQs6I0xzVPeAuhxIV8VVpPUqQ3dO0Q5XVHQsqA3xjQ/rh1Qkov2HsfK/FLrn2+ABb0xpvnx3g27M24U+w9XWrdNAyzojTHNT9586NKH5aWeeeez7ELsKVnQG2Oal6NlsPMr6DOe7B0uurSLJKVLW6erCmoW9MaY5mXbInBXQ5/xrMh3McQWAm+QBb0xpnnJmw/RnSmOSaOg9IhdiPWDX0EvIuNEJE9EtonItHr29xSRz0UkR0SWikiid/sFIrJcRDZ6900O9AcwxrQgNVWw9VPodQUrdpYB1j/vjwaDXkTCgVeB8UB/4CYR6V/nsBnAbFVNB6YDz3m3VwC3qWoqMA6YKSIdA1W8MaaF2fWNp4++z3hW7HDRrnUE/c6xhcAb4k+LPgvYpqrbVbUSeBe4ps4x/YHF3sdLju9X1S2qutX7uAjYC8QFonBjTAu0ZQGER8J5l5Kd72JQz1jCw6x/viH+BH0CsNvneYF3m691wETv4+uAGBHp7HuAiGQBkcB3Z1aqMaZFU/VMe5AygtLqSLbsKScr2ZYN9EegLsY+DIwUkTXASKAQqDm+U0TOAf4KTFVVd92TReRuEVkpIitLSkoCVJIxJqTs2wqu7dB7HCt3lgLYhVg/+RP0hUCSz/NE77ZaqlqkqhNVNQN43LvtAICItAf+CTyuqt/U9waqOktVM1U1My7OenaMMfXIm+f5vfc4svNdRIaHMTDJLvn5w5+gzwZ6iUiKiEQCNwIf+R4gIl1E5PhrPQq85t0eCbyP50Lt3MCVbYxpcbYsgO4DoGMSK3a4SE/sQFQrWwjcHw0GvapWA/cDC4FcYI6qbhSR6SIywXvYKCBPRLYA3YBnvdtvAEYAU0RkrffXBYH+EMaYEHd4P+z+FnqPp6Kymg2FZTas8jT4tZSgqs4D5tXZ9qTP47nAj1rsqvoW8FYjazTGtHRbPwV1Q5/xrN11gGq32kIjp8HujDXGBL8t86FddzjnAlbkuxCBwT1txI2/LOiNMcGt+hhs+xz6jIOwMLLzXfTr3p72UbYQuL8s6I0xwS3/X1BZDr3HU1XjZvXOA9Y/f5os6I0xwS1vPkS0gXNHsqGwjCNVNTZ+/jRZ0BtjgpeqZ1jleaOhVRuy810ADEmx/vnTYUFvjAleezZA2W7oMx6AFTtKSe4cTdcYWwj8dFjQG2OCV55nbVh6XYHbrazc6bJumzNgQW+MCV558yBhMMR0Y1tJOQcqqmz8/BmwoDfGBKdD30PRap9uG0//fJa16E+bBb0xJjhtWej5vbcn6LPzXXSNaU3PztEOFtU8WdAbY4JT3nzo0AO6pQKQvcPFkBRbCPxMWNAbY4JPZQVsX+q5G1aEgtIKisqOWrfNGbKgN8YEnx1fQPWR2v752vHzFvRnxILeGBN88uZDZAz0vBjwjJ+PiYqgT/cYhwtrnizojTHBxe323A17/hiIiARgxY79ZNpC4GfMgt4YE1yK10D5ntpum/3lx/iu5LCNn28EC3pjTHDJmw8SBr0uByA737MQuF2IPXMW9MaY4JK3AJKGQbQn2LPzXURGhDEgsYPDhTVfFvTGmOBxYBfsWe8ZVumVne/igqSOtI6whcDPlAW9MSZ4HL8bts+VABw+Vs3GooPWbdNIfgW9iIwTkTwR2SYi0+rZ31NEPheRHBFZKiKJPvtuF5Gt3l+3B7J4Y0yIyZsPnc6DLr0AWL2rlBq32opSjdRg0ItIOPAqMB7oD9wkIv3rHDYDmK2q6cB04DnvuZ2Ap4ChQBbwlIjYigHGmB87dgjyv6wdbQOeaQ/CBAbZQuCN4k+LPgvYpqrbVbUSeBe4ps4x/YHF3sdLfPZfAXymqi5VLQU+A8ZhjDF1fbcYaipPCPoV+S5S4zvQrnWEg4U1f/4EfQKw2+d5gXebr3XARO/j64AYEens57mIyN0islJEVpaUlPhbuzEmlOTNh6iOnhE3QGW1mzW7Dti0BwEQqIuxDwMjRWQNMBIoBGr8PVlVZ6lqpqpmxsXFBagkY0yz4a7xXIjtdTmEe1rv6wvLOFbtJsvWh200f34eKgSSfJ4nerfVUtUivC16EWkHXK+qB0SkEBhV59yljajXGBOKdq+AI64Tu228C41kWou+0fxp0WcDvUQkRUQigRuBj3wPEJEuInL8tR4FXvM+XghcLiKx3ouwl3u3GWPMD7bMh7AIz/w2Xtn5Ls6Na0uXdq0dLCw0NBj0qloN3I8noHOBOaq6UUSmi8gE72GjgDwR2QJ0A571nusCnsHzn0U2MN27zRhjfpA3H3peBFGeu1/dbmVlvsvGzweIX5eyVXUeMK/Otid9Hs8F5p7k3Nf4oYVvjDEn2v8d7NsCmXfUbsrbc4iDR6vtQmyA2J2xxhhn5c33/F5n2gPAbpQKEAt6Y4yztiyArv0hNrl204odLrq3jyIxto1zdYUQC3pjjHOOlMLOr6H3D615VSU730WWLQQeMBb0xhjnbF0EWlM7iRnAbtcR9hw8ZguNBJAFvTHGOVvmQ9s4SBhcu2nF8f55uxAbMBb0xhhn1FR5WvS9r4CwH6Ioe4eLDm1a0atrOweLCy0W9MYYZ+xYBsfKoPf4EzZn57sYkhxLmC0EHjAW9MaYs08Vlj4P7brBeZfWbt576Cjb9x228fMBZnN/GmPOvk0fQsEKmPAHiIyu3bzSuxC4XYgNLGvRG2POrupKWPSUZ+z8BTefsGvFDhdRrcJIi7eFwAPJWvTGmLMr+y9Qmg+3vAdhJy74nZ3vIiMplsgIa4MGkn2bxpiz50gpfPFfnn758y87Ydeho1XkFh+0bpsmYEFvjDl7ls2Ao2Uw9pkf7Vq1sxS32vj5pmBBb4w5O1w7YMUsyLgZuqf9aHd2vouIMGFQz44OFBfaLOiNMWfH59M9i4uMfrze3dk7SklN6EB0pF06DDQLemNM09udDRv/AcMfgPbxP9p9rLqGtQUHyEq29WGbggW9MaZpqcKnT0DbrjD8wXoPySkoo7LabTdKNRELemNM08r9GHZ/A6Mfg9b1z19zfCFwC/qmYUFvjGk6x2+OiusLGbee9LAVO1z06tqO2LaRZ7G4lsOC3hjTdFa+Bq7tnuGU4QcpMMQAAAxjSURBVPVfZK1xK6t3ltr4+SbkV9CLyDgRyRORbSIyrZ79PURkiYisEZEcEbnSu72ViLwpIutFJFdEHg30BzDGBKkjB+CL5yFlJPQae9LDcosPcuhYtY2fb0INBr2IhAOvAuOB/sBNItK/zmFPAHNUNQO4Efhv7/ZJQGtVHQAMBv5NRJIDU7oxJqh9+aIn7C//LZxiScDjC4Fbi77p+NOizwK2qep2Va0E3gWuqXOMAu29jzsART7b24pIBNAGqAQONrpqY0xwK90J3/4PDLwJzkk/5aHZ+S4SOrYhoaMtBN5U/An6BGC3z/MC7zZfTwO3iEgBMA94wLt9LnAYKAZ2ATNU1VX3DUTkbhFZKSIrS0pKTu8TGGOCz+fTQcLg0idOeZiqsmJHKUNs/HyTCtTF2JuAN1Q1EbgS+KuIhOH5aaAGiAdSgF+LyLl1T1bVWaqaqaqZcXFxASrJGOOIglWwYS5ceD90qNsmPFH+/gr2lR8jK6XzWSquZfIn6AuBJJ/nid5tvu4A5gCo6nIgCugC/AxYoKpVqroX+ArIbGzRxpggVXtzVBxc/MsGD8/2jp/PSrEWfVPyJ+izgV4ikiIikXgutn5U55hdwBgAEemHJ+hLvNsv9W5vCwwDNgemdGNM0Nn8T9j1NYx6FFrHNHj4inwXndpGcl6cLQTelBoMelWtBu4HFgK5eEbXbBSR6SIywXvYr4G7RGQd8A4wRVUVz2iddiKyEc9/GK+rak5TfBBjjMNqquCzJ6FLbxh0u1+nZOe7yOwZi5xiVI5pPL+miVPVeXgusvpue9Ln8SbgonrOK8czxNIYE+pWvg6u7+Cmv5/05qjjVJWC0iPs3F/BrcN6nqUCWy6bD9QYc1KqytEqN4crqzlSWcPhymoOH6upfVxRWU1FZQ1Vhw8w+evfsidmMH/ZFE/FurVUHKvxOa+m9tiKY9VUVNWg6nmPLBs/3+Qs6AOoxq2s2OHiq237qKpxO12OMSflVuVIVY03eP0L5FP5j4h3aBNRxq9Kf8rODXuIjgz3/oqgbetwOkZH0ra153l0ZDhtI8NpExnBOR2iGJBgC4E3NQv6RlJVVu86wMfripi3vpi9h44RHia0Crc+RxO8BKFNbRj/EMA/BLI3pL2B7BvS0ZHhtG0dQZtWnt/bHS2iy+uf4k6dzPsT73X6o5l6WNCfAVVlQ+FBPs4p4p85xRQeOEJkRBij+8RxVXo8Y/p1tVVyTMvx3u9ABLn0N05XYk7C0shPqkrenkN8sq6Yj3OK2Lm/gogwYUTvOH59eW/G9u9GTFQrp8s05uwqXA3r58DFD0HHpIaPN46woG/AdyXlfLKumE9yiti6t5wwgeHndeHeUedxRWp3Okbb/NmmhVKFT38D0V3g4l85XY05BQv6eux2VfBJTjEfrytiU/FBRDwr3zxzTSrj0s4hLqa10yUa47y8+bDzX3DlDIhq3/DxxjEW9F7FZUf4Z04xn+QUs3b3AQAyenTkN1f15ycDzqF7hyiHKzQmiBy/OapzLxg8xelqTANadNCXHDrG/A3FfLKumBXeObFT49szbXxffjLgHJI6RTtcoTFBatUbsH8r3PgOhNu1qWDX4oL+QEUlCzZ8z8c5RSz/bj9uhd7d2vHQ2N5clX4O59qcG8ac2tGDsPR56Hkx9BnvdDXGDy0i6A8ereKzjXv4JKeIL7fuo9qtJHeO5r7R53NVejx9ujc8+ZIxxuurmVCxDy5/5pQrR5ngEbJBX1FZzaLcvXyyroilW0qorHaT0LENd1ySwtXp8aTGt7eJlIw5XWUFsPxVGHADJAxyuhrjp5AK+qNVNSzNK+HjnCIW5+7lSFUNXWNac/PQHlw9MJ6MpI4W7sY0xuLfeoZVjrGbo5qTkAn6XfsruPLlLyk/Vk3ntpFcPziBq9LjGZLcifAwC3djGq1oLax7Fy76d+jYw+lqzGkImaBP6tSGm7KSGNE7jgvP7UxEeKBWSTTG1K4c1SYWLnnI6WrMaQqZoBcRHv9Jf6fLMCY0bf0U8r+E8S9AlM022dxYs9cYc2o11Z6pDjqdB5lTna7GnIGQadEbY5rImtmwLw8mv203RzVT1qI3xpzcsUOw5P9Cj+HQ9ydOV2POkLXojTEn99Xv4XCJZx1YG5rcbPnVoheRcSKSJyLbRGRaPft7iMgSEVkjIjkicqXPvnQRWS4iG0VkvYjY7GDGNAdlhfD1K5D2U0gc7HQ1phEabNGLSDjwKjAWKACyReQjVd3kc9gTwBxV/aOI9AfmAckiEgG8BdyqqutEpDNQFfBPYYwJvCXPgtbAmCedrsQ0kj8t+ixgm6puV9VK4F3gmjrHKHB8QuoOQJH38eVAjqquA1DV/apa0/iyjTFNqjgH1v4Nht4DsT2drsY0kj9BnwDs9nle4N3m62ngFhEpwNOaf8C7vTegIrJQRFaLyCP1vYGI3C0iK0VkZUlJyWl9AGNMgNXeHNURLvm109WYAAjUqJubgDdUNRG4EviriITh6Rq6GLjZ+/t1IjKm7smqOktVM1U1My4uLkAlGWPOyLZFsOMLGDnNE/am2fMn6AsB31V/E73bfN0BzAFQ1eVAFNAFT+t/maruU9UKPK19m/LOmGBVU+1pzXc6FzJ/7nQ1JkD8CfpsoJeIpIhIJHAj8FGdY3YBYwBEpB+eoC8BFgIDRCTae2F2JLAJY0xwWvsWlGyGy/4TImzh+1DR4KgbVa0WkfvxhHY48JqqbhSR6cBKVf0I+DXwZxH5FZ4Ls1NUVYFSEXkJz38WCsxT1X821YcxxjTCsXJY/CwkDYN+VztdjQkgv26YUtV5eLpdfLc96fN4E3DRSc59C88QS2NMMPv6ZTi8F256x26OCjE2BYIxBg4WwVcvQ+pESMx0uhoTYBb0xpgfbo667CmnKzFNwILemJbu+w2w5m3Iuhtik52uxjQBC3pjWrrPfuNZTGTEw05XYpqIBb0xLdm2RfDdYhj5H55lAk1IsqA3pqVy13hWjopNgSF3Ol2NaUI2H70xLdXat2HvJpj0pt0cFeKsRW9MS3T85qjELOhfdzJaE2qsRW9MS7T8FSj/Hib/1W6OagFCJ+grXPD6eKerMKZ5cG2H/tdCUpbTlZizIHSCPiwc4vo4XYUxzUNCJlz6uNNVmLMkdII+qgPcMNvpKowxJujYxVhjjAlxFvTGGBPiLOiNMSbEWdAbY0yIs6A3xpgQZ0FvjDEhzoLeGGNCnAW9McaEOFFVp2s4gYiUADsb8RJdgH0BKqe5s+/iRPZ9nMi+jx+EwnfRU1Xj6tsRdEHfWCKyUlVtdWPsu6jLvo8T2ffxg1D/LqzrxhhjQpwFvTHGhLhQDPpZThcQROy7OJF9Hyey7+MHIf1dhFwfvTHGmBOFYoveGGOMDwt6Y4wJcSET9CIyTkTyRGSbiExzuh4niUiSiCwRkU0islFE/t3pmpwmIuEiskZEPnG6FqeJSEcRmSsim0UkV0QudLomJ4nIr7z/TjaIyDsiEuV0TYEWEkEvIuHAq8B4oD9wk4j0d7YqR1UDv1bV/sAw4L4W/n0A/DuQ63QRQeL3wAJV7QsMpAV/LyKSADwIZKpqGhAO3OhsVYEXEkEPZAHbVHW7qlYC7wLXOFyTY1S1WFVXex8fwvMPOcHZqpwjIonAT4C/OF2L00SkAzAC+F8AVa1U1QPOVuW4CKCNiEQA0UCRw/UEXKgEfQKw2+d5AS042HyJSDKQAXzrbCWOmgk8AridLiQIpAAlwOverqy/iEhbp4tyiqoWAjOAXUAxUKaqnzpbVeCFStCbeohIO+A94JeqetDpepwgIlcBe1V1ldO1BIkIYBDwR1XNAA4DLfaalojE4vnpPwWIB9qKyC3OVhV4oRL0hUCSz/NE77YWS0Ra4Qn5t1X1H07X46CLgAkiko+nS+9SEXnL2ZIcVQAUqOrxn/Dm4gn+luoyYIeqlqhqFfAPYLjDNQVcqAR9NtBLRFJEJBLPxZSPHK7JMSIiePpgc1X1JafrcZKqPqqqiaqajOfvxWJVDbkWm79U9Xtgt4j08W4aA2xysCSn7QKGiUi099/NGELw4nSE0wUEgqpWi8j9wEI8V81fU9WNDpflpIuAW4H1IrLWu+0xVZ3nYE0meDwAvO1tFG0Hpjpcj2NU9VsRmQusxjNabQ0hOB2CTYFgjDEhLlS6bowxxpyEBb0xxoQ4C3pjjAlxFvTGGBPiLOiNMSbEWdAbY0yIs6A3xpgQ9/8BmTarT8N+TR8AAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    }
  ]
}