Self-Supervision with FastAI.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "heading_collapsed": true,
    "id": "Us1hP0a9Trxr"
   },
   "source": [
    "# Self-Supervision with FastAI\n",
    "> A tutorial of rotation-based self-supervision using FastAI2 & PyTorch!\n",
    "\n",
    "- toc: true \n",
    "- badges: true\n",
    "- comments: true\n",
    "- categories: [jupyter]\n",
    "- image: images/self-sup-rotation.png"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-03-23T14:49:40.997471Z",
     "start_time": "2020-03-23T14:49:40.985501Z"
    },
    "heading_collapsed": true,
    "id": "jOCxXuEpTrxw"
   },
   "source": [
    "# Introduction\n",
    "\n",
    "This notebook is an introduction to self-supervised learning. In short, self-supervised learning has 2 components:\n",
    "\n",
    "1. Pretrain on a pretext task, where the labels can come from the data itself!\n",
    "2. Transfer the features, and train on the actual classification labels!\n",
    "\n",
    ">\"What if we can get labels for free for unlabelled data and train unsupervised dataset in a supervised manner? We can achieve this by framing a supervised learning task in a special form to predict only a subset of information using the rest. In this way, all the information needed, both inputs and labels, has been provided. This is known as self-supervised learning.\" - Lilian Weng\n",
    "\n",
    "Using FastAI2, we'll use rotation as a pretext task for learning representations/features of our data. \n",
    "\n",
    "Here are some great overviews of self-supervised learning that I've come across:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-03-23T14:49:40.997471Z",
     "start_time": "2020-03-23T14:49:40.985501Z"
    },
    "hidden": true,
    "id": "GXcHJC68Trxx"
   },
   "source": [
    "- [Lilian Weng - Self-Supervised Representation Learning](https://lilianweng.github.io/lil-log/2019/11/10/self-supervised-learning)\n",
    "\n",
    "- [Jeremy Howard - Self-supervised learning and computer vision](https://www.fast.ai/2020/01/13/self_supervised/)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0bb547FdTrxx"
   },
   "source": [
    "# Experiment Layout"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XJ9e05yTTrxy"
   },
   "source": [
    "In this notebook, we will be using the [MNIST dataset](https://en.wikipedia.org/wiki/MNIST_database).\n",
    "\n",
    "Also check out [ImageWang](https://github.com/fastai/imagenette#image%E7%BD%91) from FastAI themselves! It's a dataset designed for self-supervision tasks!\n",
    "\n",
    "1. Train a model on a rotation prediction task.\n",
    "  - We will use *all* the training data for rotation prediction.\n",
    "  - Input: A rotated image.\n",
    "  - Target/Label: Classify the amount of degrees rotated.\n",
    "  - Our model should learn useful features that can transfer well for a classification task.\n",
    "  - (The model should learn what digits look like in order to be able to successfully predict the amount of rotation).\n",
    "\n",
    "2. Transfer our rotation pretraining features to solve the classification task with *much fewer* labels, < 1% of the original data.\n",
    "  - Input: A normal image.\n",
    "  - Target/Label: The images' *original* categorical label.\n",
    "  - Classification accuracy should be decent, even with only using < 1% of the original data.\n",
    "\n",
    "3. Train a classifier from scratch on the same amount of data used in experiment 2.\n",
    "  - Input: A normal image.\n",
    "  - Target/Label: The images' *original* categorical label.\n",
    "  - Classification accuracy should be low (lack of transfer learning & too few labeled data!)\n",
    "  - Model may overfit.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ywV2I3IhTrxy"
   },
   "source": [
    "# FastAI Vision Model Creation Methods"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "H1Exth4vTrxz"
   },
   "source": [
    "> Warning: This Jupyter notebook runs with fastai2! Make sure you have it installed, use the cell below to install it :)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T22:05:13.201652Z",
     "start_time": "2021-03-10T22:04:58.559155Z"
    },
    "collapsed": true,
    "id": "GL6S4uRYTrx0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting fastai\n",
      "  Downloading fastai-2.5.3-py3-none-any.whl (189 kB)\n",
      "Requirement already satisfied: torch<1.11,>=1.7.0 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from fastai) (1.10.0+cu113)\n",
      "Requirement already satisfied: scipy in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from fastai) (1.7.2)\n",
      "Requirement already satisfied: pyyaml in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from fastai) (6.0)\n",
      "Requirement already satisfied: pillow>6.0.0 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from fastai) (8.4.0)\n",
      "Collecting fastcore<1.4,>=1.3.22\n",
      "  Downloading fastcore-1.3.27-py3-none-any.whl (56 kB)\n",
      "Requirement already satisfied: matplotlib in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from fastai) (3.4.3)\n",
      "Requirement already satisfied: scikit-learn in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from fastai) (1.0.1)\n",
      "Collecting spacy<4\n",
      "  Downloading spacy-3.2.1-cp39-cp39-win_amd64.whl (11.8 MB)\n",
      "Requirement already satisfied: torchvision>=0.8.2 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from fastai) (0.11.1+cu113)\n",
      "Collecting fastdownload<2,>=0.0.5\n",
      "  Downloading fastdownload-0.0.5-py3-none-any.whl (13 kB)\n",
      "Requirement already satisfied: packaging in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from fastai) (21.2)\n",
      "Collecting fastprogress>=0.2.4\n",
      "  Downloading fastprogress-1.0.0-py3-none-any.whl (12 kB)\n",
      "Requirement already satisfied: pandas in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from fastai) (1.3.4)\n",
      "Requirement already satisfied: pip in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from fastai) (21.3.1)\n",
      "Requirement already satisfied: requests in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from fastai) (2.26.0)\n",
      "Requirement already satisfied: numpy in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from fastprogress>=0.2.4->fastai) (1.21.4)\n",
      "Collecting catalogue<2.1.0,>=2.0.6\n",
      "  Downloading catalogue-2.0.6-py3-none-any.whl (17 kB)\n",
      "Collecting thinc<8.1.0,>=8.0.12\n",
      "  Downloading thinc-8.0.13-cp39-cp39-win_amd64.whl (1.0 MB)\n",
      "Requirement already satisfied: setuptools in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from spacy<4->fastai) (58.5.3)\n",
      "Collecting spacy-legacy<3.1.0,>=3.0.8\n",
      "  Downloading spacy_legacy-3.0.8-py2.py3-none-any.whl (14 kB)\n",
      "Collecting preshed<3.1.0,>=3.0.2\n",
      "  Downloading preshed-3.0.6-cp39-cp39-win_amd64.whl (112 kB)\n",
      "Collecting pydantic!=1.8,!=1.8.1,<1.9.0,>=1.7.4\n",
      "  Downloading pydantic-1.8.2-cp39-cp39-win_amd64.whl (1.9 MB)\n",
      "Collecting langcodes<4.0.0,>=3.2.0\n",
      "  Downloading langcodes-3.3.0-py3-none-any.whl (181 kB)\n",
      "Collecting pathy>=0.3.5\n",
      "  Downloading pathy-0.6.1-py3-none-any.whl (42 kB)\n",
      "Collecting cymem<2.1.0,>=2.0.2\n",
      "  Downloading cymem-2.0.6-cp39-cp39-win_amd64.whl (36 kB)\n",
      "Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from spacy<4->fastai) (4.62.3)\n",
      "Collecting spacy-loggers<2.0.0,>=1.0.0\n",
      "  Downloading spacy_loggers-1.0.1-py3-none-any.whl (7.0 kB)\n",
      "Collecting wasabi<1.1.0,>=0.8.1\n",
      "  Downloading wasabi-0.9.0-py3-none-any.whl (25 kB)\n",
      "Collecting murmurhash<1.1.0,>=0.28.0\n",
      "  Downloading murmurhash-1.0.6-cp39-cp39-win_amd64.whl (21 kB)\n",
      "Collecting blis<0.8.0,>=0.4.0\n",
      "  Downloading blis-0.7.5-cp39-cp39-win_amd64.whl (6.6 MB)\n",
      "Collecting srsly<3.0.0,>=2.4.1\n",
      "  Downloading srsly-2.4.2-cp39-cp39-win_amd64.whl (451 kB)\n",
      "Requirement already satisfied: jinja2 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from spacy<4->fastai) (3.0.2)\n",
      "Collecting typer<0.5.0,>=0.3.0\n",
      "  Downloading typer-0.4.0-py3-none-any.whl (27 kB)\n",
      "Requirement already satisfied: pyparsing<3,>=2.0.2 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from packaging->fastai) (2.4.7)\n",
      "Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from requests->fastai) (1.26.7)\n",
      "Requirement already satisfied: certifi>=2017.4.17 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from requests->fastai) (2021.10.8)\n",
      "Requirement already satisfied: charset-normalizer~=2.0.0 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from requests->fastai) (2.0.7)\n",
      "Requirement already satisfied: idna<4,>=2.5 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from requests->fastai) (3.3)\n",
      "Requirement already satisfied: typing-extensions in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from torch<1.11,>=1.7.0->fastai) (3.10.0.2)\n",
      "Requirement already satisfied: cycler>=0.10 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from matplotlib->fastai) (0.11.0)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from matplotlib->fastai) (2.8.2)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from matplotlib->fastai) (1.3.2)\n",
      "Requirement already satisfied: pytz>=2017.3 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from pandas->fastai) (2021.3)\n",
      "Requirement already satisfied: joblib>=0.11 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from scikit-learn->fastai) (1.1.0)\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from scikit-learn->fastai) (3.0.0)\n",
      "Collecting smart-open<6.0.0,>=5.0.0\n",
      "  Downloading smart_open-5.2.1-py3-none-any.whl (58 kB)\n",
      "Requirement already satisfied: six>=1.5 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from python-dateutil>=2.7->matplotlib->fastai) (1.16.0)\n",
      "Requirement already satisfied: colorama in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from tqdm<5.0.0,>=4.38.0->spacy<4->fastai) (0.4.4)\n",
      "Requirement already satisfied: click<9.0.0,>=7.1.1 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from typer<0.5.0,>=0.3.0->spacy<4->fastai) (8.0.3)\n",
      "Requirement already satisfied: MarkupSafe>=2.0 in c:\\users\\abdul\\projects\\projects\\lib\\site-packages (from jinja2->spacy<4->fastai) (2.0.1)\n",
      "Installing collected packages: murmurhash, cymem, catalogue, wasabi, typer, srsly, smart-open, pydantic, preshed, blis, thinc, spacy-loggers, spacy-legacy, pathy, langcodes, fastprogress, fastcore, spacy, fastdownload, fastai\n",
      "Successfully installed blis-0.7.5 catalogue-2.0.6 cymem-2.0.6 fastai-2.5.3 fastcore-1.3.27 fastdownload-0.0.5 fastprogress-1.0.0 langcodes-3.3.0 murmurhash-1.0.6 pathy-0.6.1 preshed-3.0.6 pydantic-1.8.2 smart-open-5.2.1 spacy-3.2.1 spacy-legacy-3.0.8 spacy-loggers-1.0.1 srsly-2.4.2 thinc-8.0.13 typer-0.4.0 wasabi-0.9.0\n"
     ]
    }
   ],
   "source": [
    "!pip install fastai --upgrade\n",
    "\n",
    "# Uncomment and run the below line to get a fresh install of fastai, if needed\n",
    "# !pip install fastai --upgrade"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BJRO1KwdTrx2"
   },
   "source": [
    "> Important: Pay attention! It's important. We will be using a small ConvNet to test our self-supervised learning method. The architecture is defined below in `simple_arch`.\n",
    "\n",
    "Note that `simple_arch` takes in one argument, `pretrained`. This is to allow FastAI to pass `pretrained=True` or `pretrained=False` when creating the model body! Below are some use cases of when we would want `pretrained=True` or `pretrained=False`.\n",
    "\n",
    "1. `pretrained=False` = For training a new model on our rotation prediction task.\n",
    "2. `pretrained=True` = For transferring the learnt features from our rotation task pretraining to solve a classification task.\n",
    "3. `pretrained=False` = For training a new model from scratch on the main classification task (no transfer learning)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:11.684648Z",
     "start_time": "2021-03-10T23:31:10.265151Z"
    },
    "id": "OK-9alZaTrx2"
   },
   "outputs": [],
   "source": [
    "from fastai.vision.all import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:11.693859Z",
     "start_time": "2021-03-10T23:31:11.688343Z"
    },
    "id": "xQSCmfXrTrx3"
   },
   "outputs": [],
   "source": [
    "#collapse-show\n",
    "def simple_arch(pretrained=False):\n",
    "    # Note that FastAI will automatically cut at pooling layer for the body!\n",
    "    model = nn.Sequential(\n",
    "        nn.Conv2d(1, 4, 3, 1),\n",
    "        nn.BatchNorm2d(4),\n",
    "        nn.ReLU(),\n",
    "        nn.Conv2d(4, 16, 3, 1),\n",
    "        nn.BatchNorm2d(16),\n",
    "        nn.ReLU(),\n",
    "        nn.Conv2d(16, 32, 3, 1),\n",
    "        nn.BatchNorm2d(32),\n",
    "        nn.AdaptiveAvgPool2d(1),\n",
    "    )\n",
    "    if (pretrained):\n",
    "        print(\"Loading pretrained model...\")\n",
    "        pretrained_weights = torch.load(save_path/'rot_pretrained.pt')\n",
    "        print(model.load_state_dict(pretrained_weights))\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "V0RQxI02Trx3"
   },
   "source": [
    "> The follow below code snippets are examples of how FastAI creates CNNs. Every model will have a *body* and a *head*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:11.704827Z",
     "start_time": "2021-03-10T23:31:11.695745Z"
    },
    "id": "I0M5YiEgTrx4",
    "outputId": "4c8eab0f-36cc-4fbc-86fb-622507fc6046"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Conv2d(1, 4, kernel_size=(3, 3), stride=(1, 1))\n",
       "  (1): BatchNorm2d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  (2): ReLU()\n",
       "  (3): Conv2d(4, 16, kernel_size=(3, 3), stride=(1, 1))\n",
       "  (4): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  (5): ReLU()\n",
       "  (6): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1))\n",
       "  (7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       ")"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#collapse-show\n",
    "body = create_body(arch=simple_arch, pretrained=False)\n",
    "body"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:11.878524Z",
     "start_time": "2021-03-10T23:31:11.872169Z"
    },
    "id": "88abBSy0Trx5",
    "outputId": "a6ff46c3-1f12-42d7-9ebb-e9b96148c979"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): AdaptiveConcatPool2d(\n",
       "    (ap): AdaptiveAvgPool2d(output_size=1)\n",
       "    (mp): AdaptiveMaxPool2d(output_size=1)\n",
       "  )\n",
       "  (1): Flatten(full=False)\n",
       "  (2): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  (3): Dropout(p=0.5, inplace=False)\n",
       "  (4): Linear(in_features=64, out_features=8, bias=False)\n",
       ")"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#collapse-show\n",
    "head = create_head(nf=32, n_out=8, lin_ftrs=[])\n",
    "head"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:12.399617Z",
     "start_time": "2021-03-10T23:31:12.386435Z"
    },
    "id": "OsZXso2cTrx6",
    "outputId": "6d4a90f4-3718-4484-bced-94934d366544"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Sequential(\n",
       "    (0): Conv2d(1, 4, kernel_size=(3, 3), stride=(1, 1))\n",
       "    (1): BatchNorm2d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (2): ReLU()\n",
       "    (3): Conv2d(4, 16, kernel_size=(3, 3), stride=(1, 1))\n",
       "    (4): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (5): ReLU()\n",
       "    (6): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1))\n",
       "    (7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  )\n",
       "  (1): Sequential(\n",
       "    (0): AdaptiveConcatPool2d(\n",
       "      (ap): AdaptiveAvgPool2d(output_size=1)\n",
       "      (mp): AdaptiveMaxPool2d(output_size=1)\n",
       "    )\n",
       "    (1): Flatten(full=False)\n",
       "    (2): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (3): Dropout(p=0.5, inplace=False)\n",
       "    (4): Linear(in_features=64, out_features=8, bias=False)\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#collapse-show\n",
    "# Note that FastAI automatically determines nf for the head!\n",
    "model = create_cnn_model(arch=simple_arch, pretrained=False, n_out=8, lin_ftrs=[])\n",
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EPBzU0q0Trx6"
   },
   "source": [
    "# PyTorch Rotation/Classification Self-Supervised Dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:12.808083Z",
     "start_time": "2021-03-10T23:31:12.804890Z"
    },
    "id": "EkTG299eTrx6"
   },
   "outputs": [],
   "source": [
    "# --- Functions to convert between Torch Tensors and PIL Images ---\n",
    "import torchvision\n",
    "tensorToImage = torchvision.transforms.ToPILImage()\n",
    "imageToTensor = torchvision.transforms.ToTensor()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:28:37.733862Z",
     "start_time": "2021-03-10T23:28:37.731334Z"
    },
    "id": "bg2defioTrx6"
   },
   "outputs": [],
   "source": [
    "#collapse-hide\n",
    "# Uncomment and run the below lines if torchvision has trouble downloading MNIST (in the next cell)\n",
    "\n",
    "# !wget -P data/MNIST/raw/ http://yann.lecun.com/exdb/mnist/train-images-idx3-ubyte.gz\n",
    "# !wget -P data/MNIST/raw/ http://yann.lecun.com/exdb/mnist/train-labels-idx1-ubyte.gz\n",
    "# !wget -P data/MNIST/raw/ http://yann.lecun.com/exdb/mnist/t10k-images-idx3-ubyte.gz\n",
    "# !wget -P data/MNIST/raw/ http://yann.lecun.com/exdb/mnist/t10k-labels-idx1-ubyte.gz"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:14.084740Z",
     "start_time": "2021-03-10T23:31:14.045646Z"
    },
    "id": "sohx0YSKTrx7",
    "outputId": "da7f6798-8822-41ac-e5b3-6e9fdce7f58b",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Dataset MNIST\n",
       "    Number of datapoints: 60000\n",
       "    Root location: data/\n",
       "    Split: Train"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Download MNIST dataset from PyTorch if not downloaded already!\n",
    "torchvision.datasets.MNIST('data/', download=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "G0EzKmHQTrx7"
   },
   "source": [
    "Below we define a dataset, here's the docstring:\n",
    "\n",
    "A Dataset for Rotation-based Self-Supervision! Images are rotated clockwise.\n",
    "- `file` - MNIST processed .pt file.\n",
    "- `pct` - percent of data to use\n",
    "- `classification` - False=Use rotation labels. True=Use original classification labels."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:14.623324Z",
     "start_time": "2021-03-10T23:31:14.612994Z"
    },
    "id": "Wfc1LiwcTrx7"
   },
   "outputs": [],
   "source": [
    "#collapse-hide\n",
    "class Custom_Dataset_MNIST():\n",
    "    '''\n",
    "    A Dataset for Rotation-based Self-Supervision! Images are rotated clockwise.\n",
    "    - file - MNIST processed .pt file.\n",
    "    - pct - percent of data to use\n",
    "    - classification - False=Use rotation labels. True=Use original classification labels.\n",
    "    '''\n",
    "    \n",
    "    def __init__(self, file, pct, classification):\n",
    "        \n",
    "        data = torch.load(file)\n",
    "        self.imgs = data[0]\n",
    "        self.labels = data[1]\n",
    "        self.pct = pct\n",
    "        self.classification = classification\n",
    "                    \n",
    "        slice_idx = int(len(self.imgs)*self.pct)\n",
    "        self.imgs = self.imgs[:slice_idx]\n",
    "                    \n",
    "    def __len__(self):\n",
    "        return len(self.imgs)\n",
    "    \n",
    "    def __getitem__(self, idx):\n",
    "        img = self.imgs[idx].unsqueeze(0)\n",
    "        img = tensorToImage(img)\n",
    "        img = img.resize((32, 32), resample=1)\n",
    "        img = imageToTensor(img)\n",
    "        \n",
    "        if (not self.classification):\n",
    "            # 4 classes for rotation\n",
    "            degrees = [0, 45, 90, 135, 180, 225, 270, 315]\n",
    "            rand_choice = random.randint(0, len(degrees)-1)\n",
    "            \n",
    "            img = tensorToImage(img)\n",
    "            img = img.rotate(degrees[rand_choice])\n",
    "            img = imageToTensor(img)\n",
    "            return img, torch.tensor(rand_choice).long()\n",
    "        \n",
    "        return img, self.labels[idx]\n",
    "    \n",
    "    def show_batch(self, n=3):\n",
    "        fig, axs = plt.subplots(n, n)\n",
    "        fig.tight_layout()\n",
    "        for i in range(n):\n",
    "            for j in range(n):\n",
    "                rand_idx = random.randint(0, len(self)-1)\n",
    "                img, label = self.__getitem__(rand_idx)\n",
    "                axs[i, j].imshow(tensorToImage(img), cmap='gray')\n",
    "                if self.classification:\n",
    "                    axs[i, j].set_title('Label: {0} (Digit #{1})'.format(label.item(), label.item()))\n",
    "                else:\n",
    "                    axs[i, j].set_title('Label: {0} ({1} Degrees)'.format(label.item(), label.item()*45))\n",
    "                axs[i, j].axis('off')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VEcTmnZyTrx8"
   },
   "source": [
    "# Rotation Prediction Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-03-23T16:28:12.061325Z",
     "start_time": "2020-03-23T16:28:12.055543Z"
    },
    "id": "wt9xW9a2Trx8"
   },
   "source": [
    "> Important: 60k training data and 10k validation data!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:15.348643Z",
     "start_time": "2021-03-10T23:31:15.302426Z"
    },
    "id": "IBMTeUdcTrx8",
    "outputId": "e3357de7-f4d4-420c-b942-046ca1db9a67"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "60000 Training Samples | 10000 Validation Samples\n"
     ]
    }
   ],
   "source": [
    "# Make rotation datasets\n",
    "train_ds = Custom_Dataset_MNIST('data/MNIST/processed/training.pt', pct=1.0, classification=False)\n",
    "valid_ds = Custom_Dataset_MNIST('data/MNIST/processed/test.pt', pct=1.0, classification=False)\n",
    "print('{0} Training Samples | {1} Validation Samples'.format(len(train_ds), len(valid_ds)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-03-23T16:28:42.437483Z",
     "start_time": "2020-03-23T16:28:42.431824Z"
    },
    "id": "R_zUS7tXTrx8"
   },
   "source": [
    "> Note: Notice that our labels don't correspond to digits! They correspond to the amount of degrees rotated! Specifically from this predefined set: `[0, 45, 90, 135, 180, 225, 270, 315]`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:16.188918Z",
     "start_time": "2021-03-10T23:31:15.648884Z"
    },
    "id": "6jLJc94CTrx9",
    "outputId": "054543ee-8cc1-4383-fa00-93b0f42f78b8"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Due to IPython and Windows limitation, python multiprocessing isn't available now.\n",
      "So `number_workers` is changed to 0 to avoid getting stuck\n",
      "Due to IPython and Windows limitation, python multiprocessing isn't available now.\n",
      "So `number_workers` is changed to 0 to avoid getting stuck\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbEAAAETCAYAAAC4BDhUAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAACQF0lEQVR4nOz9eXxk11ngD39PrapNVZJK+y51q/fNbru9xXYcx1lISCBkCG8IyUDmZeDHMPMyDDOEYcgAAwwzmYUlTIYfIUCATCYh2AlxnMR220ls3O5939Tad6mqVKp9u+8ft87pq2pJ3W23bUl1vp9PfVSqu9/73PM853me8xxhGAYajUaj0WxEbG/1CWg0Go1G81rRSkyj0Wg0GxatxDQajUazYdFKTKPRaDQbFq3ENBqNRrNh0UpMo9FoNBuWN02JCSEOCyE++WZve4v7/1shxAffqP1vdIQQe4UQL70Jx9EyskkRQnxGCPFzd3ifWl42KbcjL7etxIQQw0KIx2//tN4chBAfF0IcE0LEhRDjQojfF0I41lh/L7APeLL8vxBC/JoQYrS8jy8JIWot67uFEJ8vL5sWQvzSGvv+hBCiKIRIlD9DQog/F0IM3MlrfqMxDOM0EBNCvP9W1l/vMgIghOgTQnxDCLEkhJgXQvz+GutWysgPCSG+L4SIlWXg/xVCBCzrn7M884QQoiCE+Lpl+f6yjKbKf/evcezDQohM+Tzj5fX/nRDCfUduxJvHfwU+JYRwVS5Y7/IihPiIEOKSEGJRCDErhPgLa5uwwvqV8tIqhHhKCDEphDCEED0V6/++EGKs/HxHhBCfqlhuCCGSFnn6f9c49qaXl0o2ozvRC/wrIAwcAt4B/PIa6/8s8NfG9VHfPwV8DHgQaAM8wB9a1v80sBXoBt4O/IoQ4t1r7P9lwzD8QBB4HEgDx4QQu2/rqm6BtZT1HeCvMe/Vhqf8YnwHeA5oATqAL66xSaWMBIHfxpSPHUA78F/kyoZh7DIMw19+7gFgDPi/lmM/WT5eHfAXwJM3eVl/wTCMANAK/GvgI8A3hRDidq77VnijZMgwjCngIvDDb8T+32B+ADxoGEYQ6AMcmM9/NSrlpQR8C/jQKuv/GbDdMIxa4AHgo0KIH61YZ5+UKcMwbtaDrCp5uWNKTAhRV7Zs54QQ0fL3jorV+oUQR8oWwpNCiHrL9vcJIV4qW7enhBCPvpbzMAzjTwzD+J5hGDnDMCYwG98H19jkPcALlv/fD/yZYRhjhmEkgP8M/LgQwlte/nHgtwzDiBqGcQH4U+ATt3BeRcMwBg3D+Pny8T4tl6117UKIXiHEi2XL6rtCiD8WQnyxvKynbKX9jBBiFLNRRgjx00KIC+Xn8IwQotuyv+1CiO8IISJl6/KfWJa9VwhxvnysCSGEVfkfBt7xeiy69SIjmM9r0jCM/2YYRtIwjEy5t7kay2TEMIy/MQzjW4ZhpAzDiGLKwGoy9jCmQfXV8v+PYjaC/8MwjKxhGH8ACOCxm510+VwPY77Y9wM/BCCEsJWt7UEhxIIQ4ssV9+2nyhb+ghDi14Wl5yOE+LQQ4itCiC8KIeLAJ4QQQSHEnwkhpspy8NtCCLtlfyvKlzD578LsrcSFEGcqjLXD8pxvhfUiL+W2YN7yUxHYssYmlfIyYxjGZ4FXV9n/JcMwkpafSjfZ/y1RLfJyJ3tiNuDPMXsoXZg9jj+qWOengJ/GtBAKwB8ACCHagX/AtG7qMXtOXxVCNFYeRAjRVRbKrls8r4eBcystEEL4gF7gUuWiiu9uYKsQoq587qcsy08Bu27xXCR/B7ytfA43u/a/AY4ADZiK72Mr7O8RzB7Bu4QQHwA+Bfwo0Ah8D/hby/V+p7zPJkwL7bNCiJ3l/fwZ8LNlK243ZaUIUDYI8sC227xWK+tFRu4DhoUQTwvTlXhYCLFnpRXXkBErq8oYptHzVUsjtQs4bbHSAU5zGzJkGMYocJSyDAH/Avggphy0AVHgj8vnvxP4LPBRzHsaxOw5WvkA8BUghGn0fQHz3m8BDgBPAJ8s729V+Sqv9zAwUD7OPwEWLMe5gOlmu1XWi7wghHhICLEILGH2qP7HKuvdirystN2/E0IkgHHAh/mOWnlRmK7rvxMV7sibsenlxTCM2/oAw8Djt7DefiBq+f8w8HuW/3cCOcAO/Fvgryq2fwb4uGXbT76Gc/1pTKEIr7K8HTCAGstvnwQuAz3lG/tUeZ37gc4V1n8nMLzK/j8BfH+F398N5MvfV712zBe3AHgty74IfLH8vad8Pn2W5U8DP2P53wakMBuCHwe+V3GszwG/Uf4+iukKqV3leiaAhze6jADfxlTI7wFcwL8BrgGuW5GRiuXvxGwEBlZY5gXiwKOW334d+FLFen8NfHqV/a94XcCXgD8tf78AvMOyrLV8fQ7gPwB/W3FOOfl8MA2jFy3Lm4Es4LH89hPA87cgX49hvjv3AbZV7tW1jSYvK8jDp1d63jeTl/LzMICeVbYVmErgPwIBy+8Pl+U0hKnEzwKOapWXys+ddCd6hRCfK3dD48CLQMjarcSMDUhGACemq6Ub+HDZGooJIWLAQ5g397WezweB3wXeYyx3BViJlf8GLL99HtNSOIxpXT9f/n0cSJS/W4O6tZjW2e3QDkTK39e69jYgYhhGyrLtGDdi/a0b+J+WfUUwX4728rJDFcf6KGZcCEwL873AiBDiBSHE/RXHCXD9nt0260hG0pjGxdOGYeQwg8gNmL3ZSmLlv4HKBUKI+zAt5h8zDOPyCtv+KOb9t7qrEyyXH7gzMvQ1y325gOnyasaUIXVPy7K0sHxXN8iPE5iy7O9zmD13uXxF+TIM4znMRvaPgVkhxP8WyxMgbkt+1pG8KAzTI/EtTKWwErHy3xvk5Rb2bRiGcQJTPv+j5fcXDTM8EgP+JWZPbyVZXYtNKy930p34rzFdTYcMM0D5cPl3q2uu0/K9C1P7z2PelL8yDCNk+fgMw/i913Iiwky0+FPg/YZhnFltPcN08QxidmflbyXDMH7DMIwewzA6MBXZBDBhmPGPKZZ3cfexuitpNX4Es1sNa1/7FFAvrsfjYPk9VKdt+T6G6RK07s9jGMZL5WUvVCzzG4bxc+Vrf9UwjA9gCuDfA1+WOy27Z1zcppukgvUiI6dZfs9WZSUZARBCHMDspf+0YRjPrrL5x4G/NMpmZZlzwF4hlgXZ93IbMiSE6ATuZrkMvafi3tSUG9wpzMQVua0HU2Evu0zL9zFMyzps2VetYRi7LMtXky8Mw/gDwzDuxuwVDWD2ciU7WO6KvxnrRV4qcQD9Ky1YTV7u1P7lYVh+D9Zks8vLa1ViTiFEjeXjwNSaacxU7HrgN1bY7ieFEDvLjfJvAl8xDKOI6SJ7vxDiXUIIe3mfj4obg7g3RQjxGKZ75kOGYRy5hU2+iekbltvXCyH6hclO4L8Bv2kYRqm8yl8C/16YQeftwD/D9Anf7LzswkzS+EPM4L60tFa9dsMwRjB92Z8WQrjKPaObpbn/L+BXhRC7yscNCiE+XF72DWBACPExIYSz/LlHCLGjvP+PCiGChmHkMd1gJct+HwGeMwwje7NrLbNuZaS8r/uEEI+Xrfp/hdnwXVhl/UoZ2Y1pjf8LwzC+vtIG5fN6O2b2oZXDmFbvLwpzuMYvlH9/jptQ7pk8gpndeKR8XmA+8/8krgfMG4UZiwAzdvF+IcQDwsyA/DRrNICGmRX2beAzQohaYSYB9JePK4+1onyVZemQEMIJJIEMN8rQ06scet3KS/m96Cp/7wb+E7Ca4QIV8lLergYztg7gLv8vkyx+ttyeCCHEvcD/I/cvhNglzCEZdiGEH/gMplG9mqxaj7mZ5WXZSdyuT3gYUxNbPzLd+DCmu+QyZmzFoOy7LS/73fLNjANfxxKrwkyHfwGzuzmHGZTtsmz7yfL3rvIxulY5v+cx40gJy+fpNa5nN6YVLMr/D2D2NlKY7olfqljfjelyjAMzlcsr1v0EZoOVKD+kEcxGbUfFemtdez+mBbWEKdj/GzN7Eq7HxBwV+/sYcKZ8jmPA5y3LtpX3P4fpJngOM9bgwmyYo+XtXgUesmz3D8APbwYZKa/zo8DV8nEOA7tuQ0b+HPNls8rYuYptfpWK+KNl2QHgGGYDfRw4sMaxD2O+3Evlzwng11gel7UBv4Qpt0uYPYHfqZDD0fLz/nXMRvBtxvUYxxcrjhkE/gTThb5YPuZHbiZfmMNZTpfvxzymMek3rsddxlk57riu5QVTaY1jvsPjmO9gw63KS/m3yuszLM/uW+VzlNf5Ka7L2mPl55oEZjE9JFurWV4qP/JGVTVCiL8BvmwYxt+/1edyM4QQ/we4aBjGSlbpG3XMvcDnDMOojJFVDRtJRtaibM3HMBvCoTfxuJ8BBg0z1XzTo+XldR/3luVFK7F1jhDiHkwrbQgzJfXvgfsNMwCs0dwUYVZaeRbTLfQZzB7KXYZ++TUrsNHkZTNW7NhstHDdpfIHwM9pBaa5TT4ATJY/WzFdPeuyQdKsCzaUvOiemEaj0Wg2LLonptFoNJoNyxtZMPYNRQhRdV1IwzDueAHPakHLi+Z20TKzMdA9MY1Go9FsWLQS02g0Gs2GRSsxjUaj0WxYtBLTaDQazYZlwyZ2rFdEua6rqJhE1VKCRaPRaDR3CK3E7iBCCGw2G0II7HZztgipuEqlkvpoNJrqQhq1lYbsar9XrlNpFFuxtinW9arFaK5qJbZSb+n1slKPy/qbEKJqhKtaWKmB0c9YAzfKxlrvvzSC5XLZbjgcDmw2G4VCYUWFdSuKcDNTtUpMWjc3E4CbCUal5WPtea2E7Klp9+LmYSVZKpVK+vlqsNnMtAOrLAghVPtQ+bvVg1MsFjEMA6fTidPpJJlM3rC+Ve7k+tbfq6Gd2bBlp+7EQMQ70ROzNlxCCPx+Py6Xi1AohBCCfD5PPp9naWmJQqFANpt9zYK1EQcirhfeyIGrq1nEb/W7peXl9fF6ZabSsBFC4HA48Hg8dHZ2kslkmJiYwGazUVNTQ0tLCz09PWr7UqlEsVhkcnKSRCLBnj178Pv9nDhxgng8jt1up1gsEovFVKjCMAwKhQKwXIHeapuzEWWmanticOfchxIhBLW1tQQCAXp7e3E4HCwtLZFKpZiYmCCVSpHJZN7yxk1zZ1lNaWnXcfVSqcCkHDgcDoLBIAcOHCASiRCJRHA6nYRCIXbv3s0jjzyi9iF79M899xxjY2N84AMfoLu7m8997nOMjo7icDjIZrNkMhmluKTRbO2NCSEoFAqbVharuifmdDpV115232/zHJSFVV9fTyAQ4NFHH6W1tZV9+/bhdrtJpVJEIhFOnjzJ6OgoL7/8Mrlcjlwud9vnuxGtpPXCG9UTs9lsuN1uHA4HbrebQqFALpdTjclbiZaX18ed8vYIIXA6nQSDQd7+9rfj9Xqx2WzU1tbS19eHw+HA4XAQCASor6+npqaGmpoa7HY7QgiGh4dZXFykvb0dj8fDzMwMuVwOm81GPB7n5ZdfZnZ2lrNnz+J2u2lqamJ2dparV68qF6Vs427m5t6IMlO1PTEpWNaA6e3EMayuSCEEoVCI5uZm7r//frZs2cK9996Lz+cjk8kwMzODz+cjEAhw6tQpgNekxDTrD5vNhtfrxeVyEQgEyOVyLC0tKVeyRiOEwOVyUVdXx8MPP4zdbufUqVN0d3fzsz/7s9jtdgqFAouLiywsLFBbW0ttbS0ulwuHw8HOnTvJ5/MMDw+TTqd517vehd/vByASieB2u7l69Srj4+MEg0H27t3LpUuXuHr1KnDdlblZY2RVp8SEENTV1eHz+di9ezd+v59EIkEmkyEajZLP58lms+RyORYXF8nlcqTT6Rv2IwWjsbGR2tpa3v3ud9Pf38+hQ4doamrCZrORz+dxOBzU19dz3333EQ6HSafTXL58mVdffXVZj0y7njYG1t631+ulsbGRJ554gtbWVvbu3cvMzAynTp3i3LlzvPzyyzrBo8qRiRl333236iE1NDTwoQ99iKamJux2OzabDYfDgd/vx26343K5cLvdqicmsxPb2tooFAo4nU6VGOL1ennwwQfZs2cP+/btIx6PMzU1RU1NDU1NTVy8eJETJ05gs9nUx+p92gxUlRKTKay1tbU0NDSwfft26uvricVipFIppqamyGazpNNpEokExWKRVCp1gxKzWjSBQIDm5mZ2797Nzp076e3tJRAIkMlkKBaLuN1ufD4fvb292O12pqamKBaLnDlzhlKpRC6XW2YladYv1t63DMaHw2Huvvtu+vv7efTRRxkeHlaZZMePH18XbkXNW4vdbqejo4NwOEwqlaK+vp677roLn8+nki9sNhsul2uZopHIdsvr9VIqlbDZbJRKJeVN6u/vp1QqsX37dkZHR3nuuecIBAJ0dHSQSqU4derUMuNrs/XIqkqJ1dfXEwqF+LEf+zF27NjBrl278Pv9KjCayWRURtD09DQvvfQSFy9e5Nlnn1UCAMsbs507d7Jnzx7uvvtu+vr6qKmpwTCMZamy0p3Q1tbGu9/9bmw2Gy+//DLz8/OkUim1L63I1jeyASgWi9TU1LB//362bt3Kvn37VO+7qamJt73tbQSDQerq6jh27BjHjh2jUCiomKt+ztVFLpfj6NGj9Pb28qlPfYr29nbV67IOxZmdnWVkZIRYLMbi4iJ33XUXW7duZXx8nIWFBb773e8SiUT4mZ/5GTo7O3E4zOY7nU6rNqa9vZ33vve9jI6OcvLkSXbv3k1DQwNHjx7l1KlTqncnlaVMy9/IbFoltlJD4ff7qaurY8eOHdx111309PTg8XhuyNwpFotMTEwQjUZJJpN4vV6KxaJ64NIastlsNDY20t3dTXNzM/X19crCqUzft9ls+P1+/H4/bW1t1NXVkUwmbzoaX7N+sD4nh8NBS0sLbW1tNDU1EQqFAPB4PLS3t6ve/NzcHJcuXSKZTOpqLVVKqVRifn6ecDhMX18fbW1ty5YVi0Xy+TwLCwsMDw8zNzfH3Nwcra2tNDc3Mz09zeTkJMePH2d2dpb3vve91NbW4vP5VOahVGI+n08Z5tKlWFdXx9jYGG63W7kRN1Obs2mVmBW73Y7D4aCvr49t27YxMDBAd3c3Ho9HWSZWhBC0tbXxjne8g97eXlpaWhgeHuby5ctMTU0xNTVFY2MjdXV17N69m71791JbW7sslVVaScVi8YZBiT09PfzYj/0Yr7zyimrsksnkMutIs75xu9309fXR29tLOBzG7Xar1GaPx0Nvby+hUAifz0dHRwff//73OXv2rOr1SzeSVmybH5vNRjgcJhwOKw+NlbGxMX7wgx9w+vRpvve975HNZikUCrz00kuEQiEmJiZIJBIqTvZrv/Zr+Hw+Hn/8cdrb23nwwQepqakhl8vhcDhwuVy0trby/ve/X8XE7r33XgYGBnj++ec5d+6cavfk+cgU/Y3IplZisjfmdruVVdLe3k4oFFJprpVWiVQ2cvChHEzodruVspmdnSUQCNDS0kJjYyP19fVKaeXzeUqlkmqoZGMF112LPp+P7u5url27hsfjIZPJrHjemvWL7FkHAgGVYi+VmN1ux+fz4XK56OnpIZFIMDg4yPj4ONFoVBk2murAbrcTDodpaGhQQ3mkKzGXy5FIJJiZmWF8fJyhoSGKxaLqvblcLhYWFkin0/T29uLxeLhy5YryBCSTSQYGBpQc2u12nE6nau9mZmbw+/1qLFpjY6PKorXK4UZuczadErMOMJRZP1u2bKGvr4/3vOc9HDhwgLa2Nmw2G8lkUsU3ZJaQTLWX1Tf6+vpobm5mYGCA3t5evvvd75JIJHjb297GwYMHOXDgAF1dXbjdbuWGXFpaIhqN4vF4uPvuu5dlE4Hpcuro6KC5uZlQKEQ6nWZxcfGtumWa14jT6VxmWctnXCwWVeLH9u3baWtrw+Fw0NnZyYsvvsi1a9dIpVK6R1YFyHbk4x//OFu2bCEWi5HP52lvbyeZTKoxYJ2dnUxOTtLY2Mj8/Dzz8/PLPDhOp5OJiQkA1T499dRThEIhrl69Snt7O4888gj5fJ65uTm6urq4++676erqorOzk+PHj3PmzBkeeeQR7r77bp566imuXLmyLANyo7Jxz3wVrBaFzWbDbrcTCARobGykqamJpqYm3G43YAZEZZq7LAdjzQySGUNOp5PGxkY6Ozvp6Oigo6ODrq4uurq6CIVCuFwuVVJqenqaaDTK/Pw8fr+fnTt3Luu2g9n4yfiYtJI0mw+Zhu90OmlvbycSiXDu3DlmZmaUy0izuZFKSHplVioBJYTA7XZTV1dHe3s7+XyeaDSqxq/Kogy5XI5SqaTakkgkQjab5erVq6TTaRVry2azBAIBFhYWVIJRKBSivr5eZUs3NTURi8VUOTx5HhuRTafErEjF1NTURH9/P+3t7TQ2Nqq409jYGNFolKWlJZxOpxo3JhM0stms6p43NzcTCATw+/3s3buXgYEBOjo6CAQCgOnXnpub42tf+xrj4+PEYjGampro7OyktbWVrq4uNT7D6/XS2tpKZ2cn/f39xGIxJiYmtDW+gTAMQ7lkJFYDSP51OBzY7XZ2795NW1sb09PTlEolTp06RTab3bANR7Vzq6nqNpuNTCbD3/zN3zAwMMCv/uqvEg6HMQwDj8fDtm3bmJycZH5+nj179rB3716eeeYZvv3tbzM3N0c8Hl9WRspafQMglUpx/PhxTp06xXPPPcfWrVv5kR/5EcbHx5menmbHjh0cOHCArVu3sm3bNr7//e9z8eJF3vve9/Loo4/yxS9+kZGREd0TW8/IrrL8SCtG+pynpqaYmZlRRXvD4TCBQGBZgyRT5j0eD42NjZRKJRobG1XWYjKZZHJykomJCYaHh5mamiKRSFAqlZiYmMBut9Pe3q6OLXt4Mla3kgBtZB91NWAYBplMhnQ6TaFQWJYgJHvlmUwGr9eLz+fD6/ViGAbNzc20t7czNDSkXIrSPST3q1nfVI4XtBou8iNj42DKw9TUFD6fj8XFRXw+nwpfOJ1OPB4P9fX1KhbW3NxMX18f+XxeZbXKeHqlfJRKJRVTTyaTBAIBRkdH8fv9hEIhIpEIk5OTNDU1UV9fT319PS0tLSwtLamU/EKhwMLCwoYdz7jplRiYQmQVKmlFv/rqq5w7d44zZ87gdDqZm5ujv7+fUCiE3+/H5/NhGIayhJxOJx0dHUohCSGYmJggEonw7W9/m4sXL/KDH/yAaDSKy+UiHo/zne98h507d7Jt2zblrpS9O4/Ho5RYpXtBs74pFApMTk4SDAbVUAmZpLO4uMjw8DDnzp1j37597NmzB6/Xi9fr5Z577qG+vp75+XlsNhsTExMUCgVlyGgX4/rG2hsqlUp4PB48Ho9aLt/nubk5ZeAUCgUuXbpEKpXi5MmTxONx9u/fj9PppFAoUF9fz/33369kZseOHfT39/PlL3+ZaDSq9iPbL2kMr5TFPDo6yuc//3n27dvHD/3QDzE4OMjIyAj33HMP+/btY9euXezdu5dvf/vbJJNJ3ve+9xGPx/nCF77A1NTUm3MT7zBVocQqR8DLsV6RSIS5uTmi0Sh2u52hoSEMw2B4eJhwOEx3d/eK6ffWTzweZ2ZmRqXep9NppfTS6bQqMxOPxzEMQ9U8AzM25vV6qampUSnaUlB1T2x9UygUVOwzGo0CZgMmB81PT09z9uxZgsEgbW1tBAIBvF4vDQ0N5HI52tvbicfjzM/PKwtYP++NgfTMuFwuVaxXFlIIh8N4PB7Onz/PwsKCUjTBYJBwOMz8/Dxer5dUKkVNTY2q0iEzWsPhMLFYjFwuRzgcZuvWrYyOjhKLxdQ41bUm2C2VSqTTaebn57ly5QqNjY00NzerjGm3263CI/l8XsXTNnIooyqUGCzv3RSLRXK5HGNjYwwODjI/P0+pVOLZZ59VmWTbtm1T43+klSzTY4vFIi6XC7vdzsjICOfPn+fs2bNKIABVCPbixYsIIRgaGqKlpQWv16vORQ58lgOlFxcXSaVSuif2JnE797mywcjlcly5coVCocDFixfp7OykoaGBQqFALBbj5MmT/PVf/zXpdJpAIMDWrVtpb2+nv7+f7u5uhoaGCAQCjI2NKUvbek5aoa0/5LMpFovKBSgnrHz00Uc5dOgQu3btorGxkS996UucP3+eZDKJ3W7nwIEDOBwOLl++TCQSYevWrdTV1REOhwHzeTc1NdHS0sIPfvADzp8/z44dO9i2bRtPPfUU58+fJ5FILOuRrTXB7uDgIMPDwzz++ONs27ZNDQGRZe7uvvtudu/ezW//9m9z9uzZFevDbhSqRondDOlblhlkuVyOvr4+mpqa6O3tBZZXg5YCnUwmVWUPOVeYFKxisUgikSAajTI6OooQgq6uLuUOCIVCbNmyhatXr9LY2Eg2m1VKbLPVN1uP3E7lgsp1S6US0WiUYDCossCsz6pQKJBKpRgZGeH48ePKhSwbvu7uborFIgMDA3g8HkZHR1UxaP3M1zdyKE42m6Wvr489e/Zw1113sWXLFurr6/F4POzcuZO6ujqlHOrr69W4MJ/PRzqdVuO5ZI9OyldjYyM7duxQXiLpEbp8+TKxWEwVVFgpBCF/kz1Fn89HXV0dgUAAj8ejhoU4HA4Vg9vopaeqUomtVAOxVCqRTCZJp9M888wzXL58Gbvdzl133UV/fz9wvfqGjIcZhkE0GmVycpJYLKasLjm1QqFQUAMWjx49Si6XUxYZQHt7O+FwmOnpaSYmJojH40QiEZWSqyt3vPHczstrNWJksN5utzM2NkYgEFDL5Yy76XSaU6dOMTMzo5I/du7cSVNTkyoavLCwwOXLl1VgfSM3Jpsd69hTWQThwIED/PIv/7IqA5VMJikUCjz++OM4nU4Vfz937hypVIquri4MwyAej5PNZimVSgQCAVVztVgsqkzC5557jmg0yl133cWePXvIZDJqlnir4lnJEHO73dTX19PU1ERHR8ey0miV17TRZa4qlZik8uFJ5SQFZXR0lLa2NpLJJE6nc8UsQinUlfu0Zqolk0mGhoYIh8MqIKx565BZYTIWsdqLbJ3u3TqhoDWwns/nSSQSy9wxUiacTqeaT+7atWvU19dTV1enXNG1tbXs2bOHUCjE+Pg4U1NTXLlyRc81t46RiiYYDNLY2Ijb7WZhYQGXy4Xf71dJWlZXnywMLccGyt/sdjs1NTWq/J1VDg3DoKOjAyHMSTEXFhYYGBigtraWM2fOqNT7lWYTF8KcpHf37t14vV6uXbuG3++nublZyfG1a9eYm5ujrq6OgYEBLly4sKwY+UaiqpWY7E5LC1oKYCaTIRaLcfHiRTUoUJYYqkSm7lsrL1gtIznZ3fnz52lqalrm05YvxEoN6Ea3jtYzNpsNn8+n6szJF7uSfD6velRWF4581jLGsLi4SDKZVHIklZjb7SaZTBKJRDhz5gzFYpHW1la8Xi/t7e0EAgEefPBBIpEI0WiUK1euMDw8rJXYOka+s7W1tezcuROv18v4+LgaXyoNJGtIAaCjowPDMFQmqzXz2WpMWdukrVu3MjAwwD/8wz8Qi8W4++67VbJYPp+/wa0oFacQgtbWVh5++GGy2SynTp2isbGR/v5+ZZQdOXKEK1eu0NvbS2NjI4cPH96wVYOqWolVYh1EmM1mmZyc5Ny5c3z9619nx44dHDx4UPXIpKKSMbOWlhYSiQTxeHxZOaHKKVys8TS73a5G6re1tanqH9JPrblzWGMGciLL+vp6urq6VEZhZRbr0tIS6XRaTY8hx3ZJl1HlvsE0arxeL93d3TzwwANcuXKFoaEhpqamyOfzNDY2srCwwH333Udraytut5tgMMiuXbuoqanh6NGjzM/PMzs7q3vs65jFxUWuXLlCZ2cnYBqruVxOxZsqE4Hkd5fLtawNkH+tyRqyHZLbbNu2jfr6ei5cuEChUODAgQN0dHRw7NgxEokE2Wx2WSUQIQSTk5McPnyYpqYmWltbeeGFF/jBD36g1pEeoZdeeonFxUUikYgeJ7YRqSzOW6nEpqam1PiwbDbLrl27AJZZWo2NjfT09NDc3EwkElEDWCv91FYryarEhBCEQiHa2trUlORyXjPNncNa8qexsZH3ve999Pb2ctddd5HNZkkkEiqeCaY8zM/Ps7i4yKuvvsr4+DiAsoKtPW7rM3U4HKrA8/33308ul2N0dJSpqSkmJyfx+XzMzc3R0tKC2+2ms7OTYDDIzp07VfV7Oc5Is36Rve89e/YA113LUolVKiIpL1KJSaxKzCpHcF3xbdmyhd7eXqampojFYmoGZ1ksOJfL3eCKnJqaYmFhgQMHDigldvz4ccBsd97znvfQ3d3NP/7jPzI+Pq4mAd6IbGolZi3VYnX5yeKsd911Fy6XixdffJFYLKayC60N3uLiIufOnVP1DQ8ePMjdd9+t4hr19fXk83nC4TD19fVMT0+rY69EZfzFZrOp2aHlWJOFhYVlilC7Fl8/1vueyWQYHx/HMAxV+dtutxMMBpfNCdfQ0KBcyFu3bqWvr0/FvzKZDGNjY3g8Hnbt2kVPT4+SG9m77urqUuOGZJ3Oa9euEYlECAQCXLx4Uc3wOz09zdTUFEtLS0oONesX6Za7fPkyTz31FB/84Afp7e1VMgDXB0bL75VULrfGy6V70aoM9+/fT3d3N6dPn6ZUKvHII48wOzvL4cOHSaVSy0IT0lUpi03PzMwoN7UQgmPHjqmppaxZ1RuRTa3EJDJOUVnYd8uWLaqOXTqdVoP+rNZ4MpkklUqpbn59fT379u1TJawCgQDFYlHNHSUV5e0MVvZ4PNTV1akMp1gspvYhz0Pz+qgc4zU7O4sQ5pQ7Xq9XpSBbq6dIw8Xj8ZDL5ejv71dZhplMhnPnzlEqldR4H2k0ySB/U1MTtbW1uN1u1YBMT08zOztLbW0tk5OTuN1uQqGQGjgtlZ1mfSMVy/j4OMVikfvvv/8GN2Jlz6pyW1juDRJCqGlaVnIr9vX1kc1mlVt7z549TE1N8corryzrjclPoVBgenpaGdZWBgcHlx3X+nejsemUWGVjVSwWGRkZwe/3s2/fPjo7O1WZGNl4uVyuNQtgGoZBKpVidnZWTQV+uw9+NYGG69X2pXtxowrTesYqF9FolOeee04psNraWurr69myZQsDAwM0NzfT0NCg5l7yeDy43W41+wGY7h+3262yFWtqapRLyOFwEAqF6O7uxu/3k81ml7lqDMNgdHSUSCTC0tISNTU1ZDIZNa4snU5rw2UdY1VUcn7BhYUFYrGYmkeuEjlGTK776quvquzU+vp62traliWXSXmxeoVkzOree+8lFovxve99j1KpRFtbGzU1NYyNjb2mMMRG7oXBJldisrpGLBZjcnJSjc1wu93KCpe1ziqD+pXIVGoZRJVjxuR4sNcTw7Ied6WZpjV3llQqxdWrVwFTXmprawmHw0o+ZDaiLAkG16fLkN8BpcSk8pK9eJvNpnrXNTU1ykiRsRJAjStMpVLLxhXG4/ENP/i0GpCKLJ/Pk0qlVFKXrOBR+Q7LntH8/DxjY2O88sorKgQh47RSdmSGovV/uJ541tnZSV1dnUrU8Pv9qgjw61FIG1XmNp0SgxunSYhEIthsNmZnZ1lcXMTj8aiq9Y2NjdTV1RGLxZbVOqtENlajo6O8/PLLeL1eXC4XFy5cYGpqipMnTzIyMkI2mwWWT3IohX214q7SF74ZRs9vBKSykKTTaSKRCLOzs5w7d47Ozk6am5vp7u5WczDJAcw2m436+noCgQADAwOqJyflTTYgUr7uu+8+kskkCwsLxONxNf1PJBIhk8moMkBWF5B+/usf+YwaGhro7Ozk3Llz/OEf/iGPPfYY27dvp6WlBZ/Pp9a32+0kEgm+8pWvMDk5ycjICGNjY4yOjtLa2sru3bvZtWsX9913n1JecF1ZVrYLDoeDnp4estmsGkcm267XIj8bWeY2pRKD5amq2WyWpaUlNdhQKhjpTpQ9srViULKA5sLCAsPDw3i9XtxuN1evXmViYoKZmRllRa+0j5tZRxtZiDYaVtcMmL1smawRjUZJpVLMzc2xuLioelOycrnNZqO1tVXFwYrFIl6vd1lsQ06A6HQ6aWtrY8+ePUxPTxOLxSiVSjgcDtLpNMVikUKhsKywqy4Avf6xPhdZxFu6E/v6+ggGg8vCFNKolnMYygo/xWKR6elpFhYWKJVK+Hw+du/ereLtKxm0sscvPUGlUol4PE4ymXyzb8O6YdMqMSsyC0jGQKTrsKenh0AgQFNTkyr3BCzLMJKkUimmp6f5/ve/z5UrV1QNsqmpKeLxONFoVJWRkcIlcTqdKnFjpZhXZeKJ5o3HaunCdSOlWCwyOjrK5OSkmr7dWmbMZrPR2NhIMBjk3LlzNDY2snfvXkKhEF1dXdTU1Kj56Ox2O9u2baOrq4tkMkkymVQ993PnzrGwsMDMzAyJRELFWxcXF7Ui20DImdxbW1sJh8M8+eSTfOc73+EjH/kIu3btoq+vT/XIXC4XW7duBeD06dMqK3BsbIyzZ89y/vx5Lly4QF9fH729vczMzLC4uMijjz5KX18fYMrpwsIC4+Pj/Jf/8l+4du0aS0tLb9n1rwc2rRKrfPmtA/ysPbFsNovP51OlX1ZTJHJA7MLCAplMRq0bi8XU9CuV1Trg+qScwWAQn8+3pqKSykzz5mB1O8N1GZG96UQiodaTy202G9lsllgshtfrJRKJqMlSPR4PgUBAxUXk9BqBQIBsNksulyObzeL3+1V8xOfzsbS0hGEYKuYq42Naga1/stks2WxWJQDFYjGEEExNTdHc3KxmdAfTBdja2sri4iIul4tsNqt6UIuLi8ozJGVAegVmZ2dVgelCocD4+DgjIyNcuXJFjV+sZjatEoPrikwOCJyenubatWu4XK5l8/h0dXWRSqU4ceLEstiVtfEqlUrk83kWFxdZWlpatsyaUmtFjkdramri3nvvZdu2bapmnzV+spKC1byxVN5nay+8UrlVEo/HWVpaUsWdX3rpJVpaWrjnnnvYsmULDz30EPX19bS2tiqlKMuT7dy5k4GBAe69914KhQJzc3PEYjFeeuklpqamOHLkCAsLC1y9elXLwgZiZmaGhYUFwuEwtbW1XLlyRc1SIAvvBoNBPvzhDzMyMkKhUODKlSs8++yzymAZGxtjbm6Ol156CY/Hw6FDh9i+fTv/9//+X9LptJot/OzZs8RisRVT56uRTavEKrMU8/k8CwsLTE1N0dPTo5bZbDYaGhoIh8P4/X5V/Nc66Nm6T2vcAm605ivjarKGXjgcJhQKLStHJdeXJWtyudyyiTE1bx63k9UljZxcLqfGdRUKBRobG6mpqVG9s8rqCzId32azLUvP9/l89Pf3q9iKx+NhYmKCbDa7YUsBVQPW9172nuV7bB13KrHb7Wp6lu7ubhKJhJoNXL77MtvR4XAwMzNDfX29muFCrjc0NFTVMbBKNqUSs7qHAGXFvPLKK8zNzalah7K6wsGDBwmHw5w5c4axsTHOnz+vMtGsYzYkK6XPrjR4UTZWoVCI7du309HRsazuomRpaYmpqSnm5uaIRCJ6Xqm3iMpqKrB6Qo51TE8ymVTJIdFolLq6Okqlkop/gOmalCXJpDwJIfB6vdTX1/Oud72LbDbLXXfdxcWLF4nFYszNzTE2NqblYAMg5WRpaUl5c6xxbvnMZYGE+++/H5/Px6lTp9RzlsgxZUeOHOHkyZNqvCvcmJSk2aRKrBLZe5IDSq3xBmvVBpfLpao03O7+10LGuqwKTrqZ5Di2ubk5EomEGnmvkzzWBys920pXsoxVJBIJYrGYSvaxPselpSWVwSYtdCHMauN+v5/29nY1bXwymWTnzp2MjIywsLCgrHTN+mIlg0eGLmQij9VgsW4jCz93d3djs9mYmZlR7YFExlH1sJu1qQolBtenUJCWsFVYZLziVgca36pAyQZOCrZsiGQm3OLiIvPz85w9e1YVmbVOrKmFd31iVV5wPe05mUwyNTXF8ePHVXV86T4cGhri7NmzHDt2jImJCTXI+ZFHHqG7u5v3vOc9BINBNZA1GAxy9OhRZmdn1Rg2zfrGOh5UzuQu55mTXh1rT7ylpYUf/dEf5fjx4+rdXynTUGeprk3VKDEwYxipVIqlpSWSyaQq4ut2u1VmmTWDcCX30s2o7EXJrMb5+Xlqa2uVqyCVSjE8PMzFixdVIU5Z0qrSHapZn8hnbX1m2WyWaDTKzMwMIyMjBINBamtrcTqdqs6m7JUVi0UuX75MIpGgra2NcDhMS0sLAOFwmLa2Nvr7+3E4HEQikWWJP7phW39YS9HFYjFcLhcnTpwgnU6zY8cOFSd1OBxqJnCfz0dLSwv79+9nbGxMJYRYZ9SQ+9TPe2WqSoktLS0xNzfH5OQkExMTeDwefD4fwWCQbDZLe3s7yWRyWb2y16LE5DbShRmNRjl79iw2m41Dhw6RTCYZHx/n+eef58tf/jLT09PMzMwol6POUtw4VCqyZDLJ6Ogop06d4rvf/S47d+5k+/bt1NXVsX37dl566SVSqRQTExMqw9Hv9zM8PExnZyfvfe971QSG+XyeJ554gqNHjzI6OqrSua1jDXXDtn6wjvccGRlhfHycz372s/T39/Nbv/VbdHV1qfnmurq6mJ+fZ25ujp07d7Jt2zaee+45ZmZmSKVSapZlq2LUz3plNq0SW+mBZzIZlUQxOjpKU1OTGs9TU1NDc3Mz8Xic+vp61VuD115NXroSk8kkY2Nj1NXVMTMzw/T0NKdPn+by5cvMz8+rWYFlnEQL68bC+rxKpZKaUPXIkSNEIhEmJyfV0AqZ+SoNFVnsd3BwkEQiQWdnJz09PfT39+PxeFTKtsPhWBYX03Ky/pAVXSSGYc7rZRgGhw8fpq+vj4MHD+J2u9W8c21tbWr2746ODh555BEuXbqkemSyModmdTatElsJWXrq4sWLCCHo7e3F4/EQCoXw+/1s27YNm81Ge3u7Ui5wew1G5XpybNnp06ex2WxcvXqV8+fP8/TTTzM0NLRssKJulDYmlcM50uk0Fy9eVIaS7Fm1trYyPz+/LNNMFgFeWFhQ7ub9+/fzzne+E5/PR3t7Ow0NDTidzmXzQWnWF9YQgBx/CjA0NMTExASf+9zn2LFjB9u2bVNFx4PBINu3b2dqaor5+Xl27NjBnj17+OpXv8rU1JSaQVy3C2tTVUpMJlQMDw8D8NhjjykhcTgctLS0kEql1JTjo6OjNwxKfi3I0ffXrl3jO9/5DhMTEwwPDxONRm9YVwvs5qBQKJBOp1lYWFDjvWZmZtRsCrJXZVVIuVyOiYkJ/H4/3//+91Vx17GxMfL5/DLlpxXZ+kTGsqQSk5NbTkxMAPDkk0/S29vLww8/rBLKamtr6evrIx6PE4lE6Ovr44knnuDEiRNcuXLlhviYZjlVp8RyuRxXrlwhGo0Si8XI5/MYhoHT6aSjo4NSqURPT8+y6uKv13WTz+eZn58nk8moYp3z8/MrVgbRbA6kK0gm9czMzOB2u1XQPp/P3+B6yufzjI6OUigUeOaZZyiVSsRiMYaGhpaNFQJ0w7ZOsSbeyCxjwzDnj5ubm+Mv/uIv2L9/P4cOHVKhDJmNeu3aNWZmZti+fTvbt28nmUwyNDSk2ijNylSVEpMkk0kcDgeDg4N4vV71kcVbGxsb1cy/Mlj7eoXIMMxq+nNzcyoNV2chbm6sxo+sxGKt+rJSfc9MJsPc3Bwvv/yyMrrkAPjXM9WG5s1lpR5ZqVRiZGQEwzD4whe+wMDAAO94xzvUcBtZIPjkyZOcOXOGyclJbazcAlWpxDKZjMog8vv99Pf343Q68fl8+Hw+NV+U1WVzJ9w3siSN3J8Wzs1LZQahLEu0FtLQyefzHDt2bFm1Bx0b2VjIHpnMVpTjPsfGxpTCuu+++3jkkUeWFZTu6enh5Zdf5tixY8zOzq46v6HmOlWpxGQa+5EjRxgbGyOZTKoxOrFYjFOnTjE0NKSsoDuV8q5To6uHyl72SqXKVtpGDpy2KrzXMtRDsz6Q2YWykILMMr148aKqkdjX18d9993H8ePHOXr0KCdPnuTy5cssLi6+1ae/IahKJSZ7QVevXmVmZgaPx0NDQwO9vb3E43GGh4eZmZlZNt7rTjQiVtfknYi1adY3lYNVV/p9tW20Bb45WKlHVigUmJqaIpFIkEwmeeihh7j77rsZHBzk61//OrOzs8zOzuq24RYRG/VGCSFe84nLHpGc96m+vh6Xy4XP51OzN8sJCu80r6cnZhiGTkl7jbweeXkruFWltxZaXl4fd1JmZJvjcJj9hmKxqManNjY2snXrVqanpxkeHlaD2t+KHvhGlJmqVGIblY0oYOuFjSYvt+J+vBlaXl4fd1pmpBKTHhhZBFz20GRxhLeSjSgzVelOlKw2EaI1i0yjeSvQsrf5kAk+lTO4S+Wln/lrw3bzVTY31mQL63eNRqO500gjWaITvV4/Vd0Ts47ZAS1QGo3mzeFmwy00t05VK7FKtPLSaDSajcWGTezQaDQajabqY2IajUaj2bhoJabRaDSaDYtWYhqNRqPZsGglptFoNJoNi1ZiGo1Go9mwaCWm0Wg0mg2LVmIajUaj2bBoJabRaDSaDYtWYhqNRqPZsGglptFoNJoNi1ZiGo1Go9mwaCWm0Wg0mg2LVmIajUaj2bBoJabRaDSaDYtWYhqNRqPZsGglptFoNJoNyxumxIQQh4UQn3yzt73F/f9ACHHgjdr/ZkcI8RkhxM+9wcdYz/Lzt0KID75R+9/oCCH2CiFeehOOs55lRLcxrwMhxFeFEO+5lXVvqsSEEMNCiMdf/2m98QghnhVCGEIIxxrrvB9YMgzjRPn/3UKIZ4QQ80KIG6a5FkL0CCG+KYSICiGmhRB/ZN2/EGK/EOKYECJV/rt/jWMfFkJkhBBLQoh4ef1/J4Rwv74rf9P5r8CnhBCum6243uVHCPHx8nOICyHGhRC/fxP52QvsA54s/y+EEL8mhBgt7+NLQohay/puIcTny8umhRC/tMa+PyGEKAohEuXPkBDiz4UQA3fymt9oDMM4DcTK79pN2QAyUvlcEkKIR9dYv7KNWVXGyvLxZ0KIkXK7cNLaeJfbH6Pi2L++xrGHhRDp8r5iQoiXhBD/XAix0bxu/xn47VtZcaNd2KoIIT4KOG9h1X8O/JXl/zzwZeBnVln/s8As0ArsBx4Bfr58TBdmY/ZFoA74C+DJmzTuv2AYRqC8v38NfAT4phBC3MK53xZrNcavB8MwpoCLwA+/Eft/k/EC/woIA4eAdwC/vMb6Pwv8tWEY0uD5KeBjwINAG+AB/tCy/qeBrUA38HbgV4QQ715j/y8bhuEHgsDjQBo4JoTYfVtXdQu8UfJR5q8x79Vm4WXDMPyWz+E11q1sY9aSMQcwhtmuBIF/D3xZCNFTsc+Q5di/dZNzfX+5jekGfg/4t8Cf3WSb14QQwv5G7NcwjCNArRDi4M3Wfc1KTAhRJ4T4hhBirtxL+YYQoqNitX4hxJGyBfKkEKLesv19ZSshJoQ4tZZlcwvnEgR+A/iVm6znAh4DXpC/GYZxyTCMPwPOrbJZL/BlwzAyhmFMA98CdpWXPYophP/DMIysYRh/AIjyMdbEMIxk+UX4YeB+4IfK52gr984GhRALQogvV9y3nypbbQtCiF+3WrFCiE8LIb4ihPiiECIOfEIIESxbelNCiAkhxG9bBU8I8dNCiAvlZ/iMEKK7/LsQQvx3IcRs+fmdqWhID8tzfi2sF/kxDONPDMP4nmEYOcMwJjAb3wfX2OQ9WOQHeD/wZ4ZhjBmGkcC0IH9cCOEtL/848FuGYUQNw7gA/CnwiVs4r6JhGIOGYfx8+XiflsvWunYhRK8Q4sWyJf5dIcQfCyG+WF4mrfqfEUKMAs+Vf19RBsrLtgshviOEiAghLgkh/oll2XuFEOfLx5oQQliV/2HgHeJ1eBnWi4zc5jmv1MasKmPlduDThmEMG4ZRMgzjG8AQcPfrPRfDMBYNw3gK+HHg4/L9FWbv778K03swI4T4X0IIj+UafqXcXkwKIT5Zlpkt5WVfEEL8iTC9U0ng7UKINmG6/+aE6T34Rcu+Vm3PhBA15bZqofyMXhVCNFsu4TC30Ma8np6YDfhzTG3fhWkx/lHFOj8F/DRmr6MA/EH55NuBf8DsLtZjWiVfFUI0Vh5ECNFVvsCuNc7ld4A/AaZvcs5bgZJhGOM3Wc/K/wA+IoTwls/7PZiKDExldtpilQOc5rqSuymGYYwCR4G3lX/6F8AHMS2zNiAK/DGAEGInZs/wo5j3NAi0V+zyA8BXgBDmy/IFzHu/BTgAPAF8sry/DwCfAn4UaAS+B/xteT9PAA8DA+Xj/BNgwXKcC5hutdfKepIfKw+zikEjhPBhGjWXKhdVfHcDW4UQdeVzP2VZforbkI8yf0dZPm7h2v8GOAI0YCq+j62wv0eAHcC71pKB8vV+p7zPJkyvwWfLcgimdf+zZat/N2WlCFBurPPAttu8VivrSUYOCDPkcFmYxuNqvdhbaWPWkrFmzHeucvmIMF2Rfy6ECK+x7xso92rGud7G/F75GPsx24V24D+Uj/9u4JcwvQBbMA31Sv4/wH8CAsBLwNcx5bods5f5r4QQ7yqvu2p7hmngBYFOTHn955jPWHJrbYxhGGt+gGHg8VtYbz8Qtfx/GPg9y/87gRxgx+ze/lXF9s8AH7ds+8mbHbO87kHgJGaPqAcwAMcq6z4ITK+ybIt5O274fQdwDPMFMTCVgigv+3XgSxXr/zXw6VWOseJ1AV8C/rT8/QLwDsuyVszGwIEpaH9rWeYt39PHy/9/GnjRsrwZyAIey28/ATxf/v408DOWZTYghdloPAZcBu4DbCuc8zuBaxtdfir28dOYL3t4leXtZRmosfz2yfJ96sF8IZ8qr3M/5stZuf47geFV9v8J4Psr/P5uIF/+vuq1Yzb0BcBrWfZF4Ivl7z3l8+mzLF9LBn4c+F7FsT4H/Eb5+yimy7B2leuZAB7e6DIC9GEaLzZgD3Ae+NVV1l21jbmZjGGGQ74LfM7ymx+zjXNgvs9fAZ653XsJ/CPwa5hGVhLotyy7Hxgqf/888LuWZVvKMrOl/P8XgL+0LD8EjFYc61eBPy9/X6s9+2lMJbh3lWv5Z8BzN3s+r8ed6BVCfE6Yrq048CIQEst9pGOW7yPlhxTGfEE+XLZ+YkKIGPBQ+QJv5xxsmD2Tf2kYRuEWNoliWg+3s/9vYVrCvvK512G6jAASQG3FZrXA0q0eo0w7ECl/7wa+ZrkvF4AipgC3YbmnhmGkWN47guX3vBvznk9Z9vc5TKtaLv+flmURTCFvNwzjOUyr94+BWSHE/xaWhAXM+xi7zetUrAf5qTifDwK/C7zHMIz5VVaLlf9aZejzmD2Xw5jW8/Pl38cx5QOWy8idkI/Vrr0NiJTlQjLGjVTKyIoyUF52qOJYHwVaytt+CHgvZi/hBSHE/RXH2RQyYhjGNcMwhgzT3XcG+E3gx1ZZfdU2Zi0ZK7c1f4WphH/BcuyEYRhHDcMoGIYxU172hBDiltuxMlKGGjGN32OW+/Kt8u9Q0cZwa/LTVnGfP4XZXsnlq7Vnf4VpWHyp7Lr8fSGENa/hluTn9bgT/zWmq+CQYRi1mF1kWO5a6bR878LUwPOYN+GvDMMIWT4+wzB+7zbPoRbTSvk/Qohp4NXy7+NCiLetsP5VzHBPpQtuNerL5/1HhhnzWsB0b7y3vPwcsFeIZUkZe1k9vnYDQohOTP/398o/jWEKufXe1Bime2YK6LBs68HshluxujbHMHtiYcu+ag3D2GVZ/rMVx/IYhvESgGEYf2AYxt2YFu4A8G8s+97BcjfZ7bIe5Mc8oOlC+VPMgPiZ1dYzDCMJDGLeC/lbyTCM3zAMo8cwjA7MZz8BTBiGEcV8ZlaXyD5uQz7K/AjL5WO1a58C6sX1eBwsv4fqtC3f15KBMeCFimV+wzB+rnztrxqG8QFMo+jvMROkAOXOc3Gj6/V2WDcyUoFRcQ5WVmxj1pKxcvvxZ5gN+4cMw8jf5NhwG223EOIeTCX2fcx7kwZ2We5L0DCTiaCijeHW5Geo4j4HDMN4r2X5iu2ZYRh5wzD+o2EYO4EHgPdhuoclt9TG3OqNcJaDcPLjwNSSacxU2nrMxIpKflIIsbP8Uv0m8BXDMIqYLo73CyHeJYSwl/f5qLgxaHszFjEth/3lj7xxdwOvVK5sGEYOs7v+iPxNmNRgvnAy2Ogurz+PGWT9OSGEQwgRwnTbnC5vfhjTqvhFYQZLpQWlYgOrUbYyH8HMbjwCfLO86H8B/0lcT7BoFGbcAkxXwvuFEA8IM4D8aVZ/mTDMLMJvA58RQtQKM8jaXz6uPNavCiF2lY8VFEJ8uPz9HiHEobJllAQyQMmy+0cwXVG3wnqVH4QQj2G6gD9kmLGDm/FNlstPffmeCmHGiv4b8JuGYch79ZfAvxdmksJ2TBfJF27hvOzCTNL4Q8y4xH8sL1r12g3DGMGMr35aCOEq94xulua+qgwA3wAGhBAfE0I4y597hBA7yvv/qBAiWG5049woH88ZhpG92bWWWc8y8h5RTjgoP8NfpzzEopJV2pibydifYDbY7zcMwxoTovwObiu/uw2YMb/DhmEs3sJ51woh3ocZrviiYRhnynL5p8B/F0I0lddrF9djWF8G/mn5GXvL17oWR4AlIcS/FUJ4yvd6d1lxwhrtmRDi7UKIPcLsWccxDZDbb2Nu5m/E9LEaFZ/fxlQehzFdJpcxfeMqHlVe9rvli4xjBv/CxnJf6guYXdw5zCBsl2XbT5a/d5WP0XUL59rDGjGx8jo/BDy9wjbWz7Bl+f7y+UQxrZgvA82W5QcwY2Zp4DhwYI1jH8ZUBkvlzwlMP7U1ZmLDDKxeKq8zCPyOZfknMGMRC5gCNgG8rbzs05TjH5b1g5gvyTim0j8BfMSy/GPAmfIzGgM+X/79HZjKOlG+7r8G/MZ1v/Y44Nro8oPp/iuU15Gfp9e4nt2YPSkZFx0oP6sUpjvrlyrWd2O6HOPATOXyinU/gWkUJTANhxHMYRs7KtZb69r7MXttS8CzwP/GzJ6EVd6P1WSgvGxbef9zmDL3HOY74cJ0Q0XL270KPGTZ7h+AH76ZfGwQGfmv5WeXBK5hKkvnbbQxq8oYprvNwGwXrMs/Wl7+E5iGdBKzl/SXQMtN7mW6/PwXgZeB/wewW9apwUyGu1a+bxeAX7Qs/1XMJLlJ4OfK59dZXvYF4LcrjtmG6VKfLsvDP3I9Tr9qe1a+tkvla5vBVNDy2d4DHL8V+ZEvYlUhhPgB5nitE2/1ubwehBB+TJ/xVsMwht7E434GGDQM47Nv1jHXE0KIv8EcdvH3b/W53AwhxP8BLhqGsVIv5o065l7M5ITKGFnVsInamB3AWcBt3FrewZ067lcxja9v3nTdalRiGxlhVgN4FtON+BlMa/MuQz9IDSr+EcG03p/AjFXdv9EbU82bhxDiRzDd5l5MT0DJMIwPvqUntQabpmJHFfEBzG7+JOaYlI9oBaax0MJ1F9wfAD+nFZjmNvlZzCpFg5ju7Te0TurrRffENBqNRrNh0T0xjUaj0WxY3sgCoG8oYoWK82/CMRHlIWGyB/tm9mQNw7jjRYKrhbdCXt5qtLy8PrTMbAw2rBJ7sxCWccyWlFKNRqPRrAO0EluByt4WQG1tLXV1ddTU1OB2u0mn0+RyOSKRCOl0mkKhoBWcRqNZkZXaFM2dQSuxCoQQ2O12DMOgWCyag+mEYOvWrbzzne+kra2Njo4OhoeHmZ6e5vDhwwwODrK4uEg+n8dmM8OMpVLpJkfSaDTVgt1ulnssFN60oVZVg1ZiKyCtJZvNhs/no7Gxka1bt7Jz506am5tpbGxUwuj1erHb7cvcjprq4Vaeu7a+qxdp1FbKgDSUtbH7+tFKzIIUuGKxiM1mw+Fw0NbWxgMPPMBDDz3Eu971LjweDz6fD5fLhdfrJRQK4XA4VGOmlVn1YLPZliX7rMQKZYE0VYRsG3K53DLj2OVyUSqVyGZvtbSkZjW0EmO5v1oIgcvlIhAI0NnZydatWzl48CC9vb14PB4cDgf5fJ7Z2VkGBweJRqNks9m3JFtR89Zyq89cLtdxkeqjWCwihMAwDFwuFz09PdTV1bFt2zbm5ub43ve+Ry6XI5fLvdWnumHRSozl/mqbzYbH46G1tZVHH32UHTt28M53vpNgMIjf7yebzZJOpxkcHOTVV19lamqKZDK5TFh1I7X5kQrpdtxBq/XWtbxsXorFovru9Xq5//77GRgY4Md//Mc5fvw4586dIxaLKSUm2xDNraOVGNcbIhkD2759OwMDAzz88MO0trYSCoWw2WzE43GmpqaYnJzk3LlzXL58mWg0uiwzUQtgdSCEwOPxYLfbqampAa7LkbUhyufzFAoFcrncDRms2vVcHdhsNrxeLy0tLbz97W+np6eHYDBId3c3P/zDP8yFCxd47rnndNvxGtFKjOuNj8PhwO/3s3PnTvbt28djjz2Gx+PB5XKRSCSIRqMMDg5y5swZTp48ycWLF0mlUuTzed0gVRlCCHw+HzU1NdTV1QGmwgKz0ZI98lQqRSaTYWlpSWW7SirdjJW/azYu1lipzWajtraW1tZWnnjiCVpaWigWi/T09PChD32I7373uzz//PPa7fwaqXolJoTA4XBQU1NDX18fnZ2dPPjgg/T09OByucjlcsTjcYaGhjhz5gwnTpzg5MmTjIyMkMlkKJVKWoFVCXa7Hbvdjt/vp7a2lvvuu4+Ghgb6+voASKVS2O12HA6HyjxbWFhgYWGB0dFRZmZmyOfzFItFSqUSxWKRxcVFcrmckiXN5qAyrJBKpVhaWmJ6ehqn00ltbS0ej4eenh5aWlrw+/1kMhkdG3sNaCUmBE6nE7/fz5YtW+jv72f//v00NDTgdDrVgObBwUFefvlljh49ysmTJ5dtL9GW0+ZGZqzW1dXR3NzMoUOHaG9vZ9++fQAsLS1ht9txu92q1zU5Ocnk5CRnzpxhaGiIbDZLPp9Xn1KpRDKZ1FlqmxjDMMhkMiQSCebn5/H7/QQCAdxuNy0tLTQ2NuLz+SiVSlqJvQY2bBX711vXTMYt3G4399xzD52dnXzwgx+ktbWVHTt2AKb1NDQ0xEsvvcSFCxd49dVXmZ+fZ2Fh4S1RWBuxrtl64U7Ii9vtJhQK8VM/9VP09fXxwAMPEAgEqK2tBVC9cqs7KJ1Ok8lkiEajJBIJSqWSSq1Op9M888wzjI6O8tJLL5FMJu/o+CEtL6+PO1E7UabT9/f309DQwL333suWLVv42Mc+Ri6X49q1a4yMjPDqq6/y6quv8t3vflceW7cxt0hV9sSsWWJOp5Oenh4GBgY4ePAg9fX1BINBlpaWmJiYYHR0lFdffZWhoSEGBwe1y6eKsdls1NTUcODAAbZv3862bdtwOp0rxsIkwWBQZb/CddnL5XIkEgkmJiaw2+2cOHGCbDaLzWZTik6z8bH23n0+H5cuXVJGSj6fZ2ZmBofDwb333svc3BxwPZ62UTsYbzZVqcTkeDBZD1G6Eevq6vB6vWQyGWZmZnj55Zc5ffo0R44cIZFIKBeRFq7qpXLgsixPlk6niUQiLC4uMjU1peJnwWCQYDBIY2OjUmhWF/bb3/52du3aRSgUYnJykhMnTrC4uMjk5KRWZJsA2es+f/48jY2N/OiP/ij9/f3EYjHcbjd33303ly5d4plnnmFqagq/368yWrUiuzWqUokBSok1NDSoj9vtRghBKpUiGo0yPDzM6Ogo09PTqkGpFCpZtcHpdCor2jCMZSP0NZsLmZhh/V/GTmdnZ7l8+TJ2ux2n00k4HCYcDquEj5qaGiUrTqeTzs5OgsEgc3NzNDQ0MD09jRCCmZkZbTBtAgzDoFAoEIlEcLlc1NXVEQwGyeVyuN1uGhsbGRkZYX5+nlQqhcPhWDa2THNzqlKJSSv4J37iJ9ixYweHDh0iGAxSLBaZn5/n8OHDXLx4ka9//etEIhElVLKbL5WWbKRCoRAPPvggjY2NzMzMEIlE+Na3vkU8HteN0CaiVCpRKBSIx+PE43GlyOx2O8lkknPnznHmzBm+/vWv43A4cLlc1NbWEgqF6OjooLm5mUceeYT+/n6ampqoqanB7/dTU1PD448/Tjwep6uri8uXL/OXf/mXxGIxFhYW3uKr1rwerGnzhUKBsbExXC4X+/btU714t9tNe3s74+Pjy+KmmlujKpWYtJJlLKypqQm3200+nyeRSDA6OsrY2BgTExNkMpll28qYhsvloqamhqamJhobG9m+fTsdHR2MjIwwPT2Nx+MhlUrpKVo2ETKWkU6nSSaTFAoFVRvPMAyy2azKQJMua6/Xi9frJRaLMTMzQ09PD36/H4/Hg2EYqkfW1NSkMmTz+TwNDQ0Ui0UikYiWn02CNJLr6uqW9eZlyr0cNG9FuxRvTlUqMdlwtLS00N7ejt/vV+nQo6OjHD9+nLGxMZLJJIZh4HCYt0k2YoZh0NTURHt7O48//jjbtm3j7rvvJhQKEQ6HGR8fp6enB4fDwdTUlHYPbHCsyRjJZJKrV69is9nYuXMnoVCIQCBAXV0d+/fvx+fzUSgUmJiYYHBwkHg8ztjYGLOzs7hcLtLpNF1dXdx33320tbWxZcsWAoGAcmf39fXhcDh45zvfyfnz55menlZjyzQbF8MwiMfjPP3004yMjPD4448r4yUYDDIwMMDU1BTd3d3EYjFisZjaVrY5mpWpOiUmywVJa1jGwfL5PNFolPn5eZUOLWMSMusMzF6czWajrq6O1tZWOjs76e7upqGhAZ/Ppyp8OBwOPUXLJkPGN2KxGJFIhFQqhdfrVUWjpUxs374dt9tNJpPBMAwWFxdJp9Ok02nGx8cpFArU19eTTCYJBoMYhkEoFMLpdOLxeAgGg3R2dhKLxfD7/aRSKVKp1Ft9+ZrXSbFYVLKztLREOp0GzEpBcnaMQCCgftfcGlWlxGR1jh07dtDf308oFMJut6txPN/73vcYHBzk2rVrJBIJnE7nst5XsVikvr6euro67rvvPu69917uvfdeOjs7qampoVAoEI1GmZubIx6Pq4ZHl5HZ2EjXIJhFoi9evEg2m2VqagqbzaYGq3Z3d9PW1sa+ffuYnJxkcHCQZ555hm9/+9tEIhHi8TiXLl1SpctCoRA/+ZM/qVzaTqcTh8NBOBzmscceo76+nitXrjA+Ps7Fixd1oscmIZPJcPHiRQA1JhXM2eN7e3vJZDJMTU3ddJofjcmmV2JWn7IQApvNRktLC11dXfj9fpUNlM1mWVhYUBa2zC6UQVZpLUk3YkdHB21tbcqXnc1mSaVSTE9PMzMzs2wfoBXYRkc+v1KpxNLSEpFIRKXSNzU14XK5cDqduFwuPB4PxWKRQqFAT08Pvb291NTUqN6ZTA7J5/NMT08TDAbJZDJ4PB6VNBQMBmloaKC5uZl4PI7dbl9We1HHSjYmcm6x4eFhAoGAqtTi8/mUQQ1mnEy7EW+NTa3EKqsnyNHz9957Lw899BCdnZ34fD7lPhweHmZkZITFxcUbEjIaGhro6urinnvu4Z577mH37t309vYSCAQAmJiYYHp6mueee47BwUGmpqZIpVJ64OImQghBqVRienqadDrNt7/9bTo6OshkMtTV1dHR0YHP56O+vp7GxkZVGLi9vZ2LFy8yOjrK0NAQCwsLKhPt7NmzpFIpHnnkEWw2G/X19bjdbrW/u+66C8MwuHDhAplMhmw2e4NcazYODoeDaDTKl770JcbHx9VEu1u2bOHo0aNcuHCBYrFIOBxmcXGRVCqln/FN2NRKDG7siQkhCAQCKgYBkEgklPtPxjGsmWM+n4/Ozk727NnDtm3b6O3tpaGhgZqaGlUXbXx8nJGRESYnJ5mdnV0WiNdCuLnI5/Ok02lGRkZIpVLU1tYSDoeVMpMTINbU1FBfX09vby82m41wOEwgEGBmZoZisUg8HieZTBKJRJiZmVFlrWQs1e12Ew6HCQaDOBwONfM46J7YRkQaQcVikUwmQzweZ3x8nKamJtra2pSszMzMMDs7q3tit8imV2JWpDsxFAopF1CxWGRsbIyRkREWFhZYWlqiVCopBRYOh+nv7+f+++/nh37oh+js7KSjo0MNcl5cXGRpaYnDhw9z+vRpjh49SjQaVRUbdFbZ5kEaN/l8nsXFRV5++WWcTidHjhyhtbWVQ4cO0d3dzd13361c1m1tbXR3d3Po0CEKhQIXLlxgcnKSr33tawwODjI7O8vIyAjHjh1jcXGRzs5OPB4PpVIJn8/Hjh07GB8fp6amRilPHSfZWFifl7U9WFhY4Hvf+x67du2iv7+fPXv28JM/+ZM8+eSTnD9//q041Q1JVSkxSWWyxuzsLFNTU8RiMVKpFG63G4/HQ0tLCx0dHRw4cIDdu3fT2tpKbW2tsohLpRLj4+NMTk4yNDTExMSEioNVlibSbC7kMy0WiyQSCebm5jh//jyLi4sUi0X6+voolUqEw2Hq6upUdfvGxkYcDge7d++mtraW0dFR7HY79fX1+P1+JVs2mw23201DQ8OqPTHNxsCaGGT9ze/3s3XrVtrb21UFIVkoWBrYujd2c6pSiRUKBbLZrEqZvnLlChcuXGB8fJxMJqMmsHvggQfYuXMn7373u6mrqyMcDi9TUKVSiZMnT3L8+HGOHj3K6OjosiKuOptscyIbJTl+MJlMqkHywWCQkydPcuDAARKJhCoU7PV6cbvdqifvdruZn5/n8uXL5PN5tm7dSn19vXIlOhwOAoEAgUCA5uZmampqSCaTb/GVa14rKymycDjMu9/9brxeLwCtra00NDTw4osvqhquMgYq96G5kapUYtYGyOpe3Lp1KwC9vb20tbVx77330t7eTigUUqPppUBFIhFisRjXrl1jcHCQxcVF8vm87n1VEdI1ZH3m2WyW+fl5Lly4QKlUUnLR09NDc3MzLpcLIQR1dXUqo7FYLNLY2KiyE1dq8DQbH6uc5PN5RkdH+du//VsGBgZ48MEHAdT4UlkJRnNzqlKJyXiVjJE1NjaSz+c5ePAgHo+HBx54gLa2Nu666y7lxrE2LkIIpqenuXbtGqdPn+bUqVNEIhFyudwNVlNlJplWbpsD6YqWyOecyWRIp9PMzs5y6tQp5ubmSKfT2Gw2VcXe6XTS2NhIY2OjmhVa7lMO6bBO3yKXaTYHsrL9hQsX+J3f+R3e9773cejQIVVIweFw4HQ6bzBodDLPylSlErMqFofDwbZt22hpaaG7uxun00lXVxe1tbXLFJhEFn89duwYx44dY3BwkFgsRrFYXCZ0TqdTFfaUDZKcP0r7udcvr3WmbvncK6domZmZ4ezZszgcDmZmZujr6yMcDlNfX09NTY2Kc1mPpS3wzYt1zrlCoaASw7LZrJrd4NChQ/zLf/kveeqpp3j55ZeBld2RGpOqVGJwvaFwOp1s3bpVFeOUlpDNZlsWSAfTgorH40xNTXH8+HGee+45xsfHWVxcVMrOWtRTFnR1uVwAzMzMMD09vaxHpq2r9YHV2oXX1vOxbiO/z87Oqh7a8PAw6XSanp4eVfLM2jjphmrzIw1oORh+cXGRZDKpYvQOh0ONRR0ZGeGVV15ZdRoojcmmVmLWLES4PpXGlStXaGho4ODBg2q8mHWdTCaj0pk9Ho+qOl0oFDh27BjPPPMMx48fZ2pqinw+r7a3jgV68MEHaW5uZt++fTgcDlKpFKdPn+bMmTOAWUxW89bg9Xrp6OhQ0+nE43Hm5ubI5XLLAumSygHrt9OYxONxisUiqVSKkZERxsfHaWho4IknnqC3t5fdu3fj9/uXNVQrKTKt3DYH1ilWampqaG1tJZ/P853vfIetW7eyf/9+Jicn1aSonZ2dzM/Pq6QerchuZFMrMbjROpZz+jQ0NLBjxw6CweAyl45UYul0mlgsRjAYJBQKqYkPr1y5wrPPPsvs7CyLi4uq3l2xWFRJIi0tLTz00EN0dXWxZ88eAGKxGLlcDpfLRaFQeEvuRTVjVULSzVtTU4PP52N6elrVuczn8zdsa+2Rr6ZkVmtc0uk0mUyGxcVFHA4HQ0ND1NTUqDhsX18fPp9vmYKsdGnqhmvzYH2WLpeLcDhMoVDg+PHjeL1eDh48SDweZ2RkBIDm5mYSiYTOTF2DTa/ErMie2Kuvvsr4+DilUon29nYCgQCGYRCJRMhkMkQiETwej6qR2NnZycLCAoODgwwODjI9Pa0qexQKBVXTzuFwqExHWVsxEAjgcDjw+Xy0trYSDoeJRCKqUrVuoN585BistrY29u7dq6onyFT5bDZLNpvlyJEjjI6OKut5rYKsa/WUpJyUSiXy+TyZTIbnn3+eK1eu4PF46OnpYdeuXdTU1ChZkq5rWaoqmUwqBasV28ZFyonNZiOXyzEyMoLNZmPbtm1kMhkWFhbw+Xxs27aNkydPLttWP/OVqSolJoPtY2NjxGIx1VVvaGjAMAwmJiZIpVJEo1HC4TB+v5+GhgYAlpaWGB8fVxXqK5EVPmRJq7q6Ompra/F4PNjtdlwul5qqRZa70rx5VDYAQgh8Ph89PT2qpyWVWDKZVO6/6elppYDWipetpsQqxxXKntbw8DDxeJyhoSFcLhfbt29ftv9SqUQqlWJ0dJT5+XlyudyyHrxu0DYu0hiSs140NjZSLBZVCEPOM+bz+VSyh2Z1qkqJwfXJ6dLpNM899xx+vx+/3w/A/Pw8DoeD1tZW6urq6O/vp76+nqWlJS5fvszTTz/NlStXAFRVcjmXWEdHB8FgkB07dhAOh2lubiYQCCCEYGlpiXPnznH27Fni8fgNs0Vr3jyEECQSCU6cOEGpVOLgwYN0dXWxc+dOFfdMpVKqMdm9ezfXrl0jEokwPDxMMplcpoxg+XCKSlZaJoSgubmZtrY22traaGpqUuOCpGs6k8kwNjbGN7/5TS5evKiKUlv3qRXZxqOymg/A4uIi165dY9u2bYTDYSUrjz76KK2trfzpn/4pc3Nzb9Upr3uqTomBGfcoFApMTU3hdDrxer0YhkEsFsPr9RIOh7Hb7dTV1eHxeMjlcsRiMTXbsywK7PP5aG9vp62tTVVc6Onpoba2Fp/Ph8PhIJ/Pk0gkGBsbU27IleIumjePfD7PwsICCwsLqkxUIBBQ43SkO3Hbtm3Y7XZKpRJer5doNKq2l4k8sndfOW6sEjmI1TAMJVuyuK+13BRcL2W1sLDA8PAws7Ozy6b1Aa3ANjrW5yfdiIlEglKppBKO2tvbAairq8PpdKqZNfSzX05VKjHrwNRMJqOCpsViEb/fT3t7O52dnbS3t6vZWOW8Py0tLdTV1bF9+3Y6Ozs5cOAA7e3ttLa24vP5cLvdasR9Op3m1KlTDA8P89WvfpXx8XHm5+d1UeC3GGuCz1NPPcWlS5eYnJyks7OTrq4u5Qp+4IEH2L9/PwcPHiQajXL06FEikYiSl0AgQD6fVy7m0dHRVRVNZ2enmjzT4/Fw8OBB2tra2LNnD7W1tapWXjweZ2Zmhueff54LFy5w9uxZlpaWdMO1ybD2zGOxGKdOnVLtx6FDh7jnnnvw+/00Njayd+9e0uk0Z8+eVYYUaENGUpVKTCKViRyoLN05fr8fn8+nJrs0DAOn06mK/zocDrZs2UJPTw/9/f20tLSoeaBk0c5kMsni4iLDw8MMDQ0xPDzMwsLCstJUmrcOwzBIp9OMjY3hcrnUMAq3260mQJW1CwHq6+tJJBJEo1E1T1wgECCXyzE5OUk0Gr1hOIeV3t5elUTk9Xrp7++nubl5WUmzYrHI0tIS8/PzXLlyRc1tJw0ozeZDxsaWlpaYmZlRbkVAxdLr6+tpbm7m6tWruo7iClSlEpNdcunCkW4kOSZM9rZk+R+/38/OnTtVCn0oFKKtrY2GhgbV6MnU/NnZWeLxOCdPnmRiYoJvfetbzM3NMTQ0pFPr1wEylmW320mlUly7do3p6WnOnDnDwMAAu3bt4uDBg+zatYuGhgZ8Ph+hUIiGhgba29vVfFBgyk0+nycSiai0aGvMwxoP6+vro7W1FbfbrdxFcpJWQM32fOTIEc6fP8+XvvQllWSiG6zNx0pDNRKJhCqeIBFC0N3dTT6f5/Tp08zMzOiKPxVUpRKzYq3U4Ha78Xq91NbW4vV6VW1Fp9NJKBSis7OTUCikpo4PBALLel7Sso9Go8pFNT4+TjweJ5vN3mCda94aKge/l0olcrkcfr8fj8ejahx2dnZSX19PbW2tmp6npqZGGT02m41CoaBipNaKLVKJSflqa2sjHA7jdDrVPHMyC1EqsPn5eSYnJ5mamiISiege2CanUpHJ6Xry+TyxWIx8Pq8MZB0PW52qV2IyPmK322loaKCtrY2BgQE6OjpUpXshBF1dXXR2di4rR2Wdnffq1avMzs7y7LPPMjY2xvHjx9WYI11iav0hXcjyWRYKBQYHB7l27RqvvPIKfr+fBx54gC1bthAMBvH5fOzevZu6ujra2tpwu924XC7sdjvBYJDa2lqam5tXPZ4sOi0L/MqkkFgsRjwe55VXXmFycpIXXniBqakp3WuvEqyKSU7/lM1m+da3vsXu3bvp6uri6tWrvPjiiyqeLtsgHVs3qWolZi3aKt2BqVRqWS0zKTDSBSWFLpfLkcvlmJiYIBKJcPbsWeXTltU8rKn0eqzH+qPy+cuPjJcNDw+rzEU523JDQ4OqSC/n/4Ll0/ushty/XL9YLDI9Pc3c3Bznzp1jenpa9cK0sVMdWA1cmcXc0tKixpc6HA7cbveyYtHW7TRaiQHXg6vDw8MUCgXOnDlDqVTirrvuWhZILRaLyn04NzdHNBrl29/+NufPn+ell15iamqKTCajXEXWitXalbg+sT4Xmdwjh2C88sorHDlyRJUWa2hooL6+nn/6T/8p/f393HfffarKhnVfK83AXNnolEol0uk0L7zwApcuXeLLX/4y0Wh0mULVVAfSQD5z5gznz5/n4x//OAcPHqSuro6amhr6+vqIxWJcunSJubk5LRsVVLUSA25QNul0mvHxcUKhEHNzc/h8Pvx+P4lEglgsxtLSEqlUSlXvOHfuHMPDwyr1WjZoOoto42HtmcH1OoqVLpyLFy+SSCSUlfxayOfzJJNJLly4wMjICEtLS2QyGd1jr1IMw5woM5fLkclkllVoaW1tZfv27Wp6qEo3YrW3MVWvxCSy8UokErzyyiukUin27NlDe3s7u3fvZmZmhqNHj3Lt2jVGRka4ePEi4+PjRCIRUqmU6n1Jql2wNiorVcOQve9oNMrS0hJPPvkkgUCAw4cPq8otwDJX4Wr7le7HWCymsiMTiYQqQKyt7OqjcqB8Mplkfn6e9vZ2DMPg3nvvZc+ePXzta1/j2rVrytsjqfbYmFZiLG+wCoUCsViMkZERXnjhBRobG7l06RITExNcuXKF6elp9ZFxr0oFptl8WDMal5aW1CzeDofjprEwK9Y6jdlsVk2IqOVHI0mlUszPz7O0tEQ6nWZhYYFIJIIQgtra2hvqaFY7WomxXInl83lmZmaIRqOMjIzgcrnwer2k02nV8GSzWWWda6oL2SMDc8JLeG1zfa1UQ09TvVRW8BgZGaG/v59kMsmZM2cYGhoCIBwOq4H1GhOtxFZApt0nk0k1t1ihUFDKS/e8NICqbP9aZEHLj2Y1xsfHeeWVV+jq6qK1tRWPx0NLSwtut1vFZmUcH7QsaSW2CrII61ro5A2NfvaaO4FVjoaHhxkeHmb37t0MDAyoYuOyPJlUYrpyh4lWYmtQmSlWmbihBUij0dxpbDabKou2sLBAOBymtraWYDCI1+sF0ArMglZiqyDHDK00VbwWHo1G80Zht9txOp1kMhmi0Shut5vGxkY16B50FqsVrcRWYbUBylqBaTSaNxJZZFr2tqzxeN3+3IhWYmugBUaj0bzZWKu2FAoFcrkc2WyWQqGgFdkKaCWm0Wg06wjZA3vllVcYGRnh8uXLNDU18eKLLzI0NKQGxmtMtBLTaDSadYZhGExOTjI3N4fdbqepqYmRkRHm5ube6lNbd4iN2jUVQmzME38dGIZx+6NqNYCWF83t81bIjHXcoZwqyO/343K5WFxcJJfLqQoxb0TVjo0oM7onptFoNOsQWVPROtMzvLYKMZsZrcQ0Go1mnXArnjFdN3E5et4HjUajWWes1dvSY1WXs2FjYhqNRqPR6J6YRqPRaDYsWolpNBqNZsOilZhGo9FoNixaiWk0Go1mw6KVmEaj0Wg2LFqJaTQajWbDopWYRqPRaDYsWolpNBqNZsOilZhGo9FoNixaiWk0Go1mw6KVmEaj0Wg2LFqJaTQajWbDopWYRqPRaDYsWolpNBqNZsOilZhGo9FoNixaiWk0Go1mw/KmKTEhxGEhxCff7G1vcf9/K4T44Bu1/42OEGKvEOKlN+E461lGfiCEOPBG7X+zI4T4jBDi597kY2p5WqcIIdxCiItCiMbXu6/bVmJCiGEhxOOv98BvFEKIjwshjgkh4kKIcSHE7wshHGusvxfYBzxZ/l8IIX5NCDFa3seXhBC1lvXdQojPl5dNCyF+aY19f0IIURRCJMqfISHEnwshBu7kNb/RGIZxGogJId5/K+uvdxmxIoR4Vghh3ERG3g8sGYZxovz/biHEM0KIeSHEDVOjCyF6hBDfFEJEyzLyR9b9CyH2l2U0Vf67f41jHxZCZIQQS2WZOyaE+HdCCPfru/I3nf8KfEoI4brdDTeCPAkh/n/lZx0vtw+rPp9Kebqd7cuyZVjalBkhxDeEEO98Ay7rDcMwjCzweeDfvd59bUZ3ohf4V0AYOAS8A/jlNdb/WeCvDcOQjdFPAR8DHgTaAA/wh5b1Pw1sBbqBtwO/IoR49xr7f9kwDD8QBB4H0sAxIcTu27qqW2CthvgO8NeY92rTIIT4KOC8hVX/OfBXlv/zwJeBn1ll/c8Cs0ArsB94BPj58jFdmAbTF4E64C+AJ2/SuP+CYRiB8v7+NfAR4JtCCHEL535bvFEyZBjGFHAR+OE3Yv9vJUKId2E2xu/AbBf6gP+4xibL5Ok1bA8QKrcr+4DvAF8TQnziNV7CqrzBbcrfAB9/3QaZYRi39QGGgcdX+L0O+AYwB0TL3zssyw8DvwscAeKYL3K9Zfl9wEtADDgFPFqx7Sdv91zL2/4S8PU1ll8DHrL8/xXg31j+fwDIAN7y/5PAE5blvwV8aZV9fwL4/gq/fwP4yi1eey/wIrAEfBf4Y+CL5WU9gIHZmI4CL5Z//2ngQvk5PAN0W/a3HVPoI8Al4J9Ylr0XOF8+1gTwy5Zl7ZgK2L0ZZATTqLhc3qcBOFZZz1W+7o4Vlm0BjBV+vwC81/L/fwE+V/7+RPneCsvyUeDdqxz/husCuoAU8L7y/zbMRnAQWMBUsNb79lPASHnZr1ufD6ZR9hVMpRoHPlm+N38GTJXP9bcBu2V/K8oXIID/jqnA48AZYLdlu18D/vw1vMPrWp4wG+Pfsfz/DmD6VuXpNrfvWUleMQ31GcBW/r8N+Gr53gwBv2hZ14NpPEXLz/FXgPGK+/1vgdNAFnDc5F6tKi+Y78gLwCIwD/yfivO+AjxyuzJh/dzJnpgN+HNMS6IL80H9UcU6P4X5ArQCBeAPAIQQ7cA/lC++HvOBfHUlf6kQoksIERNCdN3ieT0MnFtpgRDCh6kkLlUuqvjuBrYKIerK537KsvwUsOsWz0Xyd8Dbyudws2v/G8yXsAGzwfnYCvt7BNgBvEsI8QHgU8CPAo3A94C/tVzvd8r7bMK06D8rhNhZ3s+fAT9rmFb/buA5eQDDMCYweyDbbvNarawnGfkd4E+A6Zuc81agZBjG+E3Ws/I/gI8IIbzl834P8K3ysl3AaaP8Bpc5zW3IkGEYo8BRyjIE/Avgg5hy0IbZOP0xQPnZfhb4KOY9DWIaJFY+gKnIQpg97i9g3vstwAFMxfvJ8v5Wla/yeg8DA+Xj/BNMxSm5gNlzuFOsF3naxY1tQrMQomGFdVeSp9vZfjX+DvOd3iaEsAFfL++nHVMp/qtyjw/gNzCVYR/wTuAnV9jfTwA/hCkTzax9r77AKvKCaeR/G9Pg6GC5VwvuhEzcKatohfX2A9EKy+b3LP/vBHKAHVPr/1XF9s8AH79dq6hiHz8NjAPhVZa3Y1o1NZbfPolpofdgvohPlde5H+hcYf13AsOr7P8TrNwTezeQL39f9doxX8wC5V5gedkXubEn1mdZ/jTwM5b/bZhWezfw48D3Ko71OeA3yt9HMV2GtatczwTw8EaXEeAgcBLTwpT3cLWe2IOsbhWv1hPbARwrPzsD8yUX5WW/TkXPHVNxfHqVY6x4XcCXgD8tf78AvMOyrBXT4HAA/wH4W8syb/meWntiL1qWN2Na3x7Lbz8BPH8L8vUY13u3thXO+Z3AtVt5RhtMngax9KQxXdQG0HMr8nSb268or0BN+fcHMcMooxXLf5VyLxjT+/Quy7JPcmNP7Kct/6/VRt1MXv4S+N+s4MmwyP5/uF2ZsH7uWE+sbHV+TggxIoSIY7rAQkIIu2W1Mcv3EcyHFcZ8AT5ctnZiQogY8BDmy/haz+eDmK6E9xiGMb/KarHy34Dlt89jWpaHMXtwz5d/HwcS5e+1lvVrMd1vt0M7pjsP1r72NiBiGEbKsu0YN2L9rRv4n5Z9RTB7k+3lZYcqjvVRoKW87YcwXYojQogXhBD3VxwnwPV7dtusBxkpW6mfBf6lYRiFW9gkynL5uJX9fwvTMvaVz70O+M/lVRIslx+4MzL0Nct9uQAUMRuYNiz3tCxLC8t3dYP8OIEpy/4+h2nly+UrypdhGM9h9oT+GJgVQvxvYUmK4nXKTyXrQZ7KVD5T+X2lZ7qSPN3O9qshe9cRzGtrq7i2T2HKA1TIBLfWpqx2r24mL7+CKR9HhBDnhBA/XXGc1y0Td9Kd+K8xXU2HDMOoxXQrwHLXXKflexemtTiPecP+yjCMkOXjMwzj917LiZQTLf4UeL9hGGdWW88wjCSmFTRg+a1kGMZvGIbRYxhGB6YimwAmDMOIYvp9rd3ffazirlyDH8F0w8Da1z4F1AshvJZtOyt3hmmBScYwXYLW/XkMw3ipvOyFimV+wzB+rnztrxqG8QFMAfx7zNgKoNwvLm50vd4O60FGajF7Yv9HCDENvFr+fVwI8bYV1r+KmbRa6YJbjfryef+RYRhZwzAWMF1e7y0vPwfsrUjK2MttyJAQohO4m+Uy9J6Ke1NjmC7gKUw3jtzWg+matlIpP1lM74XcV61hGLssy1eTLwzD+APDMO7G7PUMAP/Gsu8dLHebvV7WgzyB+ewq24SZ8rOvZCV5up3tV+NHMGORlzCvbaji2gKGYUgZXCYT3Fqbstq9WlNeDMOYNgzjnxmG0Ybp5fmsEGKLZd+vWyZeqxJzCiFqLB8HpkZNY6Zi12P6XSv5SSHEznKj/JuYyQ1FTBfZ+4UQ7xJC2Mv7fFQI0bHCPtZECPEYZhf1Q4ZhHLmFTb6JGUuQ29cLIfqFyU7gvwG/aRhGqbzKXwL/XghRJ4TYDvwzTHfRzc7LLoToFUL8IfAo17OPVr12wzBGMGMfnxZCuMo9o5uluf8v4FeFELvKxw0KIT5cXvYNYEAI8TEhhLP8uUcIsaO8/48KIYKGYeQxA+Ely34fAZ4zzNTYW2G9ysgipiW6v/yRL/bdwCuVKxuGkcNMqLHKiBBC1GAqdcrn4i6vP48ZSP85IYRDCBHCdLucLm9+GLOX9IvCHK7xC+XfVfxxNco9j0cwExSOYMoumM/8PwkhusvrNQozdgVmrOv9QogHhJkB+WmWN/KV1zuFGcP4jBCiVghhK78P8vpXla+yLB0SQjiBJGZCVKUMPX2z61yF9SpPYLYJP1M+Tgj496zSJqwkT7ezfSVCiOayDP0G8KvlduoIsCSE+LdCCE/5+nYLIe4pb/ZlzGdYV1amv7DK7iVrtVFryosQ4sOWexrFVI6l8rJ2TKPvH2/lWlfldv2PmP5So+Lz25gNw2HMrvFlTK2rfLfcmCn0dSyxKkw/7guY3eE5zEBil2XbT5a/d5WP0bXK+T2PGYtIWD5Pr3E9uzEtIRmzGMC0ZlKY7odfqljfjelyjGNmA/3SGvv+BGaDlcB8qUcws4J2VKy31rX3Y1rcS8CzmP7lPzPW9o9/DDMzLI5pKX3esmxbef9zmG6l5zAbcxemGyxa3u5Vlmdt/gPww5tBRirOdcV7WLHOD1llyLKN9TNsWb6/fD5RTKv/y0CzZfkBzJhZGjgOHFjj2IcxlcFS+XMCM8vPGpe1YWbhXiqvM8jybLdPYMY7ZXbiBPC28rJPU46xWtYPYia9jGMq/RPAR24mX5gJBKfL934e05j0l5e1lvfn2mxtTnmdX8JsD+KYPe9Vs3gr5el2tue67Mk2ZRbTmHl3xXptmGGRaUw5/Eeux0F9mCn+MUzX878HBivu9+MV+1vrXq0qL8Dvl+UtgSmX/1/LPv8N8N9uVx4qP7LhrmqEEH8DfNkwjL9/q8/lZggh/g9w0TCMlazON+qYezFTxCtjZFWDEOIHmOO1TrzV5/J6EEL4MRuvrYZhDL2Jx/0MZkP52TfrmOuZ9SRPwqyk8hHDMB656cp37phuTDfiw4ZhzL6ufWkltr4puwAimC6qJzBjVfevB+HXbAyEWSHiWUw34mcwreq7DP3yVyVCiFbM9PqXMVP+/wEzhvs/3srzeq28kaOxNXeGFsxMtwbM7vrPaQWmuU0+gOk+Epgx1o9oBVbVuDAzCHsxe+VfwszY3ZDonphGo9FoNiybsXaiRqPRaKqEDetOFCtUD9/sGIZxxwu+VgtaXjS3i5aZjYHuiWk0Go1mw6KVmEaj0Wg2LBvWnajRaDSbBVGuQiaEYLVkO52EtzJaiWk0Gs1biBACm82mPpaKFsD1qkqlUkkrshXQSkyj0WjeYqSSkt8rl2nltTpaiVmw2ZaHCLXwaDSaNxprO1MsFm+6vnQ96rbJRCsxCysJhRArZ5xqAdKshjW+sRpSfrQcVTd2ux2Xy0UgEKCmpgaPx0M2myWfz6t1stkshUKBRCJBoXArU+BVF1WtxKyNzGq9LmuDtJoFpBsijUTKiYxzrIRVga0kO9oDUB3YbDYcDgfBYJDu7m7q6+tpa2sjFosRj8cBUxai0SjJZJLx8XESiYSWjQqqWolJDMNACIHL5VKKqlQqkcvl1LKVgq0aTSVWubgV15CWo+rCmrhhs9moqamhpaWFQ4cO0dHRwZYtW0gkEiQSCYrFIqVSienpaeLxOOfOnWN+fp6JiQnS6TTZbFbLD1WuxKwCIITA4/Eo6zmfz5PL5VZcdzW061EDd6YnpeMemxPZSy8Wi9jtdjweDx0dHTz44IMMDAywf/9+0uk06XSafD5PPp9ndHSUhYUFamtrGRsbI5PJsLCwsMzIhuqVlapWYna7XXXnA4EAu3btwuVyUSwWicfjDA0NUSgUKBaLZLNZUqmUEphcLkehUNC9M41CuhAdDgdOpxOXy4XNZltx7E8+n6dYLKqGSKLTqTc3lc+6UCiQTCaZmpoiGAyysLCgFJ3T6cTpdNLV1UVLSwu1tbXMzc1hGAYjIyMcP36cdDq9Ylp+NVH1SszpdFJfX09TUxMHDx7E4/GQz+eZm5tTvbFcLqd81NbYWKlU0o2NBliuwNxuNx6Ph0AgsGzsj5VUKkU+nyeZTFIsFpVMSRfSrbgiNRuPSiVWKpVIpVLMzc3R0NBAJBLB4/Hg9Xqx2+3Y7XZqa2txOBy0t7eztLTE4OAgDoeDc+fOkclk1kwgqgaqVonZ7XYGBgZobGzkscceo62tjf379+NyuTAMg0wmwxNPPEE6nWZpaYloNMrMzIza/uLFi4yOjpJMJsnlciQSCbLZrPJlazYv1mQfh8NBIBDA6/XS3NxMMBiktbWVnp4edu/ejcPhwOFwqMZLCEGpVGJkZIRoNMrFixdJpVK43W5KpRKJRIJoNMrp06cpFApq7JBmc1AZM00kEgwODpJMJnn11Vf5/ve/T1NTE42NjTQ0NFBbW8uWLVsIh8P4/X5qamro7++nWCxSW1tLJpO5oTdfbVStErPZbDQ3N9PT08OhQ4dob29n69atOJ1OJRClUol0Ok0sFmNhYYHJyUnVgDkcDmw2G4uLi6TTaebn50mlUmQyGUql0g1+6moWss2G7I3b7XbcbjehUIhQKERfXx8NDQ1s2bKFPXv28NBDD+F0OnE4lr9mpVKJ8+fPMzMzg9vtJh6P4/V6KZVKRCIRpqamuHz5smqgNJsTmTw2Pz/P/Pw8o6OjDA4O0t7eTnt7Ox0dHTQ2NuL3+3G5XHi9XlwuF+FwmMXFRTweD06nk2w2q9qcamxnqlqJdXV1MTAwQE9PD+FwmEKhoBSS9Em73W5lEbW1tSn3oUyFla6iEydOMDk5yQ9+8AOi0Sj5fJ5SqUShUCCXy7GwsKDia1Yq0/w16xObzYbP58PpdOLz+fD5fAwMDNDQ0MBdd91FKBSis7MTj8dDKBQiGAzidrtXdPVI2WtqaqK1tZV8Pq88AIlEgsnJSRoaGhgaGuL73/++ir9qNjepVIqpqSkWFxcZGhrC4/FQU1PDwsICO3fu5LHHHqO5uRm3243b7aampga3200ikQCqt/2oaiUWCoVobGwkGAzi9XpJJpMkk0kmJycB8Pl81NbW4vP5cLlc1NTUqEaptraWYrGIx+NRv9XX1zM5OYnf7yebzVIsFikUCqRSKXK5HPl8XllNslGqVsHbSMielxyMGg6HCQaDbNu2jdbWVg4ePEgwGKSlpUUZPjIWthp+v1/JF6Bcjul0mlAoxPj4OEIITpw4gRBCK7EqoFAoKENaJgTZbDa2bdtGbW2tSqmXsXyHw4Hdbn+rT/stpyqVmNvtxu/3s337dnbt2qUU2FNPPcXw8DDPP/88Qgjq6uqoq6ujp6eH+vp6Wltb6e3tpbe3l5qaGrxer2qodu7cSU9PD1u2bCGTyah4hsw+Gh4eZmlpiZmZGcbGxjh//jzxeJxoNKrOy5o0Auh4yDpBpjF3dXXR2trKu971Ltra2ujv78fn89HQ0IDT6VRGjpQJay28yh63bKRcLhdwvUfu8/loa2vjXe96F319fQBcunSJI0eOLBv2Ua2uo2rAMAyV7FMqlZiamsLv97OwsEBDQ4OKxWpMqvJOyCyyQCBAMBjE6XSSSqUYGxtjcHCQM2fOIISgvr6eUChEPB6nqamJpaUlHA4Hfr9fBfOlJeR2u1VvTWaYySyzVCpFbW0t8Xic+vp63G430WhUxd+km1FuV83psusJaVRIt05LSwudnZ1s376dlpYWOjo6cLlcKpXe2vOqjIWulEZvre4hkWOHWltbSafT9PT0EIvFcDgcqjcmt9EysrmRzzefzy8zjGHtkmbVRlUqMWtQXsYtCoUCExMTTExMkEql1NiwmZkZrly5olw/nZ2dtLe3U1dXRzAYpLm5Gb/fT19fn4qbeTwe3G63Ol6pVKKjo4NsNsvi4iJzc3M88MADxGIxIpEIY2NjTExMcOXKFaanp0kmk8tqp2neGrxeL16vl8cff5ytW7dyzz330NTUxNatW/F6vTidTmD5wGSrgqpcBmZGmjRc5HLrx263Y7PZ8Hq9hMNhduzYwdzcnPpdohXZ5qNyShaptJqamujp6aG2tlYZvqVSSbkTq12hVa0Sk3/lR2YKyYSMYrG4LDkjk8mQSqWUe7Curg6/308sFiMYDALQ0NCA3W5XCs/hcKhSVjU1NcqP7XQ6sdvtLC0tqSwjj8dDJpPBMAxmZ2fVOCLtUnxzkYpBKhSXy0VXVxdbt26lu7tbPXepwGDluGalC9G6/9W2sWKz2XC5XCom63Q6dabiJqayRy5/s9vt+P1+Ghoalhnc+XxeD8EoU5VKzFoVYbXBytIqkstkZelkMsnY2JiyjIPBIB6Ph6amJvx+P729vdTV1bF//37q6uro7u7G6XQqt1Q4HMbn89Hc3KxciNFolMXFRU6dOqUy0kZGRhgbGyOVSr3Zt0cDywLoBw8e5OGHH1aGCaCqtUg5Wa1yvZQvKS/S/WwdN2b9X36XlnY4HKaxsZHGxkaEEKoArG68NgeVhcWlAS3bC4/Hw8DAAPv27aOurg6n00kkEmFmZoZYLKYyE6u5V16VSuy1IpWeFZlRVigU8Hq9ahCix+Ohrq6OYrGolJjf7yefz1NTU4Pf71fVHQzDwOVyKYU3MTFBqVRSPbJqdxe8mVgbAo/HQzAYxO/3K/ehzWZTLsFb2Zd8drJXncvlVOwTUPG0mpqaZcoQrhtSsjzaSjE3zcbH2vN3OBzquYfDYUKhEOFwmLq6OoQQZLNZFYaonLKlWtFKbBWs9etWsrJlby6ZTJJKpVhcXEQIwdWrV3E4HJw4cYJgMMj27dtxOBwUi0VCoRBbtmyhs7OTvXv3UldXp3pmPp+P+vp6isUidXV1XL58mdHRUaLRqFZibwJWa1hax11dXWzZsoW6ujqVAm9VQCvFIyqTcmTjND8/rwYyLy0tqXTplpYWAoEAfX19KpZqPRfZ46tUYprNgTXxx+/3Ew6HqampweVyceDAAbZs2cI999xDd3e3Gm969uxZLl26xOLiItlsturjo1WrxFbKDKvEKmA3W0+OmJcuwsXFRfL5PCMjI9jtdorFIn6/n1QqRTwex+Vy0draSrFYxOVy4XQ6VSPm8Xjw+Xw3BPI1bxzWRkAW721tbaWvr2/Zs1hNZlZzScuxPxMTE4yMjDAyMqKsaCEEO3fupKGhgc7OTmpqatS2slrM9PQ0CwsLqqq5ZvNgTTDz+/00NjbS2dmpEop27typqsC43W7VA1tYWCAajS5zaVczVanErD2rShdOJStNYFiZGAKoRk6uF4vFiMViTE9Pq9/l+t3d3Vy6dImBgQH27t1LQ0ODqvggrX7putK8+QQCAerq6rj//vt56KGHaGlpWZYduJbVK3+XcpVIJIhEIrz44ou89NJLnD17lqmpKbLZLE6nkw9+8IMMDAywe/duvF6v2kehUGBubo4f/OAHnDt3jqmpKdLptDq+9ViajUPlcAqv10tLSwu7du1iy5Yt7N27l2AwSG1tLb29vbS2tmK32zEMg6mpKYaGhrhy5QrDw8MUCoWqV2BQpUpM8lqnvFhpDh9r7TKrwrPG0ORvcpqXTCZDNBqlqamJcDjMzMwMjY2NXLx4kfHxcZXUoRurNx7rPZZunYaGhmVZYbeCNFRkdZaRkREGBwe5dOkSw8PDRKNRMpmMios2NzfT1NSksletCjKdTjM6OsrMzIwaJyTPVTdeGxNpCMlYeV1dHX19fTz66KO0tLTQ1dWlEjoCgYCKkZVKJTUO1e12L8uOrWZXIlSpEpNKRrp6JDcrFVS5j7X+X8tdGYlEiEQiaru2tjZaWlrYvn07bW1tyuW0uLi44r41dx5rD6qhoYHe3l46OzvVuD+bzbasAkfldnJb+bzT6TRzc3McOXKE559/npMnTzI4OKgyz7q7u2lpaWHPnj1s2bJF1VqE67KzuLjIiRMnmJub01mqmwDpPpSx1ZqaGjo7O7n//vv5+Z//eZW5ulpPW3oIQqEQtbW1y7Knq9moqUolJicjjEajzM/P09jYqLrmd8qFd6vZa4ZhsLS0pIRwYmKCaDRKKpUim83e1v40t4/VipXKQ5aQklVYrO7jlXpB1mcjxxQODQ1x8uRJTpw4waVLl4hGo8sUpRzA2tnZSWtrq7K2ZcMkMyBvJwanWf9UGkCyQHgqlVKTYMrkITluVfbSQ6EQpVKJtrY2lpaWuHz5MqB7YlWpxKSrZ2ZmhqmpKbq6usjlcjeNj90ua5WPsvbU4vE48XicsbGxO3Zsza1TOcC5UondDrlcjlgsxtmzZ3nyySe5cOECly5dWva8bTYbPT097Ny5k23bttHW1qZS761KTI4V07HRzUFlCbJSqUQ+nyeVShGNRvF6vXg8HqXYlpaWSKfT1NfX4/P5aGpqIhQK0d/fTzab5R//8R9vmpxWDVSlEpONRDweJxaLLZu36bU0GGtZ5mudQzVbTxuRyuclp9WRJYISiQTDw8O89NJLnD59mosXL7KwsLBsm/r6eoLBoHJVOhwOSqWScjPl83ni8TgXLlzg4sWLzM3NsbS0pLavdqt7o2OdiimbzTI1NcWxY8f4whe+QDAYpKGhgfn5eRYWFlT5u3e84x0MDAzQ0dGhar5a3YnV3pZUtRJbWlpSSky6E2/FqlmpPMxqx9FsHlZSYrL3ls/niUQiXLhwgSeffJKxsTGuXr16QxxNzobQ0dGhlJhhGKpBSqVSxGIxTp48qZSYtdyUVmIbG2uil1RiS0tLjI6O0tTURHt7O0NDQwwPD6shFaFQiEAgQFNTE06nE7/fj9/vv6H6S7VSlUpMunR8Pp9KZ83n84TDYSKRyLJ0ebixd1Zp+VS7EG10rG4eWTMzk8moHrp06UkFIhsiq/s5mUxy6dIlLl26xOXLl1laWlIZq/Jjt9vZt28fe/fuZe/evXR0dKgYiOzNzc/Pc+3aNV544QWGh4eVm1Giy01tLgzDIJPJqLGACwsLxONxUqmUipHmcjlyuZx69nJcmRxPJucorFaqWonJOcG8Xi+FQkGVGKrMDqqsbyapDPJXrqOV2/qn8nlKl14mk1Hzd1VOPFiZEVYqlUgmk4yOjjI6Osr4+PgNsuNwOHA4HPT19bF//341m7h1mEehUCAajTI1NcXJkyfVgFbN5iafz5PP50kkEszNzS0zkqVxk8/nVXsjZ3WWlT1k4fBq7aVXpRKTRTaTySRLS0sYhoHT6SQcDqsyUIZh3DABoXT72O12pfDC4TCBQECVi5FB2cHBQWKxGOPj47ohWudYDZFSqcTCwgIjIyNMTEwwOztLY2MjHo9H9YhkGbGZmRmi0ShHjx5lbGyMl19+WSkwq+Ejp+JpbW1l586dKqVe9sJyuRwTExNMTU3x5JNPMjQ0xNzcnBrcrKkOVhquIec+lOPCZFvlcrluGC9WjQoMqlSJAUpJyRp20r3o9XpxuVw4HI5lSkz+lWmwoVCIhoYGNetzZ2cnPp9PKcdcLofb7WZqauqWC8Zq3jqs48BSqRQLCwvEYjHi8bgaw2V12ZRKJWKxGBMTExw5coSJiQnOnTunqorD8viVnCG8paWFcDiM2+1WyRwACwsLTExMcOzYMaampkgkEsvciJrqQLZF1uLP8mNdLn/TmatVpMSsiki6b5LJpGosXC6Xqlno8XhU911Wow8GgzQ2NtLS0qIaolAoRGNjoxqEKCfZzOVy9PT0MDw8zOLiIpFIRLkJNOufRCJBqVTi2rVrqppKfX39sh7Y3NwcTz31FCMjIzz77LOqVy/jW2AqulAoRF1dHW9729t48MEH6e/vX9bTl/G2eDzO3Nwcg4ODaiC8VbFqqgfZTni9Xvx+/7I55QBisRjz8/MsLi6SSqWWyVs1UjVKDG7MIpQVOyp9zX6/X43R8fv9NDU10dTURGdnJ11dXXR1dVFXV6eUl1R+LpdLxdfk4Nba2lo1nYpWYuuXygHL6XSaaDS6bIJSGS+LRCJMT09z+fJlhoeHmZiYUJlk0oKW+5Py09XVRX9/P8FgUKXVSxd1qVQilUqRSCSIx+Mkk0ltYVcp1ri62+0mEAgsmym+VCopObFOxVLNY8WqSolVWipyQKtMr+/q6qKmpoYPf/jDKhtINkKNjY20t7cTCoVUQ2Tt0lsHsgLK7Xg7paw0bw0rDUItFApcunSJbDZLKBRicnKSpaUlkskkR48eZXJykhdffJF4PE6hUFAuHhlvrampwefzcf/99/PYY49xzz33MDAwoGrhyWnmZVXy48ePc/Xq1WUDnrXRs7GojIO+VuRz7+rqYufOnfT399PU1EQikSCdTnP06FEuX75MPB5XRRqqmapQYlKwrNUPrBk+Er/fT6FQYGBgQMXDfD4fDQ0NanyPHFUP11OyrTNEy8y2eDxOIpFQ04hr1jeVWYqlUonFxUWmp6dVxfBYLKbK/czOzjI/P6+mVJHI7263m7q6Otrb29m6dStNTU14vd5lPTA5q/fMzAyTk5PMzs6uWJ9Rs/lZKfs5FArR2tqqJtldWFggEokwOzvLzMyM8g5UO1WhxGSGT3NzM4FAACEEXq+XgYEBent7Vapqc3OzmtPHuq3scVmVoEx7nZubU8oqn8+rhIDvfve7jI+Pc+XKFVKplBa2DYQ0SsbHx5mfn2dmZga3263mAUun02o8WeUElmBOsdHb28vb3vY23v72t3PvvfficrmWJWpIl9DXv/51rly5wtNPP60q3L+WmRU0bz2vtfe8miuwr6+PBx98kO7ubvx+P88//zwXL17k2LFjyoUt5zCsZjatErPGoKQSC4fDNDc3q7Fhra2tNDQ0KLefw2HeDul/rnQzWZNCZNd+fHycSCRCKpVSM68uLi4yPDzM7Ows6XRa98Q2KHKgsSwLtbi4qGKocD0N2ionbrcbn89Hc3Mzvb29NDY2qkQO63oyOD88PMzw8DCxWIxkMlnVsY2NjDVxDF6bQpPry/T5uro6NRUQoGKxsu2xjg2rZqNnUyqxysHJbrcbr9fL29/+du677z62bt1KfX09dXV1alblleofFgoFVaQTTAtbVnIYHBxkcnKS7373u1y+fJmZmRnS6bSKp8RisRumetFsLGTv2lpXUyZlwI0zf8ve/v79+3nooYd43/veRygUWlbSTBpCR44c4fz58zz99NPKqq6sFKPZOMjB7NKwkb31m7FSwldHRwf9/f3s2rWL3t5enE4nS0tLDA4Ocv78eZUFK0uWVbu8bEolVokMtmezWTKZDB6Ph1AoRE1NjRqrI0sEASq2Jcd7ybmchBDKLy2V2NDQEBMTE8rVJI9X7aVgNhPyOa5UM1M2IDKTVc7S29vbq2RM7sMwDOWOnJmZUVZ1JpPRCmyDIg0Tr9eLz+fD7/fjcrkYHx9Xw3fWeqbWZXLSy66uLnbv3k1LSwsej4dEIkEikVAxWakctayYbEolVlkOKpvNUigUOH36NOl0mp6eHpqampSASeUjFVqhUCAejzM6OkokEmFiYkJZPCdPnuTUqVNMTk6ysLBAPp9f1eLSafWbi8rnLN3PYI7p2bp1K4888gi/+Iu/iMfjwev1Kstczg83NDTE7Owsx48f58qVK6TT6WWyquVlYyFj5p2dnfT09LB9+3bC4TB/93d/x5UrV1S8fC1kVnM4HKarq4v3v//9fOhDH6Kurg6/38/Zs2e5evWqquQiEzq0rJhsSiUGy33Sspc1Pz+Py+Xi6NGjRCIRXC6XKrAJy5VYKpVicnKSeDzO9PS02teVK1eYnp4mFostm21XxzKqC6uBYrfb8Xq9tLS00NjYiN/vV3FWaanLGNjZs2cZHx9nbGyM+fn5Ze5m3ShtPKQXx+FwUFNTQ0dHBz09Pezbtw+fz8fQ0BCJRIJoNLqisWuz2QgEAjQ3N9Pd3c327dvp6+sjGAxis9lIpVJMTU0xNDSkJsuV7ZmWF5NNrcSs3wuFAleuXGFoaIjLly8rKxluHNMh15fuRGvAvVAorOgi0AJVPUi5kWO63G439fX17N+/ny1btuDz+YDrxpMQgmvXrnHq1Cm+8Y1vcOHC/7+9O+lJpYnCAPxKM6kJAZxAnC5EZWPiwg1x65/wf5r4A4wxLtA4z910INqtDYgyOnFXVTbE+znd72rL+yTGYJgWZZ3uOlXnHCCfz8sVAJEnI+cRO1kVRUF3dzdmZ2eRSqUwODiIbDaLpaUl6LqOdDqNSqXS8loxdiYmJrCwsICZmRnMz88jHA4jGAyiWCyiVCphY2MDa2tr0DRNdtlwuVwdvytR+LFB7CUi+Nzc3Ly4jGPPiYnCrGIrPdC6O5E6j328uN1u9PX1IRAIYGpqSuYxYrGYXAUQpc3q9Tqurq5weXmJUqkkl5jseVhyHvvd0O3tLUzThGVZKBQKCIVC8Hg8SKVSSCQSiMVisto88FwD0efzYXR0FHNzcxgdHUUwGJSlzQ4ODpDJZHB0dATDMFCv1+Vr6VlHBTFx1XR9ff3VX4UcSCwPilqbyWQSiUQCi4uLMp8h6isCkBU5TNOUjQ4ty5LJ+fbcLTlH+w5oy7Jwd3eH09NTDA8PI5lMYnp6GpOTk2g0GvJ4hlg+FkHM6/XKqkDiDkvXdei6jqWlJayurkJVVRQKhZZxxUD2rKOCGNFn2M8L+nw+jI+P49evXxgZGUEgEJC7XUX5qWazCcMwsLW1hcPDQ6iq2lLl3v6e5CztFyCibU46nUa1WoVpmohEIhgaGoLf70cwGASAlpUc0ShVHNkRneY1TYOmadje3sb5+bnMg4nPpVYdF8Q+euXLhpedrf3wvN/vRyKRwOTkJMbHx+WBVLtms4lcLoe1tTXs7OxAVdWWQ6riOeRM9mBUq9VQq9WwsrKCnZ0dXFxcyFxXNBpFf39/S+8vu2w2i93dXWSzWRwfH0NVVWQyGZimyVWjN+i4IGYfeG953p8eU+ewjxVRUioWiyEejyMajeLp6UlW92g0GiiXy8jn8zAMA+vr69jb28PFxQWq1ao8QE/O99Jccnt7i/v7e6TTaWQyGdzc3KCvrw8DAwNwu93w+XwtGzNqtRoMw8D+/j4sy0Iul5N97EQOTHwG56CXdVwQEzgg6L2azSYURcHY2Bji8TjGxsbkeUOxEUgk+DVNw97eHjY3N3F0dIRarSaPctDPYp9LKpWK7C3n9XphmiYCgYDsN9jT0wNFUeDxeNBoNFAsFlEoFKDrulxOtGPO9HUdG8SIPkIcSo1EIohGowiHw/D7/bJElaZpWF5eRjabxdnZGQzDkCWIeDX984kNH4+Pj6jX68jlcvB4PPIOTJSmcrlceHh4kN3l7YeixXswdfE2DGJE7+ByudDb24tAICA77iqKImssmqaJ9fV1GIYBXdfljjR7gVhOSj9Te31MsTv1o+8BdG635vdgECN6hX0jxv39PU5OTtBoNDAyMoJwOIz+/n5UKhWoqop0Oo39/X3U6/WWruEMXD+f2Ppuv2BpD0ovFRq3/7T/jV7HIEb0BvYSZpZlQVEUHB8fIxQKoVwuy2aZ4iwYwHxGpxKBzN7pXfz+UxATy82883q/LqdG+66uLmd+8U9oNpucFT/ob40XUbHc6/UiFArB7XbD6/Xi6ekJ1WoVlUoF+Xz+W+QzOF4+52+MmbdeyHyXediJY4ZBzEGcOMC+i381XuxJ+a/+3+J4+RzOMc7A5USiD7DnOQT7kiMR/Ruur/4CRE72X0l7Ivr/OXY5kYiIiHdiRETkWAxiRETkWAxiRETkWAxiRETkWAxiRETkWAxiRETkWL8Bvw0cp1W9TRMAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 9 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from fastai.data.core import DataLoaders\n",
    "dls = DataLoaders.from_dsets(train_ds, valid_ds).cuda()\n",
    "\n",
    "# Override the show_batch function of dls to the one used in our dataset!\n",
    "dls.show_batch = train_ds.show_batch\n",
    "\n",
    "# We have 8 classes! [0, 1, 2, 3, 4, 5, 6, 7] that correspond to the [0, 45, 90, 135, 180, 225, 270, 315] degrees of rotation.\n",
    "dls.c = 8\n",
    "\n",
    "dls.show_batch()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zTmLjo4PTrx9"
   },
   "source": [
    "# FastAI Vision Learner [Rotation]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:16.196844Z",
     "start_time": "2021-03-10T23:31:16.191449Z"
    },
    "id": "zVVzf6-wTrx9",
    "outputId": "f86c9735-61c7-4705-8d67-2b2be53b102c"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): AdaptiveConcatPool2d(\n",
       "    (ap): AdaptiveAvgPool2d(output_size=1)\n",
       "    (mp): AdaptiveMaxPool2d(output_size=1)\n",
       "  )\n",
       "  (1): Flatten(full=False)\n",
       "  (2): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  (3): Dropout(p=0.5, inplace=False)\n",
       "  (4): Linear(in_features=64, out_features=8, bias=False)\n",
       ")"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a config for our model's head!\n",
    "rotation_head = create_head(nf=32, n_out=8, lin_ftrs=[])\n",
    "rotation_head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2020-03-23T16:28:42.437483Z",
     "start_time": "2020-03-23T16:28:42.431824Z"
    },
    "id": "VAxNJxUTTrx9"
   },
   "source": [
    "> Note: We want to measure `top_2_accuracy` along with regular (top_1) accuracy, because there are hard-cases where it's understandable why our model got it wrong. For example: '0' rotated 90 or 270 degrees, or '1' rotated 0 or 180 degrees. (They can look the same!)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:16.284539Z",
     "start_time": "2021-03-10T23:31:16.280697Z"
    },
    "id": "58nYDDW0Trx9",
    "outputId": "40c85e77-d6f4-4775-d97e-56fe3db6ba94"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<function __main__.<lambda>(inp, targ)>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Top_2 accuracy is a nice metric for hard-cases:\n",
    "# - A zero rotated 90 or 270 degrees?\n",
    "# - A one rotated 0 or 180 degrees?\n",
    "# etc :P\n",
    "\n",
    "top_2_accuracy = lambda inp, targ: top_k_accuracy(inp, targ, k=2)\n",
    "top_2_accuracy"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "S7aNVvzHTrx-"
   },
   "source": [
    "Here, we train a model on the rotation prediction task!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:16.593891Z",
     "start_time": "2021-03-10T23:31:16.585743Z"
    },
    "id": "THQ11dpLTrx-",
    "outputId": "70f913ef-f77f-4d6f-951e-319248126984"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Sequential(\n",
       "    (0): Conv2d(1, 4, kernel_size=(3, 3), stride=(1, 1))\n",
       "    (1): BatchNorm2d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (2): ReLU()\n",
       "    (3): Conv2d(4, 16, kernel_size=(3, 3), stride=(1, 1))\n",
       "    (4): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (5): ReLU()\n",
       "    (6): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1))\n",
       "    (7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  )\n",
       "  (1): Sequential(\n",
       "    (0): AdaptiveConcatPool2d(\n",
       "      (ap): AdaptiveAvgPool2d(output_size=1)\n",
       "      (mp): AdaptiveMaxPool2d(output_size=1)\n",
       "    )\n",
       "    (1): Flatten(full=False)\n",
       "    (2): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (3): Dropout(p=0.5, inplace=False)\n",
       "    (4): Linear(in_features=64, out_features=8, bias=False)\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#collapse-show\n",
    "# Note to set a value for lin_ftrs, we use the defined config above.\n",
    "learner = cnn_learner(dls,\n",
    "                      simple_arch,\n",
    "                      pretrained=False,\n",
    "                      loss_func=CrossEntropyLossFlat(),\n",
    "                      custom_head=rotation_head,\n",
    "                      metrics=[accuracy, top_2_accuracy])\n",
    "\n",
    "learner.model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:19.140895Z",
     "start_time": "2021-03-10T23:31:16.752265Z"
    },
    "id": "mJru6eNyTrx-",
    "outputId": "2cb7fed3-9bb8-499b-fd5f-56fe1436eee2",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "Sequential (Input shape: 64 x 1 x 32 x 32)\n",
       "============================================================================\n",
       "Layer (type)         Output Shape         Param #    Trainable \n",
       "============================================================================\n",
       "                     64 x 4 x 30 x 30    \n",
       "Conv2d                                    40         True      \n",
       "BatchNorm2d                               8          True      \n",
       "ReLU                                                           \n",
       "____________________________________________________________________________\n",
       "                     64 x 16 x 28 x 28   \n",
       "Conv2d                                    592        True      \n",
       "BatchNorm2d                               32         True      \n",
       "ReLU                                                           \n",
       "____________________________________________________________________________\n",
       "                     64 x 32 x 26 x 26   \n",
       "Conv2d                                    4640       True      \n",
       "BatchNorm2d                               64         True      \n",
       "____________________________________________________________________________\n",
       "                     64 x 32 x 1 x 1     \n",
       "AdaptiveAvgPool2d                                              \n",
       "AdaptiveMaxPool2d                                              \n",
       "____________________________________________________________________________\n",
       "                     64 x 64             \n",
       "Flatten                                                        \n",
       "BatchNorm1d                               128        True      \n",
       "Dropout                                                        \n",
       "____________________________________________________________________________\n",
       "                     64 x 8              \n",
       "Linear                                    512        True      \n",
       "____________________________________________________________________________\n",
       "\n",
       "Total params: 6,016\n",
       "Total trainable params: 6,016\n",
       "Total non-trainable params: 0\n",
       "\n",
       "Optimizer used: <function Adam at 0x000001BF07FB7670>\n",
       "Loss function: FlattenedLoss of CrossEntropyLoss()\n",
       "\n",
       "Callbacks:\n",
       "  - TrainEvalCallback\n",
       "  - Recorder\n",
       "  - ProgressCallback"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learner.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:31:21.664562Z",
     "start_time": "2021-03-10T23:31:19.143280Z"
    },
    "id": "NzNeDBwJTrx-",
    "outputId": "49d128e1-eb2c-4c62-dc4b-d14db01aa238"
   },
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "SuggestedLRs(valley=0.005248074419796467)"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAx3UlEQVR4nO3dd3hVVbrH8e+bRippBAhJIPTQWyiKFXFQbFiAUcd2bYyO4uhY586Mo47lOjqOddSxy8ggYkOxIVYU6b0kQEISQwjpva77xznBEFIhO/uU9/M8eXKyz977/EJC3rP2WnstMcaglFLKe/nYHUAppZS9tBAopZSX00KglFJeTguBUkp5OS0ESinl5bQQKKWUl/OzO0BH9ejRwyQmJtodQyml3MratWsPGmNimnvO7QpBYmIia9assTuGUkq5FRFJb+k5vTSklFJeTguBUkp5OS0ESinl5dyuj0AppVpSU1NDZmYmlZWVdkexTWBgIPHx8fj7+7f7GC0ESimPkZmZSVhYGImJiYiI3XG6nDGGvLw8MjMz6d+/f7uP00tDSimPUVlZSXR0tFcWAQARITo6usMtIi0EHVBbV8/O/SV2x1BKtcJbi0CDo/n+tRB0wIvf7mXGE9/w/Ne77Y6ilPIQoaGhAKSlpTFy5EhbMmgh6IAPN/6Mn4/w0LIdPLMi1e44SqljtWkR/GMk3Bvh+Lxpkd2JbKGFoJ3S88rYll3M7TOGct7YPjz66U6eXJ5idyyl1NHatAg+vBmKMgDj+PzhzcdcDO666y6eeeaZQ1/fe++9PPDAA5x22mmMHz+eUaNG8f7777d6jrq6Om6//XYmTpzI6NGjef755wG4/PLLee+99w7td+mll7Z5rvbQQtBOy7bsB+Cs0bE8PmcsF4yL4/HPd7F00882J1NKHZXl90FNxeHbaioc24/B3LlzWbTol2KyaNEirrjiCt59913WrVvHihUruO2222htmeCXXnqJ8PBwVq9ezerVq3nxxRfZu3cvV199Na+++ioARUVFrFy5krPOOuuY8oIXF4LUAyUUV9a0e/9lW/YzOj6c+MhgfH2ER2ePISLYn+9TD1qYUillmaLMjm1vp3HjxnHgwAF+/vlnNm7cSGRkJL179+aee+5h9OjRTJ8+naysLHJyclo8x2effcbrr7/O2LFjmTx5Mnl5eaSkpHDyySeTkpJCbm4ub731FhdeeCF+fsd+F4BX3kdQV284/5mVXDK5L3fPHNbm/lmFFWzMKOSOM4Ye2ubrIwzr3Z3t2TqKSCm3FB7vvCzUzPZjNHv2bBYvXsz+/fuZO3cuCxYsIDc3l7Vr1+Lv709iYmKrQzyNMTz11FPMmDHjiOcuv/xy3nzzTRYuXMgrr7xyzFnBS1sEaXlllFTVsju3tF37f+K8LHTmyNjDtifFhrFzfwl19S038ZRSLuq0P4N/0OHb/IMc24/R3LlzWbhwIYsXL2b27NkUFRXRs2dP/P39WbFiBenpLU4ECsCMGTN47rnnqKlxXLXYtWsXZWVlAFx55ZU88cQTAAwfPvyYs4KXtgh2Oe8FyMivaGNPh2Wbs0nqHUb/HiGHbR8W252KmjrS88oYEBPa6TmVUhYaPcfxefl9jstB4fGOItCw/RiMGDGCkpIS4uLiiI2N5dJLL+Wcc85h1KhRJCcnk5SU1Orx11xzDWlpaYwfPx5jDDExMYc6iXv16sWwYcOYNWvWMeds4JWFYEdDISgoxxjT6g0YB4orWbuvgFtOG3LEc8N6dz90Pi0ESrmh0XM65Q9/czZv3nzocY8ePfjhhx+a3a+01HFlIjExkS1btgDg4+PDgw8+yIMPPnjE/uXl5aSkpHDxxRd3WlavvDTUcHdweXUd+WXVhz1XV294+bu9vLM2ky1ZRXyw8WeMgZmjeh9xnsG9QvER2J5d3CW5lVLe7YsvvmDYsGHcdNNNhIeHd9p5LWsRiEgg8A3Qzfk6i40xf2myTzfgdWACkAfMNcakWZWpwa6cEoIDfCmvriOjoILo0G6Hnlu3r4D7lm47bP+BMSEM7hV2xHkC/X0ZEBOqHcZKqS4xffr0NvsXjoaVl4aqgGnGmFIR8Qe+E5FlxpgfG+1zNVBgjBkkIr8GHgHmWpiJypo60vLK+NXw3nyydT8Z+eWMTYg49HzqAUcz7c2rJ1NUUcPOnBKmDIhq8XzDYruzLr3AyshKKWUpywqBcdwt0TAsx9/50XR4zXnAvc7Hi4GnRURMa3daHKPUA6XUGzhtWE8+2bqfzIKKI54P9Pfh+IHR+PgIZxHbwpkcknqH8eHGnymqqCE8qP3zfyulrNFWv5+nO5o/n5b2EYiIr4hsAA4AnxtjVjXZJQ7IADDG1AJFQLSVmRo6isf3iyQy2J+MgvLDnt+dW8qAHqH4+LTvF2l4rKPDWGclVcp+gYGB5OXlHdUfQ0/QsB5BYGBgh46zdNSQMaYOGCsiEcC7IjLSGLOlo+cRkeuA6wD69u17TJl27i8mwM+HxOgQEqKCycg/shCMTYhs9/mSYh19B9uzi5nUv+VLSEop68XHx5OZmUlubq7dUWzTsEJZR3TJ8FFjTKGIrADOABoXgiwgAcgUET8gHEencdPjXwBeAEhOTj6mUr8zp5TBPUPx9RESIoPZ1mjET2VNHZkFFVw0PqHd5+vdPZCIYH927NeRQ0rZzd/fv0MrcykHyy4NiUiMsyWAiAQBpwM7muz2AXCF8/FFwJdW9g+Ao0UwtLfjXXx8VBBZBRXUO+8M3pNbhjEwsGdIa6c4jIhjqoltOnJIKeWmrGwRxAKviYgvjoKzyBizVETuA9YYYz4AXgLeEJFUIB/4tYV5KCyvJqe4iqHOoaAJkcFU19WTU1JJbHjQoSknBnbw5rCk2DDe+mkfdfUGXx8h9UAp9y3dRm1dPSHd/AgJ8CUhKpgRfcIZGdeduIggr+7MUkq5FitHDW0CxjWz/c+NHlcCs63K0FRDh25DiyAhKhhwTDXRUAhEOGIqibYMi+1OZU096Xll9OoeyLw315JbUsXgnqEUlldQWlXLh5uyD81JFNbNj9iIQGLDg4iPDOLm0wbTq3vHOndU875NySXtYBmXTO6Hbzs7/JXydl41xcTOHEchSHJODZEQ6ZhwKiO/nEn9o0g9UEpCZDCB/r4dOm/DVBPbs0v4xxcp7Mkt5c1rJnP8wB6H9qmsqWPH/hK2ZBWRklNCdlEl2UWVfJuSS3iQP3ec0frcI6p1+WXV3PfhVt7b4Fgf4sNN2fzz12OJDQ9q40illHcVgv0ldA/0o1d3x53EcQ2FwDmEdHduGQNjOtYagF+mmnjss53sOVjG7TOGHlYEwHEX8tiEiMNuXgO47KVVLN2Uze0zhurloqP0/oYs/vrhNkoqa7j5tMHERwZx7wdbOfOf3/J/F47mVyOOnB5EKfULr5praOf+EpJ6dz/0B7ebny+9uncjI9/RYbwnt5RBPTs+eVzDVBN7DpYxLaknvz15YLuPPWd0H/bll7Mps6jDr+vtKmvquGPxRuYv3EDfqGCW3nQit54+hDnJCSy96QTiIoK47o21zP7XSt7fkEVVbZ3dkZVySV7TIjDGsDOnhPPG9jlse0JkMBkF5WQVVlBVW9/hjuIGExOjqKmr5/E5Y9p9MxrAjBG9+eN7m1m66WfGNGktqJbtPVjGDQvWsT27mJumDeKW6UMO6xMYEBPKkhuO5/WV6by5Kp35CzcQHRLA6cN7MSo+nNFxEfSNDuZgaRU5RZUUVtRwwuAedA/Uu8OV9/GaQpBdVElJZS1DndfzGyREBbNqTx6pDSOGjqJFAPDArJHU1NV3uH8hPNifkwbH8NGmbO4+c1iHioi32pJVxMUv/Iivr/DKVRM5dWjPZvfr5ufLtScN4OoT+vNd6kH+s2ofy7bsZ+HqZlalwtHpv+CayUSFBFgZXymX4zWFoGHEUFLvw2cRTYgM4r3iSnY47wMYdJQtAl8fwdenY0WgwdljYlm+4wDr9hWQnKh3J7fGGMP9S7fRzd+X9383lbiItjuDfXyEk4bEcNKQGIwxZBZUsCmziKzCcmLCutGreyAFZTXcumgDl7z4I29eM5kejWakVcrTeU0hiA4NYG5yAkOaTCcdHxWMMY5hh1EhAUTa8G5w+rBedPPzYemmbC0Ebfgm5SCr9ubz13NHtKsINCUiJEQFHxo63FhEsD9Xv7aai1/4kQXXTqZnmA7pVd7BazqLR8dH8MhFo4+YITQh0vEHYXVa/lGNGOoMYYH+nDq0Jx9tzvaI9Y+ra+vZmFHIgZKWF+c+GvX1hkc/3UF8ZBAXTzq2OaeaM3VQD169ahJZhRVc+NxKNmUWdvprKOWKvKZF0JKEKMe7ypo6c1QjhjrL2WNi+WTrflbtzTti6Kk7KCyv5r+rM/h+dx6r9+ZTUeMYoTO4ZyhTB/XgnDF9mNCv/ZP5NefjLdlsySrm8TljCPCz5j3MlAHRLLhmMjcuWMeFz63k7jOHcdXUREqranl/w88sWZdJvXHcfT6wZwjDYrtz3IDoDvcNKeVKvL4QxIYH4ecj1Nabox4x1BmmJfUkOMCX99f/7HaFYHVaPvPfWs/PRZUM7hnKnOR4JvaPIrOggu9TD7Jw9T7e+DGdf8wdy7lj+rR9QuCTLdk8+9VuRsaF8z9T+5MYHczjn+1iSK9QzhsbZ+n3M65vJB/PP5E/vL2J+5Zu470NWaTklFJRU0dS7zCiQgL4LjWXd9ZlAhDk78uJg3swfVgvxvWNoH+PEPx8vaaxrTyA1xcCXx+hT0QQ+/LLbS0EwQF+XDg+njdXpXPe2D4cP8j1i0FdveG5r1L5xxcpxEcG8cHvpjI6PuKwfeadPJCSyhqufm0N8xeup7KmjjnJLc/umlNcyV/e38onW/fTLzqYxWsz+c+qfST1DmPPwTJeuGxCl0wdEREcwIuXT+Dl79N4bWUa547pw8WT+zImPvzQfSgllTWs21fI8u05fLEth8+25QAQ4OfDkF6hxEcEExboR2igH32jgrlsSj8tEMolibst4JCcnGzWrFnTqee89N8/8n1qHt/cfip9o4/sROwq5dW1nPPUdxRX1rJs/okuO3LFGMNXO3N58ssU1u8r5Nwxffjb+SMJa2UMfkV1Hde9sYZvUw5y/3kjuOy4xCPOuWhNBg98tJ3q2npumT6Ea07sT3FFDQtW7eP1H9IZ1DOEt66d4pJ3YBtj2JVTytafi9ixv4Tt2cXsL6qktKqW0spaSqpqmT0hnv+7aLRL5leeT0TWGmOSm31OCwHcvWQz767PZOtfz7B9orLt2cWc98z3TBkQzatXTrTlvoLaunr2F1eSVVBBVmEFRRU1BPn7EtzNj4rqWl5dmc727GL6hAdy66+GcuH4uHb9cauqrePGBev5YnsOJw7uwe9OHcTkAdFkFVZw1zub+DblIJP7R/HwhaOPmPivrt5gjHHbd9T/+HwX/1yewvUnDeDumcPsjqO8kBaCNmQVVpB+sMxlLse8+WM6//veFu48I4nfntL+6So6Q0pOCVe+spqswooW9xkYE8K8kwdy3ti4Dnfa1tTV8/J3e3nx2z0cLK1mbEKEcx1pw91nJnHp5H4eeVOdMYa/fLCV139I5+4zk7i+A9OQKNUZWisEXt9HABAXEXRUY9KtcunkvvywO4+/f7aT6NCAVq+pd6ZNmYVc8fJP+Pr48MCskfSNCiYuMojI4AAqauqoqK6ltt4wpGfYUf+x9vf14fqTB3LF8Yks/Gkfr/2QzoR+kTwwa2SzY/s9hYhw7zkjKCiv4aFlO+jVPZBZ46zt9FaqvbQQuCAR4ZGLRlNcWcMdizeRmV/O708fYum15R9253Ht62uICPbnzasnk9jBNRk6KtDflyun9ufKqd6zrKCPj/DY7DHkllRy15JNJMWGHZoSXSk7uecFVy8Q2s2Pl6+cyJzkeJ78MpVbF22ksqbzZ89MPVDCXz/cyhWv/ERseCCL5x1veRHwZgF+Pjx58TjCAv254c11lFbV2h1JKe0jcHXGGJ5ZkcrfP9tFWDc/pg/vxZkje3PSkJgO38RUU1dPRn45ew+WsfdgGZ9ty+Gnvfn4+wpnjozlr+eOsGWKDW+0ak8el/x7FWeM7M3TF4/TkUTKctpH4MZEhN9NG0xyYhTvrsvi0237eXd9FglRQSy6/rh2rcBljOHttZncv3QbJZW/vAPtFx3MnWckMTs53mWHqnqqyQOiuX3GUB5etoP+0SHMnZhAfKSuZa3soS0CN1NTV883u3KZv3ADseGBvD3vOCKCW34Xf6CkknuWbOGL7TlM6h/F3OQEEnuE0L9HCJHB/vqHx0b19YYbFqzjk637AegR2o3JA6K479wRRGthVp1Mh496oB9253HFyz8xMq47b14zmeCAwxt3eaVVvLMuk+e+2k1ZdR13zBjK/0zt75FDM91Zfb1h+/5i1u8rZN2+Aj7c+DPnjonjsTlj7I6mPIwWAg/1yZZsbliwjqmDejBrbBwGqKuv5+tduXy+LYeaOsOk/lE8eP5IBvUMa/N8yn4PL9vBv77ezTu/Pf6YJ+lTqjEtBB7sP6v28cf3NtP4xxgZ7M8F4+OZO/HI9ReUayurqmXaY18RE9aN9288wfY73ZXn0M5iD3bJ5L6cPrwX5dW1CIII9OzejW5+Oi2yOwrp5sc9M4cxf+EG/rs6g0smd/66C0o1pfcReICYsG70iw6hb7Rj5S0tAu7t3DF9mNQ/ikc/3UFhebXdcZQX0EKglIsREf567giKKmq49vU1ulKaspwWAqVc0LDY7jx8wWhSD5Ry7tPfM++NtaTklNgdS3koLQRKuag5ExP45o5T+f30IXyXepCZT37LojUZdsdSHkgLgVIuLCzQn/nTB/PNHacyZUA0dyzexEMfb6eu3r1G+ynXpoVAKTcQFRLAy1dO5LIp/Xj+mz1c/8Zayqt1wjrVObQQKOUm/H19uH/WSO47bwTLd+Tw1JepdkdSHkILgVJu5vLjEpk5KpY3fkinqKLG7jjKA2ghUMoN3XDKQEqrannjhzS7oygPoIVAKTc0ok84pw6N4eXv07SvQB0zLQRKuakbTx1Eflk1C3/SIaXq2GghUMpNJSdGMal/FC98s4fq2nq74yg3poVAKTf2u1MHsb+4knfXZ9odRbkxywqBiCSIyAoR2SYiW0VkfjP7hIvIhyKy0bnPVVblUcoTnTi4B6Piwnnqy1SqauvsjqPclJUtglrgNmPMcGAKcKOIDG+yz43ANmPMGOAU4DER0dXTlWonEeHOM5LILKjgtZVpdsdRbsqyQmCMyTbGrHM+LgG2A3FNdwPCxLFwbiiQj6OAKKXa6YTBPZiW1JOnlqeSV1pldxzlhrqkj0BEEoFxwKomTz0NDAN+BjYD840xR/R6ich1IrJGRNbk5uZaHVcpt3PPzCTKa+r45/IUu6MoN2R5IRCRUOAd4BZjTHGTp2cAG4A+wFjgaRHp3vQcxpgXjDHJxpjkmJgYixMr5X4G9Qzj0sl9WbBqH6kHdLpq1TGWFgIR8cdRBBYYY5Y0s8tVwBLjkArsBZKszKSUp5p/2mCC/X158OMddkdRbsbKUUMCvARsN8Y83sJu+4DTnPv3AoYCe6zKpJQniw7txu+mDeLLHQf4LuWg3XGUG7GyRTAVuAyYJiIbnB8zRWSeiMxz7nM/cLyIbAaWA3caY/Q3WKmjdMXxicRFBPHQsu3U65oFqp38rDqxMeY7QNrY52fgV1ZlUMrbBPr7cvuModzy3w28vzGL88fF2x1JuQG9s1gpD3PumD6MjOvO3z/dRWWN3mSm2qaFQCkP4+Mj3HPmMLIKK3hdp6lW7aCFQCkPdPygHpw6NIanv0ylsLza7jjKxWkhUMpD3XXmMEqrann88112R1EuTguBUh5qaO8wLj8ukTd+TGdteoHdcZQL00KglAf7w4yhxHYP5K53NumaBapFWgiU8mCh3fx44PyRpBwo5bmvdtsdR7koLQRKebhpSb04d0wfnl6RQkqOzkOkjqSFQCkv8OdzhhPSzY+7l2zGGL3jWB1OC4FSXqBHaDfuPjOJNekFfLQ52+44ysVoIVDKS1w0IYGk3mE88skOXdZSHUYLgVJewtdH+ONZw8jIr+CNH9LtjqNciBYCpbzIiYNjOHlIDE8uT9E7jtUhWgiU8jL3zHTccfzUl6l2R1EuQguBUl5maO8w5iQn8PoPaWzPbrp6rPJGWgiU8kK3/moIkcEBXPrvVWR8/Sr8YyTcG+H4vGmR3fFUF9NCoJQX6hkWyKLrj+McvqPHituhKAMwjs8f3qzFwMtoIVDKSyX2COFPwYsJokmncU0FLL/PnlDKFloIlPJifiVZzT9RlNm1QZSttBAo5c3Cm1/TuCa0TxcHUW2Zv3A9761voXAfIy0ESnmz0/4M/kGHbaoggLuKz2fhT/t0XiIXUVdv+GDjz+w5WGbJ+bUQKOXNRs+Bc56E8ARAIDyB6jOfYH/fc7hryWZuXriBksoau1N6veKKGoyByGB/S87vZ8lZlVLuY/Qcx4dTOPD6RMO/vt7N45/vYlNmIa9cOZEBMaH2ZfRyBc67wCMsKgTaIlBKHcHXR7jx1EH897opFJbXcP/SbXZH8moF5Y5WWURwgCXn10KglGpRcmIU1500gBU7c9mUWWh3HK/VMC9UpBYCpZQdLj+uH+FB/jy5XOcmskuhs0VgVR9BuwqBiISIiI/z8RAROVdErEmklHIpYYH+XH1Cf77YnsOWrCK743ilQ30EQfa2CL4BAkUkDvgMuAx41ZJESimXc8XxiYQF+vHUlyl2R/FKheU1+AiEBVozvqe9hUCMMeXABcCzxpjZwAhLEimlXE54kD9XTe3Pp1tzdMZSGxSUVxMRHICPj1hy/nYXAhE5DrgU+Mi5zdeSREopl/Q/UxMJ7aatAjsUVtRYNnQU2l8IbgHuBt41xmwVkQHACstSKaVcTkRwAFcc349lW/aTeqDU7jhepbC8moggmwuBMeZrY8y5xphHnJ3GB40xN1uWSinlkq6a2p8AXx+e/3q33VG8SkFZjWVDR6H9o4b+IyLdRSQE2AJsE5HbLUullHJJPUK7cfGkvry7PoufCyvsjuM1iipqLLuZDNp/aWi4MaYYmAUsA/rjGDmklPIy1540AIAXv91jcxLv4egstr+PwN9538As4ANjTA2g0xIq5YXiIoI4b2wcb/20j7zSKrvjeLyq2jrKq+ssu5kM2l8IngfSgBDgGxHpB+gYMqW81G9PGUBVbT2vrkyzO4rHK7R4niFof2fxk8aYOGPMTOOQDpxqWSqllEsb1DOMGcN789rKNIoqdJpqK/0yvYT9ncXhIvK4iKxxfjyGo3XQ2jEJIrJCRLaJyFYRmd/CfqeIyAbnPl8fxfeglLLB76YNoqy6jj+8vZH6er1SbBWrp6CG9l8aehkoAeY4P4qBV9o4pha4zRgzHJgC3CgiwxvvICIRwLPAucaYEcDs9kdXStlpZFw4f5w5jM+35fCcDie1TGEXFIL2Tlwx0BhzYaOv/yoiG1o7wBiTDWQ7H5eIyHYgDmg8sfklwBJjzD7nfgfaG1wpZb+rpiayMbOQv3+2k5Fx4Zw8JMbuSB6nwFUuDQEVInJCwxciMhVo9yBiEUkExgGrmjw1BIgUka9EZK2IXN7C8dc1XJbKzc1t78sqpSwmIjx0wSiG9gpj/sL1ZOSX2x3J47hMHwEwD3hGRNJEJA14Gri+PQeKSCjwDnCL816ExvyACcBZwAzgTyIypOk5jDEvGGOSjTHJMTH6jkMpVxIc4Mfzl02grt5w66IN2l/QyQrLqwnw8yHQ37rlY9o7amijMWYMMBoYbYwZB0xr6zjnvQfvAAuMMUua2SUT+NQYU2aMOYhjuusx7U6vlHIJ/aJD+N+zhrE6rYC312bYHcejFJRXExnsj4g1M49CB1coM8YUN3pXf2tr+4oj9UvAdmPM4y3s9j5wgoj4iUgwMBnY3pFMSinXMHtCAhMTI3lo2Q690awTFZRbO88QHNtSlW2Vp6k4pqGY5hweukFEZorIPBGZB2CM2Q58AmwCfgL+bYzZcgyZlFI28fER/nb+KEora3nw4x12x/EYReXWTkEN7R811JxWLwQaY76j7WKBMeZR4NFjyKGUchFDeoVx3UkDePar3Vw0IZ7jBkbbHcntFZRXMzAm1NLXaLVFICIlIlLczEcJ0MfSZEopt3TTtMEkRAXxx/c2U1NXb3cct1dQXkNkiLUtglYLgTEmzBjTvZmPMGOMNYtnKqXcWlCAL386azh7cstYsi7T7jhuzRhDUUW1pfMMwbH1ESilVLNOH96LMfHhPLk8laraOrvjuK2y6jpq6oylq5OBFgKllAVEhFt/NZSswgoWrdbhpEeroMwxvYQrjxpSSqkWnTS4BxMTI3l6RSqVNdoqOBq/TEGtLQKllBsSEW49fSg5xVW8+WO63XHcUmGFs0UQoi0CpZSbOm5gNFMHRfOvr3dTVlVrdxy30zDhnPYRKKXc2q2nD+VgaTX3fbgNY3Qeoo74ZQpqbREopdzYhH6R3DRtEP9dk8FL3+21O45bKSjrmj4CvRdAKWW5308fQuqBUh78eDsDYkKYltTL7khuobCimrBufvj7WvueXVsESinL+fgIj80Zw7DY7tz81gZ27i+xO5JbKCyvIdzi1gBoIVBKdZHgAD/+fUUywQG+zF+4XtctaAfHFNTW9g+AFgKlVBeKDQ/inpnD2LG/hE+27rc7jssr6IKZR0ELgVKqi50zpg8DYkL45xcp2ipoQ5G2CJRSnsjXR5h/2mB25pSwbIu2ClqjLQKllMc6e3QfBvUM5Z/Ld2mroAV19YbiyhrL7yEALQRKKRv4+gg3nzaYXTmlfLwl2+44Lqm4ogZjIFJbBEopT3XWqFhHq+CLFOq0VXCEgkN3FWshUEp5KF8f4Zbpg0k5UMq767PsjuNyDs0zpJeGlFKebObIWMbEh/PYZzt1quomGuYZ0lFDSimP5uMj3D1zGNlFlToPURMHS6sAiLZ4CmrQQqCUstmUAdFMH9aT577aTZ7zj5+C9Lxy/HyE2PBAy19LC4FSynZ3nZlERU0dT32ZancUl5GeV05CVDB+Fk84B1oIlFIuYFDPMOZOTODNH9PZe7DM7jguIS2vjMTo4C55LS0ESimXcMv0wQT4+fDQx9vtjmI7YwzpeeX0iw7pktfTQqCUcgk9wwK58dRBfLYth5WpB+2OY6u8smpKq2q1RaCU8j5Xn9CfuIgg7lu6zatvMkvPc1we69dDWwRKKS8T6O97aJrqhav32R3HNmkHywFI1EtDSilvNHNUbyYlRvHYZ7soqqixO44t0vLK8PUR4iKCuuT1tBAopVyKiPDnc4ZTUF7NU8tT7I5ji7S8cuIiggjw65o/0VoIlFIuZ2RcOHMmJPDaD2leOZw0Pa+Mfl3UUQxaCJRSLuq2GUPw9/Xh4WXeNZzUGMPeg2Vd1j8AWgiUUi6qZ1ggN5wykE+35vDjnjy743SZwvIaSiprtUWglFIA15w4gD7hgTzw0TavWckszTl0VFsESimFYzjpnWcmsSWr2GvWLEjPcw4d7aEtAqWUAuCc0X0YkxDB/326g/LqWrvjWC4trwwRiI/UQqCUUoBjzYI/nz2MnOIqHlm2w+44lkvPK6dPeBCB/r5d9pqWFQIRSRCRFSKyTUS2isj8VvadKCK1InKRVXmUUu5rQr8orpqayGs/pPP1rly741gqrYuHjoK1LYJa4DZjzHBgCnCjiAxvupOI+AKPAJ9ZmEUp5ebuPCOJwT1Duf3tjRSUVdsdxzJdOetoA8sKgTEm2xizzvm4BNgOxDWz603AO8ABq7IopdxfoL8v/5g7loLyav743maM8bxRREUVNeSXVXfZrKMNuqSPQEQSgXHAqibb44DzgefaOP46EVkjImtycz27WaiUatnIuHB+f/oQPt68n3fWed4oon3OEUMe0yJoICKhON7x32KMKW7y9BPAncaY+tbOYYx5wRiTbIxJjomJsSipUsodXH/SQCYlRvGn97awK6fE7jid6tA9BF04dBQsLgQi4o+jCCwwxixpZpdkYKGIpAEXAc+KyCwrMyml3Juvj/DUJeMI6ebLvDfXUlrlOUNKG9Yh6BvlIYVARAR4CdhujHm8uX2MMf2NMYnGmERgMXCDMeY9qzIppTxDr+6BPHXxeNIOlnHnO5s8pr8gLa+cXt27ERzg16Wva2WLYCpwGTBNRDY4P2aKyDwRmWfh6yqlvMBxA6P5w4yhfLQpm1dXptkdp1Ok55XRL6pr+wcALCs7xpjvAOnA/ldalUUp5ZnmnTSQdekF/O2j7YyOD2dCvyi7Ix2TjPwKjh8U3eWvq3cWK6Xclo+P8NicscRFBnHDgnXkllTZHemoVdXWkVNSSUIXTi3RQAuBUsqthQf589ylEyiqqOGmt9ZRW9fqIESXlVVQgTFd31EMWgiUUh5geJ/u/G3WKH7ck8+jn+60O85RySioACBBC4FSSh2dCyfE85spfXn+mz0s25xtd5wOy8h33EyWENU1C9Y3poVAKeUx/nT2cMYmRPCHtzeSesC9bjbLKCgnwNeHXmGBXf7aWgiUUh6jm58vz/1mPEEBvlz3xlpKKmvsjtRumfkVxEUG4ePT7sGWnUYLgVLKo8SGB/H0JeNJzyvntkUb3WaJy4yCcuIju/6yEGghUEp5oCkDorn7zCQ+25bDc1/vtjtOu2Tkl9vSUQxaCJRSHurqE/pz9uhYHv98FxszCu2O06rSqloKymtsuYcAtBAopTyUiPC380cRE9qN3y/aQGVNnd2RWmTniCHQQqCU8mDhQf78ffYY9uSW8cgnrrve8aFCoC0CpZTqfCcM7sGVxyfyyvdpfJ960O44zbLzZjLQQqCU8gJ3npHEgJgQ/uCi6x1n5JcTEuBLZLC/La+vhUAp5fGCAnx5Yu5Y8sqquf7NtVTVulZ/QWaBY8SQYxmXrqeFQCnlFUbHR/DoRaP5aW8+d7+z2aUWs8nIryDepv4B0EKglPIi542N49bTh7BkfRZPfZlqdxwAjDFkFJTbMutog65dD00ppWx207RBpOWV8fjnuxgQE8LZo/vYmievrJry6jrbho6CtgiUUl5GRHjoglGM7xvB3Us2k1VYYWseu4eOghYCpZQX6ubnyxNzx1Ffb7ht0QZb5yOye+goaCFQSnmpvtHB/OWcEfy4J5+XvttrW46GFoFdE86BFgKllBebnRzPr4b34tFPd7I9u9iWDJkF5USHBBDSzb4uWy0ESimv1dBf0D3In/kL11NU0fXrF2TkVxBv42Uh0EKglPJy0aHd+Oevx7L3YBnXvLaaiuquvdkso6CcBBsvC4EWAqWUYuqgHjwxdxxr0gv47YK1VNfWd8nr1tUbfi6ssLWjGLQQKKUUAGeNjuXB80fx1c5cbnt7I3VdMJIos6Ccmjpj681koDeUKaXUIRdP6ktRRQ0PL9uBAH+fPYYAP+veL/+0Nx+A8X0jLXuN9tBCoJRSjcw7eSAADy/bQVFFDc/9ZjzBAdb8qVy1N5/IYH8G9wy15PztpZeGlFKqiXknD+ThC0bxbUouv/n3KgrLrZm6etXePCb1j8LHx55ZRxtoIVBKqWb8elJfnr10PFuyirng2ZWkHijt1PNnFVaQkV/B5P7RnXreo6GFQCmlWnDGyFjeuHoSRRU1zHrmez7bur/Tzr1qTx4AUwZoIVBKKZc2eUA0H950AgNiQrjujbU8/vmuTpmbaNWefMKD/EnqHdYJKY+NFgKllGpDn4ggFl1/HBdNiOfJ5Slc+/oaiiuP7S7kVXvzmJhof/8AaCFQSql2CfT35dGLRnP/eSP4elcu5z39PSk5JUd1rv1FlaTllTNlQFQnpzw6WgiUUqqdRITLjkvkP9dOoaTS0W/w5Y6cDp9n1V7X6R8ALQRKKdVhk/pHsfSmExkQE8p1r6/l/Q1ZHTr+xz35hAX6MSy2u0UJO0YLgVJKHYXe4YH859rJJCdGcst/N/DGj+ntPrahf8DXBfoHQAuBUkodtbBAf169ahKnJfXkT+9t4f8+2UFJG53IB4or2ZNbxuT+rtE/ABYWAhFJEJEVIrJNRLaKyPxm9rlURDaJyGYRWSkiY6zKo5RSVgj09+W530zgwvHxPPvVbqY+/CV//3QnB0urmt1/5W7X6h8Aa1sEtcBtxpjhwBTgRhEZ3mSfvcDJxphRwP3ACxbmUUopS/j7+vDYnDG8f+NUpg7qwTNfpXLCI1/y0nd7D7vn4Otdufzve1uIiwhiRB/X6B8ACyedM8ZkA9nOxyUish2IA7Y12mdlo0N+BOKtyqOUUlYbkxDBc7+ZwO7cUh78aDv3L93G8u05PDp7DF/uOMC9H2xlSK8wXr4yGT9f17kyL8ZYP+e2iCQC3wAjjTHNLgwqIn8Akowx1zTz3HXAdQB9+/adkJ7e/k4ZpZSygzGG/67O4P6l26itN1TV1jMtqSdPXjyOUBvWJxaRtcaY5Gafs7oQiEgo8DXwN2PMkhb2ORV4FjjBGJPX2vmSk5PNmjVrOj+oUkpZYF9eOX/5YAtDeoVxxxlJto0Uaq0QWFqWRMQfeAdY0EoRGA38GzizrSKglFLupm90MK9cNcnuGK2yctSQAC8B240xj7ewT19gCXCZMWaXVVmUUkq1zMoWwVTgMmCziGxwbrsH6AtgjPkX8GcgGnjWUTeobanpopRSyhpWjhr6Dmj1YpizY/iIzmGllFJdx3XGLymllLKFFgKllPJyWgiUUsrLaSFQSikvp4VAKaW8XJdMMdGZRCQXaJhjIhwoauVx0889gIMdfMnG523v8023tZWzucwdzdrZOZvL1Bk528ranpxNt9nxs2/puY7+7JvbZvfP3qrf0bayHsvvaHP53OVn31U5+xljYprd0xjjth/AC609bubzmmN5jfY+33RbWzk7I2tn52wuU1f8m7Ynpyv87Ft6rqM/+xa22fqzt+p3tDN+9u3J7W4/+67O2dyHu18a+rCNx00/H+trtPf5ptvaytn48dFm7eycjb/uzJxtHduenE232fGzb+m5jv7sW3q+I9zld7StY4/ld7TxY3f72Xd1ziO43aWhYyEia4yb3LnsLlk1Z+dzl6yas3PZmdPdWwQd5U4L37hLVs3Z+dwlq+bsXLbl9KoWgVJKqSN5W4tAKaVUE1oIlFLKy2khUEopL6eFwElEThSRf4nIv0Vkpd15WiIiPiLyNxF5SkSusDtPa0TkFBH51vnveordeVojIiEiskZEzrY7S0tEZJjz33KxiPzW7jytEZFZIvKiiPxXRH5ld56WiMgAEXlJRBbbnaUp5+/ka85/x0utfC2PKAQi8rKIHBCRLU22nyEiO0UkVUTuau0cxphvjTHzgKXAa66aEzgPiAdqgEwrcnZiVgOUAoFWZe2knAB3AousyOjM0xm/o9udv6NzcCz85MpZ3zPGXAvMA+a6cM49xpirrcjXnA5mvgBY7Px3PNfSYB29k80VP4CTgPHAlkbbfIHdwAAgANgIDAdG4fhj3/ijZ6PjFgFhrpoTuAu43nnsYlf+NwV8nMf1wrFutavmPB34NXAlcLar5nQecy6wDLjElX/2jY57DBjvBjkt+790DJnvBsY69/mPlbksXby+qxhjvhGRxCabJwGpxpg9ACKyEDjPGPMQ0Gzz37mGcpExpsRVc4pIJlDt/LLOipydlbWRAqCbq+Z0XrYKwfGfr0JEPjbG1LtaTud5PgA+EJGPgP90ZsbOzOpcs/xhYJkxZp2r5uxqHcmMoxUdD2zA4qs3HlEIWhAHZDT6OhOY3MYxVwOvWJaoeR3NuQR4SkROBL6xMlgzOpRVRC4AZgARwNOWJjtch3IaY/4IICJXAgc7uwi0oqP/nqfguFzQDfjYymDN6Ojv6U3AdCBcRAYZxxrlXaGj/6bRwN+AcSJyt7NgdLWWMj8JPC0iZ3Fs01C0yZMLQYcZY/5id4a2GGPKcRQsl2eMWYKjcLkFY8yrdmdojTHmK+Arm2O0izHmSRx/yFyaMSYPRz+GyzHGlAFXdcVreURncQuygIRGX8c7t7kad8kJ7pNVc3Y+d8nqLjkbsz2zJxeC1cBgEekvIgE4OgM/sDlTc9wlJ7hPVs3Z+dwlq7vkbMz+zF3RU94FPfFvAdn8MqTyauf2mcAuHD3yf9ScnpdVc3pvVnfJ6Q6ZddI5pZTycp58aUgppVQ7aCFQSikvp4VAKaW8nBYCpZTycloIlFLKy2khUEopL6eFQHkEESnt4tfrlDUrxLFmQ5GIbBCRHSLy93YcM0tEhnfG6ysFWgiUapaItDoPlzHm+E58uW+NMWOBccDZItLWWgOzcMyUqlSn0EKgPJaIDBSRT0RkrThWSktybj9HRFaJyHoR+UJEejm33ysib4jI98Abzq9fFpGvRGSPiNzc6Nylzs+nOJ9f7HxHv8A5BTMiMtO5ba2IPCkiS1vLa4ypwDHlcJzz+GtFZLWIbBSRd0QkWESOx7EmwaPOVsTAlr5PpdpLC4HyZC8ANxljJgB/AJ51bv8OmGKMGQcsBO5odMxwYLox5mLn10k4ptKeBPxFRPybeZ1xwC3OYwcAU0UkEHgeONP5+jFthRWRSGAwv0wvvsQYM9EYMwbYjmM6gpU45qG53Rgz1hizu5XvU6l20WmolUcSkVDgeOBt5xt0+GVxnHjgvyISi2NFqL2NDv3A+c68wUfGmCqgSkQO4Fhtremymz8ZYzKdr7sBSMSxROceY0zDud8Crmsh7okishFHEXjCGLPfuX2kiDyAYz2HUODTDn6fSrWLFgLlqXyAQue196aeAh43xnzgXOzl3kbPlTXZt6rR4zqa/z/Tnn1a860x5mwR6Q/8KCKLjDEbgFeBWcaYjc5Fc05p5tjWvk+l2kUvDSmPZIwpBvaKyGxwLJ0oImOcT4fzy3zvV1gUYScwoNGyhG0u4O5sPTwM3OncFAZkOy9HXdpo1xLnc219n0q1ixYC5SmCRSSz0cetOP54Xu287LIVxzqw4GgBvC0ia4GDVoRxXl66AfjE+TolQFE7Dv0XcJKzgPwJWAV8D+xotM9C4HZnZ/dAWv4+lWoXnYZaKYuISKgxptQ5iugZIMUY8w+7cynVlLYIlLLOtc7O4604Lkc9b28cpZqnLQKllPJy2iJQSikvp4VAKaW8nBYCpZTycloIlFLKy2khUEopL6eFQCmlvNz/A/hedWP1i5qdAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.lr_find()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:32:22.574170Z",
     "start_time": "2021-03-10T23:31:21.666666Z"
    },
    "id": "aXlyFh4KTrx-",
    "outputId": "18356fe0-b9ff-44a9-fd98-91ba68c99c88"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th><lambda></th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>0.927067</td>\n",
       "      <td>0.842473</td>\n",
       "      <td>0.618400</td>\n",
       "      <td>0.918600</td>\n",
       "      <td>00:37</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>0.841698</td>\n",
       "      <td>1.955620</td>\n",
       "      <td>0.438600</td>\n",
       "      <td>0.742700</td>\n",
       "      <td>00:44</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>0.727653</td>\n",
       "      <td>1.701742</td>\n",
       "      <td>0.506800</td>\n",
       "      <td>0.720600</td>\n",
       "      <td>00:31</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>0.618809</td>\n",
       "      <td>0.643281</td>\n",
       "      <td>0.722800</td>\n",
       "      <td>0.944600</td>\n",
       "      <td>00:32</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>0.574905</td>\n",
       "      <td>0.482118</td>\n",
       "      <td>0.798800</td>\n",
       "      <td>0.971000</td>\n",
       "      <td>00:31</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.fit_one_cycle(5, lr_max=3e-2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "obQzmN0xTrx_"
   },
   "source": [
    "> Important: We were able to achieve 76.2% top-1 accuracy, and 96.8% top-2 accuracy after just 5 epochs! Now we want to grab our `model` from our `Learner`, and save the body of it!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "F3WEWbnSTrx_"
   },
   "source": [
    "> Note: Our `model` has two components, the *body* and the *head*. `model` is a list of size 2, where `model[0]` is the body, and `model[1]` is the head!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:32:22.581345Z",
     "start_time": "2021-03-10T23:32:22.577015Z"
    },
    "id": "l3Bk9yjtTrx_",
    "outputId": "1cd1d9c7-7abd-4bed-9f94-d0779e811760"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Conv2d(1, 4, kernel_size=(3, 3), stride=(1, 1))\n",
       "  (1): BatchNorm2d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  (2): ReLU()\n",
       "  (3): Conv2d(4, 16, kernel_size=(3, 3), stride=(1, 1))\n",
       "  (4): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  (5): ReLU()\n",
       "  (6): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1))\n",
       "  (7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       ")"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Access the body of our model\n",
    "trained_body = learner.model[0]\n",
    "trained_body"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ty7GmGzCTrx_"
   },
   "source": [
    "> Tip: To save a model in PyTorch, save it's `state_dict` function! You can use `model.load_state_dict` to re-load the weights."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:32:22.589471Z",
     "start_time": "2021-03-10T23:32:22.583958Z"
    },
    "id": "uzqjT8TeTrx_"
   },
   "outputs": [],
   "source": [
    "# Make save directory if it doesn't exist\n",
    "save_path = Path('rotation_cps/')\n",
    "if not save_path.exists():\n",
    "    save_path.mkdir()\n",
    "\n",
    "# Save the rotation-pretraining weights of our model body\n",
    "torch.save(trained_body.state_dict(), save_path/'rot_pretrained.pt')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Lqy1uDA6TryA"
   },
   "source": [
    "# Original Classification Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Tcg-yF0oTryA"
   },
   "source": [
    "Now that we have pretrained our model on the rotation prediction task, we want to switch over to the original labeled data for the classification task."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LMOAauhnTryA"
   },
   "source": [
    "> Important: We're only using 180 samples for training!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:32:22.635149Z",
     "start_time": "2021-03-10T23:32:22.591062Z"
    },
    "id": "_S8ZjRWiTryA",
    "outputId": "47209ec0-af88-42ca-8c73-271ef328646a"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "180 Training Samples | 10000 Validation Samples\n"
     ]
    }
   ],
   "source": [
    "# Use 0.3% classification labeled data for training!\n",
    "# Use 100% classification labeled data for validation!\n",
    "train_ds = Custom_Dataset_MNIST('data/MNIST/processed/training.pt', pct=0.003, classification=True)\n",
    "valid_ds = Custom_Dataset_MNIST('data/MNIST/processed/test.pt', pct=1.0, classification=True)\n",
    "print('{0} Training Samples | {1} Validation Samples'.format(len(train_ds), len(valid_ds)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0hAYSJTbTryA"
   },
   "source": [
    "> Note: Notice the labels now correspond to the digit class!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:32:23.187943Z",
     "start_time": "2021-03-10T23:32:22.636841Z"
    },
    "id": "LflAJ6bcTryA",
    "outputId": "d9ea6a60-a541-4127-f3e6-aadb079d050f"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Due to IPython and Windows limitation, python multiprocessing isn't available now.\n",
      "So `number_workers` is changed to 0 to avoid getting stuck\n",
      "Due to IPython and Windows limitation, python multiprocessing isn't available now.\n",
      "So `number_workers` is changed to 0 to avoid getting stuck\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZYAAAETCAYAAAAVhSD9AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAACEM0lEQVR4nO39d5xk133YiX5P5VzVOeeenDCYQSQADkiCoCiBoijJQbIkStY+21rrrZ+9z/Z760BpqbXk56d9a0tehydbEmmRlESZpJggEkSOg4nATE/nnLurK3TlcPePqnNwu6Z7ZgA0MF3V5/v51KfSubfurfu753d+4fyOMAwDjUaj0Wh2C8vdPgCNRqPR1BZasWg0Go1mV9GKRaPRaDS7ilYsGo1Go9lVtGLRaDQaza6iFYtGo9FodpUPTLEIIZ4TQvzqh73tHe7/K0KIz76L9t1CiE0hhHU32+4mQohfF0L8zof5m7uNlhktM+8GLS97V15uq1iEEFNCiE+8/8P6YBBC/JIQ4oIQIiaEmBNC/GshhO0W7U8Cp4Bvlt9/XghRKF+oTSHEpBDivwohDsptDMOYMQzDZxhG4XbHU9n23QiwEMInhJgtv/4VIcTvmr47KIT4phBiVQgRFkI8LYQ4ZNr8PwM/L4RovpPf+iDRMqNl5t1QBfLyH0zXelMIkRFCxG/Rvlrl5dGK89wUQhhCiJ8uN7ljeakFV5gH+AdAI/AA8HHgf75F+78D/Ddj68zQVw3D8AFB4BNACrgghDj+gRzxzpwGLpVfnwEumr4LAd8CDgEtwBuUBRfAMIw08D3gFz+MA61ytMygZeZOMQzj75Y7cl/5mn8F+LNbbFKV8mIYxosV5/kTwCbw/fL3dywv71mxCCHqhBDfLo+GNsqvOyuaDQgh3iiPDL8phKg3bf+gEOIVIURECHFFCHHuvRyHYRj/Z/kPyRqGMQ/8N+Ajt9jkx4Dnd9hXwTCMccMwfq3c5gvlY+0ta25b+X2fEOIFIURcCPFDIcTvCyG+XNlWCPFbwKPA75W1/+/d5nTOAhdMr80X/Q3DMP7AMIywYRg54H8HDgkhGkzbPwf8+G1+466hZUbLzLthr8hLxTF5gZ8G/ugWzapSXrbhl4A/NwwjYfrsOe5EXgzDuOUDmAI+sc3nDZT+YA/gp6TBv2H6/jlgHjgOeIGvA18uf9cBrAOfpqTcnii/bzJt+6vl191ABOi+3bGW238D+O0dvvMChvyd8mefB17apu2vAMvl173l7Wzl968C/wZwAI8AMdO5VbZV53KLY/6D8jlmKY0QIkCh/Hxth20+CyxWfHYvEL6T/+mDfGiZ0TJTq/JCabQ+AYhalpfyecSBc+9FXt7zRd+m3T3ARsVF/23T+6Plk7IC/wT4UsX2TwO/dKd/1A7H8CvAHNC4w/cd5QviuoOL/ikgV3khy0KYBzymtl9+Pxe93K4OGAFcwM8Bv3+Ltp2Ubqi/WfH5AaDwbv+33X5omdEyU8Py8gzwhVt8Xyvy8gvAJBUK9E7l5f24wjxCiP8ohJgWQsSAF4CQ2JqpMGt6PQ3YKfm1e4CfLZuoESFEhJJWbnsfx/NZ4F8BP2YYxtoOzSLlZ/8d7LIDCG/zeTsljZ00fTa7Tbs7QgjxmfL5z1H6X5Yomdm/WP5vzla0bwL+Cvj3hmF8pWJ3fiD6Xo/lg0bLjJaZd8MelJdu4Bzwx7doFik/V628lPkl4I+NsjYxcUfy8n6C9/+IUlDwAcMwAsBj5c+FqU2X6XU3kAPWKP1JXzIMI2R6eA3D+O33ciBCiE9Rylh4yjCMt3ZqZ5R8hePAwZ3amPgp4MVtPl8E6oUQHtNnXdu0Uz97qx8xDONbhmGEgC8Bny+/DlMypUOGYbwp2woh6ih1EN8yDOO3ttndEeDKrX7vLqNl5h20zNyePSMvZX4BeNkwjImdGlS7vAAIIbrYWYHekbzcqWKxCyFcpoeNkuZKAZFywOxfbrPd3xJCHC3/Qb9JKRBUoGTWPSWEeFIIYS3v89w2gbnbIoT4GKXg608bhvHGHWzyXeCjO+zLWg6a/TtKf+xvVLYxDGMaeBP4ghDCIYR4CHjqFr+3DPTfwXGdAS4KIfoo+cHTFccWoGTKv2wYxj/dYR8fpZS1sRfQMlNGy8wdsWflxcQvAn94B+2qUl5M/ALwimEY49t8d2fycgd+uSlKGtH8+CIlc+05SoGgEUopdpV+v39FKcUxBvwlJj82pTTP5ylpzlXgO5SDZ9wcWNtkh8Aa8Cwlf+Sm6fG9W5zPceAaZd8hJf9nobxdgpI5/UfAEdM2vRXnNkBppBGn5HP9T8Af7ND2ofL/swH82x2OyV7+DwSlYOUfbNPml8r7TVScq/zPXJRM3ZbbXdMP+qFlRstMLcmL6ZokAP8dnE9Vyoup7Q3gb2/z+R3LizzxfYUQ4k+APzUM4xu7tL+vATcMw9huRPWhIIT4daDLMIx/fLeOoZbRMqN5N+x3edmXiuX9IoS4j9IoaBL4JKV01YcMw7h0N49Ls3fRMqN5N1S7vOxYxkJzS1qBv6CUZz8H/L1queCau4aWGc27oarlRVssGo1Go9lVaqFWmEaj0Wj2EHfdFSaE2Hcmk2EY4vatNDuhZUbzbtDy8uGjLRaNRqPR7CpasWg0Go1mV9GKRaPRaDS7ilYsGo1Go9lV7nrwXqP5oBFCbPv6dphT8XVavkZz52jFoqlphBAIIbBYSsa5fL4TDMOgWCyq16aaSRqN5hbUlGKRo1F982vMSAUhhKBQKNyx1WJWJDvJ1K32peVQs1+pCcUiR6Xm93p0qZFyAB9sJ2+xWG4a1HwYv6vR7FWqWrGYlUnlDaxvaI1hGNhsNqxWK16vF4fDgcPhwGKx3OQSky4vi8VCsVjEMAxyuRzpdJpcLkc2myWfz1MoFG76DWkNmT8zP2s0+42qVSxmK0V2ChpNJTabDZfLRX19PYFAgEAggMPhuMmFJZWAEIJisUg+nyeZTLK+vk4qlWJzc5N0On2TYpHbaiWi0bxD1SoW841st9vVKNRisaiRqd/vx2az4XA4gPevgAzDIJ1Ok0gkWF5e1p3JHsOsLFwuF263m/7+frq6ujh8+DCtra3U19fj8XhusZcSxWKRcDjM1NQU6+vrLC4usrKywurqKjabDZvNpqyZZDJJPp9Xv5/L5SgWi6TTaS0jmn1J1SoWeEe52O12dbNLt4fX66WtrQ2n04nH48EwjG1Hm2ZulzGUz+eJxWKsrq6yurqqXCaavYO0ZF0uF6FQiAMHDnDixAkeeOAB+vr6aGlpuSPFArC8vMy1a9eYn59nYmKCyclJnE4nbrcbh8NBLBYjlUoRDofJZDJYLBYMwyCTyZDNZslkMlo+NPuSu142/70UiJOdh9VqxeFw8JGPfISmpibl5ggEAvj9fgYHB3G5XKojuZNzrfSVm99ns1nC4TCjo6P89//+31ldXWVubu5ddx53u0BctVMpM+bAud1ux+l0cvjwYU6dOsVjjz3G6dOnaWxsxOv14nK5sFqtd/IbpNNpYrEYyWSSeDyuLBY5kEmlUmSzWWKxGPl8HpfLRbFYZGlpibW1NZ555hmi0Shra2vvW8FomXnv6CKUHz5VZ7FIhWKxWJQ1cujQIbq7u2lsbMTlcilf+tGjR7colvdLNptlfX2dYDDI5cuXAVhYWNhiuZgzkTQfDmbFIl2hLS0tHDx4kKNHj3LixAkVZH838ZBK2YnFYkSjUWw2GxaLRQX0E4kEhUIBn89HsVhkYmKC2dlZhoaGsFgshMPh21rLGk0tsecVi7mjtlqtWK1Wjh49SldXF4899hg9PT0MDAzg9/tVrMVut2O32/H5fHc0Or1T7HY79fX1nD59mn/4D/8hP/rRj1haWiIejxOPx1XaqXaR3T3MlqzH48FmK4m4WfGb32+3vXyuDPB7PB7sdrv6zpw9JjPQAILBIH19fVitVkZHR/nKV75CNBplY2PjAzlnzd2nUlb2+5SHqlAs8M5o1Gq10tbWxuDgIA8++CCHDh1SSsUcnJezrW+Vkly5f/P7nY7F6XTS0NCAx+NhfHwcl8tFOp2+aX+aD4/KwYfD4VDWhuzs4R3X5u2usRmzO1TG8CpxuVxbtvd6vfh8Pk6cOIHNZqOlpQWbzUYmkyGfz5PP53VKcg2xnVKBkiyav5Oelsp5T8VikWKxuMWqNceEq1FG9rRikZ2A/GPlKPTkyZM88sgj9Pb2EgqFsNlsSpHIC1IoFFSHL+cmyFnXlZMpLRYLhUJBjTql1SP3J9vJfS8vL/PWW29x/fp1Njc3yWQygE47vRuYr3uxWKSpqYnTp0/zyCOP8PGPfxy/3086nd6SNXg75X8niR7wzsRImfRhlhWXy8WhQ4doaWmhoaGB1dVVhoeHGR0d5ZVXXmFzc5N4PK6rRVQJZkvW/Lydd0K+7+3tpb+/Xw1Ojhw5QnNzMx0dHbjdbvL5PNlslqmpKSKRCGNjY+RyOYQQxGIxJicnVap7tcnHnlYssFW5yA6/rq6OlpYWfD4fdrtdtZXzD6LRqFIu0ueezWbZ3NxU+6t0eUgB8fv9OBwONdqonN+QyWSIx+NMTU2xvLxMNpu9aaSh+XCpHHwEg0FCoRD19fXqusk2+XweeCcl2DwxUsrXTlbuTgppu2tusViUJX38+HHW1tZwOp3kcjkuX75MNpu9KTlEUz2Y+w+r1aoGpGarpL29fYtiOXr0KJ2dnXR3d+Pz+dQEXL/fTzgcxjAMstksFouF9fV1EokEGxsbJJPJqnOv73nFYkZeOK/XSzAY3OLmKBQKrKyssLS0xFe/+lUymQwtLS2EQiH6+/uZmJjgueeeU/uQHUqhUKBYLKpO5cknn6Svr49Tp07h9/uVJVMoFEgmk0xNTXHx4kX+8A//kJWVFeLx+Ba3RjVd/FrB/J+n02mWl5dZWVlheXmZ+vp6gsGgUiLLy8vEYjHefvttIpEIiUQCKMVFZMJHMBiko6MDQA1OtnOByestlZTZgpEdjtPppK2tjcbGRnp6erBYLLz22mvk83mVLaaVy96hcuAprVHpvpSyJq+3y+Wirq6OI0eOUFdXR0dHB8FgkIaGBjo7O+ns7FTXuK2tTWWumq3sjo4O0uk09913n5K3aDTK5OQkr7/+Ol/96ldV+rqUr72uaKpKsUjkBcnn8+RyOQqFArlcjuXlZebm5hgeHiaTyRCLxWhoaMBqtTI5OcnIyAhQ8pVLIZGWTTAYxOfzkUwmVUexnRssFosRDodZWFggHo+rdjob7O4i//tsNqtSg2dmZkin02rEVygUmJ+fJxKJMDo6SjgcVu6ouro6QqEQHo9HWTrSRSonW8pORg5KpGV7K4QQKj3Z6XQSDAbx+/24XC7tBqsipCKQCURyEnYoFKKxsZH+/n4aGhro6ekhGAzS1NREU1MTjY2Nah+hUGjLdZfYbDa8Xq/ql4QQauA8NzeHz+fb4nKvBqpKscjOQXbsQgg8Hg/RaJRIJMJ3v/tdpqameOmll8hms0pZtLa2sra2xujo6E1KQI5GH330UQ4cOEB3dzetra0IIcjlcsolZrVayefzLC4usrCwwOrqqsoG0tw9zO4tgHA4rKyRq1ev0tDQQDAYJJ/Pq9n0m5ubjI+PE4/HSaVSAPh8Pnw+H319ffj9fpqbm7Fardjtdnp6ejh27Jhyc0gr98CBAzQ2Nqp4XGWHUZlMYrVaqaur4/Dhw6TTaaanp8nlcltm7Wt5urtUeh1kXFZmBNbX1+Pz+RgYGKCpqYlTp05RV1dHX18fPp+P5uZmHA4HTqdTKSHzvsx9hnSZycndskIIQENDA+3t7UQiEY4ePcr09DTxeByojhJWe16xVJqf+Xye9fV15ubmyOVyuN1uwuEwkUiEyclJ5ubmSCQSKggmfdnxeJxMJoPNZsNut6uL7/F4cLlcDAwMMDg4SGNjIz6fT41OpRJKJpPEYjGWl5eJRCKqc9HcfSo7glQqxfr6OgCRSETNL8nn88TjcdLpNGtra6RSKTUKlP5um82Gx+NhY2ND3fCZTEZZHVarVVnJcpKknIkvLRiXy7WlioOUISEEbrebtrY2ZmZmcLvdWxSjtnrvPrKzl4MIh8OBzWajrq4Oj8dDU1MTfr+fw4cP09jYyODgIIFAgNbWVtxuN8FgUCmU7TIMzX2GfC+Vj1lm5HE4nU51DNXkMt3TiqVy9CCVxOuvv87c3Bxerxer1cr6+jqbm5tcuXJFTVYDSCaTJBIJ1tfX1Y0tR6NdXV10dnbS399Pe3s7Dz/8MD09Pcr/KbFYLKTTaSYmJpiYmODFF19kZmZGXXitXPYW0rUp4yzSkqiMg1X6qJPJJMlkko2NjS2ZXgBNTU288cYbqrORKcP33HMP7e3tqrORCSUHDhzA7XbjdDrViFQeW3NzM48++iiJRIJr166xurq6JWtNx+nuDjKuIq9bIBDA4/GoOO3hw4cJhUK0trYSCAQ4duwYPp+P+vp6JRcS6TavrKJdmfouE39kG2kdmbNbpYVTbTKxpxUL7GyxFItFNTKUNZtSqZSyVMzbyjRit9utZmT39PTQ09NDd3c3zc3Nam6KRAZ74/E4sViM0dFRJiYmmJ+fZ21t7SaFUm0XvtaRmWA7Kf6dUkTNSR2SeDzO8vKyGoXKQO74+DiRSESNZtvb26mrqyMYDBIMBmlsbLzJcjGPgrcb1Wo+PMwDDumKam1tpaGhgfr6erxer5rS0Nvbi9/vV0VM6+rqcLlcN1XKNg9cZGUG+ZmUhWw2q+TMarUSCARucqXKZKFEIkEkElEFTSunYOxVqkKxSORodGJiYsey5zvh9Xrp7Ozkvvvu45Of/CR9fX309fXh9XrxeDw3CYesTjsyMsLc3Bx/9md/xuzsLOfPn9exlSrAPGdlpxu/clKsOSPIPE8hkUgo/7aZubk5tZ3NZqOzs5PW1lZsNhs9PT1qjpXsEG43j0bL1IdH5fwnqSwefPBBTp06RWtrK6FQiOPHj9PY2KhiJpXZYtshY8Fra2usra0pt7l0Z62vr5PJZJTbvXLahMViIZlMsrS0xOzsLBMTEyp70ZxUspfZ84plO253A5q/l5ZKW1sbJ06c4MiRI/T399Pc3KwWfzJ3JNLqWV1dJRqNcvHiRebn55mcnFSWiu4A9j5ma3W7eSnma7id9VJp9e70G0IIlZKey+W2LApWWf3BMAxisRgjIyMsLCwQi8XIZrPbHoPmg8fsypYZe93d3Zw4cYJgMIjH41HpwdKDIa+n7Njl5zJDNZlMkslkSCaTyn0uB8Ryrsvm5iaAUlpyX9ISKRaLykuysLCgYnpQPVMaqlKxVI5EzZ1I5Z/ucDiUUvnJn/xJDh48yOnTp9X3cnQhRyDRaJRYLMabb77J/Pw83/ve91heXmZ0dJR8Pq9jK1XCnc6er9xmO7YrAyOzvCwWi3K/yt/MZrNks9mbsoIMw2B+fp5nnnmGa9eusby8vKWDqgYXR61gtlgAlT169uxZnnzySdVGKo5oNEo+n9+SASgTL3K5nKoXuLKyoipav/LKK7z++usq809ua7VacbvdCCEYHBzk1KlTwDuKTg5sn3/+ea5fv04mk9miWKphcFuViqVy1Lnd7GibzUYwGKSlpYX777+fI0eOcPDgQZqbm2/al4ylpFIprly5wuLiIq+99hrLy8vMzs6q+Sqy/V6/qJrdZbvrLX3mZl95d3c3XV1dHDx4kI6ODvW9xWIhk8kQiURYXl5mcXFRVYeozBLS3B2kApGPTCZDLpdTmYNDQ0MkEgnq6uq2VLeWyReRSIR4PM7q6qpSMhMTE0QikS3WjsViwePx3JTYAe+kzks5mZiYYHl5WaXKm9vtdapWsWz355pHli6Xi/b2do4cOcJnP/tZenp6OHXqlFIk5vpOhUKB1dVVtYbG8PAwL7/88pZqtGafu2b/sZ28ySweKFW+Pnz4MEeOHOHs2bPU19dvmZGfSqWYm5tjZmZGrUppdoHs9BuaD45Kd6i0PnK5HNFoVC1NHYvF+Ku/+ivW1tbo7e3dkjQk6wWura2p50wmc1NhWrM1arValWussoSQVFazs7O8/fbbbGxsKHdpNVGViqUSs4npcrno7e2lqamJBx98kO7ubg4ePEgoFFImpHR/GYbB3Nwcy8vLnD9/nqmpKd58801WVla2CAbc/qbXboz9h4yxDAwM0NzczJkzZxgYGMDpdCpXGaB85kNDQ0xOTqr6T5X70nx4yH5ADhSTySSrq6ucP38ei8XC6uoqm5ubLC0tqWuXSqWYnZ1VQfhMJsPy8jKZTEYt+pZOp29ywZoVixCChoYG2tratiR7FItFdQyXLl1iaGiIaDR6Uz9ULdSMYpHpgoFAgJMnT9LX18dTTz1FQ0MDHR0dWwpNSsVSKBSYmZnhxo0bfP/73+fatWusr6+r2dh3SmW8R1O7VCYCCCHo7+/nwIEDnD59mu7ublVGX9akKxQKRKNRpVjC4TC5XO6m/Wk+PCpjcHKJ6QsXLrC0tMTq6iqRSITZ2VmVZCFLulTu407ueXPmoeyTOjo6aGtr21axjIyMqLiOeR/V0r9UnWIxa35ZsuWJJ56gtbVVZXJ0d3cTCoVoa2vD4/GoSW3JZJLR0VEuXrxIJBJhY2OD+fl5VldXuXHjhjI7b+dmk1ROujO3NX+vqU4qq2DLgUl9fT2BQEBZxo899hjd3d10dHTg9/uVW0XWlhsaGuLatWu8/PLLKsvHHLPT3B3M/30mkyEajTI+Ps7KygqJREJVMjfPO6msZH6r61c54Ozs7KSuro6HHnqIgYEBOjs7CYVCqq7hSy+9xMTEBK+++iqrq6s3Ka1qkpWqUSzmTl2+lrPoz507x8GDB2lsbFSLPNlsNrWCpJzUlkqlGB8f54c//CELCwvMz8+r4JwsA3Or39zOMtkunVUrltpgu3kthUJBlfC455576O3t5f7771cVjB0Oh1Iq0lcvFcu1a9dIJpNbRqGavYGMraRSKZaWlnbMvLrTe9osN3IbWaTyyJEjqnyU1+slGo0SDoe5fPky4+Pjyu1WzfHcPa9YtpuD0NTURDAY5Mknn+TAgQOcPXuWlpYWlWkhlYnNZiOXy7GxscHU1BQvv/wyly9f5vz58yQSCZLJpHKJVZq5ZnZKZZZFCtvb2+nt7WVjY4NIJMLa2tq2E+o0e5/t0tjl54FAgGAwyOOPP869996rFm5qbm5WRQplpk86nWZ2dpbR0VG++c1vsrCwwObmplYqe5zt0r7N79/NxGxz1Q+73c6xY8c4deoUJ0+epKOjQyUNXbx4kZmZGd544w2Wl5dJpVJVLyd7XrFIzOUMgsEgra2tnDhxghMnTtDZ2UkgELhpG5kTHo1GmZub49KlS9y4cYPZ2dmbRiTmej87jVQq3WF2ux2Px0NrayuHDx9mYWEBl8tFMplUk6C01VJdbGcZy+vu9/tpamri0KFDnDlzhp6eHurq6tRARrrL5FyWtbU1ZmdnuX79OpFIpKrKnu9ntnNr7xQLu5O4h/SiyIW/ZNmYSCRCLBZTEymnp6eJRCI1UdmjKhSL2ZQMhUJ86lOf4sSJEzz88MN0dnaqKrHy4kuTdnp6mqWlJd58802Gh4f54Q9/yObm5rZmrjlDpBK73U5bWxter5eWlhY1QaqpqYmBgQH6+vo4fPgwKysrrK+v8+yzz/L222+rwB+846PX7G3M16ihoYG+vj4aGhpoaWmho6ODzs5Ozp49y8GDB3G73SpdtFAosLCwQCQS4c0332RjY4Pp6Wnm5uZUCup21rdm73EnVRduh2EYNDQ0UFdXxz333MPAwAAf/ehHGRgYUMs4XL58menpab7zne+wtLTE2toa2Wy2Jlzpe1axmP9c+drr9apFdY4cOUJbW5tKI5bKQhZvi0ajzM7OMjc3x40bNxgfH2dxcVHt27zaX2VlUnMpEJnC3NjYSDAYpLe3d8vSo8eOHaOjo4P+/n7q6uqIRCKMj4+r6rrmc9HsbaTVIa+v3++nq6uL1tZW+vr66OjooL29XQVdJVL+wuEwy8vLXLt2jbW1NZaXl1VdKL3mSnXwXu/V7forOZv/wIEDnDhxgq6uLhoaGlSq8sLCAtPT02pe03apytXKnlYs8ga02+24XC6OHz/Ovffey4MPPsjx48dxOBzqQuRyOUZGRlhdXeX1119neXmZy5cvE41GWVhYwDAM6urq1Fos0jxta2ujrq5OxWfkHBZZNK67u5tAIKBGGn19fUpwgsEg7e3tal91dXXkcjmmpqbIZDLMzs6ysrKiYjd61v7eRAih1r2oq6vD7/crt8VHPvIRuru7GRwc3CI38M71XFtbIxKJ8PTTTzMxMcEPf/hDlSoqq9tWU6rofmY3rpGc+nD27Fk++clPcurUKQYGBtQyH0NDQ8zPz/P0008zOTnJ/Py8ql4sp0NUO3tSsVQGUG02m7Iauru7VVVQ87LAxWKRjY0NlpeXGRsbY2lpienpaVKpFIlEAo/Ho6qUOhwOfD4fbrebnp6eLYH/UCikMnccDodSKD09PQQCATo7O1Un4fV6qauru+m45W9UlkzX7C3MqetyfXpZMr2/v5++vj6VRtze3q62kzd/Npsll8uxtrbG6uoqk5OTTE1NsbS0RCKR0Nd8n+J0Oqmrq6Ojo4PBwUE6OjpobGxUmWeyVNTc3BxLS0uk02lVh7BWBh97UrHA1sqjfr+ftrY2Tp48ySOPPKLKZZhT+orFIjMzM0xOTjI6OkoymVTB1lAoREtLC4cPH8blcuFyudTa4/39/bS1talOYG1tbcvCS83Nzdjtdux2u8oCkxiGoea9GIbB4uIiy8vLXLlyhbfeekvFV3SBwb1HZacvl5195JFH6O3t5dFHH8Xj8eD3+/F6vVsKkGYyGTKZjJrz8OqrrzI7O8vrr79OOBymWCwqa7paigZq3hvmxdmkG/XgwYM8/vjjnDt3jvvvv1+loE9PT7O6usp3vvMdbty4wY0bN4hGo0DJyqn2TDAze1KxVGZfSYvF7/cTCoXU2tBmxWKxWPB6vQSDQTo7O0mlUlgsFlwuFw0NDTQ3N3PgwAHlypDrILS1tdHU1AS8Yx1ls1mVthwMBm9ae0F2Ful0ekvK8vz8PNPT0ywsLNRUIK7WEUJgt9txu900NTXR0tJCa2ursjrlYEJmfMViMWKxmFoKe3x8nIWFBcLh8I7JIZrap3Ixwfb2dnw+nyr1srS0xNTUFLOzsywsLCjPiFyzB2qnn9iTikViLvDn8/kIhUI0NTVtu1iS2+3mk5/8JLlcjp/92Z+lUCiogKn0jcsFvSqDtOZRpd/v37Lv7ZRKNpslHo+ztLTE6OiocrddunSJ69evq6qkskhhLfhMaw2zwpcDiKamJg4cOEB3dzctLS3q2kl52dzcJB6Pc/nyZUZGRnj66acZHh5mY2NDVcPVcbT9hfnerq+v58iRI3ziE5/g537u57Db7RQKBTW/7Zvf/CavvfYaw8PDRCIRoDTNQcbhaok9rVjkn53P50mn08TjccLhsFqgS3b6spNwu924XC68Xq9afAdQy8BWurHkc+VFNWeGmUtvSCslHo8zNzfH3Nwcb7/9Nul0mlQqxeTkpCpaJ60V7Wffu8hrY7ValVXb3NxMXV3dtqtPJhIJ1tbWWFlZYWVlhUgkwubmJul0ekvVBp1WvL+w2Wx4PB41x6mjowOfz6cW/FpdXWVpaUnVIEsmk+RyOVUyvxZlZE8qlkq/9ObmJgsLC1y9epW6ujoOHz6sirfJxZRkAFY+Ayp7x4zcb+WsavnIZrMUCgUymQyG8c461cVikVQqpVaT/MEPfsDExATnz59X+5Lra9xKaWn2BuaVAN1uN/feey9Hjx7l7NmzuN1utfibLBBYKBSYm5vj8uXLDA0NMT09TTKZVNawmcrrXysppJrt8fl8DA4Oct999/ELv/ALtLW1YRgG0WiUlZUVXnvtNa5du8b169dZWVlRg85a9mRUhWLJZrMkEgnGx8dxu90sLy/T0NBAY2MjbrebQCCgMjGky2unNanlvsxrJhiGoSa6LS8vk0gkWF1dJZfLbVEsmUyGtbU1FhYWmJiYYHFxkVQqpZVHFWPO5HM6naosy3alz2USSaFQUGnJq6urbGxsqIQPc32oXC5HOp1WnUllqXxN9VGZrSrrxp04cUItJOjz+TAMQ5WSGh8fZ2xsjHA4rAasch+1yp5ULLDVd5lKpUin0/zgBz/g2WefVbV3zpw5Q3t7OwcOHKChoYF7772X+vp6uru7d1QscuKkNEulleH3+7Farbz88svMzs5y4cIFotHoFsUiFZ5MNdVB2tpAJnk4nc4tn5tdoRaLhYGBAbq6uoCSPIyMjLCysqICsVJWnE6nWoNlZWWF5557jrW1Naanp7W8VDnyGktL9+DBgxw7doyf//mfp62tjb6+PtV2YmKCZ555hldeeYVr167tq4mye1axmJEXIJfLqaqx2WyW2dlZNjc3SSQSBAIBEokEwWCQjo4OtRiPmWKxyPr6OgsLC6yvr6tV/IrFIm63G4vFwtDQEKurq0SjUVKp1LblHcwz/TXViUwPlZmEjY2NhEKhLfXiKl2ZFotFKR/DMGhubsbpdOL1eslkMspike6zRCJBW1sbVquVWCzG0tKSihfKWfqxWIx4PK5laY9jrhnmcDhob2+nubmZhx56iP7+ftrb2/H7/WSzWeXROH/+PDdu3FAWq3nd+lqPvVaFYoF3bnSzJTM8PKxudofDwRtvvKEmQm6nWGSqaDgcVje0eUQKqPL52i9e28hsr/r6elpbW+nt7aW9vX2LYpFtJObK2QCdnZ10dHSobLDKoH0ulyOfz/PRj35UKRS5fO2lS5d46aWXGB0dJZFI6IHKHsdcAsrn8/HII4/Q39/PZz7zGerr6+ns7CSbzbK5ucn58+f5kz/5E1VcMpvN3jRHpdavddUoFjPyBpY3oxwNrKys4HA4iEQiW1KSpc9bVp1NJpM3xVikbzyfz9/UoZipdYHYb6TTaSKRCNevXyeZTNLQ0IDX68Xn891UR267NHeZ4GFOBJHbyDkwciXJfD6Px+PB6XRiGAaBQIDXXnsNm83G0tLSFrnVA5u9gTlmZrFYaGtrUzGVnp4empqasNvtqjbc0NAQ58+fZ2xsjPX19S2Luu0nqkqxVKZ/mhUL8K6XFL4VO3Ui5hn0WslUN4ZhkEwmWV9f59KlS4TDYdra2mhubmZwcHBLsdKdtjcPSiQyjVQ+u91u9V02m1WTeO+77z5cLpca5JgVi7Zg7j7m6y+zBLu6uujv71eLu7W0tJBMJpmcnOTatWtqVv3169dv2hfsnz6jqhSL+UaGd7foTiW3s0h2ShXeL4KxHzAMQ62RImNr0j3W29uL3++nrq4Ot9uN1+tV28k6c36//6ZUY0kmk2FycpLNzU21X4fDQSKRIBwO09DQQHt7O/F4/CbLSHP3MVsqQgja2tqor6/n4Ycfpre3l87OTlwuF5OTkywsLPDCCy8wNTXFlStXWF9f37Kf/TjtoKoUi6Ry/sl2392K201c3I+CsB+RcZBcLsfo6Kha5TEUCtHV1UVdXR1dXV2q4oOUiZaWFlURe7u5UlBysY2OjrKyssKNGzdUyaFIJMLS0hKHDx+mWCyyublZ84HcaqOyCC5Aa2srHR0d3HvvvfT29tLW1kY6neb69evcuHGDZ555RlXikK6v/Xxdxd3uQIUQ7/sA3u1M59splQ8awzD2r8TtArshM9vsU81LcDgceL1evF4v9fX1+Hw+gsEgUJKP1tZWZdXIpRjMQf9kMsnGxgbPPfcc4XCY6elpADweD8lkkng8rhYOW1tbIxwOMzs7Szgc3pLebkbLzHvn3cqL2SPS0tJCfX09P/VTP0V/fz8PP/wwHo+HtbU1lpaW+P73v68KkCYSCZXhd7f71bstL1VpsVTybi/i3b7omr2HtF7MbgypYNxut1qmAVCKZW1tjcbGRlwul6r2UCgUiEQihMNhzp8/r9YDktmLuVyOTCbD0tISk5OTKt1dTp7cj4HevYY5btbQ0EBHRwf33HMPBw4coKuri1wux6VLlxgfH+fNN99kZWWFxcVF3a+YqAmLpdq426OJaueDkhlzRpd8L0sEyUA8oNLbvV4vTqdzSwqyTCbJ5XKqekM6nVb7lqNZWS5GvpfZiDuhZea9817kpb29ncbGRn7mZ35GlZByOp3EYjFWV1f5xje+weLiItevXyeTyajMVLlUwt3mbstLTVgsGs1uUelWzefzWwpMQql23bvFPC9GZhnV0vob1Yw501MOAOrr6+no6OD48eMcP36cSCRCIpFgbGyMhYUFrly5QjgcJhwOA6gUcp0mXkIrFo2mzHYdQ+VcKPn6TipXm/dVmW24XYBY8+Fjtkrlwn4NDQ18+tOf5sSJE7S2thKPx/n617/O5OQkN27cUKV6ZHkWmV24F2IrewWtWDQaE5Udg1kBmJWL+fud9nOrVNP9UNZjr1NZVUGuItrV1UVvby+9vb0UCgXW1tYYGxtjeHiYsbExNbEaULE1HRvbilYsGs0t2E4xmN+/nwoNenR795BWiqz7J5cx/7Ef+zHOnTuHw+EgmUzypS99ibfffpvp6WkSicSWtem162tn9KwsjeZ9YC5WuV3hSk11YLVacTqdKgvM7XZTLBZZXFxkYmKCSCSilsjQlubtuetZYRqNRqOpLbTFotFoNJpdRSsWjUaj0ewqWrFoNBqNZlfRikWj0Wg0u4pWLBqNRqPZVbRi0Wg0Gs2uohWLRqPRaHYVrVg0Go1Gs6toxaLRaDSaXUUrFo1Go9HsKlqxaDQajWZX0YpFo9FoNLuKViwajUaj2VW0YtFoNBrNrqIVi0aj0Wh2Fa1YNBqNRrOrfGCKRQjxnBDiVz/sbe9w/18RQnz2XbTvFkJsCiGsu9l2NxFC/LoQ4nc+zN/cTfa4vPwrIcQ/eJfbbAoh+ne77W4hhDgphHjlw/zN3WaPy8y+7mNuq1iEEFNCiE+8/8P64BBC9Ashvi2EiAsh1oQQ//oWbU8Cp4Bvlt9/XghRKF+oTSHEpBDivwohDsptDMOYMQzDZxjGbRe4rmz7bgRYCOETQsyWX/+KEOJ3K763CiG+KIRYKJ/rJSFEqPz1fwZ+XgjRfCe/9UGx1+VFlPiiEGJeCBEtX59jt2jfBPwi8B/L788JIYomeZkTQvypEOI+83ZlGZi4k2MytxVC/KEQ4ovv4nzmhRBuIcTHhBB/UfHdPUKIF8vnOSeE+Oem37wKRIQQT93pb31QVIHMOIUQ/3v5vtsQQvx7IYT9Fu2ruY95Sgjxdvk4XxFCHDV9fcd9TNW7woQQDuAHwI+AVqAT+PItNvk7wH8ztq7J/KphGD4gCHwCSAEXhBDHP5ij3pHTwKXy6zPAxYrvfwN4GHgICAC/AKQBDMNIA9+j1AlqduZngV8BHgXqgVeBL92i/eeB7xqGkTJ9tlCWFz/wIHADeFEI8fEP5Ih3QAjRBayXj207efkT4AVK5/lR4NeEEJ8xff/fKN0PmlvzT4GzwHHgIHAv8M9u0b4q+xghxAFKMvF3gRDwl8C3hBA2eHd9zHtWLEKIurKVsFrW4t8WQnRWNBsQQrwhhIgJIb4phKg3bf9gWSNGhBBXhBDn3uOhfJ7Sjf67hmEkDMNIl0djO/FjwPPbfWEYRsEwjHHDMH6t3OYL5WPtFUIY8g8WQvQJIV4oWw0/FEL8vhDiy5VthRC/RakD+73yCOD3bnMuZ4ELptfmi14H/APgfzAMY9oo8Xb5YkueA378Nr9xV9hD8tIHvGQYxkR5xPdl4Ogt2t9KXgzDMOYMw/gXwP8fUG6CsgwMll83CCH+snxe58sW00uVbYUQ/zfg54F/XJaXv7zNuewoL2V6KXVwBcMwxoGXALN19hzwcSGE8za/c1fYQzLzFPBvDcMIG4axCvxbSoOTnajKPgZ4EnjRMIyXDMPIU5LnDkqDEslz3EkfYxjGLR/AFPCJbT5vAH4a8FAauf0Z8A3T988B85S0vBf4OvDl8ncdwDrwaUrK7Yny+ybTtr9aft0NRIDuHY7vv1AacX4PWCtve2KHtl7AkL9T/uzzlDqayra/AiyXX/eWt7OV378K/BvAATwCxEznVtlWncst/uM/KJ9jFtgsvy6Un6+V2zxWfv9PgCVgBPgfK/ZzLxC+3TX9IB9VIC89lG6sg4Ad+Nfm49im/Spwn+n9OWBum3YfA4qAt/zeAAbLr79afngoKbFZs8xVtP1D4Iu3+Y//Zfkc00DSJC/R8mtrud3/Bvx2+TwPAXPmcym3iQEntczcUmbeBP6a6f3Pl69ZcJu21dzH/H1K1rncxlqWsf/J9Nkd9THv+aJv0+4eYKPiov+26f3R8klZKXWOX6rY/mngl+70jzJt91dAjtIowQH8P4EJwLFN247yBXHdwUX/FJCrvJBlIcwDHlPbL7+fi15uV0dJWbiAnwN+v+L7nyvv9w8AN3CSUqf3hKnNAaDwXm7u3XpUgbw4gP+j/F/mgUmg7xbtc8Bh0/tzbK9YDpf32VF+bwCD5ePPAYdMbb/I+1As5XY2YAhooeQe/c42bR4GxsrnaQC/sU2beeAxLTO3/N0vAi8DTZTc7a+X/8+2bdpWcx9zGEiUZdwB/HNKg6X/l6nNHfUx78cV5hFC/EchxLQQIkbJlxsSWzMVZk2vpymNnBopjRp/tmyiRoQQEUpaue09HEqK0kX7nmEYWUpavgE4sk3bSPnZfwf77QDC23zeTkljJ02fzW7T7o4QQnymfP5zlP6XJeCPgF8s/zdny02lj/83DcNIGSV331cpjcgkfkqj1j3HHpKXfwHcB3RRusF+A/iREMKzQ/sN7lxeDN6RMUkTpc7CfG7vR17uKZ//BiXFNQw8C5wr/zefK7erB74P/Cal8+wCnhRC/FrFLv3bHPOeYA/JzG9RiktcBl4BvkFpsLC8TdtI+bnq+hjDMG4AvwT8HrBI6X+8Xt5Ockd9zPsJ3v8jSub1A4ZhBCi5agCEqU2X6XU3pYuxRulP+pJhGCHTw2sYxm+/h+O4SumGvi2GYSSAcUpukNvxU8CL23y+CNRXdERd27RTP3ubY/qWYRghSu68z5dfhymZ0iHDMN4sN5VxI/P+Kvd9BLhyq9+7i+wVebkH+JpRio3kDcP4Q0ojuZ3iLFe5c3m5WJYxM6uURp/m2MD7kZfLZRn5LeBflF9fB06V/xeZGdZPaWT5x+XznKNiICKE6KA0Mh2+3cndJfaEzJQHcn/fMIwOwzD6KbnULhiGUdymbTX3MRiG8eeGYRw3DKOBksu1Fzhv2t0d9TF3qljsQgiX6WGjpLlSlFIW68sHUcnfEkIcLf9Bvwn8ufFOwPQpIcSTopRC6xKlNM7KwNyd8GXgQSHEJ8ojmX9ASbCGdmj/XbYGoxTlY+kTQvw7Subgb1S2MQxjmpLP9QtCCIcQ4iFKwb2dWKZ0k9+OM8BFIUQfsGhsDcpjlIKvLwL/iyilPx4B/gbwbVOzj1KKNd1t9rK8nKc0km0RQliEEL9AaZQ7tkP7W8mLEEJ0CCH+JfCrwP+7sk35+P+Ckrx4hBCHuXVWzbuVFwfQbhhG5fGPlA/x58rn2Qr8dd4ZoFA+rx8ZhpG5g9/7oNmzMlO+xu3l6/0gJRfRdsciqco+pnx8Z8rH2AT8J+BbZUtGcmd9zB345aYoaUTz44uUzLXnKAWCRiil2FX6/f4V8AalwNNfAo2m/T5AKSsiTGlU9x3KwTNuDqxtskNgrdzmc5Q6hlh522O3aHscuAYIk/+zUP6NBCVz+o+AI6ZteivObYBSJx8HnilfgD/Yoe1D5f9ng1JmyXbHZC//B4JSsPIPdmjXQcm9sUkpjvR3TN+5KJmsLbe7ph/kY6/LS/l/+n1Ko8IYpayYT93ifBrL/6u7/P4cJb+zlJcF4M+BByu2M8dNmsrHG6Ok2H4HeGaHtgcouVwi3DqpYKK83zPmfVW0+Vj596KU3B//ma1+++8An7mb8lIlMvNY+RiTlKy7n7/N+VRzH/NS+TfDlOZueSvunTvqY+SJ7yuEEH8C/KlhGN/Ypf19DbhhGMatRjEfKEKIXwe6DMP4x3frGGoVIcT/BqwYhvH/26X9/Q7QahjGL+3G/t7jMZwE/qNhGA/drWOoZfZ7H7MvFcv7RZRmWYcpZRR9klIw7yHDMC7dzePS7E3K7i8H8BalxIHvUhotf+NuHpdm71LtfYztbh9AldJKyW/eQMk0/HvVcsE1dwU/8BVKrp1l4P9LudyHRrMDVd3HaItFo9FoNLtK1dcK02g0Gs3e4q67woQQe8pkEuKdFPkPypozDEPcvpVmJ/aazHwYaJl572h5+fC564plr2C1WhFCKMUilUqxWDSn4mk0Go3mNmjFQslKqVQc8r1WKBozcuBhtmwlZlnRcqPZz+x7xWKxlMJMhmFQLN5UoUGjUUrEYrEghFDPlVRaufJZo9lv7HvFYhgGQggaGxvxer34fD5sNhvpdJp8Ps/6+jrZbJZkMqk7iX3E7RTH7b43v9do9htasZRv/iNHjtDf38/AwABer5f19XWi0SivvfYa4XCYmZkZ8vn8XT5azQeNjLNJ96g5vvZeFYW0irVFrNkv7FvFIjuP5uZm6uvrOXPmDIcPH6a9vR2Px0M4HCYcDjM+Pk4+n992hKqpTcyKRFq0FosFj8eD2+3G4/HgcrmUwpBks1ny+TyRSIRcLqesXG25aPYb+06xmP3lVquVwcFBjh07xo//+I9z8uRJgsEgdrudaDTKysoKly9fJpfLYbVayeVyd/noNR8UldmAZmVgsViw2+00NDTQ0tJCe3s7jY2NKpNQto9GoySTSYaHh9nc3CSTyZDP57Vi0ew79p1ikTd5Q0MDTU1NPPDAA9x33310dXXhdruxWq0Ui0XW19dZWlpidXWVjY0Ntd12GWSa2sFms+F0Oqmrq8PlcuHz+XA6nbjdbrq7u+nq6qK7u5vm5uYtQXzDMMhkMiSTSS5evEg4HGZkZIRYLMb8/DzZbJZMZi9Up9d8WNyJl6NW+5J9p1gkjY2NHD58mIceeoiPfexjeL1eHA4HALlcjtXVVRYWFlhaWmJ9fX3HgK2mujErBiEENpsNn89HT08PgUCA9vZ2vF4vgUCAAwcO0N/fz+DgIK2trdvuL5VK0draytLSEnV1dSwuLhKPx0kkElqx7COkm3SnPmMnhVIrimbfKBZ5gV0uFx6Ph2PHjvH444/T39+Py+UCSgpF+sbHxsYYGxtjaWmJjY0NoCQshULhrp2DZvcxx0m8Xi+Dg4P09PTwyU9+kmAwSGNjI3a7HYfDQSgUIhQK4fF4yOVy206eLRQKHDx4UFk2i4uLBINBpqeneeWVV1QcRlPbyIHorRRFrSiR7dg3igVKnYjL5aKuro7BwUEeeOABOjo6cDqd5HI5stks6XSaeDzOzMwMk5OTrK6uEovFsNls2mKpQcxZYB6Ph/7+fk6cOMGnP/1pAoEAoVDopm2y2Sy5XI58Pk+xWFQKRgb5u7u7sVqtHD16lKWlJTKZDF6vl4sXL2IYhlYs+4RaVhy3o6YVi9nNYbfb8Xg8nDhxgkcffZQHH3yQrq4ufD4fAAsLC6ytrXH+/Hnm5uZ48cUXWVlZIZVKAWhLpcaQslEoFHA6nXR3d9Pd3c0TTzxBb28v9fX12O32LdddKpHZ2VlWV1dJJBIqdpLP5ykUCthsNtrb2/H7/fT19eFyuXjooYdwOp288cYbLC0tMTU1dZfOWvNhIIS4qUTUdsj088p5T7WgkGpasZixWq243W46Ojq47777GBwcpL6+Xpmr4XCY2dlZzp8/z9jYGNevXycej6usnlq42Jp3kDd8sVjEYrHQ1NREZ2cnR48epbW1FY/HA7Dl+ufzeXK5HEtLS8zMzBCLxUilUiQSCQqFArlcDpvNRiKRoKGhgcbGRtxuN729vaytrdHW1qYGKpraRSaAWK3Wm1LSzUiLt1AoKBkrFos1YdHWrGIxT3IDcDgcNDQ0MDAwwP3334/P58NisRCLxYjH47zwwgtcunSJ8+fPq9GoLslRu0j5KBaL2O12ent7GRgYYGBgAL/fT6FQIJPJEIvFWFlZYWZmhqtXr3L16lXC4bAadBQKBfUM7wxgvF4v/f399PT08JnPfIZ4PM7g4CDpdJrR0VEVz9NUL2aLpFgs4nA48Pl8nD59mnPnztHS0kJTU5Nqbx6g5vN5JiYmiEajzM7OKqs3Eolw5coV0ul0VQ9CalKxmM1P+drpdBIKhdRcBPl5KpUiEokwPT3NyMgIS0tLRKNRHU+pccyxFavVit/vJxQKEQwGcTqdqozP2toac3NzjIyMcP78eV566SUymcwWpSAHIFarVX3mcDhUuvq9995LMpnE6/Wq7EPDMCgUClgsFiwWi+p05OhVUx2YZcjr9VJfX8/AwAAPPfQQnZ2ddHR0qLZmxZLL5WhubmZ9fZ1AIEAqlSKbzbKyssLU1BQWi4V0Ol21slDTikWOIoLBIMeOHeNzn/scp06d2mKejoyMcPnyZS5fvszY2BiJREIFYjX7Bxk/EUKQSqWYmpri0qVLfOUrXyEejxOLxVQix3aWbGUR00KhoNxlv//7v4/D4cBms7G+vo7f78fn82EYBp2dnbS0tBCNRtnc3GR8fJxIJLLFotLsHczleeRAoLGxkdOnT3Po0CE+8YlP0NbWRl9fn6rSIKmsfn369Gmy2SwnT54kl8uRyWRYXl6moaGBkZERvv/97yt3GVTXHLqaViwWiwWHw0FdXR2tra0MDg4q01S6MFZXV5mZmWF9fZ3NzU0dpN+nmF2nmUyG9fV1Zmdneeutt8hkMipj8Fb+78q041QqhWEYDA8P4/P5aGpqolAoUFdXp36vs7OT7u5uldY+Nzenjkez95DXxW63qyzTxsZG+vv7OXToEKdOncLn8xEIBCgWi6TTaZVBKLHZbFgsFqV4vF4vhUKBbDarEkk2NjZuqqKtFctdwuzzlBPdOjo6ePzxxzl16hT3338/DoeDXC7H4uIiCwsLvPzyy7zwwgssLy9TKBSwWq1YrVZ1Ac1l9XVp/dpECIHb7cZut5NIJFhcXOT555/n6tWrrKysbHFPbdfhy8/MmUCGYaiMsYWFBUKhED6fj+7ubo4fP47D4cDlctHZ2UlbWxsXL15Uc6eWl5e1xbLHkG5O6QXp6+ujtbWVRx99lI6ODs6cOUN9fT3t7e0qaH/t2jWuX7/O0NAQs7Ozyn3a09NDQ0MDH/nIR2hqaqKtrU3Nr8vlcrS0tFBXV3fLwP9ep+YUi0QWDayrq6O3t5e2tjb8fj9QEo5oNMr8/DwrKyusra0pf6YeKe4vzCme+XyeeDzOxsYGS0tLhMPhm1wRZm4nK1arFZvNpuI3shMZGBjA6XTidDqpr68nEAjgcDi07O1xLBYLTqdTDRA6Ojo4cuQILS0taj5coVAgnU6TzWaZm5tjbGyM4eFhJicnldUihCCbzZJKpbZYMnIuXSKR2Da+Ui3WCtSYYpEavlAoYLfb6enp4cSJE3zqU59S8xLkxbt+/Trf+973uHz5MgsLCxQKBZX6d7vf0OnH1Y/5GhaLRWKxGEtLS1y4cIHZ2VmuXr3K8vKyGn1ut3CX2aqotGalQmlvb+fTn/40LS0tHDp0iPb2dg4dOgSUZGl+fp7FxUUmJycZGhoiHo9vOUbN3kDOUTp58iTd3d189rOfpb29nXvuuQen04ndbiccDvPWW28xNzfH6OgoV65c4erVq6yurhKNRoHSYKOhoQGPx4PVasXpdFIsFkkmk0xMTDA+Ps5f/uVfKgunWi3WmlIsULpZbTYbXq+XtrY2ZVZ6PB7lP4/H46yurjI3N0csFiOfz+NyubDZbLhcri3554VCgUKhoOYq1EKOuYYt1mmhUCASieByudjY2CCRSGCz2bDZ7vz2kP52m82mSsA0NzfT1dXF4cOHaWpqoquri1AohNVqJZPJkEql1ITJpaUl1tbWVD0xbT3vHYQQylLp7e2lt7eXrq4umpqacDgc5PN5wuEwy8vLjI2NMTMzw40bN5ienmZ1dZV4PE4mk8HlcqnSQA0NDSpDMJ/Pk0gkmJubY3Z2loWFBcLh8E0DmWoaaNSEYjHHVqxWKy0tLXR2dvL4448zMDBAQ0ODMj9XV1eZmJjgrbfe4uLFi6RSKYQQdHV10djYSE9PD8FgELfbrea5xGIxrly5oka11XSBNdtjtljS6bSyUOrr6ykUCvT39wMwPDwM3OxmlZZKsVjEZrPhcDgYHBwkEAjQ1NREMBjk5MmTdHV18eSTT+J0OpU8TU9PMz09zejoKNevX+fGjRtMTEywvLxMNpsFdl6lUvPhIlOJOzo6aG1t5TOf+Qz9/f0cO3YMq9XKxsYG8/PzvPjii8zNzXH9+nXm5+eZnJxUc5WknLW3t9Pc3My5c+c4duwYvb29eDweVldXmZ+f5/vf/z4TExNcvXqVTCZTtdYK1JBiAdRcgsbGRjo6Oujq6qKlpQWLxUKxWFTZPmNjY6ysrJDNZvH5fLjdbg4ePEhHRwcDAwMEAgFcLhdCCDY3N4lGoyrHPJVKVf3kJc1WisUi8XhclfeRJfJjsdiWlSQl8rXMDGpsbMTn83H06FHq6uro7OxUJV0aGxsxDIPNzU0ikQhra2vMzMwwNTXFyMgIU1NTqgKyuRMy/47m7iEzS7u7u+np6aG1tZX6+noAkskk09PTTE1N8dZbb7G2tsb8/DzhcFhlBBqGgcPhwOFw0N7eTl9fn1rPR1ZTz2QyJBIJZbVWswtMUjOKRQhBPp/Hbrdz6tQpjhw5wsMPP6xm2KfTaaLRKNevX+eb3/wmExMT5PN5BgYGOHDgAJ/85Cc5dOgQAwMDBIPBLfuPRCIcOHCA8fFxcrkcKysrTE9P6xu/ipGuJiEEhUKB1dVVXC4XTqeTtrY2BgcHAfj2t7+tAviVK0t6vV78fj8PPPAAnZ2dfPKTn9wSnLdYLMrdNTc3x+uvv87CwgKjo6NqQq6MzVQuNKbZGzidTgKBAJ/4xCc4duwYx44dw+PxEIvFWFxc5Hvf+x7Dw8N861vfUgMD80BECEEoFKK+vp5HH32Ue+65h5MnT9Lc3KzcYJubm6yvrzM0NMTKygqAGgxXKzWhWCRWqxW73U4gEFCZNhaLhVwuRzQaZWpqiunpaWZnZwFoa2vj6NGj3HPPPfT19dHc3Kw6BHO1Wq/XS19fHzabjVOnTjExMcH8/PyW1QGrKcdcU8IwjJvcWqlUikKhgM/nU5lc8XhcBV+hZNG4XC4OHjxIa2sr999/P52dnXR2dhIKhbDZbOTzedbX14lEIly7do25uTkuXLiwJePMnCiiZWdv4nK58Hq9NDY20tDQgNPpxDAM5ufnmZqaYmhoiJmZGbLZ7JbsQTkT3+PxcPz4cfr6+jhx4gR9fX14vd4tc1Sk7ElqITmoJhSLvAgy+N7c3ExDQwN2ux0olW1ZWFjg1Vdf5c033+TatWscOnSI06dP8+STT3Lu3DmCwSAul0tljUFJQBwOB263mzNnztDX14fT6eS1117j9ddf1yXQawB5g0vLQQZaA4EAra2tHDp0iOnpaVXmRwhBfX09jY2NfOxjH+PEiRN89KMfpa2tTSmoZDJJPB7n0qVLTE9P88Mf/pD5+XkuXbqkLB+5NLZ5hFvtnUmtIYTA7/fT2NhIV1cXPT09eDweNjc3uXTpEkNDQzzzzDNsbm5uUQxWqxWHw0FraysdHR385E/+JI888gg9PT2EQiHl6jLPeTIMQ02crAVZqAnFIjNxurq6lADIpWMTiQRjY2NcvXqVV155hZmZGdxuNz09Pdx77710dHSo1L9iscjGxoYKnMlEAKmgbDYbwWCQQCCA1+vViqVGkDdyJpNhfn4en8/HxsYGNpuNEydOYLfbiUajOJ1OXC4X9957LwcOHODs2bN0d3fj8XhUx5LL5dTKoy+88ALz8/OMj48TjUZvcqeZy6VXe0dSC1R6HaRiqaurU9WKAZUlmkgklJtUejfsdjuNjY10dnZy5MgRjh49ypEjR6irqyObzbKxsaEmYTscDuWGXVlZIRKJkEwm79bp7ypVr1iEENjtdnw+H6dOnaKvr48jR47Q2NiIEIJIJMLFixd59dVX+da3voXb7SYYDHL48GHOnTtHf3+/qudTKBTUBYZSIcH6+nq1yJfNZqO+vp76+nqCwSDFYpFEInEXz16zWxiGQSqVYmJiQhWQdDgcPPLIIzidTrXUcCgU4qmnnuKxxx6jsbFRzZaWA4xUKsXIyAijo6P8+Z//uaqEbI7pyN/TymTvIhVFKBSiublZTUOAUj8Ri8XY3NxU7nAZ5JfLJDz88MM88sgjPPzww9hsNqxWK2tra6yvrxMKhXC5XLhcLvL5PHNzc6qsVDwev6kyezVSE4rF5/PR0NDA8ePH6e/vVzd8OBxmcXFRLTEMEAgE6Ovro6urS61nLoRQCuW5555jbm6OTCaD2+0mm83S0tKiYizBYJD6+nqam5spFAqsra3pTqKKkVaDXHZ6eXkZu93OSy+9RE9PD2fPnlXrtcjikcePHycYDGK32zEMg+XlZeLxONeuXWNtbY0LFy6oKtmZTKamFnDaL5jTjGUaudPpVHXj5AC1p6cHi8Wi3KN9fX1qm76+PjweD+fPn1dp61arlQcffJDW1lZSqRTxeFxV/6jmYH0lVa9YAFWu+vjx4wwMDNDU1ESxWGR6epqlpSXGx8dZXl4GSoqlp6eHrq4uWltbVSmN1dVVZmdnef755xkaGiKRSODz+Whvb2dwcJCuri4cDocyjZuamtjc3NTZPFWOvJntdjvFYlGlocskjqeeeorOzk7uu+8+vF6vqilmrh0l5yF897vfZWpqiuvXrytXiZaL6sFsJUjFIlOEQ6EQTqdTFZV0u92EQiHlCj1w4AB9fX088MAD+P1+gsEgHo8Ht9vNlStX+OY3v0lDQwPBYFCVgZGKZXl5WSuWvYz0XcoU0ng8TjgcZm5ujnQ6TXt7O2fOnOGv/bW/xqFDh3A4HCSTSdLpNOfPn+ftt9/mxo0bLC4uksvlSCQS/PCHP2RkZAS73U5zc7OyXKR5q6l+zBMRZaxF+r0XFxdpampSsTabzcbq6iobGxuqjP6bb77JwsICFy5cYH19XVVz0EqlujBfL7mS49jYGFarlXvuuUdZqcFgkNOnTzM4OKj6kebmZurr6+nu7iafz5NOp7l8+TI3btxQfcjZs2cJBAJ4PB7sdjtTU1PMz89z7do1ZmZmlDu1FuSmphSL9ItWZuesr68D0NDQwMGDB3nkkUfwer3YbDYymQwbGxsMDw9z6dIlVU4BSv7yS5cusby8zMGDB0mlUvT19anFmfTM6NrBnJ2Tz+eJxWJEIhE2NjYIBoNq8iSU5jXNzs4yMjLCysqKmp8yMTFRM8HX/Ujl5NRCocD8/Dw2m03FUzweDzabjYGBAQqFAocPH1YlpBwOB16vl0gkQiwW46233uI73/kOo6OjLCwscO+99+L1elXpH7lkx/T0NIuLizW1ZEfVKxbp32xqalIBNsMwyOVyrK+vs7a2Rjgcxuv10tzcjN/vx+/3qzpQ6+vrTE1NMTs7qywbiSxOKJeqNQteNZe01mwtdQ+lpWJlenldXZ2aeyBdX2bkiHR4eJjh4WFGRkaIRCJb0tShNkae+xnpGi0Wi1y8eJFEIsHJkyfxer2EQiGV2SfrE4bDYa5evcrIyAiXLl3irbfe4vr162QyGXw+nyoFEwgEANTE2Wg0qkpLVfvESEnVKxYhBF6vV5mp5uKRyWSSRCJBKpVSZcrlQ7K5ucna2hobGxvKhWFG1vvRGT21R+U1lcUGA4EAnZ2dNDU14XQ6txSjlJ2JTBOdmZlhbW1NZQdqK7Z2kKV4rFYrMzMzuFwu5Qr3eDxbBpeGYRCPx9XUhpdffpmFhQWWl5cJBAJqobe2tjZV0TgWi6npDblcTrnxa4GqVywWi4Wenh4OHDhAQ0MDfr9/S1qgfJiXEpVrjQshWFhY4K233lJ1wKA0X6VYLOJ0OlWm2QMPPEBTUxOGYZBOp1XOuVYy1Yl5UqJcuycUCnH69Gn6+/t54oknVEzNbrertTMKhYKqAxYKhbYMZszxPS0TtUEmkyEajfLss89y5coV3n77bQKBgOpnzHUIl5eXuXbtmqpYLStV33fffZw5c4YHHniAvr4+isUikUiE9fV1NjY2KBQKNZFibKbqFYsQgkAgQCgUwu12q8Ju0lcub3KzApCvhRAkk0k2NjZIJpNbFuKRayW0tbWpYpZ+v59UKkUikSCZTCrXh6Y6kXOgbDYbgUCAhoYG+vv7Vf04mV6cy+XY3NxUqaZy4pzH48HlcqlBSq11DhpUnyCLS+ZyOTweD4FAQCULyWWo19bWmJycVBXRpTy0t7dz9OhRWltb8fv9ypMSj8fV4LTWqAnF4vV6lbnpdDpJJBJsbGywsLDA+vq6UhSyjIbZ5LTb7bjdbmw2m7JooFRHTJbJ7u3tJRQKEYlEePrpp7l+/TqTk5Nq4pum+pCT2e69914aGxsZGBigubmZ++67T2X3ZLNZFYMbGhpSE9zOnTvHqVOnOHjwINlslkgkotwZcvQJOsZSC8jB6erqKhaLRT1Xuq2KxSK5XG5L1Q7pdj9y5Agf+chH1ERLWV/sxo0bTE5OqgFqLXk/ql6xwDuLe0nlkc1mSafTxGIxlaVTaa0AW3zqPp9PWSRCCBobG2lra6Ojo4OmpibS6TTr6+sMDw8zPT2tRrCa6sM8GBkYGKC1tZUDBw5QX19PR0cHbrdbxeiWl5dVBtj6+jrr6+vce++9qmpta2srXq9XrU4q918rHcR+R/YXsvM3J/dIKgcSMmvU5XIpb4p0mxqGoRYF29jY2FJnrJZkpiYUi8w5l6OIaDSqVnOTVYhzuZyqUhuNRlWF2iNHjhAMBnE4HIyPj6sskHPnztHR0UFPTw+5XI5vfetbjIyM8NWvfpXNzU09+a2KsdlsPPjggwwMDPCLv/iLykUhFcL6+jqvvPIKc3NzXL16VU16lLXC5Op+fX19+Hw+Xn/9dbVmTyaTUa6xWkof3e/IOJrZ7SkxKxZ5zYvFIr29vZw6dYr29naKxaKSkQsXLvD2228zOTnJ+vq6csvXQjaYpCYUy3YlM8w+b1kxNJ/Pk81mSSQSylT1+/1bJj42NjZSKBTo6emhvr6ezc1N4vE4IyMjTExMsL6+rmMrVYpMJ/Z6vXR3dzMwMEBbWxtNTU1YrVa1wujy8jIjIyPMz88zMTHB6uoqqVSKQCCgFvWy2+04nU48Ho/KHKuVjB7Nu2O7ZaSFEASDQXp7ewkGg1itVuLxOLFYjJWVFZUslM/na3LqQs0oFpkCahgGoVCIlpYWDhw4AMCNGzcQQpBKpdRM/I6ODlVjrL6+ntbWVnK5nLrIMhXwO9/5DhMTE3znO99RZmut5JrvJ2S1hN7eXtra2vjc5z7H0aNHVRXsaDTKysoKzzzzDCMjI3zrW99ScZNQKMThw4d57LHHeOSRRzh27BjNzc2sr68rmZGlgTS1ibzft7vvK8vfy3WhDh8+zKc//Wm6uroIhUJcuHCBGzducPHiRcbGxkin0zUrM1WvWGQJDlnDp1AoqFFpR0cH6XSajo4ObDYbfr9fLTss5yZI09br9aq0wUwmw+zsLEtLS4yOjjI7O0ssFiOTydSsINQ6brcbj8ejigO2tLSozJ58Ps/a2hqLi4uMj4+zsLCwJduwqamJzs5ODh48SHd3N4FAQA1UYrEY6XSabDarXaP7lMrrbrVaVXylrq5OZQ5Kb4nMLqzlwWlNKJbV1VVVTdblchEMBmlvb+eJJ55QtXwKhQI2m03NUzCvFCmzOCwWC0NDQ8zOzvLHf/zHTExMMDIyQjqdVmsu1LIw1DItLS20t7fzC7/wC5w+fZq2tjbcbjdWq5XNzU3Onz/P6Ogo3/72t3E4HBw7dowDBw5w7tw5mpub6ejoUPMXZGWH2dlZhoaGWFxcZGNjo6ZqPWnePfK6u91uGhsbaWpqUhVBALLZrHJ/yXl1tSorVa9Y5GSjtbW1LXNLpIXS0tLC8ePHlWJoa2vbUp1WIkcRs7OzaunhlZUVZQnVoh+0VtkuK0uuLiqtELlsNbxTFqi9vZ2zZ8/icrno7++ns7NTuTFkdVur1UoqlSKVSrGyssL8/DzxeFyXc9nnmJcyDwQCSm7kNAa5yNfKyopaIAxqt1JDTSiWyclJcrkc4XCYUChEY2OjWpQrFAoxODgIoJb/NM+Wlv7RaDRKOBzm5ZdfVlWONzY2tvyOpjqRaeVutxufz0cgEMBut6ub2uVyceLECRVHcblcNDY2YrVa1XKx5rIusrDptWvXeOONN1haWlKlP6RrTbM/MCsGmRDU3d3Nww8/TE9PDzabTU1/GB0d5dKlS6ytrZFKpZR8yTT1WqLqFYthGESjURwOBzdu3CCTyaj16+UsfPMoUuaYy7TjjY0N4vE4c3NzrK+vMzQ0pIpR6lFn7ZDJZJRFm81mlRKQ7gi3243T6VRzUlwu15a00kKhwMbGBpFIhJGREWZnZxkfHyccDqv4irSKNfuHyjXrvV4vTU1NDA4OUl9fD8DExATT09NMTEywtrZWkxMiK6kJxbKyssLm5ibPPvssMzMzdHV10dzcTEtLy7YuLMMwWFxcZGhoiKtXrzIxMcHY2Bhra2sqtbQWRxH7FbnscDQaZXNzk2QyqSoWy0xCn8+nLJPKeQpyHpScz/LCCy9w5coVpqamVK0n0FbtfkUqF7vdTigUoq+vj/vvv5+GhgYA3njjDX7wgx9w/vx55ubm9oW81IRiKRaLZLNZJiYmSCQSfPe73yUYDNLY2KgC9Ob2hUKBmZkZFU+RK7glEgnS6bSe2FblbDcK3NzcxGazcePGDdxuN2fOnCEUCt20aqCc2CgrN8zPzyvf+NTUlCrvsrS0RDKZrOnOQbMzcsAqvSAul4uuri4ee+wxTp48SX19PR6PR81fWVhYUPKyH6qkV71igXdSjm/cuMHU1BQLCwt4PB6CweCWTC4hhKr9E4lEVFwlHo/f5TPQfBCYlYYMsF++fJlsNsuhQ4eUq6KSQqFALBZjamqKl19+menpaa5fv65KukQiEVUmX7M/kW5UOcHR6/UyMDDAZz7zGbq6umhoaFCZppFIhLm5ORKJhMpArfXKDDWhWCRyAabl5WVsNhsrKys3tZEWTjqdVnNWQGfz1DoyoD40NEQsFiMUCtHW1qZcX+brLivVLiwscPnyZcLhsLJQUqmUrrygUfPlmpubaWxs5NixY5w+fZqDBw+qmfZra2vKvS6LU5qp5b6mphSLXGtldXX1PW2vlUttUbmGuXSXLi0tqaxBOXnNXAgwnU6r0i5jY2NkMhm1Vo9Ey8r+RloeshzU448/zsGDB+nv71eyEQ6HGRkZUQH7SjdYLVNTikVeNHOdsO3YrtLxdu81tYdMzLh8+bKaJFtZoiefz6tqDtIvLl0foOVkv2LuT2SgXlopDz30EPX19RSLRZLJJNFolDfeeIMf/ehHDA0NbZm7sh+yB2tKsQBqktLtFItWKPsTudT09PT0HW8jlY/ZZablZX9RWc3Ybrfj8/no7e1lcHCQQ4cOqUW/kskka2trjI6O8uabb7K4uLilLth+6G9qSrFUrhR5O4ul8rWm9jFXvJbvt8MsS+Zy6PI7zf5EFpn0+/3U1dXR3d2tCpnm83k2Nze5du0azz33HK+99hqzs7PbruFS69SUYpHoUaVmJ8xu0ltZtZWLL2lZ0kjk8guykoPL5VIxvEQiwcLCAm+//TZzc3PEYrG7fbh3hZpULBrNTtyq/Pl2aIWiqURmlcqU9Hw+TygUYmVlhaGhIV599VVeeuklksnkvnWfasWi2Zfsp5tcs/vkcjlSqRTLy8tYrVZmZmZYWlpibGyMmZkZlZla60H6nRB3+wYTQuy7O9wwjP0pbbuElhnNu+GDkBe5mFcwGFS15fL5PKlUimQyedcnXd9tedGK5S5wty96taNlRvNuuBvyst3SDR8md1te7rpi0Wg0Gk1toVev0mg0Gs2uohWLRqPRaHYVrVg0Go1Gs6toxaLRaDSaXUUrFo1Go9HsKlqxaDQajWZX0YpFo9FoNLuKViwajUaj2VW0YtFoNBrNrqIVi0aj0Wh2Fa1YNBqNRrOraMWi0Wg0ml1FKxaNRqPR7CpasWg0Go1mV9GKRaPRaDS7ilYsGo1Go9lVPjDFIoR4Tgjxqx/2tne4/68IIT77Ltp3CyE2hRDW3Wy7mwghfl0I8Tsf5m/uNntcZl4WQpx+F+0fFUIM73bb3UQI8XUhxI992L+7W+xxednXfcxtFYsQYkoI8Yn3f1gfDEKIvyGEGBZCRIUQK0KIPxJCBG7R/iRwCvhm+f3nhRCF8oXaFEJMCiH+qxDioNzGMIwZwzB8hmEUbnc8lW3fjQALIXxCiNny618RQvyu6bvGcue2LoSICCFeFUJ8xLT5fwZ+XgjRfCe/9UGy12UGQAjx/xBCLAkhYkKI/yKEcN6i7VNA3DCMS+X3XxBC5IQQ8fJjRAjxe0KINrmNYRgvGoZx6E6OpbLtu/n/hBBHhRCvlF//phDi/17x/ceFEDeEEEkhxLNCiB7T178DfPFOfueDpErkpV8I8e3y9V4TQvzrW7Styj6m/JlVCPFFIcRC+VwvCSFC5a/vuI+pBVfYy8BHDMMIAv2AjVvfLH8H+G/G1jWZXzUMwwcEgU8AKeCCEOL4B3TMO3EauFR+fQa4aPpuE/gVoAmoo9Qp/KUQwgZgGEYa+B7wix/a0VYpQogngX8KfBzooSQ3v3GLTf4u8KWKz75mGIYfqAd+CmilJDNtlRt/wJwB3jS9VjIjhGgE/gL45+XjfBP4mvzeMIw3gIAQ4uyHdrRViBDCAfwA+BGl69wJfPkWm1RrHwOl++Bh4CEgAPwCkIZ318e8Z8UihKgra/BVIcRG+XVnRbMBIcQb5VHhN4UQ9abtHxRCvFIefV8RQpx7L8dhGMasYRhrpo8KwOAtNvkx4Pkd9lUwDGPcMIxfK7f5QvlYe4UQhuzEhRB9QogXyhr9h0KI3xdCfLmyrRDit4BHgd8rj1R+7zancxa4YHqtLrphGGnDMIYNwygConyedZQ6DMlzwI/f5jfuGntFZoBfAv7AMIxrhmFsAP8r8PkdjtkBfIydZSZnGMY14K8Dq8A/Km93TggxZ9rPveXRX1wI8WdCiK8JIb5Y2VYI8SWgm9KgYVMI8Y9vcy5mmTkNXDZ99zngmmEYf1buFL4AnBJCHDa1eY49KjN7SF4+DywYhvG7hmEkyvfi1Vu0r8o+RghRB/wD4H8wDGPaKPF2WXYkz3En8mIYxi0fwBTwiW0+bwB+GvAAfuDPgG+Yvn8OmAeOA17g68CXy991AOvApykptyfK75tM2/5q+XU3EAG6b3GMjwBRwAASwCd3aOctt2kyffZ54KVt2v4KsFx+3VvezlZ+/yrwbwBH+bdjpnOrbKvO5RbH/wflc8xSskwilBRHhFLHYG57tdzOAP5zxXf3AuHbXdMP+rHXZQa4Avx10/vG8v/ZsE3bY0Ci4rMvyOOq+Pw3gdfLr88Bc+XXDmAa+J8AO6UOPwt8sbLtrf6/it/6Qfkc82X5i5lk5nvlNv8H8H9WbPc28NOm9/8Q+AstL7eUl/9CyWL9HrBW3vbEDm2rto8BHiu//yfAEjAC/I8V+7mjPuY9WyyGYawbhvF1wzCShmHEgd8CPlrR7EtGSeMlKJnjf02UAk5/C/iuYRjfNQyjaBjGDyiZ6Z/e5ndmDMMIGYYxc4tjeckoucI6gf8PJUHdjlD5OX4Hp7jAVmsAKAXOgPuAf2EYRtYwjJeAb93B/nbEMIy/DfRROu5G4NeA/1A+72MVbU9SMlF/DnipYldxSqb2nmQPyYyP0kBEIl/7t2kb4s7kBXaQGeBBSi7af2uULJy/AN64w31ui2EYTwD3A5cNwwgAvw380/J5y4B85XlSfm8+zzjv3Bd7ij0kL53A3wD+LdAOfAf4ZtmarSRUfq7GPqaTUv9xsNz2Z4AvCCGeMO3qjvqY9+MK8wgh/qMQYloIEQNeAEJia6bCrOn1NKXRWiMlv/bPlk3UiBAiQkkrvy//tGEY88D3ga/u0CRSft6uA6mkAwhv83k7JY2dNH02u027O0II8Zny+c9R+l+WgD8CfrH839zk/zZKpvhXgH8qhDhl+srPzR3JnmEPycwmJeUska+36ww2uDN5gVvLzLxRHvKVeT8y8/fL538FOFZ+/b8C/6z838jgauV5Un5vPk8/79wXe4o9JC8pShbH9wzDyFKyJBqAI9u0jZSfq7GPSZWff9MwjJRRcvd9la3K+I76mPcTvP9HwCHggfKI6TF5HqY2XabX3UCOkik5S2mkETI9vIZh/Pb7OB6JDRjY7ovyqGackka+HT8FvLjN54tAvRDCY/qsa5t26mdv9SOGYXzLMIwQJVP78+XXYUqmdMgwjDdvsbmdUuBZcoRSZ7NX2Ssyc41S1o7kFCWXxPo2bccAIYTouNUOhRAW4Cl2lpkOIcRO51nJ7WTm98py8jyl+E8PJcUVLP8vK+WmW85TCOGldG9cM+1uL8vMXpGXq9zmmkiqvI+RcSPz/ir3fUfycqeKxS6EcJkeNkqaKwVEygGzf7nNdn9LlNIhPZT8z39ulFLkvgw8JYR4UpTS21zlAGZlYO62CCF+vmw6IkqplL8FPHOLTb7Lzea03Je1HDT7d5T83jdlChmGMU3JpP6CEMIhhHiIUoeyE8ts7fx34gxwUQjRBywaWwNmMhD5SPk33UKIfwK0AK+bmn2Ukh94L7BnZQb4Y+Bvl38nBPwz4A+3a1geof6QnWXGJoQ4AnyFUsbQ727T7FVK/uy/X27/k5TcWDtxpzJzD6Wb/F5uzu4B+O/AcSHETwshXMC/AK4ahnHD1GavyMxelpcvAw8KIT5Rtpb+ASXlNbRD+6rsYwzDGKek6P4XIYSzLNd/A/i2qdmdycvtgjCUfHJGxeOLlMy15yiZ2yOUUuwqA0r/ipIvOQb8JdBo2u8DlEZcYUrZNN+hHDzj5sDaJjsH1n6LkomXKD//J7YJwpraH6c0YhPGO4G1Qvk3EpTM6T8Cjpi26a04t4HyBYhTUmL/iVKW0XZtHyr/PxuUfOzbHZO9/B8ISsHKP9imzUcpdSLx8n/2PPCY6XtX+fxbbndNP+jHXpeZcpt/SOmGjAH/FXDeou2PUw6Il99/gdLIWMrMKPDvgQ5Tm3NsDcifpZSxtUkpCP0XwD/foe1PAjOU3Cr/8w7H1M07Qdd/JPe1TbtPADcoddDPAb2m7+4DLmp5uSN5+Rwl6zVW3vbYLdpWZR9TbtdBKZywCUwAf8f03R33MfLE9xVCiD8B/tQwjG/s0v6+BtwwDGO7EdWHghDi14EuwzBul56qeQ8IIV4G/r5RniS5C/t7nVLw9L/uxv7e4zF8nVIH8927dQy1yn7vY/alYnm/CCHuozQKmgQ+CXwDeGi3Oh1N7SGE+CgwTMmF8vPAfwD6DcNYvKsHptmTVHsfY7vbB1CltFJyZTRQMg3/XrVccM1d4xDwp5TmOUwAP6OViuYWVHUfoy0WjUaj0ewqtVArTKPRaDR7iLvuChNC7DuTyTAMcftWmp3QMqN5N+xFeRFCqAcgs67M2Vnvi7stL9pi0Wg0mg+Z7RRJLYUl7rrFotFoNPsBi8WCEIJisYhhGBSLxbt9SB8YWrGAMke3+8w8iqilEYXm/SNlZDt3hkYjqexf/H4/TqeThoYGHI5SHctCoUAsFiOdTrO2tlb1MrSvFYsQAovFol4DN72XF9g8yqj2i655/wghsFqtahQK78hIoVDQMqJRWCwWrFYr+XwewzA4ePAgHR0dfOpTn6KpqQnDMNjc3OTChQtMT0/z/e9/n1wud7cP+32xLxWLeQRR2QFI83Q7xXIn22v2B9I3XiwWtygWM9tZvZr9hVk2AoEAHo+HQ4cO0dfXx+DgIPX1par50WiU2dlZotHoth6UamPfKRaLxYLFYqFYLL4n60NaORaLRY9O9zmFwm2XJ9fsY2RfIQccnZ2ddHV18alPfYojR45w6NAhvF4vAOFwmLm5OeLxuPKaVDP7TrHIUaY0T202GzabDYfDgc1mw+12q/dCCPL5PLlcjkgkQjabJZlMKkHZrdRATfUghFAyU1dXh8fjwWazYbFYiEajZDIZNjY2lNLR8rG/MQyDUCiEz+fj1KlTHDhwgN7eXpqbm7FareRyOcLhMEtLS4yOjjIzM1MTQf19oVjMLgn5sNlsOJ1OvF4vLpeLUCiEx+OhpaUFn8+H1+tFCEE6nSaRSHD9+nWi0SjJZFJZKpr9hRyBut1u5dJobW3F7/djsVgYGxtjY2ODzc1NbclqVF/T0tJCT08PH//4xzl9+jT9/f0EAgEymQzJZJLR0VGmp6d59dVXWVhY0IqlGvH5fASDQZqammhsbFSjiaamJvx+P11dXfh8vi0WSzwep6WlhbW1Na5fv046nSaZTJJIJAiHt1sATlOLSGvXZrPhcrk4deoUx48fp66uDovFQn9/P2tra3i9XqLRKBsbGyrLpxY6C827w+Vy4XK5OHHiBGfPnuXQoUO0tLTgdDopFovEYjE2Nja4ePEiU1NTzMzMEIlEakJW9oVikSNHi8WilMfg4CCDg4M0NDQQCoXo7Oykrq6O/v5+/P6tq4rGYjHq6+tZWFjA4XAQi8VYXV1laWnpJsUihNAj1RpGWrsul4t7772Xxx9/nKamJmw2G8PDw6ysrGCxWFheXmZiYoL19XXC4XBNdBaaW1OZrCE9IadPn+bJJ5+kp6eHUCgElOJzkUiExcVFXn/9daanp5mamiKbzd6tw99V9oVisVgsBINBOjo6OHXqFI899hgtLS20tLTgdrtxOBwqt9xms5HP51UaqRACl8vFsWPH6Onpoaenh0wmQywW48KFC3z9618nHo/flM2hlUvt4XA4sNvtBAIB6uvrMQyDVCpFJpPBYrHQ0tJCIBDAZrOxurrKCy+8wOTkJCMjI3f70DUfMGalYrVasVqtHD58mNOnT3P8+HFaWlpwuVwA5PN50uk0165dY2JigqGhIVZWVlQKey242WtesUi/eCAQYHBwkPvvv5+nnnoKv9+Pz+e7qX02myWfz6uArBACh8NBX18fACdOnKBYLJLL5fB4PLz00ksAxOPxm/allUvtIITAbrfj9XoJBoMEg0EAUqkU+XyeYrFIfX09Qgg6OztZW1tjaWmJTCZTE+mjmjvHYrGoPuORRx5hYGCA+vp6bLZSd1soFEin04yPjzM0NMTU1BTRaBSn06kVy17GfCMHg0EGBwc5fvw4P/ETP0Fvby/BYBCbzbYlmD8/P69mvhqGQVtbGx6PB5/Pt2XSpHxttVppamri3nvv5fr164TDYXK5nJrYpF1itYPD4cDj8XD69Gnuuece+vv7aW9v5/DhwzQ3N6t4nIzBRKNRVlZWmJ+fr4lZ1JqdMVsqFosFm81GR0cHBw4c4OzZs5w8eZLm5malVPL5PFeuXGF2dpZXX32VyclJMpmM+q5WZKXmFIu5zIbFYsHn89Hb28uRI0d44IEH8Pv9yiSVc1mKxSKrq6ssLi6STqeBUmeSy+VUaqnVat1SjVQIgdfrpaOjg8XFReVCMwxDj1BrDLvdjsfjobe3lwceeIADBw7Q0dGB1+vF6XTeVPZHJnZsbm6SSqWU5VsrnYZme6Sl0tjYyMGDB+nr66OzsxOPx4PFYiGfz5PNZpmenmZ4eJjx8XHm5+fJ5/MIIWrCUpHUlGKxWq1AydR0uVz09PRw6NAhPve5z9HT00NjYyNWq1X5xlOpFDMzMywuLvLMM88wOjqqUkU7OjoIBAIcOXKElpYWHnvsMfx+P4FAAMMwyOfzrK+vc+3aNWZnZ0kmk1sEQ3ci1Y25rHkwGKS7u5ve3l41B0FmgsmigvDOhMn6+nqcTief/exnGR4eZmlpiZWVFWZnZ7Vc1AiVg0er1UpjYyOHDh3i4Ycf5id+4ifo6OhQ85wMw+DKlStMTU3xzW9+k7GxMWZmZtjc3NwyL65WqCnFYr7YVqsVn89HfX09XV1dNDc3q9FlsVgknU4TjUaZm5tjcnKSa9euMTQ0RDwep1AoMDc3RzAYpFgskkgkuP/++/F4PMA7aaepVIrV1VXi8Ti5XG7b9RU01YdZjmSMLRAIqIfX68XhcKjKDdIaMWcDWa1WBgcHKRaLdHR0UCwWWVpaUvEYTe1g9o50dXXR19fH4cOHcbvd2O128vk8+Xye2dlZhoeHGR0dZWJiglgsphKFao2aUizmGzaTyTA9PQ3AD37wA06ePEl3d7ea3Pj2229z/vx5Lly4wOjoKHNzc0QiETXqXFpaIpFIqPQ/r9eL2+3e0oGkUilWVlbY3NwE3hGwWjJp9yOVa2WY/ecOh0NZxrKNnAxpHlTYbDaampqwWq388i//MsPDw7jdbpaXl5mamlIuWNDxuGpEKoNCoYDNZsPv99PT08Pjjz/OkSNHCAQCyqK9ePEi169f5+mnn2ZoaIjZ2VnlGYHaHITWlGKRF0i6JxKJBBsbG0xPT9PQ0MD6+jqAskjGxsaYnJxkamqKzc1NpUSk5WGeZW2321WHYhgGuVyObDZLOp2u+kqkmhLymlutVpxOp3pdV1dHIBBQI9BbWaXmGJzT6SQYDDIwMEAmk6GpqYlEInGTRaSpLsyeCTmnqa6ujubmZjo6OlQGmPRsLC8vMzo6yuTkpFIq5j5DK5Y9jvkCSVfV4uIizz77LOPj40xOTqoR5vDwMCMjI0QiETY3N1U6qewQHnjgAbq6uviZn/kZOjo68Pv9KrNjc3OTyclJ5ufnSSaTWrFUMeY4idPppKWlhba2Nh566CHq6upobW0lEAhQV1dHV1cXdXV1yg0mtzVbMLBVWbhcLgYHB0mn07S1tRGNRj/sU9TsItJSMQwDp9NJW1sb3d3dPPHEExw4cIDTp0/j8Xi2xHGvXr3KD37wA6ampojFYsoCruVagzWlWMzICyaLAsqZ0cViUVks6+vrZDIZCoWCyvqy2+24XC46Ojro7e2lra1NBf2hlBK4ubmpts/lcltM2loVlFrFHKS3WCzKyujr66OxsZGuri7cbjder5dQKITdbt/SuRiGQTab3eLWknOg5HotTqcTl8ulyqabR7M6i7C6qJwIKQcfg4ODdHV14ff7sVqtamC7sbFBOBwmHA6TTqe3DEjMrvta6zdqVrFIisWiSvtcXFxUn8uAmsViwW63UygUKBQKhEIhmpqaePzxxzl06BC9vb243W4V8I9EIrz99tv84R/+ocrqyOfzW36v1oSkVjFbGzJOYrFYCIVCarZ0f3//lioMUgEJIUilUmSzWWZmZojH46qOWHt7Ox6PR2WOFQoFPB4Pg4ODRKNRgsEgiURi20m1mr2LeR5bPp/H7XZz4sQJTp06xcc//nHcbjcul4tCoUA2m1VJQbI4abFYxOFwqH3IvsK8to/8vNqpacViHglKRVKJVAIy06ejo4O2tjaampqU20OmKJvjNrOzs6yvr28J3GqFUn2YA/UWiwWPx4Pb7VYPOfnR3F7KQiQSIRaLMTExQSQSAcDpdKqBiKwLBaV5UVKmPB7PTTWhtPzsbeSgQt7rcm5Ta2srDQ0N+Hy+LfG3QqFAOBxmamqKjY0NZdWaF4czK5XK36p2WahJxWJeLtZsum6XLy4/GxgYoLu7m4985CN0dXVx6NChLWUY5ATIhYUFJiYmuHbtGplMZssKgprqQiZhSBwOB+3t7TQ3N+NyubBYLKoOmNm1lc1myWazXLhwgfHxcV5++WVWVlaUG+0zn/kMXV1dtLW1KbdYfX099957L7FYjPb2dorFoq6MXUVIxSLd5g0NDXR2dnLmzBl6enqw2WxbVossFovcuHGDb3/728zMzKjlNvYLNalYtruA5owxucCX1WrF4/GoIpODg4McOHCAlpYW/H4/drtddT6bm5usr6+zsLBAOBzest6G9pNXJxaLBa/Xi81mw+fz0dDQoGZLBwIBnE4nhUJBleqRi74lk0lSqRRjY2OMjIwwOztLJBKhsbERj8ejihDK0aiMu/h8Ptxud03OW6h1pGKRFm1vby99fX00NTURDAaVlZHP54lGoywvL7O0tMTa2pqasC29H5Xz3WTcN5VKqdfVTs0qlp1GBzI47/f78fv9KkD/N//m3+T06dNKocgLn8vlSCQSXLt2jcXFRV555RXGx8dVAHY795pm7yPLb8hS5ocPH6a1tZVHH31ULcxULBbJZDKq9tfi4iIrKytsbGwQj8d54YUXGB8fV51HQ0ODSj2VRSqli83hcKiaY5rqwpxa3NbWRktLC5/+9KdVqSi5KGA2myWRSDA8PMzrr7/OhQsXmJ2dJRQK0dzcTFNTEz6fTykXKMlHOp0mlUoxNTVFJpMhkUjczdPdFWpSscjMHfMFlCtGypLn8kL39fXR2tpKR0eHcn9IhZHNZpmfn2d1dZUXX3yR5eVlbty4wcrKippBvZ/M21rC7Xbj9/s5evQora2tnDhxgvr6etrb2wmFQluKjSaTSebm5lSKeiKRUMkgMnlDBv2bmprUGj+AygKSyArJZlmT1IJvvdaozAI7fPgwXV1dSm5klQWpIGTpnhs3bpDL5ejq6qK7u1v1NeZSQHK/cvDi8/mIRqPMz8+TyWSqOrmj5hSLDMDa7Xbcbre6gB6PR01gki6vwcFBenp6aG5u3mKh5HI5UqkU8XicV155hdHRUf74j/+YSCSy73yltYhUAi0tLXzqU59icHCQs2fPquKk8E4GkN1uJxKJcPXqVZ599lmee+65LW2k3DgcDrWAnOxAZCq6eYDjdDppampifX19SzaiZm9ijps4HA4+/vGPc/ToUR555BGVpGFO5hgeHubixYs8++yzdHR08OCDD3L69GkOHjzImTNn6Ojo2LJ/6WpfXV3la1/7GrOzs7zxxhusra2RSCSqNkOsphSL1WrF6/Vy+vRp6urq6O3tBUrKwu12U19fT2NjowrQNjU14fV6t4wopftjbGyM5eVlXnvttS2zZbVSqX6EEMq6qK+vJxgMbinRIZVKPp8nk8mQzWa3xOjMZV7kfIR8Ps/i4iJut5u5uTlyuRz19fVbqjUAhEIhjhw5QiKRYHJykkQiQTKZ3LIvLWN7F1lxQyLlIZPJKLd4Z2cnjz/+OO3t7XR0dCiviN/v39LXmDPMAoEAZ86coaurC6/Xy8TEBNFoVFXKrjZqRrHICxQKhXj88cfp6+vjox/9KEIIotHolglq5gW+zAFWQM1XuXTpEiMjI3z3u99VEyl1WnFtYLFYaGpqUgMM6Z6QCkIG22V8TV5785wnmQ1oDtqOj4+TTqcZGRkhk8nQ0NCgsgqlzDQ1NfHggw8Si8V4++23KRQKJJPJLdaPZm8i+4Z0Oq2sTDlPKZlMks1msVgsHDx4kOPHj9Pe3k5raysejwen0wlsTW+XWK1WgsEgjz/+OJubmxw8eJA333yToaEh1tbWqtJLUjOKxWq10tnZSWdnJ/39/XR1dREIBJTCkTV9HA7Hlu0q6z5JH/iBAwdwOp28/fbbzM/PMzo6qioYa6oXaY34/X5CoRBOp1Mla8hCk5ubm8zMzDA9Pc3rr7/OwsICMzMzTE5OqjV3dkoQMU+kNJPP50kmk4yPj/Pss89y7do11tfXSaVSALfcp+bus90EWXhnuoLL5aK+vp7BwUElR3LtJ/PgwlxxQSIHrDLJo6mpScVxJiYmVHKIVGbVMLitasVSGViTZmdvby/t7e3K9JSxFrOvu3I/8qI7HA5sNhuDg4MEAgEOHz6MzWZjYmJCu8JqACkHPp+PYDCIy+VSyR7yu2QyyfXr13n99df5oz/6oy3pxuZ4iHmim3n/2ymWQqFAPB5namqK559/nrm5OcLh8LZl9zV7E/PcOPPseSEELpdLKRfZRrY3lwCS6cTm621eAt1ut9PS0kIkEuHw4cNkMhmuXbsGvOOmrQaqWrGYXVjmEad5vXrZWch1pldWVpiZmVEmrczuOnr0KC0tLQQCAWw2Gw0NDdjtdu6//37q6up466232NjYqEqzdL9i7vSlfBw5coT29nbOnTun6oHJgUcul2NtbY2JiQneeOMNhoeHSSaTWzqDyv3LTsNms9Hb28vAwAB9fX20tLQAkM1mSSaTzMzM8Oyzz3LlyhXGxsZUCZjK49TsXcyKoTKobo6dVFavltu8/vrrTE1NMTs7q+ImNpuN5uZmmpub+dSnPqXS0aX1IvuyaqOqFQvc7KuUN2llZyAnIC0uLnL16lXi8TixWEwVofT5fHg8HpVRJks0HDp0CACfz0cqlSKZTN6V89S8eyo7bKvVSm9vLwcOHODEiRPKXSpdYfl8no2NDZaWltTER7ke+Xb7Ng9qrFYrTU1NqiSQHLlms1ni8Thzc3O8/PLLjI+Ps7CwcJMrRFMd3Mqy3M5SNU9duHbtGm+++SaXL18mHA4r5TE4OMjAwAAf+9jHlGKRMiWTP6rFUpFUvWKR5PN5JiYmVLE36eqQcwVkYbjl5WWmp6fJZDJkMhlyuRzFYpG6ujpSqRRnzpyhqakJp9OpTFxZM6qyPLqmepDW7IEDBzh79iwdHR00NDRsuaZSRszWrHmuSaUyqKxOK61luXbP5uYm8/PzfOMb32BiYoLz58+rsuma6kAmbchAvJxQLeMm5nawVQFMTk4yOTnJ3NwcKysrakLtysoK6XQaQK1OWl9fr+IomUyGVCpFLBYjmUze5IKtBmpGsRQKBdbX11UWjywGCO+s8lcsFonFYoTD4S0+c8MwmJ6eVmtWh0IhVXyw0rWmqR4qYx+yZpdcY8Xtdt90w5rnJZi33W6fZswjTGk5ZzIZ1tfXuXjxIrOzs8zOzt60X61k9jbS3R4KhWhsbKS+vn7LBNqdtikUCqysrDA8PMyNGzeYmZnh+vXrLC0tqawyaSXLxQKlNSQtnGQyqUrtV5uc1IxiMQxDzYhOJBKqcGBlG1nvyXyxZEry6uqqWrhLdi7SspHZQJrqRF77mZkZ6uvrVeFAOQMeSpk9vb295PN57rvvPq5du8b4+PhNo1GpPHw+n1ppUpbJ9/v9AKTT6S21oszZZJrqQghBQ0MDLS0tajXRyuC9tGwzmQwzMzPcuHGDF198URUojcVixONxstkshUIBi8VCZ2cnTU1NfPazn2VwcFC5wbLZLGtra1y+fJn5+XkVB64makaxAGqCUmVJ8tthsVjUxCfzGgnwTqVS0L7wakZe083NTaLRqIqtVVo1LpcLj8dDKBRSNaDM3wOqiKnf71cxGrfbTTAYVJ2DLEZoDtJrqhcpF5VLKZgHHcVikVwux8bGBhMTE4yNjTE2NkYikSCdTqsYjMz+knOpDh8+THd3N3a7nWKxSDKZJB6Ps7q6SiwW27IgXLX0QVWlWMwT0rb7g831nWT7yu3NbjHzPurq6mhrayMQCGyp4+RwOFQuuo6xVC/m6rQyMUOWwQdU8b+JiQkmJiYYGhpibm5uW0tVxmvOnDnDoUOHaGpqIhAIcPDgQVV8cmNjg9dff53FxUVVmt88ytVUD9IlLuOslUUkzSX1M5kMCwsLyv2ZSqUIBAK0tLTgdrux2+3K6nnkkUdU2ZdgMEihUGB5eZnvf//7XLlyRRWlrBZlYqaqFMutMGfobPcsX5snIxWLRTX6lMF+p9O5ZW0FiXZjVD9CCLXMsHmJYUCNNsPhMGtra2q0aJYX+ZCLgMlgbnt7O16vl5aWFpxOJ9lsllgsxsLCAsvLyyQSCbLZ7LYypakOzGnGt1pS2pyR6nA48Pv9qmyQdJ22trYSDAYZHByktbVVxXQ3NjZYXV1ldHSUubk5UqnUlhhgNclLVSmWys7dPGowf3e7iUTSEnG73aoS7ZEjRzhy5Ah1dXUqI8wwDGWWptNp1TlU0wXWlJDlWAYGBjh58iT19fW4XK4tM6FTqRTj4+MMDQ1x4cIF0um0KnUfCATw+/0Eg0EaGxsJhUKcO3eOY8eO0djYqKrcZjIZLl26xPj4OM8//zxra2tsbGyo2J6m+jAMg/X1dbxeLxsbGyqLy5xeLCdpu1wu+vv7eeKJJ1hdXWVtbY3m5mZCoRA+n099HwqFVKwml8uxsrLCM888w8jICH/6p39KPB6vOveXmT2rWGSWjTkjS2Zy3YkrYbvZ0ObZ1X6/H6/XS3d3Ny0tLWrkIK2VXC6n1riXPnm99kp1UXlTSiWy3cRE6d7yeDx4vV7lEi0UCsqVIYuYyhFoS0uLisU4HA6y2Sy5XI7l5WUWFxfZ2NhQKaPV2DloShiGQSwWIxKJsLm5ueV6Vs5dsVqtBAIBOjs71eJx9fX1qk6h0+ncUhUkn88reRkZGWF8fJz19fUthU+rkT2pWORNLm9eWS5BLrSUSqW29X1vlxZq9o/KdTA8Hg8HDhxgYGCAhx9+mGPHjtHW1obf71ez9FdXV1lfX1cLe8ky1nKf1XzR9yOywOTY2JhSFA6HQ1mncgb0E088weHDhwmFQipTsKenh6NHj9LX10dfX9+WpYrl4KdYLKpVA5955hkmJiZYXV0lnU6r9TqqbS6C5p2g/Pj4OOFwmBs3biCEYHBwcMvcNpkd5nQ66e7upqOjQ7nEZCzPXAlECMHm5iaRSIS/+qu/YnR0lK985SvEYjFl2VZzH7MnFYvVasXpdFJfX09/fz+BQIBQKITf78ftdrO2tsbm5qbK4pJuiO2Kvcnlh30+H21tbep1f38/fX19aj0WuaSszOCYmppiaWmJmZkZlpaWlCLTc1mqF8MwlCVamREmJ8L5fD6ampo4dOgQ8XiclpYW2tra6O7uVi4Nidn6KRQKRKNRwuEwKysrhMPh2xas1FQHUm4ymQybm5tsbm6STqe3lFsx14iTc98qkS5XmX6+sLDA6uoqIyMjTE5OEovFSKVSNTFw3TOKxeyrdDgcNDQ0cOrUKZ566il6e3vp7+/nwoULqjjg5OSkmmzU29urJrzJoLws4eLxeGhvb6evr4+HH35YuTmkiSpHn1J4JiYmWFpa4lvf+hYzMzNcvnyZeDyuSntU+wXfr1ROPpOznOVgQVoTXq8Xj8dDZ2cn8M5IVMqnLKtvTgQxy874+Djj4+MsLS2pfVbjBDfNVqTlsrq6SigUYmVlhWw2q2oLVlZN345sNks+nyeRSLC5ucnTTz/N2NgYf/Znf8bGxoaKDddC1uCeUSxmZNqeLLlis9kIBAJ0dHSoIGhLS4vKnJCViD0ej1IscnKj0+mksbGR1tZWZbFIq8XpdKq2S0tLRCIRNSlJdg5ydKKpbmSMze12q7knlYtwSaQr1lzKpfJhrmy8vLzMxsYG169fZ3Jykmg0Sjqd1pZKDWD2UhQKBaanp8nn8zQ1NdHU1MSRI0dULOV23oxoNEosFlOu9bfeeou5uTnlfaklb8ieUSzmfHBZ02txcZH5+Xl6e3vx+XwcO3aMgwcP8sADD5BKpZieniaRSHDo0CECgYCa0FZ5Q0urRBYbNFellaPYN998k7fffpunn36a6elp1tbWlKtNttNUL9K33dbWRk9PD36/H7vdrmSl8qY2WxmVsiRLAUlL59VXX2VkZIQ///M/Z35+Xo0+NbWBlI90Os2PfvQjXC4Xr732Gn19ffzyL/8y3d3dKkvsVvuYnJxkYmKCr33ta4yMjLCwsHBTOZdaYc8oFol0S8kSLdIPGYlEcDgcKghvt9tpbW0lm80SCoVwu92qvEalKVmpTAA1K1r6w998801GR0dZWlpSATTtwqhuKmMo+Xye0dFR5WptbW1Vqzxul8pufsh05GQySTQaJZFIsLq6SjQaVctXy4W7asGVodmKlA1Z22t9fR2r1cpLL71Ec3MzU1NTt60fNjw8rOK24XBYFTqtRfacYpFav1gssrGxwdDQEN3d3SwsLNDS0qIq0haLRfx+P8ViUWVn3KkpKd0XIyMjvPnmm4yMjHD16lUWFxfV2vaa6sesKGQ85YUXXmBoaIj6+noOHTqEz+dTWVsyNicHIeay5YCa9Dg1NcXi4iKvvvoqk5OTzMzMqMCrtlRqFzn4yGazLC0tsb6+zuLiIg6HQy13vt1AVPZLm5ubZDIZYrHYuy47VW3sKcVSmUGTSqVYWVlhYmKCK1eucOTIEVU4ULq1ZDbPdq6MXC5HMplkcXFRVQuNx+OEw2Hm5uaYn59nbGyM+fl51tbWdMdQ4xiGQTQapVgs8sYbb7C6ukogEKC5uZnGxkaV5SPjblJOpMKZnZ1lZmaG+fl5VlZWGBkZYXV1VRUX1Nbt/kKudS/jwbdDBu9r1Uoxs6cUC2yto5RMJllaWmJ4eBiPx6PWLmhublZur50wDEOtGHnx4kVisRirq6vMzs4yPDxMPB4nHo+zvr6u18jYJxiGQSQSIRaL8dxzzzEyMsLAwAC5XE5ZwnIeUyqVYmlpieeff14VKJ2cnGRqaorl5WUikQiRSGRLYofZbaapfYrFoprbptnKnlMs5psyn8+TTCaZm5sDShp/ampKLSG807Zy5nwkEmF9fZ1r166RSCSIRCJsbGyoVMFMJqOydyqrlWpqm/X1dfL5PD/60Y8YGhpiYmJCDVZyuRyxWIyZmRnOnz+v1rwPh8OEw2G1ToZMPTaj5Wd/sV1Nwp3YLhmkVhF3+ySFELc9AJki3N7eTnNzs6pCDNtXipWdgywkOD4+TiaTUS6L7fybH6ZiMQyjdvIK7wJ3IjO32BYoXWe59HQoFGJwcBCn04ndbieTyRAOh1laWuL69etqzoss2WLel7lD+SCD9lpm3jvvR15us98tj9tRmbL+QXK35aUqFIucVW8uySInJO2kWIrFoiocGY/HtxQBrJyfYP7sw+BuX/RqZzc6ChmgDwQCqgqtjLGYl4fd2NjYMnflVnLyQcqQlpn3zgelWMr7ftfb7IfBa1Uolt1EZvtsN+ntw+JuX/RqZzdk5r2sjVLp9vgw5UbLzHvnw+5j9gJ3W16qRrFUzkXZLdPzbpz/3b7o1c5udRRmN8Z2y1jfSnY+bLnRMvPe0Yrlw2fPBe9vhblC6J0oFvOseXO2zt1Wppq9gUxVl4/K72TBQC07Gs27o2oUi7zJ3+u2Gs12mCsUb4eWHY3m3VM1igX0Ta7ZfbRMaTS7z84zDDUajUajeQ9oxaLRaDSaXeWuZ4VpNBqNprbQFotGo9FodhWtWDQajUazq2jFotFoNJpdRSsWjUaj0ewqWrFoNBqNZlfRikWj0Wg0u8r/BZA7KU74Jf5NAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 9 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from fastai.data.core import DataLoaders\n",
    "dls = DataLoaders.from_dsets(train_ds, valid_ds).cuda()\n",
    "dls.show_batch = train_ds.show_batch\n",
    "\n",
    "# We have 10 classes! One for each digit label!\n",
    "dls.c = 10\n",
    "\n",
    "dls.show_batch()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yqHZW3_rTryB"
   },
   "source": [
    "# FastAI Vision Learner [Transfer-Classification]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Rm3lBPctTryB"
   },
   "source": [
    "Here we will toggle `pretrained=True` to transfer our rotation prediction features, and train on the original 180 labeled data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:32:23.198800Z",
     "start_time": "2021-03-10T23:32:23.189837Z"
    },
    "id": "tg_O-5MFTryB",
    "outputId": "46b1947f-6c2f-4534-8e71-be6f8e149a2f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): AdaptiveConcatPool2d(\n",
       "    (ap): AdaptiveAvgPool2d(output_size=1)\n",
       "    (mp): AdaptiveMaxPool2d(output_size=1)\n",
       "  )\n",
       "  (1): Flatten(full=False)\n",
       "  (2): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  (3): Dropout(p=0.5, inplace=False)\n",
       "  (4): Linear(in_features=64, out_features=10, bias=False)\n",
       ")"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#collapse-show\n",
    "classification_head = create_head(nf=32, n_out=10, lin_ftrs=[])\n",
    "classification_head"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QIYC_t6fTryB"
   },
   "source": [
    "> Note: We have `n_out=10` because of the 10 different digit classes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:32:23.217156Z",
     "start_time": "2021-03-10T23:32:23.201091Z"
    },
    "id": "J_ugDkRuTryB",
    "outputId": "b47b8b43-d62c-4aae-a88b-3e3ea8999e9b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading pretrained model...\n",
      "<All keys matched successfully>\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Sequential(\n",
       "    (0): Conv2d(1, 4, kernel_size=(3, 3), stride=(1, 1))\n",
       "    (1): BatchNorm2d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (2): ReLU()\n",
       "    (3): Conv2d(4, 16, kernel_size=(3, 3), stride=(1, 1))\n",
       "    (4): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (5): ReLU()\n",
       "    (6): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1))\n",
       "    (7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  )\n",
       "  (1): Sequential(\n",
       "    (0): AdaptiveConcatPool2d(\n",
       "      (ap): AdaptiveAvgPool2d(output_size=1)\n",
       "      (mp): AdaptiveMaxPool2d(output_size=1)\n",
       "    )\n",
       "    (1): Flatten(full=False)\n",
       "    (2): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (3): Dropout(p=0.5, inplace=False)\n",
       "    (4): Linear(in_features=64, out_features=10, bias=False)\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#collapse-show\n",
    "# pretrained=True will load the saved rotation pretraining weights into our model's body!\n",
    "# See simple_arch() function definition for more details!\n",
    "learner = cnn_learner(dls,\n",
    "                      simple_arch,\n",
    "                      pretrained=True,\n",
    "                      loss_func=CrossEntropyLossFlat(),\n",
    "                      custom_head=classification_head,\n",
    "                      metrics=[accuracy])\n",
    "\n",
    "learner.model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LxODfK_YTryB"
   },
   "source": [
    "> Tip: Freezing a model's body after transferring the weights over, allows the new head to get *calibrated* with the rest of the model!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:32:23.221977Z",
     "start_time": "2021-03-10T23:32:23.218810Z"
    },
    "id": "ppd7iY_QTryC"
   },
   "outputs": [],
   "source": [
    "learner.freeze()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Gw9vh3U1TryC"
   },
   "source": [
    "> Note: Looking at the model summary, we can see that the model is frozen up to the new head! Good!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:32:23.254536Z",
     "start_time": "2021-03-10T23:32:23.223686Z"
    },
    "id": "TdN3oCv4TryC",
    "outputId": "fce2c4ea-2dd0-476f-aaf4-3f62240ab0f5",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "Sequential (Input shape: 64 x 1 x 32 x 32)\n",
       "============================================================================\n",
       "Layer (type)         Output Shape         Param #    Trainable \n",
       "============================================================================\n",
       "                     64 x 4 x 30 x 30    \n",
       "Conv2d                                    40         False     \n",
       "BatchNorm2d                               8          True      \n",
       "ReLU                                                           \n",
       "____________________________________________________________________________\n",
       "                     64 x 16 x 28 x 28   \n",
       "Conv2d                                    592        False     \n",
       "BatchNorm2d                               32         True      \n",
       "ReLU                                                           \n",
       "____________________________________________________________________________\n",
       "                     64 x 32 x 26 x 26   \n",
       "Conv2d                                    4640       False     \n",
       "BatchNorm2d                               64         True      \n",
       "____________________________________________________________________________\n",
       "                     64 x 32 x 1 x 1     \n",
       "AdaptiveAvgPool2d                                              \n",
       "AdaptiveMaxPool2d                                              \n",
       "____________________________________________________________________________\n",
       "                     64 x 64             \n",
       "Flatten                                                        \n",
       "BatchNorm1d                               128        True      \n",
       "Dropout                                                        \n",
       "____________________________________________________________________________\n",
       "                     64 x 10             \n",
       "Linear                                    640        True      \n",
       "____________________________________________________________________________\n",
       "\n",
       "Total params: 6,144\n",
       "Total trainable params: 872\n",
       "Total non-trainable params: 5,272\n",
       "\n",
       "Optimizer used: <function Adam at 0x000001BF07FB7670>\n",
       "Loss function: FlattenedLoss of CrossEntropyLoss()\n",
       "\n",
       "Model frozen up to parameter group #1\n",
       "\n",
       "Callbacks:\n",
       "  - TrainEvalCallback\n",
       "  - Recorder\n",
       "  - ProgressCallback"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learner.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:32:45.606872Z",
     "start_time": "2021-03-10T23:32:23.256375Z"
    },
    "id": "aukCo9HCTryC",
    "outputId": "497b6278-80df-4202-d960-09fbef10d650",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "SuggestedLRs(valley=0.005248074419796467)"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEOCAYAAACEiBAqAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAxfklEQVR4nO3dd3yV9dnH8c+Vk5OcJCRhJCSQQQgbZMeAUhRxUQdaFRyode9qtUufPo+1dtfWtuJorbZaN+Io4kZAVGQEZO9NWEkYScge1/PHOWiImST3OUnO9X698vKce34TQ67zu3/3/fuJqmKMMSZ4hQQ6gDHGmMCyQmCMMUHOCoExxgQ5KwTGGBPkrBAYY0yQs0JgjDFBzvFCICIuEflKRGY3sM2lIqIikuF0HmOMMcfzR4vgHmB9fStFJNq3zWI/ZDHGGFOLo4VARJKB84FnGtjsV8AfgFInsxhjjKmb0y2CvwI/BarrWikio4AUVX3X4RzGGGPqEerUgUXkAiBHVZeJyIQ61ocAjwLXNeFYtwC3AERFRY0eOHBgq2Y1xpiObtmyZXmqGl/XOnFqrCER+R1wDVAJeIAY4E1Vvdq3PhbYChz17ZIIHAImq2pWfcfNyMjQrKx6VxtjjKmDiCxT1TpvyHHs0pCqPqCqyaqaBlwBzD1WBHzr81U1TlXTfNssopEiYIwxpvX5/TkCEXlYRCb7+7zGGGPq5lgfQU2qOh+Y73v9YD3bTPBHFmOMMcfzSyEwxhh/qKioIDs7m9LS4L0b3ePxkJycjNvtbvI+VgiMMR1GdnY20dHRpKWlISKBjuN3qsrBgwfJzs6md+/eTd7PxhoyxnQYpaWldOvWLSiLAICI0K1bt2a3iIK+EBwuKmfJ9kMcKAjepqQxHUmwFoFjTuT7D8pLQ6UVVfxk5iq+3JpH3tFyAEb36sIbt58a4GTGmGDSqVMnjh49yo4dO7jgggtYs2ZNQHIEZSGYuyGHd1bu5fxhPRiR3JmNBwp5Y3k2OYWldI/2BDpem6eqfLh2P6UV1Uwc1J0YT9M7pYxpU1bNgE8ehvxsiE2GMx+EYVMDncrvgrIQvLtqH3GdwnjsipG4QoR1ewuYuSybeRtyuPzk1EDHa9MOFZXzszdW8fG6AwCEuUIY3y+O741K4vyhPQLSLK+squaFRTuZPncL3aPDOXtwAmcNSmBoUiwhIcF9mcA0YNUMeOduqCjxvs/f7X0PJ1wM7r//flJSUrjzzjsBeOihhwgNDWXevHkcPnyYiooKfv3rX3PRRRfVe4yqqiruv/9+5s+fT1lZGXfeeSe33nor1157LZdccgkXX3wxANOmTWPq1KkNHqupgq4QFJdX8smGA0wZnYLL90diUI9okjpHMGe9FYKaisoq2ZJzlFCXEOYKYXteET9/ew35xRX87/mDGJnahfdX7+P9Nfu56+WveH/Yfn5/yVCi62kh7DxYxOxV+9iac5StuUfZeagYlwiR4S4i3aF0jwlnQEI0/ROi6R4Tzp4jJew6WMyeIyWkx0UxJr0bo1K7EBHm+vqYy3Ye4n/fXsv6fQWc2qcbldXKE/O2MH3uFsJCQ4jvFE5cdDgpXSK4KjOVU/oEb0eiqeWTh78pAsdUlHiXn2AhuPzyy/nhD3/4dSGYMWMGH374IXfffTcxMTHk5eUxduxYJk+eXO/v4bPPPktsbCxLly6lrKyMcePGcc4553DjjTfyl7/8hYsvvpj8/HwWLlzI888/f0I5awu6QvDJ+hxKK6o5f1iPr5eJCGcO6s6MrN2UVlThcbsaOELboaos3XGY7tHhpMVFtdoxl2w/xOvLsnlv9T6Ky6uOW9+veyeevz6TwT1jAG/fyv+cN4h/LNjGnz7ayNo9+TwxbRRDesZ+vU9uYRnT527m5cW7qKxWesR6SI+P4ryhPRCguLyKorJK9uaX8OLinZRWfDNYbZgrhITYcN5bvY/H5m7B7RLiOoVTXF5FSXkV5VXV9Ij18NS0UUw6KRER4XBROXM35LDxQCF5hWXkHi3jy60Hmb1qH8OTY7l9Qh/OGZxorYVgl5/dvOVNMHLkSHJycti7dy+5ubl06dKFxMRE7r33XhYsWEBISAh79uzhwIEDJCYm1nmMjz76iFWrVjFz5kxvnPx8Nm/ezDnnnMMdd9xBbm4ub7zxBpdeeimhoa3zJzzoCsG7q/bRPTqck9O6Hrf8zEEJ/OfLnSzcmsfEgQkBStc0VdXK+2v28fjcLWzYXwjA+H5xXDO2FxMHdifU1fybwcoqq3hr+R6e/mwb23KLiApzceGwnpwxsDsiUFFVjUuEMwZ2/1ahDAkRbp/Qh9G9uvCDV5bzvScWkhYXSefIMGI8bhZuzaOssporTk7h7jP7kRBTfz9MVbWy+1AxuUfLSOocQWKMh5AQoaC0gmU7DrNo+0EOHi0nMsxFRJiLhGgPV2SmEBn2za9yl6gwLh2dfNxxSyuqeGN5Nv/4dBu3vbicQT1ieOC7Azmtf52DMZpgEJvsvRxU1/IWmDJlCjNnzmT//v1cfvnlvPTSS+Tm5rJs2TLcbjdpaWkN3t6pqkyfPp1zzz33W+uuvfZaXnzxRV599VX+/e9/tyhnTUFVCI6WVTJvYw5XZqZ+fVnomLHpXYkKczFnfY5fC4GqNutSxcrdR7h3xgq25RbRJz6KP142jAP5pby8ZBe3vLCMHrEepoxOZkpGCildI+s8RlllFRv3F5JfUkF+SQXbcot4YdFOcgvLGNIzhj9NGc55QxOP++PaFJm9u/Le3eN5Yt5W9hwp5nBxBTsPFnHmoATuPasf6fGdGj2GK0RIi4v6VgsnxuPmjIHdOWNg92ZlOsbjdjFtTC+uODmV2av28qePNnLtv5Ywvl8c08akEu524RLB43YxICGa2MjjL2+pKqpYK6IjOfPB4/sIANwR3uUtcPnll3PzzTeTl5fHp59+yowZM+jevTtut5t58+axc+fOBvc/99xzeeqpp5g4cSJut5tNmzaRlJREVFQU1113HZmZmSQmJjJ48OAW5awpqArBJ+sPUFZ5/GWhY8JDXZzWP55P1h9ALz6p1a4jV1ZVszL7CCNSuhxXfHYfKubHr68ka+dhukS66RwZRo9YDzeM682EAfF1nv/dVfu4b8YK4qPDeXLaKM4dkvj1MW+f0Ic56w/w8pLdTJ+3henztvCdvnFcd2oaZwzoTkiIoKrMWZ/Dw7PXsvvQ8ddGv9M3jkenDuc7feNa9L136xTOgxe23i9oa3OFCBeNSGLSSYm8uGgX0+du5rYXl39ru5SuEZzUM5byymp2HSom+3AJldXVJHeJJKVrJOlxUZw7JJExvbtacWivjvUDtPJdQ0OGDKGwsJCkpCR69OjBtGnTuPDCCxk6dCgZGRk0Np/KTTfdxI4dOxg1ahSqSnx8PG+//TYACQkJDBo06OsO49bi2HwETmnJfAQ3/yeL1dn5LLx/Yp3/eGcuy+bHr6/knbu+w9Dk2DqO0Hz/9/YaXli0k/S4KO6a2JfJw3vy9oq9PDRrLQJMPTmFkooqDheVs3pPPtmHS8js3ZWfTRrI6F5dAO+n0SfmbeFPH20io1cX/nHNaLp1Cq/3nNmHi3k9K5vXlu5mf0Ep6XFRXD22F59tzmXexlz6de/EXRP70rNzBLERbrpEhhEfXf/xOrKjZZVsyz1KVbVSrUphaSXr9hWwZk8+6/YW4HG7SO3q/ePvdoWw+1AxOw8VsTWniJKKKnrEepg8oifTMnuR2q3uFpjxn/Xr1zNo0KBAx3BMcXExQ4cOZfny5cTG1v83qq6fQ0PzEQRNi6CwtIJPN+Zy9dhe9X6CO2NAPCECc9YfaJVC8P7qfbywaCfnDU1kW24R981YycOz13GkuILM3l15dOpwkrt888ejvLKa15bu4m+fbOHSpxYSHhpCaIgQIkJhWSXfG5nE7y4Z2mhndnKXSO49uz93TezLe6v38a/Pt/Pw7HV0Cg/lf88fxPdPTcN9Av0IHVGn8FCGJXc+btmEAY1ffiopr+Lj9Qd4+6s9PPPZdp75bDuXjEzirol96dWtdTrujalpzpw53Hjjjdx7770NFoETETQtgjeXZ3PfjJW8cfupX3/SrstlTy2kpKKKd+8e35KY7D5UzHmPfUZ6fCdev/UUQkOEOesP8NrS3WT27spN49O/1U9xTFFZJTOyvJ/mq6qUymqlf0I0V2amnNBlG1Vl44FC4jqFE9dAS8KcmAMFpfz9061f3xV16agkfjCxX719NMY5Hb1F0FTWIqhHbISbSUMSGZXaucHtzhqcwO/f38COvKITviWzvLKau175CoDHrxxJWKj30/c5QxI5Z0jdt4zVFBUeyvXjmj5yYGNEhIGJMa12PHO8hBgPv7hwCLef3oenPt3KS4t38dZXe7ji5FTuPKMvHncIuYVl5PnudkruEkHXqDB7nsG0GUFTCM4clMCZgxq/G+h7I5N45MONvLJkFw+c17xPFpVV1Xy2JY//LNzByt1HeHLaKPtUGES6+wrCLaelM33uFl5ZsosXFtV9h0iE20VCTDhR4aFEhYXSyRNKv+6dGNwzhiE9Y0iP62Sd0CeouXfidTQncpXH8UIgIi4gC9ijqhfUWncfcBPeCe5zgRtUteF7qxyWEOPh7EEJzMjazX3n9Cc8tP7r8dmHi9mSc5TteUVsOlDIR2sPcLConBhPKD85dwDnDf323Umm4+sRG8FvvzeU207rw39X7CEyPJT46HDiosIoKq8i+7D3LqScwjJKyispKqtiX34pn2/Oo7zK+zBdatdIrj2lF1MyUoiNsLGcmsrj8XDw4MGgHYr62HwEHk/zxkxzvI/A98c+A4ipoxCcASxW1WIRuR2YoKqXN3S8ltw11FSfb87j6mcX89fLR3DxyKQ6t/nLx5v42yebv34fHR7K+P5xXDQiiQkD4hssIMbUpaKqmi05R1mVfYSZy7JZuuMwkWEupmZ4H8TrGhUW6Ihtns1QVv8MZQ31EThaCEQkGXge+A1wX+1CUGvbkcDjqjquoWP6oxBUVysT/zyf+OhwXr/t20NTz1i6m5++sYqLRvTk6rG96B0XRTe75mta2Zo9+fz7ix28vWIPUWEufnTOAKaNST2hJ8eNaagQOP0b9Vfgp0B1I9sB3Ai872iaJgoJEa4ak8rSHYfZsL/guHULNuXywFurGd8vjj9NGc7JaV2J6xRuRcC0upOSYvnz1OG8f894hibH8otZazn/sc959vPt7DxYFOh4pgNxrBCIyAVAjqoua8K2V+O9fPRIPetvEZEsEcnKzc1t5aR1u2x0CmGhIby8eNfXy1ZlH+GOl5bTr3snnpw2yu7FN37RPyGaF28cw9+vHg3Ar2av4/RH5nPmn+fzxLwtFJRWBDihae8cuzQkIr8DrsHbEewBYoA3VfXqWtudBUwHTlfVnMaO649LQ8fc+9oKPl53gAfOG8jbX+1h6Y7DJMZ4eOvOU+kRG+GXDMbUtutgMZ9sOMBHaw/w5baDxHhCuW5cb24Yl0bnSOtHMHULWB9BjQATgB/X0Vk8EpgJTFLVzXXs+i3+LATLdh7i0qe+BKBPfBSXjEpmSkayzWJm2ow1e/KZPnczH649QOdIN09eNYpT+8YFOpZpg9pUIRCRh4EsVZ0lInOAocA+36a7VHVyQ8fyZyFQVd5dvY+ULpEMS461fgDTZq3fV8A9r37F1twiHrpwMNeckhboSKaNCXghaE3+LATGtCeFpRX88NUVfLIhh6vHpvLz8wYfN5ubCW6BvGvIGOMn0R43T1+bwW2n9+HFRbsY+7tP+M2769h1sDjQ0UwbZy0CYzqgpTsO8dzCHXywZj/VqmT06sKpfeI4pU83RqZ2tgceg5BdGjImSO3PL+WVJbuYuyGHtXvzqVboGhXGE1eN4pQ+3QIdz/iRFQJjDPklFSzedpA/friRHXlFvJC5k1O2P9Gqs3OZtsuGoTbGEBvh5pwhiYxJ78ZL/3yE4V/9BaTcuzJ/t3f+XrBiEISss9iYIBMb4eb2qpeIPFYEjqko8c7fa4KOFQJjgpDk76l7RX62f4OYNsEKgTHBKDa5ectNh2aFwJhgdOaD4D5+vKxiDWPrsPsCFMgEkhUCY4LRsKlw4WMQmwIIVTHJPOq5i6sWp5JfbKOZBhu7a8iYYDVs6td3CLmAi/fk89wTX/CLWWv46xUjA5vN+JW1CIwxgHcinB9M7MfbK/by/up9je9gOgwrBMaYr91xRh+GJsXyP2+tJrewLNBxjJ9YITDGfM3tCuHRqcMpKq/igTdX095GHjAnxgqBMeY4/RKi+dmkgcxZf4C/zmnSfFGmnbPOYmPMt9wwLo0N+wr42yebSeoSwdSMlEBHMg6yQmCM+RYR4beXDGV/QSn/8+ZqEmM8nNY/PtCxjEMcvzQkIi4R+UpEZtexLlxEXhORLSKyWETSnM5jjGkatyuEJ6eNom/3Ttzx0nI27i8MdCTjEH/0EdwDrK9n3Y3AYVXtC/wF+IMf8hhjmija4+a56zPxuF3c8+pXlFdWBzqScYCjhUBEkoHzgWfq2eQi4Hnf65nAmWIzxBvTpiTGevj9JUPZsL+Qv32yKdBxjAOcbhH8FfgpUN/HiCRgN4CqVgL5wLemTRKRW0QkS0SycnNzHYpqjKnPWYMTmDI6mafmb2X5rsOBjmNamWOFQEQuAHJUdVlLj6WqT6tqhqpmxMdbh5UxgfDghYPpERvBj2espKS8KtBxTCtyskUwDpgsIjuAV4GJIvJirW32ACkAIhIKxAIHHcxkjDlB0R43j1w2jG15RTzy4cZAxzGtyLFCoKoPqGqyqqYBVwBzVfXqWpvNAr7ve32Zbxt7lNGYNurUvnFcPTaV5xZuZ93egkDHMa3E708Wi8jDIjLZ9/ZZoJuIbAHuA+73dx5jTPP8+JwBdI4M46FZa20Iig7CL4VAVeer6gW+1w+q6izf61JVnaKqfVU1U1W3+SOPMebEdY4M4yfnDmDJjkPMWrk30HFMK7CxhowxzTY1I4VhybH89r31HC2rDHQc00JWCIwxzeYKEX45eQgHCsqYPtcGpmvvrBAYY07IyNQuTBmdzL8+38623KOBjmNawAqBMeaE/XTSQMJcIfz2vQ2BjmJawAqBMeaExUeHc+fEvsxZf4AvtuQFOo45QVYIjDEtcsO43iR3ieBXs9dRVW23k7ZHVgiMMS3icbt44LuD2LC/kBlZuwMdx5wAKwTGmBY7b2giJ6d14c8fbaSwtCLQcUwzWSEwxrSYiPDgBUPIO1rOE/O2BjqOaSYrBMaYVjE0OZZLRiXxry+2s/tQcaDjmGawQmCMaTU/OXcAIQJ/tNFJ2xUrBMaYVtMjNoKbx6fzzsq9fGUT2LQbVgiMMa3qttP7EB8dzq/fXW+jk7YTVgiMMa0qKjyUH53dn2U7D/Pe6v2BjmOawAqBMabVTclIYWBiNH/6aKM9ZNYOWCEwxrQ6V4hw18S+bM8r4uN1BwIdxzTCycnrPSKyRERWishaEfllHdukisg8EflKRFaJyHlO5THG+NekIYmkdI3g6QX2XEFb52SLoAyYqKrDgRHAJBEZW2ub/wVmqOpIvPMaP+lgHmOMH4W6QrjpO+ks33WErB2HAh3HNMDJyetVVY8NUu72fdW+WKhAjO91LGDz3hnTgUzJSKZzpJt/LLBZaNsyR/sIRMQlIiuAHOBjVV1ca5OHgKtFJBt4D/iBk3mMMf4VGRbKtaekMWf9Abba5DVtlqOFQFWrVHUEkAxkishJtTa5EnhOVZOB84AXRORbmUTkFhHJEpGs3NxcJyMbY1rZ90/pRZgrhGc+s1ZBW+WXu4ZU9QgwD5hUa9WNwAzfNl8CHiCujv2fVtUMVc2Ij493OK0xpjV16xTOZaOTeWPZHnIKSgMdx9TBybuG4kWks+91BHA2UHs+u13Amb5tBuEtBPaR35gO5pbT0qmsruaZz7cHOoqpg5Mtgh7APBFZBSzF20cwW0QeFpHJvm1+BNwsIiuBV4Dr1J5JN6bD6dUtiguH9+TFRTs5XFQe6DimllCnDqyqq4CRdSx/sMbrdcA4pzIYY9qOOyb05b8r9vLvhTu47+z+gY5jarAni40xfjEgMZpzhyTw3BfbbRazNsYKgTHGb+46ox8FpZW8uGhXoKOYGqwQGGP8ZmhyLKf1j+eZz7ZRUl4V6DjGxwqBMcav7jqjLweLynl92e5ARzE+VgiMMX6V2bsrw5JjeXnxLpu4po2wQmCM8bvLT05hw/5CVmbnBzqKwQqBMSYAJg/vSYTbxatLrNO4LbBCYIzxu2iPmwuG9WDWyr0cLasMdJygZ4XAGBMQV2SmUlxexeyVNvp8oFkhMMYExKjUzvTr3olXl9rdQ4FmhcAYExAiwhWZqazYfYQN+wsCHSeoWSEwxgTM90YmEeYK4dUl1ioIJCsExpiA6RoVxrknJfLm8mxKK+xJ40CxQmCMCagrTk6hoLSSD9bsD3SUoGWFwBgTUKekdyO1aySv2DMFAWOFwBgTUCEhwuUnp7B4+yG22QT3AWGFwBgTcFNGJ+MKEV7Lsk7jQHByzmKPiCwRkZUislZEflnPdlNFZJ1vm5edymOMabu6x3iYOLA7byzLpryyOtBxgk6TCoGIRIlIiO91fxGZLCLuRnYrAyaq6nBgBDBJRMbWOm4/4AFgnKoOAX7YzPzGmA7iyswU8o6WM3fDgUBHCTpNbREsADwikgR8BFwDPNfQDup17IKf2/dVe8zZm4EnVPWwb5+cJuYxxnQwp/fvTmKMh1fsmQK/a2ohEFUtBi4BnlTVKcCQRncScYnICiAH+FhVF9fapD/QX0S+EJFFIjKpGdmNMR2IK0SYmpHMgs25ZB8uDnScoNLkQiAipwDTgHd9y1yN7aSqVao6AkgGMkXkpFqbhAL9gAnAlcA/RaRzHSe/RUSyRCQrNze3iZGNMe3N5ZmpCNitpH7W1ELwQ7zX8t9S1bUikg7Ma+pJVPWIb/van/izgVmqWqGq24FNeAtD7f2fVtUMVc2Ij49v6mmNMe1MUucIJg5M4LWluymrtCeN/aVJhUBVP1XVyar6B1+ncZ6q3t3QPiISf+zTvYhEAGcDG2pt9jbe1gAiEof3UtG25nwDxpiO5ZpTepF3tNyeNPajpt419LKIxIhIFLAGWCciP2lktx7APBFZBSzF20cwW0QeFpHJvm0+BA6KyDq8LYafqOrBE/tWjDEdwfi+cfTqFslLi+zykL809dLQYFUtAC4G3gd6471zqF6qukpVR6rqMFU9SVUf9i1/UFVn+V6rqt6nqoNVdaiqvnri34oxpiMICRGuHtOLJTsO2fDUftLUQuD2PTdwMb5r+nz7VlBjjGkVl41OJiw0hBcX7Qx0lKDQ1ELwD2AHEAUsEJFegJVqY4wjukSFceGwnry1fI/NaewHTe0sfkxVk1T1PN/lnJ3AGQ5nM8YEsWtO6UVReRVvfbUn0FE6vKZ2FseKyKPH7uUXkT/jbR0YY4wjhifHMqRnDC8v3oWqXYl2UlMvDf0LKASm+r4KgH87FcoYY0SEq8aksn5fASuz8wMdp0NraiHoo6q/UNVtvq9fAulOBjPGmMnDexIZ5uLlxdZp7KSmFoISEfnOsTciMg4ocSaSMcZ4RXvcXDSiJ++s3EdBaUWg43RYTS0EtwFPiMgOEdkBPA7c6lgqY4zxuTIzlZKKKv67Ym+go3RYTb1raKVvXoFhwDBVHQlMdDSZMcYAQ5Os09hpzZqhTFULfE8YA9znQB5jjDmOdRo7ryVTVUqrpTDGmAZYp7GzWlIIrI1mjPELb6dxEv9dsZfcwrJAx+lwGiwEIlIoIgV1fBUCPf2U0RhjuHl8b8qrqnlu4fZAR/G70ooqHv1oI6uyjzhy/AYLgapGq2pMHV/RqhrqSCJjjKlDenwnvntSIv/5cieFQXYr6aGich6bu4V1e50Z4q0ll4aMMcavbj+9L4WllbwYZHMV5Jd4C19shNuR41shMMa0G0OTYxnfL45nP99OaUXwTGVphcAYY2q4fUIf8o6WMXNZdqCj+M2xQhDT3gqBiHhEZImIrBSRtSLyywa2vVREVEQynMpjjOkYTknvxvCUzjy9YBuVVdWBjuMX7blFUAZM9D2RPAKYJCJja28kItHAPcBiB7MYYzoIEeH20/uw61Axc9YfCHQcvyhory0C3wQ2R31v3b6vup49+BXwB6DUqSzGmI7l7MEJ9Iz18NLi4Og0LiipQASiw525WdPRPgIRcYnICiAH+FhVF9daPwpIUdV3GznOLccmxcnNzXUusDGmXXCFCFdkpvLZ5jx25BUFOo7j8ksqiPG4CQlxZkAHRwuBqlap6gggGcgUkZOOrROREOBR4EdNOM7Tqpqhqhnx8fGO5TXGtB+Xn5yCK0R4ZUnHbxXkl1Q41j8AfrprSFWPAPOASTUWRwMnAfN9Q1uPBWZZh7ExpikSYjycMziBGVm7Kavs2LeStttCICLxItLZ9zoCOBvYcGy9quarapyqpqlqGrAImKyqWU5lMsZ0LNPG9OJwcQXvr94f6CiOyi+pICbCucEcnGwR9ADmicgqYCnePoLZIvKwiEx28LzGmCBxap9upHWL5KUOPiqp0y0Cx0qMqq4CRtax/MF6tp/gVBZjTMcUEuKdq+C3721g4/5CBiRGBzqSI/JLKtvnpSFjjPGHy0anEBYa0qFbBQWlFY49QwBWCIwx7VzXqDDOH9qDt5bvobi8MtBxWl1pRRXlldXWIjDGmIZMG5NKYVkl76zseBPcOz28BFghMMZ0AKN7dWFAQnSHfNLYCoExxjSBiDBtbCqrsvMdm8UrUL4eedRjhcAYYxp08cgkItwuXu5grYL8YmsRGGNMk8R43Fw0oif/XbGXgg40leWx78UKgTHGNMG0Mb0oqajireV7Ah2l1VgfgTHGNMPQ5FiGJcfy4qKdqNY16n374/TsZGCFwBjTwVwzthebc46ycOvBQEdpFfklFUSHh+JyaAhqsEJgjOlgLhzek7hOYfzr8+2BjtIqvAPOOdcaACsExpgOxuN2cdWYXnyyIYftHWDSmgIrBMYY03xXj03F7RKeX7gj0FFarKCkklgHh6AGKwTGmA6oe7SHC4f35PWs3e3+VlKnh6AGKwTGmA7qhnG9KSqvYsbS3YGO0iJWCIwx5gSdlBRLZlpXnlu4g6rq9nsrabsuBCLiEZElIrJSRNaKyC/r2OY+EVknIqtE5BMR6eVUHmNM8LnhO2lkHy7hgzXtcyrL8spqSiqq2m8hAMqAiao6HBgBTBKRsbW2+QrIUNVhwEzgjw7mMcYEmbMHJ5IeF8WT87e0ywfM/PFUMThYCNTrqO+t2/eltbaZp6rFvreLgGSn8hhjgo8rRLhtQh/W7i1g/sbcQMdpNn88VQwO9xGIiEtEVgA5eCevX9zA5jcC7zuZxxgTfL43MomkzhE8Pq/9tQqO3fHUrguBqlap6gi8n/QzReSkurYTkauBDOCRetbfIiJZIpKVm9v+qroxJnDcrhBuPT2dZTsPs3j7oUDHaZZ2f2moJlU9AswDJtVeJyJnAT8HJqtqWT37P62qGaqaER8f72hWY0zHMzUjhbhO4Twxb0ugozRLQXsvBCISLyKdfa8jgLOBDbW2GQn8A28RyHEqizEmuHncLm4a35vPNuexcveRQMdpso7QIugBzBORVcBSvH0Es0XkYRGZ7NvmEaAT8LqIrBCRWQ7mMcYEsavH9iI2ws1T87cGOkqT+WN2MgDHBrBQ1VXAyDqWP1jj9VlOnd8YY2rqFB7KtDGpPPXpVnYeLKJXt6hAR2pUfkkFkWEu3C5nr+Lbk8XGmKBx3alphIYIz7aTIaoLSiscnbT+GCsExpig0T3Gw8UjkpiRtZvDReWBjtMofwwvAVYIjDFB5ubT0imtqOalxTsDHaVRVgiMMcYB/ROiOb1/PM8t3ElpRVWg4zQov6TS8YfJwAqBMSYI3XJaOnlHy/jvij2BjtKgAmsRGGOMM07t043BPWJ4esE2qtvwENV2acgYYxwiItx6ejpbc4v4YG3bHKK6sqqao2WVVgiMMcYpFwzrSXpcFNPnts3B6ApLKwGIcXi+YrBCYIwJUq4Q4Y4z+rJ+XwFz1re9EW78NbwEWCEwxgSxi0b0JKVrBNPnbm5zrQIrBMYY4wduVwh3TOjLqux8Pt3Utoa4t0JgjDF+cumoZHrGetpcX4EVAmOM8ZOw0BBum9CHZTsP8+XWg4GO8zUrBMYY40dTM1LoHh3O9LltZ+Ka9fsKiApz0SUqzPFzWSEwxgQ9j9vFLael8+W2g2TtCPx0lqrKp5tyOaVPnONDUIMVAmOMAeCqMal0jQprE62C7XlFZB8u4fQB/pma1wqBMcYAkWGh3DS+N59uyg34dJYLfHcwnd6vnRcCEfGIyBIRWSkia0Xkl3VsEy4ir4nIFhFZLCJpTuUxxpjGXOObzjLQrYIFm/NI6xZJardIv5zPyRZBGTBRVYcDI4BJIjK21jY3AodVtS/wF+APDuYxxpgGRXvcXD8ujTnrD7Bub0FAMpRVVvHl1oOc3t8/rQFwsBCo11HfW7fvq/ZNuhcBz/tezwTOFBFxKpMxxjTm+lN7Ex0eymOfbA7I+bN2HKakoorTOkIhABARl4isAHKAj1V1ca1NkoDdAKpaCeQD3ZzMZIwxDYmNdHP9d3rzwdr9rNmT7/fzL9iUS5grhLHp/vtT6GghUNUqVR0BJAOZInLSiRxHRG4RkSwRycrNbVuPgRtjOp6bxvcmNsLNox9v8vu5P92US0ZaF6LCnR919Bi/3DWkqkeAecCkWqv2ACkAIhIKxALferRPVZ9W1QxVzYiP919zyRgTnGI8bm49PZ25G3JYtvOw3857oKCUDfsL/XpZCJy9ayheRDr7XkcAZwMbam02C/i+7/VlwFxtS4N9GGOC1nWnphHXKYxHP97ot3N+fdtoRykEQA9gnoisApbi7SOYLSIPi8hk3zbPAt1EZAtwH3C/g3mMMabJIsNCuX1CX77YcpCFW/P8cs5PN+XSPTqcgYnRfjnfMY5dhFLVVcDIOpY/WON1KTDFqQzGGNMS08ak8s8F2/jThxt54/ZuOHlTo6qyaNtBTusX7+h56mJPFhtjTD08bhf3nNWP5buO8O7qfY6ea2tuEXlHyxmT3tXR89TFCoExxjRgakYKg3rE8Lv3NlBaUeXYeRZv994nk9nb/3fQWyEwxpgGuEKEX1w4mD1HSnh6wTbHzrNk+yG6R4eT5qdhJWqyQmCMMY0Ym96N84Ym8tT8rezLL2n146sqi7cdIrN3V7/3D4AVAmOMaZIHvjuIKlX+8H7tu+BbbvehEvYXlDLGj08T12SFwBhjmiClayS3npbO2yv2tvpDZot8/QNjevu/oxisEBhjTJPdPqEPCTHhPPzOWqqrW+/Z1yXbD9E1Kox+3Tu12jGbwwqBMcY0UWRYKD+bNJCV2fm89dWeVjvu4u0HOTmtS0D6B8AKgTHGNMvFI5IYkdKZP3ywgaKyyhYfb++REnYfKmFMAG4bPcYKgTHGNEOI73bSnMIynpzf8pnMlmw/BEBmgPoHwAqBMcY028jULlwyMol/frad3YeKW3SsxdsPEu0JZVCPmFZK13xWCIwx5gT8dNJAXCI8+N81tGTQ5MXbD3FyWldcIYGbnNF/Mx8YY0wHkhjr4SfnDuDh2et4delursxMbdJ+G/cX8vi8Lew7UsK+/FL2HCnh8owUh9M2zFoExhhzgq47NY1xfbvxq9nr2HmwqEn7/O2TTXy8bj9uVwhjenflBxP7MjXAhcBaBMYYc4JCQoRHLhvOuX9dwH0zVjLj1lMavMRzpLicOetyuGpMKg9NHuLHpA2zFoExxrRAz84R/Oqik1i28zB//3Rrg9u+s2of5VXVXDY62U/pmsYKgTHGtNBFI3py/rAePPrxJt5dVf+8BW8sy2ZAQjRDegbuDqG6ODlncYqIzBORdSKyVkTuqWObWBF5R0RW+ra53qk8xhjjFBHhj5cOY2RKZ+559Ss+WLP/W9tszT3Kit1HuGx0csCeIK6Pky2CSuBHqjoYGAvcKSKDa21zJ7BOVYcDE4A/i0iYg5mMMcYRUeGh/Pv6kxmaHMtdLy/n43UHjlv/xrJsXCHCRSN7Bihh/RwrBKq6T1WX+14XAuuBpNqbAdHiLY+dgEN4C4gxxrQ70R43z9+QyZCeMdzx0jJe+HIH1dVKVbXy5vI9nNYvju7RnkDH/Ba/9BGISBreiewX11r1ODAI2AusBu5R1eo69r9FRLJEJCs3N9fpuMYYc8JiPG7+c+MYxqZ34//+u5ZpzyxmRtZu9heUcmkb6yQ+xvFCICKdgDeAH6pqQa3V5wIrgJ7ACOBxEflWL4qqPq2qGaqaER8f73BiY4xpmdgIN/+5IZPfXTKU1XvyeeDN1cR4QjlrUEKgo9XJ0ecIRMSNtwi8pKpv1rHJ9cDv1ft89hYR2Q4MBJY4mcsYY5wmIlyZmcpp/eP5zbvrGJnSBY/bFehYdXKsEPiu+z8LrFfVR+vZbBdwJvCZiCQAAwDnZoc2xhg/S+ocwZPTRgc6RoOcbBGMA64BVovICt+y/wFSAVT178CvgOdEZDUgwM9UNc/BTMYYY2pxrBCo6ud4/7g3tM1e4BynMhhjjGmcPVlsjDFBzgqBMcYEOSsExhgT5KwQGGNMkLNCYIwxQc4KgTHGBDlpyaTLgSAiucBO39tYIL+B17X/Gwc09zmFmsdtyrrGlrXVnPW9byhvc3M2lPFEcjb159oWc9a1rDVz1rfuRHP683ezveQMxL+hluTspap1j9Gjqu32C3i6odd1/DerJedoyrrGlrXVnPW9byRvs3I2lPFEcjbj59rmctazrNVy1rfuRHP683ezveQMxL+h1shZ11d7vzT0TiOva/+3pedoyrrGlrXVnPW9byhvczW2X3NzNvXn2lz+yFnf+uZo7v/zupa3xd/Nupa3xZyB+DfU2L5N/Zt0nHZ3aaglRCRLVTMCnaMxlrN1Wc7W0x4yguVsrvbeImiupwMdoIksZ+uynK2nPWQEy9ksQdUiMMYY823B1iIwxhhTixUCY4wJclYIjDEmyFkh8BGR8SLydxF5RkQWBjpPfUQkRER+IyLTReT7gc5THxGZICKf+X6mEwKdpz4iEiUiWSJyQaCz1EdEBvl+jjNF5PZA56mPiFwsIv8UkddEpM3OMyIi6SLyrIjMDHSW2ny/j8/7fo7T/HXeDlEIRORfIpIjImtqLZ8kIhtFZIuI3N/QMVT1M1W9DZgNPN9WcwIXAclABZDdhnMqcBTwOJGzlTIC/AyY0dr5auRpjd/N9b7fzal4Z/5rqznfVtWbgduAy9twzm2qeqMT+erSzMyXADN9P8fJ/srYrCfa2uoXcBowClhTY5kL2AqkA2HASmAwMBTvH/uaX91r7DcDiG6rOYH7gVt9+85swzlDfPslAC+10YxnA1cA1wEXtNWfpW+fycD7wFVtOadvvz8Do9pBTkf+/bQw8wPACN82L/sjn6o6Omex36jqAhFJq7U4E9iiqtsARORV4CJV/R1Q52UAEUkF8lW1sK3mFJFsoNz3tqqt5qzhMBDeFjP6LllF4f0HWCIi76lqdVvL6TvOLGCWiLwLvNyaGVsrp4gI8HvgfVVd3toZWyunvzUnM97WczKwAj9esekQhaAeScDuGu+zgTGN7HMj8G/HEtWtuTnfBKaLyHhggZPBamlWThG5BDgX6Aw87miybzQro6r+HEBErgPyWrsINKC5P8sJeC8ZhAPvORmslub+bv4AOAuIFZG+qvp3J8PV0NyfZzfgN8BIEXnAVzD8rb7MjwGPi8j5tGwYimbpyIWg2VT1F4HO0BhVLcZbsNo0VX0Tb9Fq81T1uUBnaIiqzgfmBzhGo1T1Mbx/yNo0VT2Itx+jzVHVIuB6f5+3Q3QW12MPkFLjfbJvWVtjOVtPe8gIlrO1tZecNbWpzB25ECwF+olIbxEJw9spOCvAmepiOVtPe8gIlrO1tZecNbWtzP7qlXa4V/4VYB/f3FJ5o2/5ecAmvL3zP7ecHSdne8hoOYM3Z3vLbIPOGWNMkOvIl4aMMcY0gRUCY4wJclYIjDEmyFkhMMaYIGeFwBhjgpwVAmOMCXJWCEyHICJH/Xy+VpmzQrzzNuSLyAoR2SAif2rCPheLyODWOL8xYIXAmDqJSIPjcKnqqa14us9UdQQwErhARBqbc+BivCOmGtMqrBCYDktE+ojIByKyTLyzpQ30Lb9QRBaLyFciMkdEEnzLHxKRF0TkC+AF3/t/ich8EdkmInfXOPZR338n+NbP9H2if8k3HDMicp5v2TIReUxEZjeUV1VL8A4/nOTb/2YRWSoiK0XkDRGJFJFT8c5N8IivFdGnvu/TmKayQmA6sqeBH6jqaODHwJO+5Z8DY1V1JPAq8NMa+wwGzlLVK33vB+IdTjsT+IWIuOs4z0jgh75904FxIuIB/gF813f++MbCikgXoB/fDC/+pqqerKrDgfV4hyZYiHdMmp+o6ghV3drA92lMk9gw1KZDEpFOwKnA674P6PDNBDnJwGsi0gPv7FDba+w6y/fJ/Jh3VbUMKBORHLwzrtWeenOJqmb7zrsCSMM7Tec2VT127FeAW+qJO15EVuItAn9V1f2+5SeJyK/xzunQCfiwmd+nMU1ihcB0VCHAEd+199qmA4+q6izfpC8P1VhXVGvbshqvq6j730xTtmnIZ6p6gYj0BhaJyAxVXQE8B1ysqit9k+dMqGPfhr5PY5rELg2ZDklVC4DtIjIFvNMoishw3+pYvhn7/fsORdgIpNeYorDRydx9rYffAz/zLYoG9vkuR02rsWmhb11j36cxTWKFwHQUkSKSXePrPrx/PG/0XXZZi3dOWPC2AF4XkWVAnhNhfJeX7gA+8J2nEMhvwq5/B07zFZD/AxYDXwAbamzzKvATX2d3H+r/Po1pEhuG2hiHiEgnVT3qu4voCWCzqv4l0LmMqc1aBMY452Zf5/FavJej/hHYOMbUzVoExhgT5KxFYIwxQc4KgTHGBDkrBMYYE+SsEBhjTJCzQmCMMUHOCoExxgS5/wcNF6fghPf+8gAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.lr_find()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:33:02.517996Z",
     "start_time": "2021-03-10T23:32:45.608826Z"
    },
    "id": "ytQMO6UJTryC",
    "outputId": "dddc22b3-7763-4fec-c9b3-9b5a973731ea"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>3.983220</td>\n",
       "      <td>3.966060</td>\n",
       "      <td>0.096700</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>4.043913</td>\n",
       "      <td>2.694285</td>\n",
       "      <td>0.092500</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>3.698680</td>\n",
       "      <td>2.298090</td>\n",
       "      <td>0.102700</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>3.372990</td>\n",
       "      <td>2.348976</td>\n",
       "      <td>0.212100</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>3.066770</td>\n",
       "      <td>2.448912</td>\n",
       "      <td>0.155700</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>2.776098</td>\n",
       "      <td>2.243555</td>\n",
       "      <td>0.174200</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>2.534087</td>\n",
       "      <td>1.828253</td>\n",
       "      <td>0.339700</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>7</td>\n",
       "      <td>2.344788</td>\n",
       "      <td>1.832147</td>\n",
       "      <td>0.294000</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8</td>\n",
       "      <td>2.201845</td>\n",
       "      <td>1.831606</td>\n",
       "      <td>0.276500</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9</td>\n",
       "      <td>2.073164</td>\n",
       "      <td>1.747288</td>\n",
       "      <td>0.342400</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.fit_one_cycle(10, lr_max=3e-2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Hnj4tGuOTryD"
   },
   "source": [
    "> Tip: Unfreeze the model after calibrating the new head with the transferred body, and train a little more!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:33:02.523201Z",
     "start_time": "2021-03-10T23:33:02.520009Z"
    },
    "id": "vgunITCVTryD"
   },
   "outputs": [],
   "source": [
    "learner.unfreeze()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:33:02.554162Z",
     "start_time": "2021-03-10T23:33:02.524646Z"
    },
    "id": "nGFfRdLWTryE",
    "outputId": "6b6576c6-b603-444e-a5a9-dc8820ac1b54"
   },
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "Sequential (Input shape: 64 x 1 x 32 x 32)\n",
       "============================================================================\n",
       "Layer (type)         Output Shape         Param #    Trainable \n",
       "============================================================================\n",
       "                     64 x 4 x 30 x 30    \n",
       "Conv2d                                    40         True      \n",
       "BatchNorm2d                               8          True      \n",
       "ReLU                                                           \n",
       "____________________________________________________________________________\n",
       "                     64 x 16 x 28 x 28   \n",
       "Conv2d                                    592        True      \n",
       "BatchNorm2d                               32         True      \n",
       "ReLU                                                           \n",
       "____________________________________________________________________________\n",
       "                     64 x 32 x 26 x 26   \n",
       "Conv2d                                    4640       True      \n",
       "BatchNorm2d                               64         True      \n",
       "____________________________________________________________________________\n",
       "                     64 x 32 x 1 x 1     \n",
       "AdaptiveAvgPool2d                                              \n",
       "AdaptiveMaxPool2d                                              \n",
       "____________________________________________________________________________\n",
       "                     64 x 64             \n",
       "Flatten                                                        \n",
       "BatchNorm1d                               128        True      \n",
       "Dropout                                                        \n",
       "____________________________________________________________________________\n",
       "                     64 x 10             \n",
       "Linear                                    640        True      \n",
       "____________________________________________________________________________\n",
       "\n",
       "Total params: 6,144\n",
       "Total trainable params: 6,144\n",
       "Total non-trainable params: 0\n",
       "\n",
       "Optimizer used: <function Adam at 0x000001BF07FB7670>\n",
       "Loss function: FlattenedLoss of CrossEntropyLoss()\n",
       "\n",
       "Model unfrozen\n",
       "\n",
       "Callbacks:\n",
       "  - TrainEvalCallback\n",
       "  - Recorder\n",
       "  - ProgressCallback"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learner.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:33:24.154098Z",
     "start_time": "2021-03-10T23:33:02.555750Z"
    },
    "id": "7vN4TXLrTryF",
    "outputId": "8d4c9a15-c4b0-407f-f776-fba1267eec25"
   },
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "SuggestedLRs(valley=0.0030199517495930195)"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAvlUlEQVR4nO3dd3xUZdbA8d+ZZNIrSUggAUKvQUoowtrWgl12FRGxoKjrrquva3f31XV33V333abYsYCdRVCxr11UFCHUSK+SQEgC6ZOe5/1jJqGlktzcmcz5fj58krn3ztwzw2TOPOWeR4wxKKWU8l8OuwNQSillL00ESinl5zQRKKWUn9NEoJRSfk4TgVJK+TlNBEop5ecC7Q6greLj401qaqrdYSillE/JyMjIN8YkNLbP5xJBamoqK1eutDsMpZTyKSKyu6l92jWklFJ+ThOBUkr5OU0ESinl53xujKAx1dXVZGVlUVFRYXcotgkJCSElJQWn02l3KEopH9MlEkFWVhaRkZGkpqYiInaH0+mMMRw4cICsrCz69u1rdzhKKR/TJbqGKioqiIuL88skACAixMXF+XWLSCl1/LpEIgD8NgnU8/fnr1RX9/GG/WzLLbHksbtMIvAlERERAOzatYsRI0bYHI1Syhfc9MoqFq/KtuSx/TMRrFsI/x4BD8S4f65baHdESinVpKqaOqpq6wgPCrDk8f0vEaxbCO/cAkV7AOP++c4t7UoG99xzD48//njD7QceeIAHH3yQ008/nTFjxpCWlsaSJUuafYza2lruvPNOxo0bx8iRI3n66acBuOqqq3jrrbcajps5c2aLj6WU6lpcVTUAhAdbM7/H/xLBp3+E6vIjt1WXu7cfp+nTp7Nw4aFEsnDhQq6++mrefPNNVq1axeeff87tt99Oc8uCPvfcc0RHR7NixQpWrFjBM888w86dO5k9ezbz588HoKioiGXLlnHeeecdd6xKKd9TWulJBEHWJIIuMX20TYqy2ra9FUaPHk1ubi579+4lLy+P2NhYkpKS+M1vfsPSpUtxOBxkZ2ezf/9+kpKSGn2Mjz76iHXr1rFo0SJ3OEVFbN26lbPOOotf/epX5OXlsXjxYi6++GICA/3vv00pf+aqqgWsaxH43ydKdIqnW6iR7e0wbdo0Fi1aRE5ODtOnT+eVV14hLy+PjIwMnE4nqampzU7vNMbw6KOPMmXKlGP2XXXVVbz88sssWLCAefPmtStOpZTvqW8RhAXrGEHHOP1+cIYeuc0Z6t7eDtOnT2fBggUsWrSIadOmUVRURPfu3XE6nXz++efs3t1k4T8ApkyZwpNPPkl1dTUAW7ZsoaysDIBZs2bx8MMPAzBs2LB2xamU8j2uSneLIEJbBB1k5KXun5/+0d0dFJ3iTgL124/T8OHDKSkpITk5mR49ejBz5kwuuOAC0tLSSE9PZ8iQIc3e/7rrrmPXrl2MGTMGYwwJCQkNg8SJiYkMHTqUqVOntitGpZRvamgRWDRrSJobwPRG6enp5uj1CDZu3MjQoUNtish6LpeLtLQ0Vq1aRXR0dJPHdfXXQSl/9caqLG5buJYv7zyVPnHhx/UYIpJhjElvbJ//dQ35mE8++YShQ4dy8803N5sElFJdV1mltdNH/a9ryMecccYZLY4vKKW6tlLPGIFV00e1RaCUUl7OVVWDQyDEac1HdpdJBL421tHR/P35K9WVlVbWEB4UaFlxyS6RCEJCQjhw4IDffhjWr0cQEhJidyhKKQu4KmstGx+ALjJGkJKSQlZWFnl5eXaHYpv6FcqUUl1PaVWNZReTQRdJBE6nU1fmUkp1Wa7KGssuJoMu0jWklFJdWVllrWUXk4EmAqWU8nplVdoiUEopv1ZWWUOYRdcQgCYCpZTyemVV1s4a0kSglFJerqyyxrJlKsHCRCAiz4tIrohktnDcOBGpEZFLrIpFKaV8VV2dweXDLYL5wNnNHSAiAcDfgI8sjEMppXyWq7p+dTIfbBEYY5YCB1s47GZgMZBrVRxKKeXLXBZXHgUbxwhEJBn4GfBkK469QURWishKf756WCnlf6xeuB7sHSx+GLjbGFPX0oHGmLnGmHRjTHpCQoL1kSmllJeweuF6sLfERDqwwFNNLx44V0RqjDFv2RiTUkp5lUMtgi5Ya8gY01AcSETmA+9qElBKqSO5qqwfI7DskUXkNeBUIF5EsoDfA04AY8xTVp1XKaW6kobVyXyx+qgxZkYbjp1lVRxKKeXLuvSsIaWUUi2rHyPQWkNKKeWnGmYN+WKJCaWUUu1XVllDcKCDwADrPq41ESillBezei0C0ESglFJerayy1tL1ikETgVJKebXSyhpLy0uAJgKllPJqrqoaS6eOgiYCpZTyaqWV1q5FAJoIlFLKq7kqa4jQMQKllPJfVi9cD5oIlFLKq5VV1er0UaWU8lfGGE+LQLuGlFLKL1XV1lFTZ3SwWCml/FVZpfV1hkATgVJKea2yTihBDZoIlFLKa5V1wupkoIlAKaW8VkPXkCYCpZTyT2WdsHA9aCJQSimv1RkL14MmAqWU8loNC9frlcVKKeWfDrUItGtIKaX8UqlOH1VKKf/mqqwlwCEEB1r7Ua2JQCmlvFSpp86QiFh6Hk0ESinlpVydsHA9aCJQSimvVVZZa3nlUdBEoJRSXqtMWwRKKeXfOmN1MtBEoJRSXqusExauBwsTgYg8LyK5IpLZxP6LRGSdiKwRkZUi8hOrYlFKKV9UVlVj+cVkYG2LYD5wdjP7PwVOMMaMAq4FnrUwFqWU8jlllTW+3SIwxiwFDjazv9QYYzw3wwHT1LFKKeWPyiprLa88CjaPEYjIz0RkE/Ae7laBUkopoLbOUF7t42MErWGMedMYMwSYCvypqeNE5AbPOMLKvLy8TotPKaXs0lBwzl9mDXm6kfqJSHwT++caY9KNMekJCQmdHJ1SSnW+zlqdDGxMBCIyQDwFNERkDBAMHLArHqWU8iZlnVSCGsCyVCMirwGnAvEikgX8HnACGGOeAi4GrhKRaqAcmH7Y4LFSSvm1Q8tUWt8isOwMxpgZLez/G/A3q86vlFK+rL5rKMzHryNQSil1nOpbBFprSCml/FRZJy1cD5oIlFLKK5V10sL1oIlAKaW8UklFNQARIZoIlFLKLxW4qgkKcHT9EhNKKaUaV1BWRUyY0/L1ikETgVJKeaUCVxWxYUGdci5NBEop5YUKXdXEhjs75VyaCJRSygsd1BaBUkr5t0JXFbHhmgiUUsovGWMocFUTG6ZdQ0op5ZeKK2qorTPaNaSUUv6q0FUFoIlAKaX8VYHLfVWxzhpSSik/VVDmbhHEaItAKaX8U4Gna6ibJgKllPJPB8t0jEAppfxaoauaAIcQ2QmVR0ETgVJKeZ0CVxUxoU4cDusLzoEmAqWU8joFLnfl0c6iiUAppbxMQVk13TqpvARoIlBKKa/jbhFoIlBKKb/lXovAy7qGRCRcRBye3weJyIUi0nlRKqWUn2goOOeFXUNLgRARSQY+Aq4E5lsVlFJK+StXVS1VNXWddg0BtD4RiDHGBfwceMIYMw0Ybl1YSinlnwoaCs55WdcQICJyIjATeM+zLcCakJRSyn8V1hec88IWwa3AvcCbxpgfRKQf8LllUSmllJ9qKC/RiWMErbp+2RjzJfAlgGfQON8Yc4uVgSmllD8q6OS1CKD1s4ZeFZEoEQkHMoENInJnC/d5XkRyRSSzif0zRWSdiKwXkWUickLbw1feZtm2fH76jy+4Z/E6Ptmwn/KqWrtDUsqnHOoa8r4xgmHGmGJgKvAB0Bf3zKHmzAfObmb/TuAUY0wa8CdgbitjUV7s5eW7ySmu4L11+7juxZWM/tNH/PbN9eSWVNgdmlI+ob5rKDq08xJBa0vbOT3XDUwFHjPGVIuIae4OxpilIpLazP5lh938DkhpZSzKS7mqavh8Ux6XjE3hvvOHsXznAd5bt4+FK/bw1upsfnFyf64/uS9hQZ1TUVEpX1ToqiI61ElgQOdd79vaMz0N7ALCgaUi0gco7sA4ZuNuaSgf9uXmPMqrazknLYmgQAcnDUzgoYtH8vFtp3Dq4AT+/ckWzvjnl+zKL7M7VKW8VoGrulO7haCVicAYM8cYk2yMOde47QZO64gAROQ03Ing7maOuUFEVorIyry8vI44rbLA+5k5xIUHMT612xHb+8aH88TMsbx+44mUV9cy89nlZBW4bIpSKe/W2XWGoPWDxdEi8q/6D2MR+Sfu1kG7iMhI4FngImPMgaaOM8bMNcakG2PSExIS2ntaZYGK6lo+27ifs4YnNdmkHZfajZdmT6CkoprLn1nOvqLyTo5SKe9X4Krq1Mqj0PquoeeBEuBSz79iYF57TiwivYE3gCuNMVva81jqSDvySlmyJpsiz+yDzrB0Sx5lVbWcm5bU7HEjkqN5cfYEDpZVMfOZ5XyYuY+1ewrJLamgrq7ZYScAFmVkMfmhz/joh5yOCl0pr1JQVt2paxFA6weL+xtjLj7s9h9EZE1zdxCR14BTgXgRyQJ+DzgBjDFPAfcDccATIgJQY4xJb1P06gjGGF5fmcX9b2dSUV1HUICDM4cncsnYFE4ZmGDpakcfZOYQE+ZkYr+4Fo8d1SuG+deMY9a8Fdz48qqG7VEhgZw+NJEpwxM5eVDCEYPK5VW13L8kk9czsgh1BvA/C9aw8BcnkpYSbcnzUcouBa6qTlu0vl5rE0G5iPzEGPM1gIhMBppt1xtjZrSw/zrgulaeX7WgpKKa/30rkyVr9jKpfxy/Pm0AH23Yz1trsnlv3T5OGZTAI5eNsqTvsbKmlk827OectCScrZzpkJ7ajW/v/Sm7D7jYV1TBvqJy1uwp5LNNuby5OpugQAeDEiMY2D2S/gnhvLN2H1tyS7jlpwOYObEPP39iGbNfWMFbN02mZ0xohz8npexQUV2Lq6q2U68qhtYnghuBF0Wk/utXAXC1NSGptqqrM0x76lu27C/hjrMG8ctTBxDgECYNiOfec4ew4Ps9PPjeBi547GuenDmWEcktf4s2xrA9r5T+CRF4WmxN+mZbPiWVNZyT1qNNcUeGOBmRHN0Qz1UnQk1tHSt2FfDZpv1syilh+Y4DvLk6m27hQbxwzXhOHuQeI3p+1jgueXIZ185fwaJfTiIiWKekKt9XfzGZV3YNGWPWAieISJTndrGI3AqsszA21Urb80rZlFPCHy8azlUnph6xLzgwgKsnpTIyJZpfvbKKi59cxo2n9CcpOoSwoABiw4KY1D/umAHeF5bt4oF3NjBzQm/+dNGIJruVSiqqWbgii8iQQCb3j2/3cwkMcHBi/zhO7H+oi6mkohpngIMQ56E6h4OTInls5hiunb+CyQ99RrfwIMKDA4gIDqR7ZAiJUcEkRoUQHxFMt/Ag4iKC6BEd2umDcEq1RX15CW/tGgLcCeCwm7cBD3doNBZzVdUQ6gxo8Ruur8nYXQDASQObnlE1uncs79z8E25dsIZHPt16xL6Lx6Twj2kjG16XLftL+OsHm0iKCuGV5T9SZ+DPUw8lg31F5SzOyGLplnxW/VhATZ3h6hP7EBRozQUwkSGNfzs6ZVACT18xlk827qe0sgZXVS3F5dWszSpkf3EFFdV1RxwvAmcPT+IXp/RnVK+Yhu2llTXsL64gOSb0iGRzuKqaOj7I3Mcbq7LpHhnMhH5xTOjbjZTY0C73flL2qU8EnT19tD3taZ9697+9di+3LljNF3ecRu+4MLvD6VAZuwvoFh5EagvPKz4imJdmj6e4vAZXdQ1llbW8sSqLJ77YTkpsKL85cxBVNXXcumANEcGBvHPzT3hh2S4e+3wbxhiunpTKM1/t4O01e6mpM6QlR3PDyf04aWAC41JjO+nZHumMYYmcMSzxmO3GGIorajhQWsmBsioOlFayNquIV77bzQeZOUzo243uUSH8kF3EDs8FbiLQMzqUvvHhJMeEkhQdQlJ0CPuLK3hl+Y/klVSSEhvK2qxCXs/IAmBcaiwvzZ5wTALJLamgoqquy73XlLUKyjx1hsK9sGuoCS3P9fMiqXFh1BnI3FvU5f44M34sYEzv2FZ9MxURosOcRLsncHHnlMHklVTyyKdbSY4JZUd+GRv2FfPMVekkRAZz+1mDcAjM+WwbC1bsIdQZwBUT+zD7J33p1c17X0cRITrUSXSok36ehtLZI3pw02kDWPD9j8xftousgnJGJEfxs9HJ9IgJJavAxa78MnYecPHZ5lzySysxnnf5aYMTmDW5LycNcHd/bckt4dONufz9v5t58L0NPDg1reHcucUVTH38G/LLqnjwohFcOq7XEbFlFbgICnDQPSqkU14L5Tu8smtIREpo/ANfAJ+aqjEoMZJAh7A+u4hz2zio6c0OllWxI6+MaWN7tXxwI0SEv/w8jZziCu59cz11xjBjfC/O9HzLFhFuO2swCZHBFFfUcPn43p0+o6EjRQQHct1J/bjupH4tHltdW0duSSUBIiRFH/mhPSQpiiFJURSVVzN36Q4m94/nnLQelFfVct2LKyksr2ZUSgx3LV7H6j2FPHDhMHbml/HoZ9t4f/0+ggIc3HrGIK47qW+rZ1qprq+gzAu7howxkZ0ViNVCnAEMSowkM7vI7lA61CrP+MDYPsffNeMMcPDEzDFcNvc7Kqpr+d/zhh1zzJVHDUL7A2eAg+QWpqbecdZglu88yF2L1zG8ZzR/eX8j67OLeObKdE4b0p1/frSZJ77Yzhebc9lXVEF4UAC/OLk/u/LL+NuHm1iyJps//yyNMb1jdKxBUeCqJjwowLLxtqb41Zy7tORoPtqQgzGmy/zRZfxYQKBDGNnOC6siQ5wsuWkyNXWmyQFTdaygQAePXjaa8+Z8xfmPfkVxRQ33nT+sYdzirrOHMDIlhn99vJlp6QO5dnJqw7e9j37I4f4lP3Dxk8uIDA6kf/cIBnSP4ISUaCYNiKdffHiXeZ+q1il0VdnS4varRDAiOYr/rNxDdmE5KbHe27/dFhm7CxieHN0hH96BAQ4CNQe0We+4MP56cRq/fnU1V0zszbWTU4/Yf/aIJM4ecWzpjbOGJ3Fi/ziWrNnLlv0lbMst5YvNeSzyDEQnRYVw6uAEfnPmIBJ1PMEvHHRVderKZPX8LBG4vzVnZhe3OhHsPlDGdzsOMH1cbytDA9wzXZ79aidbc0tIjgkjOTaUEcnuvujGVNfWsXZPITMn9LE8NtW880f25ISUGJJj2jadNDLEyRUTD/3/GWPYfcDFsu0H+GZ7Pm+uzua99fu477xhTEtP0RZCF1fgqtYWgdWG9ogiwCFkZhc1+g2tMf/6eAtL1uzlrGFJlv4H1dUZ7luSySvLfyQ2zEmB5wpDh8DiX05idO9jxwA27C2msqauXeMDquN0xCwqESE1PpzU+HAun9CbXfll3L14HXctXsc76/byv+cNY3BSlxm6U0cpdFW1OA3cCn41XSHEGcDA7hFk7m3dgHFFdS2fbswFYFNOiWVx1dYZ7l68jleW/8gvT+3PqvvOZPODZ/PJbacQGeLkiS+2N3q/+gvJxvSJsSw2Za/U+HBeu34if5o6glW7C5jy8FIueuxrXvpuN0XlnVddVlnPGENOUQXdI4M7/dx+lQjA3T2UmV2EMYdmxZZUVPPpxv1HbAP4ams+pZU1AGzK6cgF2Q6prTPctnANr2dkcesZA7lrymBEhODAAAZ0j2DWpFQ+3rCfzY0koozdBSTHhNIj2qdm8qo2cjiEKyf2Yeldp3Hf+cOorKnjvrcymfTXT3n6y+1U1dQ1eV93V1MZ1bV1sG4h/HsEPBDj/rluYec9CdWi/NIqKmvqWpypZgW/6hoCGNEzikUZWeQUVzR8gP774608/81OXrt+4hE1bt5fv4/oUCcOgU37rGkRfLE5lyVr9nL7mYO4+fSBx+yf5bma96kvt/Pv6aMathtjWLn7IOP7tlz2WXUNcRHBzP5JX66dnEpmdjGPfLqFv36wif+s2MN9FwxjTO9YKqprKa+qZXteKZ9szOWzTfvZX1zJlWHLuZ+ncdZVuB+saA+8c4v795GX2vekVIPsQndBZzsmsvhdIqivX5+ZXUyP6FB30bSVewB48svtDYmgvrTy2SOS2FtUblmLYMNe9+POPqlvo/tjw4O4fHxv5i3bxW1nDmroh95bVMH+4krSdXzA74gIaSnRPHv1OD7fnMsf39nANfNWHHNceFAAJw9KYHzfblzw2a04ayuOPKC6nJL37qd2wNROv4BJHSu7wJ0IkmO1RWC5oT2icAiszy7izGGJvL4yi9LKGs5NS+L99TlkZhcxIjmar7e6SyufO7IHX2/N55Xlu6mtMwR08OIum/eX0Ltb2BGLsBztupP68eK3u3l66XYenJpGTW0d/1nhTl46UOzfThvcncn943lrdTbFFdWEBgUQ6gwgMSqE9NRYguvnA3/c+Frf4RU5TH7kKx6ePooJrVhUSFknu9C9jrcmgk4QFhRI/4QIMrOLqK0zvPDtLsb2ieWhi0fy1ZZ8nvxiO4/PHMP763OI8pRWzi+ppKK6jt0HyuiXENGh8WzOKWFQYvOzQJKiQ7h4bAoLV2bRLz6C+ct28eNBF+P7dmOIziDxe0GBjmPqGR0jOsXdHXSUmsieBBsHM575jpt/OpCbfzqgyTWnlbWyC8qJDAkkqolqu1byy//xNM+A8Webctl9wMW1k/sSFeLkihP78H7mPrbsL+HjDTmcOSyJoEAHQ3u45/F39MyhyppaduaXterD/MZT+lFTW8cf391ATJiTZ69K5z83TNQ/WtU6p98PzqO+aTpDCTrrAd695SSmjkrmkU+3cu6cr5i7dDv7iysafxxlmezCclsGisFPE8Hw5GhySyr550eb6RkdwpTh7nIA107uS1CAg1++nEFxRU3DQuwDukd4Bow7dpxgZ34ZNXWGQa1IBH3iwnn88jHMu2YcS26azBnDEvXiItV6Iy+FC+ZAdC9A3D8vmAMjLyUiOJB/TR/FnBmjCQ0K5C/vb+LEv37Klc8t5+ut+cfMplPWyCooJ8WGbiHww64hcLcIwP0N/95zhjR8q06IDObS9F689N1uIoMD+clAd8nhEGcA/RIi2NjBLYL6KaGDW+gaqtfWpSCVOsLIS5udIXThCT258ISebM8rZcnqbBauzOKK55Yztk8st5w+kJMHxuuXDwtlF5QzoW83W87tly2CYT2jEIFQZwCXHVU64oaT+xHgEM4clnhooA0YkhTZ4TOHNueU4AwQ+saHd+jjKtUe/RMiuO2swXx516n8aeoI9hWWc/Xz33P7wrXaOrBIUXk1JZU1tgwUg5+2CCKCAzl5YAJpydFEH7VIdK9uYfznhon0iTvyw3lojyjeXbePkorqJpdObKvNOSX0i4/o9JKzSrVGcGAAV07sw6XpKfz746089eV2xqbGam0rCzRMHY2xpximXyYCgBeuHd/kvvTUY5tn9QO6W/aXMLZPxzTfNu8vYUwjNYSU8ibBgQHcNWUwG/YV84d3NjCmd2zDBArVMeovJrOrRaBfRVtpiOeNv7GDrjAurawhq6BcC4gpn+BwCP+69ASiQ53c9OoqyjylV1THyC7wXEOgs4a8W8/oECJDAjtsnGDLfndCaekaAqW8RXxEMI9MH8XO/DLuX/KD3eF0KdmF5QQHOoiPsOcKb00ErSQiDE2K6rCaQ1s8M4b0gjDlSyYNiOfmnw5k8aosXl957AVq6vhkF5aTHNu2tSw6kiaCNhjSI5JNOSUdMnNi8/4SwoICbGsKKnW8/uf0gUzs1437lmQ2tGxV+2QX2HcxGWgiaJMhSVENffvtVV9awtHBtYuUslqAQ5hz2Wgigp386hUdL+gI7uVzNRH4hPqB3Y4oNbFlf0mrLyRTytt0jwrhkctGsT2vlPveytTrC9qhvKqW/NIqbRH4iiFJkQQHOnj8822UVBz/6lD5pZXkl1a1qrSEUt5q8oB4/uf0gbyxOpu7Fq3jw8wcCl1Vdoflc+yeOgoWJgIReV5EckUks4n9Q0TkWxGpFJE7rIqjI4UHBzJnxmgys4uYNW9Fw+plbaUDxaqruPmnA7k0PYV31u3lxpczGP2nj7ls7rfaXdQGDYnApovJwNoWwXzg7Gb2HwRuAf5hYQwdbsrwJB6dMZo1ewq5Zt73x/WG36xTR1UXEeAQ/u+SE1j3+ym8fuOJ/Pq0AXy34yBzPttqd2g+w84FaepZlgiMMUtxf9g3tT/XGLMC8LkVuM9J68Gcy0az6sdCbnhpJXV1LfePGmM4WFbF+qwivtmWT1x4EAk2LFKtlBWCAh2MS+3G7WcN5tL0FJ77aifbcnVGUWtkF7oIcAiJNn4e+ESJCRG5AbgBoHfv3i0c3TnOG9mDwvIqfvdmJm+szuaSsSlH7K+ormXZ9nwydheQsbuA9VlFlFXVNuw/Y2j3zg5ZqU5x99lD+DAzh/uX/MAr103QiqUtyC4oJykqxNa1RXwiERhj5gJzAdLT071mesKMcb1ZlJHFQx9s4qzhiQ0rC1VU1zL96W9Zm1VEgEMY3jOKi8emkBoXTnJsKMkxoQzo3rErnSnlLeIigrnz7CHc91Ym767bxwUn9LQ7JK9m99RR8JFE4K0cDuEPFw7nose/4dFPt/K784ZhjOHeN9azLruIv18ykvNG9mh2PWKluqLLx/fmPyt+5MH3NnDakO5EBOvfQFOyC8qZ2N/e9aJ1+mg7jUyJYXp6L+Z9s4ttuSU889UO3lydze1nDmJaei9NAsovBTiEP140gv3FlVw291u+3JKn1xo0orq2jpziClJsrjBg5fTR14BvgcEikiUis0XkRhG50bM/SUSygNuA//Uc45O1be+YMpjQoAB++fIqHvpgE+el9eCm0wbYHZZSthrTO5Y5M0ZTUFbN1c9/z/S535Gxu8DusLxKTlEFdcbeGUNgYdeQMWZGC/tzgJTmjvEV8RHB3HbmIP7wzgaG9oji79NG6gCZUriXv5wyPJEF3+/hsc+3Mf3pb1l444m6DodHls0L0tTTrqEOcuXEPjxwwTDmzRqn3UFKHSY4MICrJ6Xy8W9OJik6hJtfXa1XIHt4w1XFoImgwwQGOJg1uS9J0SF2h6KUV4oJC+Lxy8eQW1LB7QvXtur6m65uz0EXItAzxt7PDU0ESqlOc0KvGH537lA+3ZTL3K922B2O7fYcdNEjKoTgwABb49BEoJTqVFdPSuXctCT+/t/NLNuWb3c4ttpT4CKlm73jA6CJQCnVyUSEhy4eSd/4cK57cSUrdzVZiabL23OwnF6xmgiUUn4oKsTJq9dNIDEqhFnzVrBmT6HdIXW6iupacoor6K0tAqWUv+oeFcKr10+gW3gQVz23nMzsIrtD6lT1M4Z6dbN/uVpNBEop2/SIDuXV6ycQGeLk6ue/Z19R+5eB9RV7DroA6KUtAqWUv0uJDeOFa8dRXl3Lr19dTXVtnd0hdYqGRKBjBEopBQO6R/K3i0eSsbuAv76/ye5wOsWegnKCAh1094J1STQRKKW8wgUn9GTWpFSe/2Yn767ba3c4lttz0EVKbCgOh/3laDQRKKW8xm/PHcqY3jHcvWhdl1/h7MeDLq/oFgJNBEopLxIU6ODxmWMIDQrg+hczKHL53Eq2rbbnoMsrpo6CJgKllJfpER3Kk1eMJavAxc0LVlPbBWsSFZVXU1xR4xVTR0ETgVLKC41L7cYfLxrB0i15/O3Drjd47E0zhkCXqlRKeakZ43uzcV8xc5fuYEhSJD8f0yWWLwG86xoC0BaBUsqL3Xf+ME7sF8c9i9ezogvVJNpToIlAKaVaxRng4MkrxpASG8oNL65kV36Z3SF1iD0Hy4kKCSQ61Gl3KIAmAqWUl4sJC2LeNeMAuHb+ii6xutmPB11e0xoATQRKKR/QJy6cuVelk1VQzg0vZVBZU2t3SO2yp8B7po6CJgKllI8Yl9qNv08byfc7D/p0GYq6OkNWQbm2CJRS6nhcNCqZayanMn/ZLj7MzLE7nOOSW1JJVU0dvWxesP5wmgiUUj7l3nOGMjIlmrsWrW2YhulLvG3GEGgiUEr5mKBAB4/NGIMx8OtXV1FV41tlq73tGgLQRKCU8kG948L4v0tGsjariL+8v9HucNrkR08iSI7RriGllGqXc9J6cO3kvsxftosXlu2yO5xW23OwnMSoYEKcAXaH0kBLTCilfNbvzhvKjwdd/OGdH0iOCeWMYYl2h9Qib6o6Wk9bBEopnxXgEObMGMWI5Ghufm0167IK7Q6pWTW1dfywt4jBSZF2h3IETQRKKZ8WFhTIs1en0y08iGvnrySnqMLukJq0YV8xZVW1jO8bZ3coR7AsEYjI8yKSKyKZTewXEZkjIttEZJ2IjLEqFqVU19Y9MoR514yjrLKGO15fS52XrmHw/U534bwJfbvZHMmRrGwRzAfObmb/OcBAz78bgCctjEUp1cUNSozkvvOH8fW2fJ7/Zqfd4TRq+c6DpMaFkRgVYncoR7AsERhjlgLN1Y29CHjRuH0HxIhID6viUUp1fTPG9+LMYYn834eb2biv2O5wjlBXZ1ix6yDjvaw1APaOESQDew67neXZdgwRuUFEVorIyry8vE4JTinle0SEh36eRnSYk1sXrKGi2nuK023NLaXQVe114wPgI4PFxpi5xph0Y0x6QkKC3eEopbxYXEQwf79kJJv3l/DQB95TnG75zgOA940PgL2JIBvoddjtFM82pZRql1MHd2fWJHdxuqVbvKMXYfnOg/SIDiHFi4rN1bMzEbwNXOWZPTQRKDLG7LMxHqVUF3LPOUMY2D2CO15fS0GZvYvZGGP4fqd7fEBEbI2lMVZOH30N+BYYLCJZIjJbRG4UkRs9h7wP7AC2Ac8Av7IqFqWU/wlxBvDwZaMocFVx7xvrMca+KaW7DrjIK6n0yoFisLDEhDFmRgv7DXCTVedXSqnhPaO546zB/PWDTbyekcWl6b1avpMFvvfi8QHwkcFipZQ6Xted1I+J/brxwNs/sHZPoS0xLN95kLjwIPonRNhy/pZoIlBKdWkBDuHh6aOJiwjiiueW21KPyJvHB0ATgVLKDyRFh/Da9ROJDnVyxbPLWZ9V1Gnnzi4sJ6ug3GvHB0ATgVLKT6TEhvHa9ROJDHFyxXPL+TAzp1NWN/v4B/faypMHxFt+ruOliUAp5Td6dQtjwQ3ulsGNL2cw/i+f8Ns311vaXbRoVRYjkqMYlOhdpacPp4lAKeVXenUL45PbTuH5WemcMiiBN1dl87MnlrEpp+NrE23cV0xmdjGXjEnp8MfuSJoIlFJ+JyjQwU+HJPLIZaP5+u7TiAwJ5A9vb+jwaw0WZWThDBAuHNVoGTWvoYlAKeXX4iKCuf3MQXy74wAfZOZ02ONW19bx1upsTh+SSLfwoA57XCtoIlBK+b0Z43szJCmSP7+3kfKqjqlY+sXmPA6UVTEt3bu7hUATgVJKERjg4IELh5NdWM7TS7d3yGMuythDfEQwJw/y/orJmgiUUgqY2C+O80b24MkvtrP7QFm7HutAaSWfbszlZ6N74gzw/o9Z749QKaU6yW/PHUqgQzjnka/4x383U1xRDcDW/SXc+8Z6ht//IVMf/4Z53+wkt6SiycdZsmYvNXWGi8d6f7cQgNhZke94pKenm5UrV9odhlKqi9qZX8Y/P9rMu+v2ER3qZEhSJMt3HiQ40MHZI5LYur+UDfuKcQicMTSR+84fRq9uYQ33X7Imm3sWr2dwUiRv3TTZxmdyJBHJMMakN7pPE4FSSh0rM7uIf360mR35ZVya3osZ43s3zP7ZllvCW6v3Mu+bndQZuO3MQVx5Yh8e+mAT85ftYnxqNx67fDTdvWiRek0ESillgb2F5dy/JJNPNuYSERxIaWUN107uy73nDvG6sYHmEoFl6xEopVRX1zMmlGeuSufDzByeWrqDayencpGXXzzWGE0ESinVDiLCOWk9OCeth92hHDfvarsopZTqdJoIlFLKz2kiUEopP6eJQCml/JwmAqWU8nOaCJRSys9pIlBKKT+niUAppfycz5WYEJE8YDcQDRQdtuvw2/W/H/0zHshv4ymPPk9r9re0rblYD9/W0fE2ta81r2VLcdv12jZ125dibSxGu2NtKr7GYj18mz+/Z7091j7GmMYXRzDG+OQ/YG5Tt+t/b+TnyvaepzX7W9rWXKxWxtvUvta8lq14jW15bZu67UuxNhGjvmeb2eeN71lfivXof77cNfROM7ffaeJnR5ynNftb2tZcrK05Z1vjaWlfa17Lpn63+7Vt6rYvxXr4794S69Hb9D3b8n19KdYj+FzXUHuIyErTRPU9b+RL8Wqs1vClWMG34tVYD/HlFsHxmGt3AG3kS/FqrNbwpVjBt+LVWD38qkWglFLqWP7WIlBKKXUUTQRKKeXnNBEopZSf00TgISInichTIvKsiCyzO57miIhDRP4sIo+KyNV2x9MSETlVRL7yvL6n2h1PS0QkXERWisj5dsfSHBEZ6nlNF4nIL+2OpzkiMlVEnhGR/4jIWXbH0xIR6Sciz4nIIrtjaYznPfqC5zWd2d7H6xKJQESeF5FcEck8avvZIrJZRLaJyD3NPYYx5itjzI3Au8AL3hwrcBGQAlQDWVbF6omrI+I1QCkQgoXxdlCsAHcDC62JsiGmjnjPbvS8Zy8FJnt5rG8ZY64HbgSmWxVrB8a7wxgz28o4j9bGuH8OLPK8phe2++RtvVrNG/8BJwNjgMzDtgUA24F+QBCwFhgGpOH+sD/8X/fD7rcQiPTmWIF7gF947rvI219bwOG5XyLwipfHeiZwGTALON+bY/Xc50LgA+Byb4/Vc79/AmO8/T172P0s/ftqR9z3AqM8x7za3nN3icXrjTFLRST1qM3jgW3GmB0AIrIAuMgY81eg0Sa/iPQGiowxJd4cq4hkAVWem7VWxQod99p6FADBlgRKh722pwLhuP/YykXkfWNMnTfG6nmct4G3ReQ94NWOjrOjYhURAR4CPjDGrLIizo6M1w5tiRt3yzoFWEMH9Ox0iUTQhGRgz2G3s4AJLdxnNjDPsoia1tZY3wAeFZGTgKVWBtaENsUrIj8HpgAxwGOWRnasNsVqjPkdgIjMAvKtSALNaOvreiruLoJg4H0rA2tEW9+zNwNnANEiMsAY85SVwTWira9tHPBnYLSI3OtJGHZoKu45wGMich7tK0MBdO1E0GbGmN/bHUNrGGNcuJOWTzDGvIE7efkMY8x8u2NoiTHmC+ALm8NoFWPMHNwfXj7BGHMA93iGVzLGlAHXdNTjdYnB4iZkA70Ou53i2eaNfClW8K14NVZr+FKs4Hvx1uuUuLtyIlgBDBSRviIShHsA8G2bY2qKL8UKvhWvxmoNX4oVfC/eep0Td2eNiFs82v4asI9D0ylne7afC2zBPer+O7vj9LVYfS1ejVVj9cV4vSFuLTqnlFJ+rit3DSmllGoFTQRKKeXnNBEopZSf00SglFJ+ThOBUkr5OU0ESinl5zQRqC5BREo7+XwdsmaFuNdqKBKRNSKySUT+0Yr7TBWRYR1xfqVAE4FSjRKRZutwGWMmdeDpvjLGjAJGA+eLSEtrC0zFXR1VqQ6hiUB1WSLSX0Q+FJEMca+QNsSz/QIRWS4iq0XkExFJ9Gx/QEReEpFvgJc8t58XkS9EZIeI3HLYY5d6fp7q2b/I843+FU/JZUTkXM+2DBGZIyLvNhevMaYcd1nhZM/9rxeRFSKyVkQWi0iYiEzCvQbB3z2tiP5NPU+lWksTgerK5gI3G2PGAncAT3i2fw1MNMaMBhYAdx12n2HAGcaYGZ7bQ3CX0B4P/F5EnI2cZzRwq+e+/YDJIhICPA2c4zl/QkvBikgsMJBDpcXfMMaMM8acAGzEXXJgGe5aM3caY0YZY7Y38zyVahUtQ626JBGJACYBr3u+oMOhRXFSgP+ISA/cqz7tPOyub3u+mdd7zxhTCVSKSC7uVdaOXm7ze2NMlue8a4BU3Etz7jDG1D/2a8ANTYR7koisxZ0EHjbG5Hi2jxCRB3Gv4xAB/LeNz1OpVtFEoLoqB1Do6Xs/2qPAv4wxb3sWd3ngsH1lRx1bedjvtTT+N9OaY5rzlTHmfBHpC3wnIguNMWuA+cBUY8xaz0I5pzZy3+aep1Ktol1DqksyxhQDO0VkGriXShSREzy7ozlU0/1qi0LYDPQ7bOnBFhds97QeHgLu9myKBPZ5uqNmHnZoiWdfS89TqVbRRKC6ijARyTrs3224Pzxne7pdfsC91iu4WwCvi0gGkG9FMJ7upV8BH3rOUwIUteKuTwEnexLIfcBy4Btg02HHLADu9Ax296fp56lUq2gZaqUsIiIRxphSzyyix4Gtxph/2x2XUkfTFoFS1rneM3j8A+7uqKftDUepxmmLQCml/Jy2CJRSys9pIlBKKT+niUAppfycJgKllPJzmgiUUsrPaSJQSik/9/9BeGY2qoPZAAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.lr_find()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:33:34.334222Z",
     "start_time": "2021-03-10T23:33:24.155898Z"
    },
    "id": "xgkkf53ATryF",
    "outputId": "8a998ec8-702b-46ba-884f-84c46f490fd4"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1.203546</td>\n",
       "      <td>1.595100</td>\n",
       "      <td>0.472100</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>1.139515</td>\n",
       "      <td>1.518108</td>\n",
       "      <td>0.532700</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>1.178870</td>\n",
       "      <td>1.428170</td>\n",
       "      <td>0.590200</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>1.097698</td>\n",
       "      <td>1.344938</td>\n",
       "      <td>0.625300</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>1.044257</td>\n",
       "      <td>1.279844</td>\n",
       "      <td>0.649800</td>\n",
       "      <td>00:01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>1.036695</td>\n",
       "      <td>1.239927</td>\n",
       "      <td>0.666600</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.fine_tune(5, base_lr=3e-2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yUk_BLQhTryF"
   },
   "source": [
    "> Important: We were able to get 60.6% accuracy using transfer learning from our pretraining on the rotation prediction task!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UDqktL1ITryF"
   },
   "source": [
    "# FastAI Vision Learner [From Sratch-Classification]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0MRU-GS5TryF"
   },
   "source": [
    "Here we train a model from scratch on the original 180 labeled data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:33:34.348812Z",
     "start_time": "2021-03-10T23:33:34.339252Z"
    },
    "id": "uzkluGn_TryF",
    "outputId": "ff2771f7-3bd8-4442-c96d-9af9b5f7e331"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sequential(\n",
       "  (0): Sequential(\n",
       "    (0): Conv2d(1, 4, kernel_size=(3, 3), stride=(1, 1))\n",
       "    (1): BatchNorm2d(4, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (2): ReLU()\n",
       "    (3): Conv2d(4, 16, kernel_size=(3, 3), stride=(1, 1))\n",
       "    (4): BatchNorm2d(16, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (5): ReLU()\n",
       "    (6): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1))\n",
       "    (7): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "  )\n",
       "  (1): Sequential(\n",
       "    (0): AdaptiveConcatPool2d(\n",
       "      (ap): AdaptiveAvgPool2d(output_size=1)\n",
       "      (mp): AdaptiveMaxPool2d(output_size=1)\n",
       "    )\n",
       "    (1): Flatten(full=False)\n",
       "    (2): BatchNorm1d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)\n",
       "    (3): Dropout(p=0.5, inplace=False)\n",
       "    (4): Linear(in_features=64, out_features=10, bias=False)\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 48,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#collapse-show\n",
    "# pretrained=False, Create the same model as before, but without using the rotation pretraining weights!\n",
    "learner = cnn_learner(dls,\n",
    "                      simple_arch,\n",
    "                      pretrained=False,\n",
    "                      loss_func=CrossEntropyLossFlat(),\n",
    "                      custom_head=classification_head,\n",
    "                      metrics=[accuracy])\n",
    "\n",
    "learner.model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:33:34.380621Z",
     "start_time": "2021-03-10T23:33:34.350685Z"
    },
    "id": "SZluFY2oTryF",
    "outputId": "5e3f254c-c200-44b1-ae51-600e871a3f71",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "Sequential (Input shape: 64 x 1 x 32 x 32)\n",
       "============================================================================\n",
       "Layer (type)         Output Shape         Param #    Trainable \n",
       "============================================================================\n",
       "                     64 x 4 x 30 x 30    \n",
       "Conv2d                                    40         True      \n",
       "BatchNorm2d                               8          True      \n",
       "ReLU                                                           \n",
       "____________________________________________________________________________\n",
       "                     64 x 16 x 28 x 28   \n",
       "Conv2d                                    592        True      \n",
       "BatchNorm2d                               32         True      \n",
       "ReLU                                                           \n",
       "____________________________________________________________________________\n",
       "                     64 x 32 x 26 x 26   \n",
       "Conv2d                                    4640       True      \n",
       "BatchNorm2d                               64         True      \n",
       "____________________________________________________________________________\n",
       "                     64 x 32 x 1 x 1     \n",
       "AdaptiveAvgPool2d                                              \n",
       "AdaptiveMaxPool2d                                              \n",
       "____________________________________________________________________________\n",
       "                     64 x 64             \n",
       "Flatten                                                        \n",
       "BatchNorm1d                               128        True      \n",
       "Dropout                                                        \n",
       "____________________________________________________________________________\n",
       "                     64 x 10             \n",
       "Linear                                    640        True      \n",
       "____________________________________________________________________________\n",
       "\n",
       "Total params: 6,144\n",
       "Total trainable params: 6,144\n",
       "Total non-trainable params: 0\n",
       "\n",
       "Optimizer used: <function Adam at 0x000001BF07FB7670>\n",
       "Loss function: FlattenedLoss of CrossEntropyLoss()\n",
       "\n",
       "Callbacks:\n",
       "  - TrainEvalCallback\n",
       "  - Recorder\n",
       "  - ProgressCallback"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "learner.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:33:55.922396Z",
     "start_time": "2021-03-10T23:33:34.382215Z"
    },
    "id": "Zmxvv7LfTryG",
    "outputId": "6aa0246c-2332-40ae-f189-5ef3334a083c",
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/plain": [
       "SuggestedLRs(valley=0.001737800776027143)"
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAAwJklEQVR4nO3dd3gVZdrH8e+d3gMJAQIEQpVeA1J0F7GxiIi6guVVUNR1dXV1fV117V3X1d3XLrui7FoRG2LXxYYIBKSF3gktBQgJCSHlfv84B8WYCpnMKffnus7FOTPPmfmlcO7MPDPPI6qKMcaY4BXidgBjjDHuskJgjDFBzgqBMcYEOSsExhgT5KwQGGNMkLNCYIwxQS7M7QAN1aJFC01PT3c7hjHG+JVFixblqWpKdev8rhCkp6eTmZnpdgxjjPErIrKlpnV2asgYY4KcFQJjjAlyVgiMMSbI+V0fgTHG1KSsrIzs7GwOHjzodhTXREVF0a5dO8LDw+v9HisExpiAkZ2dTXx8POnp6YiI23GanKqSn59PdnY2HTt2rPf77NSQMSZgHDx4kOTk5KAsAgAiQnJycoOPiKwQBLD8olKWbtuHDTVugkmwFoHDjubrt1NDAaCy0vNBLwKq8O36PN5YuI1PV+6irELpkZrA70d25ow+qYSGBPd/EmN8TVxcHEVFRWzevJmxY8eyYsWKJs9ghcCPFR8q58n/rueFbzdxqLwS+KkYNIsJ5+Kh6XRKieXFuZu47rUfePzTNTxwdh9GdGnhcnJjfMSyGfDFvVCQDYnt4OQ7oe8Et1M1OSsEfkhV+XjFLu6bvZIdBQc5q38bOrWIo1KVSlW6tYrntF6tiAwLBeDCIe35dOUuHv1kDZdMW8BdZ/bkkmHp7n4Rxrht2Qx4/zooK/G8LtjmeQ3HVAxuueUW0tLSuOaaawC4++67CQsLY86cOezdu5eysjLuv/9+zjrrrBq3UVFRwS233MKXX35JaWkp11xzDb/73e+45JJLOOeccxg/fjwAF110ERMmTKh1W/VhhaAOh0+7hLh8SkVVWb69gM9W7uazlbtZvauQ7q3jeeKCAWSkJ9X63pAQYXTvVEZ0acH1ry/hzveyWLu7kLvO7EV4qHUTmSD1xb0/FYHDyko8y4+hEEycOJHrr7/+x0IwY8YMPvnkE6677joSEhLIy8tj6NChjBs3rsbz+S+88AKJiYksXLiQ0tJSRowYwWmnncaUKVP4+9//zvjx4ykoKOC7775j+vTpR531MCsENTj8V/f9H6yiY4tYpk0eTERY039oqiqzl+3kb5+uYUt+MSECGelJPHB2byZmpBHWgA/y+Khwpl6SwV8/Xs3zX29k7e4inrxgAK0Sohz8CozxUQXZDVteTwMGDCAnJ4cdO3aQm5tL8+bNad26NTfccANff/01ISEhbN++nd27d9O6detqt/Hpp5+ybNkyZs6c6YlUUMC6des47bTTuPrqq8nNzeWtt97i3HPPJSzs2D/GrRBUY2NuEXfNyuKbdXl0SI7h2/V53DUriwfP7t1kVySoKou37uX+D1bxw9Z99EhN4NHf9uXkHq1Iio046u2Ghgi3junBca3jue2dFYz5v294fGJ/ft3tp0EJVbXBX6eqUqlYZ7TxH4ntPKeDqlt+jM477zxmzpzJrl27mDhxIq+88gq5ubksWrSI8PBw0tPTa73EU1V58sknOf3003+x7pJLLuHll1/m9ddf58UXXzzmrGCF4BdW79rPWU/NJSI0hLvO7MnFQzvw2GdrefbLDXRvHc+k4ek/a6+qZO8tYe3uQqLCQxnaKbleH4YHyyrYlHeA9TlFbMk/wJb8YrbsKSa3sJSCkjIKSsqoqFRaxkfy13P7cu6gdo36IXvOwHb0aZvIH179gUnTFnDBkPaAsnJnIWt3FRIRFkKH5BjaJ8UQHxXOjn0lbN9XQs7+g8RHhZMSH0lKfCTlFZXs2HeQ7ftKOFhWQb+0ZgzrlMywzsl0SoklOTbSlSMpY+p08p0/7yMACI/2LD9GEydO5IorriAvL4+vvvqKGTNm0LJlS8LDw5kzZw5bttQ4ECgAp59+Os8++yyjRo0iPDyctWvX0rZtW2JjY5k8eTJDhgyhdevW9OzZ85izghWCX3j04zVEhoXw6Q2/pnWi55TJTacdx7rdhdw7eyWtE6OICA1hweY9ZG7ew6qdhRSVlv/4/jaJUZyXkcaYPqnsKChhRXYBWTv2s7f4EOWVSllFJQUlZWzbU0zlEZf3t4yPpENyDL3aJJAYHU5idDipzaI5Z0BbYiOd+TF1bRXPe38YwT3vZ/Hagq00iwmnR+sEJg5Oo6yikq17ilmWXcCB0nJSm0XROSWW4Z2TKTpYTk5hKVvyDxAWEkL75BiGdU4mIiyEBZv28OxXG3hqzvof95MYHU6bZtEc1yqObq3j6deuGcM7B+9NP8ZHHO4HcOCqoV69elFYWEjbtm1JTU3loosu4swzz6RPnz5kZGTQvXv3Wt9/+eWXs3nzZgYOHIiqkpKSwrvvvgtAq1at6NGjx48dxo1B/O1mo4yMDHVqPoKFm/dw3nPzuHl0d34/svPP1hWVlnPuM9+xZnchAGEhQp92ifRtm0i31vF0bx3ProJSXl+4lW/X53Hkt7Vji1hS4iMJDxXCQkKIiwqjc0ocXVrG0SUljvQWMcREuFuTD5SWExMR2igfzoUHy1i0ZS879h0kt7CUvKJStu4pZt3uQnYUeA6HbzilG388pesx78uYI61atYoePXq4HcNRxcXF9OnTh8WLF5OYmFhtm+q+DyKySFUzqmtvRwReqsojH62mZXwkk6uc/gGIiwzjpcsG896SHfRtl8iAtOZER4T+ot0ZfVPZtqeY7zbk0SE5ll5tEoiPqv/gT25pzKOO+KhwRh7Xstp1BSVl3DMri79/vpYuLeM4o29qo+3XmED3+eefM2XKFG644YYai8DRsELg9d/VOWRu2csDZ/eu9gMeIDUxmqt+3bnadUdKS4phYlL7xo4YEBKjw3no3D5s2VPMjW8uoX1SDH3aNd4vtDGB7JRTTqmzf+FoONaLJyJRIrJARJaKSJaI3FNNm/YiMkdEfhCRZSIyxqk8tamoVP768Ro6tohlQkaaGxGCSmRYKM9fPIjk2Egu//dCvt+Yz4rtBazauZ+cwuAdPtgYtzh5RFAKjFLVIhEJB74VkY9U9fsj2twOzFDVZ0WkJ/AhkO5gpmq9v3QHa3YX8tSFA+wGqybSIi6Sf03K4Nxnv+P8qT/9SoSGCOcMaMt1J3clLSnGxYTGXx3N5c+B5Gj6fR0rBOpJU+R9Ge59VE2oQIL3eSKww6k8tXlrcTYdW8Qypredr25KPVIT+OT6X7E+p4iyikoqKpWFm/fy8vwtvLtkOxMy0jhnYFv6tmt2zAW6slLZuqcYEWjbLLpBN+IZ/xEVFUV+fn7QDkV9eD6CqKiG3STqaB+BiIQCi4AuwNOqOr9Kk7uBT0XkWiAWOMXJPNUpPlTO/I17uGRYB9eHkQhGaUkxP/vL/zd9UrnyV514as463li4jVfmbyU2IpTjOyVzfMckBrRvTp+2iURHhFJyqII1uwtZu7uQTi1iGdi++Y8/Q1VlybZ9vL90J8u37/vZZb5hIUJaUgy92ybylzHdSU2MduVrN42vXbt2ZGdnk5ub63YU1xyeoawhmuTyURFpBrwDXKuqK45Y/idvhsdEZBjwAtBbVSurvP9K4EqA9u3bD2rMzpIvVu1myvRMXrn8eBuV08fsPXCIeRvzmbs+j+825LMp7wDgOX3UOiGKnQUlP7sXo22zaMb2TSUpNoKZi7JZl1NEZFgIvdsm0qtNAr3aJCAibMk/wOa8YuasySEyLITHJ/bnpBqucjImUNR2+WiT3UcgIncCxar6tyOWZQGjVXWb9/VGYKiq5tS0nca+j+D2d5fzzuLtLL7z1B9H6zS+Kc870c6SbfvYkl9Mxxax9EhNoEvLOJZv38esJTv4Zl0e5ZXKgPbNmJiRxhl9U2u8fHdDbhHXvLKY1bsK+d2vO3H9yd1qvGLMGH/nSiEQkRSgTFX3iUg08CnwiKrOPqLNR8AbqvqSiPQAvgDaai2hGrMQqConPDKHnm0S+Ocl1X5/jJ/Ze+AQRaXl9e5oPlhWwX2zV/LK/K1EhoUwrHMyJx3XktG9W9tgfCag1FYInOwxSwXmiMgyYCHwmarOFpF7RWSct82NwBUishR4DZhcWxFobOtziti+r8ROCwSQ5rERDbraKCo8lAfO7sMbVw7lwuPbsyW/mLtmZXHq418xf2O+g0mN8R1BPcTEP7/eyAMfruK7W0bRppl1GBqPNbsKuebVxWzNL+axCf04s18btyMZc8zcOiLweXPW5HBcq3grAuZnjmsdz1tXDad/+2Zc+9oPPPvlBg6WVbgdyxjHBG0hKDxYxsLNexjZPaXuxiboJMaE8+/LhnBG31Qe+Xg1fe/5lAumfs+TX6xjZ0FJ3Rswxo8EbSGYuz6fsgplZDfrHzDViwoP5cnzB/DSpYOZNKwD+w+W8fjnaxn/9FzW5xTVvQFj/ETQDjr31doc4iLDyEhv7nYU48NCQoSRx7X8cTTVVTv3c/ELC5j4/Dxevvx4eqQm1LEFY3xfUB4RqCpfrsnlxK4tbGwh0yA9UhN443dDCQ8N4fyp37N02z63IxlzzILyU3Bj3gF2FhzkhK52J7FpuM4pcbx51TASosM46+m5jP7H19z7/kq+WLWbQ+WVdW/AGB8TlKeG5m3wXB8+vLMVAnN00pJiePv3I5iRuY15G/J5Zf4Wps3dRJvEKKac2InzB6c5NsWoMY0tKH9T523Ip3VCFOnJNsyxOXop8ZFcc1IXrjmpCwfLKpi7Po/nv97IfbNX8uR/13HL6O6cP8QmKDK+L+gKQWWl8v3GfH7dLSUoh6k1zogKD+XkHq04uUcrFm3Zw6OfrOGWt5eTGB3Ob/rY8ObGtwVdH8HanELyDxxiaOdkt6OYADWoQxIvXTqEge2bcf0bS1hiHcrGxwVdITjcPzCskxUC45yo8FCmXpJBy4RILp+eyfZ9dhOa8V1BWQjSkqJtGkTjuBZxkUybNJjS8goue3Ehew8ccjuSMdUKqkJQ4e0fsKMB01S6tornuf8ZxKb8A0ycOo+c/QfdjmTMLwRVIVi1cz/7D5bbZaOmSY3o0oKXLh3M9r0l/Pa5eWzbU+x2JGN+JqgKwXcb8gAYZh3FpokN79yCly8/noKSMs57bh4bcm2sIuM7gqoQzNuQT6eUWJt5yrhiQPvmvPG7oZRXVjJp2gJyC0vdjmQM4GAhEJEoEVkgIktFJEtE7qmh3QQRWelt86pTecoqKlmwaY/1DxhXdW+dwLTJg8kvOsSU6QspPlTudiRjHD0iKAVGqWo/oD8wWkSGHtlARLoCtwIjVLUXcL1TYZZvL+DAoQo7LWRc17ddM568YAArthdw3WtLqKj0r1kCTeBxrBCox+EToeHeR9Xf+CuAp1V1r/c9OU7lKSgpo1NKLEPtiMD4gFN6tuLucb34fNVu7pu90u04Jsg5OsSEiIQCi4AueD7w51dp0s3bbi4QCtytqh9Xs50rgSsB2rc/urFbTjqupU1Sb3zKJcPS2ZxXzLS5mxjUobnNjWxc42hnsapWqGp/oB0wRER6V2kSBnQFRgIXAP8UkWbVbGeqqmaoakZKik0taQLHrWO6M6hDc259ezmb8w64HccEqSa5akhV9wFzgNFVVmUDs1S1TFU3AWvxFAZjgkJ4aAhPXDCA0BDhmlcXc7Cswu1IJgg5edVQyuG/7kUkGjgVWF2l2bt4jgYQkRZ4ThVtdCqTMb6obbNo/nZeP7J27OfBD1e5HccEISePCFKBOSKyDFgIfKaqs0XkXhEZ523zCZAvIivxHDHcpKr5DmYyxied2rMVU07oyL/nbWHmomy345ggI6r+delaRkaGZmZmuh3DmEZ3qLySyS8uYMGmPUy/bAgjuthQKKbxiMgiVc2obl1Q3VlsjC+LCAvhuYsH0Tkljqv+s4jVu/a7HckECSsExviQhKhwXrx0MDGRoVz64kJ2FdhopcZ5VgiM8TFtmkXz4uQhFB4s57KXFnKg1IahMM6yQmCMD+rZJoGnLxrImt2FXPfaDzYMhXGUFQJjfNSvu6Vw97hefLE6x4ahMI5ydIgJY8yxuXhoBzbnHeCFbzeRnhzD5BEd3Y5kApAdERjj4/4ypgen9mzFfR+sYnl2gdtxTACyQmCMjwsNEf52Xj+SYyP481vLKKuodDuSCTBWCIzxA4nR4dw/vjerdu7n+a82uB3HBBjrIzDGT5zWqzVn9E1l439fomzxO4QX7YDEdnDyndB3gtvxjB+zQmCMH3moy2rC104lvOiQZ0HBNnj/Os9zKwbmKNmpIWP8SMLcB4nm0M8XlpXAF/e6E8gEBCsExviTghpGJq1puTH1YIXAGH+S2K5hy42pBysExviTk++E8OifLTpIJOUn3eFSIBMInJyhLEpEFojIUhHJEpF7aml7roioiFQ7VrYxxqvvBDjzCUhMA4Ti6FT+fGgKD27r43Yy48ecvGqoFBilqkUiEg58KyIfqer3RzYSkXjgj8B8B7MYEzj6TvjxCqEYIGlWFtPmbqJ763gmDE5zN5vxS44dEahHkfdluPdR3RCK9wGPADbwujFH4fYzenBi1xbc9u5y5m+0mV5NwznaRyAioSKyBMjBM2fx/CrrBwJpqvpBHdu5UkQyRSQzNzfXucDG+KGw0BCeunAgaUkxXPXyIrbmF7sdyfgZRwuBqlaoan+gHTBERHofXiciIcDjwI312M5UVc1Q1YyUlBTH8hrjrxKjw5k2aTCVCpdNX0hBSZnbkYwfaZKrhlR1HzAHGH3E4nigN/CliGwGhgKzrMPYmKOT3iKW5/5nEFvyD/D7lxdxqNwGpzP14+RVQyki0sz7PBo4FVh9eL2qFqhqC1VNV9V04HtgnKpmOpXJmEA3rHMyj5zbl+825HPL28tQtZnNTN2cvGooFZguIqF4Cs4MVZ0tIvcCmao6y8F9GxO0zhnYjuy9JTz+2VrSmsdww6nd3I5kfJxjhUBVlwEDqll+Zw3tRzqVxZhgc+2oLmTvLeb/vlhHp5RYzurf1u1IxofZncXGBCAR4YGz+zA4vTl3vpdFbmGp25GMD7NCYEyACg8N4aFz+lJyqIJ73s9yO47xYVYIjAlgXVrG8YdRXZi9bCf/Xb3b7TjGR1khMCbAXfXrznRrFcft76ygqLTc7TjGB1khMCbARYR5ThHt3H+Qv32yxu04xgdZITAmCAzq0JxJw9J56bvNzFq6w+04xsdYITAmSNw6pjtD0pP43zeXsnjrXrfjGB9ihcCYIBEZFspzFw+idUIUV/47k+y9Njid8bBCYEwQSYqNYNrkDErLK5nyUqZ1HhvACoExQadLy3ieuWgg63IKeeCDVW7HMT7ACoExQejErilccWInXluwlW/X5bkdx7jMCoExQeqGU7vRKSWWm99aZqeIgpwVAmOCVFR4KH87rx87C0p48EM7RRTMrBAYE8QGtm/O5Sd24tX5dooomFkhMCbI/enUbnRqEcut7yyj5FCF23GMC5ycoSxKRBaIyFIRyRKRe6pp8ycRWSkiy0TkCxHp4FQeY0z1osJDefCcPmzbU8I/vljrdhzjAiePCEqBUaraD+gPjBaRoVXa/ABkqGpfYCbwVwfzGGNqMLRTMhMz0vjXN5tYuWO/23FME3OsEKhHkfdluPehVdrMUdXDtzd+D7RzKo8xpna3julO85hwbn1nORWVNtdxMHG0j0BEQkVkCZADfKaq82tpPgX4qIbtXCkimSKSmZub60BSY0yzmAjuGNuTpdv28fL3W9yOY5qQo4VAVStUtT+ev/SHiEjv6tqJyP8AGcCjNWxnqqpmqGpGSkqKY3mNCXbj+rXhxK4tePSTNWzfV+J2HNNEmuSqIVXdB8wBRlddJyKnALcB41TVJlY1xkUiwoNn96FSlVvfXo6qnSIKBk5eNZQiIs28z6OBU4HVVdoMAJ7HUwRynMpijKm/tKQYbh7dna/X5vJmZrbbcUwTqFchEJFYEQnxPu8mIuNEJLyOt6UCc0RkGbAQTx/BbBG5V0TGeds8CsQBb4rIEhGZdZRfhzGmEV08tANDOiZx3wcr2Vlgp4gCndTn0E9EFgEnAs2BuXg+2A+p6kXOxvuljIwMzczMbOrdGhN0NucdYPT/fc2wTslMmzwYEXE7kjkGIrJIVTOqW1ffU0PivczzHOAZVT0P6NVYAY0xvie9RSx/Pr07c9bk8t4Sm94ykNW7EIjIMOAi4APvslBnIhljfMXk4en0a5fIgx+u4oCNUBqw6lsIrgduBd5R1SwR6YTnKiBjTAALCRHuPLMXOYWlPPPlerfjGIfUqxCo6leqOk5VH/F2Guep6nUOZzPG+IBBHZozvn8b/vnNJrbtsXmOA1F9rxp6VUQSRCQWWAGsFJGbnI1mjPEVN/+mO6EiNm9BgKrvqaGeqrofGI9nGIiOwMVOhTLG+JbUxGiuHtmZj1bsYt6GfLfjmEZW30IQ7r1vYDwwS1XLqDKAnDEmsF3xq060bRbNPe9nUVZR6XYc04jqWwieBzYDscDX3nkDbKxaY4JIVHgod4ztyepdhbw0d7PbcUwjqm9n8ROq2lZVx3iHl94CnORwNmOMjzm9VytO7t6Sv3++1galCyD17SxOFJHHDw8FLSKP4Tk6MMYEERHh7nG9qFTlnllZbscxjaS+p4amAYXABO9jP/CiU6GMMb4rLSmGP57cjU9X7uazlbvdjmMaQX0LQWdVvUtVN3of9wCdnAxmjPFdU07oSNeWcdw9K8vuOA4A9S0EJSJywuEXIjICsBOExgSpiLAQHji7DzsKSnjA7i3we2H1bHcV8G8RSfS+3gtMciaSMcYfDOmYxOUndOSf32zilB4tGdW9lduRzFGq71VDS1W1H9AX6KuqA4BRjiYzxvi8/z39OLq3jufPM5eTX2QTDPqrBs1Qpqr7vXcYA/yptrYiEiUiC0RkqYhkicg91bSJFJE3RGS9iMwXkfSG5DHGuCsyLJR/nN+f/SVl3GJTW/qtY5mqsq5ZKkqBUd4jif7AaBEZWqXNFGCvqnYB/g48cgx5jDEu6N46gZtOP47PVu7mzUU2taU/OpZCUGvp9954VuR9Ge59VH3PWcB07/OZwMli0yAZ43emnNCRjA7N+evHqymyq4j8Tq2FQEQKRWR/NY9CoE1dGxeRUBFZAuTgmbN4fpUmbYFtAKpaDhQAydVs58rDN7Pl5ubW7yszxjSZkBDh9rE9ySs6xNSvN7odxzRQrYVAVeNVNaGaR7yq1nnFkapWqGp/oB0wRER6H01IVZ2qqhmqmpGSknI0mzDGOKx/WjPO6JvKP7/eyO79B92OYxrgWE4N1Zuq7sMzo9noKqu2A2kAIhIGJAI2xq0xfurPpx9HeWUl//h8rdtRTAM4VghEJEVEmnmfRwOnAqurNJvFT/cj/Bb4r9plB8b4rQ7JsfzP0A68sXAb63YXuh3H1JOTRwSpwBwRWQYsxNNHMFtE7hWRcd42LwDJIrIez+WotziYxxjTBK4d1ZXYiDAe+qjq333GV9X3zuIGU9VlwIBqlt95xPODwHlOZTDGNL2k2AiuPbkLD364mveX7uDMfnVeV2Jc1iR9BMaY4HLZiI70S2vGHe+tIMc6jn2eFQJjTKMLCw3hsfP6UXKowu449gNWCIwxjujSMo6bR3fnv6tzeDPT7jj2ZVYIjDGOmTw8naGdkrh39kqy9xa7HcfUwAqBMcYxISHCo7/th6py2zsr7BSRj7JCYIxxVFpSDDeedhxfrc3l/WU73Y5jqmGFwBjjuEnD0+nbLpF7389iX/Eht+OYKqwQGGMcFxoiPHROH/YWl/HQh3ajma+xQmCMaRK92iRy+YkdeSNzG/M22JBivsQKgTGmyVx/cjfSkqK57Z3llJZXuB3HeFkhMMY0meiIUO4f34eNeQd47kubt8BXWCEwxjSpX3dLYWzfVJ7+cj2b8g64HcdghcAY44I7x/YkMjSE29+14Sd8gRUCY0yTa5kQxU2jj2Pu+nxmLd3hdpygZ4XAGOOKi47vQL92idw3eyUFxWVuxwlqTs5QliYic0RkpYhkicgfq2mTKCLvi8hSb5tLncpjjPEtoSHCA2f3Yc+BQzz6qd1b4CYnjwjKgRtVtScwFLhGRHpWaXMNsFJV+wEjgcdEJMLBTMYYH9K7bSKThqfzyvytLNm2z+04QcuxQqCqO1V1sfd5IbAKaFu1GRAvIgLEAXvwFBBjTJD406ndaBkfyW3vLKe8otLtOEGpSfoIRCQdz7SV86usegroAewAlgN/VFX7TTAmiMRHhXPH2J5k7djPf77f4nacoOR4IRCROOAt4HpV3V9l9enAEqAN0B94SkQSqtnGlSKSKSKZubm5Dic2xjS1M/qkcmLXFjz26Vp229SWTc7RQiAi4XiKwCuq+nY1TS4F3laP9cAmoHvVRqo6VVUzVDUjJSXFycjGGBeICPed1ZtDFZXc+Z7NW9DUnLxqSIAXgFWq+ngNzbYCJ3vbtwKOA+y+c2OCUHqLWG44pRufZO3mg+U2b0FTcvKIYARwMTBKRJZ4H2NE5CoRucrb5j5guIgsB74AblbVPAczGWN82BUndqRvu0Tuei+L/KJSt+MEDfG3Q7CMjAzNzMx0O4YxxiGrd+3nzCe/5Te9U3niggFuxwkYIrJIVTOqW2d3FhtjfEr31gn84aSuzFq6g0+zdrkdJyhYITDG+JyrT+pMj9QE/vLOcnLsKiLHWSEwxvic8NAQ/u/8/hworeCaVxdTZjeaOcoKgTHGJ3VrFc/D5/Zh4ea9PPKRjUXkJCsExhifdVb/tkwa1oF/fbuJD5bZJaVOsUJgjPFpt53RkwHtm/HnmUvZmFvkdpyAZIXAGOPTIsJCeOaigYSFhnDDjKU2MJ0DrBAYY3xeamI094/vzdJt+3jmyw1uxwk4VgiMMX7hzH5tGNevDU98sY7l2QVuxwkoVgiMMX7jvrN60yIukuvf+IGDZRVuxwkYVgiMMX4jMSacR8/ry4bcAzz04Sq34zSpJdv2UXjQmbmdrRAYY/zKiV1TuGxER6bP28LHK4JjCIpD5ZWcP3Uej3261pHtWyEwxvidW37Tnb7tErlp5lK27Sl2O47jlm8v4GBZJcd3THJk+1YIjDF+JyIshKcuGAgKf3h1MYfKA/uS0gWb9gAw2AqBMcb8pH1yDH/9bV+WZhfwcIAPQbFgUz5dWsbRIi7Ske07OUNZmojMEZGVIpIlIn+sod1I76Q1WSLylVN5jDGB5zd9Upk0rAPT5m4K2P6Cikolc/Nehjh0NAAQ5tiWoRy4UVUXi0g8sEhEPlPVlYcbiEgz4BlgtKpuFZGWDuYxxgSgv5zRgyXb9nHTm0vpkRpPh+RYtyM1qlU791NYWu5Y/wA4eESgqjtVdbH3eSGwCmhbpdmFeCav3+ptl+NUHmNMYIoMC+WpCwcSEiJc/crigLu/YL63f8DJI4Im6SMQkXRgADC/yqpuQHMR+VJEFonIJU2RxxgTWNKSYnh8Qj+yduznnvdX1v0GP7JgUz7tk2JITYx2bB+OFwIRiQPeAq5X1f1VVocBg4AzgNOBO0SkWzXbuFJEMkUkMzc31+nIxhg/dHKPVvx+ZGdeW7CV95ZsdztOo1BVFmza4+jRADhcCEQkHE8ReEVV366mSTbwiaoeUNU84GugX9VGqjpVVTNUNSMlJcXJyMYYP3bjqd0Y1KE5t7+7gp0FJW7HOWbrc4rYW1zmv4VARAR4AVilqo/X0Ow94AQRCRORGOB4PH0JxhjTYGGhITw+oR8VlcpNby6jslLdjnRMvvf2DzjZUQzOHhGMAC4GRnkvD10iImNE5CoRuQpAVVcBHwPLgAXAv1R1hYOZjDEBrkNyLLef0ZNv1+fx73mb3Y5zTBZs2kPrhCjaJ8U4uh/HLh9V1W8BqUe7R4FHncphjAk+FwxJ4/NVu3noo9Wc0DWFLi3j3I7UYJ7+gXyO75iM5wSLc+zOYmNMwBERHj63DzERodz45lIq/PAU0dY9xezeX+p4/wBYITDGBKiW8VHcPa4XS7ft4z9+eIros5W7ARjaKdnxfVkhMMYErHH92vCrbik8+skaduzzn6uIKiqV6fM2k9GheZOc1rJCYIwJWCLCA+N7U6HKXbOy3I5Tb5+t3M22PSVMOaFjk+zPCoExJqClJcVw/Snd+Gzlbr8ZmG7at5to1zya03q1bpL9WSEwxgS8KSd0pHvreO6atcKx6R4by/LsAhZs3sPk4emEhjh7tdBhVgiMMQEvPDSEh8/tS05hKX/9eI3bcWr1wrcbiY0IZcLgtCbbpxUCY0xQ6J/WjEnD0nl5/hYWbdnjdpxq7d5/kNnLdjJhcBoJUeFNtl8rBMaYoPG/px9HakIUt7y13Cent/zPvC1UqHLp8KbpJD7MCoExJmjERYZx/9m9WZdTxHNfbXA7zi98uGInJ3ZNoX2ys0NKVGWFwBgTVEZ1b8XYvqk89d/1rM8pcjvOj3L2H2Rj7gFO6OL8DWRVWSEwxgSdu87sRXREKDfOWOIzp4jm/zjSqBUCY4xxXEp8JA+f04el2QU8/tlat+MA8P3GfOIiw+jVJqHJ922FwBgTlH7TJ5ULj2/Pc19t4Jt17s98OH/THjLSmxMW2vQfy1YIjDFB644zetK1ZRx/mrGUvKJS13LkFpayPqeoSQaYq44VAmNM0IqOCOXJCwdQUFLGTW8uRdWd4aoXNNFMZDVxcqrKNBGZIyIrRSRLRP5YS9vBIlIuIr91Ko8xxlSne+sEbhndnTlrcpm9bKcrGb7fmE9sRCi92ya6sn8njwjKgRtVtScwFLhGRHpWbSQiocAjwKcOZjHGmBpNGp5O77YJ3Dd7pStjEc3flM+g9CTCXegfAAcLgaruVNXF3ueFeCalb1tN02uBt4Acp7IYY0xtQkOEB8b3IbeolL9/tq5J951fVMra3UUM7eTOaSFooj4CEUkHBgDzqyxvC5wNPFvH+68UkUwRyczNdb933xgTePqlNeOi49vz0nebyNpR0GT7XeDi/QOHOV4IRCQOz1/816vq/iqr/wHcrKq13tGhqlNVNUNVM1JSUhxKaowJdjed1p3mMRHc/u4KKptonuPvN+YTHR5K33bu9A+Aw4VARMLxFIFXVPXtappkAK+LyGbgt8AzIjLeyUzGGFOTxJhw/jKmBz9s3ce0uZuaZJ+H7x9wq38AnL1qSIAXgFWq+nh1bVS1o6qmq2o6MBO4WlXfdSqTMcbU5ZyBbTm9Vyse/mg1mZudHa56x74SVu8qdO2y0cOcLEEjgIuBUSKyxPsYIyJXichVDu7XGGOOmojw6Hn9aNs8mmteXezojWaPf7aWiNAQzupf3XU0TSfMqQ2r6rdAvedZU9XJTmUxxpiGSIgK59mLBnH2M3P54+s/8O/Ljm/0aSNX7tjPW4uzueLETqQlNe2w01XZncXGGFONnm0SuG98b+auz+fe97OoaOTO44c+WkVidDjXjOzSqNs9Go4dERhjjL+bkJHG6p2FTJu7iQ25B3jiggEkxUYc83a/WpvLN+vyuGNsTxJjmm5KyprYEYExxtTizjN78vA5fViweQ9jn/iGJdv2HdP2KiqVhz5cRfukGC4e2qFxQh4jOyIwxpg6nD+kPb3aJHLVy4sY//Rc0pNjGNihOYM6NGdM71SaN+Ao4ZX5W1i9q5CnLhxARJhv/C0ubo22d7QyMjI0MzPT7RjGmCC098AhZmRuY9GWvSzeuo+8olKaxYRz8+juTMxII6SODuXvN+Zz8QvzGda5BdMvHYznKvumISKLVDWj2nVWCIwxpuFUlawd+7l39koWbNpDv3aJ3D2uFwPaN6+2/db8Ys56+luax0bwztUjSIxu2r6B2gqBbxyXGGOMnxERerdN5I0rh/KPif3ZUXCQs5/5jktfXPCLfoTCg2VMmb6QSoUXJg1u8iJQFzsiMMaYRlBUWs707zbzz282sq+4jOM7JpEYHU5JWQVb9xSzfW8J/75sCMO7tHAln50aMsaYJlJUWs7L32/hrUXZhIYI0RGhxEaEcf6QNMb2beNaLisExhgT5KyPwBhjTI2sEBhjTJCzQmCMMUHOCoExxgQ5KwTGGBPkrBAYY0yQs0JgjDFBzgqBMcYEOb+7oUxEcoEt3peJQEEtz6v+2wLIa+Auj9xufddXXVZXzuoyNzRrY+esLlNj5Kwra31yVl3mxs++pnUN/dlXt8ztn71Tv6N1ZT2W39Hq8vnLz76pcnZQ1ZRqW6qq3z6AqbU9r+bfzGPZR33XV11WV87GyNrYOavL1BTf0/rk9IWffU3rGvqzr2GZqz97p35HG+NnX5/c/vazb+qc1T38/dTQ+3U8r/rvse6jvuurLqsr55HPjzZrY+c88nVj5qzrvfXJWXWZGz/7mtY19Gdf0/qG8Jff0breeyy/o0c+97effVPn/AW/OzV0LEQkU2sYa8PX+EtWy9n4/CWr5Wxcbub09yOChprqdoAG8JeslrPx+UtWy9m4XMsZVEcExhhjfinYjgiMMcZUYYXAGGOCnBUCY4wJclYIvETkRBF5TkT+JSLfuZ2nJiISIiIPiMiTIjLJ7Ty1EZGRIvKN9/s60u08tRGRWBHJFJGxbmepiYj08H4vZ4rI793OUxsRGS8i/xSRN0TkNLfz1EREOonICyIy0+0sVXl/J6d7v48XObmvgCgEIjJNRHJEZEWV5aNFZI2IrBeRW2rbhqp+o6pXAbOB6b6aEzgLaAeUAdlO5GzErAoUAVFOZW2knAA3AzOcyOjN0xi/o6u8v6MTgBE+nvVdVb0CuAqY6MM5N6rqFCfyVaeBmc8BZnq/j+McDdbQO9l88QH8ChgIrDhiWSiwAegERABLgZ5AHzwf9kc+Wh7xvhlAvK/mBG4Bfud970xf/p4CId73tQJe8eGcpwLnA5OBsb6a0/ueccBHwIW+/LM/4n2PAQP9IKdj/5eOIfOtQH9vm1edzBVGAFDVr0UkvcriIcB6Vd0IICKvA2ep6kNAtYf/ItIeKFDVQl/NKSLZwCHvywoncjZW1iPsBSJ9Naf3tFUsnv98JSLyoapW+lpO73ZmAbNE5APg1cbM2JhZRUSAh4GPVHWxr+Zsag3JjOcouh2wBIfP3gREIahBW2DbEa+zgePreM8U4EXHElWvoTnfBp4UkROBr50MVo0GZRWRc4DTgWbAU44m+7kG5VTV2wBEZDKQ19hFoBYN/X6OxHO6IBL40Mlg1Wjo7+m1wClAooh0UdXnnAx3hIZ+T5OBB4ABInKrt2A0tZoyPwE8JSJncGzDUNQpkAtBg6nqXW5nqIuqFuMpWD5PVd/GU7j8gqq+5HaG2qjql8CXLseoF1V9As8HmU9T1Xw8/Rg+R1UPAJc2xb4CorO4BtuBtCNet/Mu8zX+khP8J6vlbHz+ktVfch7J9cyBXAgWAl1FpKOIRODpDJzlcqbq+EtO8J+slrPx+UtWf8l5JPczN0VPeRP0xL8G7OSnSyqneJePAdbi6ZG/zXIGXlbLGbxZ/SWnP2S2QeeMMSbIBfKpIWOMMfVghcAYY4KcFQJjjAlyVgiMMSbIWSEwxpggZ4XAGGOCnBUCExBEpKiJ99coc1aIZ86GAhFZIiKrReRv9XjPeBHp2Rj7NwasEBhTLRGpdRwuVR3eiLv7RlX7AwOAsSJS11wD4/GMlGpMo7BCYAKWiHQWkY9FZJF4Zkrr7l1+pojMF5EfRORzEWnlXX63iPxHROYC//G+niYiX4rIRhG57ohtF3n/HeldP9P7F/0r3iGYEZEx3mWLROQJEZldW15VLcEz5HBb7/uvEJGFIrJURN4SkRgRGY5nToJHvUcRnWv6Oo2pLysEJpBNBa5V1UHA/wLPeJd/CwxV1QHA68Cfj3hPT+AUVb3A+7o7nqG0hwB3iUh4NfsZAFzvfW8nYISIRAHPA7/x7j+lrrAi0hzoyk/Di7+tqoNVtR+wCs9wBN/hGYfmJlXtr6obavk6jakXG4baBCQRiQOGA296/0CHnybHaQe8ISKpeGaE2nTEW2d5/zI/7ANVLQVKRSQHz2xrVafdXKCq2d79LgHS8UzRuVFVD2/7NeDKGuKeKCJL8RSBf6jqLu/y3iJyP575HOKATxr4dRpTL1YITKAKAfZ5z71X9STwuKrO8k72cvcR6w5UaVt6xPMKqv8/U582tflGVceKSEfgexGZoapLgJeA8aq61Dtpzshq3lvb12lMvdipIROQVHU/sElEzgPP1Iki0s+7OpGfxnuf5FCENUCnI6YlrHMCd+/Rw8PAzd5F8cBO7+moi45oWuhdV9fXaUy9WCEwgSJGRLKPePwJz4fnFO9plyw888CC5wjgTRFZBOQ5EcZ7eulq4GPvfgqBgnq89TngV94CcgcwH5gLrD6izevATd7O7s7U/HUaUy82DLUxDhGROFUt8l5F9DSwTlX/7nYuY6qyIwJjnHOFt/M4C8/pqOfdjWNM9eyIwBhjgpwdERhjTJCzQmCMMUHOCoExxgQ5KwTGGBPkrBAYY0yQs0JgjDFB7v8BFQ7Hb+8t6+8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.lr_find()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-03-10T23:34:12.533913Z",
     "start_time": "2021-03-10T23:33:55.924291Z"
    },
    "id": "MYoK6Rw_TryG",
    "outputId": "a3bd6daa-ef8a-4fd6-8dd3-9f4174c39e0e"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: left;\">\n",
       "      <th>epoch</th>\n",
       "      <th>train_loss</th>\n",
       "      <th>valid_loss</th>\n",
       "      <th>accuracy</th>\n",
       "      <th>time</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <td>0</td>\n",
       "      <td>3.620842</td>\n",
       "      <td>8.403188</td>\n",
       "      <td>0.095800</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>1</td>\n",
       "      <td>2.965809</td>\n",
       "      <td>6.965024</td>\n",
       "      <td>0.095800</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>2</td>\n",
       "      <td>2.619823</td>\n",
       "      <td>4.851040</td>\n",
       "      <td>0.195800</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>3</td>\n",
       "      <td>2.401922</td>\n",
       "      <td>6.368920</td>\n",
       "      <td>0.113500</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>4</td>\n",
       "      <td>2.197757</td>\n",
       "      <td>8.566344</td>\n",
       "      <td>0.113500</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>5</td>\n",
       "      <td>2.044650</td>\n",
       "      <td>10.258311</td>\n",
       "      <td>0.113500</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>6</td>\n",
       "      <td>1.912052</td>\n",
       "      <td>10.763972</td>\n",
       "      <td>0.113500</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>7</td>\n",
       "      <td>1.817624</td>\n",
       "      <td>10.316189</td>\n",
       "      <td>0.113500</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>8</td>\n",
       "      <td>1.710081</td>\n",
       "      <td>10.014054</td>\n",
       "      <td>0.113500</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <td>9</td>\n",
       "      <td>1.624749</td>\n",
       "      <td>9.580349</td>\n",
       "      <td>0.113700</td>\n",
       "      <td>00:02</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "learner.fit_one_cycle(10, lr_max=3e-2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aPcFOe5XTryG"
   },
   "source": [
    "> Important: We were able to only get 10.3% accuracy with training from scratch"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7bQBVMDVTryG"
   },
   "source": [
    "# Conclusion"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VhQdjIASTryG"
   },
   "source": [
    "> Important: Using self-supervision can help learn features that can transfer to a down-stream task, such as classification! In this example, we used rotation predication as our pretext task for feature representation learning. Pretraining our model on rotation prediction prior to training for classification allowed us to achieve 60.6% accuracy, on just 0.3% of the labeled data (180 samples)! Training from scratch with the same amount of data yields an accuracy of 10.3%. The motivation for using self-supervised learning is the ability to train models with decent accuracy without the need of much labeled data!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "hxL-6xXWTryG"
   },
   "source": [
    "> Note: Be sure to try other self-supervised learning methods (or perhaps find your own!) and compete on the [ImageWang Leadboard](https://github.com/fastai/imagenette#image%E7%BD%91-leaderboard)! How will model size, data difficultly, and dataset size (number of samples) affect self-supervised learning?"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "2020-03-23-Self-Supervision-with-FastAI.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}