E1/E1_Rock_Scissor_Paper_Image_Classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "serious-motion",
   "metadata": {},
   "source": [
    "## Exploration_01\n",
    "## 가위 바위 보 이미지 분류문제 (Accuracy = 98.37%)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "muslim-huntington",
   "metadata": {},
   "source": [
    "### 1. 문제 정의\n",
    "가위,바위,보 이미지를 학습하여 테스트 이미지를 입력 받았을 때\n",
    "\n",
    "\n",
    "가위(0), 바위(1), 보(2)의 클래스를 올바르게 분류하는 모델을 만들어보자!\n",
    "\n",
    "### 2. 데이터 수집 및 정의\n",
    "#### 수집\n",
    "- 웹캠을 사용해 가위,바위,보 각각 100장씩 촬영한 이미지 파일을 폴더에 저장한다.\n",
    "- 데이터를 추가로 확보하기 위해 Notion에 공유된 데이터를 추가하여 학습에 사용하였다.\n",
    "- 데이터 공유 해주신 모든 분들께 너무 너무 감사드립니다.\n",
    "- 학습에 사용된 데이터는 4286장으로 가위 1427장.바위 1432장,보 1427장이다.\n",
    "\n",
    "#### 정의\n",
    "- dir_path = ~/aiffel/rock_scissor_paper/\n",
    "- dir_path 하위 디렉토리로 scissor, rock, paper가 있고 그 안에 이미지들이 저장되어있다.\n",
    "- image shape는 (224,224)로 mnist 이미지 사이즈와 같은 (28,28)로 resizing 되어 압축된다.\n",
    "- .jpg 확장자의 이미지 파일이다.\n",
    "- color image (channel=3)\n",
    "\n",
    "### 3. 데이터 전처리\n",
    "1. 카테고리(가위,바위,보)별로 저장된 이미지 데이터를 모두 target_size=(28,28)로 resizing한다.\n",
    "2. img를 JPEG으로 변환한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "editorial-struggle",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "scissor 1427 images to be resized.\n",
      "scissor 1427 images resized.\n",
      "\n",
      "rock 1432 images to be resized.\n",
      "rock 1432 images resized.\n",
      "\n",
      "paper 1427 images to be resized.\n",
      "paper 1427 images resized.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from PIL import Image\n",
    "import os\n",
    "import glob\n",
    "\n",
    "category_list = ['scissor', 'rock', 'paper'] # '가위', '바위', '보'\n",
    "number_of_data = 0 #전체 데이터 수\n",
    "\n",
    "def resize_images(category, img_path):\n",
    "    images=glob.glob(img_path + \"/*.jpg\")\n",
    "    print(f\"{category} {len(images)} images to be resized.\")\n",
    "    # 파일마다 모두 28x28 사이즈로 바꾸어 저장합니다.\n",
    "    target_size=(28,28)\n",
    "    for img in images:\n",
    "        old_img=Image.open(img)\n",
    "        new_img=old_img.resize(target_size,Image.ANTIALIAS)\n",
    "        new_img.save(img, \"JPEG\")\n",
    "        \n",
    "    print(f\"{category} {len(images)} images resized.\\n\")\n",
    "    \n",
    "for category in category_list:\n",
    "    dir_path = os.getenv(\"HOME\") + \"/aiffel/rock_scissor_paper/\"\n",
    "    image_dir_path = os.path.join(dir_path, category)\n",
    "    number_of_data += len(os.listdir(image_dir_path)) # 각 폴더에 있는 이미지들의 개수를 더한다.\n",
    "    resize_images(category, image_dir_path)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "weekly-dynamics",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4286\n"
     ]
    }
   ],
   "source": [
    "print(number_of_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "integrated-pound",
   "metadata": {},
   "source": [
    "#### 3. 함수 load_data를 정의\n",
    "```\n",
    "def load_data(img_path, number_of_data)\n",
    "    \"\"\"\n",
    "    이미지가 저장된 폴더 경로를 받아 이미지 데이터와 정답 데이터를 담을 행렬을 생성하여\n",
    "    반환한다.\n",
    "    \n",
    "    params \n",
    "    img_path : 이미지가 담겨 있는 폴더 경로\n",
    "    number_of_data : 총 데이터 수 \n",
    "    \n",
    "    return\n",
    "    imgs: 이미지데이터 (number_of_data, 28,28,3) dtype=np.int32\n",
    "    labels : 정답데이터 (number_of_data,) 가위:0, 바위:1, 보:2로 라벨링, dtype=np.int32\n",
    "    \"\"\"\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "productive-conspiracy",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "학습데이터 이미지 개수는 4286 입니다.\n",
      "img shape: (4286, 28, 28, 3)\n",
      "label shape: (4286,)\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "def load_data(img_path, number_of_data):  # 가위바위보 이미지 개수 총합에 주의하세요.\n",
    "    # 가위 : 0, 바위 : 1, 보 : 2\n",
    "    img_size=28\n",
    "    color=3\n",
    "    #이미지 데이터와 라벨(가위 : 0, 바위 : 1, 보 : 2) 데이터를 담을 행렬(matrix) 영역을 생성합니다.\n",
    "    imgs=np.zeros(number_of_data*img_size*img_size*color,dtype=np.int32).reshape(number_of_data,img_size,img_size,color)\n",
    "    labels=np.zeros(number_of_data,dtype=np.int32)\n",
    "\n",
    "    idx=0\n",
    "    for file in glob.iglob(img_path+'/scissor/*.jpg'):\n",
    "        img = np.array(Image.open(file),dtype=np.int32)\n",
    "        imgs[idx,:,:,:]=img    # 데이터 영역에 이미지 행렬을 복사\n",
    "        labels[idx]=0   # 가위 : 0\n",
    "        idx=idx+1\n",
    "\n",
    "    for file in glob.iglob(img_path+'/rock/*.jpg'):\n",
    "        img = np.array(Image.open(file),dtype=np.int32)\n",
    "        imgs[idx,:,:,:]=img    # 데이터 영역에 이미지 행렬을 복사\n",
    "        labels[idx]=1   # 바위 : 1\n",
    "        idx=idx+1  \n",
    "    \n",
    "    for file in glob.iglob(img_path+'/paper/*.jpg'):\n",
    "        img = np.array(Image.open(file),dtype=np.int32)\n",
    "        imgs[idx,:,:,:]=img    # 데이터 영역에 이미지 행렬을 복사\n",
    "        labels[idx]=2   # 보 : 2\n",
    "        idx=idx+1\n",
    "            \n",
    "    print(\"학습데이터 이미지 개수는\", idx,\"입니다.\")\n",
    "    \n",
    "    return imgs, labels\n",
    "\n",
    "(imgs, labels) = load_data(dir_path, number_of_data)\n",
    "imgs = imgs / 255.0   # 입력은 0~1 사이의 값으로 정규화\n",
    "\n",
    "print(\"imgs shape: {}\".format(imgs.shape))\n",
    "print(\"labels shape: {}\".format(labels.shape))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "suited-night",
   "metadata": {},
   "source": [
    "#### 3. 함수 custom_train_test_split 정의\n",
    "-----\n",
    "```\n",
    "imgs, labels 데이터는 가위, 바위, 보의 순서로 이미지와 클래스가 저장되어있다.\n",
    "예를들어 imgs = [가위1,가위2, .... 가위1427, 바위1, ~], labels = [0,0,0,....., 1,1, ]\n",
    "테스트 데이터를 무작위 샘플링하기 위해 imgs, labels의 순서를 랜덤하게 shuffle하여\n",
    "테스트 데이터를 추출한다.\n",
    "```\n",
    "---------\n",
    "```\n",
    "def custom_train_test_split(imgs, labels, test_size=0.2, shuffle=True, random_state=99)\n",
    "    \"\"\"\n",
    "    sklearn.model_selection의 train_test_split과 비슷하게 동작한다.\n",
    "    np.random.permutation()의 인자로 number_of_data를 넣어 인덱스가 random하게 shuffle되어\n",
    "    test 데이터를 simple random sampling 할 수 있게 된다.\n",
    "    \n",
    "    params\n",
    "    imgs:         이미지데이터 (4D array)\n",
    "    labels:       정답데이터 (1D array)\n",
    "    test_size:    테스트 데이터 비율 지정\n",
    "    shuffle:      shuffle 실행유무(True/False)\n",
    "    random_state: random_seed값을 고정\n",
    "    \n",
    "    return\n",
    "    (X_train, y_train), (X_test, y_test)\n",
    "    \"\"\"\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "honest-crime",
   "metadata": {},
   "outputs": [],
   "source": [
    "# imgs: 4D array, (number_of_data, 28, 28, 3)\n",
    "# labels: 1D array, (number_of_data,)\n",
    "def custom_train_test_split(imgs, labels, test_size=0.2, shuffle=True, random_state=99):\n",
    "    test_num = int(number_of_data * test_size)\n",
    "    train_num = number_of_data - test_num\n",
    "    \n",
    "    if shuffle: \n",
    "        np.random.seed(random_state)\n",
    "        shuffled = np.random.permutation(number_of_data)\n",
    "        X = imgs[shuffled,:,:,:]\n",
    "        y = labels[shuffled]\n",
    "        X_train = X[:train_num]\n",
    "        y_train = y[:train_num]\n",
    "        X_test = X[train_num:]\n",
    "        y_test = y[train_num:]\n",
    "    else:\n",
    "        X_train = X[:train_num]\n",
    "        y_train = y[:train_num]\n",
    "        X_test = X[train_num:]\n",
    "        y_test = y[train_num:]\n",
    "        \n",
    "    return (X_train, y_train), (X_test, y_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "given-northwest",
   "metadata": {},
   "outputs": [],
   "source": [
    "# 훈련, 테스트 데이터 생성\n",
    "(X_train, y_train), (X_test, y_test) = custom_train_test_split(imgs, labels)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "residential-estimate",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3429, 28, 28, 3) (3429,)\n"
     ]
    }
   ],
   "source": [
    "# 훈련, 테스트 데이터 shape 확인\n",
    "print(X_train.shape, y_train.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "independent-relation",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(857, 28, 28, 3) (857,)\n"
     ]
    }
   ],
   "source": [
    "# 전체 데이터 중 20%를 테스트 데이터로 추출()\n",
    "print(X_test.shape, y_test.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "viral-google",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAqsAAAFOCAYAAAChelRdAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAkCklEQVR4nO3dSYxl53ne8fc7w53rVlVXV3V1N7vZzZmiKGpmKMsSKEtwrAQBjAwIshESBMkmCy+MwMguQHbZZkTiAIHhIFDmKA4QB8nCcRzZhCVzEmd2s6dqsrprvPOZsqASWAKr9LwQyf6s+v+W9IOvTp37ne+8dWXyCU3TGAAAABCj5F5fAAAAAHAUhlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABAthlUAAABEi2EVAAAA0WJYxccuhHAqhPAfQgjjEMI7IYS/cq+vCQDwwUII7RDCr//wvD4MIfxRCOGX7vV14eTI7vUF4ET6h2a2MLMzZvZpM/utEMLzTdO8fE+vCgDwQTIzu25mXzWza2b2TTP7dgjhyaZprt7LC8PJEGiwwscphNA3s10z+2TTNK//8J/9hpndbJrm1+7pxQEAJCGEF8zs7zZN8+/u9bXgZx//bwD4uD1iZuX/G1R/6Hkze+IeXQ8AwCGEcMbeP8v5X8PwsWBYxcdtYGYHP/bP9s1s6R5cCwDAIYSQm9lvmtm/bJrm1Xt9PTgZGFbxcRuZ2fDH/tnQzA7vwbUAAEQhhMTMfsPe/3cO/tY9vhycIAyr+Li9bmZZCOHhP/bPnjL+5yQAiFYIIZjZr9v7/2Lsn2+aprjHl4QThH/BCh+7EMK/NrPGzP66vf9fA/ivZvYl/msAABCnEMI/sffP6683TTO6x5eDE4ZhFR+7EMIpM/sXZvYNM7trZr/WNM2/urdXBQD4ICGE+83sqpnNzaz8Y/+nv9k0zW/ek4vCicKwCgAAgGjx/7MKAACAaDGsAgAAIFoMqwAAAIgWwyoAAACilR33f3zmyfvlf/uq02nJP7SoKznrGqdDLcWqoC/Z6XXlbFHpv1enP5Cze4f6fy+/0+3LWUv0z+xwNJWz9196SMo99dnPymv2+/rvdarelbP7+/ty9mBPX7epy58c+qFOpyNnl/o9OTsc/nj3wtF6PW3d0WQir9nu6s/OcHVdzi6vnZazX/35v+h42v/k+/t/7+/IZ/ai0vdo3tL3aMhyOTtfaGdm0mrLa/b6ehleHfQXjOc5TU3fdq3Wsa/hH1FX+n/aNE/0a9jduSvlzm5uyGse7uln66jU92LP8Tl4/gXyqtTmBzOz9/+Ts5qe4xycTvX37OG+9l8Py3P9ebRKvwe1HrXg+Pf4f+Vv/+oH3ly+WQUAAEC0GFYBAAAQLYZVAAAARIthFQAAANFiWAUAAEC0GFYBAAAQLYZVAAAARIthFQAAANFiWAUAAEC0GFYBAAAQrWN73mpHn1bpqNPyVKA1tZ5Nc622Ls/0qjRPrVoIqZz13FvP3xTD4YqcfeyTn5Kz585elLOrp7Q6zN5gWV6zclTZ7ry+LWfLQv8cssxRt5h66vj0KslWR8+miV7j2Ii1k+22fg9abb1m0FMJ6HnOTprnnntOzoZMv4/9gV7d2+7p1chJqlU+d5f0nz+dLeTsfK7Xl1qqn8ODrl6L3HXUcTaeilzHGTSfzaTc9Ws35TWn47GcbQ31itzC9fnO5ezEUSU9dvxuZaHvsTTRz8G+WJF969Ytec1qoV9r4fi9Ksd79lfsVz/wn/PNKgAAAKLFsAoAAIBoMawCAAAgWgyrAAAAiBbDKgAAAKLFsAoAAIBoMawCAAAgWgyrAAAAiBbDKgAAAKLFsAoAAIBoHdvHWDaOKkpHfWhteg1cUeqVXp1Mq5dMHHWNE0e1m4mVlWZm7928LWf/wl/6y3L22W/8opzNW3ol4MHBSM7OF1o16myhVwcuFvr+qhr9c2iCVvdoZhYS/RqSVN9jIdGvoTn+kf0RtSfbaM9kr69X5OZt/ffKHTWuScLf2Ed59BOPy9lOR7/ngyX9c88c50oq1uwuOWqkPfv+7u6OnK0cneKbm5tytt9z7H1HrXkS9LCa3draktfcWFuXs7uO+lJPVXmvN5Cza47rnU+1elozX91pT6xQNTN78IEHpNz6uv57FbOPpp7Wc7+OwqkPAACAaDGsAgAAIFoMqwAAAIgWwyoAAACixbAKAACAaDGsAgAAIFoMqwAAAIgWwyoAAACixbAKAACAaDGsAgAAIFrH9tI1jlm20hvQrHHUwC3meiVnkmkXkQb9YscTvSZsMNQrCf/aX/0bcvbnn31WznZ6Qzn7+htvy9npVK9hy/KulJs5qmyLSqtwNTNbPXVGzuYjvUZ24sjWjb5vG0c9rDnqYS3Vs1mu1fx1e0uOH69VaZqZ5ZleOVk7KidPmm5Xe/bMzDodvdoxF2tRzcwqT/V2odVpLxzVzNP5VM7e3npPzn7rW9+Ss488dFnOeni+XdJLzc1M/MhefPEFeckL58/L2eXTa3LWU8GeiRXsZr4a1+139br073znO/q67+n7sRLfiZ4K1yLRa8LlTWNmH0ZDNt+sAgAAIFoMqwAAAIgWwyoAAACixbAKAACAaDGsAgAAIFoMqwAAAIgWwyoAAACixbAKAACAaDGsAgAAIFoMqwAAAIjW8XWrQZ9lS08HYqoXwVWO0ji1HrZ0VMPWjV4/tnHmnJz92td/Uc7OC71q8KWXXpGz05m+7sb6ppw9HE+k3NxRt7q5qd/bdOqo+Wzp9aHdjl63WlV6JWDq+JMxy/T92OnqdaudblvKpblei5q39J8fUv0zc9XTnjCZo+I2SR3ViqleW5nU+pmdijWuLcdeqh3fwXQ7+n4OjavAVHa4r52XZma543Not/V7duvGLSl34x0tZ2Z27sx9cnY81ityR47a60G/L2c7jr2wvq5Xevd6AzmbhB05G8KHvx89taieCmYLP31HNqc+AAAAosWwCgAAgGgxrAIAACBaDKsAAACIFsMqAAAAosWwCgAAgGgxrAIAACBaDKsAAACIFsMqAAAAonVsHYaj6MmKUm/s6WSO1pDE0ZySaS08VaP/Zp2O3oDx+GNPytkrb1+Xs4PhUM72unq233O0ITnuw+HBTMrVtf63UuNoj6kdDUe5o5EpXXLsRUcViKeIJHE8lVmmX0OWi3sh01tLQuposHI857Xj+T1p0kT/fCrTW2WaRs8GRzNWMC1b1/pnnjpaD7st7Z1hZjYcOJqIHIU971y5JmeXlvTGvQvnz8vZt994W8rt3N2X18wd7/lBrytnW+J73sys1XK0cOqFjpbl+rqJYz9+FK1URaHPZmWl34TG8S76MH4vvlkFAABAtBhWAQAAEC2GVQAAAESLYRUAAADRYlgFAABAtBhWAQAAEC2GVQAAAESLYRUAAADRYlgFAABAtBhWAQAAEK1jOw49tXmLmV7p1fLUjzkqG5ugzd5VoXfhtbs9OfvQg4/K2dXV03I2d9TLHexvy9k7O3fl7KlVvVptOFiWcmmqV0NOx1qFq5nZkqNyUt0zZmZJqleCeur4PFV0TVPJ2TTTn99ErFv11N4Gx+eQOPaCpxLwpGk5zitPhann88lz/cyuxe1czBfymmXpqOOt9XdBr6NXgs5n+rovvfCynD179qycPX9Wr1u9ckWr/y4dnaSZo2711VffkrNDR/342bPrcnYxn8vZNNHfyUWhn9meZ7KqtHU9tds/YST8EYWj9joLjv7hI/DNKgAAAKLFsAoAAIBoMawCAAAgWgyrAAAAiBbDKgAAAKLFsAoAAIBoMawCAAAgWgyrAAAAiBbDKgAAAKLFsAoAAIBo/YS6Vb1iz0yvwwtBr4FsTK/pUqv75nO9GjZv6T9/dXVNziaN/nfCaDSRs8Gx7ub6hpztdvtydjrRautSRyviiqNir+XYi4uFp8ZRrxr01KKao344SfXPt9XW6w47Xa0iczrTa29D4qlK1mv+QsLf2EepHM+Up5Y0yx3VrI7PpxHrTj3PXlXpZ3biqDpeXtLPwNlMf/63trbk7MBRp9vWG0FtPB5LuaWBfg73evo++L3f/T9y9oknHpezmxt63Wqe6eeVo6Xbskw/21JHxb2qaRwzlDpEmV73amZWVvrMdRROfQAAAESLYRUAAADRYlgFAABAtBhWAQAAEC2GVQAAAESLYRUAAADRYlgFAABAtBhWAQAAEC2GVQAAAESLYRUAAADROrYHbP/wQF4oz/Vq1tF4Kmc3NvRK0OlUq/nMMv1a27leWTl2/F7Ly3oFWhArCb3ZptAr0Mqg/25VqVWY1o66trrSP4fF/FDOth2dhLmjCm+xcNStpnrl42AwkLOZ45lclNpe6PT1n187av7KQq/zbMzRKXrCFKW+77qO6s7S8awmjurcRaXV93bb+rXuT/TnP3U8e56KzdlkJGdbjgrldq5Xgu5t62f2bKJlz2+el9cs9GZmm0z0SnFffah+DZ694OGpJW219M9XvWeLhaMi23ELPLXK5mkfP+rn/fRLAAAAAB8NhlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABAthlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABAthlUAAABE69hevJWVFXmhNNGrHQ8O9BrX7e27craqtG615eVleU3PPWg5alw92Vmt1ciamY1H+r311Natrq7K2WFfq0b1/PxWS++BK6da3auZWer4ey131Eh6/gxsHBV3TaP31jWNfr0haFWyeabX09amf76VI+vZNydN1tZridNUP4OKQt93ZVnKWZWn2jE4npFarIY2M2sc9yA49qhaT21mVhT6uyDL9Hum3t6xo0a2FN/HZmYtx7WOHPOD4zVrdeE4VxyHtqfSe3t7W84+9vgjUm53V99fwdO36uCqZj1qjQ/hOgAAAICPBMMqAAAAosWwCgAAgGgxrAIAACBaDKsAAACIFsMqAAAAosWwCgAAgGgxrAIAACBaDKsAAACIFsMqAAAAonVsH+PerqPWLNOrHStHHV+305KzK8MVKTcYDOQ12y395y/mUznbFHoFWq+lVWGamQ17erXbbDaTs6HSs02h1ewVnppB/cfboNErCduJvm/zlp7NMv0zKx0tf2mq/32Zpnp1XnCsK68pVriamQVP3ap9NJWAPwvqRr/n5tj7laNesqz1z1LeI0Gv7vRUkpYLTxWlHLXc8+w1+vtwOtbfyQd7d+Ss2nZa1/q1Lhb6oX3njl4zet995+Wsx3PPPSdnez291rjnmGGmU32GmM/Vfe452z+a2uva0yl+BL5ZBQAAQLQYVgEAABAthlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABAthlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABCtY/v2PAVZ3Y5eP9ZuOypBp3od3s62Vi+3fftdec2nntKrWR+4eEHOXr9+Xc4Oh0M5mwe9Am1STOTsaH8sZztivVy3o++DXq8nZ8NoJGdzRyVo21FJmiT607OoHNWMVjiyH371ZuGoSi4d/ZSeOj5Hm+eJc7Cv7/12qytnPdW5nhrGIL5l6kLfd/Opoxq60p+nH7z0fcc16O+t6USvUPVkr1x9U84Gsc42S/Ta21defVHOZp56WtOv4faNW3L2hRf+SM6unlqWs8FxDi71+3K2L2bHY/3d3TT6vS1rR9bx3jgK36wCAAAgWgyrAAAAiBbDKgAAAKLFsAoAAIBoMawCAAAgWgyrAAAAiBbDKgAAAKLFsAoAAIBoMawCAAAgWgyrAAAAiNaxHYtVVckL7ezsyNng6EvsdvX6sc9/4XNS7sJ5vRZ1Y2NDzjrKLe3cmTNytt/Tq2wPD/fl7HSsfw5FMZezTaVVq3nuVzvTq1lDqlcdtkyvZmyFXM6miee30/9mLBq9tq6p9fvQBPUaHH/fNh/V38KeIuiT5d07WuW0mdnKyqqcTVN977tqfsXqzMXCUV860+slM0cV5r/99rf1dVP9Hty4qVeCBked9t2dbTm7f6DV9Fa1o572Fb1uNe/o++vFF5+Xsy+/rF/DHcezs7G5Lme3trbk7OHhoZy9ePminNXpZ7anVtkRPRLfrAIAACBaDKsAAACIFsMqAAAAosWwCgAAgGgxrAIAACBaDKsAAACIFsMqAAAAosWwCgAAgGgxrAIAACBaDKsAAACI1rF1q8N+T15osdBrIOtaq9gzM1tfOy1n/+w3/4yU+/IzX5LXHI20Gjozs+lEz547e17Opo4/KZYHem3d6VW9xrXd0te1oO2F6VSvRawqveZvJdf3bavTkrNpW69QrEy/Xy390bFpqW+GMujXWyZa7WTjqDptGv3nB081q1wNe/Ic7OtnUKkfw5Z7DiEHtT20LPW61cV0ql9Aru/Ri/fpNd2po2418ZzvwyU5u7u3J2dXT2nrnlodymvu79+Vs4Ou/i7qDvTz3VOheurUipxdXdaznqrgc+fOydksO3Z8+/88e7F29KImQfv5ZmbW6O/vI3/eT70CAAAA8BFhWAUAAEC0GFYBAAAQLYZVAAAARIthFQAAANFiWAUAAEC0GFYBAAAQLYZVAAAARIthFQAAANE6toLg7l29gaLXc7QGZW05+8orL8vZf/qP/rGU+85//E/ymk984jE5++Uv/ZycnU4O5GxbbKowM8sdpRIrq3obypn1ZTmbiIUZ8/lEXjN11LxkBzfkrDlanupEv7lFo2eDXnBi9VxvGKnKSs42jZZtTL9fjaMNpTZH1tG8dNLUjoaxxPFMNY1+00tHNVZL306yotDbcoLp15qm+r3t97tydjbTz8H5XG/nmszmcla9Z/P5TF4zy/X9dTjal7Pdnj4/5I4NFhL9DJrO9PZFTzNWp6PPUe+J7Vzttn6/PHznu/7sHIVvVgEAABAthlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABAthlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABAthlUAAABE69hOyEG/Ly80meiVcaGjz8itVkvO7u5p9bDXr12V10yCXin27Fe/ImfnU72ubdGUcras9Dq8vd0tOfveu/rnkIkNd0tL+v7a3NyQs21HHZ9Vet3iotYr48pa3zeJo2q0KfW9UCz0dRfifagS/X4Vjoa9yvF3c+Wo/jxp8jyXs1mmP9PlQj9XSsd51c606/VUw9ae89JRSXzj5jU5++Cly3K2cFSodnoDOes5MxcL7Z7t7e/oP//MOTk7m+n7686dbTkbgn4ILS/rleKedT31v3U9krNpqr1oa0c/de2oUK0qT1Z/zo7CN6sAAACIFsMqAAAAosWwCgAAgGgxrAIAACBaDKsAAACIFsMqAAAAosWwCgAAgGgxrAIAACBaDKsAAACIFsMqAAAAonVs3Wo21ivQWnO9Tms6OZSz7Y5eL3dnT6v/Ohgf+2v/iOXzT8nZXXPU26X6vR32OnLWCr2u7co7r8nZjVW9mnEx0ir51tf0WsTL9z0oZw9L/fNdlAs5G/RlLThqekOq1/H1HU2yxd6enN2+qe0FR4ushUTfM61OT872e0v6RdgXHdk/+b537bqcXb7/gpzdXB3K2ZW2XvnazLV3wbK+pJ3t6u+iwx29urMb9H03rM/o65Z63Wqpt3TbNNUPi1nQntVZoleSHgR9z3SDfgY2jprPXl8/V9JG7Ak3s8pR05uJtahmZonjJRNq7d2Vpfp3kk2q18g2jhrX+VyOHolvVgEAABAthlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABAthlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABAthlUAAABE69hur529XXmhqtFruizV60OzRK/DVJOLhb5mUeg1cFmmV6XN53pl3GSm1/G1HDWfi7led3r3jr4Xlgdadd/GmbPymtOJ3te2aPR7UNf6vs2CXptXm17HFxzX0Gvr9cP983p2eXlFyr1354685v6hXv17cLgvZ3d39b140uze1T+f+VSvfO7fpz+r1Vz/3Edj7WzLO/qz1+3pNZ/jA736u6j05/TqO3rt7Wiin++rm2tytnCcg2miZXsdvUJ5NjmQs3mtv4u63a6cbRz3YDrTu2w7bb3GdV7qM0Q115/JrKV1EO/v62dryPTnrCj1z2z+IfSt8s0qAAAAosWwCgAAgGgxrAIAACBaDKsAAACIFsMqAAAAosWwCgAAgGgxrAIAACBaDKsAAACIFsMqAAAAosWwCgAAgGjp/aAfouAYkWeFXo1aNrWU81SoeqrSPDVwZtq1mpnNJnolYG+5L2fPnD0nZ7euvSln727flnLra6vymp2OXm/Xben3ICRaZZ2ZWZ7pNcFNo9etFnO9Dm9R6RV3ddCzi5lWh3f+7Ka8poUtOToe6fegLib6NZwwoz39Pt7dfk/ONpcvytnpVK9WDOJjUjb6q6q3rFeSzrb0Pdpt6WfQ1t0dOVs0+gsxb+tn0Gymn0GWau+jfk+/B+OJXh1aO96Hea6f2bOFvhc9jfHDoV7pW870381TjZq3terb0VQ/L5NE34u1o8q2dFSzHoVvVgEAABAthlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABAthlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABAthlUAAABE69gOu9RRa1YVevVWrUdt4ahbDS2tki9tpfqaQb/Y/pKjju/dW3I2S/QeuNnCUWuWOCoMhytytphrn1mS6fer1RrI2fGhZy/qlYQLx71tefZYo1copon+u4XMUc0q1i2mjq7kvlgHaGa2trokZ5eX2nL2pCkc1Yo3rl6Ts9uX9LrVXKy9NjNbbmsV1VVwPE9tPTspHWdgou+7pK2f2b2u/kzNSv28qmq9KjyptWtIgr5mv6vfg2Su34PpXK9x9UhTx75xXMN0NtXXdWRne7tSzlXR7ahQrSp9L5aeoe8IfLMKAACAaDGsAgAAIFoMqwAAAIgWwyoAAACixbAKAACAaDGsAgAAIFoMqwAAAIgWwyoAAACixbAKAACAaDGsAgAAIFrHds1N53q12lSPWpbr1VuF3uhl3VyrSwuONQ9Gh3LWU5WWO6ooT59albPbWzfl7M0beuXrctdRCZprNarThV7HNy/1Krze4JScPTw8kLOj0VzOJolecdeYns31bWNprm/0mfisz0v9Hnjq+PqO/ZWkevakmd3ekrNXc71q9NqFC3J2dXkoZxcdrcL09lzfd+2W/h3MwdxRA3moV2yecpzZ7bZe4zoa6+dVLr4PzcyqWru/05H+oh8O9X1QVfpnVoz192yv15eznlrSq6+9Jmdnnr3r2AvThfZZjMdjec35Qq+3n4q16mZmZemogT8C36wCAAAgWgyrAAAAiBbDKgAAAKLFsAoAAIBoMawCAAAgWgyrAAAAiBbDKgAAAKLFsAoAAIBoMawCAAAgWgyrAAAAiNaxfXtlrS/kiJql+oxclXplY9No1XnlfCKvef3GNTl78+Z1OVsUem1dMddr/m5cvyFnQ61/DmNH9e5zf/iilMvzrrzmF77wc3L2zk29jm9e6pWTaapngx61otDr8BaOSt+w8NQaa9W3odCf9KrW6x4t6D2ydePoSz5hso5eRXvx3Dk5e/bMppw9dUqvO15MtbNtJO5PM7Osq1dsPvjJz8nZd668pV+D49lb7ujn8MzxPmz39OrOSnwnzqd6dWjd9XwXpn9mIdPPlVmhX+/Ozo6cfePNN+Xs3FG3unlefyZXVrTnbDLV3xlVo5/vngrVwlG9fRS+WQUAAEC0GFYBAAAQLYZVAAAARIthFQAAANFiWAUAAEC0GFYBAAAQLYZVAAAARIthFQAAANFiWAUAAEC0GFYBAAAQrWNLIRNHvWRS69VbWZ7L2WrmqCWtxWzwlMPqv9fSsCdnn/jEk3J2d3dXzvYH+jUER5/ui9//npz9rf/236Xc8y+8LK/50stvyNl2+7ScbWX6Hl87vSxn+wO99rKq9Dq+pWV93fsvnZWz6xsbUu7u9i15zcY8dat6HV+a8Tf2UX7pF74mZ7/8la/I2U888ZScbff06syd3T0xqX/meUt/RtZOr8rZ//2/fkfOvvGqVjltZpa09Pdh0tJriUPQK1/reiHlHD/erNRrwheNXg2bOc7sK1euyNlbt7fkbLvj+Bwc9bCj0UjOnl4/I+UGg4G8ZnDMfGmuf2Yh6HXJR+HUBwAAQLQYVgEAABAthlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABAthlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABCtY7u1ylqv4yz0ZjfLK0fYQa30yvt6JemFi+fl7FNP6RWqd3b25GyW639TrKysyNnRwaGc7Q30qtHPfO5PSbnDvX15zX/wz/65nLWwJEfTTK+B63Y9Fap6TfBsrlfs3X/pPjn7zT/9dTn79NOfl3KDJf0emOdMaOnVfe22Xgl40ly6cEHOPnz5ATnb7zk+96CfV61cO4uHa2vymkWpV/e2e/q59tQXnpazL7yk162O5/pZsdrXP4ey1utOrdHu2dDx7kwctebFQqt7NTNLU72+9K0rb8rZHcc7+avPPitnG8c5+PZVvR62asT7mziqThNHrXGu1wTnbcf5cQS+WQUAAEC0GFYBAAAQLYZVAAAARIthFQAAANFiWAUAAEC0GFYBAAAQLYZVAAAARIthFQAAANFiWAUAAEC0jq2CmZV6A0W/25Wz05mjWcMhBK0qYjjsy2v+8i//OTn7P/7nb8tZa/RWifvvvyxnH3rgYTn7B999Ts4uLQ/l7Hg2l3JXr9+S11xfOydn5+2WnG0c9SK12PJiZlaX+t+B3YHe7nFl646c7axuytk7Y+13e/HNN+Q180y/B0tL+jM5XB7I2W/IyZ8N//7ffFvO9np6G9HmOb0Z68z5++XstNSev+39A3nNvK3vpZvvvidnX3v1B3I2cTSy3XU0J7VSfe97yuaKqpRye3t78pqnVvUmwaLQ3hlmZkmizxqff/qLcnYymcjZjY0NObt3oDc1nr9Pbyj8wQ+0/TieTeU1ux392fHMBF/6zGfl7FH4ZhUAAADRYlgFAABAtBhWAQAAEC2GVQAAAESLYRUAAADRYlgFAABAtBhWAQAAEC2GVQAAAESLYRUAAADRYlgFAABAtI6tW/XMsrXezGp5rlfRWaPVwJmZVWJl21xvVbPf/+7vytmD0VjOlpVe89nr6RV7Dz/wqJw9s65XmE6nemXbrZu3pdxtR9VhnuoVqvVpvY7PpdGfh0as/jUzy9NcvwZHjWPT0esOQ2dFyv3B878jr3n27Fk5e/nyaTnbCWty9qR5+oufl7O3b96Qs//5O/9FziZt/bxKcrHy1bHv09zx/Cd67fXjj+tV1o8+qp/D3Ux/eV57+2U5e+uWfr5urmmfQwh65fTbb78tZ9cvf0bOeiqy1Qp2M7PBkn5enr+gvzvP1Po5GBwz1/mLWgXyq6++Lq9pIZWjSa6/kzt9vdr5yJ/3U68AAAAAfEQYVgEAABAthlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABAthlUAAABEi2EVAAAA0WJYBQAAQLQYVgEAABCtY+tWu6leW7eY67WoTXB0s6Z6HV5da9fw8EOPy2sOB/o9WBrqlWLPfu0X5Oxrr70lZ99846qcXerrtYgvv6JXtm29q9Wtdrp9ec3hcChn32t25KyZXi/nqaKzoP8dWDr2uGWO2tlM37tVpu3d9nBDXnPzov6cPfzYk3J2dXVVzp40zzzzjJx99bU35Owbb+jZvKufK0sr61Ju5nhlzBeFnK1Mr+N8681X5Oxrly/K2XNnVuRssxjJ2bTRe8XzRqsqX1/Tz+GNTb1m1By1qJWjbtVTpxsSfd3xVL+3s9lCzq6t6+frpUuXpNxgaVleczzT9oGZ2WQyk7Nrp/Xf6yh8swoAAIBoMawCAAAgWgyrAAAAiBbDKgAAAKLFsAoAAIBoMawCAAAgWgyrAAAAiBbDKgAAAKLFsAoAAIBoMawCAAAgWsfWrWZZW15oXumVYnVTydnMUYcZEm32/vSnnpDX/NSTemXk5jm9Xq7T06tZT6/pdWnf/b0tObu+plUdmpm9+NILcnY+OZRyra6jQvXdd+WsnevoWQ9Hy581jppgc9StOmony0oPl7V2DU99+nPymk9+8jNy9pHHLsvZzNF6e9JcvXZFzm7dvilny0qvME0WemXjYjaWcofjqbxm46i39Dx6h7m+8b53U/8cvt/WL+LRhy7I2Y3VJTl7uLct5aaT0/KajzzysJwdOepLWy29ctqTHU/0a3jrLb0C/eaW/u569NFH5eylBx6QcktL+j4oav0lN3I8k0Whnx9H4ZtVAAAARIthFQAAANFiWAUAAEC0GFYBAAAQLYZVAAAARIthFQAAANFiWAUAAEC0GFYBAAAQLYZVAAAARIthFQAAANEKTePpkAQAAAA+PnyzCgAAgGgxrAIAACBaDKsAAACIFsMqAAAAosWwCgAAgGgxrAIAACBa/xcAZL5DgZgZHAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 864x432 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# imgs 데이터 imgs[0]~imgs[1426] 까지는 가위 데이터가 저장되어있다(label=0) \n",
    "# (imgs[0], labels[0])과  (X_train[0], y_train[0])을 비교해보고 데이터 순서가 바뀌었는지 확인한다.\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "fig, ax = plt.subplots(1, 2, figsize=(12,6))\n",
    "ax[0].imshow(imgs[0])\n",
    "ax[0].axis('off')\n",
    "ax[0].set_title(labels[0])\n",
    "\n",
    "ax[1].imshow(X_train[0])\n",
    "ax[1].axis('off')\n",
    "ax[1].set_title(y_train[0])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "pacific-scanner",
   "metadata": {},
   "source": [
    "### 4. 데이터 탐색(EDA)\n",
    "1. pandas DataFrame을 이용해 label 데이터가 shuffle 되었는지 확인"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "relative-mayor",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "\n",
    "y_train_df = pd.DataFrame(y_train, columns=['category'])\n",
    "y_test_df = pd.DataFrame(y_test, columns=['category'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "returning-morgan",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>category</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   category\n",
       "0         2\n",
       "1         2\n",
       "2         2\n",
       "3         0\n",
       "4         0"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "prescription-pepper",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>category</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   category\n",
       "0         1\n",
       "1         0\n",
       "2         1\n",
       "3         0\n",
       "4         0"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_test_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "institutional-native",
   "metadata": {},
   "source": [
    "#### 2. seaborn의 countplot으로 train, test 데이터 분포 확인\n",
    "```\n",
    "모델이 모든 클래스를 balance 하게 학습하고 분류할 수 있는지 여부를 확인할 수 있다.\n",
    "train_set: 3429개 (가위:1153, 바위:1144, 보:1132)\n",
    "test_set: 857개 (가위: 274, 바위: 288, 보: 295)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "white-nylon",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAtoAAAGDCAYAAAAVh7eRAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABIp0lEQVR4nO3deXyU1d3///dkD5nskAkJCRJ22cIOsmkwIARkkUhtpUL1xp0iyrciLVq1oPftbam1t5qfrbWoVUABCypIkEXZRDbBsG8JJBPIvk8ymd8f0dGYAAPkykJez8ejj5JrznVdn0k7hzdnzjmXyeFwOAQAAACgTrk1dAEAAADA9YigDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYgKANSLrvvvu0YsWKqzo3Li5OW7dudalt586ddfr06au6z7WcCwDNVVpamjp37qyKiorLtt2xY4eGDx9+Vfe5lnNx/fJo6AKAq9W7d2/nn0tKSuTl5SV3d3dJ0h//+EfdfvvtLl/rzTffrPP6AAA11WXfLUnTpk3T7bffrsTExDqtE6gLBG00WXv27HH+OS4uTs8//7xuuummGu0qKirk4cH/1QGgMXC17wauB0wdwXXnh6/vkpKSNGTIEM2bN095eXm6//77NWjQIPXv31/333+/MjIynOdMmzZNy5YtkyR99NFHuuuuu/Tiiy+qf//+iouL06ZNm1y69/79+zV16lT169dPQ4cO1bPPPiubzVatzaZNmzRy5EgNHDhQL774oiorK52vLV++XGPGjFH//v1177336uzZs3XwGwGAxq+yslJJSUm69dZbNXDgQP32t79Vbm6uJKmsrExPPPGEBg4cqH79+umOO+7QhQsX9Oc//1m7du3Ss88+q969e+vZZ5+97H0+/PBDjRkzRr1799bIkSP1/vvv12jz+uuva+DAgYqLi9PHH3/sPG6z2fTiiy/q5ptv1k033aQFCxaotLS0zn4HuP4QtHFdunDhgvLy8vTFF1/oueeeU2VlpSZPnqwvvvhCX3zxhby9vS/ZIe/fv1/t2rXT9u3bdd9992n+/PlyOByXva+bm5vmzZun7du36/3339e2bdv03nvvVWvz+eef68MPP9SKFSu0YcMGffjhh5Kk9evX64033tCrr76qbdu2qW/fvnr88cev7RcBAE3EkiVLtH79er3zzjvasmWLAgMDnf30ihUrVFhYqI0bN2rHjh364x//KB8fHz322GPq16+fFixYoD179mjBggWXvU9oaKjeeOMN7d69W4sWLdKiRYt08OBB5+sXLlxQTk6OtmzZohdeeEELFizQiRMnJEkvvfSSTp48qZUrV2rdunXKzMzU3/72N2N+IbguELRxXXJzc9OsWbPk5eUlHx8fBQcHa/To0fL19ZXZbNaDDz6or7/++qLnR0RE6M4775S7u7smTZqk8+fP68KFC5e9b/fu3RUbGysPDw+1adNGU6dOrXGf//qv/1JQUJAiIiL061//WqtXr5Ykvf/++5o5c6bat28vDw8PPfDAA0pJSWFUG0Cz8P777+uxxx5TeHi4vLy89Mgjj2jt2rXO6X+5ubk6ffq03N3d1b17d5nN5qu6z80336zo6GiZTCYNGDBAQ4YM0a5du6q1+e1vfysvLy8NGDBAI0aM0KeffiqHw6GlS5fqqaeeUlBQkMxms+6//36tWbOmLt4+rlNMXMV1KTg4WN7e3s6fS0pKtGjRIm3ZskV5eXmSpKKiItntducinJ9q2bKl88++vr6SpOLi4sve9+TJk3rhhRd04MABlZSUyG63q1u3btXatG7d2vnnyMhIZWZmSpLOnTunhQsX6sUXX3S+7nA4ZLVaFRkZ6crbBoAm69y5c3r44Yfl5vbjGKCbm5uysrI0YcIEZWRkaM6cOcrPz9ftt9+uxx57TJ6enld8n02bNulvf/ubTp06pcrKSpWWlqpTp07O1wMCAtSiRQvnzxEREcrMzFR2drZKSko0efJk52sOh6Pa9D/g5wjauC6ZTKZqP//jH//QyZMntXTpUrVq1UopKSmaOHGiS9NBrsQzzzyjG2+8Uf/7v/8rs9msf/7zn1q7dm21Nunp6erYsaOkqr9YwsLCJFUF8AceeOCKV9wDwPUgPDxcCxcuVN++fWt9/ZFHHtEjjzyitLQ0zZw5U+3atbvinUZsNptmzZqlF198USNHjpSnp6ceeuihan8X5Ofnq7i42Bm2f+izg4OD5ePjozVr1shisVz9G0WzwtQRNAtFRUXy9vZWQECAcnNz9eqrrxp2Hz8/P/n5+en48eP697//XaPN3//+d+Xl5Sk9PV3/+te/NHbsWEnSL37xCyUlJeno0aOSpIKCAn366aeG1AkAjc1dd92lxYsXO6fLZWdna/369ZKk7du36/Dhw7Lb7TKbzfLw8HCOfLds2VKpqaku3cNms8lmsykkJEQeHh7atGmTvvrqqxrt/vrXv8pms2nXrl3auHGjbrvtNrm5uSkxMVELFy5UVlaWJMlqtWrLli118fZxnSJoo1m45557VFZWpkGDBmnq1KkaNmyYIff53e9+p9WrV6tPnz76wx/+4AzRPzVy5EhNnjxZEydO1M0336wpU6ZIkuLj43Xfffdpzpw56tOnj8aNG6fNmzcbUicANDa//vWvFRcXp9/85jfq3bu37rzzTu3fv19S1QLFWbNmqW/fvho7dqwGDBigCRMmOM9bu3at+vfvr+eff/6S9zCbzfr973+v2bNnq3///lq9erXi4uKqtWnZsqUCAgI0bNgwPfHEE3rmmWfUvn17SdLcuXPVtm1b3XnnnerTp4+mT5+ukydPGvDbwPXC5Kjr784BAAAAMKINAAAAGIGgDQAAABiAoA0AAAAYgKANAAAAGMCwoD1v3jwNHjxY48aNcx578cUXddttt2n8+PF6+OGHlZ+f73ztjTfeUHx8vEaPHl1tq5zNmzdr9OjRio+PV1JSklHlAgAAAHXKsF1Hvv76a7Vo0cK53Zkkffnllxo0aJA8PDz0P//zP5Kqtso5duyY5syZo+XLl8tqtWrGjBnOh3yMHj1ab731liwWi6ZMmaKXX35ZHTp0uOS99+7dW+2pgADQlJSVlSk2Nrahy6hX9NsAmqpL9dmGPRmyf//+SktLq3Zs6NChzj/Hxsbqs88+kyQlJycrISFBXl5eioqKUtu2bZ17Z7Zt21ZRUVGSpISEBCUnJ182aHt7e6tr1651+XYAoN6kpKQ0dAn1jn4bQFN1qT67weZof/jhhxo+fLikqicrhYeHO1+zWCyyWq0XPQ4AAAA0doaNaF/Ka6+9Jnd3d91+++2GXL+srKxZjggBAACg8aj3oP3RRx9p48aN+uc//ymTySSpaqQ6IyPD2cZqtcpisUjSRY9fCl9BAmjKGCgAgOtDvU4d2bx5s95880299tpr8vX1dR6Pi4vTmjVrZLPZlJqaqlOnTqlnz57q0aOHTp06pdTUVNlsNq1Zs0ZxcXH1WTIAAABwVQwb0Z4zZ4527typnJwcDR8+XI8++qiSkpJks9k0Y8YMSVKvXr307LPPqmPHjhozZozGjh0rd3d3LViwQO7u7pKkBQsW6L777pPdbtcdd9yhjh07GlUyAKAWZWVl+tWvfiWbzSa73a7Ro0dr1qxZSk1N1Zw5c5Sbm6tu3brpv//7v+Xl5SWbzab/9//+nw4ePKigoCD9+c9/Vps2bRr6bQBAvTNse7+GlJKSwtQRAE1WY+vDHA6HiouL5efnp/Lycv3yl7/U/Pnz9dZbb2nUqFFKSEjQggUL1KVLF/3yl7/Uu+++q8OHD+vZZ5/VmjVr9Pnnn2vx4sWXvEdje88A4KpL9V88GRIAcEkmk0l+fn6SpIqKClVUVMhkMmn79u0aPXq0JGnSpElKTk6WJG3YsEGTJk2SVPUshG3btuk6HNMBgMsiaAMALstut2vChAm66aabdNNNNykqKkoBAQHy8KiagRgeHu7cftVqtap169aSJA8PD/n7+ysnJ6fBageAhtIg2/sBAJoWd3d3rVq1Svn5+Xr44Yd14sSJOr0+27ICuB4RtAEALgsICNDAgQO1d+9e5efnq6KiQh4eHsrIyHBuv2qxWJSenq7w8HBVVFSooKBAwcHBl7wu27ICaKoa5ZMhAQBNQ3Z2tvLz8yVJpaWl2rp1q9q3b6+BAwdq7dq1kqQVK1Y4t1+Ni4vTihUrJElr167VoEGDnM9NAIDmhBFtAMAlZWZm6sknn5TdbpfD4dBtt92mW265RR06dNBjjz2mxYsXq2vXrkpMTJQkTZkyRXPnzlV8fLwCAwP15z//uYHfAQA0DLb3A4BGpjn2Yc3xPQO4Plyq/2r2I9oV+fmqLC1t6DKaNDcfH3kEBDR0GQAA4HuFZYWy2W0NXUaT5uXuJbO3+Zqu0eyDdmVpqU4veqGhy2jS2s57UiJoAwDQaNjsNv19+5sNXUaTdu+g+675GiyGBAAAAAzQ7Ee00fgUFJeprNze0GU0ad6e7vJv4d3QZQAA0KwRtNHolJXb9df/fNPQZTRpj47vK/+GLgIAgGaOqSMAAACAARjRBuASVrBfu7pYwQ4AaDoI2gBcwgr2a1cXK9gBAE0HQRsAAFw1FrBfOxawX78I2gAA4KqxgP3asYD9+sViSAAAAMAABG0AAADAAARtAAAAwAAEbQAAAMAABG0AAADAAARtAAAAwAAEbQAAAMAABG0AAADAAARtAAAAwAAEbQAAAMAABG0AAADAAARtAAAAwAAeDV0AAACuqsjPV2VpaUOX0aS5+fjIIyCgocsAmgWCNgCgyagsLdXpRS80dBlNWtt5T0oEbaBeELTrgLlXT/m2by/vyEh5RUTI3cdH+bu+kfXdd2s2dnNT0NAh8o6IlHebSHlZLDJ5eMj6/gfK37Gj1uv79++v8F/eddH7Zy5bpryt26odCxgwQH7du8krvLXc/c0ymUyqyM1VyYmTyvniC5WfP39N7xkAADRPPh4+6tiqk9q3jFFLv1Yye5tV6ajU+cLzOpD+rb5N/7bGOZ7uXhrYdqA6teqkQJ9AVVRWKD0/XTvP7NCZnDM12t/UboiGtBty0RqW7V2mU9kn6/R9GYGgXQdC4uPlHRmpytJSVeTlyd3H56Jt3by81GrSJElVX4FWFBTIMzjYpfsUfvutys6eq3G8NDW1xjH/fn3lERCg0jOnZc8vkBwOeYWHK2BAf/n376f0v/9DxYcOufgOAQAAqnQO66JRXUapsKxQZ3LOKL80X35eLdSxVSfd1nWM2oXG6OMDq5ztvT289cs+v1JLc0udLzyvvef2ysvdSx1adtDU3r/QZymf1hrOJelA+rfKK82vcTy3JMew91eXCNp14PzKVarIzVX5hQvybd9ebR55+KJtK8vLdfaNJJWdOyt7foFCRo9W6G2jXbpP4bcHVPD11y61PZf0/8lRUVHjeItOnRT54ANqOeF2nbmOg/aN0S11Q1igwkP8ZAnyk4+Xh/adzNRHXx2u0dbNZNKATq0VHuKn8GCzWgW2kIe7m1ZtP6Ldx6wu3c/dzaT7x/aWJchPeUVlennFzsue0zYsQNNv7Sk3N5M2fXtGG/advuL3CQBAfcspydZH+z7U8azj1Y5vObFFd/ebps5hndWpVScdOX9EkjSk3RC1NLfUkczD+vjgx3I4HFXtPVtoWv9fa2SnW3Uy+6QKywpr3OtA+gGl5tYcUGwq2HWkDpQcO6byCxdca2y3q/jQoapRZgPVFrIlqfjIEdmLi+XVsqWh929ow7tHaWCXCIUH+6mgxHbJtl4ebhrTv716tw+X2ddLhaWXbl+bW2NvUJCft8vtvTzcNemmziq326/4XgAANKQzOWdqhGxJKrIVae/ZvZKkqKAo5/GOrTpJkr48+aUzZEtScXmxdp35Wp7unurRuqexRTcQRrSbEO/ISLn7+srk6aGKvDyVHD2miry8K7qGT7t2cm/RotbpJteTz745ofziMmUXlOoGS6BmxF/8A1xur9SSDQeUkVOowpJy3dwzWrf0bOvyvW6wBGpQ10it2XlM4wd2dOmcMf1i5O3pri0H0nRr7xtcvhcAAI1ZpaPy+//+MVD7eflJkvJKamaW3NKqY22D22rbqa01Xo8MaqNw/3CZTCbllebrTM5plZSXGFG6IQjaTUjwiOHVfnbY7crfsUPnV6y86Ai2uVdPeYW3lsnTU16tWqnFjV1lLyrS+Q8/qo+SG8wpq+v/ALFXOnTs3NXN9fL2dNfEwZ10MiNXu45muBS0O7cJUZ8O4frwq0NyM5mu6r5NVadWnRQVFKUw/zC1MofJ28Nb32Uc1Jrv1tRo62ZyU2xkb4X5h8liDlOoX0u5u7nrs5TP9G36/lqv3yaojXpG9FKYOUxmb7M83TxVaCvUhcIL+iZtV40FN24mN7UPba+Ylu3VOqC1AnwC5GZyU25Jro6eP6qdZ3aq3H7l33AAQHNkMpnULbybJOlk9gnn8ZLyEpm9zQr0CVRWcVa1c4J8AiVJIS1qX682LGZYtZ8r7BXaeWanvjr5ZV2WbhiCdhNQkZ2tzA8/UvHhw6rIzZWbr49828UoNCFBgTfdJDdvH2W8806t55p79pJ/n97On22Zmcp45x2VpabVV/nXtTH92svXy0Orth11qb2ft6duH9RRKWcuaP/J84qNCTO4wsZl8A2DFeZvka3CpoKyAnl7XHy6jae7p0Z2GilJKiorVJGtSAE+l96SLDq4raKDo5Wel64zOWdUbi9XgE+AOrTsoA6tOmjrya3VOucg3yBN7DlJtgqbzuSe0YmsE/Jy99QNIe10U7ub1CWsi97b/W6TGj0BgIYyov0ItTK30vELx3Uq+5Tz+PGs4+oV0UtDYoboPwf+I4eqRrt9PX3VN7qfJMnbs/pGEucLMvVpyidKzUlVoa1QLTxb6IaQdhoWM1Q3tbtJbiaTtpzYUm/v7WoRtJuAkuPHVXL8x7lQ9vJyFe7bp9LTpxX9xBPy79tH2Rs2yHau5o4kGUuWKGPJErl5e8urdWuFjB6lNrNmKXPpMpcXVqJ2XaJC1bu9Rau2HVFecZlL59w+qKNMMuk/O48ZXF3jtOHoFyooK1BuSY6igqL0iz4X37ay3F6u5XuXKbMwU0W2ostu9SRJO05v19aTX9U4bvYy69cD7tGgGwZp79k9KrIVSZJsFTZ9fnidDqYfVHllubO9m8lNE3tMUvuW7XXTDTcp+WjyVb5jANfC18tDXaND1SkiRGHBfgrw9ZK90iFrbpH2Hrdqz3GrHD87x93NpL4dwtUrxqJgs4883N2UX1ym4+k52ppyVnlFNftrP29P3XRjpDpGhijIz1v2SodyC0t14PR5fX0kQ7YK1tNcTp82fdQ/eoCyirL0yc++pfzqxJdqF9JOncO6KGRAqE7nnJanu6c6tOygwrJCBfoEVpu7LUlHL1QfwCooK9C36ftlLbDq7n53q3/0AO1K3dXoB0JYDNmEVeTmqiglRZLk2z7mkm0ry8pUeuqUzr35d5VnZipsyhR5BAbWR5nXJT8fT90+sKOOnM3W7uOu7UzSu71FXaJCtWbnMRWVll/+hOtQau4Zl7dkqnRU6mT2SWcodoW9sva/DAtthTqXd1ZuJjcF+QZVO7737N5qIfuHe28/VbU3fVRwtMv3B1C3urVtqQmDOimypb/OXijQ9kPn9N2ZCwoL8tOEwZ2UOKxrtfZuJumeW3soYUAHeXu668Cp89p1NF1FpeUa1CVSDyb0UavAFtXOCfLz1kPj+mhotygVl5Zr19EMfXvyvLw83TWqT4zuHd1THu7EpUvpHdlbIzvdqguFF/TBnvdVWlH96a1FtiIt+fpf2p22W17uXuod2VvtQ9vrsPWQcxvA4vJil+6VWWhVen663N3cFREQUefvpa4xot3E2YuqtsJx8/Jy8QS7io8eVXBEhHxuaKvCfbXPdcWl3T6oo9xM0sfbXZsyEuTnrdv6xujA6fM6eMbFHWpQZ1p4tlDrgAhV2CuUXZzt0jk/LuipNLI0AJeQlV+id784qKNns6uNXCfvPaX/GhOrbm1bquupUKWkVs377RLVUm3DAnU8PUdLkg9UO+eWntG6uWdb3dQ1Uqt+0ncPubGNzL5e+mLfaW389sd1HCaT9Ou4HoppHaRu0S2172Smwe+2aerbpq/iOo3U+cLzWrrng4sG5uLyYiUfWa9kra92PPr7wYyM/AyX71ny/T083T2vsur6Q9Bu4nyiq/4PWp6VdZmWP/phJNthJ0BcjV7twtSlTag++urwZbcO/MGEwZ1UYa/UmmY6ZaS+WfzD1b5le7mZTPL39lf7lh3k7e6t5KPrXf6asUfrHpKkk1mN/8ljwPXq5EUWtheWlmvXkQzd2vsGtbMEOYN2iLlqnu/Pg7kkHUrL0s0928rPp3o4C/7+nENp1f8edTikI+eyFdM6qMY5qDIgeoBGdLhZ1gKrlu1delXTOH5YPJli/c6l9m4mN4X5WyRJuSW5V3y/+kbQbgK8o9rUXLxoMik4Lk6+7dqporBQxSk/PnzGrUULufn6qCKr5sid3403ytyjhypLS6vN+4brWoeYJUmTh3TW5CGda7we6OetP95dtUp60QdbVVpuV+sQs3y9PPS7xMG1XnNEj2iN6BGtlNQLen9TinHFNxPh/uHV5nOXVZTp00Of6LsM1zry9i07qFdkrPJL87XzzA6jygRwDWrbRi4zr2qks0NEiLYfOlctbHeKDJUknUjPrXadzLxidYwMUafIEGXk/DhVzSSpY0SwKisdOpFR/RxULW4fGjNMGfkZWrZ3aY3pIj/n6e6pcnv1aXo3ht+obuHddTY3TUfPH/1JWy+Zvc3K+dk3kG4mN8V1jKvavaQoSxkFro+CNxSCdh3w695d5h7dJUnu/lW7Ivjc0FaWu34hSbIXFenCx/9xtg8eGSevsKrdJrwjIyVJAQMHyDemnSSp5MRJ5e/48S/36DlzVHYuXWXnzlU94t3XRz43tJN3RGtVlpXJ+s47qiz7cXGHR1CQoh+fo7LUVNkyM1WRly83Xx95R0bK94Yb5KiokPWDpaosadwLCBqr1Av58jrmXutrfTuEy1Zh17enzkuSKiqruvl9J6zy9Kh5Tqi/r26wBCo9u1DnsguVkV3zqVi4cvvO7dW+c3vl7uauQJ9AxUbGKuHGcYoMbKPPD6+75LkRAREad+M4ldvLterblSqrcG2hK4D642aSerWrGtU8eu7HMHbkbLa+O3NBN0a31EPj+uhERq7slQ61DjErulWAth86q51Hqm8c8NXBNHWKDNHI2KrR8fScQrm7mdS+dbDMPp5atf1otQCOqlHooTHDVFlZqbTcNPWJ6lujTV5Jng5mHJBUFbIfGvqwTmefVm5JrhwOhyKDIhUZGKkLRRe06sDH1c719fTRvQPvVUZBhrKKslRkK5SvZwtFB0cryDdIxbZirT74nxr3bIwI2nXAOzJSAQMGVDvm1bKl8+mL5dnZ1YJ2iy5d1KJDh2rtfdu1k2+7ds6ffxq0czZ8Ie/oaLXo2EFuLVpIDocqcnKUu+VL5WzaWGPkuiInRznrk+Xbob1adO4s9xYt5LDbVZGbq7ytW5WzabPKM5lrdrUOnr6gg6drn2fdt0O4Ssoqaszd/nTXiVrbx8aE6QZLoI6czeYR7AawV9qVXZytDUc3yN3NQ7GRsTqdfcr5WOCfiwiI0JTYRDnk0If7ljeJ0RKgObq1dztZgv105Gy2jv9shPqDzSm6uWe0hnePVliQn/P48fQcfXvqvCp/NqekqKxcb362VxMGd9KN0S0V0zpIUtVI+e5jGTqRcXXPWbieBX6/qNzNzU39vt+e7+fO5JxxBm17pV2HrIfUJrCN2oZUPRAutzhHm49v1jepu1RRWf1ZIKXlpdqdtlutA1qrXcgN8vH0lb3SrtySXO04vV27zuxyefFkQzMsaM+bN08bN25UaGioVq9eLUnKzc3VY489prNnzyoyMlKLFy9WYGDVli5/+tOftGnTJvn4+OiFF15Qt25Vc3ZWrFih1157TZL04IMPatKkSUaVfNWy165V9tq1Lrc/+7f/u6LrX/jPlf2rrbKkRFmffnpF51xvurQJVZeoqq8Jzb5Vc+uiWvpr4uCqx8AWl5Vr3e4f594O7dZGLQOqVqKHB1d1zL1jLIpuVTWf/cz5PO0+5truImicTmadUGxkrKKCo2sN2pGBbXRHrzvkcDi0fN8ypeenN0CVAC5nYOcIDbmxjc7nFeujrw5Xe83DzaRJQzqrY0SwPvn6mA6lZqncXqmoVgEa26+9ZsT31NItKTqc9uMAVZCft+66uZs83d20ZMMBpZ7Pl6e7m7pEhWpUnxh1bhOqNz/bq9xatgVsrrae/KrWrVQvptJRqbWHPnO5vc1u04brZFtVw/armTx5st58881qx5KSkjR48GCtW7dOgwcPVlJSkiRp8+bNOnXqlNatW6fnnntOzzzzjKSqYP7qq69q6dKlWrZsmV599VXlXeEjx9E8hYf4qXd7i3q3t6hjRIgkKcTf13nsxuiW1dp3iAh2vvbDHOzosEDnsR8CN5ous3fV/6617SISHRytKbFTVOmo1LK9SwnZQCM1oFNrje3fXpm5Rfrn5/tVYqs+Ejq0e5S6t22l5L2ntetohgpLy1VWbtexczn6YHOKPNzdNKZf+2rnTLypk8KD/fTB5u907FyOysrtVYstj2Zow95T8vf10s0929bn28R1xLAR7f79+ystrfoCvuTkZC1ZskSSNHHiRE2bNk1z585VcnKyJk6cKJPJpNjYWOXn5yszM1M7d+7UkCFDFBQUJEkaMmSItmzZonHjxhlVNq4TG/ef0cb9Zy7f8Hv//PzbOrnv0+9c2VOq9p7I1N4TTOOpK+H+4bVO9wjyDdKgtlULUU9cqL4I+IaQGzSxxySV28u1bO9SZRbyvwfQGA3qEqEx/drLmlOkt9d/q6Kyms8j6BRZNbBy0ppb4zVrbpGKy8oVbPaRr5eHSmwV8vJwVztLkIrLymXNrTkV4YfrRHw/AANcqXqdo52VlaWw7xcBtmrVSlnfb0lntVoVHh7ubBceHi6r1VrjuMVikdV6+a/vy8rKlJLi2s4NkX5+KixgAdq1KC0t1XEXf9+u8A+1qKCgoM6u1xyVlpUqJaVu53wHWgJVWHhtn5Uu4V3UJbzqARNmn6q/uML9W+vWDvGSpGJbsdZ99+M0rKEdhqmluerbh/DAqr7gRsuNsvhVLYI6k31au8/sdrafMixRRWVFyshLV15JvtxMbgrxC1GHsA5yd3PX9hPbdTD1oLN9qF+oJvaYJE93Tx2xHlF0YFtFB9Ycudp4+Itret8/KC0tVcppdpUBrtTQG9sovk87pWcX6l/J36q4rKLWdh5uVV/U+3nX3I7P3c0kb8+qRen27ydqu7uZJEnenu5ydzM5j/+ghY/X9+3ZDhdXp8EWQ5pMJplMJkOu7e3tra5du16+oSRbZqbM/vxL9Vr4+Pioa3TdPT3vQl6x/P396+x6zZGPt4/auPgZcFV2cbbM5mv7rES3aqve0b2rHQvxC1GIX9UoVF5Jnrae+XHeX+fWnZ0PM3BeIyRa0SFVxzw9PXQk+8f51ltPfqUbQtopKjRanT1byGQyqdhWrGMXjmr/uf06lX2q2nsICwpzPvCgW0Q3dVO3Wuvedfbra3jXP/Lx8VFE18s/yczVgQKgORjRI0pxvW7Q2awCLUk+UGO6yE+dzsyTJdhPw7pH6cz5/GrB+ZaebeXu5qa0CwXOR6qX2CqUmVussKAWGtEjutqidA83k0Z0j5IktvfDVavXoB0aGqrMzEyFhYUpMzNTISFVf7laLBZlZPz4dW9GRoYsFossFot27tzpPG61WjXgZ7t7AGg6rnQBzQd73r+i6+9O263dabsv3/B7qbmp+p8N/31F9wBQf3rFhCmu1w2yVzp0JjNfA7vU/IdqbmGpcwre5gOp6twmVO1bB+vR2/vp2Lkcldvtim4VoDYtA2SrsOvTXdWnj32667h+dUs3jegRrZjwIKVeyJenu7s6RAQr2OyjrPwSfXkwrcZ9AVcYthiyNnFxcVq5cqUkaeXKlRo5cmS14w6HQ3v37pW/v7/CwsI0dOhQffnll8rLy1NeXp6+/PJLDR06tD5LBoBmLT09XdOmTdPYsWOVkJCgt99+W5L017/+VcOGDdOECRM0YcIEbdq0yXnOG2+8ofj4eI0ePVpbtlzZugXgp354aqO7m0mDu0bqlp5ta/wntr3F2b6gxKbXP9mjr75LU4W9UrHtLRrQKUJmHy/tOZ6hNz7Zo7QL1acmnsjIVdKne7TvhFX+Lbw0oFOEYmPCVF5h1+YDZ5T06Z5LjqIDl2LYiPacOXO0c+dO5eTkaPjw4Xr00Uc1c+ZMzZ49W8uXL1dERIQWL14sSRoxYoQ2bdqk+Ph4+fr6auHChZKkoKAgPfTQQ5oyZYok6eGHH3YujAQAGM/d3V1PPvmkunXrpsLCQt1xxx0aMqTqqZvTp0/XvffeW639sWPHtGbNGq1Zs0ZWq1UzZszQ2rVr5e5e+0OegEu50oXt0o/bt/50C9fLseYW66Otte+vD1wLw4L2yy+/XOvxH0ZDfspkMunpp5+utf2UKVOcQRsAUL/CwsKci9jNZrNiYmIuuSg9OTlZCQkJ8vLyUlRUlNq2bav9+/erd+/eFz2nsXBr0ULmHj3kd+ON8mrdWh6BgXLYK2RLT1f+zq+Vv3On9JPHfVvu+kWNh5X9XPGRIzr72usXfd3k7q6ox+fIu3Vrlefm6tQfn62z9wOg4fFkSACAS9LS0pSSkqJevXpp9+7devfdd7Vy5Up1795dTz75pAIDA2W1WtWrVy/nOa7uFtUY+Mf2Ulhioiry8lR87JgqcnLl7m+WuWdPWX4xVS26dlHGP38cLCr89oDKs2t/aqB/v77yatlSRSmHLnnP0IQEeQYH1+n7ANB4ELQBAJdVVFSkWbNm6amnnpLZbNZdd92lhx56SCaTSX/5y1/0wgsvaNGiRVd9fVe3ZTV0S9bTZ1Tw11eVv//baiPXHgEB6jz/Kfn36qXzHTspb3fVgtvCbdtrvYy7r6+633KLKsvLlf7FRtkvsi2muXMnBY0YrtR331P0tLvlqHTUy3azbMna+DTWLVmbu7rYkpWgDQC4pPLycs2aNUvjx4/XqFGjJEktW/74dNXExEQ98MADkmruImW1WmWxWHQ5rm7LauiWrGfPqlKS2exX/bijUvnbt6llQoJCenST/eil5/IGDhsqN28vFezeLV+TpFrqdfP2VvS9v1HJ0aMq271bmna3TG6metluli1ZG5/GuiVrc1cXW7LW664jAICmxeFwaP78+YqJidGMGTOcxzMzf3yC5vr169WxY0dJVbtIrVmzRjabTampqTp16pR69uxZ73XXNYe96oElDhceXBI4aJAkKe8iI96S1GryJLn5+sr6/gd1UyCARokRbQDARX3zzTdatWqVOnXqpAkTJkiq2lVq9erVOnSoav5xZGSknn22ahFfx44dNWbMGI0dO1bu7u5asGBB099xxM1NAf36SZKKLzPn2qdtW3lHRMiWmamSY8dqbePXo4cCBgyQ9f0PVJGbW9fVAmhECNoAgIvq16+fDh8+XOP4iBEjLnrOgw8+qAcffNDIsupVy3EJ8o5oraLvvlNxLb+LnwocPFjSxUez3c1mhd2ZqKLvUpS/Y0ed1wqgcWHqCAAAFxE4bJiCb7lFNqtVGe++d8m2bj4+Msf2UmVFhfK//rrWNmFT75TJzU3WD5gyAjQHBG0AAGoROHSowiZPUllGhtL+9n+qLC6+ZHv/vn3l5u2tov37VVlUVPP1fv1k7t5d51eslD0/36iyATQiTB0BAOBngoYPV6tJE1V2Ll1nX3vtolv0/VTg4O8XQW7dVuvr3m3aSJLCf/VLhf/qlzVe9wwKUsc/Vz3s7fi8p1RZWnq15QNoJAjaAAD8RHBcnFqOH6fStDSdff2NWkenf847OlrekZFViyCPH6+1TempU8rz9qr1tcBBg1RZVqaCPXskSQ67/erfAIBGg6ANAMD3QuLjFTp2jEpTU6tC9mWmi/zgx0WQtY9mS1Lh3r0q3Lu39vMHDZK9pESZHyy94poBNF4EbQAAJPn376fQsWPksNtVcuKEgoYNq9GmPDtbBT9b6Ojm7S3/3rGqLC9X/te76qtcAE0AQRsAAEmeIaGSJJO7u4Ivsn1h8bFjNYL2D4sgC3bvdmmaCYDmg6ANAICk7LVrlb127RWfl7d1q/K2br2mex99bM41nQ+gcWJ7PwAAAMAABG0AAADAAARtAAAAwAAEbQAAAMAABG0AAADAAARtAAAAwAAEbQAAAMAABG0AAADAAARtAAAAwAAEbQAAAMAABG0AAADAAARtAAAAwAAEbQAAAMAABG0AAADAAARtAAAAwAAEbQAAAMAABG0AAADAAARtAAAAwAAEbQAAAMAABG0AAADAAARtAAAAwAAEbQAAAMAABG0AAADAAARtAAAAwAAEbQAAAMAABG0AAADAAARtAAAAwAAEbQAAAMAABG0AAADAAARtAAAAwAAEbQAAAMAADRK0//nPfyohIUHjxo3TnDlzVFZWptTUVCUmJio+Pl6zZ8+WzWaTJNlsNs2ePVvx8fFKTExUWlpaQ5QMAAAAXJF6D9pWq1X/+te/9OGHH2r16tWy2+1as2aNXnrpJU2fPl2ff/65AgICtHz5cknSsmXLFBAQoM8//1zTp0/XSy+9VN8lAwAAAFesQUa07Xa7SktLVVFRodLSUrVq1Urbt2/X6NGjJUmTJk1ScnKyJGnDhg2aNGmSJGn06NHatm2bHA5HQ5QNAAAAuMyjvm9osVj0m9/8Rrfccou8vb01ZMgQdevWTQEBAfLwqConPDxcVqtVUtUIeOvWrauK9fCQv7+/cnJyFBISctF7lJWVKSUlxaV6Iv38VFhQeI3vqnkrLS3VcRd/367wD7WooKCgzq7XHJWWlSol5XSdXjPQEqjCQj4r16K0tFQpp+vuswIAaNzqPWjn5eUpOTlZycnJ8vf3129/+1tt2bKlTu/h7e2trl27utTWlpkps7+5Tu/f3Pj4+KhrdHSdXe9CXrH8/f3r7HrNkY+3j9q4+BlwVXZxtsxmPivXwsfHRxFdIy7bztWBAgBA41bvU0e2bt2qNm3aKCQkRJ6enho1apR2796t/Px8VVRUSJIyMjJksVgkVY2Ap6enS5IqKipUUFCg4ODg+i4bAJqt9PR0TZs2TWPHjlVCQoLefvttSVJubq5mzJihUaNGacaMGcrLy5MkORwOPf/884qPj9f48eN18ODBhiwfABpMvQftiIgI7du3TyUlJXI4HNq2bZs6dOiggQMHau3atZKkFStWKC4uTpIUFxenFStWSJLWrl2rQYMGyWQy1XfZANBsubu768knn9Qnn3yiDz74QO+9956OHTumpKQkDR48WOvWrdPgwYOVlJQkSdq8ebNOnTqldevW6bnnntMzzzzTsG8AABpIvQftXr16afTo0Zo0aZLGjx+vyspKTZ06VXPnztVbb72l+Ph45ebmKjExUZI0ZcoU5ebmKj4+Xm+99ZaeeOKJ+i4ZAJq1sLAwdevWTZJkNpsVExMjq9Wq5ORkTZw4UZI0ceJErV+/XpKcx00mk2JjY5Wfn6/MzMyGKh8AGky9z9GWpFmzZmnWrFnVjkVFRTm39Pspb29vvfLKK/VVGgDgEtLS0pSSkqJevXopKytLYWFhkqRWrVopKytLUtUi9vDwcOc5Pyxw/6FtbVxdxM4C9mvHAvbGhwXsjVNdLGBvkKANAGh6ioqKNGvWLD311FM1FsaaTKZrmtbn6iJ2FrBfOxawNz4sYG+c6mIBO49gBwBcVnl5uWbNmqXx48dr1KhRkqTQ0FDnlJDMzEzntqsWi0UZGRnOc3+6wB0AmhOCNgDgkhwOh+bPn6+YmBjNmDHDeTwuLk4rV66UJK1cuVIjR46sdtzhcGjv3r3y9/e/5LQRALheMXUEAHBJ33zzjVatWqVOnTppwoQJkqQ5c+Zo5syZmj17tpYvX66IiAgtXrxYkjRixAht2rRJ8fHx8vX11cKFCxuwegBoOARtAMAl9evXT4cPH671tR/21P4pk8mkp59+2uiyAKDRY+oIAAAAYACCNgAAAGAAgjYAAABgAII2AAAAYACCNgAAAGAAgjYAAABgAII2AAAAYACCNgAAAGAAgjYAAABgAII2AAAAYACCNgAAAGAAgjYAAABgAII2AAAAYACCNgAAAGAAgjYAAABgAII2AAAAYACCNgAAAGAAgjYAAABgAII2AAAAYACCNgAAAGAAgjYAAABgAII2AAAAYACCNgAAAGAAgjYAAABgAII2AAAAYACCNgAAAGAAgjYAAABgAII2AAAAYACCNgAAAGAAgjYAAABgAJeC9j333OPSMQBA40VfDgD1y+NSL5aVlamkpEQ5OTnKy8uTw+GQJBUWFspqtdZLgQCAa0NfDgAN45JB+/3339fbb7+tzMxMTZ482dk5m81m3X333fVSIADg2tCXA0DDuGTQvueee3TPPfdoyZIlmjZtWn3VBACoQ/TlANAwLhm0fzBt2jTt3r1bZ8+eld1udx6fOHGiUXUBAOoYfTkA1C+XgvbcuXOVmpqqLl26yN3dXZJkMpnonAGgCaEvB4D65VLQPnDggD755BOZTCaj6wEAGIS+HADql0vb+3Xs2FHnz583uhYAgIHoywGgfrk0op2Tk6OEhAT17NlTnp6ezuOvv/66YYUBAOoWfTkA1C+Xgvajjz5qdB0AAIPRlwNA/XIpaA8YMKBOb5qfn6/f//73OnLkiEwmkxYuXKh27drpscce09mzZxUZGanFixcrMDBQDodDf/rTn7Rp0yb5+PjohRdeULdu3eq0HgBoDuq6LwcAXJpLQbt3797OxTPl5eWqqKiQr6+vdu/efVU3/dOf/qRhw4bplVdekc1mU2lpqV5//XUNHjxYM2fOVFJSkpKSkjR37lxt3rxZp06d0rp167Rv3z4988wzWrZs2VXdFwCas7ruywEAl+ZS0N6zZ4/zzw6HQ8nJydq7d+9V3bCgoEBff/21XnjhBUmSl5eXvLy8lJycrCVLlkiq2tN12rRpmjt3rpKTkzVx4kSZTCbFxsYqPz9fmZmZCgsLu6r7A0BzVZd9OQDg8lwK2j9lMpl066236tVXX9UTTzxxxTdMS0tTSEiI5s2bp0OHDqlbt26aP3++srKynOG5VatWysrKkiRZrVaFh4c7zw8PD5fVar1k0C4rK1NKSopL9UT6+amwoPCK3wd+VFpaquMu/r5d4R9qUUFBQZ1drzkqLStVSsrpOr1moCVQhYV8Vq5FaWmpUk7X3WflWlxJXz5v3jxt3LhRoaGhWr16tSTpr3/9q5YuXaqQkBBJ0pw5czRixAhJ0htvvKHly5fLzc1Nv//97zVs2DBj3wwANFIuBe1169Y5/1xZWakDBw7I29v7qm5YUVGh7777Tn/4wx/Uq1cvPf/880pKSqrWxmQyXdM+r97e3uratatLbW2ZmTL7m6/6XpB8fHzUNTq6zq53Ia9Y/v7+dXa95sjH20dtXPwMuCq7OFtmM5+Va+Hj46OIrhGXbefqQMGVutq+fPLkybr77rv1u9/9rtrx6dOn695776127NixY1qzZo3WrFkjq9WqGTNmaO3atc4H5ABAc+JS0P7iiy+cf3Z3d1dkZKT+7//+76puGB4ervDwcPXq1UuSdNtttykpKUmhoaHOKSGZmZnOURKLxaKMjAzn+RkZGbJYLFd1bwBozq62L+/fv7/S0tJcukdycrISEhLk5eWlqKgotW3bVvv371fv3r2vum4AaKpcCtqLFi2qsxu2atVK4eHhOnHihGJiYrRt2za1b99e7du318qVKzVz5kytXLlSI0eOlCTFxcXpnXfeUUJCgvbt2yd/f3/mZwPAVajLvlyS3n33Xa1cuVLdu3fXk08+qcDAQFmtVudAilQ1WGK1Wuv0vgDQVLgUtDMyMvTcc885V6b369dP8+fPrzZ3+kr84Q9/0BNPPKHy8nJFRUVp0aJFqqys1OzZs7V8+XJFRERo8eLFkqQRI0Zo06ZNio+Pl6+vrxYuXHhV9wSA5q4u+/K77rpLDz30kEwmk/7yl7/ohRdeuKYg7+raGtbVXDvW1TQ+rKtpnOpiXY1LQXvevHkaN26c/vKXv0iSPv74Y82bN09vvfXWVd20a9eu+uijj2ocf/vtt2scM5lMevrpp6/qPgCAH9VlX96yZUvnnxMTE/XAAw9Iqjndz2q1ujTdz9W1NayruXasq2l8WFfTONXFuho3V26UnZ2tO+64Qx4eHvLw8NDkyZOVnZ3teqUAgAZXl315Zmam88/r169Xx44dJVVN91uzZo1sNptSU1N16tQp9ezZs07qB4CmxqUR7aCgIK1atUrjxo2TJK1evVpBQUFG1gUAqGNX25fPmTNHO3fuVE5OjoYPH65HH31UO3fu1KFDhyRJkZGRevbZZyVJHTt21JgxYzR27Fi5u7trwYIF7DgCoNlyKWgvXLhQzz33nBYtWiSTyaTevXs7HzgDAGgarrYvf/nll2scS0xMvGj7Bx98UA8++OA11QoA1wOXgvYrr7yiF198UYGBgZKk3Nxcvfjii3W+gh0AYBz6cgCoXy7N0T58+LCzY5aqvn406oEKAABj0JcDQP1yKWhXVlYqLy/P+XNubq7sdrthRQEA6h59OQDUL5emjvzmN7/R1KlTddttt0mSPvvsM+dWTgCApoG+HADql0tBe+LEierevbu2b98uSXr11VfVoUMHQwsDANQt+nIAqF8uBW1J6tChAx0yADRx9OUAUH9cmqMNAAAA4MoQtAEAAAADELQBAAAAAxC0AQAAAAMQtAEAAAADELQBAAAAAxC0AQAAAAMQtAEAAAADELQBAAAAAxC0AQAAAAMQtAEAAAADELQBAAAAAxC0AQAAAAMQtAEAAAADELQBAAAAAxC0AQAAAAMQtAEAAAADELQBAAAAAxC0AQAAAAMQtAEAAAADELQBAAAAAxC0AQAAAAMQtAEAAAADELQBAAAAAxC0AQAAAAMQtAEAAAADELQBAAAAAxC0AQAAAAMQtAEAAAADELQBAAAAAxC0AQAAAAMQtAEAAAADELQBAAAAAxC0AQAAAAMQtAEAAAADELQBAAAAAzRY0Lbb7Zo4caLuv/9+SVJqaqoSExMVHx+v2bNny2azSZJsNptmz56t+Ph4JSYmKi0traFKBgAAAFzWYEH7X//6l9q3b+/8+aWXXtL06dP1+eefKyAgQMuXL5ckLVu2TAEBAfr88881ffp0vfTSSw1VMgAAAOCyBgnaGRkZ2rhxo6ZMmSJJcjgc2r59u0aPHi1JmjRpkpKTkyVJGzZs0KRJkyRJo0eP1rZt2+RwOBqibABolubNm6fBgwdr3LhxzmO5ubmaMWOGRo0apRkzZigvL09SVX/+/PPPKz4+XuPHj9fBgwcbqmwAaHAeDXHThQsXau7cuSoqKpIk5eTkKCAgQB4eVeWEh4fLarVKkqxWq1q3bl1VrIeH/P39lZOTo5CQkItev6ysTCkpKS7VEunnp8KCwmt5O81eaWmpjrv4+3aFf6hFBQUFdXa95qi0rFQpKafr9JqBlkAVFvJZuRalpaVKOV13n5X6MnnyZN1999363e9+5zyWlJSkwYMHa+bMmUpKSlJSUpLmzp2rzZs369SpU1q3bp327dunZ555RsuWLWvA6gGg4dR70P7iiy8UEhKi7t27a8eOHYbcw9vbW127dnWprS0zU2Z/syF1NBc+Pj7qGh1dZ9e7kFcsf3//Ortec+Tj7aM2Ln4GXJVdnC2zmc/KtfDx8VFE14jLtnN1oKC+9O/fv8b6mOTkZC1ZskSSNHHiRE2bNk1z585VcnKyJk6cKJPJpNjYWOXn5yszM1NhYWENUToANKh6D9q7d+/Whg0btHnzZpWVlamwsFB/+tOflJ+fr4qKCnl4eCgjI0MWi0WSZLFYlJ6ervDwcFVUVKigoEDBwcH1XTYA4CeysrKc4blVq1bKysqSVPUtZHh4uLPdD99QErQBNEf1HrQff/xxPf7445KkHTt26B//+If+93//V7NmzdLatWuVkJCgFStWKC4uTpIUFxenFStWqHfv3lq7dq0GDRokk8lU32UDAC7CZDJdc7/s6pQ/pvtdO6b7NT5M92uc6mK6X4PM0a7N3Llz9dhjj2nx4sXq2rWrEhMTJUlTpkzR3LlzFR8fr8DAQP35z39u4EoBAKGhoc4pIZmZmc51MxaLRRkZGc52P/2G8lJcnfLHdL9rx3S/xofpfo1TXUz3a9CgPXDgQA0cOFCSFBUV5dzS76e8vb31yiuv1HdpAIBLiIuL08qVKzVz5kytXLlSI0eOdB5/5513lJCQoH379snf359pIwCarUYzog0AaJzmzJmjnTt3KicnR8OHD9ejjz6qmTNnavbs2Vq+fLkiIiK0ePFiSdKIESO0adMmxcfHy9fXVwsXLmzY4gGgARG0AQCX9PLLL9d6/O23365xzGQy6emnnza6JABoEhrsyZAAAADA9YygDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA0AAAAYoN6Ddnp6uqZNm6axY8cqISFBb7/9tiQpNzdXM2bM0KhRozRjxgzl5eVJkhwOh55//nnFx8dr/PjxOnjwYH2XDAAAAFyxeg/a7u7uevLJJ/XJJ5/ogw8+0Hvvvadjx44pKSlJgwcP1rp16zR48GAlJSVJkjZv3qxTp05p3bp1eu655/TMM8/Ud8kAAADAFav3oB0WFqZu3bpJksxms2JiYmS1WpWcnKyJEydKkiZOnKj169dLkvO4yWRSbGys8vPzlZmZWd9lAwBqERcXp/Hjx2vChAmaPHmypIt/QwkAzU2DztFOS0tTSkqKevXqpaysLIWFhUmSWrVqpaysLEmS1WpVeHi485zw8HBZrdYGqRcAUNPbb7+tVatW6aOPPpKki35DCQDNjUdD3bioqEizZs3SU089JbPZXO01k8kkk8l01dcuKytTSkqKS20j/fxUWFB41feCVFpaquMu/r5d4R9qUUFBQZ1drzkqLStVSsrpOr1moCVQhYV8Vq5FaWmpUk7X3WelsUpOTtaSJUskVX1DOW3aNM2dO7eBqwKA+tcgQbu8vFyzZs3S+PHjNWrUKElSaGioMjMzFRYWpszMTIWEhEiSLBaLMjIynOdmZGTIYrFc8vre3t7q2rWrS7XYMjNl9jdfviEuysfHR12jo+vsehfyiuXv719n12uOfLx91MbFz4Crsouza/yjGFfGx8dHEV0jLtvO1YGCxuLee++VyWTS1KlTNXXq1It+Q3kprg6QMDhy7RgcaXwYHGmc6mJwpN6DtsPh0Pz58xUTE6MZM2Y4j8fFxWnlypWaOXOmVq5cqZEjRzqPv/POO0pISNC+ffvk7+/v7MABAA3r3//+tywWi7KysjRjxgzFxMRUe93VbyhdHSBhcOTaMTjS+DA40jjVxeBIvQftb775RqtWrVKnTp00YcIESdKcOXM0c+ZMzZ49W8uXL1dERIQWL14sSRoxYoQ2bdqk+Ph4+fr6auHChfVdMgDgIn74hjE0NFTx8fHav3//Rb+hBIDmpt6Ddr9+/XT48OFaX/thT+2fMplMevrpp40uCwBwhYqLi1VZWSmz2azi4mJ99dVXeuihhy76DSUANDcNthgSANC0ZWVl6eGHH5Yk2e12jRs3TsOHD1ePHj1q/YYSAJobgjYA4KpERUXp448/rnE8ODi41m8oAaC5adB9tAEAAIDrFUEbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwQJMJ2ps3b9bo0aMVHx+vpKSkhi4HAHAJ9NkA0ESCtt1u17PPPqs333xTa9as0erVq3Xs2LGGLgsAUAv6bACo0iSC9v79+9W2bVtFRUXJy8tLCQkJSk5ObuiyAAC1oM8GgCpNImhbrVaFh4c7f7ZYLLJarQ1YEQDgYuizAaCKR0MXYISysjKlpKS4fsLM/zKumGbgeFaWlJVVp9f8Rd+WdXq95ub8udM6f67urzuu9fi6v2gzYj1tlVWXD5xlZWX1UE3jckX9Nn32NaHPbnzosxunuuizm0TQtlgsysjIcP5stVplsVgu2j42NrYeqgIA1OZK+2yJfhvA9alJTB3p0aOHTp06pdTUVNlsNq1Zs0ZxcXENXRYAoBb02QBQpUmMaHt4eGjBggW67777ZLfbdccdd6hjx44NXRYAoBb02QBQxeRwOBwNXQQAAABwvWkSU0cAAACApoagDQAAABiAoN2I8QjjxmfevHkaPHiwxo0b19Cl4Hvp6emaNm2axo4dq4SEBL399tsNXRKaKfrsxoc+u/Fpbn02QbuR4hHGjdPkyZP15ptvNnQZ+Al3d3c9+eST+uSTT/TBBx/ovffe47OCekef3TjRZzc+za3PJmg3UjzCuHHq37+/AgMDG7oM/ERYWJi6desmSTKbzYqJieEphKh39NmNE31249Pc+myCdiPFI4yBK5eWlqaUlBT16tWroUtBM0OfDVy55tBnE7QBXBeKioo0a9YsPfXUUzKbzQ1dDgDgEppLn03QbqSu5hHGQHNVXl6uWbNmafz48Ro1alRDl4NmiD4bcF1z6rMJ2o0UjzAGXONwODR//nzFxMRoxowZDV0Omin6bMA1za3P5smQjdimTZu0cOFC5yOMH3zwwYYuqdmbM2eOdu7cqZycHIWGhurRRx9VYmJiQ5fVrO3atUu/+tWv1KlTJ7m5VY0dzJkzRyNGjGjgytDc0Gc3PvTZjU9z67MJ2gAAAIABmDoCAAAAGICgDQAAABiAoA0AAAAYgKANAAAAGICgDQAAABiAoA18b8eOHdq9e3dDlwEAcAF9NpoCgjbwvZ07d2rPnj2G3sPhcKiystLQewBAc0CfjaaAfbRx3Vu5cqX+/ve/y2QyqXPnzhozZoxee+01lZeXKygoSC+99JJKS0s1depUubm5KSQkRH/4wx8UExOjp59+WufOnZMkPfXUU+rbt6+ys7P1+OOPKzMzU7Gxsdq6das+/PBDhYSE6K233tKHH34oSZoyZYqmT5+utLQ03XvvverVq5cOHjyoMWPGKC8vT/Pnz5ckLV26VMeOHdNTTz3VYL8jAGgs6LNxXXEA17EjR444Ro0a5cjKynI4HA5HTk6OIzc311FZWelwOByOpUuXOhYtWuRwOByOV155xfHmm286z50zZ47j66+/djgcDsfZs2cdt912m8PhcDj++Mc/Ol5//XWHw+FwbNq0ydGpUydHVlaW49tvv3WMGzfOUVRU5CgsLHSMHTvWcfDgQUdqaqqjc+fOjj179jgcDoejsLDQMXLkSIfNZnM4HA7H1KlTHYcOHTL+lwEAjRx9Nq43Hg0d9AEjbd++XbfddptCQkIkSUFBQTp8+LAee+wxnT9/XjabTW3atKn13K1bt+rYsWPOnwsLC1VUVKRvvvlGr776qiRp+PDhCgwMlCR98803uvXWW9WiRQtJUnx8vHbt2qW4uDhFREQoNjZWkuTn56dBgwZp48aNiomJUXl5uTp37mzUrwAAmgz6bFxvCNpodp5//nlNnz5dI0eO1I4dO5wd8M9VVlZq6dKl8vb2vuZ7/tCR/yAxMVGvv/66YmJiNHny5Gu+PgBcr+iz0ZSxGBLXtUGDBumzzz5TTk6OJCk3N1cFBQWyWCySquYC/sDPz09FRUXOn4cOHaolS5Y4f05JSZEk9enTR59++qkk6csvv1ReXp4kqV+/flq/fr1KSkpUXFys9evXq1+/frXW1atXL2VkZGj16tUaN25c3b1hAGjC6LNxvSFo47rWsWNHPfDAA5o2bZpuv/12vfDCC3rkkUf029/+VpMnT1ZQUJCz7S233KLPP/9cEyZM0K5duzR//nwdOHBA48eP19ixY/Xvf/9bkvTII4/oq6++0rhx4/TZZ5+pVatWMpvN6tatmyZPnqzExETdeeedmjJlim688caL1jZmzBj16dPH+TUmADR39Nm43rDrCHCFbDab3Nzc5OHhoT179uiZZ57RqlWrrvg6999/v6ZPn67BgwcbUCUAQKLPRsNijjZwhc6dO6fZs2ersrJSnp6eeu65567o/Pz8fCUmJqpz58502ABgMPpsNCRGtAEAAAADMEcbAAAAMABBGwAAADAAQRsAAAAwAEEbAAAAMABBGwAAADAAQRsAAAAwwP8P0WPAtDRRke8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 864x432 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "sns.set_style('whitegrid')\n",
    "plt.figure(figsize=(12,6))\n",
    "plt.subplot(1,2,1)\n",
    "ax1 = sns.countplot(x='category', data=y_train_df, palette='Set1', alpha=0.7)\n",
    "ax1.set_title(\"Train label\")\n",
    "\n",
    "for p in ax1.patches:\n",
    "    ax1.annotate(f'\\n{p.get_height()}', (p.get_x()+0.4, p.get_height()), ha='center', va='top', color='white', size=20)\n",
    "\n",
    "plt.subplot(1,2,2)\n",
    "ax2 = sns.countplot(x='category', data=y_test_df, palette='Set1', alpha=0.7)\n",
    "ax2.set_title(\"Test label\")\n",
    "\n",
    "for p in ax2.patches:\n",
    "    ax2.annotate(f'\\n{p.get_height()}', (p.get_x()+0.4, p.get_height()), ha='center', va='top', color='white', size=20)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "tribal-stations",
   "metadata": {},
   "source": [
    "### 5. 모델 구축\n",
    "\n",
    "Basic CNN Model의 성능을 개선하기 위해 [Digit-recognizer competition](https://www.kaggle.com/c/digit-recognizer)에서 CNN을 사용해 최고의 성능을 보여준 모델을 학습에 사용하였다. \n",
    "\n",
    "(MNIST 99.5% 정확도)\n",
    "\n",
    "[How to choose CNN Architecture MNIST](https://www.kaggle.com/cdeotte/how-to-choose-cnn-architecture-mnist)\n",
    "\n",
    "-------------------\n",
    "### 실험\n",
    "```\n",
    "1. n pairs of convolution-subsambling에서 n을 어떻게 정할 것인가?\n",
    "pair를 늘릴 때 이미지 shape가 (28, 28)에서 점점 축소 되므로 max(n)=3이다.\n",
    "\n",
    "2. How many feature maps?\n",
    "n_channel_1, n_channel_2의 값을 정해야 하는데 (8,16),(16,32),(24,48),(32,64),(48,96),(64,128)을 바꿔가면서 성능을 테스트하였고 그 중 best set은 (32,64)이었다.\n",
    "\n",
    "3. Dense layer의 unit 개수?\n",
    "n_dense의 값 32, 64, 128, 256, 512, 1024, 2048 중에서 best는 128이었다.\n",
    "\n",
    "4. dropout?\n",
    "드롭아웃은 네트워크에서 과적합을 막아주는 방법으로 네트워크의 일반화 성능을 높인다.\n",
    "0%, 10%, 20%, 30%, 40%, 50%, 60%, 70% 중에서 40%가 best였다.\n",
    "\n",
    "5. kernel_size = (5,5) is the best?\n",
    "필자는 1개의 kernel_size = (5,5)를 가지는 convolution layer 보다 kernel_size = (3,3)을 가지는 2개의 convolution layer를 연결시켜 비선형성을 높이는 방법을 사용했다.\n",
    "\n",
    "6. max pooling layer\n",
    "model.add(Conv2D(n_channel_1, kernel_size = 5, strides=2, padding='same', activation='relu'))에서\n",
    "max pooling layer 대신 strieds=2를 사용해 subsample 하였다.\n",
    "\n",
    "7. batch normalization\n",
    "https://keras.io/api/layers/normalization_layers/batch_normalization/\n",
    "\n",
    "8. Data augmentation\n",
    "\n",
    "필자가 작성한 github 노트북에 다음과 같은 augmentation 코드를 확인할 수 있다.\n",
    "# CREATE MORE IMAGES WITH DATA AUGMENTATION\n",
    "datagen = ImageDataGenerator(\n",
    "        rotation_range=15,\n",
    "        zoom_range = 0.15,  \n",
    "        width_shift_range=0.1, \n",
    "        height_shift_range=0.1)\n",
    "        \n",
    "9. 마지막 layer에서 Dense(128)가 아닌 128C4를 사용하였는데 이게 더 좋은 성능을 보여줬나보다.\n",
    "\n",
    "```\n",
    "-----------------\n",
    "### 결론\n",
    "\n",
    "```\n",
    "\n",
    "32C3: model.add(Conv2D(n_channel_1, kernel_size = 3\n",
    "32C5S2: model.add(Conv2D(n_channel_1, kernel_size = 5, strides=2\n",
    "\n",
    "Model Architecture\n",
    "\n",
    "784 - [32C3-32C3-32C5S2] - [64C3-64C3-64C5S2] - 128C4 - 3\n",
    "with 40% dropout, batch normalization, and data augmentation added\n",
    "\n",
    "15개의 CNN 모델을 앙상블한 결과는 이 모델(99.5%)보다 높은 99.75% 정확도를 보였다고 한다.\n",
    "```\n",
    "[25 Million Images! [0.99757] MNIST](https://www.kaggle.com/cdeotte/25-million-images-0-99757-mnist)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "spanish-windows",
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "from keras import models\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout, Flatten, Conv2D, BatchNormalization\n",
    "\n",
    "def build_model():\n",
    "    \n",
    "    n_channel_1=32\n",
    "    n_channel_2=64\n",
    "    n_dense=128\n",
    "    \n",
    "    model=Sequential()\n",
    "    model.add(Conv2D(n_channel_1, (3,3), activation='relu', input_shape=(28,28,3)))\n",
    "    model.add(BatchNormalization())\n",
    "    model.add(Conv2D(n_channel_1, kernel_size = 3, activation='relu'))\n",
    "    model.add(BatchNormalization())\n",
    "    model.add(Conv2D(n_channel_1, kernel_size = 5, strides=2, padding='same', activation='relu'))\n",
    "    model.add(BatchNormalization())\n",
    "    model.add(Dropout(0.4))\n",
    "\n",
    "    model.add(Conv2D(n_channel_2, kernel_size = 3, activation='relu'))\n",
    "    model.add(BatchNormalization())\n",
    "    model.add(Conv2D(n_channel_2, kernel_size = 3, activation='relu'))\n",
    "    model.add(BatchNormalization())\n",
    "    model.add(Conv2D(n_channel_2, kernel_size = 5, strides=2, padding='same', activation='relu'))\n",
    "    model.add(BatchNormalization())\n",
    "    model.add(Dropout(0.4))\n",
    "\n",
    "    model.add(Conv2D(n_dense, kernel_size = 4, activation='relu'))\n",
    "    model.add(BatchNormalization())\n",
    "    model.add(Flatten())\n",
    "    model.add(Dropout(0.4))\n",
    "    model.add(Dense(3, activation='softmax'))\n",
    "\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "scientific-bibliography",
   "metadata": {},
   "source": [
    "### 6. 검증 데이터 분리\n",
    "- 훈련(60%), 검증(20%), 테스트(20%) 데이터를 사용한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "assured-wayne",
   "metadata": {},
   "outputs": [],
   "source": [
    "X_train_partial , X_val = X_train[:len(X_train)-len(X_test)], X_train[len(X_train)-len(X_test):]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "structural-sphere",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "학습 데이터 수: 3429개\n",
      "검증 데이터를 제외한 학습 데이터 수 : 2572개\n",
      "검증 데이터 수: 857개\n"
     ]
    }
   ],
   "source": [
    "print(f\"학습 데이터 수: {len(X_train)}개\\n검증 데이터를 제외한 학습 데이터 수 : {len(X_train_partial)}개\\n검증 데이터 수: {len(X_val)}개\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "romantic-vegetable",
   "metadata": {},
   "outputs": [],
   "source": [
    "y_train_partial , y_val = y_train[:len(y_train)-len(y_test)], y_train[len(y_train)-len(y_test):]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "suitable-nicholas",
   "metadata": {},
   "outputs": [],
   "source": [
    "model = build_model()\n",
    "model.compile(optimizer=\"adam\",\n",
    "              loss=\"sparse_categorical_crossentropy\",\n",
    "              metrics=[\"acc\"]\n",
    "             )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "smooth-victorian",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/25\n",
      "41/41 [==============================] - 19s 261ms/step - loss: 1.5040 - acc: 0.4438 - val_loss: 1.1598 - val_acc: 0.3162\n",
      "Epoch 2/25\n",
      "41/41 [==============================] - 1s 17ms/step - loss: 0.7289 - acc: 0.7225 - val_loss: 1.5468 - val_acc: 0.3162\n",
      "Epoch 3/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.3803 - acc: 0.8581 - val_loss: 2.2087 - val_acc: 0.3022\n",
      "Epoch 4/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.2734 - acc: 0.8966 - val_loss: 4.0805 - val_acc: 0.3162\n",
      "Epoch 5/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.1718 - acc: 0.9385 - val_loss: 3.4033 - val_acc: 0.3536\n",
      "Epoch 6/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.1534 - acc: 0.9439 - val_loss: 4.2083 - val_acc: 0.3512\n",
      "Epoch 7/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.1176 - acc: 0.9586 - val_loss: 2.8499 - val_acc: 0.4142\n",
      "Epoch 8/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.1036 - acc: 0.9631 - val_loss: 2.1584 - val_acc: 0.4516\n",
      "Epoch 9/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0875 - acc: 0.9669 - val_loss: 1.7137 - val_acc: 0.6021\n",
      "Epoch 10/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0803 - acc: 0.9709 - val_loss: 2.1707 - val_acc: 0.6033\n",
      "Epoch 11/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.1058 - acc: 0.9637 - val_loss: 0.7958 - val_acc: 0.7853\n",
      "Epoch 12/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0692 - acc: 0.9759 - val_loss: 0.5758 - val_acc: 0.8191\n",
      "Epoch 13/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0528 - acc: 0.9808 - val_loss: 0.6979 - val_acc: 0.7970\n",
      "Epoch 14/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0415 - acc: 0.9846 - val_loss: 0.1233 - val_acc: 0.9522\n",
      "Epoch 15/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0358 - acc: 0.9840 - val_loss: 0.0542 - val_acc: 0.9755\n",
      "Epoch 16/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0231 - acc: 0.9963 - val_loss: 0.0447 - val_acc: 0.9837\n",
      "Epoch 17/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0237 - acc: 0.9932 - val_loss: 0.0382 - val_acc: 0.9848\n",
      "Epoch 18/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0525 - acc: 0.9811 - val_loss: 0.6241 - val_acc: 0.8436\n",
      "Epoch 19/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0731 - acc: 0.9755 - val_loss: 0.2406 - val_acc: 0.9253\n",
      "Epoch 20/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0688 - acc: 0.9811 - val_loss: 0.1755 - val_acc: 0.9277\n",
      "Epoch 21/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0485 - acc: 0.9871 - val_loss: 0.1561 - val_acc: 0.9510\n",
      "Epoch 22/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0360 - acc: 0.9869 - val_loss: 0.0590 - val_acc: 0.9813\n",
      "Epoch 23/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0256 - acc: 0.9907 - val_loss: 0.0306 - val_acc: 0.9930\n",
      "Epoch 24/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0237 - acc: 0.9925 - val_loss: 0.0456 - val_acc: 0.9860\n",
      "Epoch 25/25\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0152 - acc: 0.9946 - val_loss: 0.2166 - val_acc: 0.9487\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7fa8a0206890>"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "n_train_epoch = 25\n",
    "\n",
    "model.fit(x=X_train_partial,\n",
    "          y=y_train_partial,\n",
    "          epochs=n_train_epoch,\n",
    "          batch_size=64,\n",
    "          validation_data=(X_val, y_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "controlling-seafood",
   "metadata": {},
   "outputs": [],
   "source": [
    "history = model.history.history"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "confirmed-thing",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['loss', 'acc', 'val_loss', 'val_acc'])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "history.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "surgical-audio",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEWCAYAAABsY4yMAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAA7wElEQVR4nO3dd1hTZ/sH8G9AICA4sIjixjoQqxJEtNaqOCtS3LViqYPa9tW6taJ1YB2tWkVp3drWlg4niKK2jp/bqhWrvsYW6mCIuJEhkfH8/njeRFBG1slJcu7PdXFhkjPuJwfvnDxTxhhjIIQQYnVsxA6AEEKIMCjBE0KIlaIETwghVooSPCGEWClK8IQQYqUowRNCiJWiBE8qFBYWhl27dhl9WzEFBATg1KlTRj/ue++9h23btgEAdu/ejVGjRmm1ra5u374NHx8fFBYW6rU/kQZK8FbKx8dH89O8eXO0atVK83j37t06HWvjxo3o37+/0bc1R+vXr0dISMhLzz98+BAtW7bEP//8o/Wx3n77bWzevNkocb34geTh4YGEhATY2toa5fjFNWvWDLdu3TL6cYnpVRI7ACKMhIQEzb8DAgKwYMECvP766y9tV1BQgEqV6M9A7e2330ZkZCRSUlJQr149zfPx8fFo2rQpmjZtKmJ0hOiG7uAl5o8//sCbb76J9evXo2PHjggPD0dmZiY+/PBDtG/fHn5+fvjwww9x584dzT7FqxJ27tyJd999F19++SX8/PwQEBCAo0eP6rVtSkoKQkJC4OPjgxEjRiAiIgJTp04tNW5tYoyMjMTQoUPh4+ODUaNG4eHDh5rXY2Ji0LVrV/j7+2PNmjVlvj+1atVC+/btERsbW+L5mJgYBAcHVxhHceryq508eRK9e/eGr68v5s+fj+KDyJOTkxEaGgp/f3/4+/tjypQpePLkCQBg2rRpuH37Nj766CP4+Phgw4YNSE1NRbNmzVBQUAAAyMjIwEcffYR27dqhR48e2Lp1q+bYUVFRmDBhAqZPnw4fHx8EBgbi8uXLZb4HZcnKysL06dPRvn17dO3aFatXr0ZRUREA4NatWxg+fDh8fX3h7++PiRMnAgAYY1i0aBE6dOgAhUKBoKAgnb4FEcNQgpeg+/fvIzMzE0eOHMHnn3+OoqIiDBgwAEeOHMGRI0fg4OCA+fPnl7n/pUuX0KhRI5w5cwZhYWGYNWsWyprxorxtp06dilatWuGPP/7AuHHjXkqqxWkT4549e7B48WKcPn0a+fn5muqRpKQkREREYMmSJTh+/DgeP35cZlIGgH79+pWoxrp+/TquXbuGoKAgnd8rtYcPH2LcuHGYOHEizpw5g/r16+PChQua1xlj+PDDD3H8+HHs27cPd+7cQVRUFABg6dKl8PDwwNq1a5GQkIAPPvjgpeNPnjwZtWrVwvHjx7Fq1SosX74cp0+f1rx++PBhBAYG4vz58wgICMDnn39eYcwv+vzzz5GVlYWDBw/ihx9+QGxsLHbs2AEAWLlyJTp27Ihz587h2LFjGD58OADgxIkTOH/+PA4cOIA///wTkZGRqFatms7nJvqhBC9BNjY2GD9+POzt7SGXy1G9enX06tULjo6OcHZ2xscff4xz586Vub+HhweGDBkCW1tb9O/fH/fu3cP9+/d12vb27du4fPmyJo62bdsiICCgzHNqE+OAAQPQqFEjyOVy9O7dG0qlEgCwf/9+dOnSBX5+frC3t8eECRNgY1P2n36PHj1w//59TQKOjY1Fp06d4OrqqvN7pXbs2DE0adIEvXv3hp2dHd5//3288sormtcbNGiAjh07wt7eHq6urhg5cqRWxwWA9PR0XLhwAVOnToWDgwO8vLwwePDgEh+Yvr6+6Ny5M2xtbREcHIxr165pdWy1wsJCxMfHY8qUKXB2dkbdunUxcuRIzQdhpUqVcPv2bdy9excODg5o27at5vmcnBxcv34djDE0btwYNWvW1OncRH9U+SpB1atXh4ODg+bx06dPsXjxYhw/fhyZmZkAgJycHBQWFpbaiFc8MTk6OgIAcnNzSz1XWds+evQIVatW1TwHALVr10Z6enqpx9EmRjc3txLnUsd09+5d1KpVS/Oak5NTuXeRjo6O6N27N2JiYuDj44O4uDh8+umnWsdRmhdjkMlkqF27tubx/fv3sXDhQpw/fx45OTlgjKFKlSplHu/FY1etWhXOzs6a5zw8PHDlyhXN4+LXQS6XQ6VS6dT+8ujRI+Tn58PDw6PEOTIyMgDwaqSVK1di0KBBqFq1KkaOHIlBgwahQ4cOCAkJwfz585GWloaePXvi008/LRErEQ7dwUuQTCYr8Xjz5s24ceMGtm7digsXLiA6OhoAyqx2MQY3NzdkZmbi6dOnmufKSu6GxlizZs0SVTJPnz7F48ePy92nf//+2L9/P06ePImcnBx07drVoDjc3NxKxMAYK1He5cuXQyaTIS4uDhcuXMDSpUu1fv9r1qyJzMxMZGdna55LT0+Hu7u7Vvtro3r16rCzs8Pt27dLPYebmxsWLFiAEydOICIiAhEREZqeOKGhodi5cyfi4+Nx8+ZNbNy40WhxkfJRgifIycmBg4MDqlSpgsePH+Prr78W/Jx16tRBy5YtERUVhWfPniEhIQFHjhwRJMZevXrh//7v/3D+/Hk8e/YMq1at0jQOlqVt27ZwcXHBnDlz0KdPH9jb2xsUR+fOnZGYmIjffvsNBQUF2LJlS4lqrZycHDg5OcHFxQUZGRkvJcFXXnkFKSkppR67du3a8PHxwfLly6FSqXDt2jVs374db7/9tlaxlSY/Px8qlUrzAwC9e/fGihUrkJ2djbS0NHz77beac6jbDQCgatWqkMlksLGxwaVLl/DXX38hPz8fjo6OsLe3L7d6jBgXvdME77//PlQqFdq3b4933nkHnTp1Msl5ly1bhosXL8Lf3x+RkZElEqkxY2zSpAnmzJmDqVOnolOnTqhSpUqJ6pLSyGQy9OvXD2lpaejXr5/Bcbi6umLlypX46quv4O/vj1u3bkGhUGheHzduHK5evYq2bdtizJgx6NmzZ4n9x4wZgzVr1qBt27bYtGnTS8dfvnw50tLS0KlTJ4wbNw6ffPJJqd1itRUYGIhWrVppfnbu3InZs2fD0dER3bt3x7Bhw9C3b18MHDgQAHD58mUMHjwYPj4++PjjjzFr1izUq1cPOTk5+Oyzz9CuXTt07doV1apVw+jRo/WOi+hGRgt+EHMxceJEeHp6Yvz48WKHQohVoDt4IppLly4hOTkZRUVFOHbsGA4dOoTu3buLHRYhVoN60RDR3L9/H5988gkeP36MWrVqYd68eWjRooXYYRFiNaiKhhBCrBRV0RBCiJUyqyqaixcvwsHBASqVqsRAHKmRcvmp7NIsOyDt8htSdpVKhTZt2pT6mlklePUwa6VSCS8vL7HDEY2Uy09ll2bZAWmX35Cyq6fkKA1V0RBCiJWiBE8IIVaKEjwhhFgps6qDJ4SYRn5+PlJTU5GXlyd2KBr5+fnl1idbM23KLpfLUbduXdjZ2Wl9XErwhEhQamoqXFxc0LBhw5dmFxXL06dPS0wfLSUVlZ0xhgcPHiA1NRWNGjXS+rhURUOIBOXl5aFGjRpmk9xJ+WQyGWrUqKHzNy5K8IRIFCV3y6LP9aIEb+GuXwfKWcqUECJhlOAt3LRpwKBBQLGFkQghBAAleIuWnQ3ExwMFBcClS2JHQ4j23nvvPRw/frzEcz/++CPmzp1b5vaXL18GAHzwwQd48uTJS9tERUWVuhhKcQcPHkRSUpLm8cqVK3Hq1Cldwy/Tzp07MX/+fKMdz1CU4C3Ynj2Aus3lzz/FjYUQXfTt2xfx8fElnjtw4AD69u1b4b4bNmzQekHyF72Y4CdMmGDQylfmjrpJWrBt24DatYFnz4ALF8SOhliqLVuAzZuNe8xRo4DQ0LJf79WrFyIjI/Hs2TPY29sjNTUV9+7dw549e7B48WKoVCr06tWr1NW9AgICsH37dri6umLNmjWIiYmBq6srateuDW9vbwDA1q1b8euvvyI/Px8NGjTAkiVLoFQqcfjwYZw9exZr1qxBVFQUVq9ejS5duqB37944ffo0vvzySxQWFqJly5aIiIiAvb09AgIC0K9fPxw5cgQFBQWIjIxE48aNK3wPUlNTMXPmTDx69Aiurq5YvHgxPDw8sG/fPnzzzTewsbGBi4sLoqOjkZSUhIiICOTn56OoqAhRUVFo2LChvm+/Bt3BW6isLF49M2gQ4OtLd/DEslSrVg2tWrXCsWPHAADx8fHo2bMnJk2ahJ07d2L37t04d+4crl27VuYxrly5gvj4eMTExGDDhg2aKhwA6NGjB3bs2IHdu3fD09MT27dvh0KhQEBAAKZPn47Y2FjUr19fs71KpcKMGTOwYsUKxMXFobCwED/99JPm9erVq2PXrl0YOnQoNmv5abhgwQL0798fcXFxCAoKwoIFCwAAq1evxqZNm7B7926sWbMGALB9+3aEhoYiNjYWO3bsqHDNYG3RHbyAfv8d8PQEtPiw15m6embIEGDvXmDZMv5YLjf+uYh1Cw0t/25bKIGBgYiPj0f37t2xd+9ezJkzB/v27cPWrVtRUFCAe/fu4d9//0Xz5s1L3f/8+fPo3r27ZoBQQECA5rXExERERkYiKysLOTk5eOONN8qN5caNG6hbt65mEFH//v0RHR2NESNGAIBmEfSWLVvi999/16p8CQkJiIqKAgAEBwdj6dKlAAAfHx/MmDEDb731Fnr06AEAaNWqFdatW4c7d+6gZ8+eRrl7B+gOXjCMAQMHAh9+KMzxt20DPDyA11/nd/AFBcCVK8KcixAhdOvWDadPn8Z///tf5OXloWrVqti8eTO+++47xMXFoUuXLlCpVHode8aMGZgzZw7i4uIwbtw4PHv2zKBY1dMD2NjYoLCw0KBjzZ8/HxMnTkR6ejoGDhyIR48eoU+fPlizZg3kcjnGjBmD06dPG3QONcETfGFhIfr164cPhcp0ZurOHV6NcugQ8O+/xj128eoZGxue4AGqpiGWpXLlyvD398fMmTMRGBiI7OxsODo6wsXFBffv39dU35TFz88PBw8eRF5eHrKzs3HkyBHNazk5OXBzc0N+fj7i4uJKnDMnJ+elYzVq1AhpaWm4desWACA2NhZ+fn4Glc/Hxwd79+4FAMTFxaFt27YAgOTkZLRu3RoTJkxA9erVcefOHaSmpqJevXoIDQ1Ft27d8Pfffxt0bjXBq2i2bNmCxo0bIzs7W+hTmZXExOf/3rgRWLzYeMfeswdQqYDBg/njhg2B6tUpwRPL07dvX4wdOxbLly+Hh4cHWrRogbfeegu1atWCQqEod19vb2/06dMHwcHBcHV1xWuvvaZ5bcKECRg8eDBcXV3RunVrTVLv06cPZs+ejR9++AGrVq3SbO/g4IDFixdjwoQJmkbWd99916CyzZ49G+Hh4di0aZOmkRUAlixZglu3boExhvbt26N58+ZYvXo1Jk6ciEqVKuGVV14x3g0xE1B6ejoLDQ1lp06dYmPGjKlw+6tXr5b4bck2bmQMYKx1a8Zq1mRMpdJ+34rK368fYx4ejBUWPn+uWzfGfH31i9WcWMO115cpy26O73Nubq7YIYhG27KXdt3Ku5aC3sEvWrQI06ZNK/UrUWlUKhWUSiXy8vIsftrQM2fcUKlSDXzwQSrGjauHNWtS0bNnllb7llf+nBwbxMc3wZAhj/H33xma5xs2dMOWLa7466+/YW9vlCKIwhquvb5MWfb8/Hw8NbPhz4wxs4vJVLQtu65TKguW4I8cOQJXV1e0bNkSf/zxh1b7WNOarI8e8d4zH31UD198AezbVxcTJmi3b3nl/+kn3u/9o49c4eXlqnm+Z09g0yagsNALlvzWWcO115cpy65UKs1ual5Lmi54x44d2LJlS4nnFApFmSNxK6Jt2e3s7F76Gykv4QuW4C9cuIDDhw/j2LFjUKlUyM7OxtSpU7Fs2TKhTmlWEhOBJk0AW1sgLAyIiABu3AB0mMq5VNu2AXXqAB06lHxeXV154cLzfxNChDFw4EAMHDhQ7DAqJFgvmilTpuDYsWM4fPgwli9fjvbt20smuTMGJCXxBA/wUX0yGb/DNsSTJ8C+fc97zxTXuDFQtSo1tBJCnqN+8AK4fRvIzQVefZU/rlcPeOstPhy8oED/46p7zwwZ8vJrMhm/c6cETwhRM0mC9/f3x7p160xxKrOgnstIfQcPAGPGAOnpfNSpvrZu5dUz7duX/rpCwWeVzM/X/xyEEOtBd/ACUPeBL57g+/ThI0/Xr9fvmE+eAPv3877vL1bPqPn68jv8q1f1OwchxLpQghdAYiJgb8+rZtQqVeJ18fv2AcnJuh8zLq7k4KbS0IhWIpToaD6gzsaG/46ONux4jx49QnBwMIKDg9GxY0d06tQJQ4YMQXBwsEHTCpQ1V3xx+s4Bb25zvWuDJhsTQGIib/S0tS35/OjRwMKFvC5+3jzdjrltG1C3btnVMwCv83dx4T1pRo3SOWxCShUdzasYc3P541u3+GMACAnR75jVq1dH7P/WmoyKioKTkxOGDRum6SpYUFCASpV0T08bNmyocJsJ2vZXtgKU4AWQmPi8gbW4hg2f91f/7DN+V68Nde+Z//yn7OoZgL/m40N38MS4Zs16ntzVcnP58/om+NLMnj0bTk5OUCqVUCgUCAwMxMKFC6FSqSCXy7Fo0SJ4enpi586dOHz4MJ4+fYqUlBR0794d06dPB/B8rvjc3Fx88MEH8PX1RUJCAtzd3bF69WrI5XLMmDFDMwf80aNHsXjxYjg5OUGhUCAlJUWr9kJd5npPTExEeHi40ed61wZV0RhZURGfXKx4/XtxY8YAqam8Pl1bu3fzwU2l9Z55ka8v8NdfhvXWIaS4sqoU9alqrEhGRgZ++eUXhIeHw9PTE9HR0YiJicH48eOxYsUKzXZKpRKRkZGIi4vDvn37kJ6e/tKxbt26hZCQEOzduxcuLi44cOBAiddVKhXmzJmDDRs2YOfOnXj48KHWceoy1/svv/wiyFzv2qAEb2S3b/MFsMtK8EFBgLs7oMU3SQ119Yy/f8XbKhT8/BId7U8EUGxdDK2eN0Tv3r1h+7+6zaysLEyYMAF9+/bF4sWLkVhsBr8OHTrAxcUFDg4OaNy4MdLS0l46Vt26dTWjPr29vV/a5vr166hXrx7q/a+xLDAwUOs4ExISNMsLBgcH48//fW1Wz/W+detWzbTCbdq0wbp167B+/Xrcvn0bchMu2kAJ3shK60FTnJ0drx/fswco5W/yJZmZFfeeKU7d0EpL+BFjWbgQcHIq+ZyTE3/e2IoP11+5ciX8/f2xZ88erFmzpkTjq32xCZdsbW1LnaNdm22MrbS53oOCggSZ610blOCNTJ3gS6uDVxs9mlflaLPyV1wcr54pr/dMcU2bApUrUz08MZ6QEN69t0EDPqCuQQP+2Jj176XJysqCu7s7AGDXrl1GP36jRo2QkpKC1NRUAHhpEfDy6DLXe0pKiiBzvWuDGlmNLDERcHAo2UXyRY0bA92783niZ858ubdNcVu38mNpUz0D8GO1aUMJnhhXSIjwCf1FYWFhmDFjBtasWYPOnTsb/fhyuRxz585FWFgYnJyc0LJlS6331WWu9w0bNiA2Ntb4c71rQ7/Zi4VhDfPB9+vHWIsWFW+3dSufL37fvpdfU5f/8WPG7O0ZmzxZtxjGj2fMyYmxggLd9jMHlnztDUXzwZt+Pvjs7GzGGGNFRUVs7ty57NtvvzV5DIwJNx88VdEYmXoWyYoEBwNubuWPbFX3ntG2ekbN15d3YzPhN0FCLNK2bdsQHByMwMBAZGVl4Z133hE7JKOiKhojUneRfOutire1twdGjACWL+dz1NSu/fI227bxngraVs+oFR/R2qKFbvsSIiUjRozAiBEjSjxn7LnexUQJ3ohSU4G8vPIbWIsLCwOWLgW++w4IDy/5WmYmcOAAMG4cb9jSRbNmgKMj70nz3nu67UukgzEGma5/XBJgrnO9M8Z03oeqaIyotFkky9O0KdClC+8TX1RU8jVdBje9qFIlamgl5ZPL5Xjw4IFeSYOYHmMMDx480LkPPd3BG1FFfeBLM2YMMGwYcOgQ0KPH8+e3buXVM+3a6ReLry//ZlBUpF3/eSItdevWRWpqKu7duyd2KBr5+fmws7MTOwxRaFN2uVyOunXr6nRcSvBGlJgIyOV8znZt9e8P1KjB7+LVCf7JExv89hvwySe6V8+oKRTA11/zmJo10+8YxHrZ2dmhkaHrRxoZrcdr/LLTvZ0RqWeR1OWOWS4H3n8f2LULyMjgzx054qJX75niaOpgQggleCPStovki8LC+ORg33/PH+/f72JQ9QzAe8/I5ZTgCZEySvBGUlhY/iyS5fHyAjp14tU0jx4BJ086Y/Bg/atnAN7Q2qoVzUlDiJRRgjeS1FTe60WfBA/wxtakJGDSJKCgQKZX75kX+fryBP9iDx1CiDRQgjcSfXrQFDdwIFCtGq+m8fB4Bj8/w2Py9eWLhfz7r+HHIoRYHkrwRqLNLJLlcXQEQkP5v3v1yjKoekZNoeC/qZqGEGmiBG8kiYk8SXt46H+MsWP54Kf+/R8bJSZvbz4lAjW0EiJNlOCNJCmJ370bMqioaVM+Qdirr+q/qnxx9va8oZUSPCHSRAneSPTtIik0dUMrjUgnRHoowRtBYSFw/br+9e9CUiiAx4+BGzfEjoQQYmqU4I0gOdmwLpJCohGthEgXJXgjMLSLpJBatuQLfVOCJ0R6KMEbga7TBJuSgwNP8tRVkhDpoQRvBImJgJNT6asymQNfX34HTw2thEgLJXgjSEzkDazmujiOry/w8CFw65bYkRBCTIkSvBGYaxdJNRrRSog0UYI3UEEB7yJpzgm+VSs+uyQ1tBIiLZTgDZSczJO8OSd4uZxPW0AJnhBpoQRvIHPuIlmcQkEjWgmRGkrwBjJ0FklT8fUF7t3j89YTQqSBEryBEhMBZ2egVi2xIykfjWglRHoowRtIPYukuXaRVGvdGrC1pZ40hEhJJaEOrFKpEBISgmfPnqGwsBC9evXC+PHjhTqdaBITgTZtxI6iYo6OfO1XuoMnRDoES/D29vb4/vvvUblyZeTn52PYsGF488030cYSsqGWCgr4LI2DBokdiXZ8fYH9+3lDq7l/4yCEGE6wKhqZTIbKlSsDAAoKClBQUACZlWWVmzfNv4tkcb6+QEYGkJ4udiSEEFMQ7A4eAAoLCzFgwAAkJydj2LBhaN26dbnbq1QqKJVK5OXlQalUChmaURw7VhlAfdjZ3YRS+dRoxxWq/K6ujgAaIiYmBV27Zhv9+MZgKddeCFIuOyDt8gtVdkETvK2tLWJjY/HkyROMHTsW//zzD5o2bVrm9g4ODvDy8oJSqYSXl5eQoRnFwYP8d48eDeHubrzjClX++vX5koJ379aDub69lnLthSDlsgPSLr8hZS/vg8EkvWiqVKkCf39/HD9+3BSnM5nERMDFBahZU+xItFO5MtC8OfWkIUQqBEvwDx8+xJMnTwDwrx+nTp2Cp6enUKcThbnPIlkahYJ60hAiFYJV0dy9exczZsxAYWEhGGPo3bs3unbtKtTpRJGY+HwAkaXw9QV+/BG4c8f8B2cRQgwjWIJv3rw5YmJihDq86PLzeS+aoUPFjkQ36g+kCxeAPn3EjYUQIiwayaqnmzeBwkLL6SKp1qYNr1KiahpCrB8leD1ZyiRjL3Jx4Wu07t8vdiSEEKFRgteTpUwTXJr33gNOnQIk2uWYEMmgBK+nxESgShXAzU3sSHQXGspXeNq8WexICCFCogSvJ/U6rJbURVLN3R0ICgK+/x549kzsaAghQqEEr6ekJMusnlELC+MLgOzZI3YkhBChUILXw7NnvBeNpTWwFterF1CnDrBxo9iREEKEQgleDzduAEVFln0Hb2sLjBzJe9OkpIgdDSFECJTg9WDJPWiKGzWKzw3/3XdiR0IIEQIleD1YS4Jv1Ajo1o33pikqEjsaQoixUYLXQ1ISUK0aUKOG2JEYLiyMtyccPix2JIQQY6MErwdLnEWyLP36Aa6u1NhKiDWiBK8HdR94ayCXA8OHA7t2AQ8eiB0NIcSYKMHrSKUCkpOtJ8EDwOjRvOvnjz+KHQkhxJgowevIGrpIvqhVK8DPj1fTMCZ2NIQQY6EEryNLnUWyImFhwJUrwLlzYkdCCDEWSvA6spYuki8aOhRwcqLGVkKsCSV4HSUmAtWrW0cXyeKqVAGGDAF+/hnIzhY7GkKIMVCC15E19aB5UVgYT+7btokdCSHEGCjB68jSZ5Esz+uvA82aUTUNIdaCErwO8vJ4F0lra2BVk8n4XTyt9kSIdaAEr4Pr13k3Qmu9gweer/a0aZPYkRBCDEUJXgfW2oOmuJo1gbffptWeCLEGlOB1IIUED/Bqmvv3gbg4sSMhhBiCErwOkpL4xFzVq4sdibB69gTq1qXGVkIsHSV4HVhzF8ni1Ks9HThAqz0RYskowetAKgke4AmeMeDbb8WOhBCiL0rwWnr6lN/NSiXBN2oEdO9Oqz0RYsm0SvC5ubko+t//8hs3buDQoUPIz88XNDBz8++//LdUEjzApxG+dQs4dEjsSAgh+tAqwQ8fPhwqlQoZGRkYPXo0YmNjMWPGDKFjMytJSfy3tQ5yKg2t9kSIZdMqwTPG4OjoiN9++w3vvvsuVq1ahSR1xpMIqXSRLE4uB957D4iJ4d0mCSGWResEn5CQgLi4OHTp0gUANFU2UnH+PB8EVK2a2JGYFq32RIjl0irBz5w5E+vWrUP37t3RpEkTpKSkwN/fX+jYzMa9e/wu9p13xI7E9F57DWjXjk9dIPRqT+fOATdv2gt7EkIkpJI2G7Vr1w7t2rUDwO/cq1evjs8++0zQwMzJd9/xu9gPPxQ7EnGMHs3LfuIE0KmTMOe4fRsICABat3bHW28Jcw5CpEarO/gpU6YgOzsbubm56Nu3L/r06YONEml5KyoC1q3jic3bW+xoxDFsGFC7NjBpElBYKMw5pk/nc9H/979yWheWECPRKsEnJSXB2dkZBw8exJtvvolDhw4hNjZW6NjMwqFDvIvkRx+JHYl4nJ2B5cuBP/8E1q83/vFPngSiowFPT+Dx40o0epYQI9EqwRcUFCA/Px8HDx5EQEAA7OzsIJPJhI7NLKxdC7zyCjBwoNiRiOudd4CuXYGZM4G7d4133MJCYNw4PvfNhg38uQsXjHd8QqRMqwT/zjvvICAgAE+fPoWfnx/S0tLg7OwsdGyiu30biI3lw/YdHMSORlwyGfDNN0BODvDpp8Y77oYNwMWLwLJlQIcOgK0towRPiJFo1cgaGhqK0NBQzeM6depgy5Yt5e6Tnp6O6dOn48GDB5DJZBgyZAjef/99w6I1sU2b+B3mmDFiR2IevLyAyZOBL7/kUwp37GjY8R4+BGbNAjp35gt+y2SAp6cKFy7IjRMwIRKnVYLPysrC119/jXPnzgHgvWrGjh0LFxeXMvextbXFjBkz4O3tjezsbAwcOBAdO3bEqxYyFLSggNc39+ghrdGrFZk9G/jpJ+A//+F18pW0+gsq+1iPHwOrVvHkDgAtWuTh7FlK8IQYg9b94CtXroyVK1di5cqVcHZ2Rnh4eLn71KxZE97/63bi7OwMT09PZGRkGB6xiezbB6SmSrtxtTSVKwORkcClS7zKRl9//cXbN/7zH6BVq+fPt2iRh/R0ID3d4FAJkTyt7r+Sk5MRFRWleTxu3DgEBwdrfZLU1FQolUq0bt263O1UKhWUSiXy8vKgFHnV52XL6sHNzQGvvppk8gWozaH85WneHHjjjXqYNcsRPj7X4eZWoNP+jAGjRzdAlSr2GDbsXyiVz0dFv/qqLYBaiI1NRufOOUaO3LyZ+3UXmpTLL1TZtUrwcrkc58+fR9u2bQEAf/75J+Ry7b5G5+TkYPz48Zg5c2aFDbMODg7w8vKCUqmEl5eXVscXws2bwPHjwGefAa1amT4Oscuvjc2bgZYtgXXrmiA6Wrd9f/6ZV++sWwd06NCsxGs5OX9DJgPu368PM38LjM4SrruQpFx+Q8pe3geDVgk+IiIC06dPR3Z2NgCgSpUq+OKLLyrcLz8/H+PHj0dQUBB69uypZbji27CB1wmHhYkdiflq0oT3pvn8c/4+de2q3X7Z2cC0aYBCwUfIvqhy5SI0bco/AAghhtEqwTdv3hy7d+/WJHhnZ2d89913aN68eZn7MMYwa9YseHp6YuTIkcaJ1gSePeO9ZwIDgfr1xY7GvIWH80nIxo7ldep2dhXvs2gRkJYGbN3KlwYsjULBBz8RQgyj04pOzs7OmmqW7777rtxt//zzT8TGxuLMmTMIDg5GcHAwjh49qnegphIbC2RkUOOqNhwdeQ8YpZI3vFYkKQn46is+BfHrr5e9nUIBJCfTFMWEGErvTm6sgglD2rZti7///lvfw4tm7VqgQQOgVy+xI7EMffsCQUFARATw7rt8RGpZJk0C7O15P/ryKBT8d0IC76ZKCNGP3muyWuNUBX//DRw+zAc2lVV9QF62ciUfEDZpUtnbxMcDe/YAc+bwicvK4+PDf9OIVkIMU+4dvI+PT6mJnDEGlUolWFBiWb+eD9wZNUrsSCxLo0Z8ROrs2cBvvwEvtqerVMCECUCzZvx3RapX58ekBE+IYcpN8AkJCaaKQ3RPn/J53/v3B2rVEjsayzNtGrBlC5847PLlknP3REby+vd9+3gVjTYUCkrwhBhK7yoaa7N9O58bhRpX9ePgAERF8bVrly17/nxaGu9K+fbbQO/e2h9PoeAfCpmZxo+VEKmgBP8/a9cCTZtq35+bvKxXLz6t8sKFfLAYwBfyKCgAVqzQ7VjqhtaLF40ZISHSQgkefF6VU6f43bsVth2b1IoVgI0Nr2s/cYJPTDZtGl/MQxfU0EqI4SjBgw+Zd3AALGw2Y7NUrx7vKbN7NzB4MH88Y4bux3F3B+rUoQRPiCEkn+Czs4EffuArFrm6ih2NdZg4kc8df+cOr4+vXFm/4/j60pQFhBhC8gn+55+BrCxqXDUme3tg2zZeXTN4sP7HUSiAa9f4KlKEEN0ZsFyD5WMMWLOGz0fevr3Y0VgXb2/+YwiFgl+jv/4qf2oDQkjpJH0Hf/48Hw5PjavmSd2ThurhCdGPpBP82rW8fjgkROxISGk8PICaNSnBE6IvySb4x495/XtICFClitjRkNLIZDSilRBDSDbB//ADn56AGlfNm0IB/Pe/QF6e2JEQYnkkmeAZ49Uz7do9H1BDzJNCwUfCXrkidiSEWB5JJvj4eODqVbp7twTU0EqI/iSX4B88AD74AGjRgi9QQcxbw4ZAtWqU4AnRh6T6wTMGfPwxXwpu715ALhc7IlIRamglRH+SuoP/6Sc+wnL+fKp7tyQKBZ8QLj9f7EgIsSySSfDJycDYsUDHjnx2Q2I5FAq+KtTVq2JHQohlkUSCLyoCRozg64Zu2ULrrVoaamglRD+SSPArVwJHjvDfus5LTsTXpAng7EwJnhBdWX2Cv3IFCA8HgoOBkSPFjobow8aGt5lQgidEN1ad4FUqYPhwoGpVYP16mlDMkikUfPm+wkKxIyHEclh1gp83j081u3Ejn7SKWC6FAsjNBf75R+xICLEcVpvgT5wAvvwSCAsDgoLEjoYYihpaCdGdxSf46Gg+2tHGhv+OjgaePAHeew9o1AhYvlzsCIkxNG/OB6ZRgidEexY9kjU6Ghgzhn91B4Bbt/hjPz/e7/3YMcDFRdwYiXFUqgS0bk0JnhBdWPQd/KxZz5O7Wm4ucPQoMGMGH9RErId6yoKiIrEjIcQyWHSCT04u+7W5c00XBzENhYJXv12/LnYkhFgGi07w9euX/nzt2oC9vWljIcKjhlZCdGPRCX7hQsDJqeRzdnbA0qXixEOE5e3Nry8leEK0Y9EJPiSED2Dy8OCPHRyAzZtpEW1r5eAAtGxJCZ4QbVl0ggd4Mvf25qNVExP5yFVivdQNrYyJHQkh5s/iEzwAvP46sHUrUK+e2JEQoSkUfFWulBSxIyHE/Fl0P3i1efPEjoCYSvGG1rIa2QkhnFXcwRPpaN2az+dP9fCEVIwSPLEojo6AlxcleEK0IViCDw8PR4cOHdC3b1+hTkEkihbhJkQ7giX4AQMGYOPGjUIdnkiYQgGkp/MfQkjZBEvwfn5+qFq1qlCHJxJGI1oJ0Q7VwROL06YN/00JnpDymVU3SZVKBaVSiby8PCiVSrHDEY2Uy69t2Rs29MTRo88waFCqCaIyDSlfd0Da5Req7GaV4B0cHODl5QWlUgkvLy+xwxGNlMuvbdnbtwdOnXKwqvdJytcdkHb5DSl7eR8MVEVDLJJCwaeLvn9f7EgIMV+CJfjJkydj6NChuHHjBt58801s27ZNqFMRCVI3tCYkiBsHIeZMsCqa5bQYKhGQjw//feEC0KOHuLEQYq6oioZYJFdXvsg69aQhpGyU4InFohGthJSPEjyxWAoFkJQEZGaKHQkh5okSPLFYvr7896lT4sZBiLmiBE8sVvv2gLs7MHQosG+f2NEQYn4owROLVa0acPYs4OkJBAbyxdZpKT9CnqMETyxa/frAiRPAoEHA9OlAaCjw9KnYURFiHijBE4tXuTLw66/AggXAjz8CnTsDaWliR0WI+CjBE6sgkwGzZgG7dgFKJeDnB/zxh2HHzMsDNm3iywR++KFx4iTElCSX4KOj+QAZGxv+Ozpa7IiIMfXrB5w+Dcjl/E5+yxbdj/HgAbBwIf/7CAvj3wY2bAD+/tvY0RIiLEkl+OhoYMwY4NYt3hh36xZ/TEneurRsCZw7B7z+OvD++8CUKUBBQcX7Xb8OfPIJr9f/7DM+HcKhQ8DVq4CDA2/EJcSSSCrBz5oF5OaWfC43lz9PrEuNGsCBA8C4ccDy5UDfvsCjR6Vv+8cfwJAhQJMmwLp1/N+XL/OulwEBQM2awOjR/NuA1Or2nzzh7RqFhWJHQvQhqQSfnKzb88Sy2dkBUVHA+vXA4cOAvz9w7Rp/ragI2L0bePNN3p/+t994L5ybN4Fvv+XfAoqbMoXvs2KFyYshqrlzgffeA778UuxIiD4kleDr19fteWIdPviAJ/jHj3mSnzsX8PICgoP5h3tkJJCSAixeDHh4lH6MRo34gKp168r+JmBtHj3ibQ9yOX/Pzp8XOyKiK0kl+IULASenks85OfHniXV74w2eoDw9gfnzAWdn4Oef+Vw2EyYALi4VH+PTT4HsbGD1auHjNQdr1gA5OcD+/XzE8PDhL1dxEvMmqQQfEsK/rjdowLvVNWjAH4eEiB0ZMYX69fm8NQkJPNkPHQpU0mFFhNde4yNmV660/kSXlwesWgX07s17I33/Pe9FNG2a2JERXUgqwQM8md+8yetTb96k5C41jo5Amzb8A14fn34K3LvH6+mt2Q8/ABkZzxN6t27ApEn820t8vLixEe1JLsETYog33uDdL5cuBfLzxY5GGEVFwLJlfLbOrl2fP79oEW98HjWKf8gR80cJnhAdyGTAjBl8DMXWrWJHI4zdu4F//uF378W/6cjlfMzIo0d8/AhN7Gb+KMEToqPAQMDbG/jiC+tMckuW8F5DAwe+/FqrVvxOPiYG2LzZ5KERHVGCJ0RHNja8Lv7KFeurjz55kk/1MGVK2Q3QkybxqpsJE4B//zVtfEQ3lOAJ0cPQobxXzhdfiB2JcS1ZwkcBjxxZ9jY2NrxXTaVKvOukNtNAEHFQgtcCTVBGXmRnx+9yT5zgd73W4No1Xv8+btzL40VeVK8esHYtcOYMHyBGzBMl+ArQBGWkLKNH87tdaxnGv2wZb0gdO1a77YcOBYYNAyIi+MpaxPxQgq8ATVBGylK5MjB+PBAXx+vjLVl6Ou/7PmoU4Oam/X7ffMOndxg+nI96JeaFEnwFaIIyUp6xY3miX7JE7EgMs2oVr0ufPFm3/apV47NsJiXxKitiXijBV4AmKCPlqVGDV9n99BOvvrNEWVl83pmBA4HGjXXfv0sXYOpUPhHbnj1GD48YgBJ8BfSdoIwaZqVj8mR+nb/6SuxI9LNhA5CZadg8M59/zpc2HD0auHvXeLERw1CCr4A+E5RRw6y01K3L66A3brS8Ifz5+XyO+y5d+Dq2+nJw4H/fmZl8mUNrHABmiSjBa0HXCcqoYVZ6pk0Dnj7lC4xYkl9+AVJTjTNLpLc371EUF8fv6NPTDT8mMQwleAHo2zCrrtbx9m5O1ToWxsuLL/j99dd8znhLwBifNK1lS+Ctt4xzzE8+Afr04QuEeHjwZRBHj+YDo27coDv70jx8CKSn6zBvtQ4owQtAn4bZktU6Mq2qdaie37x8+unzVZAswYEDfO3ZqVP1nz75RTY2fLDU2bO8X32LFsCuXcCIEXyxlfr1+TfgdesApVLaCZ8xPu30q68C06bVEeok5uPq1aslfluqH39kzMmJMX4J+Y+TE3++LA0alNxe/dOggfHOYQks/dp36cJYnTqMqVS672vqsnftqn+suigsZOzSJca+/pqxIUMYq1Xr+d+smxtjAwYwtmIFY998k8xOnGBMqWQsI4Ox/Hxh4xJTYiJjAQH8PXjjDcb27EnS+1jl/d1QghfIjz/y5CyT8d8VJV6ZrPQEL5OVvr2uHwj6xqXvPvqy9Gu/fz+/Dps3676vKct+7hyPc+lSk51So6iIsX/+YWzjRsZCQxlr1Kj0v2WAsSpV+Ou+voz17MnY0KGMjR3L2OzZjK1ezVhKiunjN8SzZ4wtWsSYXM7LtnYt/wA05NpTgrcAuiZsXT8QGNPvrl/fffT9QNDl2pvyg0dbRUWMtWnDWLNm/D+uLkz5dz9kCE8wmZkmO2W50tIY+/XX62z/fsZ++omxqCjGIiIYGz+eseHDGXvrLcb8/Rl79VXGXF2f//3LZPxO+NtvzacsZfnjD8ZateJxDxjAy6xGCd7K6ZpI9bmDN8U++lYdPU/WRVola1N/8Oji5595PD16MLZ8OWMXL2qX7E31d//vv4zZ2DA2fbrxj22qD/fCQsb+/puxefMYa9yYv99yOWPvvMPYnj38TtlcZGUxNmECf088PBjbtevlbSjBS4AuSU6fBKfPXb8pqo7Muc1Cn4SVn8/Yp58y1rTp83PVqMHYwIGMffMNr2MuKnp5P1P93Y8dy5idXck7yNLoWnZTfbi/qKiIsdOneblq1Hhet//JJ/yuubT32lT27mWsfn3+Hv7nP4w9flz6dpTgJUTb8uv6H9AUd/D6fIjoE5e5fvCo91Nflzp1GPvoI8ZGjOD/0dXHqVWLsWHDeD30ihW6Jzh921Lq1uXnr1zZ+DcQpnqPyyu7SsXY7t2MDR7MmIMDP17TpvzD1cPDdNV5d+7w9gKAsRYtGDt5svztKcFLiFDlN0UdvCmStT7nMdUHT3nvV1ERrx7ZsIGxd98t2Zuk+E+lSoy9/TZj8+fzRtCvv2Zs0yZeN71rF69aUScv9Y+jI6+3vneP10Xn5ZWsFrKW66hLOR4/5h+gXl4vH9/WlrF+/XgjZ1wcYxcu8J47L75n2nyIFhQw9ugRYytXMla7Nj++jQ0/R0QEvxZlMfTbC2MiJvijR4+ynj17su7du7N169ZVuD0leE7I8gvdi8Zc7/zMMWEVFT1PCEL9VKrE3xcbG/P8QDTFN7Gy9intx86Ob9+kCU/QL34otGzJG3ubN+ffCJydyz6WXG78/yulESXBFxQUsG7durHk5GSmUqlYUFAQS0xMLHcfSvCcpZff9HW35vPBo2vCKm/7/HzGnjzhd5Y3b/K6+wsXyt4HYGzVKsaWLeNd8ebNYyw8nLEpU8reXuwqLVN8Eytvn5QUxs6cYWzHDn4HPn06YyEhL39DUv/Y2zPWqxevAho9mrFJkxibO5ex6tWN98FTUVfnF4mS4C9cuMBGjRqlebx27Vq2du3acvehBM9JsfzG+Kqq/TmE++DR9T+t9faGEuY9NtX7ZcwPamOdoyzl5QthJkAAkJGRgVq1amkeu7u749KlS+Xuo1KpoFQqkZeXB6VSKVRoZk+K5VcogH37eNnlcjkAPpRdiHMUV945FApg3rwqWLHCDXfu2KFWrXxMmnQPCsWTMvcbO7YK5sypjby857OAyOVFGDs2HUrlE4O3N9U++pRdvZ+Q77Gp3q9atRojPd2+lOefQan81+Dt9d1HZ7p9Vmhv3759bObMmZrHu3btYhEREeXuQ3fwnJTLbw1l1+cu1lS9aMxtYFhxQvUe02cfXb9ZmGoQYWmoisbCSLn8VHbpMrfy6/9Brc8HjzC9aASronnttddw8+ZNpKSkwN3dHXv37sVXlrrkDSFEckJCKl77wZDti++jVF6Dl5eXbjtrQbAEX6lSJcyZMwdhYWEoLCzEwIED0aRJE6FORwgh5AWCJXgA6Ny5Mzp37izkKQghhJSBFvwghBArRQmeEEKsFCV4QgixUjLGGBM7CLWLFy/CwcFB7DAIIcRiqFQqtGnTptTXzCrBE0IIMR6qoiGEECtFCZ4QQqwUJXhCCLFSlOAJIcRKUYInhBArRQmeEEKslKBz0ejj2LFjWLhwIYqKijB48GCMGTNG7JBMJiAgAJUrV4aNjQ1sbW2xc+dOsUMSVHh4OP7v//4PNWrUwJ49ewAAjx8/xqRJk5CWloY6deogMjISVatWFTlS4yut7FFRUdi6dStcXV0BAJMnT7bKuZzS09Mxffp0PHjwADKZDEOGDMH7778vmWtfVvkFuf66zz4sHH3WcbUmXbt2ZQ8ePBA7DJM5e/Ysu3LlCgsMDNQ89+WXX2oWaF+3bh1bsmSJWOEJqrSyr1q1im3cuFHEqEwjIyODXblyhTHGWFZWFuvZsydLTEyUzLUvq/xCXH+zqqK5dOkSGjRogHr16sHe3h6BgYE4dOiQ2GERgfj5+b10h3bo0CH069cPANCvXz8cPHhQhMiEV1rZpaJmzZrw9vYGADg7O8PT0xMZGRmSufZllV8IZpXgS1vHVaiCm6vRo0djwIAB+PXXX8UORRQPHjxAzZo1AQBubm548OCByBGZVnR0NIKCghAeHo7MzEyxwxFcamoqlEolWrduLclrX7z8gPGvv1kleKn7+eefsWvXLmzYsAHR0dE4d+6c2CGJSiaTQSaTiR2Gybz77rv4/fffERsbi5o1a+KLL74QOyRB5eTkYPz48Zg5cyacnZ1LvCaFa/9i+YW4/maV4N3d3XHnzh3N44yMDLi7u4sYkWmpy1qjRg306NEDly5dEjki06tRowbu3r0LALh7966mwUkKXnnlFdja2sLGxgaDBw/G5cuXxQ5JMPn5+Rg/fjyCgoLQs2dPANK69qWVX4jrb1YJvvg6rs+ePcPevXsREBAgdlgmkZubi+zsbM2/T548KcklDgMCAhATEwMAiImJQbdu3cQNyITUyQ0ADh48aLXXnzGGWbNmwdPTEyNHjtQ8L5VrX1b5hbj+Zjeb5NGjR7Fo0SLNOq4ff/yx2CGZREpKCsaOHQsAKCwsRN++fa2+7JMnT8bZs2fx6NEj1KhRA5988gm6d++OiRMnIj09HR4eHoiMjES1atXEDtXoSiv72bNnce3aNQBAnTp1MH/+fE2dtDU5f/48QkJC0LRpU9jY8HvMyZMno1WrVpK49mWVf8+ePUa//maX4AkhhBiHWVXREEIIMR5K8IQQYqUowRNCiJWiBE8IIVaKEjwhhFgps5tNkhBj8/LyQtOmTTWPAwMDjTZLaWpqKj766CPNjJCEmBNK8MTqyeVyxMbGih0GISZHCZ5IVkBAAHr37o3jx4/DwcEBX331FRo0aIDU1FTMnDkTjx49gqurKxYvXgwPDw/cv38fc+fORUpKCgBg3rx5qFmzJgoLC/HZZ58hISEB7u7uWL16NeRyObZs2YJffvkFtra2ePXVV7FixQqRS0ykhurgidXLy8tDcHCw5ic+Pl7zmouLC+Li4jB8+HAsWrQIALBgwQL0798fcXFxCAoKwoIFCzTP+/n5Yffu3di1a5dmKPmtW7cQEhKCvXv3wsXFBQcOHAAArF+/HjExMYiLi0NERISJS00IJXgiAeoqGvVPnz59NK/17dsXAK+Xv3jxIgAgISFB83xwcDD+/PNPAMCZM2cwbNgwAICtrS1cXFwAAHXr1oWXlxcAwNvbG2lpaQCAZs2aYerUqYiNjYWtra3wBSXkBZTgCTGQvb295t+2trYoLCwEwO/ghw0bhqtXr2LQoEEoKCgQK0QiUZTgiaTt27cPABAfHw8fHx8AgI+PD/bu3QsAiIuLQ9u2bQEAHTp0wE8//QSATwiXlZVV5nGLioqQnp6O9u3bY+rUqcjKykJubq6QRSHkJdTISqyeug5erVOnTpg6dSoAIDMzE0FBQbC3t8fy5csBALNnz0Z4eDg2bdqkaWQFgFmzZmH27NnYsWMHbGxsMG/ePLi5uZV6zsLCQkybNg3Z2dlgjCE0NBRVqlQRuKSElESzSRLJCggIwPbt2616YQkibVRFQwghVoru4AkhxErRHTwhhFgpSvCEEGKlKMETQoiVogRPCCFWihI8IYRYqf8Hv5lJnnqq+aIAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "val_loss = history['val_loss']\n",
    "loss = history['loss']\n",
    "epochs = range(1, 26)\n",
    "\n",
    "plt.plot(epochs, val_loss, 'b-', label='Validation_loss')\n",
    "plt.plot(epochs, loss, 'bo', label='Traninig_loss')\n",
    "plt.title(\"Training and Validation Loss\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"Loss\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "incomplete-grocery",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAABDrUlEQVR4nO3de1wU1fvA8c8CApLi3UUNUdPM1AzTvH0VRRQEUfBSqVnaV81Ks9QstbxfKy27WWSXn2n6NVMUVtJEAzM1TY1SLDVRRMEUb6AssMzvj4kNlMsCuyzLPu/Xixc7szOzz5mFfXbOOXOORlEUBSGEEHbLwdoBCCGEsC5JBEIIYeckEQghhJ2TRCCEEHZOEoEQQtg5SQRCCGHnJBGIfMaMGcPmzZvNvq01+fr68tNPP5n9uCNHjuSbb74BYOvWrTzzzDMmbVtSFy5cwNvbG4PBUKr9hSiOk7UDEGXn7e1tfHz79m2cnZ1xdHQEYO7cuQwYMMDkY61atcoi21ZEYWFhxMTEsHbt2nzrU1NT6dGjB5s2beL+++836VgDBgwo0Xkuiq+vLwsWLKBr164ANGzYkCNHjpjl2AVRFAU/Pz9cXFzYtm2bxV5HVFySCCqBvB8Sd36I5JWdnY2Tk7zluQYMGMC7775LYmIinp6exvXbtm3j/vvvNzkJ2LqDBw+SmppKdnY2cXFxPPTQQ+X22vI3WTFI1VAlduDAAXr06EFYWBjdunVj+vTpXL9+nWeffZbOnTvTsWNHnn32WZKTk4375K3C2LRpE8OGDWPp0qV07NgRX19fYmJiSrVtYmIiI0aMwNvbm1GjRjF37lymTp1aYNymxPjuu+/yxBNP4O3tzTPPPENqaqrx+fDwcHr16kWnTp1YuXJloefHw8ODzp07s2XLlnzrw8PDGThwYLFx5JVb/lx79+4lICCARx55hHnz5pH3Bv5z587x1FNP0alTJzp16sSUKVO4ceMGAK+88goXLlxg/PjxeHt78+mnn3L+/HlatmxJdnY2ACkpKYwfP55HH32UPn36sGHDBuOx33//fSZNmsS0adPw9vYmKCiI3377rdBzALB582Z8fX3x8fEhPDw833MnT55k9OjRPProo3Tt2pWPP/4YAIPBwMcff4yfnx/e3t4MGjSIixcv3hUr3P138sQTT7Bo0SI6derE+++/X+T5ALh48SITJkygc+fOdOrUiXnz5pGZmcmjjz7KH3/8YdzuypUrtGvXLt/fgjCNJIJK7vLly1y/fp3du3czf/58cnJyGDRoELt372b37t24uLgwb968QvePi4ujadOm7N+/nzFjxjBz5kwKG5WkqG2nTp3KQw89xIEDB5gwYcJdH755mRJjZGQkixcvZt++fWRlZfH5558DcOrUKebOncubb77Jnj17uHbtWqEf3gAhISFs3brVuPzXX39x4sQJgoODS3yucqWmpjJhwgReeukl9u/fT+PGjTl8+LDxeUVRePbZZ9mzZw9RUVEkJyfz/vvvA/DWW2/RsGFDPv74Y44cOcLYsWPvOv7kyZPx8PBgz549vPfeeyxfvpx9+/YZn9+1axdBQUEcOnQIX19f5s+fX2ist2/fZvv27QwYMIDg4GB0Oh2ZmZkApKWlMXr0aLp3786ePXvYsWMHXbp0AeCLL75Ap9MRFhbG4cOHWbRoEa6ursWeG1D/Tjw9Pdm7dy/PPfdckefDYDDw7LPP0rBhQ3bt2kVsbCyBgYE4OzsTGBiY772LjIykS5cu1K5d26Q4xL8kEVRyDg4OvPjiizg7O+Pq6kqtWrXw9/enatWqVKtWjeeee46DBw8Wun/Dhg157LHHcHR0JDQ0lL///pvLly+XaNsLFy7w22+/GePo0KEDvr6+hb6mKTEOGjSIpk2b4urqSkBAAPHx8QB899139OzZk44dO+Ls7MykSZNwcCj8z7xPnz5cvnzZ+EG9ZcsWunfvTu3atUt8rnLFxsbSokULAgICqFKlCk8//TR169Y1Pu/l5UW3bt1wdnamdu3ajB492qTjgvrt+PDhw0ydOhUXFxdatWrF0KFD8yXWRx55BB8fHxwdHRk4cCAnTpwo9Hg7duzA2dmZbt260bNnT7Kzs41Xcj/88AN169blmWeewcXFhWrVqtGuXTsAvvnmGyZNmkSzZs3QaDQ88MAD1KpVy6Qy1K9fn5EjR+Lk5ISrq2uR5yMuLo5Lly4xbdo03NzccHFxoUOHDgCEhoai0+mMXza2bNlitnYaeyOVc5VcrVq1cHFxMS7fvn2bxYsXs2fPHq5fvw5Aeno6BoPB2MCcV94PsKpVqwJw69atAl+rsG2vXr1KjRo1jOsAGjRowMWLFws8jikx1qtXL99r5cZ06dIlPDw8jM+5ublRs2bNAl8nd9+AgADCw8Px9vYmIiKCV1991eQ4CnJnDBqNhgYNGhiXL1++zMKFCzl06BDp6ekoioK7u3uhx7vz2DVq1KBatWrGdQ0bNuT33383Lud9H1xdXdHr9YXWxYeHh9OvXz+cnJxwcnKib9++bN68mT59+nDx4kUaN25cYBzJycmFPlecvOcGij4fFy9epGHDhgXG3q5dO1xdXTlw4AD16tXj3Llz9O7du1Qx2Tu5IqjkNBpNvuXPP/+cM2fOsGHDBg4fPmzsMWPJQWjr1avH9evXuX37tnFdYUmgrDHWr18/X1XQ7du3uXbtWpH7hIaG8t1337F3717S09Pp1atXmeKoV69evhgURclX3uXLl6PRaIiIiODw4cO89dZbJp//+vXrc/36ddLS0ozrLl68iFarNWn/vJKTk9m/fz9bt26lW7dudOvWje3btxMbG0tqaioNGjQgMTGxwH09PDw4d+7cXevd3NwAyMjIMK77+++/821z599kUecj9wtD3jaHvEJDQ9m6dStbt27F398/35ceYTpJBHYmPT0dFxcX3N3duXbtGh988IHFX7NRo0a0adOG999/n8zMTI4cOcLu3bstEqO/vz8//PADhw4dIjMzk/fee4+cnJwi9+nQoQPVq1dn1qxZxvrnssTh4+PDyZMn2bFjB9nZ2axevTpfdVp6ejpubm5Ur16dlJSUu7rh1q1bt9AP4AYNGuDt7c3y5cvR6/WcOHGCjRs3lqpKZMuWLTRp0oTvvvuO8PBwwsPD2b59O1qtFp1OR8+ePfn777/58ssvyczMJC0tjV9//RWAoUOHsmLFChISElAUhRMnTnD16lVq166NVqtly5YtGAwGNm7cWGhZTDkfDz30EPXq1WPZsmXcunULvV7PL7/8Ynx+wIAB7Ny5k61btxISElLicyBUkgjszNNPP41er6dz5848/vjjdO/evVxe9+233+bo0aN06tSJd999N98HrjljbNGiBbNmzWLq1Kl0794dd3f3u6oi7qTRaAgJCSEpKSnfh0lp46hduzYrVqxg2bJldOrUibNnz9K+fXvj8xMmTOD48eN06NCBcePG0bdv33z7jxs3jpUrV9KhQwc+++yzu46/fPlykpKS6N69OxMmTGDixIkFdhcuzubNmxk+fDj16tXL9/PEE0+wefNmqlWrxueff87u3bvp1q0b/v7+HDhwAIDRo0fTr18/nnnmGdq3b8/MmTPR6/UAzJ8/n88++4xOnTpx6tSpfPe5FKSo8+Ho6MjHH3/M2bNn6dWrFz169CAqKsr4fIMGDXjwwQfRaDTGtgNRchqZmEZYw0svvUSzZs148cUXrR2KsHHTp0+nfv36vPzyy9YOxWbJFYEoF3FxcZw7d46cnBxiY2OJjo7Gz8/P2mEJG3f+/Hm+//57hgwZYu1QbJr0GhLl4vLly0ycOJFr167h4eHBnDlzePDBB60dlrBh7777Lv/3f//HuHHj8t0ZLkpOqoaEEMLOSdWQEELYOZurGjp69Kixr7Ber7fbfsNSdvssO9h3+e257FC28uv1eh5++OECn7O5RJB7Wz1AfHy88bG9kbLbZ9nBvstvz2WHspU/dxiWgkjVkBBC2DlJBEIIYeckEQghhJ2zWCKYPn06Xbp0oX///gU+rygKCxYsoE+fPgQHB3Ps2DFLhSKEEKIIFksEgwYNKnJO29jYWBISEtixYwfz589nzpw5lgpFCCFEESyWCDp27EiNGjUKfT46OpqQkBA0Gg0PP/wwN27c4NKlS5YKRwghbNbatdCkCbRu/QBNmqjL5mS17qMpKSn5RoX08PAgJSWF+vXrF7mfXq83doPKyMgosktUZSZlt8+yQ8nKHxnpzjvv1CM5uQoeHlm8/PLf9O9/w2zblzdLv/flcb5K8xqzZjUgI8MB0HD2LIwZk8OFCxfN994oFpSYmKgEBQUV+Ny4ceOUgwcPGpefeuopJS4urthjHj9+vMDH9kbKbtvWrFEULy9F0WjU32vWlGSfHJP2WbNGUdzcFAX+/XFzK3y/km5vDSV570t6jsvjfJVmHy+v/Nvn/nh5FV2eOxV17qyWCN544w0lIiLCuNy3b18lJSWl2GNKIlBJ2W1XRf0AMdcHjiWZ+t5b8nxdvaoomzcrSrVqBW/v7KwoTz2lKIsWKcqmTYoSH68omZkle428NJqC99FoTDoVRhUyEezevVv573//q+Tk5ChHjhxRBg8ebNIxJRGopOwVR0m/eZbmw6A8PkBK+4FTmqubkirp1ZC5z9fOnYoyfbqidOyoKA4OBW+X9+fee/MvOzkpSsuWhW9f1DkujysCi7URTJ48mZ9//pmrV6/So0cPJk6caJx3dNiwYfj4+BATE0OfPn2oWrUqixYtslQoQljM2rUwbhzcuqUunz2rLgOMGFHwPgVM9Vvk+tLsc+UK1KsHBfW/cHGBnj1Br4fMTPW3Xg8ODmAw3L19UXPUl6b8JZX/NTQWO8eNG6vx30lRwM8PnJygUyd44w3o3RuefLLg43l5QUIC3LwJf/wB8fFw4oT6c/o0FDT9clHneOHC/OcYwM1NXW82Jcsp1idXBCopu+WU5BtueX27L2yfxo3Vqof169VvrIGBitKoUdHfPO+7T1F69FCUPn0UpX9/RRk0SFGGDVOU7t3Vb67lUX9dEc/xmjVqtc6d5ysgQFEiIxXlxo27tzdHFZ+zs2ntFyW5IiqI1aqGLEESgcoey16afwZLNxiWpjrFXB8gDg6KUqVK/uqHtm0V5cknFeWttxRlxw5F+eijkpc/t1rD3b347cuj/OXxGn//rShDhvz7wazRqOfB9A/o0jT6K4qrq/qaU6YoSnZ28fuW5f9eEkElZG9lr2wNrCX9ADl2TFE6dMifBFq1UpSXXlKUL75QlMOHFSUjo+hjlISvr3r84pTHt3VLn+OtWxVFq1WT6uLFpn0gm0tWlqJMmKCWJzj47quOO0ki+IckAlVlKHtFrB4o6bdPS3e5PHNG7YHi4KAo1asrypw5ihId/aeSk2Oe4xfmvffUspw4UfR2pSl/RTnH168ryujR6vEeekhRfv21bMcriw8/VBRHRzWOs2cL304SwT8kEagqWtkrYhVMafYpbd2yuXvNXLyoflOsUkVRXFzUqoO//1afK4/3/tw5tdxLlhS/bXn0mjJHHXleu3apx3NwUJQZMxRFry/b8cxh+3a1Ok6rVZT9+wveRhLBPyQRqCxZdkt/qCtK+VQPlPYDx5o3VaWmqo2+bm7qN8Rx4xQlMTH/NuX1d//II4rSubP5j1uWc1zWst+6pSiTJqmv2aKFouzbV6bDmd2xY4rSrJma/Netu/t5SQT/kESgslTZy+vOx/KoHijtB0559Iu/U1qaWj9ds6Ya57BhivLnnwVvW15/9wsWqLEkJZn/2KU9x2Up+4ED//blnzhRUdLTS30oi/r7b7UHFyjK7NlKvmpASQT/kESgslTZy+vOx/KqHrDGh3pJ6PWK8sEHiuLhoZY/KEhRjh4tep/y+rv//Xc1ppUry+XlTFKasuv1ivL66+oVlqenenNYRZeRoShPP62e/yeeUK9kFEUSgZEkApWlyl6eH+rWqh6oKH76SVGaNlXL3r27ouzZY9p+5VX+nBxFad5cUfr2LZeXM0lJy/7rr4ri7a2e41GjFOXaNQsFZgE5OYqydKn6v9epk9puZKlEIDOUiXwKu8OxuDsf3dzyryvuzscRIyAsTL0LU6NRf4eFme9uVFswezZkZEBUFMTEwH/+Y+2I8tNoIDQUdu2Ca9esHU3JJCTAqFHg7Q1JSRAeDl98AUWMjF/haDQwbRp8+y389hs8+iikpFhmMAhJBDamNOOS5+7j4ECx+5Tnh/qIEeo/bE6O+tuekkBamvrhP3w4BASo560iCglRh0SIirJ2JKZJSYEXX4T774f//Q8mT4bjx2HgQGtHVnqhofDjj+r/bmqqo2VepNTXGVZiz1VD5dVgWtHr1SvD+751q/pelKa+ujzLbzCo3RmHDi23lyxSYWW/elXtBpq3t9X58+UbW3mQqiHBzJn5B54CdXnmTPPuY8/f1MtLVBTcc0/Fqw66k4OD+m06Kkqtxqpo0tNhyRJo2hQWLVJjPXECPvkEGjWydnS2QxKBDSmPUSuF5SmK+sHau7c6EmhFFxqqVmVFR1s7kn9lZsKHH0Lz5jB9OnTrBkePwtdfq+tEyUgisCGlacgtzT7Csv74Q73SCgy0diSm6dULqldXG1ytzWCA1auhZUuYMEFtC/jxR4iMhHbtrB2d7bLanMWi5EozLnm5jGUuSiS34bVfP+vGYSoXFwgKgi1b4OOPwdGM7ZWKov5t3rihjt9f3O9t25px+rTaG2jlSvD3r7gN7bZEEoENya2rnzkTzp1TaNxYw8KFRdfh599HvRIobh9hWdu2wYMP2tZVWUgIrF8P+/aZp13j9Gn1Q/zMGbUtqjgODuDuDh4eChs2wODB6jphHpIIbMyIEepPfPwJWrVqVaJ9hPWlpUFsLEycaO1ISqZfP3B2hs2bzZMIXnsNkpPV+n13d/WnenX1J/dx3t9Vq6rf/OPjz5j8dy9MJ4nAitaulW/q9mb3brWh01aqhXK5u6uN2+Hh8PbbZauOOXAANm5Ub6ibM8dcEYqykIsrK8mdh/XsWbWeNHceVlNuEBO2KyoKqlWr+N1GCxISAn/9pd7lWlqKot4tW78+TJlittBEGUkisJLS9O8Xts3Wuo3eacAA9UqgLL2HdDq1amz2bLXaR1QMFk0EsbGx+Pv706dPH8LCwu56Pikpiaeffprg4GBGjhxJcnKyJcOpUKR/v/05cULtNmpr1UK5PDygS5fSJwKDQW0baNECxo41a2iijCyWCAwGA/PmzWPVqlXodDoiIyM5depUvm2WLl1KSEgIERERPP/88yxbtsxS4VQ40r/f/that9GChIbCkSNqQiup//s/OHZMvQO4ShWzhybKwGKJIC4uDi8vLzw9PXF2diYoKIjoO25NPH36NJ07dwagc+fOdz1fmZVmcDdh26KibK/b6J1CQtTfW7aUbL9bt2DWLOjUSe36KSoWi/UaSklJwcPDw7is1WqJi4vLt80DDzzAjh07ePrpp/n+++9JT0/n6tWr1KpVq9Dj6vV64uPjAcjIyDA+tjXt28OcOe688049kpOr4OGRxcsv/0379jcwpUi2XPayssWyp6driIm5nxEjrhIff6lMx7J2+Vu0aMratQb69jW9HvPTT+uQlFSfxYsTOHHidqlf29pltzZLld+q3UenTZvG/Pnz2bx5Mx06dECr1eJYzG2LLi4uxn7E8fHxNt2nuFUreOWV3CVnoNE/P8Wz9bKXhS2WPSICsrJg5Mg6tGpVp0zHsnb5H39crd6pV68VdesWv/2VK/D559C/P4wc2aRMr23tsltbWcpfVAKxWNWQVqvN1/ibkpKCVqu9a5sPPviA8PBwXn75ZQDc3d0tFZIQVrNtm+12G71TSIh6N3BEhGnbL1yoDg+xZIlFwxJlYLFE0LZtWxISEkhMTCQzMxOdToevr2++bVJTU8n55/7ysLAwBkvloaiE8nYbdXa2djRl1749eHqa1nsoIUEdJXTUKGjd2sKBiVKzWCJwcnJi1qxZjBkzhsDAQPr160eLFi1YsWKFsVH4559/JiAgAH9/fy5fvsxzzz1nqXCEsJoTJ9QbBm25t1BeGo16VbBjhzofQFFef10dE2ju3HIJTZSSRdsIfHx88PHxybdu0qRJxscBAQEEBARYMgQhrK4ydBu9U2govP8+bN8OgwYVvM2RI+qd8q+9BvfeW77xiZKRO4uFsLCoKLVaxJa7jd6pe3eoXVsdhK4wr76qbvPqq+UXlygdSQRmVJJJ4oV9yB1ttDJdDQA4OUFwsDohTFbW3c9//7368/rrULNmuYcnSkgSgZnIIHKiILt22eZoo6YICYFr19REl1dOjnoV0KQJPP+8FQITJSaJwExkEDlREFsebbQ4ffuq8wTcWT20bp3aPrBggW0OrmePJBGYiQwiJ+5U2bqN3snNTZ1lLDxcLSuAXq9WB3l7w7BhVg1PlIAkAjORQeTEnXK7jdrKJPWlERICSUlw6JC6/NFH6r0Db74pU0naEnmrzEQGkRN3qozdRu/Uv786mX14uNpesGCBWmXk52ftyERJyFSVZiKTxIs7bdumdhv19LR2JJZTpw706KG2E+TkwNWrsHSptaMSJSWJwIxkkniRKy0N9uyBF1+0diSWFxqqlvPUKfXv/+GHrR2RKCmpGhLCAipzt9E7DRyo/tZoYP5868YiSkeuCISwgMrcbfROjRvDk0+q1WBNmlg7GlEakgiEMLPcbqN+fpWz22hBvvrK2hGIspCqISHMrLKNNioqP0kEQpjZtm3qb0kEwlZIIhDCzHJHG63M3UZF5SKJQAgzyu02KlcDwpZIIhDiDpmZ8NZb8O236k1SJZHbbbQyDyshKh/pNSREHomJ8NhjsH+/uty2Lcyerd40ZcrYObndRrt1s2ycQpiTXBEI8Y/vvlNHzTx2DNavhzVr1NE0hwxR75Yt7gpBUdSGYnvqNioqB4smgtjYWPz9/enTpw9hYWF3PX/hwgVGjhxJSEgIwcHBxMTEWDIcIQpkMMCsWWp1TqNG6kiajz+uDpdw/LjpCSE+Xh1nStoHhK2xWCIwGAzMmzePVatWodPpiIyM5NSpU/m2WblyJf369SM8PJx33nmHuXPnWiocIQqUkqKOqT9/PowaBfv2wf33//u8o6PpCcEeRhsVlZPFEkFcXBxeXl54enri7OxMUFAQ0dHR+bbRaDSkpaUBcPPmTerXr2+pcIS4y549alXQ3r3w+efqz51DiefKmxDWrlUbhO9MCFFR0KaNdBsVtsdiiSAlJQUPDw/jslarJSUlJd82EyZMICIigh49ejBu3Dhef/11S4UjhJGiqBOn9OqlNuweOACjR5u2r6MjDB+utiPkTQjt2km3UWG7rNprSKfTERoayjPPPMORI0eYNm0akZGROBTRPUOv1xMfHw9ARkaG8bG9kbKXruzXrzswY0ZDdu+uTt++N1iw4CJVquRQmsN5e8PGjRAV5c7KlXXJzHShXbsE4uNvlyo2U8l7b59lB8uV32KJQKvVkpycbFxOSUlBq9Xm22bjxo2sWrUKAG9vb/R6PVevXqVOnTqFHtfFxYVWrVoBEB8fb3xsb6TsJS/7L7+o8+gmJcGKFTBxojsajXuZ42nTBiZPVsfjb9mySZmPVxx57+2z7FC28heVQCxWNdS2bVsSEhJITEwkMzMTnU6Hr69vvm0aNGjAvn37ADh9+jR6vZ7atWtbKiRhpxQFVq6Erl3VHkK5E8ZoNOZ7DUdHaNnSfMcTojxZ7IrAycmJWbNmMWbMGAwGA4MHD6ZFixasWLGCNm3a0Lt3b1577TVef/11vvzySzQaDUuWLEFjzv9OIYDFi9UpRPv1U4dLLuKCUwi7ZNE2Ah8fH3x8fPKtmzRpkvFx8+bNWb9+vSVDEIJ169R5dSMjTbs7WAh7I/8WolK7fBl+/129V0CSgBAFk38NUan9+KP6u0cP68YhREUmiUBUajEx4OoKHTtaOxIhKi5JBIVYu1adiNvBQf29dq21IxKlERsLnTuDi4u1IxGi4pJEUIC1a2HcOHXeWUVRf48bJ8nA1ly/DkePwh39FYQQd5BEUICZM+HWrfzrbt1S1wvbsXevOgaQtA8IUTRJBAU4d65k60XFFBsLVaqoVUNCiMJJIihA48YlWy8qppgYtZG4sBFFhRAqSQQFWLjw7g8PNzd1vbAN6enqBDNSLSRE8SQRFGDECAgLAy8vdTwaLy91ecQIa0cmTLV/P2RnS0OxEKaQyesLMWKEfPDbspgYtetv167WjkSIik+uCESlFBurzhfgXvZRpoWo9CQRiEonI0OtGpJqISFMI4lAVDoHD6qTzEtDsRCmkUQgKp3YWPX3f/5j3TiEsBWSCESlExMDbdvKBDRCmEoSgahUsrLgp5+kWkiIkpBEICqVI0fUm8mkoVgI05mUCCZMmMAPP/xATk6OpeMRokxiYtTf3btbNw4hbIlJiWD48OFERETQt29f3n77bf766y9LxyVEqcTGwv33g4eHtSMRwnaYdGdx165d6dq1Kzdv3iQyMpLRo0fToEEDhg4dyoABA6hSpUqB+8XGxrJw4UJycnIYOnQo48aNy/f8okWLOHDgAAAZGRlcuXKFQ4cOlbFIwl4ZDLBnDzz2mLUjEcK2mDzExNWrV9m6dStbtmyhVatWDBgwgF9++YXw8HC++uqru7Y3GAzMmzePL774Aq1Wy5AhQ/D19aV58+bGbWbMmGF8/NVXX3H8+PEyFkfYs99+UyejkYZiIUrGpETwwgsvcObMGQYOHMjHH39M/fr1AQgMDGTQoEEF7hMXF4eXlxeenp4ABAUFER0dnS8R5KXT6Zg4cWJpyiAE8O/9A9JQLETJmJQIRo4cSedCZvfYtGlTgetTUlLwyFNRq9VqiYuLK3DbpKQkzp8/X+hr5KXX64mPjwfU6qTcx/ZGyn532SMjG9GokStpaaepzKdG3nv7LDtYrvwmJYLTp0/z4IMP4v7PCF7Xr18nMjKSEWYanlOn0+Hv74+jo2Ox27q4uNCqVSsA4uPjjY/tjZQ9f9kVRe06GhhIpT8v8t7bZ9mhbOUvKoGY1Gtow4YNxiQAUKNGDb755psi99FqtSQnJxuXU1JS0Gq1BW67bds2goKCTAlFiALFx8Ply1ItJERpmJQIcnJyUBTFuGwwGMjKyipyn7Zt25KQkEBiYiKZmZnodDp8fX3v2u706dPcuHEDb2/vEoYuxL9y2wekoViIkjOpaug///kPL730Ek888QQA69evp3sxd+w4OTkxa9YsxowZg8FgYPDgwbRo0YIVK1bQpk0bevfuDahXA4GBgWg0mjIWRdiz2Fho2BDuu8/akQhhe0xKBK+88grr169n3bp1gHpfwdChQ4vdz8fHB587rtUnTZqUb1l6ComyUhT1juIePdSpRYUQJWNSInBwcGD48OEMHz7c0vEIUWJ//QUXLki1kBClZVIiSEhIYPny5Zw6dQq9Xm9cHx0dbbHAhDBV7vhC0lAsROmY1Fg8ffp0hg0bhqOjI6tXryYkJIQBAwZYOjYhTBIbC3Xrgh33KhSiTExKBHq9ni5dugDQqFEjJk6cSEzu1zAhrCw2Vh1tVNoHhCgdk6qGnJ2dycnJwcvLizVr1qDVaklPT7d0bEIUKzERzpyBO/ogCCFKwKQrghkzZnD79m1ef/11jh07xtatW1m6dKmlYxOiWHL/gBBlV+wVgcFgICoqildffZV77rmHxYsXl0dcQpgkJgZq1ICHHrJ2JELYrmKvCBwdHfnll1/KIxYhSiw2Fv7zHzBhmCohRCFMaiNo1aoV48ePJyAgADc3N+P6vn37WiwwIYqTkgJ//AHPPGPtSISwbSYlgszMTGrVqmWcTSyXJAJhTTL/gBDmYVIikHYBURHFxsI990D79taORAjbZlIimD59eoHrJUEIa4qJga5doZAps4UQJjIpEfTs2dP4WK/Xs3PnTuN0lUJYQ2qqOkexTFQvRNmZlAj8/f3zLffv318GoBNW9eOP6m+5f0CIsjPphrI7JSQkcOXKFXPHIoTJYmLAxQUefdTakQhh+0y6IvD29s43cUy9evWYOnWqxYISojixsdCpE7i6WjsSIWyfSYngyJEjlo5DCJOlpTlw+DDMmGHtSISoHEyqGvr++++5efOmcfnGjRvs3LnTYkEJUZQjR6qSkyP3DwhhLiYlgg8++IDq1asbl93d3fnggw8sFpQQRTl0yA0nJ/hnZHQhRBmZlAhycnLuWmcwGMwejBCmOHTIjQ4d1JvJhBBlZ1IiaNOmDYsXL+bcuXOcO3eOxYsX07p162L3i42Nxd/fnz59+hAWFlbgNtu2bSMwMJCgoCCmTJlSsuiF3bl1C377rap0GxXCjExqLH7jjTf46KOPeOmll9BoNHTr1o1Zs2YVuY/BYGDevHl88cUXaLVahgwZgq+vL82bNzduk5CQQFhYGOvWraNGjRrSJdUOffQRxMeDXg+ZmervOx/nXU5Ph+xsjSQCIczIpETg5uZW4u6icXFxeHl54enpCUBQUBDR0dH5EsGGDRsYMWIENWrUAKBOnToleg1h286cgRdegGrV1B8XF3B2Vn/nfVyrVv71jo5X8POTvxUhzMWkRDB69GhWrFiBu7s7ANevX2fy5Ml89tlnhe6TkpKCh4eHcVmr1RIXF5dvm4SEBACeeOIJcnJymDBhAj2K+aqn1+uJj48HICMjw/jY3lSGsq9dWwvwYMOGUzRpkmXyfhkZGfz11yXLBVbBVYb3vrTsuexgufKblAiuXr1qTAKA2apxDAYDZ8+e5auvviI5OZknn3ySiIiIfK91JxcXF1q1agVAfHy88bG9qQxl/+UXaNEC+vVrXvzGeVSGspeFPZffnssOZSt/UQnEpMZiBwcHLly4YFw+f/58vjuNC6LVaklOTjYup6SkoNVq79rG19eXKlWq4OnpSZMmTYxXCaJyS0+H3buhf39rRyKEMOmK4KWXXmL48OF07NgRRVH45ZdfmDdvXpH7tG3bloSEBBITE9Fqteh0OpYtW5ZvGz8/P3Q6HYMHDyY1NZWEhARjm4Ko3KKj1cbfoCBrRyKEMCkR9OjRg2+//Zb//e9/PPjgg/j5+eFazCAvTk5OzJo1izFjxmAwGBg8eDAtWrRgxYoVtGnTht69e9O9e3f27t1LYGAgjo6OTJs2jVq1apmlYKJii4yE6tWhe3drRyKEMCkRfPPNN6xevZrk5GQeeOABfv31Vx5++GFWr15d5H4+Pj743DEOwKRJk4yPNRoN06dPL3TiG1E5KQrodNC3r9obSAhhXSa1EaxevZqNGzfSsGFDvvrqKzZv3lxkg64QRTl6FC5ckPYBISoKkxKBs7MzLi4ugDqR/X333ceZM2csGpiovHQ69Xe/ftaNQwihMqlqyMPDgxs3buDn58fo0aNxd3enYcOGlo5NVFI6nTqhzB2dyIQQVmJSIvjwww8BmDhxIp06deLmzZt0l1Y+UQp//w0HDsCcOdaORAiRy6REkNejMjegKIOoKLWxWLqNClFxlGrOYiFKKzISGjQAb29rRyKEyCWJQJSbrCzYvh0CA8FB/vKEqDDk31GUmx9/hBs3pNuoEBWNJAJRbnQ69QYyPz9rRyKEyEsSgSg3kZHQs6c694AQouKQRCDKxalT8Mcf0ltIiIpIEoEoF7l3E0siEKLikUQgyoVOBw88APfdZ+1IhBB3sotEsHYtNGmidlls0kRdFuXn5k2IiZHeQkJUVCW+s9jWrF0L48bBrVvq8tmz6jLAiBHWi8ue7NwJmZlSLSRERVXprwhmzvw3CeS6dUtdL8qHTgc1akC3btaORAhRkEqfCM6dK9l6YV45OWoi8PeHKlWsHY0QoiCVPhE0blyy9cK8jhyB5GSpFhKiIqv0iWDhQnBzy7/OzU1dLywvMhI0GpmERoiKzKKJIDY2Fn9/f/r06UNYWNhdz2/atInOnTszcOBABg4cyDfffGP2GEaMgLAw8PJSP5C8vNRlaSguHzoddOoE9epZOxIhRGEs1mvIYDAwb948vvjiC7RaLUOGDMHX15fmzZvn2y4wMJBZs2ZZKgxA/dCXD/7yl5wMBw/CggXWjkQIURSLXRHExcXh5eWFp6cnzs7OBAUFER0dbamXExVQVJT6W9oHhKjYLHZFkJKSgoeHh3FZq9USFxd313Y7duzg4MGDNG3alOnTp9OgQYMij6vX64mPjwcgIyPD+Nje2ELZ169vhFZbFWfnU5gzVFsouyXZc/ntuexgufJb9YayXr160b9/f5ydnVm/fj2vvvoqq1evLnIfFxcXWrVqBUB8fLzxsb2p6GXPzIR9+2DYMHjwQfPGWdHLbmn2XH57LjuUrfxFJRCLVQ1ptVqSk5ONyykpKWi12nzb1KpVC2dnZwCGDh3KsWPHLBWOKGd79qhDS8iwEkJUfBZLBG3btiUhIYHExEQyMzPR6XT4+vrm2+bSpUvGx7t27eI+GZGs0oiMBBcXuOMtF0JUQBarGnJycmLWrFmMGTMGg8HA4MGDadGiBStWrKBNmzb07t2br776il27duHo6EiNGjVYvHixpcIR5Uyng1694J57rB2JEKI4Fm0j8PHxwcfHJ9+6SZMmGR9PmTKFKVOmWDIEYQV//gknT0Ket1oIUYFV+juLRfmTSWiEsC2SCITZRUZC69bq3A9CiIpPEoEwqxs3IDZWrgaEsCWSCIRZ7dgB2dnSbVQIWyKJQJiVTge1akGXLtaORAhhKkkEwmxycmDbNggIAKdKPwmqEJWHJAJhNocOwaVL0j4ghK2RRCDMJjwcHBzUKwIhhO2QC3hRZooC774LS5dCYCDUqWPtiIQQJSGJQJRJVha88AJ8+ikMGgTFDB4rhKiApGpIlFpqqloN9OmnMGMGfPONjC0khC2SKwJRKn/+qd4rcPasehUwcqS1IxJClJYkAlFiu3fD4MHg6AjR0fCf/1g7IiFEWUjVkCiRVaugb19o0AAOHJAkIERlIIlAmMRggKlTYexY6N0bfvoJmjWzdlRCCHOQqiFRrJs3YfhwdVTRiRNh+XK5c1iIykT+nUWRzp6F4GA4fhw+/BCef97aEQkhzE0SgSjU/v0wcCDo9eoYQn37WjsiIYQlSBuBKNDu3dCzJ1SrBvv2SRIQojKTKwJxl9RUePJJdYaxH3+EunWtHZEQwpIsekUQGxuLv78/ffr0ISwsrNDttm/fTsuWLfntt98sGY4wgaLAs8+qo4h+/bUkASHsgcWuCAwGA/PmzeOLL75Aq9UyZMgQfH19ad68eb7t0tLSWL16Ne3atbNUKKIEVq+GjRth8WJo397a0YjCZGVlcf78eTIyMqwdSrnKysoiPj7e2mFYjSnld3V15d5776VKlSomH9diiSAuLg4vLy88PT0BCAoKIjo6+q5EsGLFCsaOHctnn31mqVCEif76CyZMgB494JVXrB2NKMr58+epXr06TZo0QaPRWDuccnP79m2qVq1q7TCsprjyK4rClStXOH/+PE2bNjX5uBZLBCkpKXh4eBiXtVotcXFx+bY5duwYycnJ9OzZ0+REoNfrjRkxIyPDbr8dmLvs2dkwapQX4MIbb/zFn39mm+3Y5mbP7zuo5c/KykKr1drdFYGiKNy+fdvaYViNKeV3c3PjwoULJfofsVpjcU5ODkuWLGHx4sUl2s/FxYVWrVoBEB8fb3xsb8xd9oUL4fBhWLMG/PxamO24lmDP7ztg/Ad3c3OzciTlT64ITCt/lSpV7vofKSoxWKyxWKvVkpycbFxOSUlBq9Ual9PT0/nzzz956qmn8PX15ejRozz33HPSYGwFBw/CnDnwxBPqHcRCCPtisSuCtm3bkpCQQGJiIlqtFp1Ox7Jly4zPV69enQMHDhiXR44cybRp02jbtq2lQhIFSE+HESPUQeQ++gjsqLpZCPEPi10RODk5MWvWLMaMGUNgYCD9+vWjRYsWrFixgujoaEu9rCihKVPg1Cn4v/+DWrWsHY2wFSNHjmTPnj351n355ZfMnj270O1zr/bHjh3LjRs37trm/fffL7atcNeuXZw6dcq4vGLFCn766aeShl+s559/nscee8zsx62oLNpG4OPjg4+PT751kyZNKnDbr776ypKhiAJERMAnn6g9hHr1snY0orRWr4bPPzfvMZ95Bp56qvDn+/fvz7Zt2+jevbtx3bZt23jFhO5mn376aanj2r17N46Ojsbeh4V9npTFjRs3OHbsGG5ubiQmJhp7PppbdnY2ThVk9EYZYsJOpaTAf/8L7drB/PnWjkbYGn9/f3744QcyMzMBtTvrpUuXiIyMZNCgQQQFBfHee+8VuK+vry+pqakArFy5En9/f4YNG8aZM2eM22zYsIHBgwczYMAAJk6cyO3btzl8+DAxMTG8+eabDBw4kHPnzvHaa6/x3XffAbBv3z5CQkIIDg5m+vTpxth8fX157733CA0NJTg4mNOnTxdZth07dtCrVy+CgoLQ6XTG9WfPnmXUqFEMGDCA0NBQzp07B0BYWBjBwcEMGDCAt99+G8h/BZSamoqvry8AmzZtYvz48Tz11FOMGjWK9PR0nn76aWNsO3fuNL5eeHi48bivvPIKaWlpBAYGkpWVBaj3YPn6+hqXy6JipCNRrhRF/cZ38yasXQsuLtaOSJTFU08V/e3dEmrWrMlDDz1EbGwsfn5+bNu2jX79+vHss89Ss2ZNDAYDo0aN4sSJEzzwwAMFHuP3339n27ZthIeHYzAYCA0NpXXr1gD06dPHWDXzzjvvsHHjRkaOHImPjw9+fn4EBATkO5Zer+e1117jyy+/pGnTpkybNo2vv/6aUaNGAVCrVi02b97M2rVr+fzzz1m4cGGhZdPpdLzwwgvUqVOHF198kfHjxwMwdepUxo0bR58+fdDr9eTk5BATE8OuXbvYsGEDVatW5dq1a8Weu+PHj7N161Zq1qxJdnY2H374IdWqVSM1NZXHH3+c3r17c+rUKVauXMm6deuoXbs2165do1q1anTo0IGYmBj8/PzQ6XT07du3RDeOFUauCOzQxx+ro4m++Sb8838nRIkFBQWxbds2QP3wDAoKIioqitDQUEJCQjh58mSR374PHTqEn58fVatWpVq1asZvzQAnT55k+PDhBAcHExERwcmTJ4uM5cyZM9x7773Gm6hCQ0M5dOiQ8fm+/4ya2KZNG5KSkgo9zuXLlzl79iyPPPIITZs2xcnJiT///JO0tDRSUlLo06cPoHZjr1q1Kvv27WPQoEHGLp01a9YsMk6Abt26GbdTFIXly5cTHBzM6NGjSUlJ4fLly+zfv5+AgABq166d77ihoaF8++23gHp1MWjQoGJfzxSSCOzMiRNqA7G/P7zwgrWjEbasd+/e7Nu3j2PHjpGRkUGNGjX4/PPP+fLLL4mIiKBnz57o9fpSHfu1115j1qxZREREMGHCBGM1T2nlfmt2cHDAYDAUul1UVBTXr1+nd+/e+Pr6kpSUlK96yFSOjo4oigJwV+x57wOIiIggNTWVTZs2sWXLFurWrVvkOfP29iYpKYkDBw5gMBi4//77SxxbQSQR2JHMTLWrqJub2rjoIO++KIN77rmHTp06MWPGDIKCgkhPT6dq1apUr16dy5cvExsbW+T+HTt2ZOfOnWRkZJCWlsbu3buNz6Wnp1OvXj2ysrKIiIjI95rp6el3Hatp06YkJSVx9uxZALZs2ULHjh1LXCadTseqVavYtWsXu3bt4ttvv0Wn01GtWjU8PDyMdfiZmZncvn2brl27smnTJuPdvrlVQ40aNeL3338HMLZhFOTmzZvUqVOHKlWqsH//fuPVSufOnfnuu++4evVqvuMChISEMGXKFLNdDYAkArsyZ4569/Cnn0LDhtaORlQG/fv358SJEwQFBfHAAw/w4IMP0q9fP6ZMmUL7YkYtbN26NYGBgQwcOJCxY8fmu4do0qRJDB06lGHDhtEsz+TY/v7+fPbZZ4SEhBgba0Gtqlm8eDGTJk0iODgYjUbDsGHDSlSW8+fPk5SUxMMPP2xc5+npSfXq1fn111958803Wb16NcHBwTzxxBNcvnyZHj164Ovry+DBgxk4cCCf/9N965lnnmHdunWEhIQYP8wLEhwczO+//05wcDBbtmwxlrVFixaMHz+ekSNHMmDAAJYsWZJvnxs3btC/f/8Sla8oGiX3+sVG5B1ewF6HGti1C8LD/+a+++pRvTq4u0P16uR7nPs7t3fanj3g46M2Eq9aZd34y8pe3/dcuUMF2OM5kCEmbhMTE0N0dDRvvfVWodsV9D9S1P+N9BqyIYqiNvBOnw6KUs+kfapWVRNCejo0awbvvmvZGIUQlrNkyRJ++umnIud3KQ1JBDYiM1OdMObLL+Hxx2Hq1D+4776W3LihdgMt7ndmJrz8sjr1pBD27ttvv2X16tX51rVv377QO6Mritdee80iV0SSCGzA5cswaJBavTN7tvpz4kQOtWrJsBBClMbgwYMZPHiwtcOoMCQRVHDx8dC/PyQlwbp16gihQghhTpIIKrAdO+Cxx8DVFX74ATp3tnZEQojKSLqPVlAffQSBgeDlBT//LElACGE5kggqmOxsmDhRveu3Xz/48Udo3NjaUQlbt3YtNGmi3kTYpIm6LEQuqRqqQK5fV3sEbd+uDgOxdCk4Olo7KmHr1q6FcePg1i11+exZdRnUO81L6urVq8bB3C5fvoyDg4NxTJxvvvkGZ2fnUsU5duxYli1bhru7e6HbfPTRR3Tp0oWuXbuW6jWef/55Ll++zIYNG0q1f2UliaCC+OsvCA6GP/9U7/wdM8baEYnKYubMf5NArlu31PWlSQS1atViy5YtgDqZjJubG//973+Nz5d2nH1T5il4/vnnS9190h7nGTCVbUVbSe3Zo3YPNRjUBmKZJEaYU56RGExaXxqvvfYazs7OxMfH0759e4KCgli4cCF6vR5XV1cWLVpEs2bN2LRpE7t27eL27dskJibi5+fHtGnTAHXegI0bN3Lr1i3Gjh3LI488wpEjR9BqtXz00Ue4urryxhtvGIehjomJYfHixbi5udG+fXsSExP55JNPCo0xd56BunXrotPpjMNLnz17ltmzZ5OamoqjoyMrVqygcePGhIWFERERgUajoUePHkydOjXflLqpqakMGTKEXbt2sWnTJnbs2MGtW7fIycnhk08+4fnnn+fGjRtkZ2czadIk/Pz8AHWegc8++wyNRkPLli2ZPXs2AwYMYPv27VSpUoW0tLR8y+XBbhLBoUPqt+ySDmLYtCmEhMCAAaDVmi+ejAzYuRM2b4avvlJfJzISWrQw32sIAWob0z9jsd213pxSUlJYv349jo6OpKWlsXbtWpycnPjpp5945513eP/99wF1qIPw8HCcnZ0JCAhg5MiRNGjQIN+xzp49y/Lly1mwYAGTJk1i+/btDBw40Pi8Xq9n1qxZrFmzBk9PTyZPnlxsfBV5noFOnTpZZJ4BU9lNIqhZE9q0KVkiUBT45Rd17P5nn4Vu3SA0VE0MecbBMtn166DTqR/+UVHqsA/u7jB8OCxfDv9UswphVgsX5m8jAHUE2iLmZimVgIAAHP9p1Lp58yavvvoqZ8+eRaPR5JtFq0uXLlSvXh2A++67j6SkpLsSwb333mscF6d169Z3zSHw119/4enpaazeCQoKKrLeP+88AxqNxjjPQMOGDe+aZwAw2zwDBw8exMHBodh5BoYMGcKqVavw8/Nj06ZNzC/naQMtmghiY2NZuHAhOTk5DB06lHG5LVT/WLduHV9//TUODg64ubkxf/5841yk5ta8OaxZU/L9FAXi4iA8XP0AnzJF/XnoITUphIaqjzWagve/eBG2blX33bULsrLAwwOefFJNKL6+UMq2NSFMktsOMHOmWh3UuLGaBErTPlCUvHX3K1asoFOnTnz44YecP3+ep/JMoZa3MdnR0bHA+QHu3Ka08xrkyjvPAKjTPOp0OsaOHVui45RmnoEqVarg6+tbZBkeeeQR5s6da/Z5Bkxlse6jBoOBefPmsWrVKnQ6HZGRkZw6dSrfNrmzD23ZsoUxY8awePFiS4VTahqNOq/v7Nlw9CicPg3Llqnf5OfNg4cfhvvug8mT1bp+gwFOnYK33oKuXaFRIxg/Xl03aRLs3aveJfzxxxAQIElAlI8RIyAhAXJy1N/mTgJ3unnzJtp/6lI3b95s9uM3bdqUxMREzp8/D2CcKa0w9jrPgKkslgji4uLw8vLC09MTZ2dngoKCiI6OzrdNtTwjoN2+fRtNYV+rK5Bmzf790L94Ue3h06oVfPgh9OihVkG1aAHTpqntAHPnqlcUJ0/+mxxkQhhR2Y0ZM4bly5cTEhJCdna22Y/v6urK7NmzGTNmDIMGDeKee+7J93mSlz3PM2AyxUKioqKUGTNmGJc3b96szJ07967t1qxZo/Tu3Vvp0aOHcubMmWKPe/z48QIfW9v164qyfr2ijB2rKO+8oygmFKVMKlLZy5s9l11R1PLb6zm4deuW8XFaWpqiKIqSk5OjzJ49W/niiy+sFJV5REVFKVOnTi1ym7zlL0pBfx9F/c1YvbF4xIgRjBgxgoiICFauXMnSpUuL3F6v1xsn5sjIyDA+rggeekj9Abh9Wx0wzlIqWtnLkz2XHdTyOzo6Gqst7ImiKMZyr127loiICLKzs2nZsiUDBgyw2XOyZMkS9u7dywcffFBkGfKWvyhZWVkl+h+xWCLQarUkJycbl1NSUox1hgUJCgpizpw5xR7XxcXF7mcoAym7vZYd/p2hzB5n6so7Q9m4cePu6oBiq/MMzJ0716TtTJ2hrUqVKgXOUFYYiyWCtm3bkpCQQGJiIlqtFp1Ox7Jly/Jtk5CQQJMmTQD44Ycf8PLyslQ4QlQ6iqLYRLtaeZJ5BjD2aioJiyUCJycnZs2axZgxYzAYDAwePJgWLVqwYsUK2rRpQ+/evVmzZg379u3DyckJd3f3YquFhBAqV1dXrly5Qp06dSQZCCNFUbhy5Qqurq4l2s+ibQQ+Pj74+PjkWzdp0iTj49dff92SLy9EpXXvvfdy/vx5/v77b2uHUq6ysrLK9Y7bisaU8ru6unLvvfeW6LhWbywWQpRclSpVaNq0qbXDKHfSPmSZ8kuPdiGEsHOSCIQQws5JIhBCCDunUUrT18iKjh49ahwhUAghhGn0en2+YTbysrlEIIQQwrykakgIIeycJAIhhLBzkgiEEMLOSSIQQgg7J4lACCHsnCQCIYSwczY51lBsbCwLFy4kJyeHoUOH3jUmeWXn6+vLPffcg4ODA46OjmzatMnaIVnM9OnT+eGHH6hTpw6RkZGAOs/ryy+/TFJSEo0aNeLdd9+lRo0aVo7UMgoq//vvv8+GDRuoXbs2AJMnT75rcMfK4OLFi0ybNo0rV66g0Wh47LHHePrpp+3i/S+s7BZ7702a96wCyc7OVnr37q2cO3dO0ev1SnBwsHLy5Elrh1WuevXqpVy5csXaYZSLn3/+Wfn999+VoKAg47qlS5cqn3zyiaIoivLJJ58ob775prXCs7iCyv/ee+8pq1atsmJU5SMlJUX5/fffFUVRlJs3byp9+/ZVTp48aRfvf2Flt9R7b3NVQ3FxcXh5eeHp6YmzszNBQUFER0dbOyxhIR07drzr2150dDQhISEAhISEsHPnTitEVj4KKr+9qF+/Pq1btwagWrVqNGvWjJSUFLt4/wsru6XYXCJISUnBw8PDuKzVai16giqq//73vwwaNIj//e9/1g6l3F25coX69esDUK9ePa5cuWLliMrf2rVrCQ4OZvr06Vy/ft3a4Vjc+fPniY+Pp127dnb3/uctO1jmvbe5RCBg3bp1bN68mU8//ZS1a9dy8OBBa4dkNRqNxu5m6Bo2bBjff/89W7ZsoX79+ixZssTaIVlUeno6L774IjNmzKBatWr5nqvs7/+dZbfUe29ziUCr1ZKcnGxcTklJQavVWjGi8pdb3jp16tCnTx/i4uKsHFH5qlOnDpcuXQLg0qVLxoYze1G3bl0cHR1xcHBg6NCh/Pbbb9YOyWKysrJ48cUXCQ4Opm/fvoD9vP8Fld1S773NJYK2bduSkJBAYmIimZmZ6HQ6fH19rR1Wubl16xZpaWnGx3v37qVFixZWjqp8+fr6Eh4eDkB4eDi9e/e2bkDlLPdDEGDnzp2V9v1XFIWZM2fSrFkzRo8ebVxvD+9/YWW31Htvk6OPxsTEsGjRIgwGA4MHD+a5556zdkjlJjExkRdeeAEAg8FA//79K3X5J0+ezM8//8zVq1epU6cOEydOxM/Pj5deeomLFy/SsGFD3n33XWrWrGntUC2ioPL//PPPnDhxAoBGjRoxb948Y515ZXLo0CFGjBjB/fffj4OD+p118uTJPPTQQ5X+/S+s7JGRkRZ5720yEQghhDAfm6saEkIIYV6SCIQQws5JIhBCCDsniUAIIeycJAIhhLBzNjn6qBCW0KpVK+6//37jclBQkNlGtj1//jzjx483jiAqREUiiUCIf7i6urJlyxZrhyFEuZNEIEQxfH19CQgIYM+ePbi4uLBs2TK8vLw4f/48M2bM4OrVq9SuXZvFixfTsGFDLl++zOzZs0lMTARgzpw51K9fH4PBwOuvv86RI0fQarV89NFHuLq6snr1atavX4+joyPNmzfnnXfesXKJhb2RNgIh/pGRkcHAgQONP9u2bTM+V716dSIiInjyySdZtGgRAAsWLCA0NJSIiAiCg4NZsGCBcX3Hjh3ZunUrmzdvNg4DcPbsWUaMGIFOp6N69eps374dgLCwMMLDw4mIiGDu3LnlXGohJBEIYZRbNZT7ExgYaHyuf//+gNpucPToUQCOHDliXD9w4EB++eUXAPbv38/w4cMBcHR0pHr16gDce++9tGrVCoDWrVuTlJQEQMuWLZk6dSpbtmzB0dHR8gUV4g6SCIQoJ87OzsbHjo6OGAwGQL0iGD58OMePH2fIkCFkZ2dbK0RhpyQRCGGCqKgoALZt24a3tzcA3t7e6HQ6ACIiIujQoQMAXbp04euvvwbUgQFv3rxZ6HFzcnK4ePEinTt3ZurUqdy8eZNbt25ZsihC3EUai4X4R24bQa7u3bszdepUAK5fv05wcDDOzs4sX74cgDfeeIPp06fz2WefGRuLAWbOnMkbb7zBt99+i4ODA3PmzKFevXoFvqbBYOCVV14hLS0NRVF46qmncHd3t3BJhchPRh8Vohi+vr5s3Lix0k6AIoRUDQkhhJ2TKwIhhLBzckUghBB2ThKBEELYOUkEQghh5yQRCCGEnZNEIIQQdu7/ASLEpoUhOsToAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "val_acc = history['val_acc']\n",
    "acc = history['acc']\n",
    "epochs = range(1, 26)\n",
    "\n",
    "plt.plot(epochs, val_acc, 'b-', label='Validation_Accuracy')\n",
    "plt.plot(epochs, acc, 'bo', label='Traninig_Accuracy')\n",
    "plt.title(\"Training and Validation Accuracy\")\n",
    "plt.xlabel(\"Epochs\")\n",
    "plt.ylabel(\"accuracy\")\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "structured-approval",
   "metadata": {},
   "source": [
    "### 7. 성능 최적화\n",
    "```\n",
    "epoch을 25으로 하고 매 epoch 마다 검증 데이터의 val_loss, val_accuracy를 계산하여 그래프로 나타내었다.\n",
    "loss의 경우 6 epoch 이후 점점 줄어들면서 17 epoch 이후로 다시 그래프가 상승 하강을 반복하는 현상이 보인다.\n",
    "accuracy도 마찬가지로 17 epoch 이후로 그래프가 하락하기 때문에 이 지점부터 overfitting을 의심 해볼 수 있다. \n",
    "따라서 epoch을 17로 재설정하고 모델을 재학습 시켜보았다.(모델을 다시 초기화하고 학습시켜야한다.)\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "paperback-barcelona",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/17\n",
      "41/41 [==============================] - 2s 21ms/step - loss: 1.5062 - acc: 0.4599 - val_loss: 1.1405 - val_acc: 0.3944\n",
      "Epoch 2/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.6251 - acc: 0.7499 - val_loss: 1.2330 - val_acc: 0.3197\n",
      "Epoch 3/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.3813 - acc: 0.8568 - val_loss: 1.3837 - val_acc: 0.3221\n",
      "Epoch 4/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.2697 - acc: 0.9001 - val_loss: 1.4402 - val_acc: 0.3291\n",
      "Epoch 5/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.1911 - acc: 0.9312 - val_loss: 1.4642 - val_acc: 0.4387\n",
      "Epoch 6/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.1890 - acc: 0.9334 - val_loss: 1.5207 - val_acc: 0.4889\n",
      "Epoch 7/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.1183 - acc: 0.9592 - val_loss: 1.5714 - val_acc: 0.4294\n",
      "Epoch 8/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.1461 - acc: 0.9445 - val_loss: 1.6970 - val_acc: 0.4761\n",
      "Epoch 9/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.1054 - acc: 0.9654 - val_loss: 3.5189 - val_acc: 0.3559\n",
      "Epoch 10/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0723 - acc: 0.9746 - val_loss: 0.6113 - val_acc: 0.7701\n",
      "Epoch 11/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0555 - acc: 0.9835 - val_loss: 0.4646 - val_acc: 0.8226\n",
      "Epoch 12/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0460 - acc: 0.9839 - val_loss: 0.2222 - val_acc: 0.9137\n",
      "Epoch 13/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0446 - acc: 0.9853 - val_loss: 0.1183 - val_acc: 0.9557\n",
      "Epoch 14/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0712 - acc: 0.9749 - val_loss: 0.9682 - val_acc: 0.7690\n",
      "Epoch 15/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0607 - acc: 0.9795 - val_loss: 0.3875 - val_acc: 0.8763\n",
      "Epoch 16/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0523 - acc: 0.9851 - val_loss: 0.1697 - val_acc: 0.9312\n",
      "Epoch 17/17\n",
      "41/41 [==============================] - 1s 16ms/step - loss: 0.0444 - acc: 0.9865 - val_loss: 0.0466 - val_acc: 0.9825\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<tensorflow.python.keras.callbacks.History at 0x7fa96097d290>"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = build_model() # 모델 초기화\n",
    "model.compile(optimizer=\"adam\",\n",
    "              loss=\"sparse_categorical_crossentropy\",\n",
    "              metrics=[\"acc\"]\n",
    "             )\n",
    "\n",
    "n_train_epoch = 17\n",
    "\n",
    "model.fit(x=X_train_partial,\n",
    "          y=y_train_partial,\n",
    "          epochs=n_train_epoch,\n",
    "          batch_size=64,\n",
    "          validation_data=(X_val, y_val))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "executed-montana",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_7 (Conv2D)            (None, 26, 26, 32)        896       \n",
      "_________________________________________________________________\n",
      "batch_normalization_7 (Batch (None, 26, 26, 32)        128       \n",
      "_________________________________________________________________\n",
      "conv2d_8 (Conv2D)            (None, 24, 24, 32)        9248      \n",
      "_________________________________________________________________\n",
      "batch_normalization_8 (Batch (None, 24, 24, 32)        128       \n",
      "_________________________________________________________________\n",
      "conv2d_9 (Conv2D)            (None, 12, 12, 32)        25632     \n",
      "_________________________________________________________________\n",
      "batch_normalization_9 (Batch (None, 12, 12, 32)        128       \n",
      "_________________________________________________________________\n",
      "dropout_3 (Dropout)          (None, 12, 12, 32)        0         \n",
      "_________________________________________________________________\n",
      "conv2d_10 (Conv2D)           (None, 10, 10, 64)        18496     \n",
      "_________________________________________________________________\n",
      "batch_normalization_10 (Batc (None, 10, 10, 64)        256       \n",
      "_________________________________________________________________\n",
      "conv2d_11 (Conv2D)           (None, 8, 8, 64)          36928     \n",
      "_________________________________________________________________\n",
      "batch_normalization_11 (Batc (None, 8, 8, 64)          256       \n",
      "_________________________________________________________________\n",
      "conv2d_12 (Conv2D)           (None, 4, 4, 64)          102464    \n",
      "_________________________________________________________________\n",
      "batch_normalization_12 (Batc (None, 4, 4, 64)          256       \n",
      "_________________________________________________________________\n",
      "dropout_4 (Dropout)          (None, 4, 4, 64)          0         \n",
      "_________________________________________________________________\n",
      "conv2d_13 (Conv2D)           (None, 1, 1, 128)         131200    \n",
      "_________________________________________________________________\n",
      "batch_normalization_13 (Batc (None, 1, 1, 128)         512       \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dropout_5 (Dropout)          (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 3)                 387       \n",
      "=================================================================\n",
      "Total params: 326,915\n",
      "Trainable params: 326,083\n",
      "Non-trainable params: 832\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "model.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "thrown-montana",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "27/27 [==============================] - 4s 4ms/step - loss: 0.0551 - acc: 0.9837\n"
     ]
    }
   ],
   "source": [
    "results = model.evaluate(X_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "involved-counter",
   "metadata": {},
   "source": [
    "## 8. 예측\n",
    "#### 모델은 첫번째 테스트 사진을 어떻게 예측했을까?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "suited-status",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictions = model.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "respiratory-hampshire",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# 모델이 예측한 첫번째 사진의 정답은? (두근두근)\n",
    "np.argmax(predictions[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "divine-modification",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "라벨:  1\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAOcAAADnCAYAAADl9EEgAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjMuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/Il7ecAAAACXBIWXMAAAsTAAALEwEAmpwYAAAPW0lEQVR4nO3dWY8b1xGG4epu7hzOotWxJdsBEhtIkKv88CD5OYYVeR1Ls3G4NXsjOxe+1fnKGWOggvE+lz46ZLPJGgL8XHWyvu97AxBO/rEvAMCHUZxAUBQnEBTFCQRFcQJBDeTqUW8+HvR6Xoi9TSf3Xr/7Wa5vN/dyfTxK/90ZD8WFmdlsNpXr3aGR61VVOetlerHQfy8nk4lcz3L9lnadvu8/vfkxubZcLuXe3V68rt+grNL39erm2tlby/Ws0O95n+n7/s23b5Jr76+v5N7JdC7X//2v/3zwv/PNCQRFcQJBUZxAUBQnEBTFCQRFcQJBUZxAUDIU652cs+v0Pxjl6drPRzqPOzk50U/e67yuP6Yzs65r5d63b9/KdZXfmpmJl21mZqoRqHaura51nlc7+bGXc3Zl+r6VKp81s+NRfx6yLNPPLfJj77G9ez4aj/U/cN7UFy+eJ9e87Hk4HunnTl3Sg3YBeHQUJxAUxQkERXECQVGcQFAUJxAUxQkEJcNG1V9nZm4QWmTph9+XW7n38jLdV2hmtlnr3sLxMP3c46H+m3Q46qyxqvV61+n7djikG2G3Tk/kdqvvW6+jRDeTOzbpa/MGNQ4GTs9kri9O5aBZ5j23zs1HY71e7fV7drZYJNe8e9o7+W4K35xAUBQnEBTFCQRFcQJBUZxAUBQnEJT8fblwxjS2TntT1abbk1pnvGTm/Cw/Guk2nL5PX9um3Mu9uemf7Xe7jVwvSx2HqChlV+lr86KUQkRIZn7bVrnaJdcap93Me2wvillv0899v9LRWSM+a2Zmo43+vCzOL+T6ZJLeP3RGre6d6C2Fb04gKIoTCIriBIKiOIGgKE4gKIoTCIriBIKSodhmt5ab906ep/LC8VBnYqen6RYdM7P5VOd56/t0LrZc3sm9JzOnBcjJ6wrnuDnVYjSc6BGOuTMDsmke3q5mZjYQrVWdM47UGzl6dPrZBgNxbKMz2rJzzqP07svKyVHV91jtPPZ+r8eZ/v/PCOCjojiBoChOICiKEwiK4gSCojiBoChOICgZFu5K3bfoZW5qVOLWGY15e/1OrvetzpYmIq87Pz+VewtnDKOXuXlZorf+e57bGxGZi3GlZmajhXjPtk7fopPnef2es9ksfV1O/jvd6sy9qiq5fn2rs2/1njXOsYtN+7D3m29OICiKEwiK4gSCojiBoChOICiKEwiK4gSCckIxXbuzkxO5Pp+ms6m207mTN/u1bfT+0fgsuXYqjnMzM1s5xwt6+a5Hvba61lnhwemJ9LLEPNfHNjbiWMdtmZ4ra2ZW7vR74vXBqvvqZYn7vZ73u3PWT0919n08qvumPw9ZrsvsYY8K4KOhOIGgKE4gKIoTCIriBIKiOIGg5G+817c3cvPd/a1cn4sRk96PyxfPnsr1Q+1EMWKs5w/fvZV7v/z8C7n+5MnUWX8i18syfe3eEX9eJFBVupXOi2qaOr2/a3UM0zlHBHad3n8Qo1SbWj926bSE1eJ1mZllzjjTzNLr47H+PKhRqArfnEBQFCcQFMUJBEVxAkFRnEBQFCcQFMUJBCXjxm/efCs3b1f3cl01L/3pxXO59/mzC7luznFzOzkqUf9NurvTYxKfP9fXfuq00o2G6RGQL55/IvcOxiO5/v79lVx/8+aNXH/1+vPkmne0YbXXWeLtrc7FVcbbyZYtf/Tlbqvz4bc//CjXp9N0ltl0+rGv7rzjBT+Mb04gKIoTCIriBIKiOIGgKE4gKIoTCIriBIKSOeeLT17KzU+fOj2XTbp3MHeO2fN6/y7OdQ762aefJtdGhf6bdHuls0Ivr9ut9QjJz0W/qHeE3/ff6Tzup8tLud4f9ejMd79cp/c6oy29fs4s0znp4jT9nh4O+rnXW/26L9/p3uS8GMr1XvRzts61VU4vavKaHrQLwKOjOIGgKE4gKIoTCIriBIKiOIGgKE4gKBmq7Us943Q8Th/xZ2aWD9P5T73T81nbypmv6qzfihmo9V7nkEWm/2aNh7qncjKZy3V1nN18lj660Mzs73/7h1x//erPcn25dI43nKdf2/V1OgM1M/v+e53BXt3o/eL0QTMnm75f6SMjayc3f/XqS7mu5tpWje4tzif685bc96BdAB4dxQkERXECQVGcQFAUJxAUxQkERXECQcmcc744lZuLQvcGZn36XMLTuTPbNdePfej0jNQ70ZN58+693Dsd6/MUv/rLX/X+iT6v8fLn9LVtNzq//frrr+X6+ZnOnq+v9EzeF2fpmbynC92/+8nL1/q5b/VzvxPvmbd3W+qeyfFU58fLtc5JTeScrZOh7pwzU1P45gSCojiBoChOICiKEwiK4gSCojiBoGSUculEDp0YfWmmW69O5jpumDjHzXWVPnatKdPr04mOcYbOc9/e6rar3VD/LH8Ux9ltN3rvf9/8INe9Nr7pdCHXyzL9s/+ZGF1pZvbsaXocqZnZZ68Ocv3ufpVce3+r282unfdkvd3I9cxpA1SjM/uB/o5rDzpqST7ng3YBeHQUJxAUxQkERXECQVGcQFAUJxAUxQkEpUdj7nWWmGW6rWs8TWdDeaaPuttu9TjB3epeP7cYpXh+qtuHynU6bzMzW93rsZ7NSI9KPDlJt+INBjpv63vnno/0WM5nT3XOuanTOed2oz8Pd7f6vnVHnXMe83S+PJ3o6371+lw/tun7dnS+pvJROj8uxk5GOtDHCyb3PWgXgEdHcQJBUZxAUBQnEBTFCQRFcQJBUZxAUDJsvL7R4wjns5l+dNHHVh51VrjfrOV65xwBOBqkM7POObJtJjItM7PhUK97f/NWq3Rv4eGQPrrQzGw40H2wq17ft6sr3Rd5LNKZXd3p+9Y5IyIPOmo0E+NQ9eBLfzxl0+uMdXHxRK5nwyq5NnRGoeYjnYPaxYfX+eYEgqI4gaAoTiAoihMIiuIEgqI4gaAoTiAomXMul8581q3OKu2QTqdaJ6ecOtnQy6c6lzpfpGfTDsU8XTOzzBkzutvpXtPsqLPK+Tzdmzgc6Az14MxA9XpwvSxyVYrXJnpkzczyTM/7bc3JIkWOWnuzX0WubWY2cI51vFvqXlQbph8/H+lZw9lQ9y7bFx+eB8w3JxAUxQkERXECQVGcQFAUJxAUxQkERXECQckAZr25l5vPzvT812GRfvjOdH/dunayo7VuDizbdN7XO5lZ0et172zRxVz3udZF+rWtGn1fmuxerg/E7FczfTaomVnTp/9eZ05DZubkoAcnX+7y9Gs/9rqjszvqa2sbnTVWTq/qSPT49nv93E3jdKP+86sP/me+OYGgKE4gKIoTCIriBIKiOIGgKE4gKPn78mKhj12bOaMxB4P0wxfOz+7joT427eIsfYyemdl0km45652ftjNnjOKg0D+d57l+bW2bfvx6fy/33rbpI/rMzPpet6t5xzb2uYiJxOhKM7Os0O+Zc3qhNeKIwLLVUUfd6PvSOG18sxN9dGIvYqDGee6yTI/VVPjmBIKiOIGgKE4gKIoTCIriBIKiOIGgKE4gKJlzTuc6+ymcLLIX7Um509o0cEZjDkbOc4tQrWr0WM7MGeHY73Xm5o3GVDFpVelWuarUoy8HTsY6nerRm+dP022A+1rft+1aHxm52+vXVotRql67mdeudnQ+bzcrPQZWvaWtl8HWej2Fb04gKIoTCIriBIKiOIGgKE4gKIoTCIriBIKSOedcHKNnZjr8MbOqSvexHU3vPRx0T2VZ6R66Q51+7rWTaT1xekUbJ+/LndGaI3Ek3MFpevTyvpGT/87PzuW6GhHp5cNejrnc6CMjG5FzZs4Rf14v6cH5Grq+udH/QOTHRaGvLc+cIwBT+x60C8CjoziBoChOICiKEwiK4gSCojiBoChOICgZwGy3OrdqW517yZyz0znmbKL7DvcixzQzq/a75NrmfiX3jqcT57H1fTHniMHhIP038djqmbpVrfs5m07ft1wcZWdm9svVZXJNzSE287Nr55Q+60Q+3Oz1fdlUOrvelfo9y53Xpq7NmwXs3bfkNT1oF4BHR3ECQVGcQFAUJxAUxQkERXECQekoZZeOI8zMetNxyEH8/Fy4LUD65+dctF2ZmY2O6aPsZgt93W2nf7ZvnVa541Hv71oRpYhxomZmR+dn+9ppV1tXOopRbWEj5+hDc66tcSKmShzN2DjHMh6dow+94wd753jDqkzfl/1e31PvPU3hmxMIiuIEgqI4gaAoTiAoihMIiuIEgqI4gaBkWDiZ6Nap3DlurhiksyOvjWbitDZ5IyDV8YNtey73dk472nTgjGE86CPfchHJOSfV2cD5B177kmdxkX5tI+dYxkaM1TQz2+z1fa1FPpw5r3vijHEdnejjLFcr3UbYHtM5a9U6Y1qdMa8pfHMCQVGcQFAUJxAUxQkERXECQVGcQFAUJxCUDBuXa539eFnlWOSBKgM1M9sXukfOy/t60dfo9depjPTX/U6/Z6vzPjX+0jtObjhyXrfTM1k7xxeOi1lyrRjr7NlbH07TPbZmZoXISd0jI533TOWUv2Vd9YMOnMx9XOj/XyCFb04gKIoTCIriBIKiOIGgKE4gKIoTCIriBILSQaWTmWWqMdF0nti3Ouf0ssjWdJaoskZ1NKGZ2cHJKb181+t7zEX+Wzszc1f3W7m+3+mj7ppG9x6eTM6Sa5NSZ8/jmc4xd04/5/1qk1zLnDnH03k6nzUz80bH7kp9bSY+rl7f80N7bPnmBIKiOIGgKE4gKIoTCIriBIKiOIGgKE4gKBnYefmMt67m2nozb70s0c8a01nlcKj777yeR6/n0rs2dd9yZ6/X13h0DqLMnJm7qlW17pwzMiudoe6dfHkvzgbNj/qeH5yvmdrJdzsnX1a8vJ+cE/iDoTiBoChOICiKEwiK4gSCojiBoOTv9ttNuoXHzI8MVFySOVGKF3d464r3s/lyudQPUOhr/z3H8HnHxTWtc4yeEwN5r/3Z4lly7dDoyKB1HnvntJxVIorJnZax2jl+cFfqVjrvvqv31I0UM33fkvsetAvAo6M4gaAoTiAoihMIiuIEgqI4gaAoTiAoGVTOp3rcoNf2pdqbvNGX1jvZkLOuVntn78uXL+X68eEx5q/7xdV5eZt3vKC37j3+YZvOKhtxdKGZ385W1/raVAZbOPfcOcHP9nuvDfDh7ZG5k3P25JzAHwvFCQRFcQJBUZxAUBQnEBTFCQRFcQJBZb0X+gH4KPjmBIKiOIGgKE4gKIoTCIriBIKiOIGg/gd7WenZbKBGwgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 저는 이 사진 뭔지 모르겠는데 바위라고 합니다... 여러분 잘 보이시나요? ㅠ_ㅠ \n",
    "plt.imshow(X_test[0])\n",
    "plt.axis('off')\n",
    "print('라벨: ', y_test[0])\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hispanic-combination",
   "metadata": {},
   "source": [
    "### 9. 결과해석 및 회고\n",
    "\n",
    "```\n",
    "테스트 셋 837개를 사용한 모델의 성능은 정확도 98.37%를 보여주었다.\n",
    "predictions = model.predict(X_test)으로 predictions에는 모델이 예측한 이미지의 클래스가 담기게 되고\n",
    "우리는 이제 X_test의 이미지와 실제 label값, 예측 label값을 비교해볼 수 있다.\n",
    "\n",
    "분명히 Basic CNN도 이미지 분류에 꽤 좋은 성능을 보이지만, Chris Deotte씨가 설계한 이 CNN 모델의\n",
    "성능은 놀라움을 주었다. kernel_size, n_dense, Convolution layer의 개수 설정 방법과 같이\n",
    "하이퍼 파라미터를 어떻게 디테일하게 설정하고 layer를 연계할 수 있는지 통찰력을 얻을 수 있는 의미있는 실험이었다. \n",
    "많이 배웠고 앞으로 끊임없이 질문하는 사고가 필요할 것 같다.\n",
    "```"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}