imageClassificationFromScratch.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Image Classification from Scratch\n",
    "#### A Deep learning for Computer Vision Demo Environment by Tillmann Brodbeck\n",
    "This is a demo environment containing all the necessary steps to install a dataset, install a deep learning model and analyze its performance, subsequently.\n",
    "In general it was made sure that every critical step in the Python code is commented and \"speaking\" variables were used to make every bit of code understandable (and to avoid commenting every single line).\n",
    "\n",
    "The first section is about setting up your environment from scratch. Here, a working python3 environment is a prerequisite. \n",
    "It is explained how to download an dataset with `wget`. Additionally all required python libraries are listed.\n",
    "\n",
    "The second section demonstrates which hyperparameters are set and how different hyperparameter configurations can be managed. Additionally it shows how the data preprocessing is done and demonstrates two ways of installing a dataset for training: CIFAR10 is downloaded from `keras` directly and imagenette is imported from disk.\n",
    "\n",
    "Section 3 demonstrates how the models are created and displays the exact architectures that are used in this demonstration in an easy way, built with as `Sequential` `keras` models. Additionally, it demonstrates how the training process can be initiated and outputs the `accuracy` metric and the categorical crossentropy as loss in real time while training.\n",
    "\n",
    "Section 4 demonstrates how the model and its results can be saved to disk using `pickle`.\n",
    "\n",
    "Finally, section 5 shows how trained models can be analyzed. Optionally, models can be imported from disk and tested against the dataset created in section 2. Finally it is shown how simple plots of the training and testing metrics can be created using `matplotlib.pyplot`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Setup your Environment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Download the imagenette Dataset\n",
    "To download the imagenette dataset make sure you have a bash-like environment and `wget` and `tar` installed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!wget https://s3.amazonaws.com/fast-ai-imageclas/imagenette2.tgz\n",
    "!tar -xf imagenette2.tgz"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Install the required Packages into your Environment\n",
    "Anaconda is recommended.\n",
    "\n",
    "The most recent versions right now and the tested version are \n",
    "```\n",
    "keras==2.3.1\n",
    "matplotlib==3.1.3\n",
    "pickle==0.7.5\n",
    "pillow==7.1.2\n",
    "tensorflow== 2.0.0\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip3 install keras matplotlib pickle pillow tensorflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import required Libraries into Python"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from datetime import datetime\n",
    "from keras.activations import relu\n",
    "from keras.layers import Activation, Conv2D, MaxPool2D, Flatten, Dense, InputLayer, BatchNormalization, Dropout\n",
    "from keras.models import load_model, Sequential\n",
    "from keras.preprocessing.image import ImageDataGenerator\n",
    "import matplotlib.pyplot as plt\n",
    "import os\n",
    "import pickle\n",
    "import sys\n",
    "import tensorflow as tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set the Path for the imagenette Dataset\n",
    "Per default the imagenettePath is `'imagenette2/'` (There has be a slash at the end)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "imagenettePath = 'imagenette2/'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Choose a Dataset\n",
    "Possible datasets are `'cifar10'` and `'imagenette'`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# dataset = 'cifar10'\n",
    "dataset = 'imagenette'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Prepare the Dataset and the Configuration"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Chosen Hyperparameters: {'batchSize': 128, 'keepProb': 0.25, 'activationConv': 'relu', 'activationDense': 'relu', 'epochs': 30, 'denseLayers': 2, 'batchNorm': True}\n",
      "Found 9469 images belonging to 10 classes.\n",
      "Found 3925 images belonging to 10 classes.\n"
     ]
    }
   ],
   "source": [
    "\"\"\" the hyperparameters \"\"\"\n",
    "hyperparameterChoice = 2\n",
    "\n",
    "# different parameter configurations can be tracked in dictionaries for quick comparisons\n",
    "if hyperparameterChoice==0: # test configuration with one epoch only\n",
    "    hyperparameter = {\n",
    "        'batchSize': 128,\n",
    "        'keepProb': .5,\n",
    "        'activationConv': 'relu',\n",
    "        'activationDense': 'relu',\n",
    "        'epochs': 1,\n",
    "        'denseLayers': 1,\n",
    "        'batchNorm': False,\n",
    "    }\n",
    "elif hyperparameterChoice==1:\n",
    "    hyperparameter = {\n",
    "        'batchSize': 128,\n",
    "        'keepProb': .5,\n",
    "        'activationConv': 'relu',\n",
    "        'activationDense': 'relu',\n",
    "        'epochs': 30,\n",
    "        'denseLayers': 1,\n",
    "        'batchNorm': False,\n",
    "    }\n",
    "elif hyperparameterChoice==2:\n",
    "    hyperparameter = {\n",
    "        'batchSize': 128,\n",
    "        'keepProb': .25,\n",
    "        'activationConv': 'relu',\n",
    "        'activationDense': 'relu',\n",
    "        'epochs': 30,\n",
    "        'denseLayers': 2,\n",
    "        'batchNorm': True,\n",
    "    }\n",
    "\n",
    "print('Chosen Hyperparameters:', hyperparameter)\n",
    "\n",
    "batchSize = hyperparameter['batchSize']\n",
    "keepProb = hyperparameter['keepProb']\n",
    "activationConv = hyperparameter['activationConv']\n",
    "activationDense = hyperparameter['activationDense']\n",
    "epochs = hyperparameter['epochs']\n",
    "denseLayers = hyperparameter['denseLayers']\n",
    "batchNorm = hyperparameter['batchNorm']\n",
    "\n",
    "\"\"\" the loading and preprocessing the data \"\"\"\n",
    "# setup image preprocessing\n",
    "datagen = ImageDataGenerator(horizontal_flip=True, \n",
    "                             rescale=1./255, # multiplies the images with x (here: scales the images to values between 0 and 1)\n",
    "                             width_shift_range=0.1, # randomly shift images horizontally (x = fraction of total width)\n",
    "                             height_shift_range=0.1, # randomly shift images vertically (x = fraction of total height)\n",
    "                             rotation_range=0 # rotation of the images randomly in the range of x degrees (here: disabled)\n",
    "                            )\n",
    "\n",
    "if dataset=='cifar10': # downloads CIFAR10 from keras\n",
    "    from keras.datasets import cifar10\n",
    "    from keras.utils import to_categorical\n",
    "    \n",
    "    (x_train, y_train), (x_test, y_test) = cifar10.load_data()\n",
    "    x_train = x_train.astype('float32')\n",
    "    x_test = x_test.astype('float32')\n",
    "    y_train = to_categorical(y_train, 10)\n",
    "    y_test = to_categorical(y_test, 10)\n",
    "    print(x_train.shape[0], 'Train Samples')\n",
    "    print(x_test.shape[0], 'Test Samples')\n",
    "\n",
    "    datagen.fit(x_train)\n",
    "    train = datagen.flow(x_train, y_train, batch_size=batchSize, shuffle=True)\n",
    "    val = (x_test, y_test)\n",
    "    \n",
    "elif dataset=='imagenette': # installs `imagenette` from disk\n",
    "    # load train data\n",
    "    train = datagen.flow_from_directory(\"{}train/\".format(imagenettePath), class_mode=\"categorical\", shuffle=True, batch_size=batchSize, target_size=(224, 224))\n",
    "    # load val data\n",
    "    val = datagen.flow_from_directory(\"{}val/\".format(imagenettePath), class_mode=\"categorical\", shuffle=True, batch_size=batchSize, target_size=(224, 224))\n",
    "else:\n",
    "    raise ValueError(f'Invalid Value: {dataset} - only cifar10 and imagenette are valid')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Create and Train the Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential_1\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv2d_1 (Conv2D)            (None, 224, 224, 16)      1216      \n",
      "_________________________________________________________________\n",
      "activation_1 (Activation)    (None, 224, 224, 16)      0         \n",
      "_________________________________________________________________\n",
      "max_pooling2d_1 (MaxPooling2 (None, 112, 112, 16)      0         \n",
      "_________________________________________________________________\n",
      "conv2d_2 (Conv2D)            (None, 56, 56, 32)        12832     \n",
      "_________________________________________________________________\n",
      "activation_2 (Activation)    (None, 56, 56, 32)        0         \n",
      "_________________________________________________________________\n",
      "max_pooling2d_2 (MaxPooling2 (None, 28, 28, 32)        0         \n",
      "_________________________________________________________________\n",
      "batch_normalization_1 (Batch (None, 28, 28, 32)        128       \n",
      "_________________________________________________________________\n",
      "conv2d_3 (Conv2D)            (None, 14, 14, 64)        18496     \n",
      "_________________________________________________________________\n",
      "activation_3 (Activation)    (None, 14, 14, 64)        0         \n",
      "_________________________________________________________________\n",
      "max_pooling2d_3 (MaxPooling2 (None, 7, 7, 64)          0         \n",
      "_________________________________________________________________\n",
      "batch_normalization_2 (Batch (None, 7, 7, 64)          256       \n",
      "_________________________________________________________________\n",
      "dropout_1 (Dropout)          (None, 7, 7, 64)          0         \n",
      "_________________________________________________________________\n",
      "flatten_1 (Flatten)          (None, 3136)              0         \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 100)               313700    \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 100)               10100     \n",
      "_________________________________________________________________\n",
      "dropout_2 (Dropout)          (None, 100)               0         \n",
      "_________________________________________________________________\n",
      "dense_3 (Dense)              (None, 10)                1010      \n",
      "_________________________________________________________________\n",
      "activation_4 (Activation)    (None, 10)                0         \n",
      "=================================================================\n",
      "Total params: 357,738\n",
      "Trainable params: 357,546\n",
      "Non-trainable params: 192\n",
      "_________________________________________________________________\n",
      "Epoch 1/30\n",
      "74/74 [==============================] - 224s 3s/step - loss: 1.8974 - accuracy: 0.3591 - val_loss: 2.2179 - val_accuracy: 0.2594\n",
      "Epoch 2/30\n",
      "74/74 [==============================] - 234s 3s/step - loss: 1.5055 - accuracy: 0.4990 - val_loss: 2.2145 - val_accuracy: 0.2064\n",
      "Epoch 3/30\n",
      "74/74 [==============================] - 255s 3s/step - loss: 1.3595 - accuracy: 0.5500 - val_loss: 1.9456 - val_accuracy: 0.3113\n",
      "Epoch 4/30\n",
      "74/74 [==============================] - 235s 3s/step - loss: 1.2487 - accuracy: 0.5903 - val_loss: 2.7210 - val_accuracy: 0.2479\n",
      "Epoch 5/30\n",
      "74/74 [==============================] - 241s 3s/step - loss: 1.1598 - accuracy: 0.6261 - val_loss: 1.4985 - val_accuracy: 0.5185\n",
      "Epoch 6/30\n",
      "74/74 [==============================] - 233s 3s/step - loss: 1.0656 - accuracy: 0.6550 - val_loss: 1.3580 - val_accuracy: 0.5651\n",
      "Epoch 7/30\n",
      "74/74 [==============================] - 243s 3s/step - loss: 1.0008 - accuracy: 0.6697 - val_loss: 1.1682 - val_accuracy: 0.6298\n",
      "Epoch 8/30\n",
      "74/74 [==============================] - 231s 3s/step - loss: 0.9633 - accuracy: 0.6855 - val_loss: 1.4945 - val_accuracy: 0.5422\n",
      "Epoch 9/30\n",
      "74/74 [==============================] - 232s 3s/step - loss: 0.9226 - accuracy: 0.7029 - val_loss: 1.7433 - val_accuracy: 0.5623\n",
      "Epoch 10/30\n",
      "74/74 [==============================] - 232s 3s/step - loss: 0.8936 - accuracy: 0.7094 - val_loss: 1.2251 - val_accuracy: 0.6283\n",
      "Epoch 11/30\n",
      "74/74 [==============================] - 202s 3s/step - loss: 0.8504 - accuracy: 0.7259 - val_loss: 1.1220 - val_accuracy: 0.6438\n",
      "Epoch 12/30\n",
      "74/74 [==============================] - 202s 3s/step - loss: 0.8279 - accuracy: 0.7318 - val_loss: 1.1916 - val_accuracy: 0.6285\n",
      "Epoch 13/30\n",
      "74/74 [==============================] - 218s 3s/step - loss: 0.7912 - accuracy: 0.7413 - val_loss: 0.8004 - val_accuracy: 0.6797\n",
      "Epoch 14/30\n",
      "74/74 [==============================] - 200s 3s/step - loss: 0.7750 - accuracy: 0.7487 - val_loss: 1.1126 - val_accuracy: 0.6729\n",
      "Epoch 15/30\n",
      "74/74 [==============================] - 209s 3s/step - loss: 0.7598 - accuracy: 0.7539 - val_loss: 1.1220 - val_accuracy: 0.7014\n",
      "Epoch 16/30\n",
      "74/74 [==============================] - 199s 3s/step - loss: 0.7315 - accuracy: 0.7651 - val_loss: 1.0406 - val_accuracy: 0.6288\n",
      "Epoch 17/30\n",
      "74/74 [==============================] - 187s 3s/step - loss: 0.7173 - accuracy: 0.7682 - val_loss: 1.2601 - val_accuracy: 0.6746\n",
      "Epoch 18/30\n",
      "74/74 [==============================] - 188s 3s/step - loss: 0.6922 - accuracy: 0.7699 - val_loss: 0.9998 - val_accuracy: 0.6622\n",
      "Epoch 19/30\n",
      "74/74 [==============================] - 186s 3s/step - loss: 0.6815 - accuracy: 0.7755 - val_loss: 0.9884 - val_accuracy: 0.6655\n",
      "Epoch 20/30\n",
      "74/74 [==============================] - 186s 3s/step - loss: 0.6674 - accuracy: 0.7772 - val_loss: 1.0577 - val_accuracy: 0.6688\n",
      "Epoch 21/30\n",
      "74/74 [==============================] - 190s 3s/step - loss: 0.6559 - accuracy: 0.7823 - val_loss: 1.4771 - val_accuracy: 0.6117\n",
      "Epoch 22/30\n",
      "74/74 [==============================] - 191s 3s/step - loss: 0.6428 - accuracy: 0.7903 - val_loss: 1.0871 - val_accuracy: 0.6678\n",
      "Epoch 23/30\n",
      "74/74 [==============================] - 188s 3s/step - loss: 0.6119 - accuracy: 0.7947 - val_loss: 0.9899 - val_accuracy: 0.6550\n",
      "Epoch 24/30\n",
      "74/74 [==============================] - 188s 3s/step - loss: 0.6077 - accuracy: 0.7979 - val_loss: 1.2252 - val_accuracy: 0.6390\n",
      "Epoch 25/30\n",
      "74/74 [==============================] - 193s 3s/step - loss: 0.5836 - accuracy: 0.8067 - val_loss: 1.3536 - val_accuracy: 0.6283\n",
      "Epoch 26/30\n",
      "74/74 [==============================] - 182s 2s/step - loss: 0.5595 - accuracy: 0.8128 - val_loss: 0.9197 - val_accuracy: 0.7363\n",
      "Epoch 27/30\n",
      "74/74 [==============================] - 181s 2s/step - loss: 0.5739 - accuracy: 0.8079 - val_loss: 1.0881 - val_accuracy: 0.6994\n",
      "Epoch 28/30\n",
      "74/74 [==============================] - 212s 3s/step - loss: 0.5531 - accuracy: 0.8148 - val_loss: 0.8262 - val_accuracy: 0.7437\n",
      "Epoch 29/30\n",
      "74/74 [==============================] - 217s 3s/step - loss: 0.5575 - accuracy: 0.8154 - val_loss: 0.8367 - val_accuracy: 0.6693\n",
      "Epoch 30/30\n",
      "74/74 [==============================] - 229s 3s/step - loss: 0.5310 - accuracy: 0.8244 - val_loss: 0.5819 - val_accuracy: 0.7231\n"
     ]
    }
   ],
   "source": [
    "\"\"\" the architectures \"\"\"\n",
    "\n",
    "if dataset=='imagenette':\n",
    "    architecture = 1 # Architecture 1 is tuned to perform well on the imagenette dataset (but it does not perform too well on CIFAR)\n",
    "if dataset=='cifar10':\n",
    "    architecture = 2 # Architecture 2 performs better for CIFAR, but has a training time that is too large for 'imagenette' (more than 1h per epoch on my resources)\n",
    "\n",
    "if architecture==1:\n",
    "    model = Sequential()\n",
    "\n",
    "    # 1st conv block\n",
    "    model.add(Conv2D(16, (5, 5), input_shape=train[0][0].shape[1:], strides=(1, 1), padding='same'))\n",
    "    model.add(Activation(activationConv))\n",
    "    model.add(MaxPool2D(pool_size=(2, 2), padding='same'))\n",
    "\n",
    "    # 2nd conv block\n",
    "    model.add(Conv2D(32, (5, 5), strides=(2, 2), padding='same'))\n",
    "    model.add(Activation(activationConv))\n",
    "    model.add(MaxPool2D(pool_size=(2, 2)))\n",
    "    if batchNorm:\n",
    "        model.add(BatchNormalization())\n",
    "\n",
    "    # 3rd conv block\n",
    "    model.add(Conv2D(64, (3, 3), strides=(2, 2), padding='same'))\n",
    "    model.add(Activation(activationConv))\n",
    "    model.add(MaxPool2D(pool_size=(2, 2)))\n",
    "    if batchNorm:\n",
    "        model.add(BatchNormalization())\n",
    "    model.add(Dropout(keepProb))\n",
    "    \n",
    "    # ANN block\n",
    "    model.add(Flatten())\n",
    "    for i in range(denseLayers):\n",
    "        model.add(Dense(units=100, activation=activationDense))\n",
    "    model.add(Dropout(keepProb))\n",
    "\n",
    "    # output layer\n",
    "    model.add(Dense(units=10))\n",
    "    model.add(Activation('softmax'))\n",
    "\n",
    "elif architecture==2:\n",
    "    model = Sequential()\n",
    "    # 1st 2xconv block\n",
    "    model.add(Conv2D(32, (3, 3), padding='same', input_shape=train[0][0].shape[1:]))\n",
    "    model.add(Activation(activationConv))\n",
    "    model.add(Conv2D(32, (3, 3)))\n",
    "    model.add(Activation(activationConv))\n",
    "    model.add(MaxPool2D(pool_size=(2, 2)))\n",
    "    model.add(Dropout(keepProb))\n",
    "\n",
    "    # 2nd 2xconv block\n",
    "    model.add(Conv2D(64, (3, 3), padding='same'))\n",
    "    model.add(Activation(activationConv))\n",
    "    model.add(Conv2D(64, (3, 3)))\n",
    "    model.add(Activation(activationConv))\n",
    "    model.add(MaxPool2D(pool_size=(2, 2)))\n",
    "    model.add(Dropout(keepProb))\n",
    "\n",
    "    # ANN block\n",
    "    model.add(Flatten())\n",
    "    model.add(Dense(512))\n",
    "    model.add(Activation('relu'))\n",
    "    model.add(Dropout(0.5))\n",
    "\n",
    "    # output layer\n",
    "    model.add(Dense(units=10))\n",
    "    model.add(Activation('softmax'))\n",
    "\n",
    "else:\n",
    "    raise ValueError(f'architecture number {architecture} is out of range')\n",
    "    \n",
    "model.summary()\n",
    "# compile model\n",
    "model.compile(loss='categorical_crossentropy', optimizer=\"adam\", metrics=['accuracy'])\n",
    "\n",
    "\"\"\" the training \"\"\"\n",
    "# fit on data\n",
    "result = model.fit_generator(train, epochs=epochs, validation_data=val)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create a notification or alarm as soon as the training is done.\n",
    "This is optional and if you are not on MacOS you might want to adapt the code to fit your setup."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "!osascript -e 'display notification \"is done!\" with title \"Nobebook\"'\n",
    "!say 'Notebook done!''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Save the Model and the Results to Disk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Models and Training Output were saved to the Directory demoResults/\n"
     ]
    }
   ],
   "source": [
    "outDir = 'demoResults/'\n",
    "if not os.path.exists(outDir):\n",
    "    os.makedirs(outDir)\n",
    "runName = '{}{}_{}'.format(outDir, architecture, hyperparameterChoice)\n",
    "model.save('{}.model'.format(runName))\n",
    "with open('{}result.p'.format(runName), 'wb') as file:\n",
    "    pickle.dump(result, file)\n",
    "print(f'Models and Training Output were saved to the Directory {outDir}')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Analyze the Results\n",
    "### Load the data from a file\n",
    "This is optional and will overwrite the results of the previous training if not `None`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "loadResultsFrom = None # if None, it takes current ouput and model. Else specify outDir and runName (e.g: 'outDir/1_2') TODO change to default\n",
    "\n",
    "if loadResultsFrom:\n",
    "    model = load_model('{}.model'.format(loadResultsFrom))\n",
    "    with open('{}history.p'.format(loadResultsFrom), 'rb') as file: # TODO change history\n",
    "        result = pickle.load(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Evaluate the Model on the current Test Set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "This will return the current test loss and accuracy:\n",
      "31/31 [==============================] - 103s 3s/step\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[0.779028594493866, 0.7210190892219543]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "print('This will return the current test loss and accuracy:') # TODO run again\n",
    "model.evaluate_generator(val, verbose=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create Training Plots\n",
    "The plots are automatically saved with its timestamp as name to the plot directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEWCAYAAACEz/viAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXxcdb3/8denSdokbbY26Za0TfcV2tJSdmS3uFAQVHDFi6L+RLjq9V68Iiro/alXrl4F/VkVQa9eNqUUrCBCEWSzLbR0L2lpaZImTbM1adNm+/z+mJM6lKSdtjmZzMz7+XjkkTlnzsx8DofOO+d7vuf7NXdHRERS24B4FyAiIvGnMBAREYWBiIgoDEREBIWBiIigMBARERQGIsfNzNLMrNnMxvbmtiLxYLrPQFKFmTVHLWYDB4GOYPnT7v7bvq9KpH9QGEhKMrPtwCfd/S9H2Cbd3dv7riqR+FEzkUjAzL5lZveb2f+aWRPwETM7w8xeMrMGM9tlZj8ys4xg+3QzczMrDZb/J3j+T2bWZGYvmtn4Y902eP5SM9tiZo1m9mMze97Mru3b/yKSShQGIm91BfA7IA+4H2gHbgIKgbOAhcCnj/D6DwFfA4YCbwK3H+u2ZjYceAD4cvC5bwALjneHRGKhMBB5q7+5+6Pu3unuLe6+wt1fdvd2d98GLAbecYTXP+TuK929DfgtMOc4tn0PsNrdHwme+wGw58R3TaRn6fEuQKSf2Rm9YGbTgDuAeUQuOqcDLx/h9VVRj/cDQ45j29HRdbi7m1n5USsXOQE6MxB5q8N7VPwMWAdMcvdc4FbAQq5hF1DStWBmBhSH/JmS4hQGIkeWAzQC+8xsOke+XtBbHgNOMbP3mlk6kWsWRX3wuZLCFAYiR/Yl4ONAE5GzhPvD/kB3rwY+CPwXUAtMBF4lcl+ESCh0n4FIP2dmaUAlcJW7PxfveiQ56cxApB8ys4Vmlmdmg4h0P20H/h7nsiSJKQxE+qezgW1EupQuBC53dzUTSWjUTCQiIjozEBGRBLzprLCw0EtLS+NdhohIQlm1atUed++xi3LChUFpaSkrV66MdxkiIgnFzHYc6Xk1E4mIiMJAREQUBiIigsJARERQGIiICAoDERFBYSAiIigMRET6ve179vFfT25hS3VTaJ+RcDediYikgvp9rTz2WiV/eLWCV99swAyKcgYxZUROKJ+nMBAR6ScOtnfw9Mbd/OHVCp7ZvJu2DmfqiBxuvnQai+aMZlReVmifrTAQEYkjd2fljnr+8EoFf3ytkr0H2inKGcTHzyjlilOKmTEql8g02OFSGIiI9KKddft5cVste1vaONjeSWt7J20dkd+tHZHH0es37mrizbr9ZGWk8c6ZI7jilBLOmjiM9LS+vaSrMBCRlFS/r5UV2+tYvbOB4TmDmFWcx/RRuQwedGxfi/sOtvPStlqe3VLDs6/v4Y09+962TUaakZE2gIHpAyK/g8cD0wYwvnAwN104mXfOGsmQY/zs3qQwEJGUsLvpAH9/o46/v1HHy9vq2Bz0zBlg0BnM8WUGE4uGMGt0LrOK85hVnMeM0bnkZmYceh93Z8OuvTy7ZQ/Pbqlh5Y462jqcrIw0Tp8wlI+dMY5zJhcyIjfz0Bf/gAHhN/OcKIWBiCSlioYW/v5GLS9viwTAtuAv9uyBacwbV8Blc0azYPxQTi7Jo2F/G2vLG1lX2ci6ir28tK2OJasrD73X+MLBzBydS0baAJ57fQ97miMzkE4bmcM/nTWec6cUMb+0gEHpaXHZ194QahiY2ULgv4E04Bfu/p3Dnh8L3AvkB9vc7O7LwqxJRJLXrsYWHlldyZJXK9hUFfnLPycznQWlQ7l6wRgWjB926Es92ojcNEbMyOSiGSMOratpOsj6ykbWVUQCYvXOBlpaOzhzUiHnTi7k3ClFjMjN7NP9C1NocyCbWRqwBbgYKAdWANe4+4aobRYDr7r7T81sBrDM3UuP9L7z5893TW4jIl32Hmjj8bVVPPxqBS+9UYs7zB2bz7tPGsUZE4cxbWQuaQnQTBM2M1vl7vN7ej7MM4MFQJm7bwsKuQ9YBGyI2saB3OBxHlCJiMhRtLZ38tctNSx5tYInN1bT2t7J+MLB/POFU1g0ZzSlhYPjXWLCCTMMioGdUcvlwGmHbfMN4M9m9nlgMHBRiPWISILq7HR2Nx1ka00zy9bu4o9rd9Gwv41hgwfyoQVjuXxuMbNL8vqkP36yCjMMujsqh7dJXQPc4+53mNkZwG/MbJa7d77ljcyuB64HGDt2bCjFikj8uDv1+9vYWbefnfX7Ka9vCR63UF63n/KGFlrbI18LmRkDuGTGSK6YW8zZkwvf1v4vxyfMMCgHxkQtl/D2ZqDrgIUA7v6imWUChcDu6I3cfTGwGCLXDMIqWETCV7+vlU1VTWyq2sumXZHfW2v20Xyw/S3b5WdnMKYgm6kjc7hoxgjGFGRRMjSbU0uHxrU/frIK87/oCmCymY0HKoCrgQ8dts2bwIXAPWY2HcgEakKsSUT6SGt7J9v2NLNpVxMbgy/+zVVNVO09cGiboYMHMn1UDlfNK6GkIIsxQ7MZU5BNydCst/Ttl/CFFgbu3m5mNwBPEOk2ere7rzez24CV7r4U+BLwczP7ApEmpGs9rO5NIhK6vQfaWL5pN4+vq+KZzTW0tHUAkTtwJw3P4cyJw5g2KodpI3OZNiqHoiGD1M7fT4R6rhXcM7DssHW3Rj3eAJwVZg0iEq7a5oM8uaGax9dX8XzZHto6nOE5g7hyXjGnlg5l2shcJhQNVtt+P6eGNxE5ZpUNLTyxvorH11WxYnsdnQ5jhmbxibPG886ZI5k7Jj8hhmCQf1AYiMhRVe89wOqdDbxW3sDfympZs7MBgKkjcrjhgsksnDmS6aNy1OSTwBQGIvIWjS2RcXrWlDewZmcDa8obqN4bGYsnfYAxqziPf104lYUzRzKhaEicq5XeojAQSSEH2jrY29JGQ0sbjS1tNOyP/K7bd5CNu5pYs7Ph0IBuABMKB3PmxEJOLslj9ph8ZozKJTMjcQdjk54pDESSiLuzo3Y/q3bUs+rNesp2N9O4v42GllYaW9o40NbZ42uH5wxi9ph8rpxXwuySfE4qySMvS907U4XCQCSBHWjrYG1FY+TLf0c9r+yop3ZfKwA5g9KZPiqX0sJs8rPyycvOIC8r8pMfPM7PGhhZFyxL6lIYiPRjHZ3O3q4mnZZ/NOmsq9jLqh31rK9spK0jcmvO+MLBnDd1OPPGFTBvXAGThw9Rjx6JmcJAJA4aW9ooD8bgqahvoby+heq9B4Iv/dZD7flNB9q7ff2g9AHMHpPPJ8+ZwLyxBcwdm8+wIYP6eC8kmSgMRELS0ek8X7aHLdVNlAdf+OX1+6loaHnbl3xWRhqj8jLJz86gaMggJg/POdSk85ZmnewM8rIGMnZoNgPTdROX9B6FgUgvO9DWwUOryvnl3944NDn64IFplBRkU1KQxWnjh1JckHVouaQgm4LsDPXRl7hSGIj0ktrmg/z6xR385qUd1O1r5eSSPH58zVzOmVxIXpa+7KV/UxiInKBtNc384m9v8PtV5Rxs7+Si6cP51DkTWDB+qAJAEobCQOQ4uDsrd9Sz+Nlt/GVjNRlpA7jylGKuO3sCk4brrlxJPAoDkRi1tHawvrKR1TsbeOy1Xaze2UB+dgafP38SHz2jlKIc9eaRxKUwEOlGZ6eztaaZV3dGxudZvbOBTVVNdHRG+vRPKBrM7YtmctW8MWQN1PAMkvgUBiLA/tZ2Xtxay6od9cHonI2HpmHMGZTO7DH5fPYdE5k9Jp/ZY/IYnpMZ54pFepfCQFJWZUMLT23azdMbq3lhay0H2ztJH2BMH5XL5XNHM2dMAXPG5DOhcLDu5JWkpzCQlNHZ6bxW0chTG6t5auNuNuzaC8C4Ydl8+LRxXDg9MpSDRuWUVKQwkKR2oK2Dv26p4amN1Ty9qYY9zQcZYDB/3FC+cuk0Lpw+nIlFQ9QFVFKewkCSjruztqKR+1fsZOmaSpoOtJOTmc47phRx0fQRnDe1iPzsgfEuU6RfURhI0qjb18qSVyt4YOVONlU1MSh9AO86aRRXnlLCaROGakJ2kSNQGEhC6+h0nnu9hgdXlvPkhmpaOzo5uSSPb10+i/fOHq0x+kVipDCQhOPubK/dz8OvlPPQqnIqGw9QkJ3BR04fx/vnlzB9VG68SxRJOAoD6fc6Op1NVXtZub2eFdvrWLm9nqq9BzCDcycXcct7ZnDh9OEMSlcvIJHjpTCQfudAWwerdzawcnsdK7ZHpnJsCm4AG5mbyanjh3JqaQEXTR/B6PysOFcrkhwUBtIvuDtL11Ry7wvbWVvxj6kcp4wYwmVzRnNq6VDmlxZQnJ+lbqAiIVAYSNzVNB3kliVreWJ9NVNH5HDd2RM4tTQyj6+6gIr0DYWBxNVjr1XytSXr2Nfawc2XTuNT50wgTUM/iPQ5hYHERW3zQW59ZD1/XLuL2SV5fP/9s5k8IifeZYmkLIWB9Lk/rd3FLUvWsfdAG19+51Q+fe4E0nVDmEhcKQykz9Tva+XWpet5dE0ls4pz+d37T2fqSJ0NiPQHCgPpE0+sr+KrD6+jsaWVL108hc+cN1HDQ4j0IwoDCc3upgO8uLWWZWt38cT6amaMyuU31y3QHcIi/ZDCQHpNY0sbL2+r5YWttTxftofXdzcDkJuZzj9fNJnPnT9JZwMi/VSoYWBmC4H/BtKAX7j7dw57/gfA+cFiNjDc3fPDrEl6z4G2DlZur+f5rXt4oWwPaysa6XTIzBjAqaVDed8pJZw1aRgzR+epu6hIPxdaGJhZGnAXcDFQDqwws6XuvqFrG3f/QtT2nwfmhlWP9J6D7R3c8ect3PPCdlqDqSLnjMnnhgsmc+bEYcwdm69xgkQSTJhnBguAMnffBmBm9wGLgA09bH8N8PUQ65FesLmqiZvue5VNVU1cNa+Ed588ilNLhzJkkFocRRJZmP+Ci4GdUcvlwGndbWhm44DxwNM9PH89cD3A2LFje7dKiUlnp3PPC9v5zuObyM1M51fXnsr504bHuywR6SVhhkF3jcTew7ZXAw+5e0d3T7r7YmAxwPz583t6DwlJ9d4D/MuDa3ju9T1cNH0437nyZAqHDIp3WSLSi8IMg3JgTNRyCVDZw7ZXA58LsRY5To+v28XNf1jLgbYOvn3FLD60YKxGDRVJQmGGwQpgspmNByqIfOF/6PCNzGwqUAC8GGItcoyaD7Zz26PreWBlOScV5/HDq+cwsWhIvMsSkZCEFgbu3m5mNwBPEOlaere7rzez24CV7r402PQa4D53V/NPP/HKm/V84f7VvFm3n8+dP5GbLpzCwHTdHyCSzELtAuLuy4Blh6279bDlb4RZg8Sutb2TO5eXcdfyMkbmZnL/9WewYPzQeJclIn1A/QEFiJwN3Pz719hS3cwVc4v55qKZ5GZmxLssEekjCoMUt+9gO//5xGbufXE7I3Mz+cXH5nPRjBHxLktE+pjCIIUt37ybWx5eR0VDCx87YxxffudUcnQ2IJKSFAYpqLb5ILc/toElqyuZNHwID33mDOaX6tqASCpTGKQQd2fJ6gpue3QDzQfbufHCyXzu/IkaR0hEFAaporx+P199eB1/3VLDnDH5fPfKkzXLmIgcojBIcm0dndzz/HZ+8JctAHzjvTP46BmlGlJaRN5CYZDE/vb6Hr7x6HrKdjdzwbTh3LZoJiUF2fEuS0T6IYVBEqpoaOHbf9zAsrVVjB2azS8/Pp8Lp6u7qIj0TGGQRA60dfCL57Zx5/IyAL508RQ+de4EMjN0gVhEjkxhkCSe3lTNNx/dwI7a/Vw6ayRfffd0NQmJSMwUBgluR+0+bn9sA3/ZuJuJRYP5zXULOGdyUbzLEpEEozBIUO7OnU+X8ePlZWQMMP79XdO49szxGl1URI6LwiBB/eSZrdzx5BbefdIovvaeGYzMy4x3SSKSwBQGCejRNZX85xObWTRnND/84BzNPCYiJ0xtCglm1Y56vvTgGuaPK+C7V56sIBCRXqEwSCBv1u7n+l+vZFReJos/Nl9dRkWk1ygMEkTj/jY+cc/fae90fnXtqQwdPDDeJYlIElEYJIDW9k4++9tVvFm3n599dB4TNDG9iPQyXUDu59ydrz68lhe21nLH+2dz+oRh8S5JRJLQUc8MzOwGMyvoi2Lk7X7yzFYeXFXOjRdM4sp5JfEuR0SSVCzNRCOBFWb2gJktNHVf6TNdXUgvmz2aL1w8Jd7liEgSO2oYuPstwGTgl8C1wOtm9h9mNjHk2lJadBfS712lLqQiEq6YLiC7uwNVwU87UAA8ZGbfC7G2lBXdhfRnH52nLqQiErqjXkA2sxuBjwN7gF8AX3b3NjMbALwO/Gu4JaaW6C6kd197KsOGDIp3SSKSAmLpTVQIvM/dd0SvdPdOM3tPOGWlJnfn5j+8xpt1+/n1P53GRHUhFZE+Eksz0TKgrmvBzHLM7DQAd98YVmGp6OFXK/jTuiq+ePFUzpioLqQi0ndiCYOfAs1Ry/uCddKLKhpa+Poj65k/roDrz50Q73JEJMXEEgYWXEAGIs1D6Ga1XtXZ6Xz5wTV0uHPHB2aTNkA9h0Skb8USBtvM7EYzywh+bgK2hV1YKrnnhe28sLWWr71nBuOGDY53OSKSgmIJg88AZwIVQDlwGnB9mEWlkrLdTXz38U1cMG04V586Jt7liEiKOmpzj7vvBq7ug1pSTltHJ1+4fw3ZA9P4zpUn6cYyEYmbWO4zyASuA2YCh+ZWdPd/CrGulPDjp8tYW9HITz98CsNzNG2liMRPLM1EvyEyPtE7gb8CJUBTLG8ejGW02czKzOzmHrb5gJltMLP1Zva7WAtPdKt3NnDX8jLeN7eYS08aFe9yRCTFxdIraJK7v9/MFrn7vcEX9hNHe5GZpQF3ARcTudawwsyWuvuGqG0mA18BznL3ejMbfny7kVhaWjv44v2rGZ4ziK9fNjPe5YiIxHRm0Bb8bjCzWUAeUBrD6xYAZe6+zd1bgfuARYdt8yngLnevh0PXJ5Led/60kW179vH9988mLysj3uWIiMQUBouD+QxuAZYCG4DvxvC6YmBn1HJ5sC7aFGCKmT1vZi+Z2cIY3jehPfd6Dfe+uINPnFXKWZMK412OiAhwlGaiYDC6vcFf7s8Cx3JrbHddY/yw5XQiw2OfR+RaxHNmNsvdGw6r43qC7qxjx449hhL6l8b9bXz5wdeYWDSYf1s4Ld7liIgccsQzg+Bu4xuO873LgeiO8yVAZTfbPOLube7+BrCZSDgcXsdid5/v7vOLioqOs5z4u3XpOvY0H+QHH5yjYalFpF+JpZnoSTP7FzMbY2ZDu35ieN0KYLKZjTezgUTuVVh62DZLgPMBzKyQSLNRUt7d/NhrlTyyupLPXzCZk0vy412OiMhbxNKbqOt+gs9FrXOO0mTk7u1mdgORnkdpwN3uvt7MbgNWuvvS4LlLzGwD0EFkroTaY92J/q5uXyu3LFnH7DH5fO58TRAnIv1PLHcgjz/eN3f3ZUSGwI5ed2vUYwe+GPwkrf+7bCPNB9r5z6tOJj0tpsnlRET6VCx3IH+su/Xu/uveLyf5rNhex4OryvnMOyYyZUROvMsREelWLM1Ep0Y9zgQuBF4BFAZH0dbRyS0Pr6M4P4sbL5wU73JERHoUSzPR56OXzSyPyBAVchS/ev4NNlc3sfij88geqCkgRKT/Op4G7P100/1T3qqyoYUf/uV1Lpo+nEtmjox3OSIiRxTLNYNH+cfNYgOAGcADYRaVDL756Ho63fn6ezX2kIj0f7G0XXw/6nE7sMPdy0OqJyk8vamaJ9ZX868LpzJmaHa8yxEROapYwuBNYJe7HwAwsywzK3X37aFWlqBaWju49ZH1TBo+hE+erYntRSQxxHLN4EGgM2q5I1gn3bhreRnl9S3cvmgWA9N1T4GIJIZYvq3SgyGoAQgeDwyvpMRVtruZnz27lffNLeaMicPiXY6ISMxiCYMaM7usa8HMFgF7wispMbk7X1uyjqyMNP793dPjXY6IyDGJ5ZrBZ4DfmtmdwXI50O1dyals6ZpKXtxWy7cun0XhkEHxLkdE5JjEctPZVuB0MxsCmLvHNP9xKmlsaeP2xzYyuySPaxYk7nwLIpK6jtpMZGb/YWb57t7s7k1mVmBm3+qL4hLFHX/eTN2+g3z7ipNIG9DdnD4iIv1bLNcMLo2eeSyY9exd4ZWUWNaWN/Kbl3bwsTNKmVWcF+9yRESOSyxhkGZmhxrBzSwLUKM40NHpfHXJWgqHDOKLl0yJdzkiIsctlgvI/wM8ZWa/CpY/AdwbXkmJ44GVO3mtvJEfXTOX3MyMeJcjInLcYrmA/D0zew24iMgk948D48IurL9r6+jkzqfLmDs2n/eePCre5YiInJBYb5GtInIX8pVE5jPYGFpFCWLJqxVUNLRw4wWTMdNFYxFJbD2eGZjZFCKT2F8D1AL3E+laen4f1dZvdXQ6P3lmKzNH53Le1KJ4lyMicsKOdGawichZwHvd/Wx3/zGRcYlS3h/X7uKNPfv4/AWTdFYgIknhSGFwJZHmoeVm9nMzu5DINYOU1tnp3PV0GZOHD+GSGZq0RkSSQ49h4O4Pu/sHgWnAM8AXgBFm9lMzu6SP6ut3ntxYzebqJm64YBIDdIOZiCSJo15Advd97v5bd38PUAKsBm4OvbJ+yN258+kySodl8+6T1INIRJLHMQ247+517v4zd78grIL6s79uqWFtRSP/57xJpKdprgIRSR76RouRu/Pjp8sozs/i8rnF8S5HRKRXKQxi9NK2OlbtqOfT75igGcxEJOnoWy1Gdy5/naKcQXxg/ph4lyIi0usUBjFYtaOe58tquf6cCWRmpMW7HBGRXqcwiMFdy8soyM7gQ6dp4hoRSU4Kg6NYV9HI05t2c93Z4xk8KJZBXkVEEo/C4CjuWl5GTmY6HzuzNN6liIiERmFwBFuqm/jTuiquPbNU8xWISFJTGBzBT5aXkT0wjU+cNT7epYiIhEph0IPte/axdE0lHzl9HEMHD4x3OSIioQo1DMxsoZltNrMyM3vbeEZmdq2Z1ZjZ6uDnk2HWcyx++sxW0tMG8MlzdFYgIskvtO4xZpYG3AVcDJQDK8xsqbtvOGzT+939hrDqOB4VDS38/pVyPnzaWIbnZMa7HBGR0IV5ZrAAKHP3be7eCtwHLArx83rNz/66FTO4/h0T412KiEifCDMMioGdUcvlwbrDXWlmr5nZQ2bW7VgPZna9ma00s5U1NTVh1HpIe0cnD64sZ9GcYorzs0L9LBGR/iLMMOhu5hc/bPlRoNTdTwb+Atzb3Ru5+2J3n+/u84uKwp1zeHvtflraOjhjwrBQP0dEpD8JMwzKgei/9EuAyugN3L3W3Q8Giz8H5oVYT0w2Ve0FYOrInDhXIiLSd8IMgxXAZDMbb2YDgauBpdEbmFn0dGGXARtDrCcmm6uaSBtgTBo+JN6liIj0mdB6E7l7u5ndADwBpAF3u/t6M7sNWOnuS4EbzewyoB2oA64Nq55YbapqYnzhYI1OKiIpJdSR19x9GbDssHW3Rj3+CvCVMGs4VpurmjipJC/eZYiI9CndgRxl38F23qzbz7QRul4gIqlFYRBlc3UToIvHIpJ6FAZRNldFwmDayNw4VyIi0rcUBlE2VzUxeGAaJQW62UxEUovCIMqmqr1MGZnDgAHd3S8nIpK8FAYBd2dzVRPTdL1ARFKQwiBQ03SQ+v1tTFVPIhFJQQqDwMaqrp5EungsIqlHYRDYHIxJpGYiEUlFCoPApqomRuQOokBTXIpIClIYBDZXNamJSERSlsKAyIQ2r+9uVhORiKQshQGwvXYfre2d6kkkIilLYUDkegFoTCIRSV0KAzShjYiIwgBNaCMiojCgqyeRmohEJHWlfBg0a0IbERGFwRZNaCMiojDomtBm+ijdcCYiqUthEExoU5yvCW1EJHWlfBhoQhsRkRQPA3dnkya0ERFJ7TDY3XSQBk1oIyKS2mHQNQzFNF08FpEUl9JhoAltREQiUjoMuia0yc/WhDYiktpSOwx2aUIbERFI4TBo7+ikrEYT2oiIQAqHQdeENgoDEZEUDgNNaCMi8g8pGwaa0EZE5B9SNgw27opMaDMoXRPaiIiEGgZmttDMNptZmZndfITtrjIzN7P5YdYTbXP1XjURiYgEQgsDM0sD7gIuBWYA15jZjG62ywFuBF4Oq5bDNR9sZ2ddiya0EREJhHlmsAAoc/dt7t4K3Acs6ma724HvAQdCrOUtuia00TAUIiIRYYZBMbAzark8WHeImc0Fxrj7YyHW8TZdE9qoW6mISESYYdDdBAF+6EmzAcAPgC8d9Y3MrjezlWa2sqam5oQL27Rrrya0ERGJEmYYlANjopZLgMqo5RxgFvCMmW0HTgeWdncR2d0Xu/t8d59fVFR0woVtqmrShDYiIlHCDIMVwGQzG29mA4GrgaVdT7p7o7sXunupu5cCLwGXufvKEGvC3dlcrQltRESihRYG7t4O3AA8AWwEHnD39WZ2m5ldFtbnHk3XhDbTNECdiMgh6WG+ubsvA5Ydtu7WHrY9L8xaumgYChGRt0u5O5A37dKENiIih0u5MNisCW1ERN4m5cJgU5UmtBEROVxKhUHXhDbT1UQkIvIWKRUGXRPa6OKxiMhbpVQYqCeRiEj3UisMdmlCGxGR7qRWGFRpQhsRke6kVBhoQhsRke6lTBh0TWijnkQiIm+XMmHQNaGN7jEQEXm7lAmDTbs0oY2ISE9SJgwKhwzk4hkjNKGNiEg3Qh21tD+5ZOZILpk5Mt5liIj0SylzZiAiIj1TGIiIiMJAREQUBiIigsJARERQGIiICAoDERFBYSAiIoC5e7xrOCZmVgPsOM6XFwJ7erGc/iDZ9inZ9geSb5+SbX8g+fapu/0Z5+5FPb0g4cLgRJjZSnefH+86elOy7VOy7Q8k3z4l2/5A8u3T8eyPmolERERhICIiqRcGi+NdQAiSbZ+SbX8g+fYp2fYHkm+fjnl/UuqagYiIdC/VzgxERKQbCgMREUmdMDCzhWa22czKzOzmeNdzosxsu5mtNXm3Z3UAAAS2SURBVLPVZrYy3vUcDzO728x2m9m6qHVDzexJM3s9+F0QzxqPRQ/78w0zqwiO02oze1c8azxWZjbGzJab2UYzW29mNwXrE/I4HWF/EvY4mVmmmf3dzNYE+/TNYP14M3s5OEb3m9nAI75PKlwzMLM0YAtwMVAOrACucfcNcS3sBJjZdmC+uyfsjTJmdi7QDPza3WcF674H1Ln7d4LQLnD3f4tnnbHqYX++ATS7+/fjWdvxMrNRwCh3f8XMcoBVwOXAtSTgcTrC/nyABD1OZmbAYHdvNrMM4G/ATcAXgT+4+31m9v+ANe7+057eJ1XODBYAZe6+zd1bgfuARXGuKeW5+7NA3WGrFwH3Bo/vJfIPNSH0sD8Jzd13ufsrweMmYCNQTIIepyPsT8LyiOZgMSP4ceAC4KFg/VGPUaqEQTGwM2q5nAT/H4DIwf6zma0ys+vjXUwvGuHuuyDyDxcYHud6esMNZvZa0IyUEM0p3TGzUmAu8DJJcJwO2x9I4ONkZmlmthrYDTwJbAUa3L092OSo33mpEgbWzbpEbx87y91PAS4FPhc0UUj/81NgIjAH2AXcEd9yjo+ZDQF+D/yzu++Ndz0nqpv9Sejj5O4d7j4HKCHSEjK9u82O9B6pEgblwJio5RKgMk619Ap3rwx+7wYeJvI/QDKoDtp1u9p3d8e5nhPi7tXBP9RO4Ock4HEK2qF/D/zW3f8QrE7Y49Td/iTDcQJw9wbgGeB0IN/M0oOnjvqdlyphsAKYHFxdHwhcDSyNc03HzcwGBxe/MLPBwCXAuiO/KmEsBT4ePP448EgcazlhXV+YgStIsOMUXJz8JbDR3f8r6qmEPE497U8iHyczKzKz/OBxFnARkWshy4Grgs2OeoxSojcRQNBV7IdAGnC3u387ziUdNzObQORsACAd+F0i7o+Z/S9wHpHhdquBrwNLgAeAscCbwPvdPSEuyvawP+cRaXpwYDvw6a629kRgZmcDzwFrgc5g9b8TaWdPuON0hP25hgQ9TmZ2MpELxGlE/sB/wN1vC74n7gOGAq8CH3H3gz2+T6qEgYiI9CxVmolEROQIFAYiIqIwEBERhYGIiKAwEBERFAYib2NmHVGjV67uzVFuzaw0elRTkf4i/eibiKScluDWfpGUoTMDkRgFc0h8Nxg7/u9mNilYP87MngoGOXvKzMYG60eY2cPBOPNrzOzM4K3SzOznwdjzfw7uGhWJK4WByNtlHdZM9MGo5/a6+wLgTiJ3tBM8/rW7nwz8FvhRsP5HwF/dfTZwCrA+WD8ZuMvdZwINwJUh74/IUekOZJHDmFmzuw/pZv124AJ33xYMdlbl7sPMbA+RCVPagvW73L3QzGqAkughAIJhk59098nB8r8BGe7+rfD3TKRnOjMQOTbew+OetulO9PgwHejanfQDCgORY/PBqN8vBo9fIDISLsCHiUw7CPAU8Fk4NPlIbl8VKXKs9BeJyNtlBbNGdXnc3bu6lw4ys5eJ/CF1TbDuRuBuM/syUAN8Ilh/E7DYzK4jcgbwWSITp4j0O7pmIBKj4JrBfHffE+9aRHqbmolERERnBiIiojMDERFBYSAiIigMREQEhYGIiKAwEBER4P8DRw6a3pvBYT0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAEWCAYAAACEz/viAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deXzU5bX48c/JHsISSAIBEiBA2EHAsMjmgguodasb1i7Wpe3VWrW1am9re221rdraeuuvXndtVcAdKxVcECEoECiyQwIJECAbgSzAZJvz+2MmGGCSTJbJzGTO+/XKi5nv+nwZmJNnO4+oKsYYY0JbmL8LYIwxxv8sGBhjjLFgYIwxxoKBMcYYLBgYY4zBgoExxhgsGBjjMyISLSKVItLP32UxpjkWDEzIcX9B1/84ReR4g/ffasN1vxSRG+vfq2qVqnZV1QPtU3JjfCfC3wUwpqOpatf61yKSB9yiqh/7r0TG+J/VDIw5hYiEi8ivRGS3iJSIyKsiEu/eFyci80WkVESOiMhqEekpIn8CJgHPuWsYfxKRGBFREUlxnztfRP4iIktEpEJEMkVkYIP7XiIi2e7r/uXUmoYxvmTBwJjT3QtcCMwAUoAa4An3vltw1aj7A4nAHUC1qv4UWIurltHV/d6TG4AHgF7AQeB/AEQkGVgA3A0kAQeAM9v9yYxphAUDY073A+B+VT2gqg5cX9jXiYjgCgxJwBBVrVXVtap6tAXXXqiq61W1BngNGO/efhmwVlX/5d73OHC43Z7ImGZYn4ExDbi/8FOBxSLSMItjGJAAPA8kA2+KSFfgFeBXqlrn5S0KGrw+BtT3X/QD9tXvUFWniOxv3VMY03JWMzCmAXWl8d0PnKeq8Q1+YlS1xD1C6EFVHQHMAq4Brq8/vQ23PoirSQoAEQnD1RRlTIewYGDM6Z4G/iAiqQAi0ltEvuF+fb6IjHJ/WZcDtUB9raAQGNzKey4CpojIxSISAdwD9GzLQxjTEhYMjDndo8DHwKciUgGsAia69/UH3gMqgM3AYmChe98TwHdE5LCIPNqSG6rqQWAe8CRQgquWsAmoatujGOMdscVtjAk87tpBAfANVf3C3+UxnZ/VDIwJECIyV0R6iEgM8GtcHczr/FwsEyIsGBgTOGYBuUARMBu4UlWr/VskEyqsmcgYY4zVDIwxxgThpLPExEQdNGiQv4thjDFBZd26dSWqmtTY/qALBoMGDSIrK8vfxTDGmKAiInua2m/NRMYYYywYGGOMsWBgjDEGCwbGGGOwYGCMMQYLBsYYY7BgYIwxBgsGxhjTrpxOZcHavZRUBlf2cQsGxhjTjhZk7eO+tzbxt09z/F2UFrFgYIwx7aSw3MEji7cB8O6G/VTVers0tv9ZMDDGhKzqWie3v7aeVbtK2nwtVeVX726mutbJby8fzZFjNXy8tagdStkxLBgYY0LW+r2H+WDjQf7r1fXsP3K8Tdf6cHMBS7cWcvcFw7hhykD69YhhQda+diqp71kwMMaErMycEsIEauuU/3p1faubdcqO1fDgoi2M7tedW2akER4mXH1mCiuyiznQxiDTUSwYGGNCVmZOCeNS4nn8mnF8te8Iv/vXtlZd5+HFWyk9Ws0fvzmOiHDX1+rVZ6aiCm+ty2/PIvuMBQNjTEiqcNTwVX4ZM4YmMmdMX26bNZh/fLmHd/+zv0XXycwpYWFWPrfOHMyY/j1ObB+Q0IVpQxJ4Y10+TmfbV5Q8Xu3bzmgLBsaYkLR6dyl1TmX60EQAfn7RcCan9eKBtzexo6DCq2scr67jgbc3kZYYx13np5+2/9qMVPaWHmN1bmmbylp6tJrpf/yUhWt91wdhwcAYE5JW5pQQExnGxIHxAESEh/G3eROIi47gR/9cR4WjptlrPPHxTvaWHuP3V40lJjL8tP1zxiTTLSaCN9rYkfz3z3I4cqya8QPi23SdplgwMMaEpMycEiYN6kV0xNdf4r27x/DUDRPYU3qM+97aiGrjzTsb84/w3IrdzJucytTBCR6PiYkM57Iz+rF480HKvQgunhw4cpyXv9jDVRNTGNanW6uu4Q0LBsaYkFNU7iC7qPJEE1FDUwYncN+c4SzeVMDzK3M9nl9T5+Tnb24ksWs0988d2eS9rs1IxVHj5P2vDrSqrE9+kg2Kx2ao9mTBwBgTcjLdk8xmeAgGALfOHMyc0cn8/t/bWZt3env/M5/vZntBBb+9Ygw9YiObvNe4lB4M79ONhVktH1WUU1TJwqx93Dh1ICk9u7T4/JawYGCMCTmZOYeI7xLJqL7dPe4XER69ZhypPWO5/dX1FFd8nXRuV3Elf/0km4vHJnPR6ORm7yUiXDspla/2HfG6Y7renz/aQWxkOLefO6RF57WGT4OBiMwRkR0ikiMi93vY/4SIbHD/7BSRI74sjzHGqCqZOSVMG5JAWJg0elz3mEj+fuOZlDtq+PHr66mtc+J0Kg+8tYmYiDB+c9lor+95xfh+RIZLizqSv9p3hMWbCrhl5mASukZ7fV5r+SwYiEg48BQwFxgFzBORUQ2PUdW7VXW8qo4H/hd421flMcYYgN0lRzlY5vDYX3CqkX2788iVY/lydymPL93Ja2v2siavlF9eMore3WK8vmdC12jOH9mHt/+zn+pap1fnPLZkB73iorhlZprX92kLX9YMJgM5qrpbVauB+cDlTRw/D3jdh+Uxps1UlSc+2sn1z3xBbZ13/6lNYFmV4+ovmD6k+WAAcNXEFG6YMoCnl+/idx9sZfrQBK7JSGnxfa/NSKX0aDWfbi9s9tiV2SWszCnh9nOH0i2m6T6J9uLLYNAfaFgnyndvO42IDATSgE8b2X+biGSJSFZxcXG7F9QYb6gqf/xwB3/9JJsvd5fy6fbgyUhpvrYyp4T+8bEMTPC+Q/bBS0cxLsU1u/iRK8ci0njzUmNmpifSp3t0sx3JqspjS7bTPz6Wb00Z0OL7tJYvg4Gnv63GBu1eD7ypqh7nW6vqM6qaoaoZSUlJ7VZAY7xVHwieXr6LeZMH0LtbNAt8OBvU+EadU/li1yFmDE1s0Rd6TGQ4r986laV3nc3AhLhW3TsiPIxvTkzhsx1FFJY7Gj1uyZYCvsov467z0z1OZPMVXwaDfCC1wfsUoLGBttdjTUQmQDUMBN+aMoCHrxjDNRkpLNtRxMGy4MhIaVw27y+j3FHLtKGeJ4k1JS46ggEtqE14cm1GKk6Ft9Z7rh3U1jl5bMkO0nt35aqJLW+KagtfBoO1QLqIpIlIFK4v/EWnHiQiw4GewBc+LIsxrXJqIPjt5WMIC5MT/6nfbMXYceM/K939BdO87C9ob4MS45ic1os3svI9zm5+e/1+dhUf5acXDie8iZFOvuCzYKCqtcAdwBJgG7BQVbeIyEMiclmDQ+cB87Wped/G+EFjgQBgYEIc04cmsCBrX7tkpDQdY9WuEkYkdyOpm++Hajbm2oxUckuOsjbv8EnbHTV1PPHxTs5Ijeei0X06vFw+nWegqotVdZiqDlHVh93bHlTVRQ2O+Y2qnjYHwRh/aioQ1Ltu0gDyDx8/MZvVBDZHTR1r8w57NaTUly4em0xcVDgLT5lz8M8v93CwzMF9c4a3qoO6rWwGsjGn8CYQAFw0ug/xXSKZv8Y6koPBuj2Hqa51NpqCoqN0iYrgG2f044ONB6msqgVcays8tSyHmemJfmvCsmBgOrWXMnM587cfcfNLa3l6+S7W7z1MTRPzA7wNBADREeFcNSGFpVsLOFRZ5fEYEzhW5pQQESZMTuvl76Jw7aRUjtfU8cFG15iaZ1fkcvhYDT+/aITfyhThtzsb42MllVU8vnQnfbpHk3voKJ+45wXERoYzcWA8kwb1YnJaLyak9iQ2KrxFgaDevMmpvJCZy9vr93PrrMGtLqvTqVTXOTt0KGGoycwpYcKAeOKi/f+1NyE1nqG9u7IwK5/ZI/vw3IrdXDKuL2NTejR/so/4/2/FGB/5y8c7cdTU8ex3Mhic1JXiiirW5pWyJtf189dPslGFyHBhXEo8CXFRLN1a6HUgAEjv040zB/bk9bV7uWVmWqvben/3wTb+uXoPl5/Rj5umpzGqn+cEaqZ1yo7VsGl/GXee59s00N4SEa7NSOGRxdu5942vqKp18tMLhvm1TBYMTKeUU1TB62v2ceOUAQxO6gpAUrdoLh7bl4vH9gWg7HgN6/ccZnVuKWtyD/F5djHfmzaIBy8d5VUgqHfdpFR+/uZGsvYcZtKgljdBbNh3hBdX5TK6X3f+tfEgb6zL56zBCdw8I43zRvRuUVmMZ1/sLkEVZqT7t7+goSsnpPDHD3ewbEcx8yannvh36i8WDEyn9Mji7XSJCucn5zf+21aP2EjOHdGbc0f0Blz9Ba35zf7ScX156P2tvL5mb4uDQW2dk1+8vYne3aJ5/dapOJ0wf+1eXl6Vxy2vZDEooQvfmzaIqzNS6dqG5o2aOieF5Q4OHHFw4MhxDpQdd/3pfl9SWcWD3xjNZWf0a/U9GlJVnl+Zy5Ckrpw9LMnvAS0z5xBxUeGMT/XdspEtldQtmtkjerN8ZzF3zvZ/jcWCgel0MnNK+HR7EQ/MHUGvuCivz2ttE0+XqAguG9+Pt9fn8+tvjG52sZOGXlqVx9aD5fz9WxNPJCT7wdlDuHlGGku2FPJCZi6/eX8rf1q6k+smpfLdaYNI7XXyLFhV5fCxGveXu+vnYJmD/Ue+/sIvqnBw6nSI+C6R9OsRS0rPLhyvqePRD7czZ3QyURFtH1eydGshv/tgGwBDkuK4ecZgrprY3299Ipk5JUxO60VkeGCNmXnkqrEUljvo2yPW30WxYGA6lzqn8vAH2+gfH8t3pw3qsPvOmzSA11bvZdGG/Xz7LO/uu//Icf780U5mj+jNnDEnL5ISER7GJeP6csm4vq5mpMxcXlqVxwuZucwe2YeeXSJP+sJ31Jw8QioqIoz+8bH07RHDjPRE+sXH0q9HjOvP+Fj6xcfQJerr//7LdhRx04treXt9PtdPbltyNKfTldl1cGIcd85O57mVu/nFO5t4fOkObpw6kG9PHdihk74OHDnO7pKj3NCBSd+8ldg1msQOWKvAGxYMTKfy9vp8th4s58l5Ezr0t9Ax/bszqm93Vz/F1IFe1TJ+/d4WVOF/Lh/d5PHjU+P56/UTeGDuSP7xZR4L1uYTJtAvPpaRyd05b3jvE1/y/eNj6RsfQ0JcVItqOucMS+KM1Hj+tiyHqyamtKl28O/NBWwvqOCv14/n8vH9uXx8P1bnlvLcit08+Uk2Ty/fxZXj+3PzzDSfLvBeL7M+ZbWf5xcEOgsGptM4Xl3H40t3cEZqPN8Y17dD7y0izJucyq/e28Lm/eXNDhFcsqWAj7cV8sDcEV6vbZvcI4Z7LxrBvT4Yiy4i3HV+eptrB3VO5S8f7yS9d1cuHdfvxLWnDk5g6uAEdhdX8kJmLm+uy2dB1j5mDUvi1plpLc4i2hKZOSUkdo1ieAcEnmAWWA1oxrTBsyt2U1hexa8uGemX6fyXje9PTGQYr6/d2+RxlVW1/GbRFkYkd+P7MzpmFStvNKwdeLsa16n+tfEA2UWV3HX+MI+J1gYndeV3V4xl1f2z+dmFw9h2sJxvP7+GS55cycrs9k/roapk7jrEtCGJfu/EDnQWDEynUFTu4Onlu5g7JpmMVgzvbA89YiO5eGxfFm04wLHq2kaPe+KjnRSUO3j4yrEB1aFZXzvIP3yctxtJsdyU2jonf/04mxHJ3Zg7pumF4nvFRXHHeemsvO9cHrt6HOWOGm58fjXff2ktOUUtWzS+KdlFlRRXVDG9FSmrQ03g/Es0pg2e+HgnNXVO7pvjv+n8APMmD6CyqpZ/bTzocf/m/WW8mJnLDZMHcObAnh1cuua1pXbw3oYD7C45yt0XDPP6t/DoiHCuyUjl43vO5oG5I1ibW8pFf1nBg+9tpvRodWse4ST1tQ3rL2ieBQMT9HYUVLBg7T6+PXUQgxJbtwpVe8kY2JMhSXHMX3N6U1GdU/nFO5voFRfNz/0ctBrT2tpBTZ2TJz/NZnS/7lw4quXpl2Miw/nB2UP47N5zuGHyAF5dvZezH1vGM5/voqrW4wKIXlm1q4SBCV287pcJZRYMTEBw1NTx9PJd3L1gAxvzj7To3IcXb6NrdAR3zh7qo9J5T0S4ftIA1u89ws7Ck5s7/vnlHjbml/GrS0e2aC5CR2tN7eDt9fnsOXSMey4Y1qb+moSu0fz2ijF8+JOZZAzsySOLt3P+n5ezeNNBj4vBNKW2zsmXu0utVuAlCwbGr+qcysKsfZz7+Gf84d/bWbKlgMv+lsntr60nt+Ros+cv31nM5+4ZnPFdvJ9g5ktXTexPZLiclNq6sNzBY0t2MDM9sd1m+fpKS2sH1bVOnvwkhzNS4znPPZu7rdL7dOPFmybzyvcn0yUygv96dT3XPP0FG/Z5/4vCV/llVFbV+j1ldbCwYGD8QlVZtr2Ii/+6gp+/uZHe3aKZf9tUVv9iNneeN5Rl24u44M/L+e93NlHUyOLhdU7l94u3MaBXF7591sAOfoLGJXSN5sJRybz9n/wTTRwPvb+Vmjonv7tijF9GOrXUOcOSOCOlh1e1gzfW7WP/keNtrhV4MmtYEh/cOYPfXzWWvENHueKpTL713Je88598jlc33XyUmVOCCJw12DqPvWHBwHS4jflHuOHZ1dz00loctXU8dcNE3r19OlMHJ9AtJpJ7LhzO8nvP5YYpA1iwdh9nP/YZjy3ZTrmj5qTrvJG1j+0FFdw3ZwTREYGV+vm6SakcOVbDki2FLNtexAebDvLj84YyMMG/fRrectUOhjVbO3DU1PG3T3M4c2BPZvkoCVxEeBjzJg9g2c/O4acXDGPPoWPcveArJj38Mfe9uZGsvFKPTUgrc0oY3a87PVuQkiSUSbAtPZyRkaFZWVn+LoZphb2HjvHY0h28/9UBesVF8ZPZ6cybPKDJ2a55JUf500c7ef+rA8R3ieSOc4dy49SB1DmVcx7/jNSesbz1o2kB99u206nMemwZyd1jKCh3EBMZzuI7Z7ZL3p+Ooqpc8VQmh45W8+lPz/FY9pdX5fHrRVt49ZYpHdY273Qqa/JKeXNdPos3HeRYdR2DErpw9ZkpXDkxhf7xsRyrruWM/1nK96en8cDFIzukXIFORNapakaj+y0YdH45RRUs217cpnz7bVF6tJonP8nm1dV7iAgL45aZadw2a/CJxGze2Ly/jD9+uJ0V2SX06xHDqH49+HhbIW/9aFpADtEE+N9PsvnTRzsBWHDbVKYEYXPFsu1F3PTSWv5w1djTZiU7auqY9egyBiXGseC2qX75t3W0qpZ/by7gzXX7+HJ3KSIwbUgCI5O789zKXF75/mRmDUvq8HIFouaCgaWjCAEP/Wsbn+8s5uzhSR2SC6aho1W1zPnL55RUVnHdpFTuOn8YfbrHtPg6Y/r34B83T2FVTgl/+HA7H28r5JJxfQM2EABck5HKk59mc+WE/kEZCADOGf5138GpOYteXb2Xoooqnpw3wW81s7joCK4+M4Wrz0xhX+kx3lyXz1vr88nMOURURFir1pcIVRYMOrnswgo+31kMwOJNBzs8GLy9Pp+iiqp2a0aYNjSR926fzpe7SxnTP7BXA0vuEcOSu2YF9Rj3+r6Dm146OWfRsepa/v5ZDtOHunIOBYLUXl24+4Jh/GR2OqtzXf0IsVGB1ZcUyIKnAdO0yguZeURHhDEiuRsfbi7o0Hs7ncqLq/I4I6UH04a03xeGiHDWkIQWNTP5y+CkrkHVT+BJw9pB/ciif3yxh5LKau7x81KNnoSFuf59TLMhpS0S3P9KTZNKj1bz9vp8rpzQn2szUtleUMHu4soOu/+KnBJ2Fx/le9MHBVwHr/HeqSOLKqtqeXr5Ls4elsSZA60ZprOwYNCJvbZ6D1W1Tr4/I+3E4in/7sDawYuZuSR1i+aSsYE9yco0r2Ht4LkVuzl8rIa7A7BWYFrPgkEnVV3r5JUv9jAzPZFhfbrRLz6W8anxHdZUtKu4ks92FHPjlIFB30xiTq4d/PWTbGaP6B1Q6wmbtrP/pZ3UvzYeoKiiipsb5MufOyaZTfvL2Fd6zOf3f2VVHlHhYQG51KBpnfragSpWK+iELBh0QqrK8ytzGdq7K2c3GGM9d4xr9S9f1w7KHTW8uS6fS8/o26Fr3RrfEhEeu+YM/nztGYzp3/RKbib4+DQYiMgcEdkhIjkicn8jx1wrIltFZIuIvObL8oSKNbmlbDlQzvennzzJbEBCF0b3686/N3vOtd9eFq7dx9HqOm6aFjireJn2MaxPN66amOLvYhgf8FkwEJFw4ClgLjAKmCcio045Jh14AJiuqqOBu3xVnlDy/MpcenaJ5KqJ/U/bN3dMMuv3HqGgzHPyt7aqcyqvfLGHjIE9m10H2BgTOHxZM5gM5KjqblWtBuYDl59yzK3AU6p6GEBVi3xYnpCw59BRPtpWyA1TBhATefqEm7lj65uKfFM7+HR7EXtLj3HTdKsVGBNMfBkM+gP7GrzPd29raBgwTEQyReRLEZnj6UIicpuIZIlIVnFxsY+K2zm8mJlHRJjwnbMGedw/JKkrw/p09dkQ0xczc+nbI4aLRrd8tStjjP/4Mhh4mmV0ala8CCAdOAeYBzwnIqeNV1PVZ1Q1Q1UzkpIs6VRjyh01vJG1j0vH9Wsy/8+cMX1Zk1dKcUVVu95/R0EFq3Yd4ttnDSQigBZ6N8Y0z5f/Y/OB1AbvU4ADHo55T1VrVDUX2IErOJhWWLDG1XHbcDipJxePTUYVlm5t39rBS6tyiYkMY94kG05qTLDxZTBYC6SLSJqIRAHXA4tOOeZd4FwAEUnE1Wy024dl6rRq65y8tCqPyWm9mh32N7xPN9IS49p1iOnho9W885/9XDmhvy0mYkwQ8lkwUNVa4A5gCbANWKiqW0TkIRG5zH3YEuCQiGwFlgH3quohX5WpM1u6tZD9R443WysA13jxOWOSWbXrEIePVrfL/eev3Yejxsl3pw1ql+sZYzqWTxt2VXWxqg5T1SGq+rB724Oqusj9WlX1HlUdpapjVXW+L8vTmT2/MpcBvbpw/kjvOm4vHtOXOqfy0bbCNt+7ts7JP77IY9qQBEYkB3ZaaWOMZ9bL1wls2HeEdXsO871pgwgP8y476Jj+3UnpGdsuTUVLtxZyoMxhw0mNCWIWDDqB51fm0i06gmsnpTZ/sJuIMGd0Miuyi09baL6lXszMJbVXLOeN6N2m6xhj/MeCQZA7cOQ4izcd5LpJqXSNbtnCdXPHJlNTp3y6rfVz/TbvL2Nt3mG+e5b3tRJjTOCxYBDkXvliD6raqo7bCak96dM9uk25il7MzKNLVHiLaiXGmMBjwSCIHauu5fU1e5kzJpnUXi1fZzcszNVU9NmOYo5W1bb4/OKKKt7/6gBXn5lC9yBYgtIY0zgLBkHsrXX5lB2v8Wo4aWPmjOlLVa2Tz3a0PM3Ha6v3Ul1nw0mN6QwsGASpmjonL2S6FpufOKBnq68zOa0XCXFRLW4qqq518s/VezhneBJDkrq2+v7GmMBgwSBIvZSZR27JUe44L71Ni82HhwkXjk7m0+1FOGrqvDpHVfm/5bsorqjie1YrMKZTsGAQhA4cOc4TH+9k9ojenD+y7cM5545J5lh1HZ/vbL6pqKSyiltezuJPH+3k/JG9mZVuiQON6QxaNhbRBISH3t+KU5XfXDa6TbWCemcNSaBHbCQfbi7gwtHJjR73ybZC7ntrI+WOWn7zjVF856xBhNlwUmM6BQsGQWbZ9iI+3FLAvRcNb9UIIk8iw8O4YFQflmwpoLrWSVTEyRXG49V1PLx4K//8ci8j+3bntVvHM6xPt3a5tzEmMFgzURBx1NTx60VbGJIUx60zB7frteeOSabCUUvmrpKTtm/KL+OS/13Bq6v3ctuswbx7+zQLBMZ0QlYzCCJPLcthb+kxXrtlymm/vbfVjPREukZH8OGmAs4d3ps6p/L08l088dFOkrpF8+rNU5g2NLFd72mMCRwWDILEruJK/m/5bq4Y388nX8rREeHMHtmbpVsL+K9DQ7j3jY2sySvlknF9eeSKsfToYpPKjOnMLBgEAVXlwfc2Ex0Zxi8uGemz+8wdk8x7Gw5wwZ8/JyoijD9dcwZXTezfLp3UxpjAZsEgCCz66gCZOYd46PLR9O7W+NrGbXX2sN4kdo1mUEIXnrhufLt1UBtjAp8FgwBX7qjhdx9sY1xKD741ZaBP7xUbFc7K+84lOiLMagPGhBgLBgHuz0t3UlJZxfPfzeiQFNExkeE+v4cxJvDY0NIAtnl/Ga98kceNUwYyLiXe38UxxnRiFgwCVJ1T+e93NtErLpqfXTTc38UxxnRyzQYDEblDRFqfFtO0yutr9vJVfhm/vGQkPWJtWKcxxre8qRkkA2tFZKGIzBHrWfS54ooqHv1wO2cNTuDy8f38XRxjTAhoNhio6i+BdOB54HtAtog8IiJDfFy2kPX7xds4XlPHb69on0R0xhjTHK/6DFRVgQL3Ty3QE3hTRB71YdlC0tq8Ut7+z35unTmYob0tB5AxpmM0O7RURO4EvguUAM8B96pqjYiEAdnAz31bxNDy6fYiIsKEH5+X7u+iGGNCiDfzDBKBq1R1T8ONquoUkUt9U6zQVVjmoHe3aGKjbLy/MabjeNNMtBgorX8jIt1EZAqAqm7zVcFCVUG5gz49fJdywhhjPPEmGPwdqGzw/qh7m/GBgnIHyd0tGBhjOpY3wUDcHciAq3kIL9NYuIei7hCRHBG538P+74lIsYhscP/c4n3RO6fCMgd9LBgYYzqYN8Fgt4jcKSKR7p+fALubO0lEwoGngLnAKGCeiIzycOgCVR3v/nmuRaXvZCocNRytriPZmomMMR3Mm2DwQ2AasB/IB6YAt3lx3mQgR1V3q2o1MB+4vLUFDQWF5Q4AayYyxnS4Zpt7VLUIuL4V1+4P7Gvwvj6QnOqbIjIL2Ancrar7Tj1ARG7DHYAGDBjQiqIEh4KyKgBrJjLGdDhv5hnEADcDo4ET31Kq+v3mTvWwTU95/z7wuqpWicgPgZeB8047SfUZ4BmAjIyMU6/RaRTU1wysmcgY08G8abVNw/UAABOKSURBVCb6B678RBcBy4EUoMKL8/KB1AbvU4ADDQ9Q1UOqWuV++yxwphfX7bSsmcgY4y/eBIOhqvor4KiqvgxcAoz14ry1QLqIpIlIFK6mpkUNDxCRvg3eXgaE9LyFgjIH3WMibMKZMabDeTNEtMb95xERGYMrP9Gg5k5S1VoRuQNYAoQDL6jqFhF5CMhS1UXAnSJyGa58R6W4EuGFrIJyhzURGWP8wptg8Ix7PYNf4vrNvivwK28urqqLcc1gbrjtwQavHwAe8Lq0nVxhuc0xMMb4R5PBwJ2MrlxVDwOfA4M7pFQhqqDMwfA+lqnUGNPxmuwzcM82vqODyhLSauuclFRWWTORMcYvvOlA/khEfiYiqSLSq/7H5yULMcWVVTjV5hgYY/zDmz6D+vkEtzfYpliTUbsqKLNhpcYY//FmBnJaRxQk1BXahDNjjB95MwP5O562q+or7V+c0FVfM7BmImOMP3jTTDSpwesYYDawHrBg0I4KyquIDBcS4qL8XRRjTAjyppnoxw3fi0gPXCkqTDsqLHfQu1sMYWGeUjoZY4xveTOa6FTHAFutvZ0VlDno0z3a38UwxoQob/oM3ufrbKNhuBaqWejLQoWiwnIHI/rahDNjjH9402fweIPXtcAeVc33UXlCkqpSUO7g7OFJ/i6KMSZEeRMM9gIHVdUBICKxIjJIVfN8WrIQUlFVy7HqOptjYIzxG2/6DN4AnA3e17m3mXZSWGZzDIwx/uVNMIhwr2EMgPu1jX9sR/UrnNkcA2OMv3gTDIrdaw4AICKXAyW+K1LosVQUxhh/86bP4IfAqyLyN/f7fMDjrGTTOpaKwhjjb95MOtsFTBWRroCoqjfrH5sWKCh30CM2kphIW+7SGOMfzTYTicgjIhKvqpWqWiEiPUXkdx1RuFBRUFZlTUTGGL/yps9grqoeqX/jXvXsYt8VKfQUljvoY01Exhg/8iYYhIvIiTwJIhILWN6EdlRQ7iDZUlEYY/zImw7kfwKfiMiL7vc3AS/7rkihpaZ+uUtrJjLG+JE3HciPishG4HxAgA+Bgb4uWKgorqhCFWsmMsb4lbdZSwtwzUL+Jq71DLb5rEQhpn7CmdUMjDH+1GjNQESGAdcD84BDwAJcQ0vP7aCyhYRCW+HMGBMAmmom2g6sAL6hqjkAInJ3h5QqhBTYhDNjTABoqpnom7iah5aJyLMiMhtXn4FpRwXlDiLDhV5dLN2TMcZ/Gg0GqvqOql4HjAA+A+4G+ojI30Xkwg4qX6dXWGbLXRpj/K/ZDmRVPaqqr6rqpUAKsAG43+clCxEF5Q5rIjLG+F2L1kBW1VJV/T9VPc+b40VkjojsEJEcEWk0gIjI1SKiIpLRkvJ0BkXlNsfAGON/LQoGLSEi4cBTwFxc6ybPE5FRHo7rBtwJrPZVWQJV/XKXNpLIGONvPgsGwGQgR1V3uxfEmQ9c7uG43wKPAg4fliUgnVjusoelojDG+Jcvg0F/YF+D9/nubSeIyAQgVVX/1dSFROQ2EckSkazi4uL2L6mf2BwDY0yg8GUw8DQ8Rk/sFAkDngB+2tyFVPUZVc1Q1YykpKR2LKJ/2exjY0yg8GUwyAdSG7xPAQ40eN8NGAN8JiJ5wFRgUSh1Ip9Y7tJGExlj/MyXwWAtkC4iaSIShSu1xaL6napapqqJqjpIVQcBXwKXqWqWD8sUUOqXu7RmImOMv/ksGKhqLXAHsARXYruFqrpFRB4Skct8dd9gUlDuIL6LLXdpjPE/b9YzaDVVXQwsPmXbg40ce44vyxKIbLlLY0yg8GUzkWlGoc0xMMYECAsGfuRa7tKCgTHG/ywY+En9cpe2wpkxJhBYMPCT+uUurWZgjAkEFgz85OtFbSwVhTHG/ywY+ImlojDGBBILBn5iqSiMMYHEgoGfFJQ7iAoPo1ecLXdpjPE/CwZ+UljmoHf3aERsuUtjjP9ZMPATm2NgjAkkFgz8pLDc5hgYYwKHBQM/UFUKyqxmYIwJHBYM/KDcUcvxmjoLBsaYgGHBwA9OrGNgzUTGmABhwcAPTqxwZjUDY0yAsGDgBzbhzBgTaCwY+EF9Kore3S0vkTEmMFgw8IOCcgc9bblLY0wAsWDgB7bCmTEm0Fgw8IOCcgfJNpLIGBNALBj4QUFZlXUeG2MCigWDDlZT5+TQ0SprJjLGBBQLBh2sqH65S2smMsYEEAsGHcwmnBljApEFgw52IhWFBQNjTACxYNAGpUer+XhrYYvOOVEzsGYiY0wAsWDQBv9vWQ63vJLF5v1lXp9TWO4gKiKMnl0ifVgyY4xpGZ8GAxGZIyI7RCRHRO73sP+HIrJJRDaIyEoRGeXL8rS3FdklADy7YrfX5xSUO+hjy10aYwKMz4KBiIQDTwFzgVHAPA9f9q+p6lhVHQ88CvzZV+Vpb0XlDnYUVtArLop/bTzI/iPHvTrPFrUxxgQiX9YMJgM5qrpbVauB+cDlDQ9Q1fIGb+MA9WF52lV9reAPV40F4KXMXK/Os1QUxphA5Mtg0B/Y1+B9vnvbSUTkdhHZhatmcKenC4nIbSKSJSJZxcXFPilsS63MKSEhLorzR/bh4rF9mb9mHxWOmibPUVVXKgoLBsaYAOPLYOCpUfy03/xV9SlVHQLcB/zS04VU9RlVzVDVjKSkpHYuZss5ncqK7BJmpCcSFibcOjONiqpaFqzd1+R55cdrcdQ4bSSRMSbg+DIY5AOpDd6nAAeaOH4+cIUPy9NuthdUUFJZxYyhiQCMS4lnclovXszMo7bO2eh5BTbHwBgToHwZDNYC6SKSJiJRwPXAooYHiEh6g7eXANk+LE+7WZnjaqqamf51LeXWmYPZf+Q4izcXNHreiRXOrGZgjAkwPgsGqloL3AEsAbYBC1V1i4g8JCKXuQ+7Q0S2iMgG4B7gu74qT3takV3CsD5dT/pSnz2iN4MT43huxW5UPfeDF1oqCmNMgIrw5cVVdTGw+JRtDzZ4/RNf3t8XHDV1rMkt5VtTBp60PSxM+P6MNH757mbW5JYyZXDCaefW1wxsuUtjTKCxGcgttDavlKpaJzOHJZ6275sTU+jZJZJnV3geZlpQ7qBXXBTREbbcpTEmsFgwaKGV2SVEhYcxJa3Xaftio8L59tSBfLK9kF3FlaftLyyzOQbGmMBkwaCFPs8u4cyBPekS5bmF7dtnDSIyPIznV55eO3DNMbAmImNM4LFg0ALFFVVsO1jOjPTTm4jqJXWL5qoJ/XlrXT6HKqtO2ldoax8bYwKUBYMWyMxxpaCYld70xLdbZqZRVevkn1/uPbGtutZJSWW1NRMZYwKSBYMW+Dy7mJ5dIhndr3uTxw3t3Y1zhyfxjy/zcNTUAVBUYcNKjTGBy4KBl1SVldklTB/qSkHRnFtnDqakspp3/7MfaLDCmTUTGWMCkAUDL+0srKSooqrZJqJ6Zw1JYFTf7jy3MhenUykoc/UfWM3AGBOIQiYYHDn29W/prbEi25WCoqnO44ZEhFtnpZFTVMnyncVfp6KwYGCMCUAhEwxeyMzjrgUbWrREZUMrsksYkhRHv/hYr8+5dFw/krvH8OyK3SeWu4y35S6NMQEoZILBLTPTiO8SyR8/3N7ic6tq61ide+ikxHTeiAwP43vTB7Fq1yGW7ygmuXuMLXdpjAlIIRMMusdEcse5Q1mRXcJK9ypl3lqXdxhHjZOZXjYRNTRv8gDiosLZUVhhTUTGmIAVMsEA4NtnDaR/fCx//HA7Tqf3K2x+nl1CZLgw1UPyueb0iI3kukkDABtJZIwJXCEVDKIjwrnngmFs2l/GB5sOen3eypxiJgzoSVx065K83jR9EOFhQkpP7/sbjDGmI4VUMAC4YkJ/RiR34/GlO6iubXxVsnqHKqvYvL+cWa1oIqqX2qsLb/7wLG6dObjV1zDGGF8KuWAQHibcN2cEew4dY8Havc0en7nrEAAzWth5fKoJA3rSKy6qTdcwxhhfCblgAHDO8CSmpPXir59kc7SqtsljV+wspkdsJGP79+ig0hljTMcLyWAgItw/dwQlldU818hCNOBOQZFTwvShCYR7kYLCGGOCVUgGA3A128wdk8wzn++i5JRU0/V2FVdysMzR4vkFxhgTbEI2GAD87KLhOGqd/O3THI/7P9/pmo8wY2jrO4+NMSYYhHQwGJLUlWszUnl19R72Hjp22v6VOSWkJcaR2quLH0pnjDEdJ6SDAcBd56cTHib86aMdJ22vrnXy5e5DrZp1bIwxwSbkg0Gf7jHcPCON9zYcOCmJ3fq9hzlWXWdNRMaYkBDywQDgB2cPOS2J3YrsYsLDhLOGtDwFhTHGBBsLBnhOYrcyu4QJqfF0i7GU08aYzs+CgduNU79OYld6tJqN+8tsSKkxJmRYMHCLifw6id0v392EqvermhljTLCzYNBAfRK7xZsK6BYTwRkploLCGBMafBoMRGSOiOwQkRwRud/D/ntEZKuIbBSRT0RkoC/L05z6JHYA04ckEhFusdIYExp89m0nIuHAU8BcYBQwT0RGnXLYf4AMVR0HvAk86qvyeOuc4Un89IJh/OBsSzdtjAkdrVutxTuTgRxV3Q0gIvOBy4Gt9Qeo6rIGx38J3OjD8nhFRPjx7HR/F8MYYzqUL9tB+gP7GrzPd29rzM3Avz3tEJHbRCRLRLKKi4vbsYjGGGPAt8HAU85njwsPi8iNQAbwmKf9qvqMqmaoakZSkg33NMaY9ubLZqJ8ILXB+xTgwKkHicj5wH8DZ6uq51zSxhhjfMqXNYO1QLqIpIlIFHA9sKjhASIyAfg/4DJVLfJhWYwxxjTBZ8FAVWuBO4AlwDZgoapuEZGHROQy92GPAV2BN0Rkg4gsauRyxhhjfMiXzUSo6mJg8SnbHmzw+nxf3t8YY4x3bFaVMcYYCwbGGGNAVD2O9gxYIlIM7Gnl6YlASTsWJxB0tmfqbM8Dne+ZOtvzQOd7Jk/PM1BVGx2bH3TBoC1EJEtVM/xdjvbU2Z6psz0PdL5n6mzPA53vmVrzPNZMZIwxxoKBMcaY0AsGz/i7AD7Q2Z6psz0PdL5n6mzPA53vmVr8PCHVZ2CMMcazUKsZGGOM8cCCgTHGmNAJBs0twRlsRCRPRDa5czpl+bs8rSEiL4hIkYhsbrCtl4h8JCLZ7j97+rOMLdHI8/xGRPa7P6cNInKxP8vYUiKSKiLLRGSbiGwRkZ+4twfl59TE8wTt5yQiMSKyRkS+cj/T/7i3p4nIavdntMCdMLTx64RCn4F7Cc6dwAW4UmuvBeap6tYmTwxgIpKHa8nQoJ0oIyKzgErgFVUd4972KFCqqn9wB+2eqnqfP8vprUae5zdApao+7s+ytZaI9AX6qup6EekGrAOuAL5HEH5OTTzPtQTp5yQiAsSpaqWIRAIrgZ8A9wBvq+p8EXka+EpV/97YdUKlZnBiCU5VrQbql+A0fqSqnwOlp2y+HHjZ/fplXP9Rg0IjzxPUVPWgqq53v67AlYG4P0H6OTXxPEFLXSrdbyPdPwqch2ttefDiMwqVYNDSJTiDgQJLRWSdiNzm78K0oz6qehBc/3GB3n4uT3u4Q0Q2upuRgqI5xRMRGQRMAFbTCT6nU54HgvhzEpFwEdkAFAEfAbuAI+6lBMCL77xQCQZeL8EZRKar6kRgLnC7u4nCBJ6/A0OA8cBB4E/+LU7riEhX4C3gLlUt93d52srD8wT156Sqdao6HteKkpOBkZ4Oa+oaoRIMvFqCM5io6gH3n0XAO7j+AXQGhe523fr23aBeAU9VC93/UZ3AswTh5+Ruh34LeFVV33ZvDtrPydPzdIbPCUBVjwCfAVOBeBGpX7Om2e+8UAkGzS7BGUxEJM7d+YWIxAEXApubPitoLAK+6379XeA9P5alzeq/MN2uJMg+J3fn5PPANlX9c4NdQfk5NfY8wfw5iUiSiMS7X8cC5+PqC1kGXO0+rNnPKCRGEwG4h4r9BQgHXlDVh/1cpFYTkcG4agPgWq3utWB8HhF5HTgHV7rdQuDXwLvAQmAAsBe4RlWDolO2kec5B1fTgwJ5wA/q29qDgYjMAFYAmwCne/MvcLWzB93n1MTzzCNIPycRGYergzgc1y/4C1X1Iff3xHygF/Af4EZVrWr0OqESDIwxxjQuVJqJjDHGNMGCgTHGGAsGxhhjLBgYY4zBgoExxhgsGBhzGhGpa5C9ckN7ZrkVkUENs5oaEygimj/EmJBz3D2135iQYTUDY7zkXkPij+7c8WtEZKh7+0AR+cSd5OwTERng3t5HRN5x55n/SkSmuS8VLiLPunPPL3XPGjXGrywYGHO62FOaia5rsK9cVScDf8M1ox3361dUdRzwKvCke/uTwHJVPQOYCGxxb08HnlLV0cAR4Js+fh5jmmUzkI05hYhUqmpXD9vzgPNUdbc72VmBqiaISAmuBVNq3NsPqmqiiBQDKQ1TALjTJn+kqunu9/cBkar6O98/mTGNs5qBMS2jjbxu7BhPGuaHqcP67kwAsGBgTMtc1+DPL9yvV+HKhAvwLVzLDgJ8AvwITiw+0r2jCmlMS9lvJMacLta9alS9D1W1fnhptIisxvWL1Dz3tjuBF0TkXqAYuMm9/SfAMyJyM64awI9wLZxiTMCxPgNjvOTuM8hQ1RJ/l8WY9mbNRMYYY6xmYIwxxmoGxhhjsGBgjDEGCwbGGGOwYGCMMQYLBsYYY4D/DxgaQOOs7cO1AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plotDir = 'demoPlots/'\n",
    "if not os.path.exists(plotDir):\n",
    "    os.makedirs(plotDir)\n",
    "for plotType in ['accuracy', 'val_accuracy']:\n",
    "    plt.plot(result.history[plotType])\n",
    "    if plotType=='accuracy':\n",
    "        plt.title('Training')\n",
    "    if plotType=='val_accuracy':\n",
    "        plt.title('Testing')\n",
    "    plt.xlabel('Epoch')\n",
    "    plt.ylabel('Accuracy')\n",
    "    plt.savefig(f'{plotDir}{datetime.now()}.jpg')\n",
    "    plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python (myenv)",
   "language": "python",
   "name": "myenv"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}