From 64600e4abe54be3c85d47c16af80ccfd10ee5298 Mon Sep 17 00:00:00 2001 From: Marina Zhang <40069936+MarinaZhang@users.noreply.github.com> Date: Mon, 9 Oct 2023 15:32:10 -0700 Subject: [PATCH 1/4] Update README.md --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index d1bf6c3..60c6ac6 100644 --- a/README.md +++ b/README.md @@ -62,7 +62,7 @@ Please cite this reference if you use RETVec in your research: ```bibtex @article{retvec2023, - title={RetVec: Resilient and Efficient Text Vectorizer}, + title={RETVec: Resilient and Efficient Text Vectorizer}, author={Elie Bursztein, Marina Zhang, Owen Vallis, Xinyu Jia, and Alexey Kurakin}, year={2023}, eprint={2302.09207} From e25d7c54ad447b47d7d5808ed3d34316be3d5412 Mon Sep 17 00:00:00 2001 From: Marina Zhang <40069936+MarinaZhang@users.noreply.github.com> Date: Mon, 9 Oct 2023 15:33:20 -0700 Subject: [PATCH 2/4] Update CONTRIBUTING.md --- CONTRIBUTING.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index be7bb48..2895328 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,5 +1,5 @@ # How to Contribute -Thanks for considering contributing to TF similarity! +Thanks for considering contributing to RETVec! Here is what you need to know to make a successful contribution. There are just a few small guidelines you need to follow. @@ -31,7 +31,7 @@ pull request: - Ideally one PR corespond to one feature or improvement to make it easier to review. So **try** to split your contribution in meaning logical units. - Your code **must** pass the unit-tests. We use `pytest` so simply run it at the root of the project. -- Your code **must** passs static analyis. We use `mypy` so simply run `mypy tensorflow_similarity/` from the root of the project. +- Your code **must** passs static analyis. We use `mypy` so simply run `mypy retvec/` from the root of the project. - Your code **must** comes with unit-tests to ensure long term quality - Your functions **must** be documented except obvious ones using the Google style. - Your functions **must** be typed. From 23d28c0ce49f9bfbdb75b8fca46c310f7fa1c55b Mon Sep 17 00:00:00 2001 From: Marina Zhang <40069936+MarinaZhang@users.noreply.github.com> Date: Mon, 9 Oct 2023 16:23:17 -0700 Subject: [PATCH 3/4] Delete notebooks/train_hello_world_tf.ipynb --- notebooks/train_hello_world_tf.ipynb | 547 --------------------------- 1 file changed, 547 deletions(-) delete mode 100644 notebooks/train_hello_world_tf.ipynb diff --git a/notebooks/train_hello_world_tf.ipynb b/notebooks/train_hello_world_tf.ipynb deleted file mode 100644 index 89b2021..0000000 --- a/notebooks/train_hello_world_tf.ipynb +++ /dev/null @@ -1,547 +0,0 @@ -{ - "cells": [ - { - "attachments": {}, - "cell_type": "markdown", - "metadata": {}, - "source": [ - "# Using RetVec to train an emotion classifier\n", - "\n", - "RetVec is a state of art text tokenizer that works directly out of raw strings to create resilient models. Model trained with RetVec acheive state of art classification performance\n", - "and exhibit strong resilience to adversarial attacks as reported in our [paper](https://arxiv.org/abs/2302.09207).\n", - "\n", - "\n", - "RetVec speed, low , and stateless nature makes it the perfect choice to train and deploy\n", - "small and efficient on-device models. It is natively supported in TFLite via custom ops implemented in tensorflow-text for ondevice models and we provide a Javascript implementation RetVecJS that allows to deploy web models via TFJS.\n", - "\n", - "\n", - "This notebook demonstrates how to quickly train and use a text emotion classifier.\n", - "This classifier can then be easily exported to run in a webpage as demonstrate in \n", - "this notebook\n", - "\n", - "Let's get started" - ] - }, - { - "cell_type": "code", - "execution_count": 14, - "metadata": { - "tags": [ - "hide-output" - ] - }, - "outputs": [], - "source": [ - "# installing needed dependencies\n", - "try:\n", - " import retvec\n", - "except:\n", - " !pip install retvec # is retvec installed?\n", - "try:\n", - " import datasets\n", - "except:\n", - " !pip install datasets # used to get the dataset\n", - "\n", - "try:\n", - " import matplotlib\n", - "except:\n", - " !pip install matplotlib\n", - "\n" - ] - }, - { - "cell_type": "code", - "execution_count": 3, - "metadata": {}, - "outputs": [ - { - "name": "stderr", - "output_type": "stream", - "text": [ - "/Users/elieb/git/retvec/venv/lib/python3.10/site-packages/tqdm/auto.py:21: TqdmWarning: IProgress not found. Please update jupyter and ipywidgets. See https://ipywidgets.readthedocs.io/en/stable/user_install.html\n", - " from .autonotebook import tqdm as notebook_tqdm\n" - ] - } - ], - "source": [ - "import tensorflow as tf\n", - "import numpy as np\n", - "from tensorflow.keras import layers\n", - "from datasets import load_dataset\n", - "from matplotlib import pyplot as plt" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "In this notebook we are using the `RETVecTokenizer()` layer which perform the binarization and embedding in a single step. This is the best approach for GPU training. For TPU training,\n", - "it is more efficient to split the two steps -- see our TPU training notbook for this.\n" - ] - }, - { - "cell_type": "code", - "execution_count": 10, - "metadata": {}, - "outputs": [], - "source": [ - " # RetVec tokenizer layer.\n", - "from retvec.tf import RETVecTokenizer " - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Create dataset\n", - "\n", - "We are going to use the [Go Emotion dataset](https://huggingface.co/datasets/go_emotions) to create a mulit-class emotion classifier.\n", - "https://ai.googleblog.com/2021/10/goemotions-dataset-for-fine-grained.html" - ] - }, - { - "cell_type": "code", - "execution_count": 16, - "metadata": {}, - "outputs": [], - "source": [ - "# downloading\n", - "dataset = load_dataset('go_emotions')" - ] - }, - { - "cell_type": "code", - "execution_count": 17, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "num classes 28\n", - "['admiration', 'amusement', 'anger', 'annoyance', 'approval', 'caring', 'confusion', 'curiosity', 'desire', 'disappointment', 'disapproval', 'disgust', 'embarrassment', 'excitement', 'fear', 'gratitude', 'grief', 'joy', 'love', 'nervousness', 'optimism', 'pride', 'realization', 'relief', 'remorse', 'sadness', 'surprise', 'neutral']\n" - ] - } - ], - "source": [ - "# getting class name mapping and number of class\n", - "CLASSES = dataset['train'].features['labels'].feature.names\n", - "NUM_CLASSES = len(CLASSES)\n", - "print(f\"num classes {NUM_CLASSES}\")\n", - "print(CLASSES)" - ] - }, - { - "cell_type": "code", - "execution_count": 78, - "metadata": {}, - "outputs": [], - "source": [ - "# preparing data\n", - "x_train = tf.constant(dataset['train']['text'], dtype=tf.string)\n", - "\n", - "# the one-hot requires a little more due to the multi-class nature of the dataset.\n", - "y_train = np.zeros((len(x_train),NUM_CLASSES))\n", - "for idx, ex in enumerate(dataset['train']['labels']):\n", - " for val in ex:\n", - " y_train[idx][val] = 1\n", - "\n", - "# test data\n", - "x_test = tf.constant(dataset['test']['text'], dtype=tf.string)\n", - "y_test = np.zeros((len(x_test),NUM_CLASSES))\n", - "for idx, ex in enumerate(dataset['test']['labels']):\n", - " for val in ex:\n", - " y_test[idx][val] = 1\n" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## Model\n", - "\n", - "A key strength of RetVec is that input to the model are \n", - "the raw datasets string with no pre-processing which greatly simplify the training and inference process. In particular for on-device models. \n", - "\n", - "Notes:\n", - "- Using strings directly as input requires to use a shape of `(1,)` and specify the type `tf.string`\n", - "\n", - "- We are using `RetVecTokenizer()` in its default configuration which is to truncate at `128` words and use a small pretained model for the embedding. You can experiment with shorter or longer length by changing the `sequence_length`\n", - "parameter." - ] - }, - { - "cell_type": "code", - "execution_count": 128, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "WARNING:tensorflow:No training configuration found in save file, so the model was *not* compiled. Compile it manually.\n", - "Model: \"model_12\"\n", - "_________________________________________________________________\n", - " Layer (type) Output Shape Param # \n", - "=================================================================\n", - " token (InputLayer) [(None, 1)] 0 \n", - " \n", - " ret_vec_tokenizer_14 (RETV (None, 128, 256) 230144 \n", - " ecTokenizer) \n", - " \n", - " batch_normalization_14 (Ba (None, 128, 256) 1024 \n", - " tchNormalization) \n", - " \n", - " spatial_dropout1d_16 (Spat (None, 128, 256) 0 \n", - " ialDropout1D) \n", - " \n", - " bidirectional_26 (Bidirect (None, 128, 128) 164352 \n", - " ional) \n", - " \n", - " bidirectional_27 (Bidirect (None, 128, 64) 41216 \n", - " ional) \n", - " \n", - " bidirectional_28 (Bidirect (None, 64) 24832 \n", - " ional) \n", - " \n", - " dense_11 (Dense) (None, 28) 1820 \n", - " \n", - "=================================================================\n", - "Total params: 463388 (1.77 MB)\n", - "Trainable params: 232732 (909.11 KB)\n", - "Non-trainable params: 230656 (901.00 KB)\n", - "_________________________________________________________________\n" - ] - } - ], - "source": [ - "\n", - "\n", - "# Using strings directely requires to put a shape of (1, ) and a dtype: tf.string\n", - "inputs = layers.Input(shape=(1, ), name=\"token\", dtype=tf.string)\n", - "\n", - "# we are using RetVec with it's default settings\n", - "x = RETVecTokenizer(model='retvec-v1')(inputs)\n", - "\n", - "# Adding a batch norm after RetVec usually help with the convergence\n", - "x = layers.BatchNormalization()(x)\n", - "\n", - "# standard three LSTM layers\n", - "x = layers.SpatialDropout1D(0.1)(x)\n", - "x = layers.Bidirectional(layers.LSTM(64, return_sequences=True))(x)\n", - "x = layers.Bidirectional(layers.LSTM(32, return_sequences=True))(x)\n", - "x = layers.Bidirectional(layers.LSTM(32))(x)\n", - "outputs = layers.Dense(NUM_CLASSES, activation='sigmoid')(x)\n", - "model = tf.keras.Model(inputs, outputs)\n", - "model.summary()\n", - " " - ] - }, - { - "cell_type": "code", - "execution_count": 129, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "Epoch 1/50\n" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "2023-08-03 15:19:13.127544: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:19:14.165134: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:19:14.184875: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:19:14.439446: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:19:14.460276: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:19:14.691597: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:19:14.709305: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:19:15.189193: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:19:15.216765: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:19:15.649611: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:19:15.681321: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:19:16.115779: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:19:16.144417: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "340/340 [==============================] - ETA: 0s - loss: 0.1727 - acc: 0.2734" - ] - }, - { - "name": "stderr", - "output_type": "stream", - "text": [ - "2023-08-03 15:20:08.448031: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:20:08.890213: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:20:08.903641: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:20:09.091102: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:20:09.103586: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:20:09.291403: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n", - "2023-08-03 15:20:09.304346: I tensorflow/core/grappler/optimizers/custom_graph_optimizer_registry.cc:114] Plugin optimizer for device_type GPU is enabled.\n" - ] - }, - { - "name": "stdout", - "output_type": "stream", - "text": [ - "340/340 [==============================] - 67s 169ms/step - loss: 0.1727 - acc: 0.2734 - val_loss: 0.1479 - val_acc: 0.2959\n", - "Epoch 2/50\n", - "340/340 [==============================] - 49s 143ms/step - loss: 0.1490 - acc: 0.2954 - val_loss: 0.1464 - val_acc: 0.2959\n", - "Epoch 3/50\n", - "340/340 [==============================] - 49s 145ms/step - loss: 0.1455 - acc: 0.3104 - val_loss: 0.1394 - val_acc: 0.3455\n", - "Epoch 4/50\n", - "340/340 [==============================] - 48s 142ms/step - loss: 0.1381 - acc: 0.3532 - val_loss: 0.1330 - val_acc: 0.3650\n", - "Epoch 5/50\n", - "340/340 [==============================] - 49s 144ms/step - loss: 0.1339 - acc: 0.3682 - val_loss: 0.1295 - val_acc: 0.3932\n", - "Epoch 6/50\n", - "340/340 [==============================] - 49s 143ms/step - loss: 0.1309 - acc: 0.3921 - val_loss: 0.1261 - val_acc: 0.4166\n", - "Epoch 7/50\n", - "340/340 [==============================] - 49s 145ms/step - loss: 0.1279 - acc: 0.4106 - val_loss: 0.1238 - val_acc: 0.4229\n", - "Epoch 8/50\n", - "340/340 [==============================] - 48s 142ms/step - loss: 0.1251 - acc: 0.4246 - val_loss: 0.1224 - val_acc: 0.4295\n", - "Epoch 9/50\n", - "340/340 [==============================] - 48s 141ms/step - loss: 0.1225 - acc: 0.4344 - val_loss: 0.1199 - val_acc: 0.4420\n", - "Epoch 10/50\n", - "340/340 [==============================] - 48s 140ms/step - loss: 0.1206 - acc: 0.4425 - val_loss: 0.1165 - val_acc: 0.4570\n", - "Epoch 11/50\n", - "340/340 [==============================] - 48s 142ms/step - loss: 0.1186 - acc: 0.4501 - val_loss: 0.1152 - val_acc: 0.4603\n", - "Epoch 12/50\n", - "340/340 [==============================] - 49s 143ms/step - loss: 0.1165 - acc: 0.4589 - val_loss: 0.1134 - val_acc: 0.4638\n", - "Epoch 13/50\n", - "340/340 [==============================] - 49s 144ms/step - loss: 0.1148 - acc: 0.4635 - val_loss: 0.1121 - val_acc: 0.4702\n", - "Epoch 14/50\n", - "340/340 [==============================] - 48s 142ms/step - loss: 0.1133 - acc: 0.4661 - val_loss: 0.1111 - val_acc: 0.4721\n", - "Epoch 15/50\n", - "340/340 [==============================] - 49s 144ms/step - loss: 0.1119 - acc: 0.4718 - val_loss: 0.1099 - val_acc: 0.4732\n", - "Epoch 16/50\n", - "340/340 [==============================] - 48s 141ms/step - loss: 0.1102 - acc: 0.4749 - val_loss: 0.1086 - val_acc: 0.4736\n", - "Epoch 17/50\n", - "340/340 [==============================] - 50s 146ms/step - loss: 0.1090 - acc: 0.4800 - val_loss: 0.1081 - val_acc: 0.4818\n", - "Epoch 18/50\n", - "340/340 [==============================] - 48s 140ms/step - loss: 0.1078 - acc: 0.4879 - val_loss: 0.1071 - val_acc: 0.4885\n", - "Epoch 19/50\n", - "340/340 [==============================] - 49s 145ms/step - loss: 0.1065 - acc: 0.4920 - val_loss: 0.1058 - val_acc: 0.4909\n", - "Epoch 20/50\n", - "340/340 [==============================] - 48s 141ms/step - loss: 0.1053 - acc: 0.4952 - val_loss: 0.1046 - val_acc: 0.4947\n", - "Epoch 21/50\n", - "340/340 [==============================] - 48s 142ms/step - loss: 0.1044 - acc: 0.4985 - val_loss: 0.1044 - val_acc: 0.4971\n", - "Epoch 22/50\n", - "340/340 [==============================] - 48s 141ms/step - loss: 0.1031 - acc: 0.5039 - val_loss: 0.1031 - val_acc: 0.5041\n", - "Epoch 23/50\n", - "340/340 [==============================] - 48s 141ms/step - loss: 0.1022 - acc: 0.5052 - val_loss: 0.1021 - val_acc: 0.5030\n", - "Epoch 24/50\n", - "340/340 [==============================] - 48s 140ms/step - loss: 0.1015 - acc: 0.5103 - val_loss: 0.1009 - val_acc: 0.5111\n", - "Epoch 25/50\n", - "340/340 [==============================] - 48s 141ms/step - loss: 0.1007 - acc: 0.5118 - val_loss: 0.1014 - val_acc: 0.5100\n", - "Epoch 26/50\n", - "340/340 [==============================] - 48s 142ms/step - loss: 0.0997 - acc: 0.5181 - val_loss: 0.1003 - val_acc: 0.5143\n", - "Epoch 27/50\n", - "340/340 [==============================] - 47s 139ms/step - loss: 0.0992 - acc: 0.5204 - val_loss: 0.1000 - val_acc: 0.5172\n", - "Epoch 28/50\n", - "340/340 [==============================] - 48s 142ms/step - loss: 0.0987 - acc: 0.5217 - val_loss: 0.0999 - val_acc: 0.5159\n", - "Epoch 29/50\n", - "340/340 [==============================] - 48s 140ms/step - loss: 0.0980 - acc: 0.5221 - val_loss: 0.0990 - val_acc: 0.5139\n", - "Epoch 30/50\n", - "340/340 [==============================] - 48s 142ms/step - loss: 0.0972 - acc: 0.5269 - val_loss: 0.0990 - val_acc: 0.5189\n", - "Epoch 31/50\n", - "340/340 [==============================] - 48s 140ms/step - loss: 0.0968 - acc: 0.5276 - val_loss: 0.0984 - val_acc: 0.5193\n", - "Epoch 32/50\n", - "340/340 [==============================] - 48s 142ms/step - loss: 0.0965 - acc: 0.5293 - val_loss: 0.0985 - val_acc: 0.5215\n", - "Epoch 33/50\n", - "340/340 [==============================] - 48s 141ms/step - loss: 0.0958 - acc: 0.5310 - val_loss: 0.0980 - val_acc: 0.5235\n", - "Epoch 34/50\n", - "340/340 [==============================] - 48s 140ms/step - loss: 0.0951 - acc: 0.5359 - val_loss: 0.0986 - val_acc: 0.5145\n", - "Epoch 35/50\n", - "340/340 [==============================] - 48s 141ms/step - loss: 0.0949 - acc: 0.5360 - val_loss: 0.0980 - val_acc: 0.5264\n", - "Epoch 36/50\n", - "340/340 [==============================] - 47s 139ms/step - loss: 0.0944 - acc: 0.5359 - val_loss: 0.0977 - val_acc: 0.5220\n", - "Epoch 37/50\n", - "340/340 [==============================] - 48s 140ms/step - loss: 0.0939 - acc: 0.5412 - val_loss: 0.0971 - val_acc: 0.5259\n", - "Epoch 38/50\n", - "340/340 [==============================] - 48s 141ms/step - loss: 0.0936 - acc: 0.5390 - val_loss: 0.0984 - val_acc: 0.5202\n", - "Epoch 39/50\n", - "340/340 [==============================] - 48s 141ms/step - loss: 0.0930 - acc: 0.5431 - val_loss: 0.0982 - val_acc: 0.5266\n", - "Epoch 40/50\n", - "340/340 [==============================] - 47s 139ms/step - loss: 0.0928 - acc: 0.5421 - val_loss: 0.0975 - val_acc: 0.5250\n", - "Epoch 41/50\n", - "340/340 [==============================] - 48s 141ms/step - loss: 0.0921 - acc: 0.5433 - val_loss: 0.0977 - val_acc: 0.5205\n", - "Epoch 42/50\n", - "340/340 [==============================] - 48s 141ms/step - loss: 0.0919 - acc: 0.5469 - val_loss: 0.0972 - val_acc: 0.5233\n", - "Epoch 43/50\n", - "340/340 [==============================] - 47s 138ms/step - loss: 0.0915 - acc: 0.5479 - val_loss: 0.0967 - val_acc: 0.5275\n", - "Epoch 44/50\n", - "340/340 [==============================] - 47s 138ms/step - loss: 0.0913 - acc: 0.5472 - val_loss: 0.0969 - val_acc: 0.5264\n", - "Epoch 45/50\n", - "340/340 [==============================] - 47s 139ms/step - loss: 0.0911 - acc: 0.5499 - val_loss: 0.0975 - val_acc: 0.5279\n", - "Epoch 46/50\n", - "340/340 [==============================] - 48s 141ms/step - loss: 0.0906 - acc: 0.5497 - val_loss: 0.0966 - val_acc: 0.5270\n", - "Epoch 47/50\n", - "340/340 [==============================] - 993s 3s/step - loss: 0.0902 - acc: 0.5534 - val_loss: 0.0963 - val_acc: 0.5298\n", - "Epoch 48/50\n", - "340/340 [==============================] - 967s 3s/step - loss: 0.0897 - acc: 0.5556 - val_loss: 0.0962 - val_acc: 0.5310\n", - "Epoch 49/50\n", - "340/340 [==============================] - 995s 3s/step - loss: 0.0893 - acc: 0.5559 - val_loss: 0.0966 - val_acc: 0.5310\n", - "Epoch 50/50\n", - "340/340 [==============================] - 1021s 3s/step - loss: 0.0890 - acc: 0.5569 - val_loss: 0.0967 - val_acc: 0.5246\n" - ] - } - ], - "source": [ - "batch_size = 128\n", - "epochs = 50\n", - "model.compile('adam', 'binary_crossentropy', ['acc'])\n", - "history = model.fit(x_train, y_train, epochs=epochs, batch_size=batch_size, \n", - " validation_data=(x_test, y_test))" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "340/340 [==============================] - 69s 176ms/step - loss: 0.1677 - acc: 0.2777 - val_loss: 0.1479 - val_acc: 0.2959\n", - "Epoch 2/30\n", - "340/340 [==============================] - 51s 151ms/step - loss: 0.1492 - acc: 0.2954 - val_loss: 0.1471 - val_acc: 0.2959\n", - "Epoch 3/30\n", - "340/340 [==============================] - 52s 154ms/step - loss: 0.1453 - acc: 0.3122 - val_loss: 0.1368 - val_acc: 0.3586\n", - "Epoch 4/30\n", - "340/340 [==============================] - 56s 164ms/step - loss: 0.1360 - acc: 0.3648 - val_loss: 0.1306 - val_acc: 0.3833\n", - "Epoch 5/30\n", - "340/340 [==============================] - 51s 150ms/step - loss: 0.1301 - acc: 0.3988 - val_loss: 0.1246 - val_acc: 0.4150" - ] - }, - { - "cell_type": "code", - "execution_count": 130, - "metadata": {}, - "outputs": [], - "source": [ - "# saving the model so we can use it on the web\n", - "model.save('demo_models/emotions.keras') # new saving format requires .keras" - ] - }, - { - "cell_type": "code", - "execution_count": 131, - "metadata": {}, - "outputs": [ - { - "data": { - "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGzCAYAAAAMr0ziAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMiwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy8pXeV/AAAACXBIWXMAAA9hAAAPYQGoP6dpAABhp0lEQVR4nO3dd3hUZd7G8e/MpIc0SIcAIaGKgLSIiqJEsaHYFlxcioprXVl0XdEVF8vi67osFhTXXlBsqNhQjIKiCEjvJQZCgFRI7zPn/ePAQEwCSUgyk+T+XNdcM3NafnOIzp3nPM9zLIZhGIiIiIi4MaurCxARERE5GQUWERERcXsKLCIiIuL2FFhERETE7SmwiIiIiNtTYBERERG3p8AiIiIibk+BRURERNyeAouIiIi4PQUWERERcXsKLCJyQs8//zwWi4WEhARXlyIibZhF9xISkRM5++yzOXDgAHv27GHXrl3Ex8e7uiQRaYPUwiIitUpJSeHnn39m9uzZhIWFMX/+fFeXVKOioiJXlyAiTUyBRURqNX/+fEJCQrjsssu49tprawwsubm5/PWvf6Vr1654e3vTqVMnJkyYQHZ2tnOb0tJS/vnPf9KjRw98fHyIiori6quvJjk5GYClS5disVhYunRplWPv2bMHi8XC66+/7lw2adIk2rVrR3JyMpdeeikBAQGMHz8egB9//JHrrruOzp074+3tTUxMDH/9618pKSmpVvf27dv5wx/+QFhYGL6+vvTs2ZMHH3wQgO+//x6LxcLHH39cbb933nkHi8XCihUr6n0+RaThPFxdgIi4r/nz53P11Vfj5eXF9ddfzwsvvMDq1asZMmQIAIWFhQwfPpxt27Zx4403MnDgQLKzs1m0aBFpaWmEhoZit9u5/PLLSUpKYty4cdx9990UFBSwZMkSNm/eTFxcXL3rqqysZNSoUZxzzjk89dRT+Pn5AfDBBx9QXFzMbbfdRocOHVi1ahXPPvssaWlpfPDBB879N27cyPDhw/H09OSWW26ha9euJCcn89lnn/H4448zYsQIYmJimD9/PldddVW1cxIXF8ewYcNO4cyKSL0ZIiI1+PXXXw3AWLJkiWEYhuFwOIxOnToZd999t3ObGTNmGICxcOHCavs7HA7DMAzj1VdfNQBj9uzZtW7z/fffG4Dx/fffV1mfkpJiAMZrr73mXDZx4kQDMO6///5qxysuLq62bNasWYbFYjH27t3rXHbuuecaAQEBVZYdX49hGMb06dMNb29vIzc317ksMzPT8PDwMB5++OFqP0dEmpYuCYlIjebPn09ERATnn38+ABaLhbFjx7JgwQLsdjsAH330Ef3796/WCnF0+6PbhIaGctddd9W6TUPcdttt1Zb5+vo6XxcVFZGdnc1ZZ52FYRisW7cOgKysLH744QduvPFGOnfuXGs9EyZMoKysjA8//NC57L333qOyspIbbrihwXWLSMMosIhINXa7nQULFnD++eeTkpLC7t272b17NwkJCWRkZJCUlARAcnIyffv2PeGxkpOT6dmzJx4ejXcF2sPDg06dOlVbnpqayqRJk2jfvj3t2rUjLCyM8847D4C8vDwAfvvtN4CT1t2rVy+GDBlSpd/O/PnzOfPMMzVSSsQF1IdFRKr57rvvOHjwIAsWLGDBggXV1s+fP5+LLrqo0X5ebS0tR1tyfs/b2xur1Vpt2wsvvJBDhw7x97//nV69euHv78/+/fuZNGkSDoej3nVNmDCBu+++m7S0NMrKyvjll1947rnn6n0cETl1CiwiUs38+fMJDw9n7ty51dYtXLiQjz/+mHnz5hEXF8fmzZtPeKy4uDhWrlxJRUUFnp6eNW4TEhICmCOOjrd3794617xp0yZ27tzJG2+8wYQJE5zLlyxZUmW7bt26AZy0boBx48Yxbdo03n33XUpKSvD09GTs2LF1rklEGo8uCYlIFSUlJSxcuJDLL7+ca6+9ttrjzjvvpKCggEWLFnHNNdewYcOGGof/GkfmpLzmmmvIzs6usWXi6DZdunTBZrPxww8/VFn//PPP17lum81W5ZhHXz/99NNVtgsLC+Pcc8/l1VdfJTU1tcZ6jgoNDeWSSy7h7bffZv78+Vx88cWEhobWuSYRaTxqYRGRKhYtWkRBQQFXXHFFjevPPPNM5yRy77zzDh9++CHXXXcdN954I4MGDeLQoUMsWrSIefPm0b9/fyZMmMCbb77JtGnTWLVqFcOHD6eoqIhvv/2W22+/nSuvvJKgoCCuu+46nn32WSwWC3FxcXz++edkZmbWue5evXoRFxfHvffey/79+wkMDOSjjz7i8OHD1bZ95plnOOeccxg4cCC33HILsbGx7Nmzhy+++IL169dX2XbChAlce+21ADz66KN1P5Ei0rhcOURJRNzP6NGjDR8fH6OoqKjWbSZNmmR4enoa2dnZRk5OjnHnnXcaHTt2NLy8vIxOnToZEydONLKzs53bFxcXGw8++KARGxtreHp6GpGRkca1115rJCcnO7fJysoyrrnmGsPPz88ICQkx/vznPxubN2+ucVizv79/jXVt3brVSExMNNq1a2eEhoYaU6ZMMTZs2FDtGIZhGJs3bzauuuoqIzg42PDx8TF69uxpPPTQQ9WOWVZWZoSEhBhBQUFGSUlJHc+iiDQ23UtIROQEKisriY6OZvTo0bzyyiuuLkekzVIfFhGRE/jkk0/Iysqq0pFXRJqfWlhERGqwcuVKNm7cyKOPPkpoaChr1651dUkibZpaWEREavDCCy9w2223ER4ezptvvunqckTaPLWwiIiIiNtTC4uIiIi4PQUWERERcXutYuI4h8PBgQMHCAgIOKW7v4qIiEjzMQyDgoICoqOjq90f7PdaRWA5cOAAMTExri5DREREGmDfvn013oH9eK0isAQEBADmBw4MDHRxNSIiIlIX+fn5xMTEOL/HT6RVBJajl4ECAwMVWERERFqYunTnUKdbERERcXsKLCIiIuL2FFhERETE7bWKPix1YRgGlZWV2O12V5fS5tlsNjw8PDQEXURE6qxNBJby8nIOHjxIcXGxq0uRI/z8/IiKisLLy8vVpYiISAvQ6gOLw+EgJSUFm81GdHQ0Xl5e+svehQzDoLy8nKysLFJSUujevftJJwsSERFp9YGlvLwch8NBTEwMfn5+ri5HAF9fXzw9Pdm7dy/l5eX4+Pi4uiQREXFzbeZPW/0V71707yEiIvWhbw0RERFxewosIiIi4vYUWERERMTtKbCIiIiI22v1o4RERESkfhwOg4P5pezNLmLvoWL25BTRzsuDu0Z2d1lNbTKwGIZBSYVrZrz19bTVax6YxYsX89hjj7F582ZsNhvDhg3j6aefJi4uDoC0tDT+9re/8fXXX1NWVkbv3r2ZO3cuCQkJAHz22Wc88sgjbNq0iXbt2jF8+HA+/vjjJvlsIiLifgzDoKzSQVmFg5IKO6UVdkor7ZRWOCitsFNcXkna4RL2ZBezN8cMKKmHiimvdFQ5TtcOfgosza2kwk6fGV+75GdvfWQUfl51P+1FRUVMmzaNfv36UVhYyIwZM7jqqqtYv349xcXFnHfeeXTs2JFFixYRGRnJ2rVrcTjMX7IvvviCq666igcffJA333yT8vJyvvzyy6b6aCIi4mIOh8HWg/n8sCuLH3ZmsTEtj5IKO4ZR/2N5WC3EtPejSwc/unbwp1uYf+MXXJ96XPrT5aSuueaaKu9fffVVwsLC2Lp1Kz///DNZWVmsXr2a9u3bAxAfH+/c9vHHH2fcuHHMnDnTuax///7NU7iIiDSLzPxSftyVzQ+7sli+K5ucovJat7VZLfh4WPHxtB15WPH1shEV5EvXDn506eDvDChRQT542Nynq2ubDCy+nja2PjLKZT+7Pnbt2sWMGTNYuXIl2dnZztaT1NRU1q9fzxlnnOEMK7+3fv16pkyZcso1i4iIe9m8P49FGw7ww84stqcXVFnn72VjWFwo5/UI5cxuHQj288LH0wwpnm4UQOqrTQYWi8VSr8syrjR69Gi6dOnCSy+9RHR0NA6Hg759+1JeXo6vr+8J9z3ZehERaVl+3XOIZ7/bzbKdWc5lFguc3jGI4d1DObd7GAO7hLToYFKbBn2iuXPn0rVrV3x8fEhISGDVqlW1bvv6669jsViqPH5/75hJkyZV2+biiy9uSGmtSk5ODjt27OAf//gHI0eOpHfv3hw+fNi5vl+/fqxfv55Dhw7VuH+/fv1ISkpqrnJFRKQJGIbBT7uzGfe/FVw7bwXLdmZhs1q4vF8UT48bwJp/XMiiO8/hb6N6kdCtQ6sMK9CAFpb33nuPadOmMW/ePBISEpgzZw6jRo1ix44dhIeH17hPYGAgO3bscL6vaZTMxRdfzGuvveZ87+3tXd/SWp2QkBA6dOjA//73P6KiokhNTeX+++93rr/++uv517/+xZgxY5g1axZRUVGsW7eO6Ohohg0bxsMPP8zIkSOJi4tj3LhxVFZW8uWXX/L3v//dhZ9KRETqwjAMvtueybPf7Wb9vlwAPG0Wrh3UiVvPi6NLB9d2gm1u9Q4ss2fPZsqUKUyePBmAefPm8cUXX/Dqq69W+TI9nsViITIy8oTH9fb2Puk2bY3VamXBggX85S9/oW/fvvTs2ZNnnnmGESNGAODl5cU333zDPffcw6WXXkplZSV9+vRh7ty5AIwYMYIPPviARx99lCeeeILAwEDOPfdcF34iEZG26+gInsKySryPdHw9/tnb04aPhxWrxcLiLek8+91uth3MB8Dbw8r1Qztzy7ndiA5um5f76xVYysvLWbNmDdOnT3cus1qtJCYmsmLFilr3KywspEuXLjgcDgYOHMi//vUvTjvttCrbLF26lPDwcEJCQrjgggt47LHH6NChQ43HKysro6yszPk+Pz+/Ph+jRUlMTGTr1q1VlhnHjU/r0qULH374Ya37X3311Vx99dVNVp+IiNTOMMyQsmjDAT7fcJD9uSUn3cdiwTkM2d/Lxg3DunDzOd0IC2jbVx7qFViys7Ox2+1ERERUWR4REcH27dtr3Kdnz568+uqr9OvXj7y8PJ566inOOusstmzZQqdOnQDzctDVV19NbGwsycnJPPDAA1xyySWsWLECm636qJpZs2ZVGaorIiLiTlKyi1i0/gCLNuwnOavIudzfy0ZkkA+lFQ5zMrdKO2UVDsrtxyZpMwwI9PFg0tmxTD6rKyH+Xq74CG6nyYfKDBs2jGHDhjnfn3XWWfTu3ZsXX3yRRx99FIBx48Y5159++un069ePuLg4li5dysiRI6sdc/r06UybNs35Pj8/n5iYmCb8FCIi0tqVVdo5mFtK2uES0g4XO5/355aQU1ROsK8nHdp5E9rOiw7+3nRo52W+9zefPW0Wvt2WwaINB9i8/1jLv5eHlZG9whndP5oLeoXjU8P0Fg6HQbndnHm2tMJBiL8n3h71mwajtatXYAkNDcVms5GRkVFleUZGRp37n3h6enLGGWewe/fuWrfp1q0boaGh7N69u8bA4u3trU65IiJySg4VlfPGz3tYvjubtMPFZBaUNWhG2JrYrBbOiQ/liv7RXHRaBAE+nifc3mq14GO11RhmxFSvwOLl5cWgQYNISkpizJgxADgcDpKSkrjzzjvrdAy73c6mTZu49NJLa90mLS2NnJwcoqKi6lOeiIjISR3MK+GlH1J4d1VqtfvK+Xra6BjiSyfnw49OIb609/civ6SC7MJycgrLySkqI6ewnOzCMnKKyskpLKOgtJKBXUK4on80l/SNpEM7/WHdmOp9SWjatGlMnDiRwYMHM3ToUObMmUNRUZFz1NCECRPo2LEjs2bNAuCRRx7hzDPPJD4+ntzcXP7973+zd+9ebr75ZsDskDtz5kyuueYaIiMjSU5O5r777iM+Pp5Ro1wzG62IiLQ+KdlFzFuazMJ1aVTYzaaUvh0DmTisKz0iApzBpD43qD2eYRgN3ldOrt6BZezYsWRlZTFjxgzS09MZMGAAixcvdnbETU1NxWo9NmnN4cOHmTJlCunp6YSEhDBo0CB+/vln+vTpA4DNZmPjxo288cYb5ObmEh0dzUUXXcSjjz6qyz4iInLKthzI4/mlyXy16SCOI5d8EmLbc8f58QzvHtpoIUNhpWlZDKOxrti5Tn5+PkFBQeTl5REYGFhlXWlpKSkpKcTGxlabYVdcR/8uIlKbwrJKfDysDb7xXoXdQdrhEnZmFPDuqlSW7jg2jf3IXuHcfn4cg7rUfA82aV4n+v7+vZZxQx0REWn1dmUU8J9vdrJ4SzpWC0QG+tApxI+OIb50DPZ1PncK8SU62Jfc4gp+yy4kJbuIlKwi8zm7iNRDxVQ6jv0tbrXA5f2iuW1EHL2jTvylKO5LgUVERFxq36Fi/vvtTj5Zt995ycZhwIG8Ug7klcKe+h/Tx9NK1w7+DOnanpvOiaVraNuaxr41UmBpxbp27crUqVOZOnWqq0sRkVYsu7CMx7/YxtYD+QyNbc/5vcIY1i0UX68TD9HNLCjlue928+6qVGcn2FGnRTDtwp4E+3mSdriE/bkl7D9cwv7c4iPP5vuicjs2q4XO7f2IDfV3PrqF+hMb5k9EgA9Wq/qUtCYKLCIi0mCLN6fz4MebyCkqB2BHRgFv/bIXLw8rZ3brwIgeYZzfK5zY41o48oormPdDMq/9lEJphTnD6/Duodx7UU/6xwQ7t4sI9GFQl5BqP9MwDPJLK/HzsrXaOxNLdQosIiJSb3klFcz8bAsL1+4HoFdkALec2421qYf5fnsW+3NL+GFnFj/szOKRz7fStYMfI3qGE+jryWs/pVBQWgnAGZ2D+duonpwVF1rnn22xWAjyPfFEbNL6tM1oahhQXuSaRx0HZf3vf/8jOjoah8NRZfmVV17JjTfeSHJyMldeeSURERG0a9eOIUOG8O233zb4lMyePZvTTz8df39/YmJiuP322yksLKyyzU8//cSIESPw8/MjJCSEUaNGcfjwYcCcQPDJJ58kPj4eb29vOnfuzOOPP97gekTEfS3flc3Fc35g4dr9WC1w24g4Pr3zbK4e2InHxpzO8r+fz7fTzuXBS3tzVlwHPG0W9uQU8/rPe3gmaRcFpZX0jAjgpQmDWXjbWfUKK9J2tc0Wlopi+Fe0a372AwfA6+Sdv6677jruuusuvv/+e+ftCQ4dOsTixYv58ssvKSws5NJLL+Xxxx/H29ubN998k9GjR7Njxw46d+5c77KsVivPPPMMsbGx/Pbbb9x+++3cd999PP/88wCsX7+ekSNHcuONN/L000/j4eHB999/j91uzhI5ffp0XnrpJf773/9yzjnncPDgwVpviCkiLVNJuZ0nvtrGGyv2AtClgx+z/9C/2hBhi8VCfHgA8eEBTDm3G4Vllfy0O5ulOzJJO1zCNQM7Mbp/NDb1MZF6aJvzsJQXuX1gARgzZgwdOnTglVdeAcxWl5kzZ7Jv374qk/Md1bdvX2699VbnbRJOpdPthx9+yK233kp2djYAf/zjH0lNTWX58uXVti0oKCAsLIznnnvOOYPxyWgeFpGWZV3qYe55fwO/ZZt3Hv7TmV2Yfmkv/Lza5t+90jg0D8vJePqZwcFVP7uOxo8fz5QpU3j++efx9vZm/vz5jBs3DqvVSmFhIf/85z/54osvOHjwIJWVlZSUlJCamtqgsr799ltmzZrF9u3byc/Pp7KyktLSUoqLi/Hz82P9+vVcd911Ne67bds2ysrKarxRpYi4xraD+by5Yi9FZZU4DAMDs7OqYWC+NzgyhNgALFgt5g37rBYLFgtYLRZsVvN1WYWDrzabs8RGBvrw5LX9OLdHmGs/oLQ5bTOwWCx1buVwpdGjR2MYBl988QVDhgzhxx9/5L///S8A9957L0uWLOGpp54iPj4eX19frr32WsrLy+v9c/bs2cPll1/ObbfdxuOPP0779u1Zvnw5N910E+Xl5fj5+eHr61vr/idaJyLNq6Tczpyknbz8Ywp2R+M2oF91Rkf+Ofo0gvzU4VWaX9sMLC2Ej48PV199NfPnz2f37t307NmTgQMHAmYH2EmTJnHVVVcB5k0k9+zZ06Cfs2bNGhwOB//5z3+cl5ref//9Ktv069ePpKQkZs6cWW3/7t274+vrS1JSUp0vCYlI41u2M4t/fLKJfYdKAHNOk6GxHbBawAJYrRYsFov5+khLytFeJHbDwGGYrTAOh4H96GvDwO6A0zsGcU53dY4V11FgcXPjx4/n8ssvZ8uWLdxwww3O5d27d2fhwoWMHj0ai8XCQw89VG1EUV3Fx8dTUVHBs88+y+jRo/npp5+YN29elW2mT5/O6aefzu23386tt96Kl5cX33//Pddddx2hoaH8/e9/57777sPLy4uzzz6brKwstmzZwk033XRKn19ETi6roIxHP9/Kog3mpe7oIB8eubIviX0iXFyZSONpm8OaW5ALLriA9u3bs2PHDv74xz86l8+ePZuQkBDOOussRo8ezahRo5ytL/XVv39/Zs+ezf/93//Rt29f5s+fz6xZs6ps06NHD7755hs2bNjA0KFDGTZsGJ9++ikeHmbmfeihh7jnnnuYMWMGvXv3ZuzYsWRmZjb8g4vISTkcBgtWpZI4exmLNhzAaoEbz47lm2nnKaxIq9M2RwmJy+nfReTU7M4s4IGFm1m15xAAp0UHMuvq0+nXKdi1hYnUg0YJiYi0MqUVdjam5bE29TBr9h5m6Y5MKuwGvp427rmoB5PO6oqHpqlv24qyIWs7ZG6DrB3m69y95kCwE/HwgiFTIOHP5qAUN6XA0gbMnz+fP//5zzWu69KlC1u2bGnmikTkZA7mlbBm72HW7s1lTephtuzPo/J3o34u6BXOI1eeRqeQuk+XIK1E5jbYs/y4cLINinMafrzFf4eMTXDZf80A44YUWNqAK664goSEhBrXeXpqeKKIq5VW2Nm8P4/1+3JZl5rLutTDHMgrrbZdWIA3g7uEMLBzCENi29O/UxAWN/6LWBpZ3n7Y/CFs/MAMFzUJ7gJhvSC8l/ncPg5sJ/mqT/kRkmbCurch5zcY+xb4u9+IMAWWNiAgIICAgABXlyEimEOFU7KLnOFk/b5cth3Mr9Z6YrNa6B0VwKDOIQw8ElI6hfgqoLRkhgF5+8DDB/zD6nb5pSQXtn4Kmz4wW1SOXt+xekLsuRB5+rGAEtqjYXOMdRwEEafBhzdC6s/w0vlw/XsQ0af+x2pCbSawtIK+xa2K/j2kNSqvdJBTVEZWQRnZhUefy8kqKCPryPsd6QXklVRU2zcswJszYoIZ0DmYATHmQ9PeuwF7JayfD7u+gYBIaN/tyCMOQrqAh3fN+1WWQ/YOOLgR0jdB+pHnsnxzvc0bgjqZj+AYCIo58v7Ic8YW2PQ+7Pwa7MdNCNr5LOh3HfQZA37ta/zRDdL9Qrj5W3hnLBxOgVcuhGtehp6XNN7POEWt/r+Go5c8iouLNSOrGykuLgZ0SUpaPsMw+HT9Af6zZIdzwraT8faw0rdjkDOgnNE5hOggn+ZvPTEMSE6CoM4Q1qN5f3ZtDqXAmtfANwQ6DoboAeDtghZiw4Cdi2HJw2bwqInFaoaLowEmMNr8sj+40ezwaq9h5nGrJzgqwV4Gh5LNx8mE9YZ+f4DTr4Xg+t/cts7CesKU7+D9CbDnR3j3ekj8J5x9t1t0xm31w5oBDh48SG5uLuHh4fj5+alJ1YUMw6C4uJjMzEyCg4OJiopydUkiDbY7s4CHPtnCit+OdXa0WS2EtvMitJ03YQHehLXzJvS4564d/OgVGYiXhxuM6FnzOnx2t/nFO2gSnP9gw/ouFGTAiueg4KD5BRfUqWH1pPwI7/8JSg4fW2axmpc8Og4yH50Gm1/gJ+uXcSrSfoUlM2DvT+Z73xAY+mczgBxKhkO/mX09KopOfBzvIPOSTeTpENUPIvuZl20sFsjfD3lp5iN3n3mpKG/fsWW+7aHv1WZQiejbvIHBXgFf3Qe/vmq+7389jH669takU1CfYc1tIrAYhkF6ejq5ubnNX5zUKDg4mMjISIVHaZFKyu08+90uXvrxNyrsBt4eVu66IJ7rh3YmxM8Lq7UF/F7nH4C5CccuUQB4B8K5fzOHt9bly6kgHX562vxiqzzSSdgnGMY8D70uq189v74GX95rtj5E9oP2sZC2BvLTqm/r6QdRA8wv9EGTwNZILbU5yZD0CGz9xHzv4QNn3gZnTwXf4KrbGgYUZh4LMId+M4NGcGez/qh+ZgfYhvw/zjBc36JhGLD6Zfjq72DYodNQGDcf2oU36o9RYKmF3W6noqL6tWNpXp6enthsNleXIdIg327N4OFFW9ifa17+uaBXODOvOI2Y9i1oaLFhmM39O78yL7skPgxfP2j2swAIiYWLHjNDR01fnDUFlU5DzL/MD6433w/9M1z06MmDj70Svn4AVr1ovu97DVw5Fzx9j/2s/WvMVo/9a+DAuqohK7QnjPoXdE9s8OmgKBuWPQm/vmIGJiwwYDycP73hrUWtRfJ38MEkKM2DwE5w64+N2ndGgUVEpJGlHS7mn4u28u22DMC8X8/DV5zGRX0imqel0F5phoHflkLKMrOF5Mq50PnM+h9r80fmiBCrp/kFFN4bHHbY8K7ZwlBofka6DoeLZ5mXNMAMD8vnmH1MnEFlKIy4H+IuMANL0kzz8hCYLQ3Xvgah8TXXUZILH042vxQBzv8HnHvviVsXHA7I3mn2vfnhKSgxZ/ol/kIY9bjZD6OuMrbAxvdg9atQXnDsOIn/hMi+dT9Oa5e9G94dC91GwGX/adRDK7CIiDRAhd1BUVklReV2isoqKSyrpKiskvWpucxdupvSCgceVgs3DY/l7pHdm3YUj2GYk4KlLIPflpn9KY5vWQCzb8XNSdAhru7HLcqBuUOhOBtGPAAj/l51fVmBGUp+ftbsGIoFBv7JvAzz62tHlgExCWZQ6XZ+9YCx82v4+FYzTHj6w+Wzof+4qtvkJJsjUnJ2mce+6kXoc0XdPweYgeeHf8PKF8FRARYbDLnZrKu2VoC8NNj0IWx8HzKPmzQzqj9c+Ch0O69+NbQVJbnmkOnGuvx2hAKLiEgNissr2ZVRyI70AnZkFLAzo4C9OcUUHgkn5ZUnvuP50Nj2PDamLz0immjUSkE67P4Wkr+HlB+g6Hc3EPUJMls9uo2A9e/AgbXmCJWbk+reTL/wFrNVIbwP3LKs9llNc1PNETJbFlZdHnPmkaAy4sQtIfkH4KMpsHe5+b7/H+HSf4N3O/PzfTAJSnPNywzXv2v2+WionGT45iHY8YX53ifYrHHIzeYXbMlhcy6TjR8c6Uh73FwmPUaZYarnZWB1g47QbYwCi4i0efsOFbM29TA7MwrYkV7IzowCUg8V12lfL5sVf28b/t4etPP2IMjXkz8MjuHqgR0b9/KPvRLSVsGuJbB7iTlPx/E8fKHLMIg9z/zLP7IfWI/0/yrIgJcTIS8VOg+DCZ+evL/Izm/gnevMkTc3fQudBp28xtRfzMtEVhsMv8espa7nwGE3L9ssewIMB3SIh77Xmq0iht3s9zJ2PgQ00p2lf1sKix841nLSobs5Kmf3kqpDjLucbY6+6XOl2UolLqPAIiJtTqXdwbp9uSRty+S77RnszCiscbvQdt70jGxHj4gAekYE0C2sHYG+Hvh7meHE39ujaYcc5x8wW1F2LTEv9ZTlHbfSAtFnQPxIswWj05ATh5DMbfDKKPMYp18HV79Ue5gozYfnh5mjbobdafb3aC57foKPboaCA8eW9RtnDpX1bOS7tTvssPZN+O4x87LXUeGnmROu9b3WnKhN3IICi4i0CXnFFSzblcV32zJYujOL3OJjowBtVgv9OgXROyqQnhEB9IgIoEdEOzq0a/y5JKgsM/tE5O2D8iIoLzSfywqPvS4vMvuHHP+lDeZ8G/Ejzc6e8SPrPw9K8vcw/1pzdMu598EFD9a83Rf3mMNUQ7rCbT83bAr3U1F8CD6902ztOP8Bc6hwU3ZWLs2D1a9ARQmcNsacel7cjgKLiLRahmGwYPU+Plm3n1/3HsZ+3D14gv08GdEjjAt6R3Be9zCC/JphJuXUlbDortpnQ63GYk6AFp9oTocefcaxyzwNtfZNswaAMS/AgD9WXb/3Z3jtyBTrExa5tmNpZVmTTEAmLVN9vr9b/dT8ItJ6FJVV8rcPN/DlpnTnsh4R7bigVwQje4dzRkwwHjar+aW480uz42iH7uaN4YI6N26nytJ8s2/H6pcBw7yZXe8rzGnkvdqZLRjOx3HvQ7o27j1gAAZOMKe0Xz4bFv3FnDsk9lxzXUWJ2bJxdDtXj4JRWJEGUmARkRYhNaeYW976le3pBXjaLExN7MEV/aOrTth2cAOse9u8s+3x07uDOXQ2tIc550hYr2N3uG1IkNmxGL6YZk6vDnDGDeaQ2MYOIvVxwUNweI85que9G8xOtWE9YNn/mbOxtos0axRpoRRYRMTtLd+VzR3vrCWvpIKwAG/m3TCQQV2OhIOiHPOutuvmQ8Zxo2wCos37zuQkm3N9VBSbE68dnYn1KK925iibbueZI2Ai+tYeYAozzanKjw71DelqdhztNqJxP3BDWK3m5aD8/bBvpdmv5fLZ8NMz5vrLZ1efXl6kBVEfFhFxW4Zh8MryFP715TYcBvSPCebFGwYR2c7DHGmz/m2ztcNxpLOtzcucTn7ADRB3/rG+IfZK8y66WdshcztkbYOsHeaMqb+/o65ve4gdfmQo8QhznhMw5z35+gFz7hCL1RxpM2I6eLnZlPxF2fDySLO15ajTroLrXndVRSK1UqdbEWnxSivsTF+4iY/XmZddrhvUiUev7IPPto/MIat5+45tHNUfzviTeR+a+lyWsVdC5lZzkraUZWbn1PLfDYcO7GSO3DnaMhPZD654FqIHnNLna1LZu8w5WkpzzXlG7ljV6DetE2kMCiwi0qLtzy3hz2/9yub9+disFmZc3ocJ4clYljx87LKPXwfoN9a8SV1j3ffFXmHeYO+3ZWaI2bfyWOuNh485HPfMO8DWAq6mp/4C3/8Lzr7bHC4t4oYUWESkxVr5Ww63z19LTlE57f29eO1ib/pvmw2/fW9u4B0Ew/8KCbceu6NvUykvhtQVZotFj4uOXR4SkUahYc0i4n4cdvPL37c9tI/F8PDhYF4pm/fnsflAPlv257H5QB4Z+ebN9UaElzI3+iP8v1gIGOZ9X4beYt7Nt7lG43j5HZnUTS0UIq6mwCIiTc9hx/7hzdi2HruRXgYd2GuPINuIoMyIxMOIJNiIAEsAT0T/yIjchVi2H+kQe/p1cME/zFE5ItImKbCISNNyODi84FZCdi6kwrBRjDdBlmIiySHSlsMwtlbfJ+fIc+y5cOEj5mywItKmKbCISJMpr7Cz9ZU/MyD9AyoNK/db/4pX3ys4IwwG+OfQ1ZKBV94ec2KzQ7+Zc6aU5po3qrvwEfNSTFPeb0ZEWgwFFhFpEpvTctn21l+5rmwhDsPCmxH3c/+fphIWcJKp2csKzSnsFVRE5DgKLCLSqMorHTz33S6sP/wfUz3MPiubB/6TyVfcjaUuIcS7XdMWKCItkgKLiDSaTWl5/O3DDQzPepcHPT8CoPD8x+h33l0urkxEWjoFFhE5ZYeKynl1eQovLEvmess3POj5jrli5AzaDVdYEZFTp8AiInViGAZZBWXsyixkV0YBu7MK2ZVRyO7MQnKKzOHH19mW8pjna+YOw++F4fe4rmARaVUUWESkVpVFh0l7bSJhOatJM8L4zR7OXiOSFCOSvUYEexyRHCIYsHJj8FoeKn3Z3PHM2815U0REGokCi4jUaN+e3Tjeupqu9r0A9GQvPW17q23nsPlA+1isObsABwyaBKP+pVE+ItKorA3Zae7cuXTt2hUfHx8SEhJYtWpVrdu+/vrrWCyWKg8fH58q2xiGwYwZM4iKisLX15fExER27drVkNJE5BQZhsFnSd9je+0iutj3kkkISwa/SNolb1B50Sxzevz4RPO+OhYbVnsp1qxt4Kg0b0Z42X8VVkSk0dW7heW9995j2rRpzJs3j4SEBObMmcOoUaPYsWMH4eE13748MDCQHTt2ON//fmjjk08+yTPPPMMbb7xBbGwsDz30EKNGjWLr1q3Vwo2INJ3MglL+N38Bdxx8kBBLIQc8OmH700Iu7NKz5h3sFZCbCodSoKIIel4G1gb9HSQickL1vltzQkICQ4YM4bnnngPA4XAQExPDXXfdxf33319t+9dff52pU6eSm5tb4/EMwyA6Opp77rmHe++9F4C8vDwiIiJ4/fXXGTdu3Elr0t2aRU7d4s0H+eKj13nSMRtfSzmZgX0JveVTrO1CXV2aiLRS9fn+rtefQuXl5axZs4bExMRjB7BaSUxMZMWKFbXuV1hYSJcuXYiJieHKK69ky5YtznUpKSmkp6dXOWZQUBAJCQm1HrOsrIz8/PwqDxFpmPzSCu55fwNJ787mv44n8bWUU9T5fMLv/EZhRUTcRr0CS3Z2Nna7nYiIiCrLIyIiSE9Pr3Gfnj178uqrr/Lpp5/y9ttv43A4OOuss0hLSwNw7lefY86aNYugoCDnIyYmpj4fQ0QwWzd/2p3NJf/9gYgNz/Fvz//hYXFg73c9/hM/MKfHFxFxE00+SmjYsGEMGzbM+f6ss86id+/evPjiizz66KMNOub06dOZNm2a831+fr5Ci0gdlVbYWbThAG+t2MuW/YeZ4fEmkzy/MVee81dsIx9Wp1kRcTv1CiyhoaHYbDYyMjKqLM/IyCAyMrJOx/D09OSMM85g9+7dAM79MjIyiIqKqnLMAQMG1HgMb29vvL1PcgM1Eali36Fi3l65l/dW76O0uJDh1k3c6/Ud51nXY2DBcvETcOatri5TRKRG9bok5OXlxaBBg0hKSnIuczgcJCUlVWlFORG73c6mTZuc4SQ2NpbIyMgqx8zPz2flypV1PqaI1MzhMPhhZxY3v/ErV/37Ew4tf5V/VzzBep8/85LXbM6zrgerJ5ZrX1FYERG3Vu9LQtOmTWPixIkMHjyYoUOHMmfOHIqKipg8eTIAEyZMoGPHjsyaNQuARx55hDPPPJP4+Hhyc3P597//zd69e7n55psBc4jz1KlTeeyxx+jevbtzWHN0dDRjxoxpvE8q0obkl1bw0Zo0lv70E73ylnOrbQ0DvXZhtRw3KDCoM/S6FAaMh6h+ritWRKQO6h1Yxo4dS1ZWFjNmzCA9PZ0BAwawePFiZ6fZ1NRUrMfNw3D48GGmTJlCeno6ISEhDBo0iJ9//pk+ffo4t7nvvvsoKirilltuITc3l3POOYfFixdrDhaRetp6IJ/5K5JxbHiPSXzGZGsaeB63QdQA6HmpGVQi+qqvioi0GPWeh8UdaR4WacvKKu0s3pzOOz8n03n/Z9xh+5SuVrOfmd3igdF1OB69LzODSlBHF1crInJMfb6/dS8hkRZqf24J83/Zy0erUji37DuetH1CF89MACq82+MxfCq2wZPAJ8i1hYqINAIFFpEWZv2+XJ77bjc/bN/PldblfGD7mM6eWQA4fDtgPWcqnkNu0jwqItKqKLCItBDJWYU89fUOlm3ew2jbCr71/ITOVjOoGP5hWM6+G+vgGxVURKRVUmARcXPpeaW8/NUKijd/zrWWNczx3oy3pcJc6R8GZ0/FMvhG8PJzbaEiIk1IgUXEHRkG+WlbWPPNfIJTl/AAu7F6HNc/PqQrDL0FBk1WUBGRNkGBRcSd5KZSseJFijYuIrgklfMBjow8Lgrth3+/K6DnZRDeW0OSRaRNUWARcRd7V1A+fxxe5bkEA2WGBxs9+xHQ/0p6nvcH/AOjXV2hiIjLKLCIuION7+P45Ha8HBVsdMTyvvc1DL3wD1w2uAc2q1pSREQUWERcyTBg6ROw7AmswJf2oXzX+xEe/8NQvD1srq5ORMRtKLCIuEplGXx6J2x6H4B5laPZ0nsq/x07EA9bve5LKiLS6imwiLhCUQ68Nx5SV1Bh2HiocjJ5vf/Is+POUFgREamBAotIc8veBfOvg8Mp5Bt+3FZxN/69Epl7vcKKiEhtFFhEmlPKj/DeDVCaS5oRxqTyv9G110Ce++NAPBVWRERqpf9DijSXDQvgraugNJf1RneuLHuETj3OYO74gXh56D9FEZET0f8lRZrDpg/h41vBUcGXxjDGlj1In+5xzLthkEYDiYjUgQKLSFPbsRg+/jNgsMC4iDvK7mBQXBQvTRiMj6fCiohIXSiwiDSllB8xPpgIjko+dZzD9LIJDI0N5ZWJQxRWRETqQYFFpKnsX4v9nbFYKktZYh/IPeW3MLJ3JK9OGoKvl8KKiEh9aJSQSBMoP7gF+2tj8K0s4md7Hx70uIenrj6DKwdEY9FNC0VE6k2BRaSRbdmykYgPryTUyGO9oxvvd3+SL64aSliAt6tLExFpsRRYRBpJSbmd/335M2PW3Uio5RDJxJB1xXzmDO7j6tJERFo8BRaRU2QYBj8n5/DEwp/5d+EDdLFmku0ZTYebvyQuorOryxMRaRUUWEQa6FBROQvXpvHe6n0cyMxivte/6GXdR5lPOKG3fAntFVZERBqLAotIbSrLIXcvHEqB0lwoK8BRVsS+9Ex2pWWQlZNDB0r5G6XEeR8kznIAh08I3jcugvaxrq5eRKRVUWCRtq2yDA7vgUO/mY+c5COvkyEvDQxHlc2tQJcjD34/MtmrHdY/fQThvZuldBGRtkSBRdqWwixIXXHscXAjGPZaN3d4+pFGJKllfhQZPhThQ6XNl+jwMHrERBLeoQN4+YNXO+hyFgR1bMYPIyLSdiiwSOtlGGbrSeoK2Puz+Zyzu/p2XgHmJZwOcdC+25FHHN+k+zHty4MUlpmB5qy4DowdEsOlp0VqlloRkWamwCKtj2HALy/Az89AwcHfrbRAeB/oMgw6D4POZ0JgRzhuMrfSCjszP9vKu6tSARjSNYQnr+1PbKh/M34IERE5ngKLtC6VZfDZ3bDhXfO91RM6DjwSToZB5wTwDal1992ZBdwxfx07MgqwWOCOEfFMTeyOh013sRARcSUFFmk9CjNhwXhIWwUWG4x6HAZNAk/fOu3+4Zo0HvpkMyUVdkLbeTNn7ADO6R7atDWLiEidKLBI63BwI7x7PeSngU8QXPc6xF1Qp12Lyip56JPNLFy3H4Cz4zvw37EDCA/wacKCRUSkPhRYpOXb9hksvAUqiqFDPFz/HoTG12nXrQfyufPdtfyWVYTVAtMu7MFtI+KxWXWDQhERd6LAIi2XYcCP/4HvHjXfdxthtqycoI8KQFmlneW7svli00E+33iQ8koHkYE+PHP9GQyNbd/kZYuISP0psEjLVFECn94Jmz803w+9BUbNAlvNv9KlFXZ+2JnFV5vT+XZrBgVllc51F/QK56nr+tPe36s5KhcRkQZQYJGWpyAdFvwR9q8Bqwdc8iQMuanaZiXldpbtzOTLTekkbcugqPzYBHERgd5c0jeKS0+PYkjXECwWXQISEXFnCizSchzeCyvnwdo3obwQfIJh7FsQe26VzfJLK3j62128uyqV4uNCSnSQD5ecHsWlp0dyRkwIVvVTERFpMRRYxP3tXwM/PwdbPz02jX7k6XDdG+bstEc4HAYL1+3nia+2kV1YDkDHYF8u6xfFJX0j6d8pWCFFRKSFUmAR9+RwwM7FsOI52PvTseWx58FZf4H4kVVmp92UlseMRZtZl5oLQLcwf2Zc3ofzeoTpco+ISCugwCLupaLEnKV2xdxj9/2xekDfa2DYnRDVr8rmh4vK+fc3O3h3VSqGAf5eNu5O7M6ks2Lx8tDstCIirYUCi7iHvDRY/QqsfQOKc8xl3kEweBIM/XO1uyDbHQbvrkrlqW92kFtcAcCYAdFMv7Q3EYGa8E1EpLVRYBHXMQzzLsqrXoRtnx/rnxLUGc68DQb+CbwDqu22YV8uD3y8iS0H8gHoFRnAzCtOI6Fbh+asXkREmpECizS/ihLY9AGs/B9kbDq2vOtwSPgz9Lik1vlUNuzLZdz/fqGkwk6gjwf3XNST8QmddXNCEZFWToFFmk9uKqx+2RyWXHLYXObhC/3HmhO/RZx2wt1Tc4q56Y3VlFTYOSc+lKfHDaBDO+9mKFxERFxNgUWaR9oaeP1SqCw13wd3hiFT4IwbwO/k0+EfKipn4muryC4s57ToQOb9aRDtvPXrKyLSVuj/+NL0DAO++YcZVqIHwrl/gx6jwGqr0+6lFXZufmM1KdlFdAz25bVJQxRWRETaGP1fX5pechKk/gw2bxj7drURPydidxjcvWAda1NzCfTx4I0bhxCuUUAiIm2OeipK0zIM+O4x8/WQm+sVVgzD4NHPt/L1lgy8bFZemjCY+PDqo4ZERKT1U2CRprX9cziwDjz94Zy/1mvXV5an8PrPewD4zx/6a9iyiEgb1qDAMnfuXLp27YqPjw8JCQmsWrWqTvstWLAAi8XCmDFjqiyfNGkSFoulyuPiiy9uSGniThx2+O5x8/WZt0G7sDrv+vnGAzz2xTYAHri0F6P7RzdFhSIi0kLUO7C89957TJs2jYcffpi1a9fSv39/Ro0aRWZm5gn327NnD/feey/Dhw+vcf3FF1/MwYMHnY933323vqWJu9n8EWRtA58gOOuuOu+2KuUQ097bAMDEYV2YMrxbU1UoIiItRL0Dy+zZs5kyZQqTJ0+mT58+zJs3Dz8/P1599dVa97Hb7YwfP56ZM2fSrVvNXz7e3t5ERkY6HyEhIfUtTdyJvQK+P9K6cvbd4Btcp912ZxYw5c1fKbc7uKhPBDNGn6abF4qISP0CS3l5OWvWrCExMfHYAaxWEhMTWbFiRa37PfLII4SHh3PTTTfVus3SpUsJDw+nZ8+e3HbbbeTk5NS6bVlZGfn5+VUe4mbWvQ2H94B/GCTcWqdd0g4XM/HV1eSVVHBG52Ceuf4MbFaFFRERqWdgyc7Oxm63ExERUWV5REQE6enpNe6zfPlyXnnlFV566aVaj3vxxRfz5ptvkpSUxP/93/+xbNkyLrnkEux2e43bz5o1i6CgIOcjJiamPh9DmlpFKfzwb/P18HvAy/+ku6TmFDP2xV/Yn1tCbKg/L08YjI9n3eZpERGR1q9J52EpKCjgT3/6Ey+99BKhoaG1bjdu3Djn69NPP51+/foRFxfH0qVLGTlyZLXtp0+fzrRp05zv8/PzFVrcya+vQv5+COwIgyafdPM92UVc/9IvHMwrJTbUn3ennKkp90VEpIp6BZbQ0FBsNhsZGRlVlmdkZBAZGVlt++TkZPbs2cPo0aOdyxwOh/mDPTzYsWMHcXFx1fbr1q0boaGh7N69u8bA4u3tjbe3vtDcUlkh/Pgf8/V5fwfPE0/y9ltWIde/9AsZ+WXEhZlhRRPDiYjI79XrkpCXlxeDBg0iKSnJuczhcJCUlMSwYcOqbd+rVy82bdrE+vXrnY8rrriC888/n/Xr19faKpKWlkZOTg5RUVH1/DjicitfgOJsaN8NBvzxhJvuzixg7P/MsNI9vB0LbhmmsCIiIjWq9yWhadOmMXHiRAYPHszQoUOZM2cORUVFTJ5sNv1PmDCBjh07MmvWLHx8fOjbt2+V/YODgwGcywsLC5k5cybXXHMNkZGRJCcnc9999xEfH8+oUaNO8eNJsyo5DD89a74e8QDYPGvddEd6AeNf/oXswnJ6RQYw/+YEXQYSEZFa1TuwjB07lqysLGbMmEF6ejoDBgxg8eLFzo64qampWK11b7ix2Wxs3LiRN954g9zcXKKjo7nooot49NFHddmnpfn5WSjLg/A+0PeaWjfbdjCf8S+v5FBROX2iAnn75gTa+3s1Y6EiItLSWAzDMFxdxKnKz88nKCiIvLw8AgMDXV1O21SYCU8PgIoiGDsfel9e42ab9+dxwysryS2u4PSOQbx101CC/RRWRETaovp8f+tuzdI4fpxthpXogdDrsho32ZiWyw0vryS/tJL+McG8eeNQgnxrv2wkIiJylAKLnLq8NPj1FfP1yIeghplpN6blMv7llRSUVjKwczCv3ziUQB+FFRERqRsFFjl1384Eezl0OQe6nV9t9baD+fzplVUUlFYypGsIr00eSjtv/eqJiEjdNehuzSJOvy2FTe8DFrjokWqtK7szC7jh5ZXO6fYVVkREpCEUWKThKkrh8yMzDg+dAh0HVVm9J7uIP760kpyick6LDuR1hRUREWkgBRZpuOX/hUPJ0C4SLvhHlVVph4sZ//JKMgvK6BkRwFs3JaiDrYiINJgCizRM9m5YPtt8ffEs8AlyrsrIL2X8yyvZn1tCt1B/zbMiIiKnTIFF6s8w4Iu/mh1t4xPhtKucq7ILy/jjS7+wN6eYmPa+zJ+SQFiAJgAUEZFTo8Ai9bfxfUj5ATx84NKnnB1tc4vLueHllSRnFREV5MM7N59JVJCvi4sVEZHWQIFF6qfkMHz9gPn63L9B+1gA8ksrmPDqKranFxAW4M07U84kpr2fCwsVEZHWRIFF6ufbf5p3Yw7tCWf9BYCiskpufG01G9PyaO/vxTs3JxAb6u/aOkVEpFVRYJG6S10Ja143X1/+X/DwwuEwmPb+en7de5hAHw/eumko3SMCXFqmiIi0PgosUjf2Cvj8r+brATdA17MBeGFZMl9vycDLZuW1yUM5LTroBAcRERFpGAUWqZtfnofMLeDbHi58BIAfdmbx1Dc7AHjkytMY1CXElRWKiEgrpsAiJ5ebCkufMF9f9Cj4d2DfoWL+smAdhgHXD41h3NDOrq1RRERaNQUWOTHDgC//BhXF0OVsGDCeknI7f35rDbnFFfSPCeafV5zm6ipFRKSVU2CRE9v6KexcDFZPuPy/GMCDH29i68F8Ovh78cL4gXh72FxdpYiItHIKLFK7De/Bwinm67P/AmE9eXPFXhau24/NauHZP55BdLAmhhMRkaanW+dKdYZh9llZdqTfSu8r4Ly/s3rPIR79fCsA0y/pxVlxoS4sUkRE2hIFFqmqohQW3QmbPjDfnz0VRj5MRmE5t89fS6XD4PJ+Udx0TqxLyxQRkbZFgUWOKcqGBeNh3y9g9YDLZsOgiZRXOrh9/lqyCsroGRHAk9f2w3Lk/kEiIiLNQYFFTFk74Z3r4PAe8A6CsW9CtxEAPPbFVtbsPUyAjwfz/jQIPy/92oiISPPSN4+Yd15+7wYozYPgLjD+AwjrCcDH69J4c8VeAOaMHaB7BImIiEsosLR1696Gz+4GRyV0Ggrj3oF2YYB5B+ZHPjM72f5lZHdG9o5wZaUiItKGKbC0VQ47fPcoLP+v+b7vNXDl8+Dp49zk+e+TOVxcQXx4O/5yQbyLChUREVFgaZtK8+CjKbDra/P9uffBiOlgPTYtT9rhYl79KQUwhzB72DRlj4iIuI4CS1uTtRMWXA85u8HDB654Fvr9odpmT329g/JKB8O6deCCXuEuKFREROQYBZa2ZMdXZstKeQEEdoJxb0P0GdU225SWxyfrDwDw4GW9NYRZRERcToGlLXA44Mf/wPePA4Z5E8Pr3nB2rj2eYRg8/qXZ0fbqMzrSt2NQMxcrIiJSnQJLa1dWCJ/cCts+M98PmQIXzwKbZ42bJ23L5JffDuHlYeWeUT2bsVAREZHaKbC0Zod+M2euzdwKNi+49CkYNLHWzSvtDmZ9tQ2Am86JpaNubCgiIm5CgaW1Sv4OPpgMpbnQLhLGvgUxQ0+4y4LV+0jOKqK9vxe3jYhrnjpFRETqQIGlNcrbD++MA3sZdBwMY9+GwKgT7lJQWsGcb3cCMDWxO4E+NV8yEhERcQUFltZoy8dmWIk+AyZ9UWUyuNq8uOw3sgvL6Rbqz/VDOzdDkSIiInWn2cBao62fmM/9/1insHIwr4SXfvwNgL9f0gtPTRInIiJuRt9MrU1eGqStBizQ54o67fKfb3ZSVulgaNf2XNRH9wsSERH3o8DS2mz91HzuPAwCIk+6+ZYDeXy0Ng2ABzRJnIiIuCkFltZmyyfmc58rT7qpYRj868ttGAaM7h/NgJjgJi1NRESkoRRYWpO8NEhbZb6uw+WgpTuz+Gl3Dl42K/dpkjgREXFjCiytydZF5nPMmRAYfcJNDcPg/77aDsCks7sS096vqasTERFpMAWW1uTo6KDTxpx00xXJOWxPL8Dfy8YdI+KbtCwREZFTpcDSWuTth30rzde9T3456I0VewC4emAngvw0SZyIiLg3BZbWYtvRy0EJENTxhJvuzy1hydYMACYM69LUlYmIiJwyBZbWwjk6aMxJN53/y14cBpwV14HuEQFNWpaIiEhjUGBpDfIPwL5fzNcnGc5cWmFnwep9AEwY1rWJCxMREWkcCiytwbbPzOdOQ096OeiLjQc5VFROdJAPib3Dm6E4ERGRU6fA0hocvRxUh9FBbx7pbDv+zC546J5BIiLSQugbq6UrSIfUFebrk1wOWr8vlw1peXjZrIwbEtMMxYmIiDQOBZaWbusiwIBOQyCo0wk3Pdq6cnm/KDq082762kRERBpJgwLL3Llz6dq1Kz4+PiQkJLBq1ao67bdgwQIsFgtjxoypstwwDGbMmEFUVBS+vr4kJiaya9euhpTW9hydLO4ko4NyCsv4fMNBACac1bVJSxIREWls9Q4s7733HtOmTePhhx9m7dq19O/fn1GjRpGZmXnC/fbs2cO9997L8OHDq6178skneeaZZ5g3bx4rV67E39+fUaNGUVpaWt/y2paCdNj7s/n6JJeDFqzeR7ndQf9OQbrJoYiItDj1DiyzZ89mypQpTJ48mT59+jBv3jz8/Px49dVXa93Hbrczfvx4Zs6cSbdu3aqsMwyDOXPm8I9//IMrr7ySfv368eabb3LgwAE++eSTGo9XVlZGfn5+lUebtO0zwICOgyG49j4plXYH83/ZC2gos4iItEz1Cizl5eWsWbOGxMTEYwewWklMTGTFihW17vfII48QHh7OTTfdVG1dSkoK6enpVY4ZFBREQkJCrcecNWsWQUFBzkdMTBvtQFrH0UHfbsvkQF4p7f29uKxfVJOXJSIi0tjqFViys7Ox2+1ERERUWR4REUF6enqN+yxfvpxXXnmFl156qcb1R/erzzGnT59OXl6e87Fv3776fIzWoSAD9v5kvj7J5aCjnW3HDonBx9PWxIWJiIg0Po+mPHhBQQF/+tOfeOmllwgNDW2043p7e+Pt3cZHuWw7Mjqo4yAI7lzrZrsyCvg5OQerBcYn1L6diIiIO6tXYAkNDcVms5GRkVFleUZGBpGRkdW2T05OZs+ePYwePdq5zOFwmD/Yw4MdO3Y498vIyCAq6tjlioyMDAYMGFCf8tqWrZ+azycZHfTWkb4rib0j6BTi18RFiYiINI16XRLy8vJi0KBBJCUlOZc5HA6SkpIYNmxYte179erFpk2bWL9+vfNxxRVXcP7557N+/XpiYmKIjY0lMjKyyjHz8/NZuXJljccUoDCzTpeDCkor+GhNGgATNZRZRERasHpfEpo2bRoTJ05k8ODBDB06lDlz5lBUVMTkyZMBmDBhAh07dmTWrFn4+PjQt2/fKvsHBwcDVFk+depUHnvsMbp3705sbCwPPfQQ0dHR1eZrkSO2LQLDAdEDIaRLrZstXLufonI7cWH+nBXXoRkLFBERaVz1Dixjx44lKyuLGTNmkJ6ezoABA1i8eLGz02xqaipWa/1GS993330UFRVxyy23kJubyznnnMPixYvx8fGpb3ltQx1GBxmGwRtHOttOPKsrFoulycsSERFpKhbDMAxXF3Gq8vPzCQoKIi8vj8DAQFeX07QKs+A/PcwWlrs3QEjXGjdbviubG15Zib+XjV8eGEmAj2fz1ikiInIS9fn+1r2EWpqdX5lhJWpArWEFcLauXDOok8KKiIi0eAosLc3uI52Te4yqdZP0vFKStpkjuSYMq72Pi4iISEuhwNKSOOzw21LzddzIWjdbvPkgDgMGdg4mPjygeWoTERFpQgosLcmBdVCaC95B5oRxtfhqszlD8KWnaxp+ERFpHRRYWpLk78znbueCreYBXtmFZazecwiAUadVn8xPRESkJVJgaUmO9l+Ju6DWTb7ZkoHDgNM7BhHTXjPbiohI66DA0lKU5kHaavP1CQLLV5sPAnBxX7WuiIhI66HA0lKk/ACGHdrH1TqcOa+4ghXJOQBcosAiIiKtiAJLS3G0/0p87aODlmzLoNJh0DMigG5h7ZqpMBERkaanwNJSHA0sJ7gctFiXg0REpJVSYGkJcpLh8B6wekLX4TVuUlhWyQ+7sgG45HQFFhERaV0UWFqCo60rnc8E75ov9Xy3PZPySgexof70jNBkcSIi0roosLQEzstB59e6yfGXg3RnZhERaW0UWNydvcIcIQS1TsdfUm7n++1ZgEYHiYhI66TA4u72rYLyQvALhch+NW6ybGcWJRV2Ogb7cnrHoGYuUEREpOkpsLi74y8HWWv+59LlIBERae0UWNxd8omn4y+rtJO0LRPQ5SAREWm9FFjcWVEOHFhvvq4lsPy8O4eCskrCA7wZ2Dmk+WoTERFpRgos7uy37wEDwk+DgJpbT47eO2jUaZFYrbocJCIirZMCiztL/t58jq+5daXS7mDJ1gxAl4NERKR1U2BxV4Zx0v4rK1MOcbi4ghA/T4bGtm/G4kRERJqXAou7ytoOBQfBwwc6n1XjJkcvB13UJxIPm/4pRUSk9dK3nLvafaR1pcvZ4OlTbbXDYfD1FvNy0MW6d5CIiLRyCizu6uj8K/E1z267NvUwWQVlBPh4cHZcaDMWJiIi0vwUWNxRRQns/cl8XUv/la82pwOQ2DsCLw/9M4qISOumbzp3lLoCKkshIBrCelVbbRgGi48Elos1OkhERNoABRZ3tPu40UE1TLW/aX8e+3NL8POycV6PsGYuTkREpPkpsLijk8y/cvRy0Pk9w/HxtDVXVSIiIi6jwOJu8g9C5hbAAt3Or7Zal4NERKQtUmBxN78daV2JPgP8qk8Gtz29gJTsIrw8rJzfK7yZixMREXENBRZ3s/vEs9u+uCwZgAt6htPO26O5qhIREXEpBRZ34nAca2GpYf6VHekFfLrhAAB3XhDfnJWJiIi4lAKLO0nfAMU54BUAnYZUWz17yQ4MAy49PZK+HYNcUKCIiIhrKLC4k13fms+x54LNs8qqjWm5fL0lA6sFpl3YwwXFiYiIuI4Ci7twOGD9fPN1r0urrf7PNzsBGDOgI/HhAc1ZmYiIiMspsLiLPT/A4RTwDoTTrqqyavWeQyzbmYWH1cLURLWuiIhI26PA4i7WvG4+n34dePk7FxuGwb+/3gHAH4bE0LmDnwuKExERcS0FFndQlA3bPjdfD5pUZdWPu7JZlXIILw8rd2lkkIiItFEKLO5g/TvgqIDogRDVz7nYMAye+sZsXbkhoQtRQb6uqlBERMSlFFhczTCOXQ76XevKkq0ZbEzLw8/Lxu3nxzV7aSIiIu5CgcXV9iyHQ8ng1Q76XuNc7HAYzF5ijgyafHZXQtt5u6pCERERl1NgcTVnZ9trwbudc/FnGw+wPb2AAB8Pbhmu1hUREWnbFFhcqfgQbFtkvj7uclCl3cGcb3cBcMvwbgT5edaws4iISNuhwOJKG94FezlE9TfvznzER2vTSMkuor2/F5PPiXVhgSIiIu5BgcVVaulsW1Zp55mk3QDcPiJOd2QWERFBgcV1UldA9k7w9IO+1zoXL1i1j/25JUQEenPDmV1cWKCIiIj7UGBxlaOtK32vAZ9AAErK7Tz3vdm6cucF3fHxtLmoOBEREfeiwOIKxYdgyyfm60GTnYvfW51KVkEZnUJ8GTs4xjW1iYiIuKEGBZa5c+fStWtXfHx8SEhIYNWqVbVuu3DhQgYPHkxwcDD+/v4MGDCAt956q8o2kyZNwmKxVHlcfPHFDSmtZdj4PtjLIOJ06DjQuXjVnkMAjE/ogpeHsqSIiMhR9e7R+d577zFt2jTmzZtHQkICc+bMYdSoUezYsYPw8PBq27dv354HH3yQXr164eXlxeeff87kyZMJDw9n1KhRzu0uvvhiXnvtNed7b+9WOlFalc62E8Fica7adrAAgL4dA11QmIiIiPuq95/xs2fPZsqUKUyePJk+ffowb948/Pz8ePXVV2vcfsSIEVx11VX07t2buLg47r77bvr168fy5curbOft7U1kZKTzERIS0rBP5O72rYKsbeDhC/3+4FxcXF7JnpwiAHpFKrCIiIgcr16Bpby8nDVr1pCYmHjsAFYriYmJrFix4qT7G4ZBUlISO3bs4Nxzz62ybunSpYSHh9OzZ09uu+02cnJyaj1OWVkZ+fn5VR4thrOz7dXgE+RcvCO9AMOA0HbehAW00tYlERGRBqrXJaHs7GzsdjsRERFVlkdERLB9+/Za98vLy6Njx46UlZVhs9l4/vnnufDCC53rL774Yq6++mpiY2NJTk7mgQce4JJLLmHFihXYbNVHysyaNYuZM2fWp3T3UJILWz42X//uRofb083LQb2jApq3JhERkRagWWYlCwgIYP369RQWFpKUlMS0adPo1q0bI0aMAGDcuHHObU8//XT69etHXFwcS5cuZeTIkdWON336dKZNm+Z8n5+fT0xMCxhVs+kDqCyB8D7QaUiVVdsOmq1EvaN0OUhEROT36hVYQkNDsdlsZGRkVFmekZFBZGRkrftZrVbi4+MBGDBgANu2bWPWrFnOwPJ73bp1IzQ0lN27d9cYWLy9vVtep1zDgF+PdCoeNKlKZ1uA7Uc63PaKVAuLiIjI79WrD4uXlxeDBg0iKSnJuczhcJCUlMSwYcPqfByHw0FZWVmt69PS0sjJySEqKqo+5bm3/Wsgcwt4+FTpbAtm355t6WphERERqU29LwlNmzaNiRMnMnjwYIYOHcqcOXMoKipi8mRzArQJEybQsWNHZs2aBZj9TQYPHkxcXBxlZWV8+eWXvPXWW7zwwgsAFBYWMnPmTK655hoiIyNJTk7mvvvuIz4+vsqw5xZv7Rvmc58x4Ft1BNT+3BIKSivxtFmIC2vX/LWJiIi4uXoHlrFjx5KVlcWMGTNIT09nwIABLF682NkRNzU1Fav1WMNNUVERt99+O2lpafj6+tKrVy/efvttxo4dC4DNZmPjxo288cYb5ObmEh0dzUUXXcSjjz7a8i77nMi+1eZz36urrTo6/0pcWDtNGCciIlIDi2EYhquLOFX5+fkEBQWRl5dHYKAbXlKxV8LjkeCogKmbILhzldXPJu3iP0t2ctUZHfnv2AGuqVFERKSZ1ef7W3/ON4fcvWZY8fCFwE7VVh/rv6IOtyIiIjVRYGkO2TvN59B4sFY/5cdGCLlh65CIiIgbUGBpDs7A0qPaquLySlKOTMmvEUIiIiI1U2BpDicILDszCo9Mye+lKflFRERqocDSHLJ3mc+h3aut0gy3IiIiJ6fA0tQMA7J2mK9De1Zbvf1IYNEMtyIiIrVTYGlqxTlQmgtYoENctdVH52BRC4uIiEjtFFia2tHWleDO4OlbZdXxU/JrhJCIiEjtFFia2gk63B6dkt/DaiE+XFPyi4iI1EaBpak5O9xWDyxH51+JD9eU/CIiIieib8mm5mxh0QghERGRhlJgaWonuCS0Pf3oDLcaISQiInIiCixNqaIEclPN1zUElmP3EFILi4iIyIkosDSlnGTAAJ9g8A+tsqqk3M6ebE3JLyIiUhcKLE3p+MtBFkuVVTszCnBoSn4REZE6UWBpSkdHCIXVcDlIHW5FRETqTIGlKWUfnZJfHW5FREROhQJLUzrBCKGtamERERGpMwWWpuJwQPZu8/XvAothGMfd9FCBRURE5GQUWJpKfhpUloDVE4K7VFl1IK+UfE3JLyIiUmcKLE3l6OWgDnFg86iy6mjriqbkFxERqRt9WzYV5z2ENCW/iIjIqVJgaSon6HC7TSOERERE6kWBpamc4C7NamERERGpHwWWppJV8xwsx0/J3ytKLSwiIiJ1ocDSFEoOQ1Gm+fp3fViOn5I/PMDHBcWJiIi0PAosTeHo/CsB0eBdtRVle7rmXxEREakvBZam4OxwW9MIIbPDbW9dDhIREakzBZamcKIRQprhVkREpN4UWJpCLSOEDMPQCCEREZEGUGBpCrVcEjp43JT8ceH+LihMRESkZVJgaWz2CjicYr7+XQvLtuOm5Pf2sDV3ZSIiIi2WAktjO5QCjkrw9IfA6CqrtmuGWxERkQZRYGls2UcnjOsOFkuVVVvVf0VERKRBFFga29H+K2E9q606epfmXgosIiIi9aLA0thquUtzaYWdlCNT8msOFhERkfpRYGlstczBcnRK/g7+XoS183ZBYSIiIi2XAktjMoxa52A5fv4Vy+/6toiIiMiJKbA0psIMKMsHixXad6uy6uiU/BohJCIiUn8KLI3p6OWgkK7gceyyj91hsHx3NqARQiIiIg2hwNKYaum/8vnGA+zOLCTQx4PEPhEuKExERKRlU2BpTFnVp+SvtDuY863Zr+WWc7sR5OvpispERERaNAWWxlRDC8vCtftJyS6ivb8Xk8+OdVFhIiIiLZsCS2NyjhAyJ40rq7TzdJK57PYRcfh7e7iqMhERkRZNgaWxlBVCfpr5+sglofdX72N/bgnhAd7ccGYXFxYnIiLSsimwNJac3eazXyj4tae0ws6z35nL7rogHh9P3Z1ZRESkoRRYGsvvJox7a8VeMgvK6Bjsyx+GxLiwMBERkZZPgaWxZB8bIVRYVskLy5IBuHtkd7w91LoiIiJyKhoUWObOnUvXrl3x8fEhISGBVatW1brtwoULGTx4MMHBwfj7+zNgwADeeuutKtsYhsGMGTOIiorC19eXxMREdu3a1ZDSXOe4EUKv/5TCoaJyunbw4+qBHV1bl4iISCtQ78Dy3nvvMW3aNB5++GHWrl1L//79GTVqFJmZmTVu3759ex588EFWrFjBxo0bmTx5MpMnT+brr792bvPkk0/yzDPPMG/ePFauXIm/vz+jRo2itLS04Z+suR25JFQU0I0Xf/gNgL9e2AMPmxqxRERETpXFMAyjPjskJCQwZMgQnnvuOQAcDgcxMTHcdddd3H///XU6xsCBA7nssst49NFHMQyD6Oho7rnnHu69914A8vLyiIiI4PXXX2fcuHEnPV5+fj5BQUHk5eURGOiCqe8ddng8CuxlvHTGxzy+ooSeEQF8dfdwrFbd6FBERKQm9fn+rtef/+Xl5axZs4bExMRjB7BaSUxMZMWKFSfd3zAMkpKS2LFjB+eeey4AKSkppKenVzlmUFAQCQkJtR6zrKyM/Pz8Kg+Xyt0L9jIMmzdPrzFbhf56YQ+FFRERkUZSr8CSnZ2N3W4nIqLq/XAiIiJIT0+vdb+8vDzatWuHl5cXl112Gc8++ywXXnghgHO/+hxz1qxZBAUFOR8xMS4ehXPkclCmVycKyw36dgxk1Gm6Z5CIiEhjaZYOFgEBAaxfv57Vq1fz+OOPM23aNJYuXdrg402fPp28vDznY9++fY1XbEMc6XC7pigMgHsu6onFotYVERGRxlKvueJDQ0Ox2WxkZGRUWZ6RkUFkZGSt+1mtVuLj4wEYMGAA27ZtY9asWYwYMcK5X0ZGBlFRUVWOOWDAgBqP5+3tjbe3d31Kb1pHAssuRzSDuoQwokeYiwsSERFpXeoVWLy8vBg0aBBJSUmMGTMGMDvdJiUlceedd9b5OA6Hg7KyMgBiY2OJjIwkKSnJGVDy8/NZuXIlt912W33Ka3zlRbCv9iHbzs1S1+AFJDuiueeiHmpdERERaWT1vhvftGnTmDhxIoMHD2bo0KHMmTOHoqIiJk+eDMCECRPo2LEjs2bNAsz+JoMHDyYuLo6ysjK+/PJL3nrrLV544QUALBYLU6dO5bHHHqN79+7Exsby0EMPER0d7QxFLpN/EN46eQ1eR57bdezFWXGhTVqSiIhIW1TvwDJ27FiysrKYMWMG6enpDBgwgMWLFzs7zaampmK1HusaU1RUxO23305aWhq+vr706tWLt99+m7Fjxzq3ue+++ygqKuKWW24hNzeXc845h8WLF+Pj49MIH/EU2Dwhou8JN7E7DHZmFrDV0YVrLrukmQoTERFpW+o9D4s7cuU8LO+sTOWBjzfRKzKAxVPPbdafLSIi0pI12TwsUt2n6/cDMOYMTcEvIiLSVBRYTsGB3BJW7TkEwOj+0S6uRkREpPVSYDkFn204gGHA0Nj2dAz2dXU5IiIirZYCyyn4dP0BAK4coNYVERGRpqTA0kC7MgrYejAfD6uFS/tGnXwHERERaTAFlgY62royomcYIf5eJ9laREREToUCSwMYhsGnG8zRQVcM0OggERGRpqbA0gBrU3PZd6gEPy8bib3DXV2OiIhIq6fA0gCLjsy9Muq0SPy86j1ZsIiIiNSTAks9VdodfL7xIABXaHSQiIhIs1Bgqaflu7PJKSqng78X58TrRociIiLNQYGlnhYdGR10Wb8oPG06fSIiIs1B37j1UFJu5+st6YAmixMREWlOCiz18O22DIrK7XQK8WVg5xBXlyMiItJmKLDUw/FT8VssFhdXIyIi0nYosNRRbnE5y3ZmAjBGk8WJiIg0KwWWOvpyUzoVdoPeUYF0jwhwdTkiIiJtigJLHX1yZLI4dbYVERFpfgosdXAgt4RVKYcAuKK/AouIiEhzU2Cpg882mJ1th8a2JzrY18XViIiItD0KLHXwyXGjg0RERKT5KbCcxM6MArYdzMfTZuHSvlGuLkdERKRNUmA5iaNT8Z/XI4wQfy8XVyMiItI2KbCcgGEYfLrh6Oggzb0iIiLiKgosJ7A2NZd9h0rw87KR2DvC1eWIiIi0WR6uLsCdde3gxz8u601hWSW+XjZXlyMiItJmKbCcQId23tw8vJuryxAREWnzdElIRERE3J4Ci4iIiLg9BRYRERFxewosIiIi4vYUWERERMTtKbCIiIiI21NgEREREbenwCIiIiJuT4FFRERE3J4Ci4iIiLg9BRYRERFxewosIiIi4vYUWERERMTttYq7NRuGAUB+fr6LKxEREZG6Ovq9ffR7/ERaRWApKCgAICYmxsWViIiISH0VFBQQFBR0wm0sRl1ijZtzOBwcOHCAgIAALBZLox47Pz+fmJgY9u3bR2BgYKMeW6rT+W5eOt/NS+e7eel8N6+GnG/DMCgoKCA6Ohqr9cS9VFpFC4vVaqVTp05N+jMCAwP1C9+MdL6bl85389L5bl46382rvuf7ZC0rR6nTrYiIiLg9BRYRERFxewosJ+Ht7c3DDz+Mt7e3q0tpE3S+m5fOd/PS+W5eOt/Nq6nPd6vodCsiIiKtm1pYRERExO0psIiIiIjbU2ARERERt6fAIiIiIm5PgUVERETcngLLCcydO5euXbvi4+NDQkICq1atcnVJrcYPP/zA6NGjiY6OxmKx8Mknn1RZbxgGM2bMICoqCl9fXxITE9m1a5drim3hZs2axZAhQwgICCA8PJwxY8awY8eOKtuUlpZyxx130KFDB9q1a8c111xDRkaGiypu2V544QX69evnnO1z2LBhfPXVV871OtdN64knnsBisTB16lTnMp3zxvPPf/4Ti8VS5dGrVy/n+qY81wostXjvvfeYNm0aDz/8MGvXrqV///6MGjWKzMxMV5fWKhQVFdG/f3/mzp1b4/onn3ySZ555hnnz5rFy5Ur8/f0ZNWoUpaWlzVxpy7ds2TLuuOMOfvnlF5YsWUJFRQUXXXQRRUVFzm3++te/8tlnn/HBBx+wbNkyDhw4wNVXX+3CqluuTp068cQTT7BmzRp+/fVXLrjgAq688kq2bNkC6Fw3pdWrV/Piiy/Sr1+/Kst1zhvXaaedxsGDB52P5cuXO9c16bk2pEZDhw417rjjDud7u91uREdHG7NmzXJhVa0TYHz88cfO9w6Hw4iMjDT+/e9/O5fl5uYa3t7exrvvvuuCCluXzMxMAzCWLVtmGIZ5bj09PY0PPvjAuc22bdsMwFixYoWrymxVQkJCjJdfflnnugkVFBQY3bt3N5YsWWKcd955xt13320Yhn6/G9vDDz9s9O/fv8Z1TX2u1cJSg/LyctasWUNiYqJzmdVqJTExkRUrVriwsrYhJSWF9PT0Kuc/KCiIhIQEnf9GkJeXB0D79u0BWLNmDRUVFVXOd69evejcubPO9ymy2+0sWLCAoqIihg0bpnPdhO644w4uu+yyKucW9PvdFHbt2kV0dDTdunVj/PjxpKamAk1/rlvF3ZobW3Z2Nna7nYiIiCrLIyIi2L59u4uqajvS09MBajz/R9dJwzgcDqZOncrZZ59N3759AfN8e3l5ERwcXGVbne+G27RpE8OGDaO0tJR27drx8ccf06dPH9avX69z3QQWLFjA2rVrWb16dbV1+v1uXAkJCbz++uv07NmTgwcPMnPmTIYPH87mzZub/FwrsIi0IXfccQebN2+ucs1ZGl/Pnj1Zv349eXl5fPjhh0ycOJFly5a5uqxWad++fdx9990sWbIEHx8fV5fT6l1yySXO1/369SMhIYEuXbrw/vvv4+vr26Q/W5eEahAaGorNZqvWszkjI4PIyEgXVdV2HD3HOv+N68477+Tzzz/n+++/p1OnTs7lkZGRlJeXk5ubW2V7ne+G8/LyIj4+nkGDBjFr1iz69+/P008/rXPdBNasWUNmZiYDBw7Ew8MDDw8Pli1bxjPPPIOHhwcRERE6500oODiYHj16sHv37ib//VZgqYGXlxeDBg0iKSnJuczhcJCUlMSwYcNcWFnbEBsbS2RkZJXzn5+fz8qVK3X+G8AwDO68804+/vhjvvvuO2JjY6usHzRoEJ6enlXO944dO0hNTdX5biQOh4OysjKd6yYwcuRINm3axPr1652PwYMHM378eOdrnfOmU1hYSHJyMlFRUU3/+33K3XZbqQULFhje3t7G66+/bmzdutW45ZZbjODgYCM9Pd3VpbUKBQUFxrp164x169YZgDF79mxj3bp1xt69ew3DMIwnnnjCCA4ONj799FNj48aNxpVXXmnExsYaJSUlLq685bntttuMoKAgY+nSpcbBgwedj+LiYuc2t956q9G5c2fju+++M3799Vdj2LBhxrBhw1xYdct1//33G8uWLTNSUlKMjRs3Gvfff79hsViMb775xjAMnevmcPwoIcPQOW9M99xzj7F06VIjJSXF+Omnn4zExEQjNDTUyMzMNAyjac+1AssJPPvss0bnzp0NLy8vY+jQocYvv/zi6pJaje+//94Aqj0mTpxoGIY5tPmhhx4yIiIiDG9vb2PkyJHGjh07XFt0C1XTeQaM1157zblNSUmJcfvttxshISGGn5+fcdVVVxkHDx50XdEt2I033mh06dLF8PLyMsLCwoyRI0c6w4ph6Fw3h98HFp3zxjN27FgjKirK8PLyMjp27GiMHTvW2L17t3N9U55ri2EYxqm304iIiIg0HfVhEREREbenwCIiIiJuT4FFRERE3J4Ci4iIiLg9BRYRERFxewosIiIi4vYUWERERMTtKbCIiIiI21NgEREREbenwCIiIiJuT4FFRERE3N7/Axdod8bhE5FWAAAAAElFTkSuQmCC", - "text/plain": [ - "
" - ] - }, - "metadata": {}, - "output_type": "display_data" - } - ], - "source": [ - "plt.plot(history.history['acc'])\n", - "plt.plot(history.history['val_acc'])\n", - "plt.legend(['acc', 'val_acc'])\n", - "plt.title(f'Accuracy')\n", - "plt.show()" - ] - }, - { - "cell_type": "markdown", - "metadata": {}, - "source": [ - "## model testing\n", - "let's test how well the model perform" - ] - }, - { - "cell_type": "code", - "execution_count": 13, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "WARNING:tensorflow:No training configuration found in save file, so the model was *not* compiled. Compile it manually.\n" - ] - } - ], - "source": [ - "# reload\n", - "model = tf.keras.models.load_model('demo_models/emotions.keras', compile=False)" - ] - }, - { - "cell_type": "code", - "execution_count": 25, - "metadata": {}, - "outputs": [], - "source": [ - "def predict_emotions(txt, threshold=0.5):\n", - " # recall it is multi-class so we need to get all prediction above a threshold (0.5)\n", - " preds = model(tf.constant([txt]))[0]\n", - " out = 0\n", - " for i in range(NUM_CLASSES):\n", - " if preds[i] > threshold:\n", - " emotion_name = CLASSES[i]\n", - " emotion_prob = round(float(preds[i]) * 100, 1)\n", - " print(f\"{emotion_name} ({emotion_prob})%\")\n", - " out += 1\n", - " if not out:\n", - " print(\"neutral\") \n", - " " - ] - }, - { - "cell_type": "code", - "execution_count": 29, - "metadata": {}, - "outputs": [ - { - "name": "stdout", - "output_type": "stream", - "text": [ - "joy (91.6)%\n" - ] - } - ], - "source": [ - "txt = \"I enjoy having a good icecream\"\n", - "predict_emotions(txt)" - ] - } - ], - "metadata": { - "kernelspec": { - "display_name": ".env", - "language": "python", - "name": "python3" - }, - "language_info": { - "codemirror_mode": { - "name": "ipython", - "version": 3 - }, - "file_extension": ".py", - "mimetype": "text/x-python", - "name": "python", - "nbconvert_exporter": "python", - "pygments_lexer": "ipython3", - "version": "3.10.8" - }, - "orig_nbformat": 4, - "vscode": { - "interpreter": { - "hash": "bc86f0786348ca6b89e1e790af95528ab7136d16a08a53d7be83a40ce5119309" - } - } - }, - "nbformat": 4, - "nbformat_minor": 2 -} From 3e419a8c01fd62d1c309f5d3da18a0a6a37299a9 Mon Sep 17 00:00:00 2001 From: Marina Zhang <40069936+MarinaZhang@users.noreply.github.com> Date: Mon, 9 Oct 2023 16:24:31 -0700 Subject: [PATCH 4/4] Add train_tpu.ipynb notebook --- notebooks/train_tpu.ipynb | 551 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 551 insertions(+) create mode 100644 notebooks/train_tpu.ipynb diff --git a/notebooks/train_tpu.ipynb b/notebooks/train_tpu.ipynb new file mode 100644 index 0000000..fda4caf --- /dev/null +++ b/notebooks/train_tpu.ipynb @@ -0,0 +1,551 @@ +{ + "nbformat": 4, + "nbformat_minor": 0, + "metadata": { + "colab": { + "provenance": [] + }, + "kernelspec": { + "name": "python3", + "display_name": "Python 3" + }, + "language_info": { + "name": "python" + }, + "accelerator": "TPU" + }, + "cells": [ + { + "cell_type": "markdown", + "source": [ + "### Use RETVec on TPU\n", + "\n", + "You can run this notebook in Google Colab, where you can request a TPU.\n", + "\n", + "RETVec requires a slightly different setup on TPU, because TPUs do not support string tensors. Thus, we will split the default RETVecTokenizer layer into two parts -- one that converts strings into an integer representation which runs on CPU, and the remaining components of RETVec including the word embedding model which runs on TPU (as well as the rest of the model).\n", + "\n", + "We will use the same example as in the `train_retvec_model_tf.ipynb` notebook, where we train an emotion classifier." + ], + "metadata": { + "id": "KCdhdgLc-xCP" + } + }, + { + "cell_type": "code", + "source": [ + "# installing needed dependencies\n", + "try:\n", + " import retvec\n", + "except ImportError:\n", + " !pip install retvec # is retvec installed?\n", + "\n", + "try:\n", + " import datasets\n", + "except ImportError:\n", + " !pip install datasets # used to get the dataset\n", + "\n", + "try:\n", + " import matplotlib\n", + "except ImportError:\n", + " !pip install matplotlib\n", + "\n", + "try:\n", + " import tensorflow_text\n", + "except ImportError:\n", + " !pip install tensorflow_text" + ], + "metadata": { + "id": "niDxQ-1x-mx9" + }, + "execution_count": 1, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "import os\n", + "os.environ['TF_CPP_MIN_LOG_LEVEL'] = '1' # silence TF INFO messages\n", + "import tensorflow as tf\n", + "import numpy as np\n", + "from tensorflow.keras import layers\n", + "import tensorflow_text as text\n", + "from datasets import load_dataset\n", + "from matplotlib import pyplot as plt" + ], + "metadata": { + "id": "2Ovv0fvB-psR" + }, + "execution_count": 2, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "### Import RETVec layers we need for TPU\n", + "\n", + "Note that we do not import the RETVecTokenizer layer for TPU here, since TPUs do not support tf.string inputs." + ], + "metadata": { + "id": "6oLggdAk_a2B" + } + }, + { + "cell_type": "code", + "source": [ + "from retvec.tf.layers import RETVecEmbedding, RETVecIntToBinary, RETVecIntegerizer" + ], + "metadata": { + "id": "fWwzCVs7-rgT" + }, + "execution_count": 3, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "### Initialize TPU and TPU Strategy" + ], + "metadata": { + "id": "pRdgWcpXrliX" + } + }, + { + "cell_type": "code", + "source": [ + "resolver = tf.distribute.cluster_resolver.TPUClusterResolver(tpu='')\n", + "tf.config.experimental_connect_to_cluster(resolver)\n", + "# This is the TPU initialization code that has to be at the beginning.\n", + "tf.tpu.experimental.initialize_tpu_system(resolver)\n", + "print(\"All devices: \", tf.config.list_logical_devices('TPU'))\n", + "\n", + "strategy = tf.distribute.TPUStrategy(resolver)" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "Jrf9ViW6rkmE", + "outputId": "42c04ce9-b4de-47ae-d4d2-26c66cda78c5" + }, + "execution_count": 4, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "All devices: [LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:0', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:1', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:2', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:3', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:4', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:5', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:6', device_type='TPU'), LogicalDevice(name='/job:worker/replica:0/task:0/device:TPU:7', device_type='TPU')]\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "### Create dataset\n", + "\n", + "We are going to use the [Go Emotion](https://huggingface.co/datasets/go_emotions) dataset to create a mulit-class emotion classifier. https://ai.googleblog.com/2021/10/goemotions-dataset-for-fine-grained.html" + ], + "metadata": { + "id": "F9Rvip4S_wCO" + } + }, + { + "cell_type": "code", + "source": [ + "# downloading dataset\n", + "dataset = load_dataset('go_emotions')" + ], + "metadata": { + "id": "jxYGjVIR_6S6" + }, + "execution_count": 5, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "# get class name mapping and number of class\n", + "CLASSES = dataset['train'].features['labels'].feature.names\n", + "NUM_CLASSES = len(CLASSES)\n", + "print(f\"num classes {NUM_CLASSES}\")\n", + "print(CLASSES)" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "CJDELoVj_6WE", + "outputId": "0d027bb4-398a-49be-b891-a43167650e52" + }, + "execution_count": 6, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "num classes 28\n", + "['admiration', 'amusement', 'anger', 'annoyance', 'approval', 'caring', 'confusion', 'curiosity', 'desire', 'disappointment', 'disapproval', 'disgust', 'embarrassment', 'excitement', 'fear', 'gratitude', 'grief', 'joy', 'love', 'nervousness', 'optimism', 'pride', 'realization', 'relief', 'remorse', 'sadness', 'surprise', 'neutral']\n" + ] + } + ] + }, + { + "cell_type": "code", + "source": [ + "# preparing data\n", + "x_train = tf.constant(dataset['train']['text'], dtype=tf.string)\n", + "\n", + "# the one-hot requires a little more due to the multi-class nature of the dataset.\n", + "y_train = np.zeros((len(x_train),NUM_CLASSES))\n", + "for idx, ex in enumerate(dataset['train']['labels']):\n", + " for val in ex:\n", + " y_train[idx][val] = 1\n", + "\n", + "# test data\n", + "x_test = tf.constant(dataset['test']['text'], dtype=tf.string)\n", + "y_test = np.zeros((len(x_test),NUM_CLASSES))\n", + "for idx, ex in enumerate(dataset['test']['labels']):\n", + " for val in ex:\n", + " y_test[idx][val] = 1" + ], + "metadata": { + "id": "UDAQ-jsy_6YZ" + }, + "execution_count": 14, + "outputs": [] + }, + { + "cell_type": "markdown", + "source": [ + "### Pre-process dataset on CPU\n", + "\n", + "We use the `RETVecIntegerizer` class to convert the tf.strings input into integer tensors (on CPU). By default, the layer converts each word into it's UTF-8 codepoints (with max 16 characters per word).\n", + "\n", + "The `RETVecIntegerizer` has a method `integerize` which will encode a string tensor into its integer representation." + ], + "metadata": { + "id": "a7CZv1PUAIPg" + } + }, + { + "cell_type": "code", + "source": [ + "sequence_length = 128 # number of words per text, inputs will be padded or truncated to this length\n", + "word_length = 16 # max characters per word\n", + "\n", + "# initialize whitespace tokenizer\n", + "whitespace_tokenizer = text.WhitespaceTokenizer()\n", + "\n", + "# initialize RETVec integerizer\n", + "integerizer = RETVecIntegerizer()" + ], + "metadata": { + "id": "0lUZoLh1AV0Y" + }, + "execution_count": 15, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "# split text in dataset into words\n", + "x_train = whitespace_tokenizer.tokenize(x_train)\n", + "x_test = whitespace_tokenizer.tokenize(x_test)\n", + "\n", + "# convert from ragged tensor to tensor with pad/truncation to sequence_length\n", + "x_train = x_train.to_tensor(default_value=\"\", shape=(x_train.shape[0], sequence_length))\n", + "x_test = x_test.to_tensor(default_value=\"\", shape=(x_test.shape[0], sequence_length))\n", + "\n", + "# encode each word into their integer representation\n", + "x_train = integerizer.integerize(x_train)\n", + "x_test = integerizer.integerize(x_test)" + ], + "metadata": { + "id": "p4lhTH4vrhYC" + }, + "execution_count": 16, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "print('train input shape:', x_train.shape)" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "gw7QEnPKA0ZL", + "outputId": "b87bf75b-61f6-4d7b-afd8-1697b8873453" + }, + "execution_count": 17, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "train input shape: (43410, 128, 16)\n" + ] + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "### Create Model\n", + "\n", + "Now, we can create the model with RETVec which will run on TPU. To do this, we use the layers `RETVecIntToBinary` which will binarize the UTF-8 codepoints, and `RETVecEmbedding` which will call the embedding model and produce 256-dim float embeddings for each word. Then, we can build the rest of the model like usual and train." + ], + "metadata": { + "id": "EnAfdnUb__pt" + } + }, + { + "cell_type": "code", + "source": [ + "retvec_model_dir = 'gs://tensorflow/keras-applications/retvec-v1/' # currently colab TPU only supports gcs paths\n", + "batch_size = 256\n", + "epochs = 25\n", + "\n", + "# use TPU\n", + "with strategy.scope():\n", + " # input is int32 tensor with shape (sequence_length, word_length)\n", + " inputs = tf.keras.layers.Input(shape=(sequence_length, word_length), dtype=tf.int32)\n", + " x = RETVecIntToBinary(sequence_length=sequence_length, word_length=word_length)(inputs)\n", + " x = RETVecEmbedding(model=retvec_model_dir)(x)\n", + "\n", + " # build the rest of the model\n", + " x = layers.Bidirectional(layers.LSTM(64, return_sequences=True))(x)\n", + " x = layers.Bidirectional(layers.LSTM(64))(x)\n", + " outputs = layers.Dense(NUM_CLASSES, activation='sigmoid')(x)\n", + " model = tf.keras.Model(inputs, outputs)\n", + " model.summary()\n", + "\n", + " model.compile('adam', 'binary_crossentropy', ['acc'])\n", + " history = model.fit(x_train, y_train, epochs=epochs, batch_size=batch_size,\n", + " validation_data=(x_test, y_test))" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "LUzjZXZ--e5H", + "outputId": "0e19b6e4-b88a-40d9-8444-3adf5cecaa6b" + }, + "execution_count": 19, + "outputs": [ + { + "output_type": "stream", + "name": "stderr", + "text": [ + "WARNING:tensorflow:No training configuration found in save file, so the model was *not* compiled. Compile it manually.\n" + ] + }, + { + "output_type": "stream", + "name": "stdout", + "text": [ + "Model: \"model_1\"\n", + "_________________________________________________________________\n", + " Layer (type) Output Shape Param # \n", + "=================================================================\n", + " input_2 (InputLayer) [(None, 128, 16)] 0 \n", + " \n", + " ret_vec_int_to_binary_1 (R (None, 128, 16, 24) 0 \n", + " ETVecIntToBinary) \n", + " \n", + " ret_vec_embedding_1 (RETVe (None, 128, 256) 230144 \n", + " cEmbedding) \n", + " \n", + " bidirectional_2 (Bidirecti (None, 128, 128) 164352 \n", + " onal) \n", + " \n", + " bidirectional_3 (Bidirecti (None, 128) 98816 \n", + " onal) \n", + " \n", + " dense_1 (Dense) (None, 28) 3612 \n", + " \n", + "=================================================================\n", + "Total params: 496924 (1.90 MB)\n", + "Trainable params: 266780 (1.02 MB)\n", + "Non-trainable params: 230144 (899.00 KB)\n", + "_________________________________________________________________\n", + "Epoch 1/25\n", + "170/170 [==============================] - 42s 121ms/step - loss: 0.1708 - acc: 0.2827 - val_loss: 0.1474 - val_acc: 0.2959\n", + "Epoch 2/25\n", + "170/170 [==============================] - 11s 64ms/step - loss: 0.1478 - acc: 0.2955 - val_loss: 0.1443 - val_acc: 0.2959\n", + "Epoch 3/25\n", + "170/170 [==============================] - 11s 68ms/step - loss: 0.1420 - acc: 0.3297 - val_loss: 0.1360 - val_acc: 0.3575\n", + "Epoch 4/25\n", + "170/170 [==============================] - 11s 62ms/step - loss: 0.1354 - acc: 0.3607 - val_loss: 0.1314 - val_acc: 0.3717\n", + "Epoch 5/25\n", + "170/170 [==============================] - 11s 63ms/step - loss: 0.1315 - acc: 0.3757 - val_loss: 0.1273 - val_acc: 0.3893\n", + "Epoch 6/25\n", + "170/170 [==============================] - 11s 67ms/step - loss: 0.1277 - acc: 0.3958 - val_loss: 0.1245 - val_acc: 0.4024\n", + "Epoch 7/25\n", + "170/170 [==============================] - 11s 62ms/step - loss: 0.1243 - acc: 0.4169 - val_loss: 0.1205 - val_acc: 0.4315\n", + "Epoch 8/25\n", + "170/170 [==============================] - 11s 63ms/step - loss: 0.1206 - acc: 0.4334 - val_loss: 0.1178 - val_acc: 0.4468\n", + "Epoch 9/25\n", + "170/170 [==============================] - 11s 64ms/step - loss: 0.1179 - acc: 0.4447 - val_loss: 0.1156 - val_acc: 0.4516\n", + "Epoch 10/25\n", + "170/170 [==============================] - 11s 64ms/step - loss: 0.1157 - acc: 0.4528 - val_loss: 0.1136 - val_acc: 0.4655\n", + "Epoch 11/25\n", + "170/170 [==============================] - 11s 63ms/step - loss: 0.1138 - acc: 0.4591 - val_loss: 0.1126 - val_acc: 0.4608\n", + "Epoch 12/25\n", + "170/170 [==============================] - 12s 73ms/step - loss: 0.1115 - acc: 0.4691 - val_loss: 0.1102 - val_acc: 0.4691\n", + "Epoch 13/25\n", + "170/170 [==============================] - 11s 64ms/step - loss: 0.1097 - acc: 0.4768 - val_loss: 0.1098 - val_acc: 0.4728\n", + "Epoch 14/25\n", + "170/170 [==============================] - 11s 63ms/step - loss: 0.1081 - acc: 0.4831 - val_loss: 0.1085 - val_acc: 0.4806\n", + "Epoch 15/25\n", + "170/170 [==============================] - 11s 65ms/step - loss: 0.1064 - acc: 0.4896 - val_loss: 0.1068 - val_acc: 0.4854\n", + "Epoch 16/25\n", + "170/170 [==============================] - 11s 63ms/step - loss: 0.1052 - acc: 0.4951 - val_loss: 0.1067 - val_acc: 0.4824\n", + "Epoch 17/25\n", + "170/170 [==============================] - 11s 64ms/step - loss: 0.1037 - acc: 0.4998 - val_loss: 0.1065 - val_acc: 0.4822\n", + "Epoch 18/25\n", + "170/170 [==============================] - 11s 65ms/step - loss: 0.1022 - acc: 0.5053 - val_loss: 0.1046 - val_acc: 0.4918\n", + "Epoch 19/25\n", + "170/170 [==============================] - 11s 63ms/step - loss: 0.1010 - acc: 0.5102 - val_loss: 0.1047 - val_acc: 0.4911\n", + "Epoch 20/25\n", + "170/170 [==============================] - 11s 66ms/step - loss: 0.0997 - acc: 0.5140 - val_loss: 0.1033 - val_acc: 0.4973\n", + "Epoch 21/25\n", + "170/170 [==============================] - 11s 65ms/step - loss: 0.0987 - acc: 0.5198 - val_loss: 0.1051 - val_acc: 0.4924\n", + "Epoch 22/25\n", + "170/170 [==============================] - 11s 62ms/step - loss: 0.0978 - acc: 0.5210 - val_loss: 0.1030 - val_acc: 0.5027\n", + "Epoch 23/25\n", + "170/170 [==============================] - 12s 69ms/step - loss: 0.0968 - acc: 0.5266 - val_loss: 0.1023 - val_acc: 0.5001\n", + "Epoch 24/25\n", + "170/170 [==============================] - 11s 67ms/step - loss: 0.0954 - acc: 0.5309 - val_loss: 0.1021 - val_acc: 0.5045\n", + "Epoch 25/25\n", + "170/170 [==============================] - 11s 62ms/step - loss: 0.0948 - acc: 0.5334 - val_loss: 0.1025 - val_acc: 0.5025\n" + ] + } + ] + }, + { + "cell_type": "code", + "source": [ + "# visualize the training curves\n", + "plt.plot(history.history['acc'])\n", + "plt.plot(history.history['val_acc'])\n", + "plt.legend(['acc', 'val_acc'])\n", + "plt.title(f'Accuracy')\n", + "plt.show()" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/", + "height": 452 + }, + "id": "fLYBe3Yvsdqq", + "outputId": "9199edc4-7c8e-464b-9a6c-7a8e0610930e" + }, + "execution_count": 20, + "outputs": [ + { + "output_type": "display_data", + "data": { + "text/plain": [ + "
" + ], + "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjEAAAGzCAYAAADe/0a6AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABigElEQVR4nO3deVxVdf7H8de97CCgiKyiIu57bqipWVpaaa6lbS6ZTevUOG3WpNkyzq9mGrMsy8kWszTLmkqzjNHSMjXNXUkRxQ0QlH2/9/z+OIoSqKDA5cL7+XjcB+ee7X7u9Spvv+d7vl+LYRgGIiIiIk7G6ugCRERERC6FQoyIiIg4JYUYERERcUoKMSIiIuKUFGJERETEKSnEiIiIiFNSiBERERGnpBAjIiIiTkkhRkRERJySQoyIiIg4JYUYEbmgN954A4vFQnR0tKNLEREpwaK5k0TkQq688kqOHTvGwYMH2bdvHy1atHB0SSIigFpiROQC4uPj+fnnn3nllVdo1KgRixYtcnRJZcrOznZ0CSLiAAoxInJeixYtokGDBtx4442MGTOmzBCTlpbGX/7yF5o1a4aHhweNGzdm/PjxpKSkFO+Tl5fHs88+S6tWrfD09CQ0NJRRo0YRFxcHwJo1a7BYLKxZs6bEuQ8ePIjFYuG9994rXjdx4kTq1atHXFwcN9xwA76+vtx+++0ArF27lptvvpkmTZrg4eFBREQEf/nLX8jNzS1V9969e7nlllto1KgRXl5etG7dmqeffhqA1atXY7FY+Pzzz0sd99FHH2GxWFi/fn2FP08RqVyuji5ARGquRYsWMWrUKNzd3bn11lt588032bRpEz169AAgKyuLfv36sWfPHu666y66du1KSkoKX375JUeOHCEwMBCbzcbQoUOJiYlh3LhxPPzww2RmZrJq1Sp27txJVFRUhesqKipi8ODB9O3bl3/+8594e3sDsHTpUnJycrjvvvto2LAhGzdu5LXXXuPIkSMsXbq0+Pjt27fTr18/3NzcuOeee2jWrBlxcXF89dVXvPjiiwwYMICIiAgWLVrEyJEjS30mUVFR9O7d+zI+WRGpFIaISBl+/fVXAzBWrVplGIZh2O12o3HjxsbDDz9cvM/06dMNwFi2bFmp4+12u2EYhrFgwQIDMF555ZXz7rN69WoDMFavXl1ie3x8vAEY7777bvG6CRMmGIDx5JNPljpfTk5OqXWzZs0yLBaLcejQoeJ1/fv3N3x9fUusO7cewzCMadOmGR4eHkZaWlrxuuTkZMPV1dWYMWNGqdcRkeqny0kiUqZFixYRHBzM1VdfDYDFYmHs2LEsXrwYm80GwGeffUbnzp1LtVac2f/MPoGBgTz00EPn3edS3HfffaXWeXl5FS9nZ2eTkpJCnz59MAyD3377DYATJ07w448/ctddd9GkSZPz1jN+/Hjy8/P59NNPi9ctWbKEoqIi7rjjjkuuW0Qqj0KMiJRis9lYvHgxV199NfHx8ezfv5/9+/cTHR1NUlISMTExAMTFxdGhQ4cLnisuLo7WrVvj6lp5V69dXV1p3LhxqfUJCQlMnDiRgIAA6tWrR6NGjbjqqqsASE9PB+DAgQMAF627TZs29OjRo0Q/oEWLFtGrVy/doSVSQ6hPjIiU8r///Y/jx4+zePFiFi9eXGr7okWLuO666yrt9c7XInOmxeePPDw8sFqtpfa99tprOXnyJE888QRt2rTBx8eHo0ePMnHiROx2e4XrGj9+PA8//DBHjhwhPz+fX375hddff73C5xGRqqEQIyKlLFq0iKCgIObOnVtq27Jly/j888+ZN28eUVFR7Ny584LnioqKYsOGDRQWFuLm5lbmPg0aNADMO53OdejQoXLXvGPHDn7//Xfef/99xo8fX7x+1apVJfZr3rw5wEXrBhg3bhxTp07l448/Jjc3Fzc3N8aOHVvumkSkaulykoiUkJuby7Jlyxg6dChjxowp9XjwwQfJzMzkyy+/ZPTo0Wzbtq3MW5GN0+Nojh49mpSUlDJbMM7s07RpU1xcXPjxxx9LbH/jjTfKXbeLi0uJc55ZfvXVV0vs16hRI/r378+CBQtISEgos54zAgMDuf766/nwww9ZtGgRQ4YMITAwsNw1iUjVUkuMiJTw5ZdfkpmZyU033VTm9l69ehUPfPfRRx/x6aefcvPNN3PXXXfRrVs3Tp48yZdffsm8efPo3Lkz48eP54MPPmDq1Kls3LiRfv36kZ2dzffff8/999/P8OHD8ff35+abb+a1117DYrEQFRXF119/TXJycrnrbtOmDVFRUTz66KMcPXoUPz8/PvvsM06dOlVq3zlz5tC3b1+6du3KPffcQ2RkJAcPHmT58uVs3bq1xL7jx49nzJgxADz//PPl/yBFpOo58tYoEal5hg0bZnh6ehrZ2dnn3WfixImGm5ubkZKSYqSmphoPPvigER4ebri7uxuNGzc2JkyYYKSkpBTvn5OTYzz99NNGZGSk4ebmZoSEhBhjxowx4uLiivc5ceKEMXr0aMPb29to0KCB8ac//cnYuXNnmbdY+/j4lFnX7t27jUGDBhn16tUzAgMDjSlTphjbtm0rdQ7DMIydO3caI0eONOrXr294enoarVu3Np555plS58zPzzcaNGhg+Pv7G7m5ueX8FEWkOmjuJBGRCygqKiIsLIxhw4bxzjvvOLocETmH+sSIiFzAF198wYkTJ0p0FhaRmkEtMSIiZdiwYQPbt2/n+eefJzAwkC1btji6JBH5A7XEiIiU4c033+S+++4jKCiIDz74wNHliEgZ1BIjIiIiTkktMSIiIuKUFGJERETEKdWKwe7sdjvHjh3D19f3smbFFRERkepjGAaZmZmEhYWVmg+tPGpFiDl27BgRERGOLkNEREQuweHDh8ucmf5iakWI8fX1BcwPwc/Pz8HViIiISHlkZGQQERFR/Hu8ompFiDlzCcnPz08hRkRExMlcalcQdewVERERp6QQIyIiIk5JIUZEREScUq3oE1MehmFQVFSEzWZzdCl1nouLC66urrodXkRELkudCDEFBQUcP36cnJwcR5cip3l7exMaGoq7u7ujSxERESdV60OM3W4nPj4eFxcXwsLCcHd3VwuAAxmGQUFBASdOnCA+Pp6WLVte0gBHIiIitT7EFBQUYLfbiYiIwNvb29HlCODl5YWbmxuHDh2ioKAAT09PR5ckIiJOqM78F1j/269Z9OchIiKXS79JRERExCkpxIiIiIhTUogRERERp6QQIyIiIk6p1t+dJCIiIhVXaLMTn5LNnuMZ7DmeiY+7Cw8NbOnoskqokyHGMAxyCx0zcq+Xm0uFxqlZuXIlL7zwAjt37sTFxYXevXvz6quvEhUVBcCRI0d47LHH+Pbbb8nPz6dt27bMnTuX6OhoAL766iuee+45duzYQb169ejXrx+ff/55lbw3ERFxTilZ+ew9nsneRDOw7Dmewf7kLAps9uJ9mgR4K8TUBLmFNtpN/9Yhr737ucF4u5f/Y8/Ozmbq1Kl06tSJrKwspk+fzsiRI9m6dSs5OTlcddVVhIeH8+WXXxISEsKWLVuw280v3fLlyxk5ciRPP/00H3zwAQUFBaxYsaKq3pqIiNRwBUV24k5klQgrexMzOZGZX+b+Pu4utAn1o02IL+3C/DAMo0YNGFsnQ4wzGT16dInnCxYsoFGjRuzevZuff/6ZEydOsGnTJgICAgBo0aJF8b4vvvgi48aNY+bMmcXrOnfuXD2Fi4iIQ6XlFLDtSDp7TweVM60rRXaj1L4WCzRr6EObEF/ahPjRJtSXdqF+hNf3wmqtOaHlj+pkiPFyc2H3c4Md9toVsW/fPqZPn86GDRtISUkpbmVJSEhg69atXHHFFcUB5o+2bt3KlClTLrtmERFxDodSs1m1O4nvdifx68GTlJFX8PV0pe3poNL2dCtLq2BffDycLxI4X8WVwGKxVOiSjiMNGzaMpk2bMn/+fMLCwrDb7XTo0IGCggK8vLwueOzFtouIiHOz2w12HE3nu92JrNqdxO9JWSW2Nw/0oW2YH21Pt7C0DfMjzN+zRl0SuhzO8Zu8jkpNTSU2Npb58+fTr18/ANatW1e8vVOnTvznP//h5MmTZbbGdOrUiZiYGCZNmlRtNYuISNXKL7Lxc1wq3+9O4vs9SSRlnO3P4mK1EB0ZwLXtghnUNpiIgNo9Z6BCTA3WoEEDGjZsyNtvv01oaCgJCQk8+eSTxdtvvfVW/v73vzNixAhmzZpFaGgov/32G2FhYfTu3ZsZM2YwcOBAoqKiGDduHEVFRaxYsYInnnjCge9KREQqKj2nkNWxyXy3O5EfYk+QXXD2DlsfdxcGtA7i2nbBXN06CH9vNwdWWr0UYmowq9XK4sWL+fOf/0yHDh1o3bo1c+bMYcCAAQC4u7vz3Xff8de//pUbbriBoqIi2rVrx9y5cwEYMGAAS5cu5fnnn+cf//gHfn5+9O/f34HvSEREyuvIqRxW7U5i1e4kNsSfxHZOB5cgXw8GtQvm2nbB9IlqiIdrxfpb1hYWwzDK6PbjXDIyMvD39yc9PR0/P78S2/Ly8oiPjycyMhJPT08HVSh/pD8XEZHS9idn8uW246zancSe4xkltrUKrse17YK5tl0IncL9a/RdQ+V1od/f5aGWGBEREQfKK7SxfPtxFm9KYNPBU8XrrRbo3iyA6063uDRt6OPAKmsmhRgREREHiE3M5OONCSzbcoSMvCLA7Jg7oFUjhnQIYWDbYAJ83B1cZc2mECMiIlJNcgtsfL39GB9vTGBLQlrx+vD6XozrEcHN3SMI8dcl9vJSiBEREaliu49l8PHGBL747SiZ+Wari6vVwqC2wdwa3YS+LQJxqQV9XKqbQoyIiEgVyM4v4uvtx/ho42G2HU4rXt8kwJtxPSMY060xQb5qdbkcCjEiIiKVaOfRdD7amMCXW4+RdbrVxc3FwnXtQri1ZxP6RDWsFXcW1QQKMSIiIpcpK7+IL7eafV12HE0vXt+soTe39mzC6G6NCazn4cAKayeFGBERkTLY7Aancgo4mV1Aapb582R2PqnZp9dlF3Dy9PqEkznkFpqj6Lq7WBnSIYRxPSPo3bxhrZmnqCZSiBERkTonr9DGD7+f4FhabqlAkpqdz8nsAtJyC6nIcLBRjXy4tWcTRnVtrFujq4lCTC3WrFkzHnnkER555BFHlyIiUiMkpObw4YZDfPLrYdJyCst1TH1vNwJ83Gno406AjzsBPh7Fyw3rmT+D/TxpGVRPrS7VTCFGRERqNbvd4Id9J1i4/hCrY5OLW1fC63vRJaL+6WByNpCYgcWDAB93Gni74epidewbkPNSiBERkVopPaeQpZsPs/CXQxxKzSle379VI8b3asrVbYI0NouTq5vx0jCgINsxj3JeYH377bcJCwvDbreXWD98+HDuuusu4uLiGD58OMHBwdSrV48ePXrw/fffX/JH8sorr9CxY0d8fHyIiIjg/vvvJysrq8Q+P/30EwMGDMDb25sGDRowePBgTp0y5/mw2+289NJLtGjRAg8PD5o0acKLL754yfWIiFyqnUfTeeLT7UTP+p4Xlu/hUGoOvp6uTO4byepHB/DBXT0Z1C5YAaYWqJstMYU58Pcwx7z2U8fA/eKTeN1888089NBDrF69moEDBwJw8uRJVq5cyYoVK8jKyuKGG27gxRdfxMPDgw8++IBhw4YRGxtLkyZNKlyW1Wplzpw5REZGcuDAAe6//34ef/xx3njjDQC2bt3KwIEDueuuu3j11VdxdXVl9erV2Gxmb/xp06Yxf/58/v3vf9O3b1+OHz/O3r17K1yHiMilKCiy883O43yw/hCbD52dRLFNiC8T+jRjeJcwvN3r5q+82sxiGBXpe10zXWgq77y8POLj44mMjMTT8/TIiAXZNT7EAIwYMYKGDRvyzjvvAGbrzMyZMzl8+DBWa+lGtA4dOnDvvffy4IMPApfXsffTTz/l3nvvJSUlBYDbbruNhIQE1q1bV2rfzMxMGjVqxOuvv87dd99drvOX+eciIlJBx9Jy+WhDAos3JZCSVQCYw/lf3zGU8b2b0r1pA3W2rcEu9Pu7POpmLHXzNsOEo167nG6//XamTJnCG2+8gYeHB4sWLWLcuHFYrVaysrJ49tlnWb58OcePH6eoqIjc3FwSEhIuqazvv/+eWbNmsXfvXjIyMigqKiIvL4+cnBy8vb3ZunUrN998c5nH7tmzh/z8/OIWIxGRqmQYBuvjUvlg/SFW7UnCZjf/Lx7s58FtPZtya88Igvz0n6O6oG6GGIul3K0hjjRs2DAMw2D58uX06NGDtWvX8u9//xuARx99lFWrVvHPf/6TFi1a4OXlxZgxYygoKKjw6xw8eJChQ4dy33338eKLLxIQEMC6deuYPHkyBQUFeHt74+Xldd7jL7RNRKQyGIZB3Ils1sQms3jTYfYnn+2z16t5AON7N+PadsG46U6iOqVuhhgn4enpyahRo1i0aBH79++ndevWdO3aFTA72U6cOJGRI0cCkJWVxcGDBy/pdTZv3ozdbudf//pX8WWqTz75pMQ+nTp1IiYmhpkzZ5Y6vmXLlnh5eRETE1Puy0kiIhdzIjOfn+NSWLcvhXX7Uzienle8zdvdhVFdw7mzVzNah/g6sEpxJIWYGu72229n6NCh7Nq1izvuuKN4fcuWLVm2bBnDhg3DYrHwzDPPlLqTqbxatGhBYWEhr732GsOGDeOnn35i3rx5JfaZNm0aHTt25P777+fee+/F3d2d1atXc/PNNxMYGMgTTzzB448/jru7O1deeSUnTpxg165dTJ48+bLev4jUHbkFNjbEp/LT/hTW7kthb2Jmie3urlZ6NGvAde1CGNk1HD9PNwdVKjWFQkwNd8011xAQEEBsbCy33XZb8fpXXnmFu+66iz59+hSHiIyMjEt6jc6dO/PKK6/wf//3f0ybNo3+/fsza9Ysxo8fX7xPq1at+O6773jqqafo2bMnXl5eREdHc+uttwLwzDPP4OrqyvTp0zl27BihoaHce++9l/fmRaRWs9kNdhxNPx1aTrDlUBoFtpL/GWsf5kffFoH0bRlIj2YBeLq5OKhaqYnq5t1J4nD6cxGpmw6lZrN2Xwo/7U/h57hU0nNLDv0fXt+rOLT0iWpIQ838XKvp7iQREamx8ots/BB7gtWxyazbn8Lhk7kltvt6utInquHp4NKIZg29dUu0lJtCTB2waNEi/vSnP5W5rWnTpuzatauaKxKR2qzIZufnuFS+3HaMb3clkplXVLzNzcXCFU0a0O90a0vHcH/NTSSXTCGmDrjpppuIjo4uc5ubmzrGicjls9sNNiec4qttx1ix43jxwHMAIX6eXN8xhP4tG9EzMgAfD/3qkcqhb1Id4Ovri6+vbkEUkcplGAa7jmXw1bZjfLXtGMfOuQU6wMedGzqGMKxTGD2aBWDVPEVSBepMiKkF/ZdrFf15iDivuBNZfLn1GF9tP8aBE9nF6309XLmufQjDOodyZYtADTwnVa7Wh5gzl0tycnI0smwNkpOTA+hyloizOJqWy1fbjvHl1mPsPn52OAcPVysD2wZxU+cwBrQO0i3QUq1qfYhxcXGhfv36JCcnA+DtrZ7vjmQYBjk5OSQnJ1O/fn1cXPQPnkhNdSIznxU7jvPltmMlZoZ2tVro1zKQm7qEcW27EOqpj4s4SJ345oWEhAAUBxlxvPr16xf/uYhIzVFos/O/vcks3pjAD7+f4PTcilgsEB0ZwE2dw7m+QwgNfNwdW6gIdSTEWCwWQkNDCQoKorCw8OIHSJVyc3NTC4xIDXMoNZslmw6zdPMRTmTmF6/vHFGfmzqHcWPHUEL8NTClU8hKhr3LIfYbyEuHgObQsDkEREHDKPOnRz1HV1kp6kSIOcPFxUW/PEVETssvsvHtriSWbErgp/2pxesD67kzultjxnaPoHmj2vHLrtY7eQD2fA17v4bDG4Fzbp44/Evp/esFnw41fwg3Ac3B3bvayr5cdSrEiIgI7E/O5OONh1m25QincszWaYsF+rdsxLgeEQxsG4y7q+4sKrf8TEjYAL4h0Kg1uFTDDQuGAYnbzwaX5N0lt4d3gzZDoX4TOBkPJ+MgNc78mZMKWUnmI+Hn0uf2DTsdapqfDTcNo6BBJLjVrNa4Swoxc+fO5eWXXyYxMZHOnTvz2muv0bNnzzL3fe+995g0aVKJdR4eHuTlnR1PwDAMZsyYwfz580lLS+PKK6/kzTffpGXLlpdSnoiI/EFugY3lO46zeGMCv57TSTfU35Obu0dwS/fGNG7gPP8DrzGOboZP74JTB83nVjcIagPBHSGkAwR3gJCO4B1w+a9lt0HCevNS0d6vIS3h7DarKzTrawaX1jeAf/j5z5ObdjrUHCgZblLjIC8NMo+Zj4NrSx7n0wge23/576MSVTjELFmyhKlTpzJv3jyio6OZPXs2gwcPJjY2lqCgoDKP8fPzIzY2tvj5H+8Oeumll5gzZw7vv/8+kZGRPPPMMwwePJjdu3drckARkcuw61g6izce5outR4uH/3exWrimTRC39ozgqlZBuGgguoqz2+GXufD9s2AvAu+GYCuE/AxI3GE+tp2zv2/YOaGmgxlyGkaB9SJdHArz4MBqM7TEfmO2opzh6gUtBkLbYdBqMHg1KF/tXvXNlprwbqW35ZwsGWqKfx4wW2JqmArPYh0dHU2PHj14/fXXAbDb7URERPDQQw/x5JNPltr/vffe45FHHiEtLa3M8xmGQVhYGH/961959NFHAUhPTyc4OJj33nuPcePGXbSmy50FU0SkNsnKL+LLrcdYvCmB7UfSi9dHBHgxrkcTxnRrTLCf/oN4ybJT4Iv7YN935vN2w2HYHPD0N1tHknZC4k5IOh1mzrTS/JGrFwS3O9taE9wBgtsDBvz+Hez9CvZ9D4VnBxTEqwG0uh7aDoXmV1df/xXDgILsSu8QXK2zWBcUFLB582amTZtWvM5qtTJo0CDWr19/3uOysrJo2rQpdrudrl278ve//5327dsDEB8fT2JiIoMGDSre39/fn+joaNavX19miMnPzyc//2zv+YyMjFL7iIjUJTa7wcb4k3zx21G+2n6MnAIbYE64eF37EG7t0YQ+UQ01/P/liv8RPpsCWYng6glDZkG3SWanIoAGTc1HmxvPHpOXYfZZSdxxNuAk74bCHPNy1NHNJV/D4gKG7exzv8bm+doOhSZ9wMUB3Vktlhp5R1OFPomUlBRsNhvBwcEl1gcHB7N3794yj2ndujULFiygU6dOpKen889//pM+ffqwa9cuGjduTGJiYvE5/njOM9v+aNasWcycObMipYuI1DpnJl38etsxVuxMLHFrdPNGPtzaowmjuobTsJ6HA6usJWxF8ONL8MNLgAGBreHmd0+3nFyEpx806WU+zrDbzA63STtOt9rsNENOxlEzwDRqY/ZvaXMjhF1xNiRJCVUe53r37k3v3r2Ln/fp04e2bdvy1ltv8fzzz1/SOadNm8bUqVOLn2dkZBAREXHZtYqI1HSGYfDb4TS+3nacFTuOk5hx9iYJfy83BrcPZky3CHo0a6DRyStL+lH47O6zd/JccSdc/3/g7nPp57S6QGAL89F+5Nn1OSfNFhr/xpdXcx1RoRATGBiIi4sLSUlJJdYnJSWVe/RVNzc3rrjiCvbvN3s4nzkuKSmJ0NDQEufs0qVLmefw8PDAw0P/sxCRusEwDLYfSWf5juMs336co2m5xdvOTLo4tJM56WKtvzU6O9W8hdmzmvo/xn5j9n/JPQXuvjBsNnQcU3Wv5x0AVMKdTHVEhUKMu7s73bp1IyYmhhEjRgBmx96YmBgefPDBcp3DZrOxY8cObrjhBgAiIyMJCQkhJiamOLRkZGSwYcMG7rvvvoqUJyJSaxiGwa5jGcXBJeFkTvE2H3cXrm0XzI2dwujfKhAP11o+iKdhmLcW//w6xK4wQ0zL66DTLdBycNWMXVKUb9559Msb5vPQLjBmgXlHkdQYFb6cNHXqVCZMmED37t3p2bMns2fPJjs7u3gsmPHjxxMeHs6sWbMAeO655+jVqxctWrQgLS2Nl19+mUOHDnH33XcD5u3WjzzyCC+88AItW7YsvsU6LCysOCiJiNQFhmEQm5TJ8u3H+Xr7ceJTzt6V4uXmwsC2QQztFFp3Zou2FcLu/8L61+HYb+esLzBvOd77NXj4Q7th0PEWc5yUi92yXB6pcfDpJDh++h7pXg/AoBngqisANU2FQ8zYsWM5ceIE06dPJzExkS5durBy5crijrkJCQlYrWebM0+dOsWUKVNITEykQYMGdOvWjZ9//pl27doV7/P444+TnZ3NPffcQ1paGn379mXlypUaI0ZE6oT9yVl8vf0YX28/zv7krOL1Hq5WrmkTxI2dQrmmTRDe7nVkkPW8dNj8Pmx4CzKOmOtcPaHzrdDrfjPE7PgEdnxqdoT97UPz4RsKHUZDx5shtPOldYbdvhS+fgQKssArAEa8Ca2HVOrbk8pT4XFiaiKNEyMiziY7v4jl24+zeFMCWxLSite7u1i5qnUjhnYKZWDbYOp51JHgAuZ4Khvegi0fmCECzFFie94D3e8Cn8CS+9vtZmfbHUth1xfmaLNnBLYyW2c6joGAcgzSVpANKx6HrR+az5teCaPmX3jkW7lsl/v7WyFGRKSaGIbBtiPpLNmUwJdbj5F9eiwXV6uFfi0DGdopjGvbB+PnWQ1z79QkhzfB+tdgz1dg2M11jdpC7wfMVpXy9Hkpyof938P2T+D3lVB09q4tGvcwA02HUaWDEJi3OH86CVJ+B4sV+j8OVz1eOZem5IIUYlCIEZGaLS2ngM9/O8qSTYfZm5hZvD4y0IexPSIY1TWcIN86dvncbjNDy/q5cGTj2fVR15jhJWrgpY+NkpdhnnvHUoj/4WwwsriY5+90izm/kLsP/PoOrHwKbPnm5ahR8yGy3+W/PykXhRgUYkSk5rHbDX6JT2XJpsN8szORgiLzF6mHq5UbOoYyrkcEPSMD6t5YLvmZZv+VX96EtEPmOhd3s6Wk9/3lGzyuIjITYecysw/NuZ2D3bzNS07Ht5rPW15n9n8pq6VGqoxCDAoxIlJzJGfksXTzET759TCHUs/eFt021I9be0YwvHM4/t41/HKRrci8tTh2hdmh1qMeePiBez3w8DWfu59e53F63bnP3euZrRznBrT0I2Z/l83vQ/7p+Zy8AqDHZOgxBXyDy66lMqXsM1tntn8Cp+LNdVY3GPSs2WHYWsvH2KmBFGJQiBERxyqy2VkTe4LFmw6zOjYZm938Z7WehyvDu4QxrkcTOoT7OUerS8p++OJeOLLp8s5jsZ4NPe4+5izIdnMWbRq2MEND51urbwLDcxkGHN0C+1dBqyEQ1qX6axCgmieAFBGRsxJSc1jyawJLfz1C8jnzFnVv2oCxPSK4sVOo89wWbbfDpvmwagYU5ZqtKlc/ZbaW5GeYdwvlZ5mXgwqyzHUlnmeefp4BGGY/lPyM089Pa9bP7O/ScrBjWz0sFmjczXyIU3OSv10iIjWDYRis3ZfC/LUHWLsvpXh9gI87o7uGM7ZHBC2CfB1Y4SVIOwz/vd+coRmg+QAYPvfS5u8xDHPun+KAk2n+9AmCoDaVWraIQoyISDkU2uws336ct348wJ7jZuuCxQL9WjZiXI8IBrUNdr55iwwDtn4EK580W0xcveC656H75EtvKbFYzMtH7j7V089F6jSFGBGRC8jOL2LJpsO8sy6+eOJFb3cXxvaI4K4rI4kIcECfjsqQlQxfPWx23gVo3BNGztPcQOJUFGJERMpwIjOf938+yMJfDpGeWwhAYD13JvZpxh29mlLf293BFV6G3f+Fr/8COanm3TlXPwVXPqzB3cTpKMSIiJwjPiWb+WsP8OnmI8Vju0QG+nB3v0hGd23s3BMv5p4yh9bf8Yn5PLij2foS0sGxdYlcIoUYERHgt4RTvPXDAb7dnciZgSc6R9Tnvquac227EFysf7g9Oi8dsICnkwzrsP97+O9DkHnMvP2571S46glwdeIWJanzFGJEpM6y2w1Wxybz1o8H2Bh/snj9wDZB3NO/eekRdXNPwd7lsPMzOHB6OPvAVhDeDcK7mo/gDuDq4YB3cx75WbDqGfh1gfm8YQsY+RY07u7YukQqgUKMiNQ5BUV2/rv1KG//eIB9yeZsyW4uFoZ3Ceee/s1pFXzOLdL5mRD7jTl0/f7vwV5Y8mQpseZj20fmcxd3COl4OticfgREOWZclEPrzYHrTh00n0ffCwNnOGaAOZEqoBAjInVGZl4hH21IYMFP8SRlmIPT1fNw5fboJky6MpIQ/9OTMBbkwL7vzBaXfd+VnBE5qB20H2XOiOzhB8e2wNHNpx9bIPfk2ednePhB2BUlg41faNW90cI8WP0i/PwaYIB/hDnuS/Orqu41RRxA0w6ISK13NC2X936K5+ONh8nKN4e+D/L14K6+kdwW3QQ/Tzcoyof9MbBrGexdAYXZZ08QEAUdRpvBJajt+V/IMMxWjzOB5uhmOL7NHAH3j3zDzl6CCusK3g3B6mreIWRxMX8WL59Zb/3DPmeWz7nkdWwrfH4vnNhjPu9yBwz5O3j6X/bnKFLZNHcSCjEiUrYdR9KZv/YAy3ccL57PqEVQPe7p15zhV4ThYbGbfVt2LYM9X5+dmBDAv4kZWjqMgpBOJYNCRdiKzEBxbmtN8m6zP02lsZwNNEX5gAE+jWDYHGhzQyW+jkjlUohBIUZEzjrTWXf+2gP8cuBsZ93ezRsypX8kA1o0xHr4Z/NS0e4vzcs/Z/iGQvuRZqtLeLdLDy4XU5BtttCc21pTkAV2Gxg282fxcpG5TAX+qW57Ewz9N/gEVk39IpVEE0CKiAB5hTaWbTnKO+sOEHfCvBTkarUwtFMod/drToeGwE9z4OuFkJV09kDvQGg/wuzn0qR39XTAdfeBpn3MR3nZ7WcDzrnhxrCfXbYXmXdG+YZUXe0iNYhCjIg4tdSsfBb+coiF6w+Rml0AgK+HK7dGN2Fin2aE+brCr+/Con+YI9QCeNaHtsPMFpdm/cDFCf4ptFoBK7i4OboSkRrDCf7mioiUFncii/+sjWfZliPknx5ZN7y+F5OubMbYHhH4erjC3q/hgxlwMs48KLAVXP00tL5Bg7yJ1AIKMSLiNAzDYEP8Sf6z9gDf70kuXt+psT9392vODR1CcHWxwuFN8N3f4PAv5g4+jWDANOg6wTlaXUSkXPS3WURqvEKbnRU7jvOftfHsOGreQWSxwMA2wUzpF3l2ZN2TB+D7mbD7C/NAVy/o86A5uaGH7/lfQESckkKMiNRYhTY7SzYd5s01cRxNM8da8XC1MrpbYyb3jSSqUT1zx5yT8OPLsHH+6RF1LdDldrjmafALc9wbEJEqpRAjIjWOYRis2p3EP1bu5cDpO40a+rgzvncz7ujVhIb1Ts9NVJgHG9+CH/91doyXqIFw7XOamVmkDlCIEZEa5beEU8xasZeNB83xWwJ83PnzNS0Y17MJnm4u5k52uznOS8xzkJ5grgvuYIaXFgMdVLmIVDeFGBGpEQ6lZvPSyliW7zgOmJeN7u4XyZ+uijKnBTgjfq3Zaff4VvO5bxhc8zfoPM4csVZE6gyFGBFxqJPZBcyJ2ceiDYcotBlYLDCma2OmXteKUH+vszueiIVV0+H3leZzd1/o+wj0ul+zMovUUQoxIuIQeYU2FvwUz5ur48g8PSnjVa0a8eT1bWgbenr4ccOA5D2w8W3Y8oE5Uq3FBbpPgquehHqNHPgORMTRFGJEpFrZ7Aaf/3aUf30Xy/H0PADahfrx1A1t6dsyENISYMt/4cAaiP8Rss+OB0PrG2HQs9ColUNqF5GaRSFGRKrN2n0n+PuKvew5ngFAmL8nTw0I4gbf/Vj3Pg8rfoBT8SUPcvWCZldC36nmTxGR0xRiRKTK7T6Wwaxv9rB2Xwpe5DHEcz/3RhymU8FWrN/uKLmzxcWcQbr5VRB5FUT0NCc1FBH5A4UYEakyx9Jy+fe3u4nf9gN9LLt4yH0n3Vz242IUweFzdgxqZwaW5ldB0yvB089hNYuI81CIEZFKl51+kvXL5uASv4YZlj3Uc887u9EA/CNOt7QMgMj+4BvsoEpFxJkpxIhI5bHbObzmHXzWPs8gIx2s5uoijwa4Rl119hJRQHNz8iMRkcugECMilcJ+5DeSP3mIiAyzj8shSxj5He+kZe8bcQ3uCFargysUkdpGIUZELk92KjkrZ+C540NCMMgyPPmu0XiumTCD+r71HF2diNRiCjEicmlsRbD5XQpXPY93oTn54pf2vhjXzmRk325YdLlIRKqYQoyIs9u3CnLToOW14FW/el7z0M/Ylz+GNXknbsBue1Per38/f7rzDpo3UuuLiFQPhRgRZ7Z+Lnz7lLlsdYOoq6HdcGh9A3gHVP7rZRyHVc/AjqVYgTTDh38V3YLPlZN5/rr2uLuq34uIVB+FGBFn9dOr5oSIAP5NID0B9n1nPqyu5q3L7YZDm6HgE3h5r1VUABvexPjhJSwFWdgNC4ttV/Oe153MuLM/V7a4zPOLiFwCi2EYhqOLuFwZGRn4+/uTnp6On58GyZI6YO2/IOY5c/mqJ2HAk5DyO+z+Enb/F5LOGQXXYoVmfU8HmmEVH5Nl//fwzZOQug+ALfYWzCicSGjb3vzf6E408HGvpDclInXN5f7+VogRcTY/vAyrXzCXr34arnq89D6pcWaY2f0FHN92zgaLOSJuu+HQdhj4hZ7/dU4dhG+fhr1fm6fEn1mF41huvYpnhnbk1p4R6rwrIpdFIQaFGKkjDAPW/AN++If5fOB06PfXix93Mh72nG6hObq55LaIXmagaXcT+Dc21xXkwE+zzctVRXnYcOHdout4tWg0EaEhzLm1Cy2CfCv1rYlI3aQQg0KM1AGGAatfhB9fNp9f+xxc+XDFz5OWAHu+MgPN4Q0lt4V3NzsGb1ti9q8BfnPpxOM5d7DPaMzdfSN5bEhrPFxdLvPNiIiYFGJQiJFazjAgZias+7f5/LoXoc+Dl3/e9KNnA03CesxJjUxZHiFMyxnHV4U9aOTryb9u7kz/Vo0u/zVFRM6hEINCjNRihmHegfTzHPP5kP+DXvdW/utkJsKer8j//X8sPxHI00kDyMWTgW2CeGlMJxrW86j81xSROu9yf3/rFmuRmsowzI61v8w1n9/wT+g5pUpe6pQ1gA8zB/BufDNOZhfg4WrluRvbcmevpuq8KyI1lkKMSE1kGPDNE7DxLfP5ja9Aj8mV/jIHTmTxzrp4PttyhLxCOwBtQnyZc+sVtApW510RqdkUYkRqGrsdVjwKv74DWGDYq9BtQqWd3jAMNsafZP7aeGL2JnHmgnKHcD+m9GvODR1DcXPRyLsiUvMpxIjUJHY7LP8LbH4PsMDw1+GKOyrl1IU2O9/sTOQ/aw+w/Uh68fpBbYOY3Lc5vZoH6NKRiDgVhRiRmsJuh6/+DL8tBCww4k3ocutlnzYjr5AlGw/z3s8HOZqWC4CHq5XR3RozuW8kUZqwUUSclEKMSE1gt8GXD8HWReY0ASPfgk63XNYpj6bl8u66eBZvOkxWfhEAgfXcubNXM+7o1UR3HImI01OIEXE0uw2+uA+2LwGLC4yeDx1GX/Lpth1OY/7aA3yzMxGb3ezw0jKoHnf3i2R4l3A83TRYnYjUDgoxIo5kK4LP/wQ7PzUDzJh3oP3ICp/Gbjf4fk8S/1kbz8aDJ4vXX9miIXf3a85VLRthtaq/i4jULgoxIo5iK4RlU2DX52B1hTHvmnMYVUCRzc7SzUd4+8cDxKdkA+DmYmFY5zAm942kfZh/VVQuIlIjXNJ9lHPnzqVZs2Z4enoSHR3Nxo0by3Xc4sWLsVgsjBgxosT6iRMnYrFYSjyGDBlyKaWJOAdbIXx61+kA4wa3fFDhAPPT/hRunLOOact2EJ+SjZ+nK/cNiGLt49fwyi1dFGBEpNarcEvMkiVLmDp1KvPmzSM6OprZs2czePBgYmNjCQoKOu9xBw8e5NFHH6Vfv35lbh8yZAjvvvtu8XMPD3U6lFoo/agZXLYvhsQd4OIOtyyE1uUP7QdTsnlxxR5W7U4CoL63Gw9e3YJbezbBx0ONqyJSd1T4X7xXXnmFKVOmMGnSJADmzZvH8uXLWbBgAU8++WSZx9hsNm6//XZmzpzJ2rVrSUtLK7WPh4cHISEhFS1HpObLSjYnWdz52emJFk9z9TJbYFpdV67TZOYV8vrq/by77iAFNjsuVgt39mrKI4NaUt/bvYqKFxGpuSoUYgoKCti8eTPTpk0rXme1Whk0aBDr168/73HPPfccQUFBTJ48mbVr15a5z5o1awgKCqJBgwZcc801vPDCCzRs2LDMffPz88nPzy9+npGRUZG3IVL1ck7Cni9h5zI4uBYM+9ltTfpAh1HQbjjUO3/r5Rk2u8Gnmw/z8re/k5Jlfu/7t2rEMze2paWmBhCROqxCISYlJQWbzUZwcHCJ9cHBwezdu7fMY9atW8c777zD1q1bz3veIUOGMGrUKCIjI4mLi+Opp57i+uuvZ/369bi4lL4ddNasWcycObMipYtUvbx02LvCbHE5sBrsRWe3hXczb5tuNwL8w8t9yo3xJ5n51S52HTODevNAH/42tC1Xtw7S6LoiUudV6QX0zMxM7rzzTubPn09gYOB59xs3blzxcseOHenUqRNRUVGsWbOGgQMHltp/2rRpTJ06tfh5RkYGERERlVu8SHkUZEPsN2Y/l33fga3g7LaQjtB+lHnLdEBkhU575FQOs77Zy/LtxwHw9XTl4YEtGd+7Ge6umtdIRAQqGGICAwNxcXEhKSmpxPqkpKQy+7PExcVx8OBBhg0bVrzObjeb1V1dXYmNjSUqKqrUcc2bNycwMJD9+/eXGWI8PDzU8VccpzAP9q8yW1x+/xYKc85uC2xltri0HwWNWlX41DkFRby5Jo63fzxAfpEdqwXG9WzCX69tpRF2RUT+oEIhxt3dnW7duhETE1N8m7TdbicmJoYHH3yw1P5t2rRhx44dJdb97W9/IzMzk1dfffW8rSdHjhwhNTWV0NDQipQnUrVOHoA1/wd7l0NB5tn1DZqdDS7B7eESLvPY7Qb/3XaUf3yzl6QMs99Lr+YBTB/annZhfpX0BkREapcKX06aOnUqEyZMoHv37vTs2ZPZs2eTnZ1dfLfS+PHjCQ8PZ9asWXh6etKhQ4cSx9evXx+geH1WVhYzZ85k9OjRhISEEBcXx+OPP06LFi0YPHjwZb49kUqSlgDv3giZx8znfo2h/QgzvIRdcUnB5YzfEk4x86vdbD2cBkBEgBdP39CWwe1D1O9FROQCKhxixo4dy4kTJ5g+fTqJiYl06dKFlStXFnf2TUhIwGot/zV7FxcXtm/fzvvvv09aWhphYWFcd911PP/887pkJDVDdgosHGkGmMDWcNMcaNwTKvA9L0tieh7/t3Ivn/92FAAfdxfuv7oFk/tGan4jEZFysBiGYTi6iMuVkZGBv78/6enp+Pmp6V0qUX4mvD8Mjv1mtr5M/q5CdxeVxTAM3lkXz7+++53cQhsAY7o15vHBrQny86yMqkVEnMLl/v7W8J4i51OUD4tvNwOMVwDc+fllB5jcAhtPfLadL7eZl6W6NW3AjGHt6NS4fiUULCJStyjEiJTFbjMnZ4z/Adx84I5PL+luo3MdOZXDPR9sZvfxDFytFp4Z2o7xvZuq34uIyCVSiBH5I8OA5X81pwpwcYdxi8zB6i7DLwdSuX/RFk5mF9DQx503bu9KdPOyR6QWEZHyUYgR+aPVf4fN7wIWGPU2RF19yacyDIOFvxziua92U2Q36BDux1t3die8vlfl1SsiUkcpxIic65d58ONL5vKN/zJH271E+UU2nvliJ5/8egSAmzqH8X+jO+HlrjuPREQqg0KMyBnbl8LKJ8zlq5+GHpMv+VTJGXn86cPN/JaQhtUCT17fhin9mqv/i4hIJVKIEQHY9z18ca+53PNP0P+xSz7Vbwmn+NPCzSRn5uPn6cprt3XlqlaNKqlQERE5QyFG5PBG+OROc9bpDmNgyD8ueQTeT349zN8+30mBzU7LoHrMH9+dZoE+lVywiIiAQozUdcl7YNHN5iSOUQNhxJuXNBJvoc3Oi8v38N7PBwG4rl0wr4ztQj0P/RUTEakq+hdW6q60BFg4CvLSILw7jF0Iru4VPs3J7ALuX7SZXw6cBOCRQS358zUtsVrV/0VEpCopxEjd9Mf5kG5fCu4Vv+yz61g693ywmaNpufi4u/DK2C4Mbh9SBQWLiMgfKcRI3ZOfCYvGQOp+8I8wpxPwDqjwab7adozHPt1GXqGdpg29mT++O62CfaugYBERKYtCjNQt586H5N3wkuZDstkNXv42lnk/xAHQr2Ugr9/aFX9vt6qoWEREzkMhRuqOc+dDcq8Ht38KgS0rdIr03EIeXvwba2JPAPCn/s15fEgbXNT/RUSk2inESN3wx/mQxn4I4V0rdIoDJ7KY/P6vxKdk4+Fq5aUxnRje5fJmtRYRkUunECN1Q4n5kOZXeD6klKx8xi/YyJFTuYT5e/L2+O50CPevmlpFRKRcFGKk9is1H9KICh2eV2hjyge/cuRULk0bevPpvX1o5OtR+XWKiEiFVHxULxFnEve/y5oPyTAMHvt0O78lpOHn6cqCiT0UYEREagiFGKm9DAP+94K5fMWdlzQf0r+/38dX247harUw745uRDWqV8lFiojIpVKIkdrrwBo4uhlcPWHg9ArPh/TFb0eZE7MPgBdHdqBPi8AqKFJERC6VQozUXmv/Zf7sOgHqBVXo0E0HT/L4p9sB8zbqsT2aVHZ1IiJymRRipHZK+AUOrgWrG1z554odmprDnxZupsBm57p2wTwxpE0VFSkiIpdDIUZqpx//af7sciv4Ny73Yem5hUx6byMnswvoEO7H7HFdNJGjiEgNpRAjtc+xrbB/FViscOUj5T6s0Gbn/kWbiTuRTYifJ+9M6IG3u0YhEBGpqRRipPZZe7oVpsNoaBhVrkMMw2D6f3fx0/5UvN1d+M+E7gT7eVZhkSIicrkUYqR2Sd4Le74yl/v9tdyHvbMuno83JmCxwJxxV2g0XhERJ6AQI7XLulfMn22GQlDbch3y3a5EXlyxB4Cnb2jLoHbBVVWdiIhUIoUYqT1OHoAdn5rL/R8t1yE7j6bz8OKtGAbcFt2EyX0jq7BAERGpTAoxUnusmw2GDVoMgrArLrp7Ynoed7//K7mFNvq1DGTmTe2xVHBAPBERcRyFGKkd0o/C1o/M5X4Xb4XJKShi8vubSMzIo0VQPV6/rStuLvrrICLiTPSvttQOP78G9kJoeiU07X3BXe12g0cWb2XXsQwCfNxZMKEH/l5u1VSoiIhUFoUYcX5ZJ2Dze+ZyOfrC/N/KvXy3Owl3Fytv39mNJg29q7Y+ERGpEgox4vx+mQtFuRDWFZpffcFdF29M4K0fDwDw8s2d6N4soDoqFBGRKqAQI84t9xRs/I+53P/RC85U/dP+FP72xU4AHh7YkuFdwqujQhERqSIKMeLcNs6HgkwIagetrj/vbvuTs7jvw80U2Q1u6hzGI4NaVmORIiJSFRRixHnlZ8Evb5jL/f4K1rK/ziezC5j8/iYy8oro1rQBL43ppFupRURqAYUYcV6/LjAvJwVEQfuRZe6SX2TjTwt/5VBqDo0bePHWnd3wdHOp5kJFRKQqKMSIcyrMhfWvm8t9/wLWsoPJK9/9zqaDp/D1cOXdiT0IrOdRjUWKiEhVUogR5/Tbh5CVBP4R0GlsmbtsSTjF/LXmnUivjO1Cy2Df6qxQRESqmEKMOB9bIfz0qrl85cPg6l5ql7xCG48t3YbdgFFXhHOtJnUUEal1FGLE+WxfAumHwScIrrijzF1mf7+PuBPZNPL1YPqwdtVcoIiIVAeFGHEudhusfcVc7vMQuHmV2mXr4TTe/jEOgFkjO1Lfu3RLjYiIOD+FGHEuuz6Hk3Hg1QC631Vq87mXkUZeEc4gXUYSEam1FGLEedjtZ1thou8Dj3qldnk1Zh/7krMIrOfBDF1GEhGp1RRixHn8vhKSd4G7L0TfU2rztsNpvPWDeRnp7yM76DKSiEgtpxAjzsEw4MeXzeWed5uXk86RX2Tj0dOXkYZ3CeO69iEOKFJERKqTQow4hwOr4dgWcPWCXg+U2jyn+DKSO88Oa++AAkVEpLopxIhz+PFf5s9uE6BeoxKbth9JY94P5qB2L4zoSAMfXUYSEakLFGKk5ju0Hg6tA6sb9PlziU35RTYeW7odm91gWOcwhnTQZSQRkbpCIUZqvrX/NH92uQ38w0tsev1/+4lNyqShjzszb9JlJBGRukQhRmq2Y7/B/u/BYoW+j5TYtPNoOm+sMe9GemFEBwJ0GUlEpE5RiJGabe3pvjAdb4aA5sWrC4rsPLp0Gza7wY2dQrm+Y6iDChQREUdRiJGaK3kP7PnKXO47tcSm11fvZ29iJgE+7jyny0giInWSQozUXGdG5207DILaFK/eeTSdN1bvB+D54R1oWM/DEdWJiIiDKcRIzXTyAOz81Fzu92jx6jOXkYrsBjd2DOXGTrqMJCJSVynESM20bjYYdmhxLYR1KV4995zLSDOH6zKSiEhddkkhZu7cuTRr1gxPT0+io6PZuHFjuY5bvHgxFouFESNGlFhvGAbTp08nNDQULy8vBg0axL59+y6lNKkNDm+ErR+Zy/3PtsLsOpbO3NOXkZ4b3p5AXUYSEanTKhxilixZwtSpU5kxYwZbtmyhc+fODB48mOTk5Ased/DgQR599FH69etXattLL73EnDlzmDdvHhs2bMDHx4fBgweTl5dX0fLE2e2PgQ+Gg70QWg6GJr0AKLTZeXTpdorsBtd3COFG3Y0kIlLnVTjEvPLKK0yZMoVJkybRrl075s2bh7e3NwsWLDjvMTabjdtvv52ZM2fSvHnzEtsMw2D27Nn87W9/Y/jw4XTq1IkPPviAY8eO8cUXX1T4DYkT2/UFfDQWCnMgaiDc/G7xpjdWx7HneAYNvN14bngHLBaL4+oUEZEaoUIhpqCggM2bNzNo0KCzJ7BaGTRoEOvXrz/vcc899xxBQUFMnjy51Lb4+HgSExNLnNPf35/o6OjznjM/P5+MjIwSD3FyWz6ATyeZLTDtRsCti8HdB4DdxzJ47X/m5cWZwzvQyFeXkUREpIIhJiUlBZvNRnBwcIn1wcHBJCYmlnnMunXreOedd5g/f36Z288cV5Fzzpo1C39//+JHRERERd6G1DQ/vwZfPmR25O06HsYsAFdz9N1Cm53HPjXvRhrcPphhuhtJREROq9K7kzIzM7nzzjuZP38+gYGBlXbeadOmkZ6eXvw4fPhwpZ1bqpFhQMzz8N3fzOd9/gzD5oDVpXiXeWvi2HUsg/rebjw/QpeRRETkLNeK7BwYGIiLiwtJSUkl1iclJRESUnr24Li4OA4ePMiwYcOK19ntdvOFXV2JjY0tPi4pKYnQ0LP/y05KSqJLly5l1uHh4YGHhy4pODW7Hb55DDb9x3w+cLo5Ku85IWVvYgZzzlxGuqk9Qb6ejqhURERqqAq1xLi7u9OtWzdiYmKK19ntdmJiYujdu3ep/du0acOOHTvYunVr8eOmm27i6quvZuvWrURERBAZGUlISEiJc2ZkZLBhw4Yyzym1gK0QPv/T6QBjgRv/Bf3+WiLAmHcjbaPQZnBtu2Bu6hzmuHpFRKRGqlBLDMDUqVOZMGEC3bt3p2fPnsyePZvs7GwmTZoEwPjx4wkPD2fWrFl4enrSoUOHEsfXr18foMT6Rx55hBdeeIGWLVsSGRnJM888Q1hYWKnxZKQWKMyFpRPh95VgdYWRb0HHMaV2e+uHOHYezcDfy40XdRlJRETKUOEQM3bsWE6cOMH06dNJTEykS5curFy5srhjbkJCAlZrxbraPP7442RnZ3PPPfeQlpZG3759WblyJZ6eunxQq+RlwMe3wqF14OoJt3wArQaX2m3P8QxejTEvIz17UzuC/PQ9EBGR0iyGYRiOLuJyZWRk4O/vT3p6On5+fo4uR8qSnQofjoLjW8HdF25bAs2uLLVbXqGNEXN/Ym9iJoPaBjN/fDe1woiI1FKX+/u7wi0xIhWWfhQWjoCU38G7IdzxGYRdUeaur6z6nb2JmTT0cecfozsqwIiIyHkpxEjVSo2DD0ZAegL4hcOdX0CjVmXuuj4ulflrDwDwj9GdNDeSiIhckEKMVJ3EHbBwFGQnQ0AUjP8C6jcpc9eMvEIeXboNw4BxPSK4tl1wmfuJiIicoRAjVSNhA3x0M+SlQ0hHuGMZ1As67+7P/ncXR9NyaRLgzTND21VjoSIi4qwUYqTy7f8eFt8BRbkQ0cvsxOtV/7y7L99+nGW/HcVqgX+P7YyPh76WIiJycfptIZVr1xfw2d3mRI4tBsEtC8Hd+7y7J2Xk8fQXOwC4f0ALujUNqKZCRUTE2VXp3ElSx2xZeHYm6vYjYdzHFwwwhmHw2KfbScsppEO4H38e2LIaixUREWenECOVY38MfPXn0zNRT4DR7xTPRH0+C385xI+/n8DD1crssV1wd9XXUUREyk+Xk+TynToIn002A0yXO2DYqyXmQSrL/uQs/r5iDwDTrm9DiyDfaihURERqE/3XVy5PQQ4suQNyT0FYV3Myx4sEmEKbnamfbCWv0E6/loGM792semoVEZFaRSFGLp1hwNePmOPBeAfC2IXgdvF5jl6L2cf2I+n4e7nx8pjOWK0alVdERCpOIUYu3Ya3YPsSsLjAze+Bf+OLHrIl4RSvr94PwIsjOxDir8kdRUTk0ijEyKU5+BN8+5S5fN3zENnvoodk5xcxdclW7AaM6BLG0E5hVVykiIjUZgoxUnHpR2HpBDBs0PFm6HV/uQ57YfkeDqbmEObvyczhHaq4SBERqe0UYqRiivLhk/GQfQKCO8CwORftyAsQsyeJjzcmAPDPWzrj7+VW1ZWKiEgtpxAjFbPiMTj6K3jWh7EfXnAwuzNSs/J54rPtANzdN5I+UYFVXKSIiNQFCjFSfpvfgy3vAxZzMLuAyIseYhgGTy7bQUpWAa2DfXl0cOsqL1NEROoGhRgpnyO/mq0wANf8DVoOKtdhS389wqrdSbi5WPj32C54urlUYZEiIlKXKMTIxWUlw5I7wVYAbYZCv7+W67CE1BxmfrULgL9e15p2YX5VWaWIiNQxCjFyYbZCWDoRMo9BYCsY8Wa5OvLa7AZTP9lKdoGNns0CmNKvedXXKiIidYpCjFzYd8/AoZ/A3RfGLgLP8rWmzPshjl8PnaKehyv/uqUzLhqVV0REKplCjJzftiWw4U1zeeQ8aNSqXIftPJrOv1f9DsCzN7UnIuDidzCJiIhUlEKMlO34dvjqYXO536PQdmi5DssrtPGXJVspshsMaR/C6K7hVVikiIjUZQoxUlrOSVhyOxTlQotBcPVT5T70/1buZV9yFo18Pfj7qI5YytF/RkRE5FIoxEhJdht8NhnSEqBBMxj9H7CW77bodftSePengwC8NKYTAT7uVVeniIjUeQoxUtL/XoC4/4Gbt9mR16tBuQ47fDKHqZ9sBeCOXk24unVQFRYpIiKiECPn2v0lrHvFXL7pNQgp3ySNyRl53PHOBpIz82kVXI+nbmhbhUWKiIiYFGLElLwXvrjPXO79IHQcU67D0nIKGL9gI4dSc4gI8GLh5Gi83V2rsFARERGTQoxAXrrZkbcgC5r1g0Ezy3VYdn4Rk97bxN7ETIJ8PVg0uRfBfp5VXKyIiIhJIaaus9vh83shdT/4NYab3wOXi7ek5BfZuGfhr/yWkIa/lxsLJ0fTpKHGgxERkeqjEFPXrZkFsSvAxQPGLgSfwIseUmSz8+ePf+On/al4u7vw3qQetA7xrYZiRUREzlKIqcvW/Rt+fMlcHvoKhHe96CF2u8GTy3bw7a4k3F2t/Gd8d65oUr47mERERCqTQkxd9dMc+P5Zc/maZ+CKOy56iGEYPL98N59uPoKL1cLrt15BnxYXb7kRERGpCgoxddH6ubDqGXP56qeh/6PlOuzVmH3Fg9m9PKYT17UPqaICRURELk4hpq755U349vQ0Alc9CVc9Xq7DFqyLZ/b3+wCYeVN7RnVtXFUVioiIlItCTF2y4W1Y+aS53P8xGPBkuQ5b+uthnvt6NwBTr23FhD7NqqhAERGR8lOIqSs2/Qe+ecxc7jvVvIxUjskZV+5M5InPtgNwd99IHrqmRVVWKSIiUm4KMXXBrwtg+V/N5SsfhoHTyxVg1u1L4c8f/4bdgFu6N+bpG9tqVmoREakxFGJqu83vwdd/MZd7P2iOxluOILIl4RT3LPyVApud6zuEMGtUJwUYERGpURRiarMtC+Grh83lXvfDdS+UK8DsTcxg4oKN5BTY6NcykNnjuuBiVYAREZGaRSGmttr6EXz5kLkcfS8M/nu5AszBlGzufGcjGXlFdGvagLfu7IaHq0sVFysiIlJxCjG10bbF8MX9gAE9psCQf5QrwCSm53H7fzZwIjOftqF+LJjYQzNSi4hIjaUQU9tsXwpf3AcY0P0uuOHlcgWYk9kF3PHOBo6m5RIZ6MMHd/XE38ut6usVERG5RAoxtcmOT+Hze8CwQ9cJcMO/yhVgMvMKmfjuRvYnZxHq78nCyT1p5OtRDQWLiIhcOoWY2mLX57DsdIC54k4YOhusF//jzSu0cff7v7L9SDoBPu4snBxN4wbeVV+viIjIZVKIqQ12/xc+nQyGDbrcDsPmlCvAADz9+U42xJ/E18OVD+7qSYugelVcrIiISOVQiHF2e76CT+8yA0yncXDTa+UOMAmpOXz+2xEA3h7fnQ7h/lVZqYiISKVSiHFme1fA0olgL4KOt8CIN8Ba/tuhF/wUj92AAa0b0TuqYdXVKSIiUgUUYpxV7Er4ZLwZYDqMhhFvVijApOUUsGTTYQCm9GteVVWKiIhUGYUYZ3RoPXxyJ9gLof1IGPk2uFRsPJdFGxLILbTRLtSPPmqFERERJ6QQ44w2vg22Amh9A4yaX+EAk19k492fDgJwT//mmhNJRESckkKMMzq8wfzZ635wqfiAdP/97RgpWfmE+ntyY6fQSi5ORESkeijEOJv0I5BxFCwuEN61wofb7QZvrz0AwF1XRuLmoq+AiIg4J/0GczZnWmFCOoK7T4UP/+H3E+xPzsLXw5VxPSMquTgREZHqoxDjbBJOh5gmvS7p8PmnW2HG9YzA11NzI4mIiPNSiHE2Z1piInpW+NCdR9P5OS4VV6uFSVdGVnJhIiIi1euSQszcuXNp1qwZnp6eREdHs3HjxvPuu2zZMrp37079+vXx8fGhS5cuLFy4sMQ+EydOxGKxlHgMGTLkUkqr3QqyIXGHuRwRXeHDz7TCDO0USlh9r8qsTEREpNpV7N5cYMmSJUydOpV58+YRHR3N7NmzGTx4MLGxsQQFBZXaPyAggKeffpo2bdrg7u7O119/zaRJkwgKCmLw4MHF+w0ZMoR33323+LmHh2ZRLuXoZnN6Ab/G4N+4Yoem5fL19uMA3K3B7UREpBaocEvMK6+8wpQpU5g0aRLt2rVj3rx5eHt7s2DBgjL3HzBgACNHjqRt27ZERUXx8MMP06lTJ9atW1diPw8PD0JCQoofDRo0uLR3VJtdxqWkd9fFY7MbXNmioeZIEhGRWqFCIaagoIDNmzczaNCgsyewWhk0aBDr16+/6PGGYRATE0NsbCz9+/cvsW3NmjUEBQXRunVr7rvvPlJTU897nvz8fDIyMko86oTDpy/bVfBSUnpuIR9vTAA0xYCIiNQeFbqclJKSgs1mIzg4uMT64OBg9u7de97j0tPTCQ8PJz8/HxcXF9544w2uvfba4u1Dhgxh1KhRREZGEhcXx1NPPcX111/P+vXrcXEpPR/QrFmzmDlzZkVKd352+9mWmCYVCzGLNyaQXWCjVXA9rmrVqAqKExERqX4V7hNzKXx9fdm6dStZWVnExMQwdepUmjdvzoABAwAYN25c8b4dO3akU6dOREVFsWbNGgYOHFjqfNOmTWPq1KnFzzMyMoiIqOVjnqT8Dnnp4OYNwR3KfVhBkb14ioG7+2mKARERqT0qFGICAwNxcXEhKSmpxPqkpCRCQkLOe5zVaqVFixYAdOnShT179jBr1qziEPNHzZs3JzAwkP3795cZYjw8POpex98zrTDh3So01cDyHcdIzMijka8Hw7uEVVFxIiIi1a9CfWLc3d3p1q0bMTExxevsdjsxMTH07t273Oex2+3k5+efd/uRI0dITU0lNFTz+hQr7tRb/ktJhmHw9o/xAEzs0wwP19KX5kRERJxVhS8nTZ06lQkTJtC9e3d69uzJ7Nmzyc7OZtKkSQCMHz+e8PBwZs2aBZj9V7p3705UVBT5+fmsWLGChQsX8uabbwKQlZXFzJkzGT16NCEhIcTFxfH444/TokWLErdg13mXEGJ+2p/KnuMZeLu7cHt0kyoqTERExDEqHGLGjh3LiRMnmD59OomJiXTp0oWVK1cWd/ZNSEjAaj3bwJOdnc3999/PkSNH8PLyok2bNnz44YeMHTsWABcXF7Zv3877779PWloaYWFhXHfddTz//PN175LR+WSnQup+c7lx93Ifdmaix1u6R1Df270qKhMREXEYi2EYhqOLuFwZGRn4+/uTnp6On5+fo8upfHtXwOJboVEbeGBDuQ7ZczyD619di9UCPzx2NREB3lVcpIiISMVc7u9vzZ3kDC5hkLv/rDX7wlzfIVQBRkREaiWFGGdQwUHukjLy+HLbUQDu7qeJHkVEpHZSiKnpigrg2BZzOaJXuQ557+eDFNoMejYL4Iommr5BRERqJ4WYmi5xOxTlgVcANIy66O5Z+UUs+uUQAFP6a4oBERGpvRRiarpzb60ux2i7n2w6TEZeEc0DfRjYpvSs4iIiIrWFQkxNl/CL+bMc8yUV2ey8s87s0Du5XyRWq6YYEBGR2kshpiYzjAoNcvfNzkSOpuUS4OPO6K6Nq7g4ERERx1KIqcnSEiArCayuEHbFBXc1DIP5pwe3G9+7KZ5ummJARERqN4WYmuxMK0xoZ3DzuuCuG+NPsv1IOh6uVu7s1bQaihMREXEshZiarPhS0sVvrT7TCjOmW2Ma1tN0DSIiUvspxNRk5Rypd39yFt/vScZigcl9NbidiIjUDQoxNVV+JiTtMpcv0qn3nXVmK8ygtsE0b1SvqisTERGpERRiaqojv4Jhh/pNwC/0vLudyMznsy3mFAP3aHA7ERGpQxRiaqpyzpe0cP1BCorsdImoT/emmmJARETqDoWYmurw6UHuLhBicgtsLDw9xcA9/ZtjKceIviIiIrWFQkxNZLeZl5PggiHm0y1HOJVTSESAF4Pbh1RTcSIiIjWDQkxNdGIv5GeAez0IalfmLja7wTunb6u+u29zXDTFgIiI1DEKMTXRmfmSwruBi2uZu6zancTB1Bz8vdy4ubumGBARkbpHIaYmOtOpt8n5B7k7M7jdHb2a4O1edtARERGpzRRiaqKLDHK3+dBJNh86hbuLlQm9m1VfXSIiIjWIQkxNk5UMp+IBCzTuUeYuC346CMDwLmEE+XlWX20iIiI1iEJMTXOmFSaoHXj6l9qcnJnHtzsTAZh4ZbNqLExERKRmUYipaS5yKemTTYcpshtc0aQ+7cNKhxwREZG6QiGmpkk4E2JKjw9jsxt8vPEwAHdEN63OqkRERGochZiapDAPjm81l5uUDjGr9yZzNC2X+t5u3Njp/PMpiYiI1AUKMTXJ8W1gKwCfRtAgstTmDzeYUwzc3K0xnm4u1V2diIhIjaIQU5OcO1/SH+ZBOnwyhx9+PwHAbbqUJCIiohBTo1xg5upFGxIwDOjXMpDIQJ9qLkxERKTmUYipKQzjnDuTSoaY/CIbn/xqdui9Xa0wIiIigEJMzXHyAGSfABd3CO1cYtPKnYmczC4gxM+TQW2DHFSgiIhIzaIQU1OcuZQUdgW4lRyF98NfzA6943pG4OqiPzIRERFQiKk5zjPI3d7EDDYdPIWL1cK4Hk0cUJiIiEjNpBBTU5ynP8yiXxIAuLZtMCH+midJRETkDIWYmiA3DZL3mMvnhJjs/CI+/+0oAHf0UodeERGRcynE1ARHfwUMc4C7emc77n6x9ShZ+UVEBvrQJ6qh4+oTERGpgRRiaoIy5ksyDIMPT19Kuj26CVarpawjRURE6iyFmJrgTH+Yc+ZL2pKQxp7jGXi4WhnTrbGDChMREam5FGIczVYERzeby+e0xCw6fVv10E5h1Pd2d0RlIiIiNZpCjKMl74KCLPDwg0ZtADiVXcDXO44DcEcv3VYtIiJSFoUYRzszyF3jHmA1Z6ZeuvkwBUV22of50SWivuNqExERqcEUYhztD+PD2O0GizaYHXrv6NUUi0UdekVERMqiEONoCSVH6l23P4VDqTn4ergyvEuYAwsTERGp2RRiHCnjGKQngMUKjbsDZ+dJGtU1HG93V0dWJyIiUqMpxDjSmf4wwe3Bw5fj6bl8vycJgNs1Qq+IiMgFKcQ40h/6w3y88TB2A3pGBtAq2NeBhYmIiNR8CjGOdE6IKbTZWbzxbIdeERERuTCFGEcpzIXj28zliGi+351EcmY+gfXcGdI+xLG1iYiIOAGFGEc5ugXsRVAvBOo34cMNZofeW7pH4O6qPxYREZGL0W9LRzl89tbqAynZ/LQ/FYsFbu2pEXpFRETKQyHGUc7cmdSkV/Hgdle3DiIiwNuBRYmIiDgPhRhHMIzilpj80O58uvkIoHmSREREKkIhxhFS90PuSXD15KvkRqTnFhJe34urWgU5ujIRERGnoRDjCGf6w4R1ZeEmc7bq26Kb4GLVPEkiIiLlpRDjCAm/AHCiQWe2HU7DzcXC2B4RDi5KRETEuSjEOMLpTr0r0sw+MEM6hBJYz8ORFYmIiDgdhZjqlnMSUmIBmHcgEIA7otWhV0REpKIuKcTMnTuXZs2a4enpSXR0NBs3bjzvvsuWLaN79+7Ur18fHx8funTpwsKFC0vsYxgG06dPJzQ0FC8vLwYNGsS+ffsupbSa78gmANK9m3K80IdWwfXoGRng4KJEREScT4VDzJIlS5g6dSozZsxgy5YtdO7cmcGDB5OcnFzm/gEBATz99NOsX7+e7du3M2nSJCZNmsS3335bvM9LL73EnDlzmDdvHhs2bMDHx4fBgweTl5d36e+spjrdqXd9YQsAbo9uisWiDr0iIiIVZTEMw6jIAdHR0fTo0YPXX38dALvdTkREBA899BBPPvlkuc7RtWtXbrzxRp5//nkMwyAsLIy//vWvPProowCkp6cTHBzMe++9x7hx4y56voyMDPz9/UlPT8fPz68ib6f6vTcUDq7licIpfGkdxIanB+Ln6eboqkRERKrd5f7+rlBLTEFBAZs3b2bQoEFnT2C1MmjQINavX3/R4w3DICYmhtjYWPr37w9AfHw8iYmJJc7p7+9PdHT0ec+Zn59PRkZGiYdTsBXCkV8B2GxvyYgrwhRgRERELlGFQkxKSgo2m43g4OAS64ODg0lMTDzvcenp6dSrVw93d3duvPFGXnvtNa699lqA4uMqcs5Zs2bh7+9f/IiIcJLbkxN3QFEu6YYPcUYYt0c3dXRFIiIiTqta7k7y9fVl69atbNq0iRdffJGpU6eyZs2aSz7ftGnTSE9PL34cPny48oqtSqdvrd5sb0nniAA6hPs7uCARERHn5VqRnQMDA3FxcSEpKanE+qSkJEJCQs57nNVqpUULsyNrly5d2LNnD7NmzWLAgAHFxyUlJREaGlrinF26dCnzfB4eHnh4ON+4KvaEX7ACm+2tuKOXWmFEREQuR4VaYtzd3enWrRsxMTHF6+x2OzExMfTu3bvc57Hb7eTn5wMQGRlJSEhIiXNmZGSwYcOGCp3TGRTEm318Yt3aMrRT6EX2FhERkQupUEsMwNSpU5kwYQLdu3enZ8+ezJ49m+zsbCZNmgTA+PHjCQ8PZ9asWYDZf6V79+5ERUWRn5/PihUrWLhwIW+++SYAFouFRx55hBdeeIGWLVsSGRnJM888Q1hYGCNGjKi8d3op8jMhbnWlncszN5Eiw0rLrlfh6eZSOecVERGpoyocYsaOHcuJEyeYPn06iYmJdOnShZUrVxZ3zE1ISMBqPdvAk52dzf3338+RI0fw8vKiTZs2fPjhh4wdO7Z4n8cff5zs7Gzuuece0tLS6Nu3LytXrsTT07MS3uJlyEyCT+6s1FPuMZpwy5VtKvWcIiIidVGFx4mpiapsnJj0I/DZ3Zd9mpyCIn5PyiK3yMJPIbfx6AMPVUJxIiIizu1yf39XuCWmTvFvDHetvKxTbDp4ksnvbSIjr4jWwb68f0fPSipORESkblOIqULf707igY+2kF9kp3vTBrwzoQf+3hrcTkREpDIoxFSRpb8e5sllO7DZDa5pE8Tc27ri5a7OvCIiIpVFIaYKvPVDHLO+2QvA6K6N+cfojri5VMu4giIiInWGQkwlMgyDWd/s5e0fDwBwT//mTLu+jWapFhERqQIKMZWkyGbnic928NmWIwBMu74Nf7oqysFViYiI1F4KMZUgt8DGgx9tIWZvMi5WC7NGdeSW7k4yKaWIiIiTUoi5TOk5hUx+fxO/HjqFh6uVubd1ZVC74IsfKCIiIpdFIeYyJGXkMf6djcQmZeLr6cqCiT3o0SzA0WWJiIjUCQoxl+jAiSzufGcjR9NyCfL14P27etI2tBJHCxYREZELUoi5BDuOpDPx3Y2kZhcQGejDB3f1JCLA29FliYiI1CkKMRX00/4U7vngV7ILbHQI9+O9ST0JrOfh6LJERETqHIWYCli+/Th/WbKVApudPlENeevObvh6ahoBERERR1CIKaeFvxxi+n93YhhwQ8cQ/j22Cx6umkZARETEURRiLsIwDObE7Off3/8OwO3RTXhueAdcrBqFV0RExJEUYi7AZjeY+dUuPlh/CICHB7bkkUEtNY2AiIhIDaAQcwErdybywfpDWCzw7LD2TOjTzNEliYiIyGkKMRdwQ8cQJvZpRremDRjWOczR5YiIiMg5FGIuwGKx8OxN7R1dhoiIiJTB6ugCRERERC6FQoyIiIg4JYUYERERcUoKMSIiIuKUFGJERETEKSnEiIiIiFNSiBERERGnpBAjIiIiTkkhRkRERJySQoyIiIg4JYUYERERcUoKMSIiIuKUFGJERETEKdWKWawNwwAgIyPDwZWIiIhIeZ35vX3m93hF1YoQk5mZCUBERISDKxEREZGKyszMxN/fv8LHWYxLjT81iN1u59ixY/j6+mKxWCr13BkZGURERHD48GH8/Pwq9dxyfvrcHUOfu2Poc3cMfe6Oce7n7uvrS2ZmJmFhYVitFe/hUitaYqxWK40bN67S1/Dz89OX3AH0uTuGPnfH0OfuGPrcHePM534pLTBnqGOviIiIOCWFGBEREXFKCjEX4eHhwYwZM/Dw8HB0KXWKPnfH0OfuGPrcHUOfu2NU5udeKzr2ioiISN2jlhgRERFxSgoxIiIi4pQUYkRERMQpKcSIiIiIU1KIEREREaekEHMBc+fOpVmzZnh6ehIdHc3GjRsdXVKt9+yzz2KxWEo82rRp4+iyap0ff/yRYcOGERYWhsVi4Ysvviix3TAMpk+fTmhoKF5eXgwaNIh9+/Y5ptha4mKf+cSJE0t994cMGeKYYmuRWbNm0aNHD3x9fQkKCmLEiBHExsaW2CcvL48HHniAhg0bUq9ePUaPHk1SUpKDKq4dyvO5DxgwoNR3/t57763Q6yjEnMeSJUuYOnUqM2bMYMuWLXTu3JnBgweTnJzs6NJqvfbt23P8+PHix7p16xxdUq2TnZ1N586dmTt3bpnbX3rpJebMmcO8efPYsGEDPj4+DB48mLy8vGqutPa42GcOMGTIkBLf/Y8//rgaK6ydfvjhBx544AF++eUXVq1aRWFhIddddx3Z2dnF+/zlL3/hq6++YunSpfzwww8cO3aMUaNGObBq51eezx1gypQpJb7zL730UsVeyJAy9ezZ03jggQeKn9tsNiMsLMyYNWuWA6uq/WbMmGF07tzZ0WXUKYDx+eefFz+32+1GSEiI8fLLLxevS0tLMzw8PIyPP/7YARXWPn/8zA3DMCZMmGAMHz7cIfXUJcnJyQZg/PDDD4ZhmN9tNzc3Y+nSpcX77NmzxwCM9evXO6rMWuePn7thGMZVV11lPPzww5d1XrXElKGgoIDNmzczaNCg4nVWq5VBgwaxfv16B1ZWN+zbt4+wsDCaN2/O7bffTkJCgqNLqlPi4+NJTEws8f339/cnOjpa3/8qtmbNGoKCgmjdujX33Xcfqampji6p1klPTwcgICAAgM2bN1NYWFji+96mTRuaNGmi73sl+uPnfsaiRYsIDAykQ4cOTJs2jZycnAqdt1bMYl3ZUlJSsNlsBAcHl1gfHBzM3r17HVRV3RAdHc17771H69atOX78ODNnzqRfv37s3LkTX19fR5dXJyQmJgKU+f0/s00q35AhQxg1ahSRkZHExcXx1FNPcf3117N+/XpcXFwcXV6tYLfbeeSRR7jyyivp0KEDYH7f3d3dqV+/fol99X2vPGV97gC33XYbTZs2JSwsjO3bt/PEE08QGxvLsmXLyn1uhRipUa6//vri5U6dOhEdHU3Tpk355JNPmDx5sgMrE6la48aNK17u2LEjnTp1IioqijVr1jBw4EAHVlZ7PPDAA+zcuVP97KrZ+T73e+65p3i5Y8eOhIaGMnDgQOLi4oiKiirXuXU5qQyBgYG4uLiU6p2elJRESEiIg6qqm+rXr0+rVq3Yv3+/o0upM858x/X9d6zmzZsTGBio734lefDBB/n6669ZvXo1jRs3Ll4fEhJCQUEBaWlpJfbX971ynO9zL0t0dDRAhb7zCjFlcHd3p1u3bsTExBSvs9vtxMTE0Lt3bwdWVvdkZWURFxdHaGioo0upMyIjIwkJCSnx/c/IyGDDhg36/lejI0eOkJqaqu/+ZTIMgwcffJDPP/+c//3vf0RGRpbY3q1bN9zc3Ep832NjY0lISND3/TJc7HMvy9atWwEq9J3X5aTzmDp1KhMmTKB79+707NmT2bNnk52dzaRJkxxdWq326KOPMmzYMJo2bcqxY8eYMWMGLi4u3HrrrY4urVbJysoq8b+d+Ph4tm7dSkBAAE2aNOGRRx7hhRdeoGXLlkRGRvLMM88QFhbGiBEjHFe0k7vQZx4QEMDMmTMZPXo0ISEhxMXF8fjjj9OiRQsGDx7swKqd3wMPPMBHH33Ef//7X3x9fYv7ufj7++Pl5YW/vz+TJ09m6tSpBAQE4Ofnx0MPPUTv3r3p1auXg6t3Xhf73OPi4vjoo4+44YYbaNiwIdu3b+cvf/kL/fv3p1OnTuV/ocu6t6mWe+2114wmTZoY7u7uRs+ePY1ffvnF0SXVemPHjjVCQ0MNd3d3Izw83Bg7dqyxf/9+R5dV66xevdoASj0mTJhgGIZ5m/UzzzxjBAcHGx4eHsbAgQON2NhYxxbt5C70mefk5BjXXXed0ahRI8PNzc1o2rSpMWXKFCMxMdHRZTu9sj5zwHj33XeL98nNzTXuv/9+o0GDBoa3t7cxcuRI4/jx444ruha42OeekJBg9O/f3wgICDA8PDyMFi1aGI899piRnp5eodexnH4xEREREaeiPjEiIiLilBRiRERExCkpxIiIiIhTUogRERERp6QQIyIiIk5JIUZERESckkKMiIiIOCWFGBEREXFKCjEiIiLilBRiRERExCkpxIiIiIhT+n91NfiB8pb6gwAAAABJRU5ErkJggg==\n" + }, + "metadata": {} + } + ] + }, + { + "cell_type": "markdown", + "source": [ + "## Test Model\n", + "\n", + "Let's test our model on some examples, noting that we have to preprocess our inputs before feeding them to the model." + ], + "metadata": { + "id": "qRnwfkjatMOV" + } + }, + { + "cell_type": "code", + "source": [ + "def preprocess_text(text):\n", + " # we need to tokenize our inputs and convert them into integer codepoints\n", + " # before passing them to the model\n", + " text = tf.constant([txt])\n", + " text = whitespace_tokenizer.tokenize(text)\n", + " text = text.to_tensor(default_value=\"\", shape=(text.shape[0], sequence_length))\n", + " text = integerizer.integerize(text)\n", + " return text\n", + "\n", + "def predict_emotions(txt, threshold=0.5):\n", + " # recall it is multi-class so we need to get all prediction above a threshold (0.5)\n", + " preds = model(preprocess_text(text))[0]\n", + " out = 0\n", + " for i in range(NUM_CLASSES):\n", + " if preds[i] > threshold:\n", + " emotion_name = CLASSES[i]\n", + " emotion_prob = round(float(preds[i]) * 100, 1)\n", + " print(f\"{emotion_name} ({emotion_prob})%\")\n", + " out += 1\n", + " if not out:\n", + " print(\"neutral\")" + ], + "metadata": { + "id": "So39n-nKrd_K" + }, + "execution_count": 22, + "outputs": [] + }, + { + "cell_type": "code", + "source": [ + "txt = \"I enjoy having a good icecream.\"\n", + "predict_emotions(txt)" + ], + "metadata": { + "id": "isWJPbWf-h4R", + "colab": { + "base_uri": "https://localhost:8080/" + }, + "outputId": "d7fe825c-0ee1-412f-9919-64f35ef9c718" + }, + "execution_count": 23, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "joy (84.5)%\n" + ] + } + ] + }, + { + "cell_type": "code", + "source": [ + "# the model works even with typos, substitutions, and emojis!\n", + "txt = \"I enjoy hving a g00d ic3cream!!! 🍦\"\n", + "predict_emotions(txt)" + ], + "metadata": { + "colab": { + "base_uri": "https://localhost:8080/" + }, + "id": "twyBz-_lvGyb", + "outputId": "e79754c3-e414-425d-be6b-7d68b2eaadb2" + }, + "execution_count": 24, + "outputs": [ + { + "output_type": "stream", + "name": "stdout", + "text": [ + "joy (85.9)%\n" + ] + } + ] + } + ] +} \ No newline at end of file