Sentiment Analysis in Czech.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sentiment Analysis in Czech\n",
    "### Capstone Project for Machine Learning Nanodegree at Udacity\n",
    "\n",
    "- [Part 1: Loading the data and extracting samples](#Part-1:-Load-our-datasets-and-extract-one-sentence-samples)\n",
    "- [Part 2: Data preprocessing](#Part-2:-Preprocess-our-datasets)\n",
    "- [Part 3: Defining the neural network and benchmark](#Part-3:-Define-our-neural-network-and-benchmark-models)\n",
    "- [Part 4: Creating the individual models](#Part-4:-Create-the-individual-models-and-train)\n",
    "    - [Facebook Model](#Facebook-Model)\n",
    "    - [ČSFD Model](#ČSFD-Model)\n",
    "    - [MALL.CZ Model](#MALL.CZ-Model)\n",
    "    - [Combined Model](#Combined-Model)\n",
    "- [Part 5: Running tests with new sentences](#Part-5:-Running-tests-with-new-sentences)\n",
    "\n",
    "#### ⏱ NOTE:\n",
    "Some cells in Part 1 and 2 contain time-consuming tasks like data transformations. If you run the whole notebook at least once, their outputs will already be saved in this project subdirectories and you don't have to run them again in the future. These cells are marked with a stopwatch emoji ⏱ in the titles above them.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import subprocess\n",
    "import string\n",
    "import random\n",
    "import re\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import f1_score\n",
    "from nltk.tokenize import sent_tokenize, word_tokenize\n",
    "from nltk.sentiment.util import SAD, HAPPY\n",
    "from lib.czech_stemmer import cz_stem\n",
    "\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Having problem with NLTK tokenizer? \n",
    "You might need to run `nltk.download()` and download `Punkt Tokenizer Models`. More information in [this SO thread](https://stackoverflow.com/a/10852888)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Part 1: Load our datasets and extract one-sentence samples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "labels = {'n':'NEG', '0':'NEU', 'p':'POS', 'b':'BIP'}\n",
    "target_labels = [labels['n'], labels['0'], labels['p']]\n",
    "max_sentence_length = 30 # no. of words\n",
    "\n",
    "def data_stats(data, dataset='Dataset', do_print=False):\n",
    "    '''\n",
    "    Returns basic statistics about a dataset.\n",
    "    \n",
    "    If the third argument is set to True, the function will\n",
    "    print out the stats instead.\n",
    "    '''\n",
    "    num_of_neg = len(data[data['Sentiment'] == labels['n']])\n",
    "    num_of_neu = len(data[data['Sentiment'] == labels['0']])\n",
    "    num_of_pos = len(data[data['Sentiment'] == labels['p']])\n",
    "    num_of_chars = np.mean( data['Post'].apply(lambda post: len(post)) )\n",
    "    \n",
    "    if do_print is True:\n",
    "        print(dataset)\n",
    "        print('{} positive, {} neutral and {} negative posts.'.format(num_of_pos, num_of_neu, num_of_neg))\n",
    "        print('Posts are on average {0:3.0f} characters long.'.format(num_of_chars))\n",
    "        print('='*65 + '\\n')\n",
    "        return None\n",
    "    \n",
    "    return (num_of_neg, num_of_neu, num_of_pos)\n",
    "\n",
    "\n",
    "\n",
    "def load_gold_data(directory, filter_out):\n",
    "    '''\n",
    "    Loads a dataset with separate contents and labels. Maps labels to our format.\n",
    "    Filters out any samples that have a label equal to the second argument.\n",
    "    \n",
    "    Returns a new DataFrame.\n",
    "    '''\n",
    "    return pd\\\n",
    "        .concat([\n",
    "            pd.read_csv('data/{}/gold-posts.txt'.format(directory), sep='\\n', header=None, names=['Post']),\n",
    "            pd.read_csv('data/{}/gold-labels.txt'.format(directory), sep=' ', header=None, names=['Sentiment']).iloc[:,0].map(labels)\n",
    "        ], axis=1)\\\n",
    "        .query('Sentiment != @filter_out')\\\n",
    "        .reset_index(drop=True)\n",
    "\n",
    "        \n",
    "\n",
    "def load_data(directory):\n",
    "    '''\n",
    "    Loads a dataset whose samples are split to individual files/per class.\n",
    "    \n",
    "    Returns a new DataFrame.\n",
    "    '''\n",
    "    return pd\\\n",
    "        .concat([\n",
    "            pd.read_csv('data/{}/positive.txt'.format(directory), sep='\\n', header=None, names=['Post']).assign(Sentiment=labels['p']),\n",
    "            pd.read_csv('data/{}/neutral.txt'.format(directory),  sep='\\n', header=None, names=['Post']).assign(Sentiment=labels['0']),\n",
    "            pd.read_csv('data/{}/negative.txt'.format(directory), sep='\\n', header=None, names=['Post']).assign(Sentiment=labels['n'])\n",
    "        ], axis=0)\\\n",
    "        .reset_index(drop=True)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⏱ Load the raw datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Facebook CZ\n",
      "2587 positive, 5174 neutral and 1991 negative posts.\n",
      "Posts are on average  60 characters long.\n",
      "=================================================================\n",
      "\n",
      "ČSFD\n",
      "30852 positive, 30753 neutral and 29699 negative posts.\n",
      "Posts are on average 311 characters long.\n",
      "=================================================================\n",
      "\n",
      "MALL CZ\n",
      "102976 positive, 31943 neutral and 10387 negative posts.\n",
      "Posts are on average 122 characters long.\n",
      "=================================================================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Load the Facebook CZ comments, map labels to nicer values and remove posts with bipolar sentiment.\n",
    "facebook_data_raw = load_gold_data('facebook', labels['b'])\n",
    "data_stats(facebook_data_raw, 'Facebook CZ', True)\n",
    "\n",
    "\n",
    "# Load the film reviews from ČSFD.\n",
    "csfd_data_raw = load_data('csfd')\n",
    "data_stats(csfd_data_raw, 'ČSFD', True)\n",
    "\n",
    "\n",
    "# Load the product reviews from MALL.\n",
    "mall_data_raw = load_data('mallcz')\n",
    "data_stats(mall_data_raw, 'MALL CZ', True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### ⏱ Extract one-sentence posts subsets for each of the corpora"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ONE SENTENCE: Facebook CZ\n",
      "1968 positive, 3605 neutral and 1296 negative posts.\n",
      "Posts are on average  53 characters long.\n",
      "=================================================================\n",
      "\n",
      "ONE SENTENCE: ČSFD\n",
      "5039 positive, 6037 neutral and 8464 negative posts.\n",
      "Posts are on average  71 characters long.\n",
      "=================================================================\n",
      "\n",
      "ONE SENTENCE: MALL CZ\n",
      "56712 positive, 13959 neutral and 3682 negative posts.\n",
      "Posts are on average  53 characters long.\n",
      "=================================================================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "def filter_one_sentence(data, max_sentence_length):\n",
    "    '''\n",
    "    Takes a dataset and filters out any post that is longer than one sentence.\n",
    "    Also filter out any sentences that are longer than `max_sentence_length`\n",
    "    \n",
    "    Returns a tuple (filtered DataFrame, original sentence counts Series).\n",
    "    '''\n",
    "    data_sentence_counts = data['Post'].apply( lambda post: len(sent_tokenize(post)) )\n",
    "    data_indices_multisentence = data_sentence_counts[data_sentence_counts > 1].index.values\n",
    "    data_onesentence = data.drop(data_indices_multisentence)\n",
    "    data_word_counts = data_onesentence['Post'].apply( lambda sen: len(word_tokenize(sen)) )\n",
    "    data_indices_longsentense = data_word_counts[data_word_counts > max_sentence_length].index.values\n",
    "    \n",
    "    return (data_onesentence.drop(data_indices_longsentense), data_sentence_counts)\n",
    "\n",
    "# Filter out any multi-sentence posts for Facebook CZ dataset\n",
    "facebook_data_onesentence, fb_s_counts = filter_one_sentence(facebook_data_raw, max_sentence_length)\n",
    "data_stats(facebook_data_onesentence, 'ONE SENTENCE: Facebook CZ', True)\n",
    "\n",
    "# Filter out any multi-sentence posts for ČSFD dataset\n",
    "csfd_data_onesentence, csfd_s_counts = filter_one_sentence(csfd_data_raw, max_sentence_length)\n",
    "data_stats(csfd_data_onesentence, 'ONE SENTENCE: ČSFD', True)\n",
    "\n",
    "# Filter out any multi-sentence posts for MALL CZ dataset\n",
    "mall_data_onesentence, mall_s_counts = filter_one_sentence(mall_data_raw, max_sentence_length)\n",
    "data_stats(mall_data_onesentence, 'ONE SENTENCE: MALL CZ', True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⏱ Plot the sentence counts in our raw data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABCUAAAH4CAYAAAB9kFfEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3X2YXHV58PHvhoAhEmvUGF8qBhNy4wUPFasVKJoKahAq\nolFAUSrxoeWtqAhisaj4hlAVRMFYS6xVEW2DBgm4WtTIi6I+ioLinewivlTBVF4MJBEw+/xxZpbJ\nsLuZsDNzzs58P9e1l2fOnJm5l+Pec3Kf+/f7DYyMjCBJkiRJktRt08oOQJIkSZIk9SeLEpIkSZIk\nqRQWJSRJkiRJUimmlx2AJEnqLRGxK3Ak8GlgB+Bw4F8z8zelBiZJkirHooQkSWq3W4ADgX2BGcDv\nLUhIkqSxDLj6hiRJareIeDrwQ+APwB6Z+buSQ5IkSRVkUUKSJEmSJJXC4RuqnIj4JrBonKevyswX\ndCGGecDPgRdn5lc6/FkjwHGZuayTnyNJ3RIRrwb+AdgTGAF+DJydmVc2HBPAO4EDgD8DfgNcBrw7\nM/+34biJ7p6ckZnviYh3Au9o2L+ZokPjh8AHM3NVG34tSeq6iLgVeCpwZma+c4znXwVcDFyfmXs3\n7J8G/AKYCzw5M9c1ve6dwLGZ+YStfPYlmfnWhxF3K98D9d9tPLtk5q3b+tmaeixKqKr+GzhjjP13\ndzsQSVJrImIA+BSwBPgocBbFtcargCsi4tjM/HhE/DlwLfAD4Djg98BuwOnACyLiLzNzU8NbnwN8\ncYyP/FXD9h3AwbXt7YDH1T738oh4fWYub9OvKUndNgIcSlHIbbZknNf8DfAo4HbgtcCHOhFYs1a/\nB2qHvwx4RNNb7AR8jqKwfHs3Ylb5LEqoqn6fmd8pOwhJ0jb5e4pVN16Qmd9o2H95RPwROC8iLgVe\nD9wHHJyZ99eO+WZEXE9RqHgF8JmG1/+8he+E+8c4ZmVEbALOj4iVmfn7h/l7SVKZvgPsExHzGjsH\nImJHikmFbxzjNa8BVlN0SxxNl4oStPg9kJnrMvOHzS+OiE8Djwb+NjM3didklc2ihKakiPhb4J+A\nZwDTKFp0T83MaxuOeRXwVmBXirtp78/MTzY8fzRwGrALMAy8IzP/s+mjnh4RbweeCfy09hlXNbzH\nM4GzgWcD91PcyTs1M+9uOObw2ufsBtwGLAP+JTMf0pIcEY8AvgrsDPy1s9VLmmLeCHyp6UK07kyK\nbrdZwONr+6Y1HpCZP4yIU4FsY0zvAv4OOAz4WBvfV5K65RpgPkW3xHkN+w8E1lEUc3er74yIGRSd\nCqcDNwAnRsSzM/N7XYi11e+Bdc1PRsSRFMWUt2Xm9R2NUpUybeuHSKUYiIjpTT/bAUTEPsBKiqrx\n3wJHUYxHvrjhmMMoxtddDbwUuAS4KCIOqT3/98C/UYxfPoSiEPD5iHhJUxxnA9+gaC/7FUXb2e61\n9/hL4DqKu31HUhRAXgpc2RDHiRQtaN+k+CL5d+A9tffdQm3s32eAAF5oQULSVBIRT6a4KB5zHp7M\n/EVmvikzb6HIuU8EromIv4+IpzUc94ExLpynjfGd0NI1TO3zbgX23sqhklRVmymuWQ9t2v9yYMUY\nxx8CPBL4T4pr1Z8DSzsZIGzz90Dza+cBF1BcM7+/g2GqguyUUFUdVvtpdDvwBODpwMWZ+eb6ExHx\nAHApRYfBzym6KL6UmSfWDvlaRCwAnhcRl1PcOftEZr6l9vxgRDwWeDfw5YbPXJ6Zb6t9xteAW4CT\nKCbu+WeKC91DMvNPtWPWAN8CXhIRX6YY+3dRZp5ce7+v1iZtOyMi/qVp0qFzgRcCizJzaJv+a0lS\n+Z5U+99fbu3AzFwZEW+huGv2cYCI+AVFHj8nM29reskFtZ9GnweOaDG23/Fgd4YkTUWXAl+OiMdm\n5u8jYnvgJcCLKa5LG70G+Fp9KeaIuBg4ISLe1DRfT7u1/D3QqHYz7zPAn4DXZObmdgemarNTQlX1\nVYohEY0/iwEyc3lmvjYiZkXEX0XEURTJF2CH2vi6ZwCXN75hZh6ZmadQdCLMBb7SeNeNoqr7FxHx\nmIaXfbHh9Q8Ag8Bf13btB1xaL0jUjrka+G3tud2Ax1JUqRt9HtgeeE7Dvn+gKHZ8LDN/1Op/JEmq\nkHou3K6VgzPzXyguYF8LfJZisrM3AT+pd6Q1OIuHfiec3oaYJWmquAq4l6IQAcXKRfdSdA6Pqt1k\nO5BiTp1HR8SjKa6JH03RWdFJ2/Q90OBtFNfX/zcz/6e9IWkqsFNCVXVnZn5/rCciYieKoRevBB6g\nmOvh57WnB4B6UeEhY9VqHlv737FmcoeiG2NDbft3Tc/9L8VQEYDZjD0r8O8oZjueXXvcfEz9PR/V\nsG8P4OvA8RHx4THuEkpS1dVXwnjKeAdExJ9n5q/rjzPzLoq7Y5+pzdh+KPAfFMPcXtbw0l+O953Q\noicCP5nE6yWpVJl5X0RcwYPDgZcAX8zMkWKF5VGHU9z8+hgPnUdnKcXw5k7Z5u+BiNibYsW9ZZk5\n3rW5epydEpqKzqfoRHg+sFNm7gV8ouH5P9T+93GNL4rCc4C7aruO5qF33p7NgwUOKKrKjR7Pg8WO\nOyk6LprNpVia7s6Gx83PUzum7t0UXy5/BD4wxntKUqXVhqP9iGIY2kNExFOBX0XECRHxy9qcO42v\nH6ldkNbn1mmLiNiF4gL5una9pySV5FLgRbUbdIcw9nwSr6GYD+35TT/nAfvXcnFHbMv3QO3xLIpO\nuTXAyWO9Rv3BooSmon2AyzLzWw1LydWT37TMXE9xR+ygpte9t/bzM4qCwBMy8/v1H2BPirkoGsex\njSbV2soYB1FMnglwLfDy+qSWtWP2o+i0+Hbtc35P0dHR6DCK9rbvNuz7Xe2O4duAIyPieS39l5Ck\navko8LJxctiZPLhK0W3AMbUZ4pvtStEB1y6nU8z2/l9tfE9JKsOVFF3BZ1D8O+5bjU/WJg3eB/j3\nzPxm4w8PLgn6ug7H2Or3ABRzBT0ZeJXLf/Y3h29oKvo+8MqIuIbiwvYQoH7H7ZG1/30v8NmIOBdY\nBTyPohX44Mx8ICLeB7ynNknQNcBfAO8DPpOZf2xogzs+In5HUWB4EzCTB5P6eykKE5dFxAUUxYj3\nAt8DVmXmnyLi3cC5EbGe4ouk3qJ2fmY2dkrUXQQcB1wQEXvV5rGQpKliOUVOvjIiPkxxt24WxZKc\nh1CMF/5NRLwZ+BpwfUScD6ylGHp3FEWe3OdhfPb2tTZgKC7WH0/Rxnw48Lpa4VeSpqzMvKc28fqb\ngE81zmtW8xqKVeEuG+O1v4qIa4HXRcS7artnRsQbx/iobzTMcfascY65IDPvr+XddZk5XNvf6vfA\noRRzCi2rxTHWCknDTZPCq0dZlNBU9GaK4sPHKOaU+DHFZD9XUFzMfi8zP1ebvPJ0in/kDwFHZOZX\nADLzgxGxgWIt5X+mmJzygxSrcjR6Q+1nd4o1oPfPzF/U3uN7EfECignYLqW4E3cp8NZ6MSEzPxwR\nmyha0k6gGGt3eu2zHiIzN0fEP1IUSk7iwQKIJFVeLYe9nKJQ/HcUee8BinbeF2Xm12rHXV0bTnc6\nRd59HMXQu28Cf5WZD6dT4jEUXWpQdKPdRtE1d2BmfvVh/1KSVC1fpJjs8tIxnjsSuGqCIuzFwIXA\n/rXHsyhWf2v2jxR5G4pr7APGOObfKLoevg18iloHRqvfAxST0gMcW/sZy9EU82eoxw2MjIyUHYMk\nSZIkSepDzikhSZIkSZJKYVFCkiRJkiSVwqKEJEmSJEkqhUUJSZIkSZJUip5ZfWPduvXO2CmpkubM\nmTVQdgzdYi6WVEXmYUkq33i52E4JSZIkSZJUCosSkiRJkiSpFBYlJEmSJElSKSxKSJIkSZKkUliU\nkCRJkiRJpbAoIUmSJEmSSmFRQpIkSZIklcKihCRJkiRJKoVFCUmSJEmSVAqLEpIkSZIkqRTTyw5A\nklSeiNgXOB9YCPwWODMzL46I2cByYH/g7tr+i8qLVJIkSb3ITglJ6lMRsR3wJeD9mfko4P8Cn4qI\necAngHuAucArgHMiYu+yYpUkSVJvslNCkvrXo4E5wPSIGAA2A/cBfwIOBRZm5ibguxFxMXAU8J2y\ngpUkSVLvsVNCkvpUZv4euBD4HHA/cDVwIvA44P7MvKXxcGC3rgcpSZKknmanhCT1qYiYBmwAXglc\nBrwQuBg4BNjYdPgGYObW3nP27JlMn75dmyOVJElSr+pqUSIiTgHeR9EeXPfizLy66bhXAe+lGMv8\nDeD1mXl71wItwYoVlwCwZMkRJUeioaE1ACxYsLDkSPrb8uXLAFi69NiSI+lpLweek5mn1h6viojL\ngXcCM5qOnUkxx8SE7rxzQ1sD7Db//qXeNGfOrLJDkKYcvxOroR/OQ7c7JfYCTs/MD4x3QETsCSwD\nXgT8GPgI8EngoK5EWJLrrrsGsChRBYODq4De/sOfCm666cdlh9APdgYe0bTvAeAHwHMjYufM/GVt\nfwA/7WZwZfDvX5Kkgt+J1dAP56Hbc0rsBdywlWOOBFZm5vWZuRE4DTgwIuZ2PLqSrFhxCSMjmxkZ\n2TzaMaFyDA2tYXh4LcPDa0erkuq+epdE87ba7mvAMyLi6IgYiIhFwMso5phYCZwVETMj4tnAq4HP\nlhhrx/n3L0lSwe/EauiX89C1okREzKS40/aGiLgtIm6OiKVjHLobDXfjahOx3VF7bU+qd0k0b6v7\n6pXI5m11V2OXhB0TnZOZN1Is9/kG4G7gAuDvMvP7wDHA9sCvgRXAqZl5fVmxdoN//5IkFfxOrIZ+\nOQ/dHL4xF7gG+BiwBHgO8OWI+G1mXtlw3CMpJlRrtNUJ1qby5GoDAzAy8uC24x7Ls/32222x7bmo\nBs9D52Tml4Evj7H/DuCw7kckSZKkftK1okRm/hxY1LDr6oj4NHAo0FiU2ADs2PTyrU6wNpUnV9tn\nn/249tpvjW6vW7e+5Ij61/77H8jPfvaz0W3PRTn22GPP0Q6JPfbYc8qfB4sqU8PixQdz4YXnjW5L\nktSv/E6shn45D90cvvHMiHhr0+4ZwKamfTfTMFQjIh4HPKa2vyctWXIEAwPTGBiY5kSXJVuwYCHz\n5+/K/Pm79vRkMlXXuOKGq2+oW/z7lySp4HdiNfTLeejm8I17gHdExBBwKfB84Ai27J6AYoK11RGx\nHPg+cBZwZW1uiZ617777lR2Canq5CjmV7LHHnmWHoD7k378kSQW/E6uhH87DwEh9MoMuiIiXAO8F\n5lNMnva2zPyviFgGkJnH1o47DHgP8ATgauDozPzdRO+9bt367v0ikrQN5syZNVB2DN1iLpZUReZh\nSSrfeLm4q0WJTjIBS6oqL4YlqVzm4amjvuxhL7eqS/1qvFzczeEbkiRJkjSu+rKHFiWk/tG1iS4l\nSZIkaTxDQ2sYHl7L8PDa0Y4JSb3PooQkSZKk0tW7JJq3JfU2ixKSJEmSJKkUFiUkSZIkla5x6cN+\nWAZRUsGJLiVJkiSVbsGChcyfv+votqT+YFFCkiRJUiXYISH1H4sSkiRJkirBDgmp/zinhCRJkiRJ\nKoWdEpIkSVLJImIucCOwNDMvj4jZwHJgf+Bu4MzMvKjMGCWpE+yUkCRJksp3EfDYhsefAO4B5gKv\nAM6JiL3LCEySOslOCUmSJKlEEXEscC/wq9rjnYBDgYWZuQn4bkRcDBwFfKe0QCWpA+yUkCRJkkoS\nEQuBNwPHNezeFbg/M29p2JfAbt2MTZK6wU4JSZIkqQQRMR34NHBSZt4REfWnHglsbDp8AzBza+85\ne/ZMpk/frq1xSlInWZSQJEmSynEGcENmXtm0fwMwo2nfTIo5JiZ0550b2hSaJLXXnDmzxtzv8A1J\nkiSpHIcDR0TEXRFxF7AzcAlwMLBDROzccGwAPy0hRknqKDslJEmSpBJk5hZzRETErcCJtSVBnwGc\nFRHHALsDrwYO6nqQktRhdkpIkiRJ1XMMsD3wa2AFcGpmXl9uSJLUfgMjIyNlx9AW69at741fRFLP\nmTNn1kDZMXSLuVhSFZmHJal84+ViOyUkSZIkSaqg1auvYvXqq8oOo6OcU0KSpJqhoTUALFiwsORI\nJEmSYHDwCgAWLTqg5Eg6x04JSZJqBgdXMTi4quwwJEmSWL36KjZt2simTRt7ulvCooQkSRRdEsPD\naxkeXjvaMSFJklSWepdE83avsSghSRJs0SFht4QkSVJ3WJSQJEmSJKliFi8+aMztXmNRQpIkYPHi\ng8fcliRJKsOiRQcwY8aOzJixY09PdOnqG5IkUay4MX/+rqPbkiRJZevlDok6ixKSJNXYISFJkqqk\nlzsk6ixKSJJUY4eEJElSdzmnhCRJkiRJKoVFCUmSJEmSVAqLEpIkSZIkqRQWJSRJkiRJUiksSkiS\nJEmSpFJYlJAkSZIkSaWwKCFJkiRJkkoxvewAJEnliIgjgY837Z4J/BtwGrAc2B+4GzgzMy/qboSS\nJEnqdQMjIyNlx9AW69at741fRFLPmTNn1kDZMbQiIl4A/AfwV8B5wEbgGGBP4Erg4Mz8zkTvYS6W\nVEVTJQ+3g3lYUlWNl4vtlJAkERE7Af8OHA/cBRwKLMzMTcB3I+Ji4ChgwqKEJEmStC26XpSIiLnA\njcDSzLx8jOcvBw4A/lTfl5k7dS9CSepLbwFuzMwvRcRewP2ZeUvD8wm8fGtvMnv2TKZP365TMUqS\nJKnHlNEpcRHw2Ame3wt4bmZ+v0vxSFJfq3VJ/CPw4tquR1IM3Wi0gWK+iQndeeeG9gYnSW0wZ86s\nskOQJI2jq6tvRMSxwL3Ar8Z5/vHA44GbuhmXJPW5Q4FfNMwXsQGY0XTMTOCerkYlSZKknte1okRE\nLATeDBw3wWF7AeuByyNiXURcGxH7dCVASepfLwG+0PB4LbBDROzcsC+An3Y1KkmSJPW8rgzfiIjp\nwKeBkzLzjogY79AZwLcpxjYPAUuBKyNit8y8baLPcByzJD1sewPL6g8yc31ErATOiohjgN2BVwMH\nlRSfJEmSelS35pQ4A7ghM6+c6KDMXAmsbNj1sYg4Hng+8LmJXus4ZklVVeWxzBGxHfAU4LdNTx1D\nUaj4NcWwjVMz8/ouhydJkqQe162ixOHAEyPi8NrjRwGXRMR7MvP99YMi4hXAtMxsbCOeAWzqUpyS\n1Fcy80+MMZQvM+8ADut+RJIkSeonXSlKZOZujY8j4lbgxDGWBN0JeH9E3EQxpvmNwI7AV7sQpiRJ\nkiRJ6qKurr4xlohYFhHLADLz34EPA18B7gIOAV6cmfeWF6EkSZIkSeqEgZGRkbJjaIt169b3xi8i\nqefMmTNroOwYusVcLKmKzMOSVL7xcnHpnRKSJEmSJKk/WZSQJEmSJEmlsCghSZIkSZJKYVFCkiRJ\nkiSVwqKEJEmSJEkqhUUJSZIkSZJUCosSkiRJkiSpFBYlJEmSJElSKSxKSJIkSZKkUliUkCRJkiRJ\npbAoIUmSJEmSSmFRQpIkSZIklcKihCRJkiRJKoVFCUmSJEmSVAqLEpIkSZIkqRQWJSRJkiRJUiks\nSkiSJEmSpFJYlJAkSZIkSaWwKCFJkiRJUgUNDa1haGhN2WF01PSyA5AkSZIkSQ81OLgKgAULFpYc\nSefYKSFJkiRJUsUMDa1heHgtw8Nre7pbwqKEJEmSJEkVU++SaN7uNRYlJEmSJElSKSxKSJIkSZJU\nMYsXHzzmdq9xoktJkiRJkipmwYKFzJ+/6+h2r7IoIUmSJElSBfVyh0SdRQlJkiRJkiqolzsk6pxT\nQpIkSZIklcKihCRJkiRJKoVFCUmSJEmSVAqLEpIkSZIkqRQWJSRJkiRJUiksSkiSJEmSpFJYlJAk\nSZIkSaWwKCFJkiRJkkphUUKSJEmSJJXCooQkSZIkSSrF9LIDkCSVJyL+HFgGPA/4A3BOZp4fEbOB\n5cD+wN3AmZl5UXmRSpIkqRfZKSFJfSoiBoAvATcDjwUWA++MiH2BTwD3AHOBVwDnRMTeZcUqSZKk\n3mSnhCT1r+cATwLempl/An4SEfsAm4BDgYWZuQn4bkRcDBwFfKe0aCVJktRzul6UiIi5wI3A0sy8\nfIznXwW8l+Lu3DeA12fm7d2NsvtOPvl4AD70oQtLjkSei2rwPHTFM4GfUHRBHEkxfOO9wI+B+zPz\nloZjE3h590OUJElSLyujU+Iiijbhh4iIPSnGNr+I4qL4I8AngYO6Fp0k9Y/HAM8Hvg7sDDwL+Apw\nMLCx6dgNwMytveHs2TOZPn27NocpSZKkXtXVokREHAvcC/xqnEOOBFZm5vW1408D1kXE3F7ulqjf\nEa5ve2e4PJ6LavA8dM0fgTsy86za4+siYgVwJjCj6diZFHNMTOjOOze0N0L1paGhNQAsWLCw5EjU\nK+bMmVV2CJKkcXRtosuIWAi8GThugsN2A35af5CZvwfuAKKz0UlSX0pgekQ0tjZsB/wQ2CEidm7Y\nHzTkZ6mTBgdXMTi4quwwJElSF3SlUyIipgOfBk7KzDsixq0xPJKiRbhR37UMW82vDs9FNXgeOuZr\nFDn2HRHxLuCvgJcBLwTmAWdFxDHA7sCrcSidumBoaA3Dw2tHt+2WkCSpt3Vr+MYZwA2ZeeVWjtsA\n7Ni0r+9ahtetW192CKrxXFTDVD8PVS2qZObGiPgb4KPA7ygmujwpM79TK0YsA35NkYNPrQ+tkzqp\nsUNicHCVRQlJknrcwMjISMc/JCJ+BjwRqH/YoygKEO/JzPc3HHc2MCczl9YeP47iQnlObSjHuNat\nW9/5X6SDXGmgOjwX1dBL52HOnFkDZcfQLVM9F6t8F1xw7minxPz5u3LCCW8qOSL1AvOwJJVvvFzc\nlU6JzNyt8XFE3AqcOMaSoJ8DVkfEcuD7wFnAlVsrSEiSpN6wePHBXHjheaPbkiT1s36Y/LmMJUG3\nEBHLADLz2My8odYyvBx4AnA1cHSZ8XVLL9wN7hWei2rwPEj9acGChcyYsePotiRJ/aw+rLGXvxNL\nKUpk5ryG7WObnvsC8IVuxyRJkso3NLSGTZs2jm738kWYJEkT6ZfJn7u2JKgkSdLWNE90KUlSv+qX\n70SLEpIkSZIkqRQWJSRJqhkaWjM6oZTK0Ti5pRNdSpL6Wb98J5Y+0aUkSVXRD5NJVd2CBQuZP3/X\n0W1JkvpVv3wnWpSQJIn+mUxqKujlu0GSJG2LfvhOtCghSRIPnUzKokR5/G8vSVKhH74TnVNCkiRJ\nkrSF1auvYvXqq8oOQ33ATglJkijaIy+88LzRbZWnPtloP9wdkiLiMOBM4CnAL4C3ZeaXImI2sBzY\nH7gbODMzLyovUvWbwcErAFi06ICSI1Gvs1NCkiSKfwDPmLEjM2bs6D+GSzY4uKqn12OX6iJiIfBJ\n4PWZuRPwBuDzEfE44BPAPcBc4BXAORGxd2nBqq+sXn0VmzZtZNOmjXZLqOMsSkiSRHF3vn4B5rKg\n5alPODo8vNbzoJ6XmWuAuZl5XURMpyhArAfuAw4F3pGZmzLzu8DFwFHlRat+Uu+SaN6WOsHhG5Ik\n4USXVeF5UL/JzHsiYhdgLcUNw+OA+cD9mXlL46HAy7f2frNnz2T69O06Eqv6x8DAlttz5swqLxj1\nPIsSkiRJUrl+BewIPBe4DDgH2Nh0zAZg5tbe6M47N7Q9OPWfF73oIFauXDG6vW7d+pIjUi8Yr7jl\n8A1JkthycksnuiyP50H9KDMfyMz7M/PrwArgWcCMpsNmUswxIXXcokUHjM6z5ESX6jQ7JSRJopjo\ncv78XUe3VQ7Pg/pJRBwEnJyZL2jYvQMwDBwUETtn5i/rhwM/7XaM6l+LFx9UdgjqExYlJEmq8c58\nNXge1Ed+ADwrIl4LfBY4EDgIeA6wM3BWRBwD7A68uvac1BV2SKhbHL4hSVLNggULvTtfAZ4H9YvM\nvA14CcVSoHcB7wIOzcyfAccA2wO/phjScWpmXl9WrJLUKQMjIyNlx9AW69at741fRFLPmTNn1sDW\nj+oN5mJJVWQenjqWL18GwNKlx5YciVQN9eWxe6FYP14udviGJEmSpEq46aYflx2CVCn1pbJ7oSgx\nHodvSJIkSSpdvUuieVvqV0NDaxgeXsvw8NrRjoleZFFCkiRJUukauyTsmJAe7JJo3u41FiUkSZIk\nSVIpLEpIkiRJKt0ee+w55rbUrxqXyO7l5bItSkiSJEkqXeOKG66+IRWTW86fvyvz5+/a0xNduvqG\nJEmSpEqwQ0LaUi93SNQNjIxM6aWMR031NZkl9a7x1mTuReZiSVVkHpak8o2Xix2+IUmSJEmSSmFR\nQpIkSVIlDA2tYWhoTdlhSOoi55SQJEmSVAmDg6sAenpSP0lbslNCkiRVindKpf40NLSG4eG1DA+v\nNQdIfcSihCRJqpTBwVWjd0sl9Y/Gv3tzgNQ/LEpIkqTK8E6pJEn9xaKEJEmqDO+USv1r8eKDx9yW\n1Nuc6FKSJElS6RYsWMj8+buObkvqD3ZKSJJU4wSL5fNOqdTfFi8+2L99qc/YKSFJUo1L0ZXPO6VS\nf/PvXuo/FiUkSeLBCRbr214Yl8e7pJJUvnrnoN+H5eqH89BSUSIitgeOBb6cmbdGxPuAw4HvA8dl\n5h0djFGS1MS83H7NEyz28pd/1fnfXlOFuVi9zO7BauiH89DqnBIfAP4JeHREHAKcAvwr8ATgIx2K\nTZI0PvOyJJXPXKye5PLM1dAv56HVosThwCsy8wbgMOBrmXk2cAJgj6UkdV9b8nJEnBIR90XEPQ0/\nz42I2RHxxYi4OyJ+GRGv79DvURlOsCjpYfAaWT3J5ZmroV/OQ6tzSuwE/E9ETANeDLy9tv9PwOZW\nPywiDgPOBJ4C/AJ4W2Z+aYzjLgcOqL0/AJm5U6ufI0l9oC15GdgLOD0zP9C4MyL+C7gHmAvsCVwZ\nET/JzO9MOvKKWrBgITNm7Di6LUktaFculqS+1WqnxP8D/hl4H/BnwMqIeCpwNnBdK28QEQuBTwKv\nrxUY3gB8PiIeN8bhewHPzcyd6j8txilJ/WLSeblmL+CGxh0RsRNwKPCOzNyUmd8FLgaOakfgVTU0\ntIZNmzbF9yieAAAgAElEQVSyadPGnm6RlNRW7crFUqXYPVgN/XIeWi1KHA88i2Iin9My89fAG4En\nAie28gaZuQaYm5nXRcR0irtv64H7Go+LiMcDjwduajE2SepHk87LETETCOANEXFbRNwcEUuBXYH7\nM/OWhsMT2K2dv0DV9EuLpKS2mnQulqqovjzz/Pm72j1Yon45D60O31gP/GVmNrahvRV4gOIu262t\nvElm3hMRuwBrKQoix2XmH5oO26v2eZdHxF8Aa4BTMvPbLcYqSf2gHXl5LnAN8DFgCfAc4MvAB4GN\nTcduAGZu7Q1nz57J9OnbtfDR1bP99tttsT1nzqwSo5E0RbTlGlmqol6+Mz+V9MN5aLUo8XOKWYTX\n1Xdk5h9rQzK+RQsXqg1+BewIPBe4LCLWZubXG56fAXwbeAswBCylGMu8W2beNt6bTuULYUl6GCad\nlzPz58Cihl1XR8SngedR5OJGMynmmJjQnXdu2HrkFbX//gfys5/9bHR73br1JUckqV06WGRs5zWy\nVCm9fGd+KumH8zBuUSIijuXByXoGgBsjonnCnkexjcMsMvOB2ubXI2IFxbjlrzc8vxJY2fCSj0XE\n8cDzgc+N975T+UJYUm9r18Vwu/NyRDwTeFFmvr9h9wzgl8DzI2LnzPxl/XDgpw87+Cmg3iJZ35ak\nsXTqGlmS+tVEnRIXUbTrTgOWU0zgc3fD8yMUd82+/tCXPlREHAScnJkvaNi9A3BX03GvAKZl5hca\nds8ANrXyOZLUw9qal2vHviMihoBLKYq/R1B0TzwaOCsijgF2B14NHNSG36HS+qFFUtKktTsXS5W0\nevVVACxadEDJkfS3FSsuAWDJkiNKjqRzxi1KZOb9wH8ARMTPgWsbuhwejh8Az4qI1wKfBQ6kuMB9\nTtNxOwHvj4ibKOaeeCPFcI+vTuKzJWnKa3dezsw1taWa3wt8Cvg1cHRm/qBWjFhW23cPcGpmXj/Z\n36Hq7JCQtDUduEaWKmlw8ArAokTZrrvuGqC3ixIDIyMjLR0YEQcDzwa2p2hVG5WZp7f4Hs8FzgUW\nUkxgeWpmfiMiltXe59jacf8EHAc8lqKYcXxm3jjRe69bt761X0SSumzOnFkDWz9q27UjL7ebuVhS\nFXUqD0P1crF5WO2wevVVrFy5AoCXvnSJhYmSrFhxCdde+y0A/vqvnzflCxPj5eKWJrqMiHOBfwR+\nBDSvltFy4svMqymWTWref2zT47OAs1p9X0nqN+3Ky9rS0NAawI4JSa0xF6tX1bsk6tsWJcpR75Ko\nb0/1osR4Wl1943XA6zLzMx2MRZLUutdhXm67wcFVgEUJSS17HeZiSZqUaS0etxn4TicDkSRtE/Ny\nmw0NrWF4eC3Dw2tHOyYkaSvMxepJixcfNOa2umvfffcbc7vXtFqU+BRwckRs18lgJEktMy+3Wb1L\nonlbkiZgLlZPWrToAGbM2JEZM3Z06EaJliw5goGBaQwMTOvZoRvQ+vCNPwdeChwWEbcC9zU+mZn7\ntjcsSdJWmJclqXzmYvUsOySqoZc7JOpaLUr8pPYjSaoG83KbLV58MBdeeN7otiS1wFysnmWHRDX0\ncodEXctLgladyx9JqqpOLkVXNVM9F19wwbkAnHDCm0qORFI7mYclqXyTXRL07RM9n5nvejhBSZIe\nHvNyZ9ghIWlbmIvbb/nyZQAsXXpsyZFI1dAPy5W3OnzjxWO87qnAI4HL2hqRJKkV5uUO6OUvfEkd\nYS5us5tu+nHZIUiV0g/LlbdUlMjMfZr31WYZPh/4Q7uDkiRNzLwsSeUzF7dXvUuivm23hPpdfbny\n+navFiZaXRL0ITLzT8AHgb9vXziSpIfLvCxJ5TMXP3yNXRJ2TEj9s1z5wy5K1OwLbG5HIJKktjAv\nS1L5zMWS1KJWJ7r8NtA8k++jgN2As9sdlCRpYuZlSSqfubi99thjz9EOiT322LPkaKTy9cty5a1O\ndPmVpscjwH3A9zLzqvaGJElqgXlZkspnLm6jpUuP5eSTjx/dlvrdggULmT9/19HtXtXqRJdndjoQ\nSVLrzMuSVD5zcfvZISFtqZc7JOoGRkaaO87GFhGvAN5K0Y42HVgDnJ+Z/9a58Fq3bt361n4RSeqy\nOXNmDXTifauYl83Faod+WJNd3dWpPAzVy8XmYUlVNV4ubmmiy4j4B+BTwH8DrwKOAAaB8yLi9e0K\nUpLUGvOyetng4KqenmVcvcNc3H4rVlzCihWXlB2GVBlDQ2tGi/W9qtU5JU4FTszMTzbs+1JE3Fx7\n7qK2RyZJmoh5WT2pX9ZkV88wF7fZddddA8CSJUeUHIlUDfUifS9/H7a6JOgTgavH2P8tYF7bopEk\ntcq8rJ7UL2uyq2eYi9toxYpLGBnZzMjIZrslJB4s1A8Pr+3pbolWixI3AS8dY//LgKH2hSNJapF5\nWZLKZy5uo3qXRPO21K/6pVDf6vCNM4DLI2If4Pravr2BlwCv7ERgkqQJmZfVk/plTXb1DHOxJE1S\nS50SmflV4AW1h6+jmMRnE/CczFzZmdAkSeMxL6tX1ddknz9/154eP6veYC5ur3333W/MbalfNRbn\ne7lQ3/KSoFXn8keSqqqTS9FVjblY7eCSoGo38/DU8eY3nwjABz/40ZIjkarhggvOBeCEE95UciST\nN14unnD4RkQ8GTgdeF9m/k/D/o/UXvvOzLy9nYFKksZnXlY/sBihqjMXd44dEtKWerlDom7c4RsR\nMY9ibNyLgMc0PX0TcABwfS0pS5I6zLwsSeUzF3fWkiVHuByo1GDBgoU9X6yfaE6JdwM3Antk5o2N\nT2Tmx4G9gF8C7+pceJKkBuZlSSqfuViS2miiosT+wNsz849jPZmZ9wJvB17YicAkSQ9hXpak8pmL\nJXXN0NCa0bmWetVEc0o8Cvjfrbz+Fzy0bU2S1BnmZUkqn7lYUtcMDq4Cenu+pYk6JX5Gsc7yRPYG\nft6+cCRJEzAvS1L5zMWSumJoaA3Dw2sZHl7b090SExUlLgLeHxHzx3oyInYFzgY+3YnAJEkPYV6W\npPKZiyV1Rb1Lonm714w7fCMzl0XE84CbIuJLwHeBu4HZwF8BhwBXAB/sRqCS1O/My5JUPnOxJLXX\nwMjIyIQHRMThwNHAMyiS7f8C3wP+IzMv7XiELVq3bv3Ev4gklWTOnFkD7Xy/Kudlc7GkKmp3Hobq\n5mLzsNQ7hobWcOGF5wFw/PFvnPLzSoyXi7dalJgqTMCSqqoTF8NVZS6WVEXmYUlT1QUXnAvACSe8\nqeRIJm+8XDzR6huSJEmSJKkkixcfXHYIHWdRQpIkSZKkCprqQzZaMdHqG5IkSZIkSR1jUUKSJEmS\ntIWzz343Z5/97rLD6Hvnn/8Bzj//A2WH0VEtD9+IiP8D/Doz74yIg4AlwPcz82Mdi06SNC7zsiSV\nz1ysXnX77b8tOwQBt956S9khdFxLnRIR8ffAD4E9I2Iv4IvAk4C3R4TlM0nqMvOyJJXPXKxe1dgh\nYbdEeRo7JHq5W6LV4RunAK/LzNXA64AfZeaLgSOApR2KTZI0PvOyJJXPXKye1NglYcdEeRq7JHq5\nY6LVosRTgNW17b8Fvlzb/gXwZ+0OSirTyScfz8knH192GNLWmJclqXzmYkmapFbnlLgFeHFE/AbY\nhQcT7lLg5lY/LCIOA86kSOC/AN6WmV8a47hXAe8F5gLfAF6fmbe3+jmS1AfakpcBImIucCOwNDMv\nj4jZwHJgf+Bu4MzMvKhtkUtS72hbLpaqZO7cJ452SMyd+8SSo+lf8+Y9bbRDYt68p5UcTee02ilx\nBvARYCWwMjNviIhzgZOBt7byBhGxEPgkRYFhJ+ANwOcj4nFNx+0JLANeBTwOuK32OqnjGjsk7JZQ\nxU06Lze4CHhsw+NPAPdQFIZfAZwTEXtPPmRJ6jntzMVSZZx22hljbqu7TjrplDG3e01LnRKZeWlE\n/Dnw5My8obZ7GXBWZv6uxfdYExFzM/OeiJhOcbG7Hriv6dAjKZL69QARcRqwrvbaSnVLfOITF3Dz\nzT8pO4xxPf3pu3PMMSeUHYb6iH8T3dOOvAwQEccC9wK/qj3eCTgUWJiZm4DvRsTFwFHAd9r5O0jS\nVNeuXCxVkR0S1dDLHRJ1LS8JCvwBeGFEvAz4MPBE4K5t+bBaQWIXYC1Fl8ZxmfmHpsN2A77d8Jrf\nR8QdQADjFiVmz57J9OnbbUs4k7bDDtvyn6/7dthhOnPmzCo7jCnN/37bxr+JrptUXq51sL0ZeA7w\ng9ruXYH7M7NxNqUEXt7Ke5aRiyWpZJO+RpaqyA6JaujlDom6lv4FUSskfL12/BOATwEnAM+PiBdm\n5g+34TN/BewIPBe4LCLWZubXG55/JLCh6TUbgJkTvemddza/pPP+7u/+oW3vVR8q8KEPXdi29wRY\nt259W9+v3/jfb9v4NzG2ThRCJpuXax1rnwZOysw7IqL+1COBjU2HbzUH15WRi9tpaGgNAAsWLCw5\nkv7meVC7daog3eZrZEnqS63OKXE+8FXgqcCm2r5XAauAc7flAzPzgcy8v1aIWEHRJtxoA0XRotFM\nivHNUkc1/gO43f8Yltpssnn5DOCGzLyyaf8GYEbTvr7JwYODqxgcXFV2GH3P86AppG3XyJLUr1ot\nSuwHfCgzN9d3ZOYDwHuAv2zlDSLioIj476bdO/DQ9rabKYZq1F/3OOAxOIOxJDWabF4+HDgiIu6K\niLuAnYFLgIOBHSJi54ZjA/hp2yKvqKGhNQwPr2V4eO3onXp1n+dBU8ykr5Elqd+1OgD8j8DsMfbv\nQut3z34APCsiXgt8FjgQOIhiLHOjzwGrI2I58H3gLODKzPx9i58jTYodEpoiJpWXM3O3xscRcStw\nYm1J0GcAZ0XEMcDuwKsp8nVPa7wzPzi4yqEDJfE8aIppxzWyJPW1VjslPg18JCLqFd/ZEXEQ8HHg\n4lbeIDNvA15CsRToXcC7gEMz82cRsSwiltWOuwE4BlgO/A54EnB0i3FKUr+YdF6ewDHA9sCvKYbZ\nnVpfEUmStIVO5mJJ6gutdkr8E/A+4BrgEcD3gAeAC2vPtSQzrwaeNcb+Y5sefwH4QqvvK0l9qC15\nuS4z5zVs3wEc1pYop5DFiw/mwgvPG91WOTwPmmLamoslqR+1VJSojY17S0S8HZhfe91wZtqWJkkl\nMC+334IFC5k/f9fRbZVjwYKFzJix4+i2VGXmYkmavFaXBN0ROIciyZ5X2/fTiBgETsvM+zoYoySp\niXm5M7wzX76hoTVs2rRxdNvChKrMXCxJk9fqnBIfAfanaEmrewvwQuBf2h2UJGmrzMsdsGDBQv8R\nXLLmiS6lijMXS9IktVqUeCnwmsy8tr4jMy8HlgJHdCIwSdKEzMuSVD5zsSRNUqsTXW5X+2l2HzCj\nfeFIklpkXlZPcqJLTTGTzsURsR/wQWA34H+BczLz4xExm2I1uv2Bu4EzM/OitkRdYStWXALAkiXW\ndMp2/vkfAOCkk04pORL1ulY7Jb4MfDQior4jIhYAHwau6ERgkqQJmZfVkxYsWMiTnvRknvSkJzuU\nRlPBpHJxrfBwWe342cArgbMi4gXAJ4B7gLnAK4BzImLvtv8GFXPddddw3XXXlB2GgFtvvYVbb72l\n7DDUB1otSrwR2ATcHBHrI2I9kMAG4B87FZwkaVzmZUkq32Rz8VOBVZl5cWZuzswfAN8A9gUOBd6R\nmZsy87vAxcBRHfktKmLFiksYGdnMyMjm0Y4JlaPeJdG8LXVCq0uC3gn8TUTsDjydoiVtTWb+rJPB\nSZLGZl5WrxoaWsNvfvM/o9t2S6jKJpuLM/MG4LX1x7XOiecCPwbuz8zG29QJvLxdsVdRY4fEdddd\n4xCOEjV2SNgtoU5rdU4JImI74F7gR8BAbd9CgMxc05HoJEnjMi+rFzWvvmFRQlXXrlwcEX9GMRzk\n/1F0S7yh6ZANwMytvc/s2TOZPn2saS6qb2AARkYe3J4zZ1a5AWmU50Kd1FJRIiJeDFxEMaat0QAw\nwtgT/EiSOsS8LEnla1cujohdgMuBYeBwiq6L5okyZ1LMMTGhO+/c0MpHVtI+++zHtdd+a3R73br1\nJUfUv+bNe9poh8S8eU/zXKgtxitutTqnxHnAtcAzgKc1/OxS+19JUneZlztg9eqrWL36qrLD6GuN\nK264+oamgEnn4oh4JnA9MAgcmpkbgbXADhGxc+OhwE/bF3r1LFlyBAMD0xgYmObQjZI1rrjh6hvq\ntFaHb+wMHJiZP+9kMJKklpmXO2BwsJgsf9GiA0qOpH8tWLCQGTN2HN2WKm5SuTgi5gJfAT6YmWfX\n92fm+ohYSbESxzHA7sCrgYPaEHOl7bvvfmWHoJp587zHoe5otShxPfCXgBe/klQN5uU2W736KjZt\n2ji6bWGiHENDa0bPgxNdagqYbC5+PTAHOCMizmjY/2HgGGAZ8GuKYRunZub1k4h1SrBDojrskFC3\ntFqU+C/g4xHxXIp2svsan8zMf213YJKkCZmX26zeJVHftihRDie61BQzqVycme8D3jfBIYdNOsIp\npr4UqMWJ8tWXArU4oU5rtSjxZuAPwCFjPDcCePErSd1lXpak8pmL26y+LKhFifK5FKi6paWiRGbu\n0ulAJEmtMy+33+LFB7Fy5YrRbZVj8eKDufDC80a3pSozF7fXihWXMDKyeXTbwkR56l0S9W27JdRJ\nrXZKEBGPAJYAC4HzgT2BmzPz9g7FJkmagHm5vRYtOsCJLitgwYKFDAxMG92Wqs5c3D71Lon6tkWJ\n8jR2SdgxoU5raUnQ2trJCZwNvA14NHAC8JOI2Ktz4UmSxmJe7ozFiw+yS6Jkq1dfxcjIZkZGNrs8\nqyrPXCxJk9dSUYKi6jsIPBXYVNv3KmAVcG4H4pIkTcy83AGLFh1gl0TJmicclSrOXNxGjcuBujRo\nuRqXA3VpUHVaq0WJ/YAPZebm+o7MfAB4D8UySJKk7jIvS1L5zMVttGTJEQwMTGNgYJpDN0rWOIeE\n80mo01qdU+KPwOwx9u9CsW6yJKm7zMsdUB8uYLdEeZxwVFOMubjN7JCoDjsk1C2tFiU+DXwkIo6t\nPZ4dEQcBFwAXdyQySdJEzMsd4ESX5Vu06IDRooTnQVOAubjN7JCoDjsk1C2tDt/4J+AbwDXAI4Hv\nAV8EVtaekyR1l3m5zVavvopNmzayadNGJ1gsUfMydFLFmYslaZJaKkpk5gOZ+RaK9rT/A+wFPCYz\n30gxy7AkqYvMy+3nBIvV4DJ0mkrMxe23fPkyli9fVnYYoijWW6QvXz+ch5aGb0TEn4AnZOY64CcN\n+3cBbgR26kx4kqSxmJclqXzm4va76aYflx2CahzSWA39cB7GLUpExNHAMbWHA8AVEXF/02FPAH7T\nodgkSQ3My53lBIvVMG/e00Y7JJxkTVVkLu6cxg6J5cuXsXTpsRMcrU6qD2msb/fyP4irrF/Ow0Sd\nEl8AnkKRbPcGvsmWswiP1B6v6FRwkqQtmJc7aNGiA7jssi+ObqscJ510CieffPzotlRB5uIOaeyS\nsGOiXM1DGv1eLEe/nIdxixKZeS/wLoCIuBX4fGZu6k5YkqRm5uXOGhpaw8jI5tHtBQsWlhxRfxoa\nWrPFtudBVWMulqT2amlOicz8VETsHhHPBranqAw3Pv+vnQhOkjQ283L7DQ6u2mLbfwyXw/OgqcRc\n3F577LHnaIfEHnvsWXI0/c0hjdXQL+eh1Yku3wq8D7gDWN/09AhgwpWkLjIvS1L5zMXttXTpsaPD\nt5xPolyLFh3QFxMsVl2/nIeWihLAm4C3ZuY5nQxGktQy83KbLV58MBdeeN7otsrheaiW+jJ0vXwx\nPEnm4jazQ6I6evnO/FTSD+eh1aLETOA/OxmIJGmbmJfbrHGYgEMGyuN5qJZ+uEM3SebiNrNDojr8\nu6+GfjgP01o87lLgNZ0MRJK0TczLbbZixSVjbqu7mpcEVHnqS9Ft2rRxtGNCD2EulqRJarVT4m7g\nnyPilcAa4L7GJzPz1e0OTJI0IfNym1133TVbbC9ZckSJ0fQvlwSsjn5Zim6SzMVtdv75HwBcErgK\nTjvtjQCcffZ5JUfS30455UQAPvCBj5YcSee0WpSYBVzcyUAkSdvEvCxJ5TMXt9mtt95Sdgiquf/+\n+7Z+kDpu8+bNZYfQca0uCXp0pwORJLXOvNx+++67H9de+63RbZXDJQGro1+WopsMc3F71bsk6tt2\nS5Sn3iVR37Zbohz1Lon6dq92S7TaKUFE7A68AVgIHAm8DMjM/FqHYpMkTcC83F5LlhwxWpRw6EZ5\nXBKwOhYtOmC0KOHQjfGZi9unsUvCjolyNXZJ2DFRnsYuiV7umGhposuIOAD4HvBIYG/gEcCTgCsi\n4vDOhSdJGot5uf2c6LIaPA/V4aSjW2culqTJa3X1jfcBp2bmkcD9AJl5OnAq8PYOxSZJGp95uc2a\nJ7pUOTwP1eGkoy0xF7fRvHlPG3Nb3bf99juMua3umjZt2pjbvabV32wP4Mox9l8GtJwxImK/iLg+\nIu6OiOGI+Idxjrs8IjZGxD31n1Y/Q5L6RFvysiRpUszFbdQ4h4TzSZSrcQ4J55MoT+McEr06nwS0\nXpT4H2Cs2ab2B37ZyhtExGyKBP1hYDbwSuCsiHjBGIfvBTw3M3eq/7QYpyT1i0nnZYCIOCwibq4V\ngH8SEYfW9s+OiC/Wisi/jIjXtynuymqc3NKJLsvjeaiOxolGnXR0XG3JxXrQvHlPs0uiIrbffge7\nJCpg2rRpPd0lAa1PdHkW8K8RMR/YDjgwIuYBJ1BM7NOKpwKrMrO+bNIPIuIbwL7Af9cPiojHA48H\nbmrxfSWpH006L0fEQuCTwAsz87pakXhVRDwZWAbcA8yluOC+MiJ+kpnfaf+vUg1OdFkNnofqcNLR\nlrTjGlkN7JCoDjskqqGXOyTqWiq5ZOYngdcBhwD3Au8F9gOOzMx/a/E9bsjM19Yf1zonngv8qOnQ\nvYD1wOURsS4iro2IfVr5DEnqF23Ky2uAubWCxHSKAsR64D7gUOAdmbkpM78LXAwc1fZfpEKc1K8a\nVq++asxtdd/Q0Joxt/WgduRibWnFikuc5LYiTjnlxC2WpFQ5zj773Zx99rvLDqOjWl4SNDOvAK5o\nx4dGxJ8BXwb+X+1/G80Avg28BRgCllLcodstM28b7z1nz57J9OnbtSO8Us2ZM6vsEKRK8W9ifO3I\ny5l5T0TsAqylKFQfB8wH7s/MxvXYEnj5ZD6r6pzUrxoGB6/YYtulKMszOLhqi+0FCxaWGE11tfMa\nWQ9OcGunVPl6eQnKqeT2239bdggdN2FRIiJmAAcBg5l5b23ficCLgHXAeZl547Z8YO3i93JgGDg8\nM7f4f3tmrgRWNuz6WEQcDzwf+Nx473vnnRu2JYzKWrdufdkhSJXSC38T7SysdCIvA78CdqToXrsM\nOAfY2HTMBmDm1t6oVwrEYEGsLAMDW257Hsqz/fbbbbHtuXhQh3Jx31ux4hJGRjaPbluYKE9jh8Qp\np5zYF0MIqqixQ+Lss9/NaaedUWI0nTNuUSIidgZWA0+kmFl4KCL+BTiZ4qJ1OnBNRByQmd9v5cMi\n4pn/v727D5azrhI8/g0kmZAE9SqZoFvlG8gRRUa3lBotXZWdUsSaXUZdHLHQQSbKIBAngBlxwBc0\n2SBEQYlxWaPjjOgwMhJlFqQsX5dRXHVZefMgIs44AwgSNa8aIftH9w2dS9+bl9v9/H63n++nKsXT\n3U93zuXcPvfm9O93HuBa4O+AsyY2JLrnvAbYLzOv6Ll7HrBtz74kSRpNw6jLAJn5u+7hVyLiSuC5\ndOpur/l0ZkxMaSY3iI844sidKySOOOLIkWiIzUQve9mxrF9/5c5j81DO0Ucfww9/+MOdxzM5FwNu\nDg+lFuuRlwS2KVFO7yoJV0yU07tKYpRXTEw1U+J8Ost1F2XmHRFxEJ2BPZ/PzD/JzDfSGe6zRxtc\nImIxnYbERZm5rF9DomshcElEPCMi5kTE2XQ+wbtuD78mSRpVg67Lx0bElyfcPZfOSra53V+8d54O\n3Drtr6BivYP8HOpXTu92DbdulHXooYcxb94BzJt3gFs3djXQWixJbTdVU+LlwHmZOd4WP4bOVOG/\n6TnnWjpXz9gTJwOLgHO7l54b//P+iFgbEWsBMvOTdC4bei3wSzqDg14xvjROklps0HX5+8BzI+LE\niNgvIo6lsxz5Y3S20a2MiPkR8TzgBODTg/giatU7WM0ha+Wcd97yvsdq3h133M62bVvZtm2rgy53\nNeharC4vCVyP3ktQjvrlKGu2ePHj+x6PmqlmSjwG6B0s+VLgd8BXeu77NZ0ivFuZuQJYsYfnrqTT\nYZYkPWzQdfmeiPhj4IPApcDtwHGZ+cOIWELnsqA/o7Nt4+zMvGH6X0K9XDZch02bNvY9VvMcdDmp\ngdZiPezVr/5TB11W4sILP7LzksDOkyhn+fJzd+ZhVOdJwNRNiZ/SWa77LxGxP/AK4JsTViy8BPjJ\n8MKTJPUYeF3OzG/SmSEx8f4HgOOnFa0kjSZ/Rx4iV0jUwxUSdRjlFRLjpmpK/A2d2Q7vAv4IOBg4\nffzBiHgRnWsxXzrUCCVJ46zLQ/SCF7yQ66//xs5jlbFw4YE7V0gsXOjVHkp6+ctfyZo1H9p5rJ2s\nxUPkCol6uEKiDqO8QmLcVO2vVcAXgTXAccA5mXklQERcQmfq8PXd8yRJw2ddHqLeX4T9pbic9753\nVd9jNe/QQw/jkEOexiGHPM2tG7uyFkvSAE26UiIzHwTe3v0z0WXAusy8cViBSZJ2ZV0ernXr1u5y\n7BU4ypg4cNQGUVmukHgka/FwjQ+4tSlZ3vgsg9Wr1xSOpN3akIeptm9MKjNvGnQgkqR9Z12evptv\n/kHfYzXLgaN1cYXE3rEWT58DbqX2cXqJJEmSpOK8JHA9xj+dn3isZrUlDzYlJEkCjjjiyL7Halbv\nkHTIiDsAABr/SURBVFEHjpa3bt3aXbY2ScPkJYGldrIpIUkS7DJDwnkS5ThwtC433/wDtzNJkobK\npoQkSTxywKLKuOSSC/seq3kTh79Kw9Z7GWAvCSy1h00JSZJ45IBFlXHXXXf2PVbzHP6qpnlJ4Hr0\nXulhlK/6ULu25GGfrr4hSZIkSYPmCgmpfWxKSJJEZ6ji9dd/Y+exynjyk5+6c4XEk5/81MLRtNsR\nRxy5c4WEw1/VFFdI1GOUP5mfSdqQB7dvSJKEAxZrccYZZ/U9VvMc/ipJaoJNCUmSgHPOObPvsZp1\n3nnL+x6reeZCJZx//l9z/vl/XToMAcuWncqyZaeWDqP1zjrrNM4667TSYQyV2zckSQK2bdva91jN\n2rRpY99jNc9cqIQNGx4oHYJUlYceeqh0CEPnSglJkiRJxfWukHC1RFm9KyRcLVFO7wqJUV4tYVNC\nkiRg3rwD+h6rWb2T953CX5a5UNN6V0m4YkLadZXEKK+YsCkhSRKwYsVFfY/VrN7J+07hL8tcSJKa\nYFNCkiQc6leL5cvf1vdYzXP4q5o2NvbYvsdSW+233359j0fN6H5lkiTtBYf61WH79t/2PVbzHP6q\npp177vv6Hqt5q1ev6XusZl144Uf6Ho8ar74hSZIkqQqukJB2NcorJMbZlJAkic4gv/EVEg71K2fO\nnLk7V0jMmTO3cDTtNm/eATtXSDj8VU1xhUQ9XCFRh1FeITFu9NsukiTtAYf61WHVqg/1PVbzHP6q\nEpYtO9VLUFbCXNShDXmwKSFJEu25FnjtHHRZj1Wrzu97LEnSINmUkCSJ9lwLvHYOuqzHvffe3fdY\nGpbeT4NH/ZPh2pmLOrQlDzYlJEmSJElSETYlJEmiPdcCr13vcEsHXZa1ePHj+x5LkjRI/tYlSRLt\nuRZ47Rx0WY/ly8/teyxJ0iDZlJAkSZJUXO8lKL0cZVnmog5tyYNNCUmSaM8wqdqZh3qceeZb+x5L\nkjRIs0sHIEmSpPrs2LGj77E0TKP8afBMYy7q0IY8uFJCkiRJkiQVYVNCkiRJjzBr1qy+x9IwLVt2\nqlu3KmEu6tCGPNiUkCSJ9gyTqp15qMdFF13a91iSpEGyKSFJEnD22Wf0PVazHHRZj3POObPvsTQs\nvv/rYS7q0JY82JSQJAl48MHf9T2W2mrbtq19jyVJGiSbEpIkSZIkqQibEpIkAfvvP7vvsdRW8+Yd\n0PdYkqRBsikhSRLwgQ9c0vdYzXLQZT1WrLio77E0LL7/62Eu6tCWPDTalIiIF0bEDRHxq4j4cUS8\nZZLzXhcRd0bE5oi4OiIWNxmn2q0Nl92R9EhtGSZVO/NQj+XL39b3WJKkQWpsfWpEjAFfAE4DPgs8\nG/hyRPw4M7/cc96RwFrgZcAPgA8DnwCObSpWSZKkttu+/bd9j6VhGuVPg2cac1GHNuShyZUSTwL+\nKTMvz8yHMvP7wFeBF0w47/XA+sy8ITO3AsuBY1wtoSb4CZ0kSZIkNaexlRKZeSNw4vjt7sqJFwGf\nmnDq04Fv9TzvFxHxABDAvdOJYSb8I7P2GNvQqWuT2r/foP4YZ/p7IiJeCFxEp/beD1yQmR/r1uh1\nwNHAr4D3ZObHy0UqqWlz5szduUJizpy5haNRW4z/3J/pP19HgbmoQxvyUGS8eEQ8Gvgi8L3uf3st\nALZMuG8LMH+q1xwbm8/s2fsPLEb1t2jRgaVDaFTbvl7tvZn8PTLVtjrgFGATsBg4ErgmIm7JzG+X\ninfYVq9e04of/LUzD/VYtepDO3OxatWHCkcjSRpVjTclIuIpwNXAj4HXZuZDE07ZAky87tR8Or8c\nT2rDhol9jMktHVuwx+eq4+INmwG4776NhSNpVlu+3rEFS0uHMONs2HwxsGffIxU3LnZuq+ve/n5E\njG+rOw44LDO3Ad+JiMuBNwAj25SYuH3LfxCXYR7qYS7UNL/n6mEu6tCWPDR99Y3/CNwAfAk4rjsz\nYqLb6GzVGH/OQcBju/dLQ9WWy+5I0NlWl5n9ttXNArZn5p29p9PZ4iFJkiQNTJNX31gMXAtclJmr\npjj1M8DXI2Id8F1gJXBNZv6igTAlqZUmbKv7KjBx+cxut9HBaG2lq3iFS6uYh3qYC0nSMDS5feNk\nYBFwbkSc23P/xcDjADLzlMy8MSKW0BmwdjDwTeCkBuNUy7lCQm0zcVsdcDgwb8Jpu91GB3u3la52\nbdm+VTvzUI+ZnAsbKpJUr8a2b2TmisyclZkLJ/x5Z7cZcUrPuVdk5mGZ+ajMfGVm/rypOCWpTSbZ\nVvcjYG5EPLH3VODWAiE2xu1bdTAP9TAXaprfc/UwF3VoSx4anSkhSarHhG11y8YHD2fmRmA9sDIi\n5kfE84ATgE+Xi3b4Jg6TUhnmoR7mQpLUhCKXBJUkVWGqbXVLgLXAz+hs2zg7M29oPkRJUpuM8qfB\nM425qEMb8mBTQpJaKjNXACumOOX4pmKRJAkeXpXThn+I1c5c1KENebApIUmSJBUWEUcBV2XmE7q3\nx+gMfj8a+BXwnsz8eMEQJWkonCkhSRLtGSZVO/NQD3PRjIiYFRFvAq4D5vY8dBmd7XOLgdcAF0TE\nHxYIsTHOMamHuahDW/JgU0KSJNrzg7925qEe5qIx5wBLgfeP3xERC4HjgHdl5rbM/A5wOfCGMiFK\n0vC4fUOSJEkqZx2d+T4v7rnvacD2zLyz574EXrW7Fxsbm8/s2fsPNsJCFi06sHQI6jIXdRjVPNiU\nkCRJkgrJzLsBIqL37gXA1gmnbgHm7+71NmzYMrDYSrvvvo2lQ1CXuajDTM/DZE0Vt29IkiRJddkC\nzJtw33w6MyZGlnNM6mEu6tCWPNiUkCSJ9vzgr515qIe5KOpHwNyIeGLPfQHcWigeSRoamxKSJOFQ\nv1qYh3qYi3IycyOwHlgZEfMj4nnACcCny0Y2fKtXr7EJVglzUYc25MGmhCRJklSfJcAc4GfAlcDZ\nmXlD2ZAkafAcdClJkiQVlplfAw7quf0AcHyxgAoZX5Uz6p8MzwTmog5tyIMrJSRJkiRJUhE2JSRJ\nwqF+tTAP9TAXappzTOphLurQljzYlJAkifb84K+deaiHuZAkNcGmhCRJkiRJKsKmhCRJkiRJKsKm\nhCRJkqTinGNSD3NRh7bkwaaEJEm05wd/7cxDPcyFJKkJNiUkScKhfrUwD/UwFyph9eo1NsEqYS7q\n0IY82JSQJEmSJElF2JSQJEmSVIVly051ZU4lzEUd2pAHmxKSJEmSJKkImxKSJOFQv1qYh3qYCzXN\nOSb1MBd1aEsebEpIkkR7fvDXzjzUw1xIkppgU0KSJEmSJBVhU0KSJEmSJBVhU0KSJElScc4xqYe5\nqENb8mBTQpIk2vODv3bmoR7mQpLUBJsSkiThUL9amId6mAuVsHr1GptglTAXdWhDHmxKSJIkSarC\nsmWn2gSrhLmoQxvyYFNCkiRJkiQVYVNCkiRJUnFuGaqHuahDW/JgU0KSJBzqVwvzUA9zIUlqgk0J\nSZJoz6cRtTMP9TAXkqQm2JSQJEmSJElF2JSQJEmSVJxbhuphLurQljzYlJAkSZIkSUXYlJAkifZ8\nGlE781APc6ESVq9e4/dbJcxFHdqQhyJNiYg4KiL+fYrHr46IrRGxafxPk/FJktrHoX51MA/1MBeS\npCbMbvIvi4hZwEnAauB3U5z6HOBFmfndRgKTJEmSJO003owc9U/pVV7TKyXOAZYC75/shIj4feD3\ngZubCkqSJEmSJDWv6abEOuDZwP+Z4pznABuBqyPivoi4PiKe30h0kiRJktRybt9SkxrdvpGZdwNE\nxFSnzQO+BbwduAN4E3BNRDw9M++Z7EljY/OZPXv/AUarfhYtOrB0CFJVRuU9ERFHAVdl5hO6t8fo\nNJKPBn4FvCczP14wxKFbvXqNS1UrYB7qYS60py677FJuu+2W0mFM6vDDn8mSJW8tHYZaxPfE3mm0\nKbEnMnM9sL7nro9GxKnAS4HPTPa8DRu2DDs0Affdt7F0CFJV9uQ9UXPjYopZP5cBm4DFwJF0msO3\nZOa3m4+yGRM/FfIfYWWYh3qYC0lSE6prSkTEa4D9MvOKnrvnAdsKhSRJo+wc4Hg6s36WA0TEQuA4\n4LDM3AZ8JyIuB94AjGxTQpK0bwb5iaurczQKfE/sneqaEsBC4L9HxM3Aj4C3AQcA1xWNSpJG0zpg\nBfDinvueBmzPzDt77kvgVbt7sVHaSlfzCpc2MQ/1MBdSe7h9S02qoikREWsBMvOUzPxkRDweuBZ4\nHPB94BWZublkjJI0iiaZ9bMA2Drh1C3A/N293ihtpXO7Wh3MQz1mci5sqEhSvYo0JTLza8BBPbdP\nmfD4SmBlw2FJkjq20Nk212s+nRkTI8tPhepgHuphLqR2832vpjR9SVBJUv1+BMyNiCf23BfArYXi\naYSXP5Mk6WHLlp3qz0M1oortG5KkemTmxohYD6yMiCXAM4ETgGPLRqaaDePyZ4P8Zbi2y58Ny7Au\nQ2cuJEnD4koJSVI/S4A5wM+AK4GzM/OGsiFJkqQmuHpQTXKlhCSp36yfB+hcKlTaI17+rA6DXoFg\nLiRJw+ZKCUmS2PUfXf4DTJIkqRk2JSRJkiRJUhE2JSRJ6lq9eo2rJCRJrefqQTXJmRKSpBlrWFca\nGCSvNCBJkjQ5mxKSJEmSpF24QkJNsSkhSZqxvNKAJEnSzOZMCUmSJEnSLpYtO3Vns14aJpsSkiRJ\nkiSpCJsSkiRJkqSdeldIuFpCw2ZTQpIkSZIkFWFTQpIkSZIkFWFTQpIkSZIkFWFTQpIkSZK0U++l\nsb1MtobNpoQkSZIkSSpidukAJEmSJEl1cYWEmuJKCUmSJEmSVIRNCUmSJEmSVIRNCUmSJEmSVIQz\nJSRJaqFly04tHcJu1R7jIPZb1/41jqs5Tve9S9LMZlNCkiRJapGam0zjao+xxmbYZZddym233VI6\njCkdfvgzWbLkraXDeITav9+g/hin856wKSFJUostWDpWOoQZZ/PFGwb+mmMLlg78NUfdhs0Xlw5B\nkjQANiUkSZKkFlo6tqB0CDPOxRs2lw5hUoNegTD+yXyNq0KGxQbx3htEg9hBl5IkSZIkqQhXSkiS\nGlX7nkioO8Y2fWIlSZJGnyslJEmSJElSEa6UkCQV4V7mvVPzPmZJkqR95UoJSZIkSZJUhE0JSZIk\nSZJUhNs3JElqsc0XbygdghjMJdWkveW2sPJqHqw8ruYYBz382VpchislJEmSJElSEa6UkCSpxRYs\nHSsdwowzjNUlYwuWDvw1R52faE6fA4f33rBWl1iL986wVvlZi/feIGqxKyUkSZIkSVIRNiUkSZIk\nSVIRbt+QJBXhgLU6OOiyDm5FkCS1lSslJEmSJElSEUVWSkTEUcBVmfmESR5/HfB+YDHwVeDkzLy3\nwRAlSUPmgLW9M+iVJYO+jNogjV9+ruYYB6X2r7FNuZAkldHoSomImBURbwKuA+ZOcs6RwFrgdcBB\nwD3AJxoLUpIkSZIkNaLp7RvnAEvprIKYzOuB9Zl5Q2ZuBZYDx0TE4iYClCRJkiRJzWh6+8Y6YAXw\n4inOeTrwrfEbmfmLiHgACGAgWzgcrlbeZZddym233VI6jBnp8MOfyZIlbx3oazpgTSVYi0fLMOr6\n+NaBQRhG7azRsH6+movRZB2uh0OH6+DvxGU02pTIzLsBImKq0xYAWybctwWYP9WTxsbmM3v2/tOK\nT7u3aNGBA3mduXO98Mu+mjt39sDyoOkzF5IkSdK+m7Vjx47G/9KIeAnwucw8qM9jXwCuz8xVPffd\nDxyXmf97ste8776NzX8hkrQHFi06cFbpGPZFRDwH+BjwTOBHwCmZ+e2pnmMtllSjmVqH94V1WFKt\nJqvFNV4S9DY6WzUAiIiDgMd275ckNSAi5gFfpDNo+DHAJcAXImJh0cAkSZI0UmpcQ/8Z4OsRsQ74\nLrASuCYzf1E2LElqlZcCD2XmR7u310XEXwLHAleUC0uSJEmjpIqVEhGxNiLWAmTmjcASOkMxfw48\nATipYHiS1EZPB26dcF9275ckSZIGoshKicz8GnBQz+1TJjx+BX4SJ0klOXRYkiRJQ1fj9g1JUnlb\ngAMm3Dcf2DTVkzZsmNjHkKTyZuqVkvZl4LAkzTRVbN+QJFVnl6HDXcEjt3RIkobAgcOS2sKVEpKk\nfr4C/F5EnA6sBU4EFgNfKhqVJLWHA4cltYIrJSRJj5CZvwFeAbwOeAA4Hfgvmbm5aGCS1B4OHJbU\nCq6UkCT1lZk/AF5QOg5JaikHDktqBZsSkiRJUn0cOCxppEw2dNjtG5IkSVJ9HDgsqRVcKSFJkiTV\nx4HDklrBlRKSJElSZRw4LKktXCkhSZIkVciBw5LawJUSkiRJkiSpCJsSkiRJkiSpCJsSkiRJkiSp\nCJsSkiRJkiSpiFk7duwoHYMkSZIkSWohV0pIkiRJkqQibEpIkiRJkqQibEpIkiRJkqQibEpIkiRJ\nkqQibEpIkiRJkqQibEpIkiRJkqQibEpIkiRJkqQiZpcOYFRFxA7gY5l5yoT77wJOy8yrI+KTwAnA\nb/u8xOLM3Nx9zrOAdwIvBh4FPAD8E/DOzPzFsL6GUdHNxVbgIWBH98+3gDMz8+buObOANwOnAocA\nW4BvAO/JzJt6XusoYCXwPGAWcAuwIjO/0NgXNIMN6n3RfZ1njeev3+sMPnrNRNbieliL62AdVtOs\nw/WwDtfDWrwrV0oM15sj4pjdnHNJZi7s82e8+P4h8M/ArcAzgIXAfwLmA9d1C4d276ju/9cDgccB\nNwHXRMT+3cfXAH8JnNl9/FDgO8D1EfF8gIh4DHAd8Hfdc8aAC4DPdguz9sy03xfSXrIW18NaXAfr\nsJpmHa6Hdbge1uIuV0oM1/8E1kXEEZn5wD6+xho634zv7bnvJxFxMvBe4DHAhmnG2SqZuT0iPkGn\n2I5FxCHAScChmfmz7mm/AS6IiAV0cvAc4DDgAOCzmbm9e94/RsR7gEc3+kXMbIN4X0h7w1pcIWtx\nUdZhNc06XCHrcHHW4i6bEsP1YeCpwEeB1+7tkyPiiXTe+K+e+Fi3ALxjugG2UUSMAWcAN2fm/RGx\nFLi+p/j2+iRwXkQ8Gfh/wE+AGyLicjpL2b6XmauaiXxkTOt9Ie0Da3GFrMVFWYfVNOtwhazDxVmL\nu2xKDNcOOt3GmyLihMy8vM85p0XEn0+479rM/FPgCd3b/zb+QESsBP6ie3Mu8JbM/NsBxz2K/jki\nHuoe/wa4gYd/sC0G7pnkeXd3/3twZt7VXTp4GvAq4H3Atoj4FJ29eFuHE/rIme77Qtpb1uJ6WIvr\nYB1W06zD9bAO18Na3GVTYsgy818j4nTg0oj4Rp9TPpKZZ03y9J93//t44Kfd13sH3W5wRHwX2L//\nUzXBCyYOgOlxL52BSf08qfvfewAy85d0Cu/7ImIh8J+B1cB2YOngwh1t03xfQGfgT7/6NZvOD1hp\nF9bialiLK2EdVtOsw9WwDlfEWtzhoMsGdLu2XwHW0ZlOu6fPuxO4GXjTkEJTx1XAURERfR57I3BT\ntyP89oj4+vgDmbkpM9cDHwSe3VCsI2Nf3xdd/8bDPxwB6O51XAz0W3IoWYvrZy1umHVYTbMOV886\nXIC12JUSTXoLnWK6eC+ftwT4UkQ8SOeyMfd293KdAfwBD3eOtY8y83sR8T+AL0bEXwDfpHOZqT8H\nTgfGp+KuB94VEe+gM+hnE53pzycDn2488NGwr++LvwfeHRG3ZebtEbEIWAH8IDNvG3SQGinW4kpZ\ni4uxDqtp1uFKWYeLanUtdqVEQzLzfjrFdKIzImJTnz8v6T7v28BzgacA34uIzXQuh3Qw8PzM/F8N\nfQkjLTPPoHMpowuA+4Ef0rnu8gsz8/ruOQm8lM7lp+4Efg18Dvhb4KICYc94+/q+AN4FXA1cGxGb\n6Fwb+wDg2EYC14xlLa6btbh51mE1zTpcN+twGW2vxbN27NhROgZJkiRJktRCrpSQJEmSJElF2JSQ\nJEmSJElF2JSQJEmSJElF2JSQJEmSJElF2JSQJEmSJElF2JSQJEmSJElFzC4dgDSZiJgNvB34M+BJ\nwC+BLwHnZuZPB/D6C4HjM3Nd9/bXgG9n5l9N97WnG4sk1cJaLEllWYc16lwpoZqtAN4InAEE8F+B\nxcDXI2L+AF7/TODNPbdfBbxvAK87iFgkqRbWYkkqyzqskeZKCdXsZOCtmXlt9/ZdEfHfgPuBY4HP\nTfP1Z/XeyMwHpvl60zFr96dIUhHWYkkqyzqskTZrx44dpWOQ+oqI+4D1wCmZ+bue+w8D7s7MjRFx\nEnAO8B+AW4B3ZOaXu+d9EtgIPJpOx3cL8MHMXBkRfwZ8Yvw1M3NW71K1iHg3cDjwU+AUYDPwV3SK\n/8V0utNXASeNxzbIWKb/f0+SBsNaLEllWYc16ty+oZqtptMZ/teIWBcRJ0bEosy8vVt8jwU+QKcw\nHgl8Crg6Ip7V8xpvAe4E/gC4CFjRffzvu7e/Dzx+kr//OGAO8Bw6HeiPAucCrwVeDxwPvBqggVgk\nqRRrsSSVZR3WSLMpoWpl5ko6Re424EQ6Re3fI+LCiJhFp9ityswrM/OOzPww8I/A0p6XuT0z352Z\nP87MVcADwPMycyuwCdiemfdMEsIm4KzM/DGd4nsA8L7M/F5mfgG4EXhG99xhxyJJRViLJaks67BG\nnTMlVLXM/AfgHyLiQOBoOkN+zgT+hU7xOyoi3tXzlLnADT23b5/wkhvpdHr3xF2Z+WD3eEv3v3f2\nPL4V+L3u8bBjkaRirMWSVJZ1WKPMpoSqFBFHAksy83SAzNxIZy/d+oi4CngZne/fs4FrJjz9Nz3H\nv+3z8nu6P217n/semuTcYcciSY2zFktSWdZhtYFNCdVqf+C0iPhcZn59wmO/Bh6ks4TtSZl5x/gD\nEbECuAe4ZA/+jkFOea0pFkkaFGuxJJVlHdbIsymhKmXm/42IzwNXRMQ5wNeARwEvB/4EeBHwFODy\niEjgK8AfA8uBV+7hX7MJODginpqZd+727KldUFEskjQQ1mJJKss6rDZw0KVq9jrgUmAZcBPwDeCl\nwB9l5o2Z+Xk6Q3PeDtxKZ5LvifnwNZx350o6y9FujoiDpxNoTbFI0oBZiyWpLOuwRtqsHTtcISNJ\nkiRJkprnSglJkiRJklSETQlJkiRJklSETQlJkiRJklSETQlJkiRJklSETQlJkiRJklSETQlJkiRJ\nklSETQlJkiRJklSETQlJkiRJklTE/wfFJdNIk7ldJAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x117c60668>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.set_context(\"notebook\", font_scale=1.3, rc={\"lines.linewidth\": 2})\n",
    "\n",
    "fig = plt.figure(figsize=(18,8), dpi=72)\n",
    "subplots = []\n",
    "\n",
    "datasets_to_plot = [\n",
    "    (facebook_data_raw, fb_s_counts, 'Facebook'),\n",
    "    (csfd_data_raw, csfd_s_counts, 'ČSFD'),\n",
    "    (mall_data_raw, mall_s_counts, 'MALL.CZ'),\n",
    "]\n",
    "\n",
    "for i, dataset in enumerate(datasets_to_plot):\n",
    "    subplots.append(fig.add_subplot(1, 3, i+1))\n",
    "    sns.boxplot(x='Sentiment', y=dataset[1], \n",
    "                data=dataset[0], \n",
    "                palette='hls', order=['NEG', 'POS', 'NEU'],\n",
    "                ax=subplots[i])\n",
    "    subplots[i].set(title=dataset[2], xlabel='Sentiment', ylabel='Sentences Counts')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⏱ Plot one-sentence vs multi-sentence occurences"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABDAAAAH4CAYAAAC8Fx1oAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3XmcndP9wPHPmJHEEhLJiKLorz/9WloEXWhQSxAhVG2R\n0qK0SlSopUgFTdIiCCWWilrbUioRYqulIZba6tdqj9ijtmyyr5P5/fE8M24mM8kkZubezHzer9d9\nmfuc85x7noyc58n3nvM9ZdXV1UiSJEmSJJWy1YrdAUmSJEmSpOUxgCFJkiRJkkqeAQxJkiRJklTy\nKordAUmS1HZFxOZAP+BWoB1wOHB9SumDonZMkiSVHAMYkiSpmN4C9gV2BjoAUwxeSJKk+pS5C4kk\nSSqmiNgSeBmYAXw1pfRJkbskSZJKkAEMSZIkSZJU8lxColVaRDwB7NZA8V9TSnu1QB82A94GeqWU\nHmzmz6oGTkwpXducnyNJLSUijgR+DGwDVAOvAr9JKY0tqBPAIGBPYF3gA2A0cFFKaXJBvWV9KzMw\npfSriBgEnF9wfDHZzI+XgWEppfub4LIkqcVFxDvApsAFKaVB9ZT3Be4Ankspfavg+GrAu0A3YKOU\n0qQ65w0CfpJS2mA5n/3HlNLZK9HvxtwHaq6tIV9KKb2zop+tVY8BDLUGjwID6zk+vaU7IklqnIgo\nA24Gvgf8FhhK9lzSF3ggIn6SUrouIjYGngZeAk4EpgBbAOcAe0XEDimleQVNXwz8pZ6PnFjw81Sg\nd/5zOdA1/9wxEXFcSmlkE12mJLW0auAgsqBvXd9r4JzvAOsAHwNHAZc1R8fqaux9IK/+XaB9nSbW\nBv5AFoT+uCX6rOIzgKHWYEpK6dlid0KStEJOINt9ZK+U0uMFx8dExHzgioi4BzgOWAD0TiktzOs8\nERHPkQU1DgFuKzj/7UbcExbWU2dURMwDroyIUSmlKSt5XZJUTM8CO0XEZoUzEiJiDbKEyf9Xzznf\nB54km4VxDC0UwKCR94GU0qSU0st1T46IW4FOwP4ppbkt02UVmwEMtXoRsT/wC2A7YDWyacJnpJSe\nLqjTFzgb2JzsW7pfp5RuKig/BjgL+BLwJnB+SumuOh+1ZUT8EtgeeC3/jL8WtLE98Bvg68BCsm8I\nz0gpTS+oc3j+OVsAHwHXApeklJaaFh0R7YGHgU2Ab5u1X9Iq5lTg3joPrTUuIJtF1xFYPz+2WmGF\nlNLLEXEGkJqwTxcCPwAOA0Y0YbuS1FKeAr5MNgvjioLj+wKTyAK/W9QcjIgOZDMgzgFeAU6OiK+n\nlP7eAn1t7H1gUt3CiOhHFng5N6X0XLP2UiVlteVXkUpeWURU1HmVA0TETsAosmj0/sDRZOun7yio\ncxjZesBxwIHAH4EbI6JPXn4C8Duy9dZ9yIIGf4qIA+r04zfA42RT3CaSTX3bOm9jB2A82beI/ciC\nJQcCYwv6cTLZNLgnyG46vwd+lbe7hHyt4m1AAD0NXkhalUTERmQP0PXmDUopvZtSGpBSeotszP0C\n8FREnBAR/1NQ79J6HrJXq+ee0Kjnnfzz3gG+tZyqklSqFpM9sx5U5/jBwN311O8DrAXcRfas+jZw\nbHN2EFb4PlD33M2Aq8memX/djN1UCXIGhlqDw/JXoY+BDYAtgTtSSqfXFETEIuAespkLb5PNzrg3\npXRyXuWRiPhfYNeIGEP2jdwNKaUz8/KHIqILcBFwX8FnjkwpnZt/xiPAW8ApZEmJziN7KO6TUqrK\n67wO/A04ICLuI1ureGNK6bS8vYfzhHQDI+KSOgmVLgd6ArullN5YoT8tSSq+DfP/vre8iimlURFx\nJtm3cdcBRMS7ZOP4xSmlj+qccnX+KvQn4IhG9u0TPpv1IUmronuA+yKiS0ppSkSsDhwA9CJ7Li30\nfeCRmu2rI+IO4KSIGFAnv1BTa/R9oFD+xd9tQBXw/ZTS4qbumEqbMzDUGjxMtiyj8LUPQEppZErp\nqIjoGBHfiIijyQZqgHb5esDtgDGFDaaU+qWUfk42w6Eb8GDht3lk0eJtI2K9gtP+UnD+IuAh4Nv5\noR7APTXBi7zOOODDvGwLoAtZ9LvQn4DVgW8WHPsxWWBkRErpH439Q5KkElIzFpY3pnJK6RKyh92j\ngNvJErkNAP5VM9OtwFCWviec0wR9lqRVxV+B2WRBC8h2cJpNNiO5Vv6F3L5kOYA6RUQnsmfiTmQz\nNprTCt0HCpxL9nz9o5TSf5u2S1oVOANDrcG0lNIL9RVExNpkyz8OBRaR5aZ4Oy8uA2oCEEutrct1\nyf9bX0Z7yGZ5zMl//qRO2WSy5SoAnak/O/InZFmfO+fv69apaXOdgmNfBR4DfhoRw+v59lGSSl3N\njiBfbKhCRGycUnq/5n1K6VOyb91uyzPXHwTcQrbU7rsFp77X0D2hkb4A/OtznC9JRZVSWhARD/DZ\nkuTvAX9JKVVnu1LXOpzsi7IRLJ3351iyJdbNZYXvAxHxLbKdB69NKTX0bK5WzhkYau2uJJvhsDuw\ndkqpO3BDQfmM/L9dC0+KzDeBT/NDx7D0N3pf57NgCGTR6kLr81lgZBrZTI66upFt5zet4H3dcvI6\nNS4iuxHNBy6tp01JKmn5krh/kC2FW0pEbApMjIiTIuK9PEdQ4fnV+cNrTS6gJhERXyJ7mB7fVG1K\nUpHcA+ydf5nXh/rzX3yfLH/b7nVeVwB75GNxs1iR+0D+viPZDLzXgdPqO0dtgwEMtXY7AaNTSn8r\n2H6vZqBcLaU0k+ybtv3qnDc4f/2HLHiwQUrphZoXsA1Z7ozCdXe1A3C+Q8h+ZIlBAZ4GDq5J2JnX\n6UE2g+OZ/HOmkM0UKXQY2RS75wuOfZJ/E3ku0C8idm3Un4QklZbfAt9tYAy7gM92a/oIOD7PlF/X\n5mQz65rKOWRZ7//chG1KUjGMJZttPJDs33x/KyzMEyLvBPw+pfRE4YvPtlH9YTP3sbH3AchyG20E\n9HXL1LbNJSRq7V4ADo2Ip8gegvsANd/krZX/dzBwe0RcDtwP7Eo2Hbl3SmlRRAwBfpUnQHoK2BYY\nAtyWUppfMBXvpxHxCVkwYgCwJp/dAAaTBTFGR8TVZIGLwcDfgftTSlURcRFweUTMJLvp1EyTuzKl\nVDgDo8aNwInA1RHRPc+7IUmripFkY/LYiBhO9i1gR7JtTPuQrW/+ICJOBx4BnouIK4EJZMv/jiYb\nJ3daic9ePZ+KDNmD/fpkU6kPB36YB4klaZWVUpqVJ5UfANxcmIct932y3fFG13PuxIh4GvhhRFyY\nH14zIk6t56MeL8jJtmMDda5OKS3Mx91JKaU38+ONvQ8cRJYD6dq8H/XtFPVmnYT3aqUMYKi1O50s\nUDGCLAfGq2SJjB4ge/D9e0rpD3liznPIAgJvAEeklB4ESCkNi4g5ZHtVn0eWeHMY2e4khX6Wv7Ym\n22N7j5TSu3kbf4+IvciSy91D9g3fPcDZNYGHlNLwiJhHNi3uJLK1gefkn7WUlNLiiOhPFlQ5hc+C\nJZJU8vIx7GCyoPIPyMa9RWRTivdOKT2S1xuXL+k7h2zc7Uq2/O8J4BsppZWZgbEe2ew3yGa5fUQ2\nG2/flNLDK31RklRa/kKWyPOeesr6AX9dRsD2DuAaYI/8fUeyXfDq6k82bkP2jL1nPXV+Rzab4hng\nZvKZHY29D5Al3Af4Sf6qzzFk+T7UypVVV1cXuw+SJEmSJEnLZA4MSZIkSZJU8gxgSJIkSZKkkmcA\nQ5IkSZIklTwDGJIkSZIkqeS1yV1IJk2aaeZSSSWpsrJjWbH70FIciyWVIsdhSSq+hsZiZ2BIkiRJ\nkqSSZwBDkiRJkiSVPAMYkiRJkiSp5BnAkCRJkiRJJc8AhiRJkiRJKnltchcSSWrNIqIHMAzYApgM\nXJxSui4idgSeA+YWVB+SUhoSEWXAEOBHZPeGW4DTUkpVeZt9gcFAN+Bx4LiU0sd5WXfgOmBrYALw\nk5TSs81/pZIkSWpLnIEhSa1IRHQGRgPDgc7AocDQiNgL6A6MTSmtXfAakp96EtAb2AbYEvg2cHre\n5jbAtUBfoCvwEXBTXtYBuC9/3wm4EhgdEWu3wOVKkiSpDXEGhiS1LpsC96eU7sjfvxQRjwM7AxsA\nrzRw3lHAFSmlDwEiYihwEXAx0A8YlVJ6Li87C5gUEd2A7YHFKaUReTsjI2IAsB9wZ5NfnSRJktos\nAxiS1IqklF4hC0YAtTMydiFbEtILmBcRbwPlZAGGc1NK88mWm7xW2FR2epTlZc8UfMaUiJgKRD3n\n1Zy7RRNfmiRJkto4AxiS1EpFxLpkyztezP97HPAEWb6KbsBdwAXA2cBawJyC0+eQLTNsX09ZTfma\nyylbps6d16SionxFLkmSJEltmAEMSWqFIuJLwBjgTeDwlNJioE9BlbciYghZ4s6zyYIOaxSUrwks\nSinNi4i6ZTXls+o5r7BsmaZNqxv3kKTiq6zsWOwuSJIaYBLPNujDDz9gjz12bpHPOu20k5k6dQoA\nhxxyAP/85/+1yOdKbVlEbE+228hDwEEppbkR0TkiLo2IwifzDsC8/Od/ky0JqW0mP7ZUWUR0BdbL\nj9c9r+bcustKVMBxWGrdImLjiBgTETMi4v2IOCU/3jki/hIR0yPivYg4ruCcsogYGhGTImJaRAyP\niPKC8r4R8VZEzM7b7lZQ1j0ins/LXomIb7XsFUutg/fn0mcAQ83q+eefpbq6utjdkNqM/IH2QWBY\nSum0fOYFwHTgu8CgiFg9Iv4XOJd8NxHgNuCM/KG7G/AL4Na87A/A9yKiR77ryFCy3UymAI8B7SOi\nf97usWTLUx5qgctVIzgOSy0rzx10L1mAtwuwD9nYuzNwA9kMtW7AIcDFBcEGd4OS2hDvzyvHJSSq\ntXDhQq699re8/PILVFUtZrPNvsSAAWfSqVMnTj75BLbe+mv861//x0cffcjWW3+VgQMvoqKigmee\neYoRI65itdXK2Xzzr/DCC89zzTW/43e/uxaAU0/9Kb/+9WUAjBlzL5dffjHTpk1l9933pH//04p5\nyVJrdBxQCQyMiIEFx4cDB5A92E4G5pLlwhiel19D9kD9PFnei9uAyyBLDBoRxwMjyXYyGQcck5fN\nj4heZA/WQ4A3gD4ppdnNeI2tluOw1Cp8E9gQODulVAX8KyJ2IpvxdhDwlZTSPOD5iLgDOBp4FneD\nkkqW9+fSYQBDtW677fdUVy/mxhtvo6ysjJtvvpErrriEQYMGA/D+++8xfPgIFixYwFFHHcbf//4s\nW231VS688JdcccU1RGzB2LFjGDt2DAADB17IQw89wBVXXEOXLl0BqKhYnRtvvJVPP/2UQw89gAMO\n+C6bbfalol2z1NqklGryWjRkrwbOqwLOy1/1ld9JAw/CKaVXybZp1efkOCy1CtsD/yKbXdEPmAEM\nBl4FFqaU3iqom4CD85/dDUoqUd6fS4cBDNUaP34cM2fO5KWXXgSgqmoRFRWf/S+y8867UF5ezhpr\nrMEmm2zKp59+yiuvvMwmm2xKRHaP7NVrf6644pIGP6Nnz30A6NSpE127VjJt2lT/YkpSznFYahXW\nA3YnW2K3CbAj2dK+3mSz3woV7trkblBSkc2fvxawdDLf558fz4wZM3j11ZcBqKqqoqKigsrKjrRr\nV8E++/Rkgw06AfC///tlqqrm8fbb/+HLX/4fevT4OgBHH92X4cMvZb311qptv0uXtams7Eh5+Woc\neuh3qazsSGVlR7p160Z19TyTCtfDAIZqVVUt5uSTB9Cjx64AzJs3j7lzP7sftm/fvqB2GdXV1ZSX\nly+1dqusrOHUKuXlFQX1ylz3JUkFHIelVmE+MDWlNDR/Pz4i7ibbtrpDnbqFuza5G5RUZFOnZitg\nJ02aucTx+fMXcuKJP1vq/jxp0kwWLFjEggXVtecsWFDFjBlzgXYsWLCoTltlTJ06m/bts2NTpswC\nOlBVtZiZMxfU1l28uJpp02Yv1Y+2pKHgjUk8Vetb39qZe+65i/nz51NdXc3w4Zdy+eUNf4sHsM02\n2/LBB+8zYUIC4Ikn/sqsWTMpKysDoLy8nKqqqmbvuyS1Bo7DUquQgIrCHUSAcuBloF1EbFJwvHDX\nJneDkkqU9+fS4QyMNmrBggX07LnLEscuvfRK5syZw3HHHUV19WI23fRLnH12vcvha62zzroMGjSE\nwYMvYLXVyojYivLyctq3z75g2G23PTj55BO46KLfNNu1SNKqyHFYarUeIZsVcX5EXAh8g2wXqJ7A\nZsDQPDHy1sCRZMk24bPdoB4DFrL0blBPRsRI4AUKdoPK67ePiP5kCZWPwt2gpJXm/bm0lbXFqaOT\nJs1sexfdTGbPnsXNN9/Iscf+mA4dOvCf/7zGWWcN4N57H6yNLkpqvMrKjm3mL45jcdNwHJaaVlOM\nw/lW1b8lC17MAC5IKd0UEeuRBRn2IlviMSilNDI/p5xsmcmxfLYb1Gl5kmUi4jDgVxTsBpVS+iQv\nq9lm9Wtku0GdmFJ6dnn9dByWmo/358+nobHYAIY+t+uvv4a//e0JKioqqKiooH//AWy7bfdid0vN\nYNGIm4vdhQZVnPiDYnehSRjA0Mpo6+NwvzG3F7sLy3T7/v2K3QWtAMdhlTKfxVYtbf3+/HkYwCjg\nYC2tHG+azc8HZ2nFGcBQU3IcVinzWUxtRUNjsUk8JUmSJElSyTOAIUmSJEmSSp4BDEmSJEmSVPIM\nYEiSJEmSpJJnAEOSJEmSJJW8imJ3YFXQUtl+zdy7fB988F823HCjYndDUgtzHC4djsOSpPo0573a\n+/PytZX7szMwVmHvvfcuZ599Gvvuuzs9e+7KD37QlzFj7gXglltGctFFAz9X+x9++AE9euzInDlz\nmqK7n9tTTz3J+ef/otjdkKRajsOSJJUe78+tlzMwVlGLFy/m9NNPoXfvA7jggiGsvno7/vGPlznn\nnDNYe+2OHH30scXuYpObMWMGixe7Xbmk0uA4LElS6fH+3Lq1aAAjIg4DLgC+CLwLnJtSujcidgSe\nA+YWVB+SUhoSEWXAEOBHeX9vAU5LKVXlbfYFBgPdgMeB41JKH7fUNRXL9Omf8uGH/6Vnz31p374D\nAN2778BPf3oKCxcu5MYbr+Ptt9/kV7+6mMGDB7HWWmvx+uuJCRMSm2yyGWeeeS4RW1BdXc1NN93A\nPffcRbt27Tj00L5ce+1V/PGPf1nqM9988w0uv/xi3njjddZfvxsnntifnXbqUW//JkxIXHLJUN57\n7x06d16PPn0Opm/f7y+3nUMOOYADD/we998/mmnTprDttt0577wL+e9/J3LppUNZtGgRffrsw+jR\nDzFjxnSGD7+U5557lg4dOnDggd/j+9//AWVlZcu8ZoC//OXP3HHHrUyf/ilbbrkVZ555LhtttPEy\n25SkQo7DjsOSpNLj/bl1359bbAlJRHwFuIkswLA28DPgTxHRFegOjE0prV3wGpKfehLQG9gG2BL4\nNnB63uY2wLVAX6Ar8FH+Ga1e587r0b37DgwYcBI33ngdL730AnPnzuWAAw6iZ899l6r/4IMPMGDA\nmYwZ8wgbb7wx1133WwDuv380DzxwHyNG3Mitt/6Jf/zjJaqqqpY6f86c2Zx22knssUdPxox5lFNP\nPYMLL/wl7733br39u/zyS9h997148MEnGDz4En7/+xt47713G9XOuHFPMGLE77jjjruZOPE9Ro26\nmy233Jqf//wXbL55MHr0QwBcdNEvgTLuums0V111HQ8//AAPPHDfcq/52WfHc/3113DBBYMZO/Yx\nIrbM21p+m5JUw3HYcViSVHq8P7fu+3OLBTBSSq8D3VJK4yOigmzGxExgAVkA45UGTj0KuCKl9GFK\n6SNgKPDDvKwfMCql9FxKaS5wFrBvRHRrxkspGZdeeiWHHHI4L730Aqef3p/99tuD888/h+nTP12q\nbo8eu7L55l+hffsO7LHH3kycOBGAhx8ey2GH9eWLX9yEtdZam5/+9JR6P2v8+Kfo1Gk9Dj74UCoq\nKth++x3ZZZfdGDt2TL3127Vrx/jx43j66XFsuOFGjB37OJtssmmj2jnwwIPp3Hk9unTpyje/uTMT\nJ763VPtTpkzm2WfHc8opp7HGGmvwhS9syJFHHs1999273Gt+9NGH6NWrN1tt9VXKy8s55pjjOfXU\nnzeqTUkq5DjsOCxJKj3en1vv/blFl5CklGZFxJeACWTBkxNTSjMiojswLyLeBsqBO8mWl8wHtgBe\nK2wGiHxpyRbAMwXtT4mIqUAADS4j6dx5TSoqyhvd7w8bXfPzqazsuIJndOSkk37MSSf9mPnz5/Pi\niy9y6aWXMmzYELbaaivat1+dysqOdOiwOp07d65tv0uXjqy2WvZ506ZN4X//d7PasnXW+QoA6623\nVu2ndO26NrNmTePdd9+mV6/da49XVVXRs2dPnnnmcc4///za4/fffz9XXTWcK664gssv/w1Tp06l\nd+/eDBw4cJntVFZ2pLx8NTbddMOC/qxJVdV8Kis70rFjByoqVqOysiMffvg21dXVHHHEd2vbWbx4\nMZ06dVruNc+aNZ1tt/1qwZ93RzbZZH1effXVZbaplvu7sDL8HakY2rdvz2GHHclhhx3J/PnzefXV\nVxgx4iqGDr2QzTePJep26tS59ueKigqqqxcDMHnyJNZf/7O4e7duX6j3sz7++CPeeect9t33O7XH\nqqqq2HXX3Xn44bFccsmQ2uO33noXgwYN4frrr2bYsF/z6afT2HPPvRkw4MxlttNwX5deV/vxxx9R\nXV3N4YcfVHts8eJq1llnneVe89SpU/nylzevLVtjjTXYYouteO21fy63TUmSlsf7c+u9PxcjiedE\nYA1gF2B0REwAJgFPANeRzcy4iyxXxtnAWkBhetc5ZMGP9vWU1ZSvuawOTJtWGtli65o0aWaj6/71\nrw9z7bVXc+ed99auO9p8869x9NE/4rLLfsOmm36Z+fMXMmnSTObNW8icOQtq258+fS5VVYuZNGkm\nXbpUMmHC22y/fVb2/vtZ9G3q1Nm1nzV58iw6dOjI1lt/jauvvqH2+CeffEz79u1Zd91OPPzwZ3+x\nqqur+fvfX+T44/vTv/8ZvPHGBAYNOofrrhtJZWVlg+1MmjSTqqrFTJ8+t7avc+YsYN687DpmzpzH\nokVZv1dbbQ3Ky8sZNeoh2rVrB2TJa+bMmbPca1533fV4552JtWWzZ89i5MgbOPTQI5bZpkpba/kd\nGYhZddQdh9u3b8/Xv/5NFixYwGWX/WapB6SGrL9+Nz7++KPa95MmfVJvvS5duvLVr27T4Di89969\nao9XV1fz8ssvcsopp3PmmefWjsN3330nlZWVDbazIrp06Up5eTmjRz+81Ji5PJWVlUye/Nl11h2H\nV6ZNSZLA+3Nrvz+3+DaqKaVFKaWFKaXHgLuBg1JKfVJKl6WUZqeU3iJL2lnzNfgcsoBHjTWBRSml\nefWU1ZTPat6rKL4dd/wGc+fOZvjwYUybNpXq6mref38if/7zH/n2t3dpdDv77XcAf/7zn3j//YnM\nnTuX66+/pt56O+/cg3fffYdHHnmQqqoq3nnnbU444YeMG/fEUnXLysq4/PKLuf32m1m0aBFdu1ZS\nVlbGuuuuu0Lt1LX66qszZ85sqqur6dZtA7bdtjsjRlzF/PnzmDFjOgMHnsUNN1y93Hb23rsXDz54\nP6+//h8WLVrELbfcxGuv/ZMNNvjCSrcpqe1xHHYcliSVHu/Prfv+3JJJPPeLiEfrHG4HlEXEpRFR\n+LVjB2Be/vO/yZaE1DaVH1uqLE8Iul5Beau17rqduOaa3zF58iSOOupw9tqrB6ee+lO23HJr+vc/\nrdHt9Oy5L3vttQ/HH/8D+vU7hI022hjI/hIUWmeddRk27CruvfduevfekwEDTuKgg77H/vsfVF+z\nDBo0hBdf/Du9e+/J979/CDvu+E169+6zwu0U6t59BwB69dqd+fPnM2jQYKZOncIhh/ThiCMOpmvX\nrpx22lnLbWeHHb7OiSeewi9/eQ69e+/Jm29OYNCgwXm/V65NSW2P47DjsCSp9Hh/bt3357L61s00\nh4jYAPgP0B+4HdgX+AOwE3AfcC/ZkpFNgdHA9SmlKyKiP3AC0AtYCNwP/CmldElEbAc8SbZLyQvA\nVcCGKaXey+rLpEkz28YmuY0wYcLrdO68Hl27dgXgnXfe5uijD+eRR/5Wu+2QVGPRiJuL3YUGVZz4\ng2J3oUlUVnZsM3tFOhZnHIc/v35jbi92F5bp9v37FbsLWgGOwyplPou1HO/PxdXQWNySu5B8BBxA\ntn3qp8CFZMtHXsuPbwtMBp4iy4ExPD/1GmAU8DxZMs+ngcvyNl8BjgdGAp8AGwLHtMwVtQ7PPjue\niy76JXPmzGH+/HncfvvNbLfd9v6llKQW4jgsSVLp8f5cmlp6F5JxwI71HH8N2KuBc6qA8/JXfeV3\nku1aopVw+OFH8t//TuSww/qwcOFCunffgYEDLyx2tySpzXAcliSp9Hh/Lk3F2IVEJaRdu3acffZA\nYGCxuyJJbZLjsFqTUl7O41IeSSvC+3NpavFdSCRJkiRJklaUAQxJkiRJklTyDGBIkiRJkqSSZwBD\nkiRJkiSVPAMYkiRJkiSp5LkLSSO0VEbtYmfHXrRoEVOnTmH99bsVtR+f1wcf/JcNN9yo2N2Q1ITa\nyjgMjsWSpFXHD75Y8M/JZrxXF/v+7L25dDgDYxXVo8eO7Lnnt5kzZ/YSxxctWkTv3ntyyCEHNLqd\nt956A4BBg85h3LgnAPjHP15uVBsffvgBPXrsyJw5c1bsAprJU089yfnn/6LY3ZDURjgW18+xWJJU\nLN6b69da7s0GMFZhHTp0YNy4J5c49uyz41m4cNFKtffpp5/W/rzttt3585/v+1z9K4YZM2aweHF1\nsbshqQ1xLF6aY7EkqZi8Ny+ttdybDWCswr7znT159NGHljj2yCNj2W233Wvfv/TSC/TuvecSdXr3\n3pOXXnphiWPDhw/j1Vdf4be/vYKrrrp8ifMeeOA+fvazn3LeeWey1149+P73D+OFF56vt09vvvkG\nJ598Avubav7qAAAgAElEQVTu+x2OPvpwnnnmqQb7P2FC4oQTfsi++36Hvn0P5g9/uK1R7RxyyAHc\neuvvOeKIg9lnn90488xTmTFjBv/+97+49NKhTJiQ6NNnHwBmzJjORRcNZP/9e9aeV12d/cUdPHgQ\nV1xxCT/96Y/o2XMXjjvuKFL6T+3n/OUvf+bQQw9k771342c/O5H//vf95bYpqe1xLC7+WPz+NTcy\n/ZnnHYslSYD35lK4NzfXv5MMYKzC9txzb1566QWmT88ignPmzOaVV17m29/eZYXb+tnPTmebbbbj\n5JNPpX//AUuVv/ji82y11dd48MEnOPLIozjnnDOYNm3aEnXmzJnNaaedxB579GTMmEc59dQzuPDC\nX/Lee+/W+5mXX34Ju+++Fw8++ASDB1/C739/A++9926j2hk37glGjPgdd9xxNxMnvseoUXez5ZZb\n8/Of/4LNNw9Gj84GrIsu+iVQxl13jeaqq67j4Ycf4IEHPouYPvjgAwwYcCZjxjzCxhtvzHXX/RbI\nIrTXX38NF1wwmLFjHyNiy7yt5bcpqW1xLC7+WLzBkYcw+1//Yfb/vbbCf+aSpNbHe3Px783N9e8k\nAxirsE6dOrHddjvwxBOPAfDkk4+z8849WH31dk3+WRtvvAlHHnkUFRUV7LffAWy44UaMHz9uiTrj\nxz9Fp07rcfDBh1JRUcH22+/ILrvsxtixY+pts127dowfP46nnx7HhhtuxNixj7PJJps2qp0DDzyY\nzp3Xo0uXrnzzmzszceJ7S7U/Zcpknn12PKecchprrLEGX/jChhx55NHcd9+9tXV69NiVzTf/Cu3b\nd2CPPfZm4sSJADz66EP06tWbrbb6KuXl5RxzzPGceurPG9WmpLbFsbj4Y3FFp3VZ55s7MOsf/9cU\nf8ySpFWc9+bi35ub699J7kKyiuvZcx/uv380Bx54MA8/PJajjz62WRLFbLTRxku8X3/99Zk6dcoS\nxz7++CPeeect9t33O7XHqqqq2HXX3Xn44bFccsmQ2uO33noXgwYN4frrr2bYsF/z6afT2HPPvRkw\n4MxltlOjU6fOtT9XVFTUOzXp448/orq6msMPP6j22OLF1ayzzjrLaGcxAFOnTuXLX968tmyNNdZg\niy224rXX/rncNiW1PY7FxR2LZy9cCNXVrNahQ/1/cJKkNsd7c+v8d5IBjFXcrrvuzrBhv+E///k3\n//3v+2y33faMH//ZOqjy8vIlktUsWrSI2bNn19fUMk2ePGmJ9x999CF77rn3Ese6dOnKV7+6DVdf\nfUPtsU8++Zj27duz7rqd2HvvXrXHq6urefnlFznllNM588xzeeONCQwadA53330nlZWVDbazIrp0\n6Up5eTmjRz9Mu3ZZtHXGjBmNGrgqKyuZPPmT2vezZ89i5MgbOPTQI1a6TUmtl2Nxw1piLO435naq\n5s2jev6CFeqbJKn18t7csFX530kuIVnFrbnmmuy887f51a/OZ88996asrGyJ8o022ph58+by4ot/\np6qqittu+z1VVVX1ttWuXbsG/9K++eYExo4dw6JFi7jvvnuZPHkyO+3UY4k6O+/cg3fffYdHHnmQ\nqqoq3nnnbU444Ye1Ww4VKisr4/LLL+b2229m0aJFdO1aSVlZGeuuu+4KtVPX6quvzpw5s6murqZb\ntw3YdtvujBhxFfPnz2PGjOkMHHgWN9xw9XLb2XvvXjz44P28/vp/WLRoEbfcchOvvfZPNtjgCyvd\npqTWy7F4SS09FlfNncfkv4zh0789vdw2JUltg/fmJbWWfyc5A6MRbt+/X7G7sEw9e/bir389jYsu\n+vVSZV27VnLiiacwePAg5s2bx/7792HzzaOBdvblsssu5sMPP6Rnz32WKNt0080YP/4phg+/lI03\n3oRLLx3OOuusw+zZs2rrrLPOugwbdhVXXjmMYcN+zRprrMlBB32P/fc/qO5HATBo0BCGDfs1d955\nB6uvvjo9e/aid+8+lJeXr1A7hbp33wGAXr12Z9Sohxg0aDDDhw/jkEP6UFVVxU477cxpp5213HZ2\n2OHrnHjiKfzyl+cwbdoUvva1bRk0aHDe75VrU9LKK/VxGByLC7X0WDx97hzW+PJmrLf3HsttU5LU\ndG6e+NkMhooTf1DEntTPe/NnWsu/k8ra4pZjkybNbHsX/Tk88MB93H33ndx4463F7oqKbNGIm4vd\nhQaV4k1zZVRWdixbfq3WwbF4xTgWN6zfmNuL3YVlWhUCcE2llH8Xjf09OA6rlPksVlq8NzefhsZi\nl5BIkiRJkqSSZwBDkiRJkiSVPHNgaLn22+8A9tvvgGJ3Q5LaNMdiSZJKi/fmlucMDEmSJEmSVPIM\nYEiSJEmSpJJnAEOSJEmSJJU8AxiSJEmSJKnkGcCQJEmSJEklzwCGJEmSJEkqeQYwJEmSJElSyTOA\nIUmSJEmSSp4BDEmSJEmSVPIMYEiSJEmSpJJnAEOSJEmSJJU8AxiSJEmSJKnkGcCQJEmSJEklzwCG\nJEmSJEkqeQYwJEmSJElSyTOAIUmSJEmSSp4BDEmSJEmSVPIMYEiSJEmSpJJnAEOSJEmSJJU8AxiS\nJEmSJKnkGcCQJEmSJEklr6LYHZAkSZJak4j4OTAEWFBwuBfwT2AksAcwHbggpXRjfk5Zfs6PyJ7R\nbwFOSylV5eV9gcFAN+Bx4LiU0sd5WXfgOmBrYALwk5TSs818mZLU4pyBIUmSJDWt7sA5KaW1C17j\ngBuAWWRBiEOAiyPiW/k5JwG9gW2ALYFvA6cDRMQ2wLVAX6Ar8BFwU17WAbgvf98JuBIYHRFrt8SF\nSlJLMoAhSZIkNa3uwCuFB/KAwkHA+SmleSml54E7gKPzKkcBV6SUPkwpfQQMBX6Yl/UDRqWUnksp\nzQXOAvaNiG7A7sDilNKIlNLClNJI4GNgv+a9RElqeQYwJEmSpCYSEWsCAfwsIj6KiH9HxLHA5sDC\nlNJbBdUTsEX+8xbAa3XKIl9askRZSmkKMDX/nLrn1W1XkloNc2BIkiRJTacb8BQwAvge8E2yJR7D\ngLl16s4B1sx/Xit/X1i2GtC+nrLCc5dVtkydO69JRUX58qqphHxY7A4sQ2Vlx2J3QW2AAQxJkiSp\niaSU3gZ2Kzg0LiJuBXYFOtSpviZZTgzIgg5r1ClblFKaFxF1ywrPXVbZMk2bVjfuIa28SZNmFrsL\nakUaCoi1aAAjIg4DLgC+CLwLnJtSujciOtMMGZklSZKklhQR2wN7p5R+XXC4A/AesHtEbJJSeq+m\nOp8t//h3/v65grJ/1ymr+YyuwHr58Y7AyXW7QZZfQ5JalRYLYETEV8iyI/dMKY2PiL2A+yNiI7Ks\nyjUZmbcBxkbEv/LtnwozMlcDY8gyMl9ckJF5b+BV4Kr8M0xaJEmSpGKYBZwfEW8A95Al2TyCbFZG\nJ2BoRBxPtuXpkXz23HobcEZEPAYsBH4B3JqX/QF4MiJGAi+QJfgcm1KaktdvHxH9yZ6LjyJ7pn6o\n2a9UklpYiyXxTCm9DnTLgxcVZAPrTLL9sZsjI7MkSZLUovJn3sOAX5I9614DHJNSegk4HlgdeB+4\nGzgjpVQz4+IaYBTwPNmsjKeBy/I2X8nPHQl8AmwIHJOXzQd6kW2xOhXoD/RJKc1u7muVpJbWoktI\nUkqzIuJLwASy4MmJwJepPyPzwfnPy8vI/ExB+1MioiYjs8tIJLVJEdGDLFncFsBk4OKU0nXNtVwv\nIroD15F9mzgB+Ek+g06S2qSU0n1kiTvrHp9KFtyo75wq4Lz8VV/5ncCdDZS9Cuy8sv2VpFVFMZJ4\nTiRLNLQLMBq4mObJyNwgMy5LK8fM16UvD1KMJlsP/UdgO+DRiHgT+AlNvFwvIjqQPaQPBn5HNmtu\ndET8T0ppuQnkJEmSpMZq8QBGSmlR/uNjEXE3sCPNk5G5QWZcllqf1pL5ugkCMZsC96eUapK3vRQR\nj5N9M3cQ8JWU0jzg+YioWa73LAXL9QAiYihwEVmQuXa5Xl52FjApX663PbA4pTQi/7yRETGAbE13\nvd8USpIkSSujxXJgRMR+EfFoncPtgDeBdhGxSWF1ls7IXFjWmIzMktTmpJReSSkdVfM+n5GxC1BG\n/cv1tsh/Xt5yvdqylNIUsnXWUc95dduVJEmSmkRLzsB4CdgxIo4Cbgf2JfuG7pvAJjRxRuaWuSRJ\nKl0RsS7Z8o4XyfJW/KxOlaZYrrdSS/nA5XxqO1ziVhr8PUjSqq/FAhgppY8i4gDgcuBq4HXgoJTS\nf/LAxbVkGZlnsXRG5m5kGZnbkwU0ajMy5+eOBDYAxpFnZJaktixPmDyGbJbb4cCWNM9yvZVayget\nYzlfvzG3F7sLDbp9/37F7oJyrWWJ26qusb8HAx2SVLpaeheScWQ5L+oeb5aMzJLUFkXE9sCDZAHf\nn6eUFkfEBPLleiml92qqsvRyvecKyhqzXK8jWcLQJbpAth22JEmS1GSKsQuJJKmZ5Ik1HwSGpZR+\nU3M8pTQzIkbRxMv18vrtI6I/2Uy6o8hmzT3UzJcqSZKkNqbFknhKklrEcUAlMDAiZhW8BgPHA6uT\nLde7m6WX640iW673GvA0Bcv18nNHAp8AG5Iv10spzQd6AX3JEnv2B/qklGa3wLVKkiSpDXEGhiS1\nIimlIcCQZVRp8uV6KaVXybZplSRJkpqNMzAkSZIkSVLJM4AhSZIkSZJKngEMSZIkSZJU8gxgSJIk\nSZKkkmcAQ5IkSZIklTwDGJIkSZIkqeQZwJAkSZIkSSXPAIYkSZIkSSp5BjAkSZIkSVLJM4AhSZIk\nSZJKngEMSZIkSZJU8gxgSJIkSZKkkmcAQ5IkSZIklTwDGJIkSZIkqeQZwJAkSZIkSSXPAIYkSZIk\nSSp5BjAkSZIkSVLJM4AhSZIkSZJKngEMSZIkSZJU8gxgSJIkSZKkkmcAQ5IkSZIklTwDGJIkSZIk\nqeQZwJAkSZIkSSXPAIYkSZIkSSp5BjAkSZIkSVLJM4AhSZIkSZJKngEMSZIkSZJU8gxgSJIkSZKk\nkmcAQ5IkSZIklbyKYndAkiRJkrRq6zfm9mJ3oUG379+v2F1QE3EGhiRJkiRJKnkGMCRJkiRJUskz\ngCFJkiRJkkqeAQxJkiRJklTyDGBIkiRJkqSSZwBDkiRJkiSVPAMYkiRJkiSp5BnAkCRJkiRJJc8A\nhiRJkiRJKnkGMCRJkiRJUskzgCFJkiRJkkqeAQxJkiRJklTyDGBIkiRJkqSSZwBDkiRJkiSVvIqW\n/LCI6AEMA7YAJgMXp5Sui4gdgeeAuQXVh6SUhkREGTAE+FHe31uA01JKVXmbfYHBQDfgceC4lNLH\nLXVNkiRJkiSp+bVYACMiOgOjgZOBPwLbAY9GxJvAl4CxKaX96zn1JKA3sA1QDYwBTgcujohtgGuB\nvYFXgauAm4D9mvdqJEmSJElSS2rJJSSbAvenlO5IKS1OKb1ENmNiZ6A78EoD5x0FXJFS+jCl9BEw\nFPhhXtYPGJVSei6lNBc4C9g3Iro154VIkiRJkqSW1WIzMFJKr5AFI4DaGRm7kC0J6QXMi4i3gXLg\nTuDclNJ8suUmrxU2lZ0eZXnZMwWfMSUipgIBuIxEkiRJkqRWokVzYNSIiHWB+4AX8/8eBzwBXEeW\ny+Iu4ALgbGAtYE7B6XPIZo60r6espnzNZX1+585rUlFR/nkvQ2pzPix2B5ahsrJjsbsgSZIkqRm1\neAAjIr5ElsfiTeDwlNJioE9BlbciYghZ4s6zyQISaxSUrwksSinNi4i6ZTXls5bVh2nT6sY8JK3q\nJk2aWewuNAkDMZIkSVL9WnQb1YjYnmy3kYeAg1JKcyOic0RcGhGFT+0dgHn5z/8mWxJS20x+bKmy\niOgKrFdQLkmSJEmSWoGW3IWkG/AgMCyl9JuCounAd4GyiDibLNnnucD1efltwBkR8RiwEPgFcGte\n9gfgyYgYCbxAluBzbEppSnNfjyRJkrQs+fPv/wHHppTG5DngRgJ7kD0DX5BSujGvW0Y2A/lHZM/o\ntwCnpZSq8vK+wGCy5daPA8ellD7Oy7qTLcXeGpgA/CSl9GyLXagktZCWnIFxHFAJDIyIWTUv4CLg\nAGBbYDLwFFkOjOH5edcAo4DnyZJ5Pg1cBrWJQY8nuxF8AmwIHNNSFyRJkiQtw41Al4L3N5Atde4G\nHAJcHBHfystOAnoD2wBbAt8GTgeIiG2Aa4G+QFfgI+CmvKwDWU65m4BOwJXA6IhYuzkvTJKKoSV3\nIanJa9GQvRo4rwo4L3/VV34n2a4lkiRJUkmIiJ8As4GJ+fu1gYOAr6SU5gHPR8QdwNHAs2S79V2R\nUvowrz+U7Iu+i4F+wKiU0nN52VnApHyGx/bA4pTSiPyjR0bEAGA/fEaW1Mq0aA4MSZIkqbWLiK+Q\nzZ44seDw5sDClNJbBccSsEX+8xZks40LyyJfWrJEWb5ceipZLri659VtV5JajaJsoypJkiS1RhFR\nQZav7ZSU0tSI2nzzawFz61SfQ7aDXk35nDplqwHt6ykrPHdZZcvUufOaVFSUL6+aSkgpb2lfytzl\nrfUwgCFJkiQ1nYHAKymlsXWOzyHbaa/QmmQ5MWrK16hTtiilNC8i6pYVnrussmWaNq1u3ENqnSZN\nmlnsLmgFNRR0cgmJJEmS1HQOB46IiE8j4lNgE+CPZAk620XEJgV1g8+Wf/w7f19Y9u/6yiKiK7Be\nfrzueXXblaRWwxkYkiRJUhNJKS2ReyIi3gFOzrdR3Q4YGhHHk215eiRZsk2A24AzIuIxYCHwC7Kl\nKAB/AJ6MiJHAC8BQYGxKaUpev31E9CfbqeQosl1OHmq+q5Sk4nAGhiRJktQyjgdWB94H7gbOqNlZ\nBLgGGAU8TzZ74mngMoCU0iv5uSOBT4ANgWPysvlAL7ItVqcC/YE+KaXZLXNJktRynIEhSZIkNZOU\n0mYFP08FDmugXhVwXv6qr/xOGtgWNaX0KrDz5+2rJJU6Z2BIkiRJkqSSZwBDkiRJkiSVPAMYkiRJ\nkiSp5BnAkCRJkiRJJc8AhiRJkiRJKnkrHcCIiC9GRHlTdkaStCTHWkkqPsdiSSoNjdpGNSI2Aq4E\nBpPtS/0I8G3gg4jYP9+bWpL0OUya9Ak9eux4N461klQ0PvdKUulq7AyMq4H1gMnAD4CvAjsB95IN\n8JKkz+myy34DjrWSVGw+90pSiWpsAGMP4CcppfeAg4D7UkrPAZcBOzZX5ySpLXnxxRfAsVaSis3n\nXkkqUY0NYCwEKiJibeA7wAP58S8A05uhX5LU5lRUVIBjrSQVm8+9klSiGhvAeAQYCYwG5gNjImIf\n4CbgL83UN0lqU77+9W+AY60kFZvPvZJUohobwDgeeAaYARyYUpoFbAGMAgY0U98kqU0566zzwLFW\nkorN515JKlGN2oUE+C5wVkppfs2BlNLwiFgLOBG4ojk6J0ltyZNPPg6OtZJUbD73SlKJajCAERFf\nADrmb28CJkTElDrVtgWG4kAuSStl8uTJzJkzG4ChQy8E2N6xVpJaVuFYjM+9klSyljUDYyfgz0B1\n/v7pgrJqoCz/+fdN3y1Jahv+9a9XOe+8sygrqxlSm26sjYhvAPemlDbM3+8IPAfMLag2JKU0JCLK\ngCHAj8juDbcAp6WUqvJz+wKDgW7A48BxKaWP87LuwHXA1sAEsuz9z65ofyWpWJpzLJYkNZ0GAxgp\npXsiYjOyPBlvAd8AJhVUqQZmpZSmNmsPJakV2223Pbjrrvuorl7MYYcdCE0w1ubBiGPItvxbVFDU\nHRibUtq/ntNOAnoD2+SfOQY4Hbg4IrYBrgX2Bl4FriL7hnK/iOgA3EcW3PgdcBQwOiL+J183Lkkl\nr3AsPvTQPmX43CtJJWmZOTDy/a+h8ck+JUkraIMNNgBg3Li/U1nZ8YUmaPIc4DCyoMJZBce7A680\ncM5RwBUppQ8BImIocBFwMdAPGJVSei4vOwuYFBHdgO2BxSmlEXk7IyNiALAfcGcTXIsktYiasTil\n5HOvJJWoRiXxjIguwNnA14HV+WwaHQAppZ2bvmuS1LZMn/4pPXrseAmff6wdSbYcZLc6x7sD8yLi\nbaCcLMBwbp6obgvgtcKPAyKfzbEFWUb+mn5MiYipQNRzXs25WzSyr5JUUnzulaTS1dhdSG4mG8Rv\nI9tSSpLUxAYPHgRwNJ9zrC2YRVG3aBLwBFm+im7AXcAFZA/qawFzCurOIZt9176espryNZdTtkyd\nO69JRUX58qppJVVWdlx+JbUIfxelYQV+Dz73SlKJamwAYw9g95rpw5Kkpvfii38H6NNcY21KqU/B\n27ciYgjZTI2zyYIOaxSUrwksSinNi4i6ZTXls+o5r7BsmaZNqxv3UFOaNGlmsbugnL+L0rACvwef\neyWpRDV2jd8nwLzm7IgktXWdO68HzTTWRkTniLg0Igq/guxQ8Hn/JlsSUntKfmypsojoCqyXH697\nXs25dZeVSNKqwudeSSpRjZ2BMQi4KiJ+RrZF3oLCwpTSgvpOkiQ13rHHnsCQIRc011g7HfguUBYR\nZwObAucC1+fltwFnRMRjwELgF8CtedkfgCcjYiTwAjCUbDeTKXn99hHRn2ynkqPIlqc89Dn6KknF\nNAifeyWpJDV2BsZQ4FtkD67Tgbl1XpKkz+m6634LzTTWppQWAwcA2wKTgafIcmAMz6tcA4wCnieb\nPfE02TaspJReAY4nSw76CbAh2Tat5AlAewF9galAf7JlMLM/T38lqYh87pWkEtXYGRhHNGsvJEkM\nGjSE/v1/3LOp2kspPQF0LXj/GrBXA3WrgPPyV33ld9LAtqgppVcBs/JLai187pWkEtWoAEZK6cnm\n7ogktXXdu+/geCtJReY4LEmlq1EBjIh4HKhuqDyltEeT9UiS2qj+/X/Myy+/+FhD5Y61ktT8fO6V\npNLV2CUkz9Zz3v8APcnWCUqSPqett/4aL7/8YuG2fY61ktTyfO6VpBLV2CUkv6jveEQcD+zbpD2S\npDbqJz85mYEDf7HUeOtYK0ktx+deSSpdjd2FpCGP4EAuSc3NsVaSis+xWJKKrLE5MNrVc3gd4FTg\noybtkSS1UQsXLqxvvHWslaQW5HOvJJWuxubAmEf9yYzmAcc0XXckqe3aY4+dAebWU+RYK0ktx+de\nSSpRjQ1g7F7nfTWwAPhXSmlm03ZJktqmK6+8lv79f1yY3d6xVpJans+9klSiGpvE80mAiFgXCKAc\neMNBXJKaTvfuO5BSetKxVpKKx+deSSpdK5ID4zLghIJzqiLiT8CxKaUFzdQ/SWoz8hwYv8WxVpKK\nxudeSSpdjd2F5GJgP6APsC6wHnAQsDNwUfN0TZLalmuuuRIcayWp2HzulaQS1dgcGH2BvimlxwqO\n3R8R84FbgP9v797j7Cqrg4//QsItgBKaNJRWrDdWlDYaXqVUsRa0Vi7lRUUU0iAIqShE2mAEIoqI\nJBq5qYUQkaTcQovFGogCalGkWIjgmyISFyB4wSYQSKQJuZiEvH/sPeVwnJmczJyZvTPn9/18zifn\n7LX3PuvkZNZsFs9+njPanpkkdZjvfOc2gJOstZJUKa97JammWh2BsQPw391s/zVFZ1qS1E8bNmwA\na60kVc3rXkmqqVYbGHcC0yNi+64N5fPpwA8GIjFJ6jSvfe3rwForSVXzuleSaqrVW0imAv8B/Dwi\nFpfbXgc8B/z1QCQmSZ1mypSp3HXXnW/HWitJVfK6V5JqqtVlVB+JiFcDfwuMA9YBXweuy8w1A5if\nJHWMP/qjlwBYayWpQl73SlJ9tToCg8xcCXwJICJ+LzOf3to3i4gDgQspfhk8BczKzDkRMQqYCxwM\nPAOcm5lXlscMA2YAJ5X5Xg1MzcxNZfwY4HxgLPBd4MTMfGJrc5OkOmhHrZUk9Y+1WJLqqdc5MCLi\nryLixxHxp02heRGxJCL+otU3KpsUNwFfAEYB7wFmRsTbgCuA1RRNiKOAWRFxQHnoKcBhwHiK/zP5\nJuD08pzjgcspZoseDSwD5rWakyTVwQ9/eDfHHfdefvazR5pDW11rJUl901WL23HdK0kaGD02MMoi\n/Q3gv4AVTeFzgcXAtyJi/xbf66XANzJzfmY+l5k/ohgx8UaKtbXPycx1mbkImA8cVx43CbgkM5dm\n5jJgJnB8GZsILMjMezJzLcWyVu+IiLEt5iRJlVq8+EdMm/b3vPKV+/CiF72oOdyXWitJ2kqNtZj2\nXPdKkgZAbyMwPgl8ITP/NjN/3RjIzPsy8xjgGuDTrbxRZi7OzEldr8sRGW8GhgEbMvPRxt0pbjOh\n/PPBpliUt5a8IFYO71sBRCs5SVLV5s37Cu95zzF88pPnMWbM778g1pdaK0naeo21uB3XvZKkgdHb\nHBj7Aadu4fh/BP59a980Il4M3AzcRzEK47SmXdYAI8vnu5SvG2PbATt2E2s+tlujRo1kxIjhW5u2\n1PGWVp1AL8aM2a3qFPrkkUeS88771Jby71OtlSS15qGHfsrUqR/b0m7WYkmqWG8NjM0UoyN6s5Yt\nzKPRLCJeBiwEfga8l2Jei52adhtJMScGFA2JnZtiGzNzXUQ0x5qP7dbKlU4gLQ01y5evqjqFPtm8\nGVaseJYXvajIv4dGxlbXWklS64YNG8bmzZu3tJu1WJIq1lsD44fA4cCSXvZ5J/CTVt8sIvYDbgWu\nBT6amc9FxMPADhGxd2b+smtXnr81ZEn5+p6G2JKmWNf5RwN7bCFnbYM2zr6q6hR6NOJD7686BW3D\nxo17DT/4wZ388R+/rLfdtqrWSpK2jrVYkrYNvXWRLwA+FRGTugtGxPHAp4CLW3mjcmLNW4ELM3Nq\nZj4HkJmrgAUUK5KMjIg3AMcC15WHXgtMi4g/Ks9xFsU9iADXA++OiAMjYieKCT5vcakrSduKY475\nW+bO/TK33vqNbuNbW2slSVvPWixJ24YeR2Bk5nci4nRgTkTMpJiv4hmKJVBfD7wIODMzv9bie50I\njExz174AACAASURBVAE+ERGfaNj+BWAyxXKoj1Pc/jEtM7tGXFxGsbzqIop5L64FLipzXBwRk4G5\nwJ7AncAJLeYjSZV7wxv+jFNP/Xs+//kZzJlzKcuXP7mA/tVaSdJWaqzFn/nMOY/T/+teSdIA6O0W\nEjLz8oj4JsVypa+laCQ8BXwO+GrzLM1bONcMYEYvuxzdw3GbgLPLR3fxG4AbWs1DkurmyCOP4oAD\nDuTb376FOXMuXUs/aq0kqW+6avFRRx1+Kf287pUkDYxeGxgA5bwUMwchF0nqWHvuuSeTJp3A1Kkf\neV/VuUhSp9pzzz3JTK97JammnElZkiRJkiTVng0MSZIkSZJUezYwJEmSJElS7dnAkCRJkiRJtbfF\nSTwBImJ74GTg5sz8eUTMAN4L3At8KDNXDGCOktQRNm7cSERMwVorSZXxuleS6qvVERgXAGcBu0fE\nEcBHgS8DewJfGqDcJKmj/OM/XgLWWkmqmte9klRTrTYw3gsclZmLgaOBb2fm54BTgMMGKjlJ6iS3\n3/5tsNZKUtW87pWkmmq1gbEr8OuI2A44BPhmuX0T8NxAJCZJnWbt2jVgrZWkqnndK0k11dIcGMB9\nwNnA08CLgQUR8VLgc8APBig3SeooEa9m8eIfWWslqVpe90pSTbXawPgwcC3wMuDMzHw8Ii4G/gB4\nz0AlJ0mdZOrUMzjuuPe+HmutJFWp39e9EXE0cC7wEuAXwMcz8+sRMQqYCxwMPAOcm5lXlscMA2YA\nJ1Fco18NTM3MTWX8GOB8YCzwXeDEzHyijE0A5gD7Ag8DJ2fm3f39i5CkummpgZGZPwEmNG0+MzPX\ntz8lSepML3/5K8hMa60kVai/170RsQ8wD/irzPxBRLwN+EZE/CFwObCaogkxHrglIn5SNhu65tgY\nD2wGFgKnA7MiYnx57NuB+ykmE50HHBoROwE3UzQ3vgJMAm6KiJdn5uq+/j1IUh21OgKDiNgXOA3Y\nB5gIvDMiMjO/PVDJSVKnsdZKUvX6U4sz86GIGJuZqyNiBEWzYhXwW+BIYJ/MXAcsioj5wHHA3RSN\nh0syc2mZw0zgPGBWmcOCzLynjJ0BLI+IscB+wHOZObtMYW5E/ANwKHBDO/4+JKkuWmpgRMRbKTq7\n/wYcAOwI7AVcHBF/m5n/MnApSlJnuPfeRQA/xForSZVpx3Vv2bx4GcXtHNsBHwJeAWzIzEcbdwXe\nVT4fBzzYFIvy1pJxwH82nP/piFgBRDfHdR07bkt5jho1khEjhm9pN9XI0qoT2EaNGbNb1SmoTVod\ngTEDmJaZl5brYZOZ0yPiSeCTgBfVktRPc+ZcCtZaSapau657fwXsDLwZuIliJMXapn3WACPL57uU\nrxtj21E0UJpjjcf2FuvVypXNh0lD0/Llq6pOQVupp6ZTq8uo/glwSzfbbwJe3secJEkNHnvsZ2Ct\nlaSqteW6NzM3ZuaGzLwduBF4PbBT024jKebEgKLpsHNTbGN5u0lzrPHY3mKSNKS02sD4NcWEQs0O\nBn7ZvnQkqXONHv37YK2VpKr167o3Ig6NiO80bd4B+BmwQ0Ts3bg7z9/+saR83Rhb0l0sIkYDe5Tb\nm49rPq8kDRmt3kIyE/hyRLwCGA68IyL+mGK25NMGKDdJ6iiTJh3PzJmfttZKUrX6e937I+D1ETEJ\nuA54B8WEmn8G7A3MjIjJFEueHlvGoFi6dVpE3A5sAM4Crilj1wN3RMRc4N4yx1vKuTBuB3aMiCkU\nK5VMopg49La+/xVIUj21NAIjM+cBxwNHAM9SLNN0IDAxM78yYNlJUgc57LAjwForSZXq73VvZi4D\n/oai2fEb4NPAkZn5U2AysD3wOMVtJdO6VhYBLgMWAIsoRk/cBVxUnnNxeexc4EmKSUVPKGPrgUOA\nY4AVwBTgiMx8th9/DZJUSy0vo5qZ3wS+OYC5SFLHs9ZKUvX6W4sz806KOS+at68Aju7hmE3A2eWj\nu/gN9LAsambeD7yxr/lK0raixwZGRMxo9SSZOb096UhSZylXHvlf11wzr8faa62VpIHRWIt7q8Ng\nLZakKvU2AuPPWzzH5nYkIkmd6IEH7m/e1FPttdZK0gBpqsW9XQNbiyWpQj02MDLzoMFMRJI60Ze+\nNOcFr8eM2c3aK0mDrLEWW4clqb5angMjIsYDJwGvBjYBi4E5mfnYAOUmSR3HWitJ1bMWS1I9tbQK\nSUQcCdwHvBb4L4r1pt8M/CQi3jxw6UlS5/j+978H1lpJqpTXvZJUX62OwJgBfDIzZzZujIhzgIvp\nZpZlSdLW+fKXLwVrrSRVzeteSaqplkZgAC+lWKu62XzgNe1LR5I617JlS8FaK0lV87pXkmqq1QbG\nQuDD3Ww/mn6skS1Jet4b3/hmsNZKUtW87pWkmmr1FpKngJMj4q+Au4CNwARgf+CWiJjftWNmHtv2\nLCWpA7z4xbuDtVaSquZ1ryTVVKsNjJHA9eXz7cvHT8uHJKkN1q9fB9ZaSaqa172SVFMtNTAy84SB\nTkSSOt306edw8cUXWG8lqUJe90pSfbU6AoOIeAPFxEU7NoU2Z+YVbc1KkjqUtVaSqmctlqR6aqmB\nERGfB04HngTWNoU3AxZySeqnSy/9AsA9WGslqTJe90pSfbU6AuNEYHJmXjmQyUhSJ1u4cAFYayWp\nal73SlJNtbqM6rPADwYyEUnqdDvvvDNYayWpal73SlJNtToC45PAJRHx98BjwHONwcz8bbsTk6RO\nc+KJH2TmzE9bayWpWl73SlJNtdrA+G+Kta8f6CE+vD3pSFLnGj16DFhrJalqXvdKUk212sC4DPh3\n4EpgzcClI0md68ILPwvWWkmqmte9klRTrTYwxgLTMvOxgUxGkjrZypUrwForSVXzuleSaqrVSTy/\nCrxzIBORpE530EFvA2utJFXN615JqqlWR2CsBM6PiEnAI8CGxmBmHtvuxCSp0+y2225grZWkqnnd\nK0k11WoDYxTwzwOZiCR1ulWrVoG1VpKq5nWvJNVUSw2MzDxhoBORpE43ffo5XHzxBdZbSaqQ172S\nVF+tjsAgIg4A9uX5paOGATsC/ycz3z8AuUlSx7HWSlL1rMWSVE8tNTAi4jxgOsW62HsBv6aYoXkE\ncOOAZSdJHeSKK2YD3IW1VpIq43WvJNVXqyMwPgB8KDO/HBG/AP6SYoKjr1JMbiRJlZq48LqqU+jV\ndYdP3OI+3/jGTWCtlaSqed0rSTXV6jKqY4Bby+eLgQMy8zfAx4H3DURiktRpfvOblWCtlaSqed0r\nSTXVagNjKfCH5fOfAq8rnz9FMaROktRPv/d7o8FaK0lV87pXkmqq1VtI/hm4NiKOB24BvhoR9wOH\nURT2rRIR+wNfz8y9ytevB+4B1jbsNiMzZ0TEMGAGcFKZ79XA1MzcVB57DHA+xS+U7wInZuYTW5uT\nJFXtrW99O/PnX922WitJ6pO2XvdKktqn1QbGdOB/gFGZeVNEXAbMppjUqOWZmMtmxAnARcDGhtAE\n4JbMPLybw06h+IUxHtgMLAROB2ZFxHjgcuDtwP3Al4B5wKGt5iRJdfHBD57C/PlXz6WftbZLN83i\nUcBc4GDgGeDczLyyjPW5WRwRE4A5FDP2PwycnJl39+1vQZIq15brXklS+7XUwCgvYM9veH0OcE4f\n3m86cHR5rjMatk+guMewO5OASzJzKUBEzATOA2YBE4EFmXlPGTsDWB4RYx2FIWlbM3z4cDKz37W2\nl2bxFcBqiibEeOCWiPhJ2WzoU7M4InYCbqao61+hqNk3RcTLM3P11uYuSVVr43WvJKnNem1glBem\nhwK3Zeaz5bZTKS5il1M0Fn68Fe83l+L/8L2lafsEYF1EPEax3vYNwMczcz0wDniwYd8s0ohhZew/\n/zeQ+XRErAAC6LGBMWrUSEaMGN5TWDW0tOoEejFmzG5VpzBo6vw91F1v/07Wr1/PHXfcwYEHHvi/\n2/pZa3+nWRwRuwJHAvtk5jpgUUTMB44D7qaPzWJgP+C5zJxdvvfciPgHit8dN2xFzpJUqfXr13P3\n3Xfx8Y9/bJc2XfdKktqsxwZGROwN3AH8AfAnwCMR8XlgKnBTeex/RMRbM/PeVt6s4cK4ObQc+B7F\nEOSxFMtUnQucCewCrGnYdw3F5KM7dhPrio/sLY+VK5sPkfpu+fJVVaegbUBP/06WLVvGlCl/x9NP\nP8XVV/8LEya8hv7WWrpvFr8K2JCZjzZsS+Bd5fO+Noubj+s6dlyLuUpS5RprMcW1b7+veyVJ7dfb\nCIzzKC5Cx2fmqogYDZwG/FtmHgUQEWeW+x3SnyQy84iGl49GxAyKi+8zKRoSOzfERwIbM3NdRDTH\nuuIOW5a0TfjKV2bzkpe8lKuuup6RI3ehHbW2h2bxLrxwomR4YcO3r83iPjWSwdFwA62TRofVnd9F\nPfT2PVxwwWd4xStezsKFN7Prrrs+MpDXvZKkvuutgfHXwBGZ2fW/Dd9BcXvHVQ373Aqc1Z8Eyknl\nPk4xmVzXe+0ErCufL6H4v3z3dB1SbmuMdZ1rNLBHQ1ySam3Rorv57GcvZOTIXbo2DUitpWgq7NS0\nrbHh29dmcZ8byY6GG1iODqsPv4t66O17+P737+Szn72QtWs3s+uuwMDVYklSP2zXS2x3YFnD64Mo\nJoO7vWHb/1AU9/54Bngn8KmI2D4iXknR0JhXxq8FpkXEH5X3W58FXFPGrgfeHREHlvN1zKRYzeTp\nfuYkSYNi9epV7LHH6MZNA1VrHwZ2KG8P7BI8f/vHCxrCtN4sbj6u+bySVHuDWIslSf3Q2wiMX1Bc\nhP4yIoZTDJe7s2tSo9JfAo/1J4HMfC4i/gb4IvAUxRDnOcAXyl0uo5gXYxHFUOZrKWbWJzMXR8Rk\nivu99wTupJh5X5K2CWPH7smvfvVz9txzTzZt2gQDV2tXRcQCYGZZN/cFjuX5Zae7msW3Axv43Wbx\nHRExF7iXhmZxuf+OETGFYqWSSRQ1+7b+5Nto4+yrtrxTVV7S6mrkkuqssRYP5HWvJKl/ervyugr4\nYkScA7yNokEwpSsYEW+mmOH+0q1908z8HjC64fWD5Xt0t+8m4Ozy0V38BpzpXtI26pBDDueSSy7g\nAx/4IPfeuwjaWGu7MZmiyfA4xS0e07pWFqGPzeLMXB8Rh5TnnQE8QnH7YeNFvyTVWmMtBmYzsLVY\nktRHvTUwPkdxG8llwHPA9My8ESAivgicCnyt3E+S1AcTJ76f1atXcdFFn2XYsO2gjbW2m2bxCorl\nVbvbt8/N4sy8H3jj1uYnSXXRWIsplpz2uleSaqjHBkZ5Mfux8tHsCmBuZi4eqMQkqRMMHz6cD3/4\nND784dMAGDNmt882hK21kjQIGmvxmDG7/X5T2FosSTXRp5t3M/PH7U5EkvRC1lpJqp61WJLqo7dV\nSCRJkiRJkmrBBoYkSZIkSao9GxiSJEmSJKn2bGBIkiRJkqTas4EhSZIkSZJqzwaGJEmSJEmqPRsY\nkiRJkiSp9mxgSJIkSZKk2rOBIUmSJEmSas8GhiRJkiRJqj0bGJIkSZIkqfZGVJ1AnW2cfVXVKfRo\nxIfeX3UKkiRJkiQNGkdgSJIkSZKk2rOBIUmSJEmSas8GhiRJkiRJqj0bGJIkSZIkqfZsYEiSJEmS\npNqzgSFJkiRJkmrPZVQlSZKkNoqIA4ELgXHAU8CszJwTEaOAucDBwDPAuZl5ZXnMMGAGcBLFNfrV\nwNTM3FTGjwHOB8YC3wVOzMwnytgEYA6wL/AwcHJm3j1IH1eSBo0jMCRJkqQ2KZsUNwFfAEYB7wFm\nRsTbgCuA1RRNiKOAWRFxQHnoKcBhwHjg1cCbgNPLc44HLgeOAUYDy4B5ZWwn4Oby9e7AF4GbImLX\ngf6skjTYHIEhSZIktc9LgW9k5vzy9Y8i4rvAG4EjgX0ycx2wKCLmA8cBdwOTgEsycylARMwEzgNm\nAROBBZl5Txk7A1geEWOB/YDnMnN2+X5zI+IfgEOBGwb+40rS4HEEhiRJktQmmbk4Myd1vS5HZLwZ\nGAZsyMxHG3enuM2E8s8Hm2JR3lryglhmPg2sAKKb45rPK0lDhiMwJEmSpAEQES+muL3jPop5K05r\n2mUNMLJ8vkv5ujG2HbBjN7HGY3uL9WrUqJGMGDF8i59D9bG06gS2UWPG7FZ1CmoTGxiSJElSm0XE\ny4CFwM+A91LMa7FT024jKebEgKLpsHNTbGNmrouI5ljjsb3FerVyZXPfQxqali9fVXUK2ko9NZ28\nhUSSJElqo4jYD7gHuA04MjPXUqwOskNE7N24K8/f/rGkfN0YW9JdLCJGA3uU25uPaz6vJA0ZjsCQ\nJEmS2qScWPNW4MLM/FzX9sxcFRELKFYkmUyx5OmxFJNtAlwLTIuI24ENwFnANWXseuCOiJgL3AvM\nBG7JzKfL/XeMiCkUK5VMoljl5LYB/qiSNOgcgSFJkiS1z4nAGOATEbG64XE+MBnYHngcuBGY1rWy\nCHAZsABYRDF64i7gIigmBi2PnQs8CewFnFDG1gOHUCyxugKYAhyRmc8OwmeVpEHlCAxJkiSpTTJz\nBjCjl12O7uG4TcDZ5aO7+A30sCxqZt5PsUyrJA1pjsCQJEmSJEm1ZwNDkiRJkiTVng0MSZIkSZJU\nezYwJEmSJElS7dnAkCRJkiRJtecqJJIkSduIjbOvqjqF3r3ES0tJ0sBxBIYkSZIkSao9GxiSJEmS\nJKn2bGBIkiRJkqTas4EhSZIkSZJqzwaGJEmSJEmqPRsYkiRJkiSp9mxgSJIkSZKk2rOBIUmSJEmS\nas8GhiRJkiRJqj0bGJIkSZIkqfZGVPGmEbE/8PXM3Kt8PQqYCxwMPAOcm5lXlrFhwAzgpDLfq4Gp\nmbmpjB8DnA+MBb4LnJiZTwzuJ5IkSZIkSQNpUEdgRMSwiPgA8C1gh4bQFcBqiibEUcCsiDigjJ0C\nHAaMB14NvAk4vTzfeOBy4BhgNLAMmDfwn0SSJEmSJA2mwR6BMR04mmLExBkAEbErcCSwT2auAxZF\nxHzgOOBuYBJwSWYuLfefCZwHzAImAgsy854ydgawPCLGOgpDkiRJktRJJi68ruoUenXd4RP7dfxg\nNzDmUtwO8paGba8CNmTmow3bEnhX+Xwc8GBTLMpbS8YB//m/gcynI2IFEECPDYxRo0YyYsTwLSa7\ndIt7VGfMmN2qTmFQ+V3UQ52/h7rrpH8nkiRJ0kAY1AZGwyiKxs27AGubdl0DjGyIr2mKbQfs2E2s\n+dhurVzZfMi2Z/nyVVWnoJLfhVrR6r8TGx2SJElS9+qwCskaYKembSMp5sToiu/cFNtY3m7SHGs+\nVpIkSZIkDQF1aGA8DOwQEXs3bAuev21kSfm6Mbaku1hEjAb2aIhLkiRJkqQhoJJlVBtl5qqIWADM\njIjJwL7AscCh5S7XAtMi4nZgA3AWcE0Zux64IyLmAvcCM4FbMvPpwfwMkiQNdRtnX1V1Cj17SeWX\nM5IkaRDUYQQGwGRge+Bx4EZgWtfKIsBlwAJgEcWojLuAiwAyc3F57FzgSWAv4IRBzVySJEmSJA24\nSv6XRWZ+Dxjd8HoFxfKq3e27CTi7fHQXvwG4of1ZSpIkSZKkuqjLCAxJkiRJkqQeedOoJEmSJEkt\ncl6o6jgCQ5IkSZIk1Z4NDEmSJEmSVHs2MCRJkiRJUu3ZwJAkSZIkSbVnA0OSJEmSJNWeDQxJkiRJ\nklR7NjAkSZIkSVLt2cCQJEmSJEm1ZwNDkiRJkiTVng0MSZIkSZJUezYwJEmSJElS7dnAkCRJkiRJ\ntWcDQ5IkSZIk1Z4NDEmSJEmSVHs2MCRJkiRJUu3ZwJAkSZIkSbVnA0OSJEmSJNXeiKoTkCQNnoj4\nKDAD+G3D5kOAB4C5wMHAM8C5mXllecyw8piTKH5vXA1MzcxNZfwY4HxgLPBd4MTMfGJQPpAkSZI6\nhiMwJKmzTACmZ+auDY87gSuA1RRNiKOAWRFxQHnMKcBhwHjg1cCbgNMBImI8cDlwDDAaWAbMG8TP\nI0mSpA5hA0OSOssEYHHjhojYFTgSOCcz12XmImA+cFy5yyTgksxcmpnLgJnA8WVsIrAgM+/JzLXA\nGcA7ImLswH8USZIkdRIbGJLUISJiJBDAaRGxLCKWRMQHgFcBGzLz0YbdExhXPh8HPNgUi/LWkhfE\nMvNpYEX5PpIkSVLbOAeGJHWOscB/ALOBdwN/BtwMXAisbdp3DTCyfL5L+boxth2wYzex5mN7NGrU\nSEaMGL7FpJducQ91Z8yY3dp6Pr+Hvmvnd+H30Hft/pmQJA0+GxiS1CEy8zHgLQ2b7oyIa4C/AHZq\n2n0kxZwYUDQkdm6KbczMdRHRHGs+tkcrVzb3PdROy5evqjoFlfwu6qHV78FGhyTVl7eQSFKHiIj9\nIuLMps07Ab8EdoiIvRt35/lbQ5bwwltCotz2O7GIGA3s0RCXJEmS2sIRGJLUOVYD50TEI8DXgIOA\n91GMytgdmBkRk4F9gWOBQ8vjrgWmRcTtwAbgLOCaMnY9cEdEzAXupZjg85ZyLgxJkiSpbRyBIUkd\nIjMfAo4GPgmsAi4DTsjMHwGTge2Bx4EbgWmZeU956GXAAmARxaiMu4CLynMuLo+dCzwJ7AWcMEgf\nSZIkSR3EERiS1EEy82aKiTubt6+gaG50d8wm4Ozy0V38BuCGNqYpSZIk/Q5HYEiSJEmSpNqzgSFJ\nkiRJkmrPW0i2URMXXld1Cr267vCJVacgSZIkSRpCHIEhSZIkSZJqzwaGJEmSJEmqPW8hkSRJkgZA\nROwPfD0z9ypfj6JYdvpg4Bng3My8sowNA2YAJ1Fco18NTC1XgiIijgHOB8YC3wVOzMwnytgEYA6w\nL/AwcHJm3j1Yn1OSBosjMCRJkqQ2iohhEfEB4FvADg2hK4DVFE2Io4BZEXFAGTsFOAwYD7waeBNw\nenm+8cDlwDHAaGAZMK+M7USxPPY8YHfgi8BNEbHrAH5ESaqEDQxJkiSpvaYDp1GMmACgbCgcCZyT\nmesycxEwHziu3GUScElmLs3MZcBM4PgyNhFYkJn3ZOZa4AzgHRExFjgIeC4zZ2fmhsycCzwBHDrg\nn1KSBpkNDEmSJKm95gKvA37YsO1VwIbMfLRhWwLjyufjgAebYlHeWvKCWGY+DawAopvjms8rSUOG\nc2BIkiRJbZSZSwEionHzLsDapl3XACMb4muaYtsBO3YTazy2t1ivRo0ayYgRw7e0m2pkadUJbKPG\njNmtrefze+i7/n4XNjCkfpq48LqqU+jRdYdPrDoFSZJUWAPs1LRtJMWcGF3xnZtiGzNzXUQ0xxqP\n7S3Wq5Urm/se0tC0fPmqqlNQqdXvoqdGh7eQSJIkSQPvYWCHiNi7YVvw/O0fS8rXjbEl3cUiYjSw\nR7m9+bjm80rSkOEIDEmSJGmAZeaqiFgAzIyIyRRLnh7L85NtXgtMi4jbgQ3AWcA1Zex64I6ImAvc\nSzHB5y2Z+XS5/44RMYVipZJJFKuc3DZIH02SBo0jMCRJkqTBMRnYHngcuBGYlpn3lLHLgAXAIorR\nE3cBFwFk5uLy2LnAk8BewAllbD1wCMUSqyuAKcARmfns4HwkSRo8jsCQJEmSBkBmfg8Y3fB6BXB0\nD/tuAs4uH93FbwBu6CF2P/DGfqYrSbXnCAxJkiRJklR7NjAkSZIkSVLt2cCQJEmSJEm1V5s5MCLi\no8AM4LcNmw8BHqCYsOhg4Bng3My8sjxmWHnMSRSf5WpgankPoSRJkiRJGiJq08AAJgDTM/OCxo0R\n8a/AaorloMYDt0TETzLzbuAU4LBy+2ZgIXA6MGswE5ckSZIkSQOrTreQTAAWN26IiF2BI4FzMnNd\nZi4C5gPHlbtMAi7JzKWZuYxiTezjBy9lSZIkSZI0GGrRwIiIkUAAp0XEsohYEhEfAF4FbMjMRxt2\nT2Bc+XwcxTrZjbEoby2RJEmSJElDRF1uIRkL/AcwG3g38GfAzcCFwNqmfdcAI8vnu5SvG2PbATsC\n63p6s1GjRjJixPAtJrW0tdzVjTFjdmvr+fwu+sbvoT7a/V1IkiRJnaYWDYzMfAx4S8OmOyPiGuAv\ngJ2adh9JMScGFA2LnZtiGzOzx+YFwMqVa3oLqw2WL19VdQrC76FOWv0ubHRIkiRJ3avLLST7RcSZ\nTZt3An4J7BARezfuzvO3jSwpXzfGlgxYopIkSZIkqRK1GIFBMaLinIh4BPgacBDwPopRGbsDMyNi\nMrAvcCxwaHnctcC0iLgd2ACcBVwzyLlLkiRJkqQBVosRGJn5EHA08ElgFXAZcEJm/giYDGwPPA7c\nCEzLzHvKQy8DFgCLKEZl3AVcNLjZS5IkSZKkgVaXERhk5s0UE3c2b19B0dzo7phNwNnlQ5IkSZIk\nDVG1GIEhSZIkSZLUGxsYkiRJkiSp9mxgSJIkSZKk2rOBIUmSJEmSas8GhiRJkiRJqj0bGJIkSZIk\nqfZsYEiSJEmSpNqzgSFJkiRJkmrPBoYkSZIkSaq9EVUnIEmSJEl9MXHhdVWn0KvrDp9YdQrSkOII\nDEmSJEmSVHs2MCRJkiRJUu3ZwJAkSZIkSbVnA0OSJEmSJNWeDQxJkiRJklR7NjAkSZIkSVLt2cCQ\nJEmSJEm1ZwNDkiRJkiTVng0MSZIkSZJUezYwJEmSJElS7dnAkCRJkiRJtTei6gQkSZIk1dfG2VdV\nnULPXuJ/zkidxBEYkiRJkiSp9mxgSJIkSZKk2rOBIUmSJEmSas8GhiRJkiRJqj0bGJIkSZIkqfZs\nYEiSJEmSpNqzgSFJkiRJkmrPBoYkSZIkSao9GxiSJEmSJKn2bGBIkiRJkqTas4EhSZIkSZJqzwaG\nJEmSJEmqPRsYkiRJkiSp9mxgSJIkSZKk2rOBIUmSJEmSas8GhiRJkiRJqj0bGJIkSZIkqfZsYEiS\nJEmSpNqzgSFJkiRJkmrPBoYkSZIkSao9GxiSJEmSJKn2bGBIkiRJkqTas4EhSZIkSZJqzwaGhqJD\n0AAACU1JREFUJEmSJEmqPRsYkiRJkiSp9kZUnUB/RcQEYA6wL/AwcHJm3l1tVpLUWazFklQt67Ck\nTrBNj8CIiJ2Am4F5wO7AF4GbImLXShOTpA5iLZakalmHJXWKbbqBARwEPJeZszNzQ2bOBZ4ADq04\nL0nqJNZiSaqWdVhSR9jWGxjjgAebtmW5XZI0OKzFklQt67CkjjBs8+bNVefQZxFxNrBfZr6rYdvV\nwH9n5pnVZSZJncNaLEnVsg5L6hTb+giMNcDOTdtGAqsryEWSOpW1WJKqZR2W1BG29QbGEiCatgW/\nO4ROkjRwrMWSVC3rsKSOsK0vo3o7sGNETAEuByYBY4HbKs1KkjqLtViSqmUdltQRtukRGJm5HjgE\nOAZYAUwBjsjMZytNTJI6iLVYkqplHZbUKbbpSTwlSZIkSVJn2KZHYEiSJEmSpM5gA0OSJEmSJNXe\ntj6J55AQEZuBOZl5ctP2nwOnZubCiPgn4Fjgt92cYmzXPY4R8afAx4G3AC+iuA/yG8DHM/PpgfoM\nQ0X5XawFngM2l4//BE7PzAfKfYYBfwd8GHgFxdJl3wfOzcwfN5xrf2Am8AZgGPATYEZm3jRoH2gb\n1q6fi/I8f9r1/XV3nvZnr22Rtbg+rMX1YB3WYLMO14d1uD6sxS/kCIz6+LuIeMcW9vliZu7azaOr\nUB8A/IBiyazXALsCf0GxDvi3yiKjLdu//HvdDfg94MfALRExvIxfBvwDcHoZfyWwCLgrIv4cICJ2\nB74FXFvuMwqYBfxzWcTVmn7/XEhbyVpcH9bierAOa7BZh+vDOlwf1uKSIzDq4yvA3Ij4k8xc0cdz\nXEbxD/fTDdsei4gTgU8DuwMr+5lnR8nMDRExj6Iwj4qIVwAnAK/MzMfL3dYDsyJiF4rvYAKwD7Az\n8M+ZuaHc72sRcS7w4kH9ENu2dvxcSFvDWlxD1uJKWYc12KzDNWQdrpy1uGQDoz6+BLwcmA28d2sP\njoi9KYrEu5tjZbE4q78JdqKIGAV8BHggM5+KiNOAuxoKdaN/Aj4ZEX8M/BfwGHBPRMynGE53X2Z+\nbnAyHzL69XMh9YG1uIasxZWyDmuwWYdryDpcOWtxyQZGfWym6GL+OCKOzcz53exzakSc1LTt1sx8\nH7BX+frXXYGImAl8qHy5A/DBzLymzXkPRT+IiOfK5+uBe3j+l+BYYFkPxy0t/9wzM39eDl88FXgX\n8BlgXURcTXHv4NqBSX3I6e/PhbS1rMX1YS2uB+uwBpt1uD6sw/VhLS7ZwKiRzPxVREwBLo2I73ez\nyz9m5kd7OPzJ8s8/AH5Rnu8syi5zRNwLDO/+UDV5Y/PkNg2eoJgMqjsvLf9cBpCZv6Eo0p+JiF2B\ntwIXARuA09qX7tDWz58LKCYz6q7WjaD4ZSy9gLW4NqzFNWEd1mCzDteGdbhGrMUFJ/GsmbIbfDsw\nl2KW3laPexR4APjAAKWmwteB/SMiuom9H/hx2Wn+WETc0RXIzNWZuQC4GHjdIOU6ZPT156L0a57/\nRQpAeW/mWKC7YY+Stbj+rMWDzDqswWYdrj3rcAWsxY7AqKsPUhTesVt53GTgtojYRLHUzhPlvWcf\nAV7L8x1p9VFm3hcRXwZujogPAXdSLM11EjAF6JodeAFwTkScRTGJ0WqKWbBPBK4b9MSHhr7+XPwL\n8KmIWJKZD0XEGGAGcH9mLml3khpSrMU1ZS2ujHVYg806XFPW4Up1dC12BEYNZeZTFIW32UciYnU3\nj78sj7sbeD3wMuC+iHiWYgmpPYE/z8xvDtJHGNIy8yMUyz/NAp4CfkqxrvWBmXlXuU8CB1Es2fUo\n8D/AvwLXABdWkPY2r68/F8A5wELg1ohYTbH2+M7AoYOSuLZZ1uJ6sxYPPuuwBpt1uN6sw9Xo9Fo8\nbPPmzVXnIEmSJEmS1CtHYEiSJEmSpNqzgSFJkiRJkmrPBoYkSZIkSao9GxiSJEmSJKn2bGBIkiRJ\nkqTas4EhSZIkSZJqb0TVCUjtEBEjgI8BxwMvBX4D3AZ8IjN/0Ybz7wocnZlzy9ffA+7OzDP7e+7+\n5iJJdWEtlqRqWYc11DkCQ0PFDOD9wEeAAP4vMBa4IyJGtuH8pwN/1/D6XcBn2nDeduQiSXVhLZak\nalmHNaQ5AkNDxYnAKZl5a/n65xHxHuAp4FDgX/t5/mGNLzJzRT/P1x/DtryLJFXCWixJ1bIOa0gb\ntnnz5qpzkPotIpYDC4CTM3Njw/Z9gKWZuSoiTgCmA38I/AQ4KzO/U+73T8Aq4MUUneQ1wMWZOTMi\njgfmdZ0zM4c1DpeLiE8BrwZ+AZwMPAucSfGL4gsUXe+vAyd05dbOXPr/tydJ7WEtlqRqWYc11HkL\niYaKiyg6zr+KiLkRMSkixmTmQ2WhPhT4PEURHQ9cDSyMiD9tOMcHgUeB1wIXAjPK+L+Ur38E/EEP\n738ksD0wgaKzPRv4BPBeYCJwNPBugEHIRZKqYi2WpGpZhzWk2cDQkJCZMykK4hJgEkUB/O+IuCAi\nhlEUxs9l5o2Z+Uhmfgn4GnBaw2keysxPZebPMvNzwArgDZm5FlgNbMjMZT2ksBr4aGb+jKJQ7wx8\nJjPvy8ybgMXAa8p9BzoXSaqEtViSqmUd1lDnHBgaMjLzq8BXI2I34GCKCYxOB35JUSj3j4hzGg7Z\nAbin4fVDTadcRdFBbsXPM3NT+XxN+eejDfG1wI7l84HORZIqYy2WpGpZhzWU2cDQNi8ixgOTM3MK\nQGauorj3b0FEfB14O8W/9WnALU2Hr294/ttuTt/q/XQbutn2XA/7DnQukjTorMWSVC3rsDqBDQwN\nBcOBUyPiXzPzjqbY/wCbKIbRvTQzH+kKRMQMYBnwxRbeo52z3dYpF0lqF2uxJFXLOqwhzwaGtnmZ\n+f8i4t+AGyJiOvA94EXAXwPvBN4MvAyYHxEJ3A78DXAGcFiLb7Ma2DMiXp6Zj25x797NqlEuktQW\n1mJJqpZ1WJ3ASTw1VBwDXApMBX4MfB84CHhbZi7OzH+jmBDoY8CDFDMaT8rn18jekhsphsQ9EBF7\n9ifROuUiSW1mLZakalmHNaQN27zZkTeSJEmSJKneHIEhSZIkSZJqzwaGJEmSJEmqPRsYkiRJkiSp\n9mxgSJIkSZKk2rOBIUmSJEmSas8GhiRJkiRJqj0bGJIkSZIkqfZsYEiSJEmSpNr7/7cOxBlgD7uk\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x116b90550>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "facebook_data_raw['Length'] =   fb_s_counts.apply(lambda c: 'Multiple-sentence' if c > 1 else 'Single-sentence')\n",
    "csfd_data_raw['Length']     = csfd_s_counts.apply(lambda c: 'Multiple-sentence' if c > 1 else 'Single-sentence')\n",
    "mall_data_raw['Length']     = mall_s_counts.apply(lambda c: 'Multiple-sentence' if c > 1 else 'Single-sentence')\n",
    "\n",
    "fig = plt.figure(figsize=(18,8), dpi=72)\n",
    "subplots = []\n",
    "\n",
    "datasets_to_plot = [\n",
    "    (facebook_data_raw, 'Facebook'),\n",
    "    (csfd_data_raw, 'ČSFD'),\n",
    "    (mall_data_raw, 'MALL.CZ'),\n",
    "]\n",
    "\n",
    "for i, dataset in enumerate(datasets_to_plot):\n",
    "    subplots.append(fig.add_subplot(1, 3, i+1))\n",
    "    sns.countplot(x='Sentiment', hue='Length', data=dataset[0], \n",
    "                hue_order=['Single-sentence', 'Multiple-sentence'],\n",
    "                order=['NEG', 'POS', 'NEU'], palette='husl',\n",
    "                ax=subplots[i])\n",
    "    subplots[i].set(title=dataset[1], xlabel='Sentiment', ylabel='Samples Counts')\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⏱ Combine the three datasets to our final training data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Close your eyes, this is messy and I was lazy to refactor it...\n",
    "\n",
    "def create_joint_dataset(datasets):\n",
    "    '''\n",
    "    Takes multiple datasets (from smallest to largest), computes which class\n",
    "    has the least samples and creates a joint dataset so that all classes\n",
    "    are represented equally (strips off excessive samples).\n",
    "    \n",
    "    Returns a new joint DataFrame.\n",
    "    '''\n",
    "    \n",
    "    data_size_limit = min(\n",
    "        [ sum(x) for x in zip( *[data_stats(dataset) for dataset in datasets] ) ]\n",
    "    )\n",
    "\n",
    "    aggregate_size = [0, 0, 0]\n",
    "    results = [None, None, None]\n",
    "    \n",
    "    for i,dataset in enumerate(datasets):\n",
    "        neg, neu, pos = data_stats(dataset)\n",
    "        new_would = [\n",
    "            aggregate_size[0]+neg,\n",
    "            aggregate_size[1]+neu,\n",
    "            aggregate_size[2]+pos\n",
    "        ]\n",
    "        \n",
    "        if max(new_would) > data_size_limit:\n",
    "            results[i] = (abs(data_size_limit-new_would[0]),\n",
    "                          abs(data_size_limit-new_would[1]),\n",
    "                          abs(data_size_limit-new_would[2]))\n",
    "        else:\n",
    "            results[i] = (0, 0, 0)\n",
    "        \n",
    "        aggregate_size = new_would    \n",
    "    \n",
    "    result_datas = [None, None, None]\n",
    "    for i,res in enumerate(results):\n",
    "        result_datas[i] = datasets[i]\\\n",
    "            .drop(datasets[i].query('Sentiment == \"POS\"').iloc[:res[2]].index.values)\\\n",
    "            .drop(datasets[i].query('Sentiment == \"NEU\"').iloc[:res[1]].index.values)\\\n",
    "            .drop(datasets[i].query('Sentiment == \"NEG\"').iloc[:res[0]].index.values)\n",
    "\n",
    "    \n",
    "    return pd\\\n",
    "        .concat([\n",
    "            result_datas[0],\n",
    "            result_datas[1],\n",
    "            result_datas[2]\n",
    "        ], axis=0)\\\n",
    "        .reset_index(drop=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "FINAL ONE SENTENCE: Combined set\n",
      "13442 positive, 13442 neutral and 13442 negative posts.\n",
      "Posts are on average  64 characters long.\n",
      "=================================================================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# ⏱ Combine the one-sentence datasets into one joint set\n",
    "combined_data_onesentence = create_joint_dataset([facebook_data_onesentence, csfd_data_onesentence, mall_data_onesentence])\n",
    "data_stats(combined_data_onesentence, 'FINAL ONE SENTENCE: Combined set', True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Part 2: Preprocess our datasets\n",
    "\n",
    "### Define the sentence processor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def create_processor(aggressive = False):\n",
    "    '''\n",
    "    Creates a processor instance with set stemmer aggressivity.\n",
    "    Default stemmer aggressivity is False, which means 'light' stemming.\n",
    "    \n",
    "    Returns a sentence processor function.\n",
    "    '''\n",
    "    \n",
    "    punctuation_mapping = str.maketrans(\n",
    "        dict.fromkeys(string.punctuation+'0123456789²½¾ˇ', ' ')\n",
    "    )\n",
    "    \n",
    "    happy_emojis_regex  = \"|\".join( map(re.escape, HAPPY) )\n",
    "    sad_emojis_regex    = \"|\".join( map(re.escape, SAD) )\n",
    "\n",
    "    def sent_processor(sentence):\n",
    "        '''\n",
    "        Takes a sentence, replaces emojis with keywords, strips off punctuation,\n",
    "        numbers and excessive whitespace and converts the sentence to lower case.\n",
    "        The sentence is also capped at `max_sentence_length` words.\n",
    "        \n",
    "        Returns a list of processed words.\n",
    "        '''\n",
    "        def split_sentence(sentence):\n",
    "            split_sentence = re.sub('\\s+', ' ', sentence).split(' ')\n",
    "            return [] if len(sentence) == 0 else split_sentence\n",
    "\n",
    "        def remove_punc(sentence):\n",
    "            return \\\n",
    "                re.sub(sad_emojis_regex, ' sadmoji ',\n",
    "                    re.sub(happy_emojis_regex, ' happymoji ', sentence)\n",
    "                )\\\n",
    "                .translate(punctuation_mapping)\\\n",
    "                .lower()\\\n",
    "                .strip()\n",
    "\n",
    "        def stem(sentence, aggressive):\n",
    "            stemmed = [\n",
    "                cz_stem(word, aggressive) \n",
    "                for word in split_sentence(sentence)\n",
    "            ][:max_sentence_length]\n",
    "        \n",
    "            return ['prázdno'] if len(stemmed) < 1 else stemmed\n",
    "\n",
    "        return stem(remove_punc(sentence), aggressive)\n",
    "    \n",
    "    return sent_processor"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create a processor instance\n",
    "sent_processor = create_processor()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⏱ Create a joint dataset for word vectors learning\n",
    "\n",
    "Combine all of our three *raw* corpora, run the posts through the sentence processor and save the processed text in a `.txt` file which will serve for the word vectors training."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%capture\n",
    "\n",
    "combined_data_processed = pd\\\n",
    "    .concat([\n",
    "        facebook_data_raw,\n",
    "        csfd_data_raw,\n",
    "        mall_data_raw\n",
    "    ], axis=0)\\\n",
    "    ['Post']\\\n",
    "    .apply(lambda post: ' '.join( sent_processor(post) ))\n",
    "\n",
    "combined_data_processed.to_csv('data/combined_corpora_processed.txt', \n",
    "                               encoding='utf-8', index=False)\n",
    "\n",
    "# Delete the combined dataframe to free up the memory, we won't need it in our code anymore\n",
    "del combined_data_processed"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "### Train word vectors\n",
    "\n",
    "Now you can go and run **`train_word_vectors.sh`** file from your console. \n",
    "\n",
    "After you're finished with training and **`combined_corpora_processed.bin`** model is created in `word_models` directory, you can continue to the next cell.\n",
    "\n",
    "#### Getting fastText library\n",
    "\n",
    "You will need a working **`fasttext`** binary in order to train or use word vectors. A binary for macOS is already provided in `lib` directory. If you're on Mac, this should work out of box and you can go run the `train_word_vectors.sh` Shell script.\n",
    "\n",
    "If it doesn't work for you or you're not on Mac, you can obtain your binary via *quickly* building the fastText source. It's literally 3 commands.\n",
    "\n",
    "[More info on getting the `fasttext` binary here](https://fasttext.cc/docs/en/support.html)\n",
    "\n",
    "Once you have your binary, put it inside the project's **`lib`** directory. The following Python script will call a subprocess with the path explicitly defined as `lib/fasttext`.\n",
    "\n",
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the word represenation model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def load_word_model(model = 'word_models/combined_corpora_processed.bin'):\n",
    "    '''\n",
    "    Loads a fastText word representation model.\n",
    "    NOTE: Load a word model no more than once per kernel!\n",
    "    \n",
    "    Returns a function for getting word vectors.\n",
    "    '''\n",
    "    \n",
    "    word_model = subprocess.Popen(\n",
    "        ['lib/fasttext', 'print-word-vectors', model], \n",
    "        stdout=subprocess.PIPE, stdin=subprocess.PIPE)\n",
    "    \n",
    "    def get_word_vector(word):\n",
    "        '''\n",
    "        Takes a single processed word.\n",
    "        \n",
    "        Returns a list of word vector representation.\n",
    "        '''\n",
    "        \n",
    "        word_model.stdin.write( bytes('{}\\n'.format(word), 'utf-8') )\n",
    "        word_model.stdin.flush()\n",
    "        return [\n",
    "            float(value) for value in \\\n",
    "            word_model.stdout.readline()\\\n",
    "                .decode('utf-8')\\\n",
    "                .replace('{} '.format(word), '')\\\n",
    "                .replace(' \\n', '')\\\n",
    "                .split(' ')  \n",
    "        ]\n",
    "    \n",
    "    return get_word_vector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Load the word embedding model\n",
    "word_vector = load_word_model()\n",
    "word_vector_dim = 100"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Define a function for obtaining sentence tensors from text sentence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def sentence_tensor(sentence):\n",
    "    '''\n",
    "    Takes a sentence, runs it through the sentence processor\n",
    "    and obtains word vectors for all of the processed words.\n",
    "    Pads the array so it matches our input size.\n",
    "    \n",
    "    Returns a 2D numpy array containing word vectors.\n",
    "    '''\n",
    "\n",
    "    sentence_array = [word_vector(word) for word in sent_processor(sentence)]\n",
    "    \n",
    "    return np.pad(sentence_array, \n",
    "                  pad_width=( \n",
    "                      (0, max_sentence_length - len(sentence_array)), \n",
    "                      (0, 0) \n",
    "                  ), \n",
    "                  mode='constant')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### ⏱ Process the text data and save our final  sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "'facebook_y' labels successfully saved!\n",
      "'facebook_x' sentences successfully saved!\n",
      "\n",
      "'csfd_y' labels successfully saved!\n",
      "'csfd_x' sentences successfully saved!\n",
      "\n",
      "'mall_y' labels successfully saved!\n",
      "'mall_x' sentences successfully saved!\n",
      "\n",
      "'combined_y' labels successfully saved!\n",
      "'combined_x' sentences successfully saved!\n",
      "\n"
     ]
    }
   ],
   "source": [
    "def dataset_process_save(dataframe, name):\n",
    "    '''\n",
    "    Takes a DataFrame, obtains sentence tensors for each\n",
    "    entry and saves the 3D numpy array as .npy\n",
    "    \n",
    "    The labels are one-hot-encoded and also saved as .npy\n",
    "    '''\n",
    "    \n",
    "    y_labels = pd.get_dummies(dataframe, columns=['Sentiment']).drop('Post', axis=1).as_matrix()\n",
    "    np.save('data/{}_y.npy'.format(name), y_labels)\n",
    "    print('\\'{}_y\\' labels successfully saved!'.format(name))\n",
    "    \n",
    "    x_data = np.stack([sentence_tensor(post) for post in dataframe['Post']])\n",
    "    np.save('data/{}_x.npy'.format(name), x_data)\n",
    "    print('\\'{}_x\\' sentences successfully saved!\\n'.format(name))\n",
    "    \n",
    "    del x_data, y_labels\n",
    "    return None\n",
    "\n",
    "dataset_process_save(facebook_data_onesentence, 'facebook')\n",
    "dataset_process_save(csfd_data_onesentence, 'csfd')\n",
    "dataset_process_save(mall_data_onesentence, 'mall')\n",
    "dataset_process_save(combined_data_onesentence, 'combined')\n",
    "\n",
    "# Delete the processed DataFrames to free up the memory, we won't need them in our code anymore\n",
    "del facebook_data_onesentence, csfd_data_onesentence, mall_data_onesentence, combined_data_onesentence"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### Define a loader for our final processed datasets from `.npy` files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def load_xy(dataset = 'small_data'):\n",
    "    '''\n",
    "    Loads X and Y for specified dataset. Shuffles and splits into \n",
    "    training and testing sets.\n",
    "    \n",
    "    Returns X_train, X_test, y_train, y_test \n",
    "    '''\n",
    "    X = np.load('data/{}_x.npy'.format(dataset))\n",
    "    y = np.load('data/{}_y.npy'.format(dataset))\n",
    "    \n",
    "    return train_test_split(X, y, test_size=0.1, random_state=29, shuffle=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Part 3: Define our neural network and benchmark models\n",
    "\n",
    "<p>&nbsp;</p>\n",
    "<p>&nbsp;</p>\n",
    "\n",
    "![Convolutional Neural Network Architecture Used for the Project](images/cnn.png)\n",
    "\n",
    "<p>&nbsp;</p>\n",
    "\n",
    "---\n",
    "\n",
    "### Define our `CNN` class"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.callbacks import ModelCheckpoint\n",
    "from keras.regularizers import l2\n",
    "from keras.layers import Conv1D, MaxPooling1D, GlobalAveragePooling1D, GlobalMaxPooling1D, Dropout, Dense\n",
    "\n",
    "class CNN():\n",
    "    def __init__(self, name = 'unnamed', optimizer = 'adagrad'):\n",
    "        '''\n",
    "        Creates and compiles a new model. Prints out its summary.\n",
    "        '''\n",
    "        \n",
    "        self.name = name\n",
    "        self.model_weights = 'models/weights.best.{}.hdf5'.format(self.name)\n",
    "        \n",
    "        self.model = Sequential()\n",
    "        self.model.add(\n",
    "            Conv1D(filters = 64, kernel_size = 4, \n",
    "                   activation = 'relu', padding = 'same',\n",
    "                   input_shape = (max_sentence_length, word_vector_dim))\n",
    "        )\n",
    "        self.model.add(MaxPooling1D(pool_size = 2))\n",
    "        \n",
    "        self.model.add(\n",
    "            Conv1D(filters = 128, kernel_size = 8, \n",
    "                   activation = 'relu', padding = 'same',)\n",
    "        )\n",
    "        self.model.add(MaxPooling1D(pool_size = 2))\n",
    "        \n",
    "        self.model.add(GlobalMaxPooling1D())\n",
    "        self.model.add(Dense(units = len(target_labels), activation = 'softmax'))\n",
    "        \n",
    "        self.model.compile(optimizer = optimizer, \n",
    "                           loss = 'categorical_crossentropy', \n",
    "                           metrics = ['categorical_accuracy'])\n",
    "\n",
    "        self.model.summary()\n",
    "        \n",
    "        return None\n",
    "    \n",
    "    def fit(self, X_train, y_train, epochs = 20, batch_size = 1000, validation_split = 0.15):\n",
    "        '''\n",
    "        Trains and cross-validates the model on the provided\n",
    "        training set `X_train` and `y_train`. You can also setup\n",
    "        `epochs`, `batch_size` and `validation_split`.\n",
    "        \n",
    "        Returns self.\n",
    "        '''\n",
    "        \n",
    "        print('\\nFitting \\'{}\\' model...\\n'.format(self.name) + '_'*65)\n",
    "        self.checkpointer = ModelCheckpoint(filepath = self.model_weights, \n",
    "                                            verbose = 1, save_best_only = True)\n",
    "        \n",
    "        self.model.fit(X_train, y_train, \n",
    "                       epochs = epochs, \n",
    "                       batch_size = batch_size, \n",
    "                       validation_split = validation_split, \n",
    "                       callbacks = [self.checkpointer], \n",
    "                       verbose = 1)\n",
    "        \n",
    "        self.load()\n",
    "        \n",
    "        return self\n",
    "    \n",
    "    def load(self):\n",
    "        '''\n",
    "        Loads the saved weights for the model.\n",
    "        \n",
    "        Returns self.\n",
    "        '''\n",
    "        \n",
    "        self.model.load_weights(self.model_weights)\n",
    "        print('_'*65 + '\\nLoaded the weights for \\'{}\\' model.\\n'.format(self.name))\n",
    "        \n",
    "        return self\n",
    "    \n",
    "    def predict(self, sentence, proba = False):\n",
    "        '''\n",
    "        Takes a `sentence` and predicts its label.\n",
    "        \n",
    "        Returns the label index.\n",
    "        '''\n",
    "        \n",
    "        sentence_array = np.expand_dims(sentence_tensor(sentence), axis=0) \\\n",
    "                         if type(sentence) is str else \\\n",
    "                         np.expand_dims(sentence, axis=0)\n",
    "    \n",
    "        prediction = self.model.predict(sentence_array)\n",
    "        return np.argmax(prediction) if proba is False else prediction\n",
    "    \n",
    "    def predict_proba(self, sentence):\n",
    "        '''\n",
    "        Takes a `sentence` and predicts its label.\n",
    "        \n",
    "        Returns a vector containing probabilities of belonging to each of the labels.\n",
    "        '''\n",
    "        \n",
    "        return self.predict(sentence, True)\n",
    "    \n",
    "    def score(self, X_test, y_test):\n",
    "        '''\n",
    "        Computes the accuracy and weighted F1 score for the provided\n",
    "        testing set `X_test` and `y_test`.\n",
    "        \n",
    "        Returns tuple (accuracy, F1, individual F1s).\n",
    "        '''\n",
    "        \n",
    "        predictions = [self.predict(sentence) for sentence in X_test]\n",
    "        true = np.argmax(y_test, axis=1)\n",
    "        \n",
    "        return (\n",
    "            np.sum(np.array(predictions) == true) / len(predictions),\n",
    "            f1_score(true, predictions, average = 'weighted'),\n",
    "            f1_score(true, predictions, average = None)\n",
    "        )\n",
    "\n",
    "    \n",
    "    def test(self, X_test, y_test):\n",
    "        '''\n",
    "        Obtains and prints out accuracy and F1 scores for the provided\n",
    "        testing set `X_test` and `y_test`.\n",
    "        \n",
    "        Returns self.\n",
    "        '''\n",
    "        \n",
    "        accuracy_score_, f1_score_, f1_scores_ = self.score(X_test, y_test)\n",
    "\n",
    "        print('_'*65)\n",
    "        print('SCORES FOR \\'{}\\' MODEL'.format(self.name))\n",
    "        print('Test Accuracy: {0:2.4f}%'.format(accuracy_score_*100))\n",
    "        print('F1 Score: {0:2.4f}'.format(f1_score_))\n",
    "        print('F1 Scores: {}'.format(f1_scores_))\n",
    "        print('='*65 + '\\n')\n",
    "        \n",
    "        return self\n",
    "    \n",
    "    def benchmark(self, X_test, y_test):\n",
    "        '''\n",
    "        Obtains and prints out accuracy and F1 scores for the provided\n",
    "        testing set `X_test` and `y_test`.\n",
    "        \n",
    "        Returns self.\n",
    "        '''\n",
    "        \n",
    "        true = np.argmax(y_test, axis=1)\n",
    "        choices = list( set(true) )\n",
    "\n",
    "        weights = [\n",
    "            (true == label).sum() / true.sum()\n",
    "            for label in choices\n",
    "        ]\n",
    "        \n",
    "        def bench_predict(sentence):\n",
    "            '''\n",
    "            Takes a `sentence` and predicts weighted random label.\n",
    "\n",
    "            Returns the label index.\n",
    "            '''\n",
    "\n",
    "            return random.choices(choices, weights)[0]\n",
    "        \n",
    "        predictions = [bench_predict(sentence) for sentence in X_test]\n",
    "\n",
    "        print('_'*65)\n",
    "        print('SCORES FOR \\'{}\\' BENCHMARK MODEL'.format(self.name))\n",
    "        print('Test Accuracy: {0:2.4f}%'.format(np.sum(np.array(predictions) == true) / len(predictions)))\n",
    "        print('F1 Score: {0:2.4f}'.format(f1_score(true, predictions, average = 'weighted')))\n",
    "        print('F1 Scores: {}'.format(f1_score(true, predictions, average = None)))\n",
    "        print('='*65 + '\\n')\n",
    "\n",
    "        return self"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Part 4: Create the individual models and train\n",
    "### Facebook Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_9 (Conv1D)            (None, 30, 64)            25664     \n",
      "_________________________________________________________________\n",
      "max_pooling1d_9 (MaxPooling1 (None, 15, 64)            0         \n",
      "_________________________________________________________________\n",
      "conv1d_10 (Conv1D)           (None, 15, 128)           65664     \n",
      "_________________________________________________________________\n",
      "max_pooling1d_10 (MaxPooling (None, 7, 128)            0         \n",
      "_________________________________________________________________\n",
      "global_max_pooling1d_5 (Glob (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_5 (Dense)              (None, 3)                 387       \n",
      "=================================================================\n",
      "Total params: 91,715\n",
      "Trainable params: 91,715\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Load the processed data set\n",
    "facebook_X_train, facebook_X_test, \\\n",
    "    facebook_y_train, facebook_y_test = load_xy('facebook')\n",
    "\n",
    "# Initiate the model\n",
    "facebook_model = CNN('facebook')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Fitting 'facebook' model...\n",
      "_________________________________________________________________\n",
      "Train on 5254 samples, validate on 928 samples\n",
      "Epoch 1/5\n",
      "5220/5254 [============================>.] - ETA: 0s - loss: 0.7170 - categorical_accuracy: 0.6872Epoch 00000: val_loss improved from inf to 0.65148, saving model to models/weights.best.facebook.hdf5\n",
      "5254/5254 [==============================] - 7s - loss: 0.7154 - categorical_accuracy: 0.6880 - val_loss: 0.6515 - val_categorical_accuracy: 0.7091\n",
      "Epoch 2/5\n",
      "5240/5254 [============================>.] - ETA: 0s - loss: 0.5393 - categorical_accuracy: 0.7756Epoch 00001: val_loss improved from 0.65148 to 0.61018, saving model to models/weights.best.facebook.hdf5\n",
      "5254/5254 [==============================] - 6s - loss: 0.5392 - categorical_accuracy: 0.7758 - val_loss: 0.6102 - val_categorical_accuracy: 0.7435\n",
      "Epoch 3/5\n",
      "5240/5254 [============================>.] - ETA: 0s - loss: 0.4555 - categorical_accuracy: 0.8216Epoch 00002: val_loss did not improve\n",
      "5254/5254 [==============================] - 7s - loss: 0.4559 - categorical_accuracy: 0.8211 - val_loss: 0.6266 - val_categorical_accuracy: 0.7414\n",
      "Epoch 4/5\n",
      "5220/5254 [============================>.] - ETA: 0s - loss: 0.3832 - categorical_accuracy: 0.8529Epoch 00003: val_loss did not improve\n",
      "5254/5254 [==============================] - 6s - loss: 0.3837 - categorical_accuracy: 0.8525 - val_loss: 0.6403 - val_categorical_accuracy: 0.7392\n",
      "Epoch 5/5\n",
      "5240/5254 [============================>.] - ETA: 0s - loss: 0.3218 - categorical_accuracy: 0.8805Epoch 00004: val_loss did not improve\n",
      "5254/5254 [==============================] - 6s - loss: 0.3218 - categorical_accuracy: 0.8805 - val_loss: 0.6405 - val_categorical_accuracy: 0.7500\n",
      "_________________________________________________________________\n",
      "Loaded the weights for 'facebook' model.\n",
      "\n",
      "_________________________________________________________________\n",
      "SCORES FOR 'facebook' MODEL\n",
      "Test Accuracy: 71.6157%\n",
      "F1 Score: 0.7083\n",
      "F1 Scores: [ 0.54700855  0.76190476  0.72727273]\n",
      "=================================================================\n",
      "\n",
      "_________________________________________________________________\n",
      "SCORES FOR 'facebook' BENCHMARK MODEL\n",
      "Test Accuracy: 0.3843%\n",
      "F1 Score: 0.3865\n",
      "F1 Scores: [ 0.23728814  0.51209104  0.2606383 ]\n",
      "=================================================================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "facebook_model.fit(facebook_X_train, facebook_y_train, epochs=5, batch_size=20)\\\n",
    "              .test(facebook_X_test, facebook_y_test)\\\n",
    "              .benchmark(facebook_X_test, facebook_y_test);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "### ČSFD Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_11 (Conv1D)           (None, 30, 64)            25664     \n",
      "_________________________________________________________________\n",
      "max_pooling1d_11 (MaxPooling (None, 15, 64)            0         \n",
      "_________________________________________________________________\n",
      "conv1d_12 (Conv1D)           (None, 15, 128)           65664     \n",
      "_________________________________________________________________\n",
      "max_pooling1d_12 (MaxPooling (None, 7, 128)            0         \n",
      "_________________________________________________________________\n",
      "global_max_pooling1d_6 (Glob (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_6 (Dense)              (None, 3)                 387       \n",
      "=================================================================\n",
      "Total params: 91,715\n",
      "Trainable params: 91,715\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Load the processed data set\n",
    "csfd_X_train, csfd_X_test, \\\n",
    "    csfd_y_train, csfd_y_test = load_xy('csfd')\n",
    "\n",
    "# Initiate the model\n",
    "csfd_model = CNN('csfd')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Fitting 'csfd' model...\n",
      "_________________________________________________________________\n",
      "Train on 14948 samples, validate on 2638 samples\n",
      "Epoch 1/5\n",
      "14920/14948 [============================>.] - ETA: 0s - loss: 0.7076 - categorical_accuracy: 0.6843Epoch 00000: val_loss improved from inf to 0.65333, saving model to models/weights.best.csfd.hdf5\n",
      "14948/14948 [==============================] - 21s - loss: 0.7071 - categorical_accuracy: 0.6846 - val_loss: 0.6533 - val_categorical_accuracy: 0.7187\n",
      "Epoch 2/5\n",
      "14940/14948 [============================>.] - ETA: 0s - loss: 0.6017 - categorical_accuracy: 0.7406Epoch 00001: val_loss improved from 0.65333 to 0.63058, saving model to models/weights.best.csfd.hdf5\n",
      "14948/14948 [==============================] - 20s - loss: 0.6015 - categorical_accuracy: 0.7406 - val_loss: 0.6306 - val_categorical_accuracy: 0.7339\n",
      "Epoch 3/5\n",
      "14920/14948 [============================>.] - ETA: 0s - loss: 0.5523 - categorical_accuracy: 0.7674Epoch 00002: val_loss did not improve\n",
      "14948/14948 [==============================] - 18s - loss: 0.5526 - categorical_accuracy: 0.7672 - val_loss: 0.6354 - val_categorical_accuracy: 0.7316\n",
      "Epoch 4/5\n",
      "14900/14948 [============================>.] - ETA: 0s - loss: 0.5091 - categorical_accuracy: 0.7894Epoch 00003: val_loss improved from 0.63058 to 0.62832, saving model to models/weights.best.csfd.hdf5\n",
      "14948/14948 [==============================] - 14s - loss: 0.5091 - categorical_accuracy: 0.7893 - val_loss: 0.6283 - val_categorical_accuracy: 0.7358\n",
      "Epoch 5/5\n",
      "14940/14948 [============================>.] - ETA: 0s - loss: 0.4656 - categorical_accuracy: 0.8171Epoch 00004: val_loss did not improve\n",
      "14948/14948 [==============================] - 16s - loss: 0.4656 - categorical_accuracy: 0.8172 - val_loss: 0.6485 - val_categorical_accuracy: 0.7176\n",
      "_________________________________________________________________\n",
      "Loaded the weights for 'csfd' model.\n",
      "\n",
      "_________________________________________________________________\n",
      "SCORES FOR 'csfd' MODEL\n",
      "Test Accuracy: 71.3408%\n",
      "F1 Score: 0.7104\n",
      "F1 Scores: [ 0.76399561  0.60743427  0.73855544]\n",
      "=================================================================\n",
      "\n",
      "_________________________________________________________________\n",
      "SCORES FOR 'csfd' BENCHMARK MODEL\n",
      "Test Accuracy: 0.3388%\n",
      "F1 Score: 0.3386\n",
      "F1 Scores: [ 0.42209302  0.29783694  0.24340771]\n",
      "=================================================================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "csfd_model.fit(csfd_X_train, csfd_y_train, epochs=5, batch_size=20)\\\n",
    "          .test(csfd_X_test, csfd_y_test)\\\n",
    "          .benchmark(csfd_X_test, csfd_y_test);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### MALL.CZ Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_13 (Conv1D)           (None, 30, 64)            25664     \n",
      "_________________________________________________________________\n",
      "max_pooling1d_13 (MaxPooling (None, 15, 64)            0         \n",
      "_________________________________________________________________\n",
      "conv1d_14 (Conv1D)           (None, 15, 128)           65664     \n",
      "_________________________________________________________________\n",
      "max_pooling1d_14 (MaxPooling (None, 7, 128)            0         \n",
      "_________________________________________________________________\n",
      "global_max_pooling1d_7 (Glob (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_7 (Dense)              (None, 3)                 387       \n",
      "=================================================================\n",
      "Total params: 91,715\n",
      "Trainable params: 91,715\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Load the processed data set\n",
    "mall_X_train, mall_X_test, \\\n",
    "    mall_y_train, mall_y_test = load_xy('mall')\n",
    "\n",
    "# Initiate the model\n",
    "mall_model = CNN('mall')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Fitting 'mall' model...\n",
      "_________________________________________________________________\n",
      "Train on 56879 samples, validate on 10038 samples\n",
      "Epoch 1/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.6519 - categorical_accuracy: 0.7637Epoch 00000: val_loss improved from inf to 0.53990, saving model to models/weights.best.mall.hdf5\n",
      "56879/56879 [==============================] - 38s - loss: 0.6505 - categorical_accuracy: 0.7639 - val_loss: 0.5399 - val_categorical_accuracy: 0.7773\n",
      "Epoch 2/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.5359 - categorical_accuracy: 0.7847Epoch 00001: val_loss improved from 0.53990 to 0.52564, saving model to models/weights.best.mall.hdf5\n",
      "56879/56879 [==============================] - 32s - loss: 0.5358 - categorical_accuracy: 0.7849 - val_loss: 0.5256 - val_categorical_accuracy: 0.7859\n",
      "Epoch 3/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.5129 - categorical_accuracy: 0.7954Epoch 00002: val_loss improved from 0.52564 to 0.49949, saving model to models/weights.best.mall.hdf5\n",
      "56879/56879 [==============================] - 32s - loss: 0.5121 - categorical_accuracy: 0.7959 - val_loss: 0.4995 - val_categorical_accuracy: 0.7994\n",
      "Epoch 4/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.4935 - categorical_accuracy: 0.8023Epoch 00003: val_loss improved from 0.49949 to 0.49760, saving model to models/weights.best.mall.hdf5\n",
      "56879/56879 [==============================] - 38s - loss: 0.4935 - categorical_accuracy: 0.8023 - val_loss: 0.4976 - val_categorical_accuracy: 0.7988\n",
      "Epoch 5/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.4814 - categorical_accuracy: 0.8087Epoch 00004: val_loss did not improve\n",
      "56879/56879 [==============================] - 35s - loss: 0.4811 - categorical_accuracy: 0.8088 - val_loss: 0.5085 - val_categorical_accuracy: 0.7947\n",
      "Epoch 6/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.4708 - categorical_accuracy: 0.8125Epoch 00005: val_loss improved from 0.49760 to 0.48498, saving model to models/weights.best.mall.hdf5\n",
      "56879/56879 [==============================] - 36s - loss: 0.4710 - categorical_accuracy: 0.8124 - val_loss: 0.4850 - val_categorical_accuracy: 0.8074\n",
      "Epoch 7/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.4574 - categorical_accuracy: 0.8188Epoch 00006: val_loss improved from 0.48498 to 0.47821, saving model to models/weights.best.mall.hdf5\n",
      "56879/56879 [==============================] - 34s - loss: 0.4580 - categorical_accuracy: 0.8187 - val_loss: 0.4782 - val_categorical_accuracy: 0.8079\n",
      "Epoch 8/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.4478 - categorical_accuracy: 0.8240Epoch 00007: val_loss improved from 0.47821 to 0.47463, saving model to models/weights.best.mall.hdf5\n",
      "56879/56879 [==============================] - 34s - loss: 0.4479 - categorical_accuracy: 0.8238 - val_loss: 0.4746 - val_categorical_accuracy: 0.8114\n",
      "Epoch 9/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.4374 - categorical_accuracy: 0.8291Epoch 00008: val_loss did not improve\n",
      "56879/56879 [==============================] - 35s - loss: 0.4381 - categorical_accuracy: 0.8286 - val_loss: 0.4835 - val_categorical_accuracy: 0.8060\n",
      "Epoch 10/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.4276 - categorical_accuracy: 0.8331Epoch 00009: val_loss improved from 0.47463 to 0.46573, saving model to models/weights.best.mall.hdf5\n",
      "56879/56879 [==============================] - 44s - loss: 0.4274 - categorical_accuracy: 0.8333 - val_loss: 0.4657 - val_categorical_accuracy: 0.8138\n",
      "Epoch 11/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.4175 - categorical_accuracy: 0.8382Epoch 00010: val_loss did not improve\n",
      "56879/56879 [==============================] - 35s - loss: 0.4176 - categorical_accuracy: 0.8381 - val_loss: 0.4841 - val_categorical_accuracy: 0.8083\n",
      "Epoch 12/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.4119 - categorical_accuracy: 0.8401Epoch 00011: val_loss did not improve\n",
      "56879/56879 [==============================] - 31s - loss: 0.4115 - categorical_accuracy: 0.8406 - val_loss: 0.4665 - val_categorical_accuracy: 0.8134\n",
      "Epoch 13/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.4001 - categorical_accuracy: 0.8456Epoch 00012: val_loss improved from 0.46573 to 0.46189, saving model to models/weights.best.mall.hdf5\n",
      "56879/56879 [==============================] - 32s - loss: 0.4000 - categorical_accuracy: 0.8457 - val_loss: 0.4619 - val_categorical_accuracy: 0.8203\n",
      "Epoch 14/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.3897 - categorical_accuracy: 0.8509Epoch 00013: val_loss did not improve\n",
      "56879/56879 [==============================] - 34s - loss: 0.3900 - categorical_accuracy: 0.8506 - val_loss: 0.4882 - val_categorical_accuracy: 0.8045\n",
      "Epoch 15/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.3868 - categorical_accuracy: 0.8518Epoch 00014: val_loss improved from 0.46189 to 0.45522, saving model to models/weights.best.mall.hdf5\n",
      "56879/56879 [==============================] - 35s - loss: 0.3863 - categorical_accuracy: 0.8521 - val_loss: 0.4552 - val_categorical_accuracy: 0.8241\n",
      "Epoch 16/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.3737 - categorical_accuracy: 0.8590Epoch 00015: val_loss did not improve\n",
      "56879/56879 [==============================] - 30s - loss: 0.3740 - categorical_accuracy: 0.8588 - val_loss: 0.4709 - val_categorical_accuracy: 0.8166\n",
      "Epoch 17/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.3676 - categorical_accuracy: 0.8612Epoch 00016: val_loss improved from 0.45522 to 0.45333, saving model to models/weights.best.mall.hdf5\n",
      "56879/56879 [==============================] - 30s - loss: 0.3674 - categorical_accuracy: 0.8614 - val_loss: 0.4533 - val_categorical_accuracy: 0.8255\n",
      "Epoch 18/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.3618 - categorical_accuracy: 0.8643Epoch 00017: val_loss improved from 0.45333 to 0.45231, saving model to models/weights.best.mall.hdf5\n",
      "56879/56879 [==============================] - 30s - loss: 0.3616 - categorical_accuracy: 0.8643 - val_loss: 0.4523 - val_categorical_accuracy: 0.8240\n",
      "Epoch 19/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.3537 - categorical_accuracy: 0.8685Epoch 00018: val_loss did not improve\n",
      "56879/56879 [==============================] - 30s - loss: 0.3539 - categorical_accuracy: 0.8685 - val_loss: 0.4730 - val_categorical_accuracy: 0.8194\n",
      "Epoch 20/20\n",
      "56000/56879 [============================>.] - ETA: 0s - loss: 0.3487 - categorical_accuracy: 0.8699Epoch 00019: val_loss improved from 0.45231 to 0.44950, saving model to models/weights.best.mall.hdf5\n",
      "56879/56879 [==============================] - 32s - loss: 0.3481 - categorical_accuracy: 0.8702 - val_loss: 0.4495 - val_categorical_accuracy: 0.8269\n",
      "_________________________________________________________________\n",
      "Loaded the weights for 'mall' model.\n",
      "\n",
      "_________________________________________________________________\n",
      "SCORES FOR 'mall' MODEL\n",
      "Test Accuracy: 82.5175%\n",
      "F1 Score: 0.8059\n",
      "F1 Scores: [ 0.54639175  0.47906348  0.90231171]\n",
      "=================================================================\n",
      "\n",
      "_________________________________________________________________\n",
      "SCORES FOR 'mall' BENCHMARK MODEL\n",
      "Test Accuracy: 0.6251%\n",
      "F1 Score: 0.6246\n",
      "F1 Scores: [ 0.06906907  0.1837531   0.76683938]\n",
      "=================================================================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "mall_model.fit(mall_X_train, mall_y_train, epochs=20, batch_size=1000)\\\n",
    "          .test(mall_X_test, mall_y_test)\\\n",
    "          .benchmark(mall_X_test, mall_y_test);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Combined Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "conv1d_15 (Conv1D)           (None, 30, 64)            25664     \n",
      "_________________________________________________________________\n",
      "max_pooling1d_15 (MaxPooling (None, 15, 64)            0         \n",
      "_________________________________________________________________\n",
      "conv1d_16 (Conv1D)           (None, 15, 128)           65664     \n",
      "_________________________________________________________________\n",
      "max_pooling1d_16 (MaxPooling (None, 7, 128)            0         \n",
      "_________________________________________________________________\n",
      "global_max_pooling1d_8 (Glob (None, 128)               0         \n",
      "_________________________________________________________________\n",
      "dense_8 (Dense)              (None, 3)                 387       \n",
      "=================================================================\n",
      "Total params: 91,715\n",
      "Trainable params: 91,715\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "# Load the processed data set\n",
    "combined_X_train, combined_X_test, \\\n",
    "    combined_y_train, combined_y_test = load_xy('combined')\n",
    "\n",
    "# Initiate the model\n",
    "combined_model = CNN('combined')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Fitting 'combined' model...\n",
      "_________________________________________________________________\n",
      "Train on 30849 samples, validate on 5444 samples\n",
      "Epoch 1/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 1.0544 - categorical_accuracy: 0.5068Epoch 00000: val_loss improved from inf to 0.88363, saving model to models/weights.best.combined.hdf5\n",
      "30849/30849 [==============================] - 23s - loss: 1.0489 - categorical_accuracy: 0.5088 - val_loss: 0.8836 - val_categorical_accuracy: 0.5737\n",
      "Epoch 2/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.8153 - categorical_accuracy: 0.6154Epoch 00001: val_loss improved from 0.88363 to 0.80145, saving model to models/weights.best.combined.hdf5\n",
      "30849/30849 [==============================] - 19s - loss: 0.8142 - categorical_accuracy: 0.6160 - val_loss: 0.8014 - val_categorical_accuracy: 0.6313\n",
      "Epoch 3/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.7632 - categorical_accuracy: 0.6497Epoch 00002: val_loss improved from 0.80145 to 0.77312, saving model to models/weights.best.combined.hdf5\n",
      "30849/30849 [==============================] - 18s - loss: 0.7640 - categorical_accuracy: 0.6491 - val_loss: 0.7731 - val_categorical_accuracy: 0.6482\n",
      "Epoch 4/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.7358 - categorical_accuracy: 0.6667Epoch 00003: val_loss improved from 0.77312 to 0.76669, saving model to models/weights.best.combined.hdf5\n",
      "30849/30849 [==============================] - 19s - loss: 0.7362 - categorical_accuracy: 0.6665 - val_loss: 0.7667 - val_categorical_accuracy: 0.6565\n",
      "Epoch 5/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.7213 - categorical_accuracy: 0.6748Epoch 00004: val_loss improved from 0.76669 to 0.75616, saving model to models/weights.best.combined.hdf5\n",
      "30849/30849 [==============================] - 18s - loss: 0.7214 - categorical_accuracy: 0.6753 - val_loss: 0.7562 - val_categorical_accuracy: 0.6537\n",
      "Epoch 6/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.7004 - categorical_accuracy: 0.6881Epoch 00005: val_loss improved from 0.75616 to 0.74555, saving model to models/weights.best.combined.hdf5\n",
      "30849/30849 [==============================] - 17s - loss: 0.7005 - categorical_accuracy: 0.6884 - val_loss: 0.7456 - val_categorical_accuracy: 0.6631\n",
      "Epoch 7/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.6886 - categorical_accuracy: 0.6935Epoch 00006: val_loss did not improve\n",
      "30849/30849 [==============================] - 19s - loss: 0.6888 - categorical_accuracy: 0.6938 - val_loss: 0.7510 - val_categorical_accuracy: 0.6607\n",
      "Epoch 8/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.6733 - categorical_accuracy: 0.7024Epoch 00007: val_loss did not improve\n",
      "30849/30849 [==============================] - 19s - loss: 0.6744 - categorical_accuracy: 0.7016 - val_loss: 0.7662 - val_categorical_accuracy: 0.6594\n",
      "Epoch 9/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.6626 - categorical_accuracy: 0.7102Epoch 00008: val_loss improved from 0.74555 to 0.74387, saving model to models/weights.best.combined.hdf5\n",
      "30849/30849 [==============================] - 19s - loss: 0.6625 - categorical_accuracy: 0.7104 - val_loss: 0.7439 - val_categorical_accuracy: 0.6672\n",
      "Epoch 10/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.6558 - categorical_accuracy: 0.7137Epoch 00009: val_loss improved from 0.74387 to 0.73464, saving model to models/weights.best.combined.hdf5\n",
      "30849/30849 [==============================] - 18s - loss: 0.6564 - categorical_accuracy: 0.7132 - val_loss: 0.7346 - val_categorical_accuracy: 0.6666\n",
      "Epoch 11/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.6318 - categorical_accuracy: 0.7294Epoch 00010: val_loss did not improve\n",
      "30849/30849 [==============================] - 20s - loss: 0.6315 - categorical_accuracy: 0.7296 - val_loss: 0.7421 - val_categorical_accuracy: 0.6686\n",
      "Epoch 12/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.6343 - categorical_accuracy: 0.7271Epoch 00011: val_loss improved from 0.73464 to 0.73332, saving model to models/weights.best.combined.hdf5\n",
      "30849/30849 [==============================] - 18s - loss: 0.6336 - categorical_accuracy: 0.7280 - val_loss: 0.7333 - val_categorical_accuracy: 0.6708\n",
      "Epoch 13/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.6129 - categorical_accuracy: 0.7391Epoch 00012: val_loss did not improve\n",
      "30849/30849 [==============================] - 18s - loss: 0.6128 - categorical_accuracy: 0.7389 - val_loss: 0.7352 - val_categorical_accuracy: 0.6701\n",
      "Epoch 14/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.6061 - categorical_accuracy: 0.7435Epoch 00013: val_loss did not improve\n",
      "30849/30849 [==============================] - 19s - loss: 0.6066 - categorical_accuracy: 0.7430 - val_loss: 0.7354 - val_categorical_accuracy: 0.6695\n",
      "Epoch 15/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.5998 - categorical_accuracy: 0.7450Epoch 00014: val_loss improved from 0.73332 to 0.73325, saving model to models/weights.best.combined.hdf5\n",
      "30849/30849 [==============================] - 18s - loss: 0.5996 - categorical_accuracy: 0.7452 - val_loss: 0.7333 - val_categorical_accuracy: 0.6754\n",
      "Epoch 16/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.5858 - categorical_accuracy: 0.7548Epoch 00015: val_loss did not improve\n",
      "30849/30849 [==============================] - 19s - loss: 0.5853 - categorical_accuracy: 0.7553 - val_loss: 0.7405 - val_categorical_accuracy: 0.6752\n",
      "Epoch 17/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.5784 - categorical_accuracy: 0.7609Epoch 00016: val_loss did not improve\n",
      "30849/30849 [==============================] - 18s - loss: 0.5776 - categorical_accuracy: 0.7610 - val_loss: 0.7355 - val_categorical_accuracy: 0.6741\n",
      "Epoch 18/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.5717 - categorical_accuracy: 0.7616Epoch 00017: val_loss did not improve\n",
      "30849/30849 [==============================] - 17s - loss: 0.5721 - categorical_accuracy: 0.7613 - val_loss: 0.7388 - val_categorical_accuracy: 0.6695\n",
      "Epoch 19/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.5588 - categorical_accuracy: 0.7695Epoch 00018: val_loss did not improve\n",
      "30849/30849 [==============================] - 19s - loss: 0.5596 - categorical_accuracy: 0.7696 - val_loss: 0.7604 - val_categorical_accuracy: 0.6633\n",
      "Epoch 20/20\n",
      "30000/30849 [============================>.] - ETA: 0s - loss: 0.5478 - categorical_accuracy: 0.7757Epoch 00019: val_loss did not improve\n",
      "30849/30849 [==============================] - 18s - loss: 0.5478 - categorical_accuracy: 0.7757 - val_loss: 0.7522 - val_categorical_accuracy: 0.6692\n",
      "_________________________________________________________________\n",
      "Loaded the weights for 'combined' model.\n",
      "\n",
      "_________________________________________________________________\n",
      "SCORES FOR 'combined' MODEL\n",
      "Test Accuracy: 67.8155%\n",
      "F1 Score: 0.6749\n",
      "F1 Scores: [ 0.69775982  0.56971634  0.75492958]\n",
      "=================================================================\n",
      "\n",
      "_________________________________________________________________\n",
      "SCORES FOR 'combined' BENCHMARK MODEL\n",
      "Test Accuracy: 0.3223%\n",
      "F1 Score: 0.3223\n",
      "F1 Scores: [ 0.31375465  0.32042928  0.33249006]\n",
      "=================================================================\n",
      "\n"
     ]
    }
   ],
   "source": [
    "combined_model.fit(combined_X_train, combined_y_train, epochs=20, batch_size=1000)\\\n",
    "              .test(combined_X_test, combined_y_test)\\\n",
    "              .benchmark(combined_X_test, combined_y_test);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Part 5: Running tests with new sentences\n",
    "\n",
    "![Four example sentences and their predictions from our models. ](images/tests.png)\n",
    "\n",
    "---\n",
    "\n",
    "### Coded version of the above tests"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "=================================================================\n",
      "Trying out the sentences on 'facebook' model:\n",
      "\n",
      "TRUE: NEG ----------------------\n",
      "CZ: 'Rozbila se po prvním použití, je na hovno!' is NEG\n",
      "EN: 'It broke after the first use; it's shitty!'\n",
      "\n",
      "TRUE: NEU ----------------------\n",
      "CZ: 'Rozbila se až za rok' is NEU\n",
      "EN: 'It broke after a year of use'\n",
      "\n",
      "TRUE: POS ----------------------\n",
      "CZ: 'S manželem jsme si víkend moc užili' is POS\n",
      "EN: 'Me and my husband really enjoyed the weekend'\n",
      "\n",
      "TRUE: NEU ----------------------\n",
      "CZ: 'Ok, ale nic zajímavého' is NEU\n",
      "EN: 'Ok, but nothing interesting'\n",
      "\n",
      "=================================================================\n",
      "Trying out the sentences on 'csfd' model:\n",
      "\n",
      "TRUE: NEG ----------------------\n",
      "CZ: 'Rozbila se po prvním použití, je na hovno!' is NEG\n",
      "EN: 'It broke after the first use; it's shitty!'\n",
      "\n",
      "TRUE: NEU ----------------------\n",
      "CZ: 'Rozbila se až za rok' is NEG\n",
      "EN: 'It broke after a year of use'\n",
      "\n",
      "TRUE: POS ----------------------\n",
      "CZ: 'S manželem jsme si víkend moc užili' is POS\n",
      "EN: 'Me and my husband really enjoyed the weekend'\n",
      "\n",
      "TRUE: NEU ----------------------\n",
      "CZ: 'Ok, ale nic zajímavého' is NEU\n",
      "EN: 'Ok, but nothing interesting'\n",
      "\n",
      "=================================================================\n",
      "Trying out the sentences on 'mall' model:\n",
      "\n",
      "TRUE: NEG ----------------------\n",
      "CZ: 'Rozbila se po prvním použití, je na hovno!' is POS\n",
      "EN: 'It broke after the first use; it's shitty!'\n",
      "\n",
      "TRUE: NEU ----------------------\n",
      "CZ: 'Rozbila se až za rok' is POS\n",
      "EN: 'It broke after a year of use'\n",
      "\n",
      "TRUE: POS ----------------------\n",
      "CZ: 'S manželem jsme si víkend moc užili' is POS\n",
      "EN: 'Me and my husband really enjoyed the weekend'\n",
      "\n",
      "TRUE: NEU ----------------------\n",
      "CZ: 'Ok, ale nic zajímavého' is NEG\n",
      "EN: 'Ok, but nothing interesting'\n",
      "\n",
      "=================================================================\n",
      "Trying out the sentences on 'combined' model:\n",
      "\n",
      "TRUE: NEG ----------------------\n",
      "CZ: 'Rozbila se po prvním použití, je na hovno!' is NEG\n",
      "EN: 'It broke after the first use; it's shitty!'\n",
      "\n",
      "TRUE: NEU ----------------------\n",
      "CZ: 'Rozbila se až za rok' is NEG\n",
      "EN: 'It broke after a year of use'\n",
      "\n",
      "TRUE: POS ----------------------\n",
      "CZ: 'S manželem jsme si víkend moc užili' is POS\n",
      "EN: 'Me and my husband really enjoyed the weekend'\n",
      "\n",
      "TRUE: NEU ----------------------\n",
      "CZ: 'Ok, ale nic zajímavého' is NEU\n",
      "EN: 'Ok, but nothing interesting'\n",
      "\n"
     ]
    }
   ],
   "source": [
    "trained_models = [\n",
    "    facebook_model,\n",
    "    csfd_model,\n",
    "    mall_model,\n",
    "    combined_model\n",
    "]\n",
    "\n",
    "test_sentences = [\n",
    "    ('Rozbila se po prvním použití, je na hovno!',\n",
    "     'It broke after the first use; it\\'s shitty!',\n",
    "     'NEG'),\n",
    "    ('Rozbila se až za rok',\n",
    "     'It broke after a year of use',\n",
    "     'NEU'),\n",
    "    ('S manželem jsme si víkend moc užili',\n",
    "     'Me and my husband really enjoyed the weekend',\n",
    "     'POS'),\n",
    "    ('Ok, ale nic zajímavého',\n",
    "     'Ok, but nothing interesting',\n",
    "     'NEU')\n",
    "]\n",
    "\n",
    "for i,model in enumerate(trained_models):\n",
    "    print('='*65)\n",
    "    print('Trying out the sentences on \\'{}\\' model:\\n'.format(model.name))\n",
    "    \n",
    "    for sent in test_sentences:\n",
    "        print('TRUE: {} ----------------------\\nCZ: \\'{}\\' is {}\\nEN: \\'{}\\'\\n'\\\n",
    "              .format(sent[2],\n",
    "                      sent[0],\n",
    "                      target_labels[model.predict(sent[0])],\n",
    "                      sent[1])\n",
    "             )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "---\n",
    "\n",
    "---"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}