Quantum_MNIST.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github.com/Rishit-dagli/TFUG-Mysuru-2020/blob/master/TFQuantum_starter.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MNIST classification with QNNs\n",
    "\n",
    "In this notebook you will build a quantum neural network (QNN) to classify a simplified version of MNIST with \n",
    "[Cirq](https://cirq.readthedocs.io/en/stable/) and TensorFlow Quantum (TFQ). The tutorial in general shows using a simple quantum neural network on a classical data problem. We here follow the approach used in the [Classification with Quantum Neural Networks on Near Term Processors](https://arxiv.org/pdf/1802.06002.pdf) paper by Edward Farhi and Hartmut Neven.\n",
    "\n",
    "![](http://storage.rishit.tech/storage/TFUG-Mysuru-2020/classfication_with_qnn_paper.JPG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> Note: This notebook is designed to be run in Google Colab if you want to run it locally or on a Jupyter notebook you \n",
    "would skip the code cells with the `Colab only` comment."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Install TensorFlow 2.x (Colab only)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Colab only\n",
    "pip install -q tensorflow==2.1.0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Install TensorFlow Quantum (Colab only)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Colab only\n",
    "pip install -q tensorflow-quantum"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports\n",
    "\n",
    "Now import TensorFlow and the module dependencies:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import tensorflow_quantum as tfq\n",
    "\n",
    "import cirq\n",
    "import sympy\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import collections\n",
    "import random\n",
    "\n",
    "# visualization tools\n",
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "from cirq.contrib.svg import SVGCircuit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preprocessing the data\n",
    "\n",
    "In the last tutorial you saw how to make a model to do hybrid quantum-classical binary classification. As this is meant to be an easy to follow and reproducoble tutorial, we will perform binary classification with MNIST. We will perform binary classification for this purpose with just two labels here `1` and `5` (Choosen as they look a lot different unlike something like 1 and 7)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the raw data\n",
    "\n",
    "Load the MNIST dataset distributed with Keras."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAT4AAAIzCAYAAACZTnVzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nOzdeVxU1fsH8M+ZjWFm2FcREBCREJVcUdAyF9TUIrU0cyvXMsvsm9lm5Te1sk1z66stammLuWDmbmqaOyIqiogssu/bDMMsz+8PdX6agCDI1bnn/Xrd10vhzj3PPHPvM+fec+6FERE4juPERCJ0ABzHcU2NFz6O40SHFz6O40SHFz6O40SHFz6O40SHFz6O40SHFz6O40SHFz6uyTHGnBljGxljFYyxVMbYs0LHxImLTOgAOFFaAqAKgAeAMAB/MMbiiOicsGFxYsH4nRtcU2KMqQEUAQglosTrP1sDIIOI3hQ0OE40+Kku19SCAJhuFL3r4gC0ESgeToR44eOamgZAyb9+VgLAToBYOJHihY9rauUA7P/1M3sAZQLEwokUL3xcU0sEIGOMtbrpZ+0B8IENrsnwwQ2uyTHG1gMgABNwbVR3G4DufFSXayq8x8cJ4UUAtgByAawDMJUXPa4p8R4fx3Giw3t8HMeJDi98HMeJDi98HMeJDi98HMeJTq0PKWCM3fORDyJi97qNxsbzUj2el+rxvFRPyLzwHh/HcaLDCx/HcaLDn8fHNSkbGxs8+uij8PHxQUZGBi5duoTU1FQYDAahQ+PuQ4wxaDQaBAUFISwsDBkZGTh8+DBKS0sbtN1GL3xSqRQPPfQQ1Go1GGMIDg6GWq3Gjh07kJSU1NjN3Ze8vb0xYcIEKJXKW35+4cIFpKenIz4+Hrm5uQJFJyypVIqgoCB069YNmZmZGDhwIFJSUrB7926cO3cORqNR6BDvCy1atMBjjz2G1q1bQyqVoqSkBDt37kR6ejqysrKEDq/JODo6YvLkyXj88cfRuXNnpKenY9y4cfjnn39gNpvveruNVvgUCgVcXFwQERGBiRMnws3NDYwxuLu7Q6FQYNCgQfjuu++wdetWaLXaxmr2vuPj44NFixZhwIABkMluTW9OTg6Ki4uRkpKCrVu34tKlS4iLi0NeXp5A0TY9qVQKNzc3nD9/HmvXroWvry+ioqLw+eef4+TJk/j4449RUFAgdJhNjjGGoKAg9O3b19Ir7tChAzw8PCCRSKDT6fDEE09g3759WLBgAQoLC4UO+Z5TKpV47bXXMHnyZLi4uAAAAgIC4O7uDrlcDr1ef/cbJ6IaF1y7kfyOi1QqpX79+tH+/fspIyODqqqqyGQy3bJUVVXRzp07qX379re8trb279eltly89957VFlZSVVVVRQfH08ff/wxPf300zRy5Ejq3bs3zZkzh2JiYigvL49SUlJo/fr1FBoaett2hH6PjZ2XG4uXlxdduXKFFixYYPmZs7MzjR07ljIyMmjLli0UFhZW4+uFfo/3Ii8ymYxGjBhB27Zto/z8fCosLKz2GDKZTJSQkEBDhgwhiURi1XlxcnKir776ioqLi2/LwZUrV2jlypXk6el5x/2txrYbuiMDoKCgIDp06BAZDIZqP6wbS2VlJS1btow8PDys8gN79dVXKS8vj4xGIx09epTatGlDtra2JJPJSCaTkUQiIaVSSS4uLvThhx9SWloaVVRU0Pr166l169ZWvSPf+IKcM2cOpaen03PPPXfL7+RyOYWFhdHnn39OMTEx1K5dO9EUvvHjx1NJSYml2F2+fJmysrKqPYYMBgNt3ryZgoKCrDYvMpmMFi9eTFqtttZa8uOPP5JSqWzawscYo9atW9O8efPo2LFjVFZWZunZXbx4kWJjY+ngwYPUs2dPGj58OJ06dYpMJhNlZWVReHi41X1gEomEDh06REajkXQ6HU2YMKHWD0Qul9PQoUOpqqqKSkpKaNy4cXR9XpNV5eXmxcfHh3Jycmjfvn3UsmXLatdxcHCgn3/+mVavXk3Ozs5WX/imTJlCJSUllgP6xIkT1Lt3b/ryyy+poqKCTCYT7dq1ix555BG6cuUKmUwmqqiooLFjx1ptXjw9PSknJ8eSE6PRSGVlZVRaWkpGo/GWL4GjR4/StGnTqFmzZqRWq+994QsPD6f09PRbgtNqtZScnHzb6ayvry/t27fPsm7v3r2t7gMbMGAApaWlUVVVFe3fv58cHR1rLQIAyN3dnebPn0/p6el0+PBhCgkJsbq83LwsXryY8vPzadCgQbV+oXbq1In27t1L0dHRVl34fHx8KCEhwXIQX7p0iWbPnk12dnY0f/58S2di8+bNpFaradWqVZZjaNKkSVabl9WrV5NeryeTyUR6vZ6OHTtG06ZNo9GjR9P27dspPT2d8vPzSafTWfJRUFBAM2bMIKlUWqe83NXghoeHByZOnIhmzZoBAMrLy3HhwgUcO3YM27dvx6VLl2p8rdlsbtBozP1IJpOhc+fOsLe3R0JCAsaOHYvi4uI7vi43Nxc//vgjunfvjsjISDRr1gznz59vgoibXmBgIIYMGYKrV6/i4MGDNa5HREhNTUVcXBxatmwJqVQKk8nUhJHeexKJBAEBAfjss88QEBAAk8mEtLQ0LFiwAKtWrQIAZGdno7S0FCqVCsC1vFjzoGBN9Ho9tm7diq+//hoAcPDgQQQHB8PGxgbR0dHo0qULAgMDoVar0adPH/zzzz84ceLEHWcH1Lvw2dvbY9asWRgxYgQYY7h8+TLWrVuHmJgYnD9//rYPhzEGOzs7qNVqGI1GnDx50upGMf38/NCnTx/Y2dnhwIEDSEtLEzqk+84nn3wCd3d3fPbZZ3ecg1VVVQW9Xo/27dvD2dnZqvYXiUSCtm3b4p133kHfvn0hk8mQmpqKDz74AL/++qtlvbVr18LFxQXjx49HRUUFqqqqcPToUUycOBFyuVzAd3BvaTQa2NraArhW7BMTE3H8+HHL71NSUpCSkgIA2L9/P0JCQrB48WKEhYWhd+/eAIBXXnnljlPn6l34AgICMGTIECiVSmi1WqxYsQJffPFFjb04d3d39OzZE4GBgYiLi8MHH3yAxMTEatd9UPn5+cHV1RUA8Ndff93oxteLVqu12jlsfn5+iIyMRFJSEn799dc75sdkMqGiogJyudzqDnIPDw9MmDAB/fv3h42NDfLz8/HVV19hzZo1t6xXUFCAL7/8EklJScjNzQURoays7K72rQdJx44dLXMX4+LiMHfuXPz555/VrltcXIzY2Fjs27cPYWFhkMvl6N+/P1q0aNG4hU8qlaJPnz5wdnYGAGRmZuKnn36qsehJpVI89dRTmD59OkwmE37//XccOnQIVVVV9Wn2vhcQEABnZ2eUl5fj9OnTdd45XVxcMHDgQHh7e+PIkSPIyMi4x5E2PZlMhkmTJkGj0WDu3Ll1mnyrUCjg6OiIjIwMFBUVNUGUTcfJyckyqR8A3n///duK3g2FhYVYvXo1AMDW1haPPPIIFAoFioqK6nQp5UHj5uaGESNGwN/fH4wxxMTEYNu2bfekrXrdqztgwACMGTMG9vbX/jrgvn37at2R+/Tpg7feegteXl5YvXo1Vq1ahYqKioZFfB8KCAiAq6srDhw4UOfJt3Z2dnj55Zfx2muvwdnZGb///jtSU1PvcaRNT6PRoF+/figpKcGWLVvq9BqlUgl7e3tcvHgRlZWV9zjCpqVSqaDRaABcO1XbuHEjysvL7/g6Ly8vDB48GGazGUePHsXZs2fvdahNrn379oiMjLRc1ywsLKz1VkbGGPr374/AwMB6t1WvHl9ERAQeeughMMawd+9evP/++zWuGx0djc8++wxeXl64evUqtm/fblXXam4mk8kgkUhQVFR0x9NVhUIBX19fjB49GjNmzIBUKsXq1auxceNGq7xf1dvbG46Ojvjxxx/r1KOVy+Xo3bs31Go1NmzYYHWndm3btsXDDz8MIsLFixdRVnbnPyesUCgwduxY+Pv74+LFi/j6669x8eLFJoi2afXq1QutW7e+43pTpkxB27ZtIZVK8cQTT8DBwcHyu71799apA1GvwieXyyGVSkFEKCgouOV+U8YYGGNwdHTE66+/jnHjxsHd3R1lZWU4fPgw4uLi6tPUA+XGEHn37t0tp7z/JpFI4OLigsmTJ2PWrFlQKBSorKzE+vXr8eGHHyI7O1uAyO+9G/dt//LLL3f8UmCMoVu3bhgzZgxWrVpllbeuKRQKKBQKVFRUYNmyZbWeATHGIJFIMG/ePEyfPh3AtdseExISrG6kG7iWG6lUavl/8+bN0aFDB8v//f398eKLL6JLly6wsbGx5AcADAYD/vrrL7zzzjtITk6+Y1t3fa+uQqGAra0tJBIJPDw8MGDAALi4uKBbt26IjIyEjY0NcnNzMWLECBw4cOBum3kgnDlzBpmZmfD398eUKVOwcOFCS+9NKpUiICAAw4YNw5NPPonAwEDo9XokJibiq6++wsqVKwWO/t4qLS2F0WhE7969cfLkyRoPWJlMhoCAACxbtgxnz57F9u3bmzjSpiWTydCqVSskJCRYrnnfOIilUilcXFwQGRmJOXPmoHXr1jCbzdi8eTOmT59utQ+4OHnyJK5evQpvb28AwOuvv47XX3+91tcYjUZkZmZiw4YN+OWXX3DixIk6tVWvwnfzaUffvn3x5ptvIiQkBF26dIGbm5tlBI6IkJWVhUmTJll90QOA8+fP4/Tp03B1dcWsWbPQtm1b6HQ6ANeuV7Vr1w7e3t4oKSnB1atX8c0332DNmjVWOZjxb8eOHUNWVhaeffZZ7N27FxcvXrzlwrxMJoObmxt69OiBmTNn4sKFC5g+fbpVXrwH/n+qjkKhwAsvvACj0YiEhATodDrLZaRmzZohOjoavXr1gkqlQm5uLuLj4/HNN99Y9ZNZDh8+jBMnTsDNzQ02NjY1rqfX62EymZCdnY1Tp07h008/xYULF+p0rdSiPjOrp0yZQhkZGbfcNvLv++dSU1MpJiaG5syZQzKZ7I53Lwg9e7yxZpx36tSJduzYQWlpaWQwGMhoNFqW/Px8+uuvv+g///kP9e7d+7YbzK05L3K5nKZNm0b5+fmUnJxMCxYsoAEDBlBUVBRFRUXRyJEjaf369ZSYmEhr1qwhFxcXq85LeHg4HTly5JZj6ODBg/THH39Y7tU1m81kNpupoKCAtm/fTqNHjyZ3d3erzsuNpWPHjrRp0ybLnRs33xmWnZ1NCQkJ9N1339H8+fOpQ4cOd6wxNbVd6x8U//cz8Z2cnDBhwgQ8//zzCAwMtHTNy8vLkZWVhePHj2Pnzp2IiYmp8zc2WdHfCggLC8NDDz2Etm3b3vLzjIwM7Nq1C5cvX67ztRlryotMJsNrr72Ghx9+GAEBAWjWrBns7OxgNBpx+fJlJCQk4MCBA9i6desdB8Ae9Lw4ODhg+PDhGDVqFNq2bQsnJ6db1jUajaioqEBCQgJ27NiBJUuWoLCwELUdp8CDn5ebhYWFYfr06ZaJzMC1vJw6dQrnz5/HqVOn6jxQWlNe6lX4gGvFr3Pnzhg6dCiUSiVMJhMuXbqEEydO4OLFi8jMzKzXRFxr+sAak7XlRS6Xw9XVFS1atIC/vz/s7e0the/ChQvIzc2t062M1pAXlUqFoKAgPPLII+jQoYOlAwFcm5R75swZxMbGIjExsc5PGraGvNzM09PzlrwQEUpLS+s9Ha7RCt8NHh4elhHe8vLyOg3L1yew+xkvfNXjealeTXlRqVRwdHQEY///lqqqqlBcXFzvqU3WlJfG1OiFr7HwD6x6PC/V43mpHs9L9WrKC/8raxzHiU6tPT6O4zhrxHt8HMeJDi98HMeJDi98HMeJDi98HMeJDi98HMeJDi98HMeJDi98HMeJDi98HMeJDi98HMeJDi98HMeJDi98HMeJDi98HMeJDi98HMeJDi98HMeJDi98HMeJDi98HMeJDi98HMeJDi98XJNjjE1jjJ1gjOkZY98LHQ8nPjKhA+BEKRPAfwFEAbC9w7oc1+h44eOaHBH9DgCMsU4AvAUOhxMhfqrLcZzo8MLHcZzo8MLHcZzo8MLHcZzo8MENrskxxmS4tu9JAUgZY0oARiIyChsZJxa8x8cJ4R0AOgBvAnju+r/fETQiTlQYEQkdA8dxXJPiPT6O40SHFz6O40SHFz6O40SHFz6O40Sn1uksjLF7PvJBROxet9HYeF6qx/NSPZ6X6gmZF97j4zhOdHjh4zhOdPidG9x9wdbWFlFRUXBycsKePXuQlpYmdEiCatmyJby9vXHkyBHo9XqhwxGcg4MDunfvDl9fX+zfvx+JiYkwm813vb173uOzt7dHdHQ0pk2bBjc3t3vd3H3P1dUVM2fOxMsvvwy5XC50OPcFlUqFcePGYfHixfj6668xatQoODo6Ch2WYKRSKYYNG4bJkydDrVYLHc59gTGGQYMGYcmSJZg0aRJUKlWDtndPe3yMMfTs2ROffvopMjMzsX37duTl5d3LJu9706dPx7Rp06DT6ZCdnY1ff/1V6JAE161bN8yYMQOenp7YvXs39u7dK+pejpeXF7p16wapVCp0KPeN4uJiZGdnA7jWG25op+GeFj4bGxu89NJLkEgk+OKLL0R/+tKpUyeMGjUKdnZ2OHToEMrLy4UOSXBBQUF455134O/vj127dmHSpEnIyMiAmG+l9Pf3h5+fHzIzM4UOxWrd08I3ceJEREZG4ttvv0VMTAyMRvE+fEMqlWLy5Mlo1qwZ/vnnH4wePRoVFRVChyUoZ2dnvPXWW5bezYYNG3D16lWhwxKUQqFAREQEWrZsiZSUFFF/AdxL96zw9e/fH++//z5SUlKwePFiURc9AHjooYcQHh4OhUKBFStWoLi4WOiQBBcREYFu3bpBoVAgISEBu3fvFjokwXl4eCAsLAwajQb79++HTqcTOqT7gkajgbOzMxhrnOmKjV74GGMICQnBnDlzAACffPIJkpKSGruZBwpjDJGRkXB1dUVmZiY/wAE0a9YMAwYMgLe3N3Q6Hb7//nvR9/YAwMXFBV5eXjAajcjKyhJ9h+GGkJAQ9OjRo9EKX6OO6kokErRv3x7/+9//4O/vjy+//BJr1qxpzCYeSD4+PujRoweqqqrw2WefIT8/X+iQBHVj0Gv06NFQKpXYtWsXvv32W5hMJqFDE5xMJoNCoUBBQQEuXLjAC991paWljTow2miFTyKRIDQ0FJ9//jl8fHywePFiLFq0qLE2/8BSqVR45pln0K9fP+zfvx/ff/+96A9wOzs7hISEwMbGBgCQnp6OqqoqgaMSnkQigY+PD3x9fXH27FmUlpYKHdJ9QyKRQCZrvBPURtmSRCJBq1atMHPmTAQGBmLp0qX45ptvUFJS0hibf2BJpVJ07doVQ4YMQVVVFfbv34/KykqhwxKcl5cXHn30UchkMuTn52Pbtm3QarVChyU4R0dH9OjRA25ubsjMzOQ5uYmjoyNcXV0bbXuN0uOzs7PD8OHD0bNnT/zwww9YtWoVCgoKGmPTDzTGGJo3b47Q0FDk5OTg+PHjop6fBly7SD169Gi0bdsWRqMRu3btQlxcnOh7wcC1yf4PPfQQiIj3+G6iUqnQoUMHtGjRotG22eDCp1arMW3aNAwePBhr1qzBkiVLkJub2xixPdAYYwgICMDQoUOh0+mwevVqJCcnCx2W4JRKJSIiIuDg4ICCggJs2rSJ7y/X3Tid02q1uHLlCh/RvU4ul8PLy6tR7+ZpUOGTSqXo3LkzRo8ejdjYWCxfvtwyu1rslEolnnjiCURFReHcuXPYvXu36CcsM8bg5+cHb29vAMDu3btx6NAhfgH/OmdnZ3Ts2BHnz59HcnIyn8NXDSLC6dOnG3zm1KDC17NnT2zbtg15eXmYP38+n2l+HWMMLVq0wPjx41FQUIDdu3cjKytL6LAEJ5VKMXToUPj5+eHq1avYvn07cnJyhA7rviGVSmFjYwODwcC/DKrBGANjDF5eXg0e6Ljrwte1a1esXbsWV65cwYwZM5CamtqgQKxJQEAAfv75Z7Ro0QLbtm3D119/za954tpUjeeeew6MMZSUlCAzM5Nf27tOLpejdevWUCgUQody37rRA7azs4NE0rCrdHf1al9fXyxcuBClpaV47bXXcOLEiQYFYW2mTZuG4OBgHD9+HKtXrxb9rWnVSU1NxeXLl/kTaq5TqVTo2bMnZDIZSkpK+D5TA6PRiJUrVzZ4xki9C5+zszPmzp2Lli1bYuLEifwuhJswxhAcHIz+/fsjKSkJX375JQ4dOiR0WPcNs9lseVBF9+7d8f7778Pf31/gqO4PMpkMzZs3h8FgwOnTp5GRkSF0SPellJQUZGdnN/j6Z71OlB0dHTFp0iR0794dn3/+OU6dOsVPVW7i5uaGBQsWwNXVFW+99RY2bdokdEj3FaPRiMWLF0OlUsHDw4OPWt7EbDajoqICmZmZOH36NM9NDbZv394o18vrVfgiIyMRERGBb7/9FkuXLuUTLG/i7OyMmTNnom/fvvjss8/wyy+/CB3SfcdsNmP79u24evUq3NzccOzYMT4L4LqKigp8/fXXcHFxwd9//y10OPcVnU6HTZs2ITU1FXFxcY0yO4LV1mX8919Bat68OZycnJCWltZokyut5a9DKZVKhIaGws7ODnFxcSgsLGxQG9aSl8bG81I9npfq1ZSXehW+e4F/YNXjeakez0v1eF6qx/+8JMdx3HW19vg4juOsEe/xcRwnOrzwcRwnOrzwcRwnOrzwcRwnOrzwcRwnOrzwcRwnOrzwcRwnOrzwcRwnOrzwcRwnOrzwcRwnOrzwcRwnOrzwcRwnOrzwcRwnOrzwcRwnOrzwcRwnOrzwcRwnOrzwcU2KMWbDGFvFGEtljJUxxmIZYwOEjosTF174uKYmA5AO4BEADgDeBfALY8xPwJg4keGPnucExxg7A+ADItogdCycOPAeHycoxpgHgCAA54SOhRMP3uPjBMMYkwP4E8BlIposdDycePDCxwmCMSYB8BMAewBPEJFB4JA4EZEJHQAnPowxBmAVAA8AA3nR45oaL3ycEJYBeAhAHyLSCR0MJz78VJdrUoyxFgBSAOgBGG/61WQi+lGQoDjR4YWP4zjR4dNZOI4THV74OI4THV74OI4THV74OI4TnVqnszDG7vnIBxGxe91GY+N5qR7PS/V4XqonZF54j4/jONFp8ARmd3d3hIWFWf6flpaG9PR0VFRUNHTTHCdqzs7O8PPzg0QiQXx8PPR6vdAhWY0GFb5+/fohKioKM2bMwI35gH/99Rc2b96MVatW8eL3LxKJBDKZDL6+vnB3d8fx48dhMIjvbi2pVIpWrVohPDwcSqUSly9fxtmzZ5GVlSV0aPeFgIAAtGzZEo899hgGDRoEpVKJd999F+vXrxc6NKtR6wTmO52Dnzt3Dj4+PtBoNJbCxxhDfn4+wsLCkJmZeccAxHBtwsfHBz4+PvD29kbXrl0RHBwMDw8PbNy4EYsWLUJZWdltr7HWvGg0GgwcOBDDhw/HwIEDYWtri8TERBw8eBBvvPEGioqKan29NeaFMYZ27drBzc0NUVFRaN26NVq1agVfX18olUowxpCYmIiZM2fijz/+qHYb1piXf5NIJAgKCoKfnx+kUin++ecflJaWwmg01viamvLSoB5fYGAgpFIpiAjp6enQarVo3bo1nJ2dMWDAAHz33Xcwm80NaeKBZWNjg+DgYAwdOhQhISFo1aoV7O3t4e3tDalUCgBo1qwZzp8/j40bNwocbdOQSqUYPXo0/vOf/8BsNuP48eMIDQ21HOQHDhzAmjVrhA6zSQUGBuKpp57C0KFDoVKpEBISAgA4efIkVq5cCZ1Oh9atW6Nfv37o0aNHjYXP2qnVagwfPhzjx4+Hh4cHJBIJUlJSEB8fj3Xr1uHEiRP12yAR1bgAoNqWP//8k3Jycig5OZmio6MpOjqazp07RyaTifbt20fe3t61vv5a8zW3f78utb0fjUZDEydOpHnz5tHly5eprKyM9Ho9mc3m25bS0lKaO3euKPICgGbPnk3Z2dlUWFhIH3zwAfn5+VFUVBT9888/ZDAYKDU1lSIjI0Wxvzg6OlJkZCTFxsZSYWEhERGlp6fT1KlTKTQ0lPz8/MjJyYkcHBxo7NixZDQaKTY2ltq3b2/Vealu8fT0pA8//JCSk5PJZDKRXq8ng8FAZrOZKioqaMeOHaRSqeqVlwYF5ujoSAMGDKChQ4eSm5sbyWQyWrduHRmNRtLr9RQdHS2qwufg4EArV66k0tJS0ul0lg8mMTGRvvzyS1q6dCmVlZVZCt+lS5coLCzM6vNyY8nLyyOTyUSVlZV05swZWr9+PfXu3Zu+/PJLqqiooKqqKho8eLAo9peuXbtSXFwcGY1Gqqqqoj179tDgwYNJoVDctu7w4cPJaDTS5cuXKTw83Krz8u8lODiYYmJiSKvVkslkom3bttG4ceNo7Nix9M0335DRaKTU1FTq168fXT91vveFDwAxxm5pcM2aNZYDe8SIEaIqfM888wyVlJSQ2Wwmk8lExcXFtHz5cvLy8qKoqChKSkqy5KaiooIWLVpU72+q+3m502ddUFBAJpOJTCYTGY1G2r59O4WEhNBHH31EZWVlZDQa6X//+58o9peIiAhKSkoivV5PO3bsIKlUWu2BC4BkMhlt2bKFzGYz/fbbb1adl5sXe3t7Wr58OZlMJqqqqqIzZ87QoEGDSC6XE2OMFAoFpaSkkNlspk2bNlX7pVFT2w2ex3f9DUCpVMLR0RG2trYwm803vznRqKqqQmlpKSoqKnDmzBk888wzeOedd9CxY0d8/PHH8Pf3BwDodDp88803eOONN6DVagWOuuk8/fTTSE5Ohl6vx+nTp7Fs2TJcuHABq1atQmlpKRhj6Nmzp9BhNonKykosWbIEw4cPx9ixY2EymWo8XsxmM/Ly8kBEuPYMV3GYPHkynn/+eVRWVuLo0aOYOXMmtm7dCoPBACKC0WjE7t27727jDanIuP5t1KlTJ5o6dSqtW7eOioqKyGQyibLH5+XlRU8//TTNmzePunTpQu7u7jRlyhS6ePEimc1m0uv1FNJGE2YAACAASURBVB8fT5999hnZ2dmJJi83FolEQlFRUfTVV19RmzZtyMHBgZo1a0Zz5syh0tJSMplMdPHiRdHlpS6Lm5sbmc1m2rZtG9nb24siL+Hh4fTrr7/SmjVrqGPHjrf9XqlU0tGjR6miooK++OILkslkdc7LXY/qOjs7IyoqCg4ODpgyZQoCAgKgVqtvWcfJyeluN/9AyszMxC+//ILff/8dbm5umDhxIp544gm0atUKZrMZu3fvxuzZs3Hu3DmYTCahw21yZrMZO3bswK5du+Dt7Y3hw4cjMjIS3bp1g62tLQCIqkdzN/z9/REREYE///xT6FDuGYlEAk9PT6jVamzZsgWBgYHw9PSEr68v0tPTbxRN9O/fH8HBwcjOzsb69etrndZym7upyH379qUFCxaQ0Wi09O5uXLu5sZjNZjp48CB16tRJdN/gvXv3pu+++85yvS8zM5PWrVtHnTp1IolEUqdveKHf473Iy43F19eXli5dSnl5eZZ96MaSmZlJgwYNqrZXY4158fLyoqFDh9Lw4cNpyJAhFBgYWGuPLy0tjZ577jmrzYudnZ3l+ElKSqLMzEzS6/WUnJxMS5cupaCgIGKMUWhoKMXHx5PJZKLDhw/f+1HdZs2a0d69e6m0tJTMZjNptVratm0bffXVV5ZTuhuFz2g00sGDB2nAgAGiOMDlcjl5e3vTrl27LIMYV65coZkzZ1KLFi1IKpXW+dRG6PfYmHn594G+YsUKKi4uppycHDp+/DiVlJRYCt+Ni9hjxoyx6rxoNBqaNm0axcTE0JUrVyglJYUuXbpEe/fupQkTJtxyKUQikdDs2bPJbDZTQkIC9e/f3yrzYm9vT+PGjaOjR4+SwWCguLg4eu+992ju3Lm0d+9eKi4uppUrV9KwYcMoJiaG9Ho9HTx4kKKjo2s8tmpsu74f2AsvvEC5ubmWeUWvvPIK+fn50ciRI6m4uNgyYnej8BkMBvrzzz+t/gCXyWQ0Y8YMOnbsmGUqS0ZGBk2ePLnG3ovYCp+npyetWbOGioqK6OLFizRr1iyaPHkypaWlkclkoqKiIoqNjaWqqirat29ftWcLQr/HxspLhw4dLNNZbj5eTCYTJSUl0dKlS2n8+PGkUqmoXbt2lJCQQCaTif74449qrw8L/R4bmhd7e3t68803KTk5mbRaLe3YsYPCw8NJo9GQo6MjjRs3jnJzc6msrIzS0tJIr9fTkSNHqG/fviSXy+t9HNXrAwsODqa//vrL8gF5eXmRWq2miIgIOn78OJnNZiosLKRXX32VnnjiCctQc0JCAgUEBFjtAR4YGEhLliyhvLy826arhIWFUWBgIAUGBpJSqSRPT0/L/28sfn5+t3XVhX6PjZGXfy+bNm0inU5Hubm5NGPGDLK3t6dFixZZ5mhNmjSJgoKCaNu2baTVamn9+vXUsmVLq8zL6tWrqaqqinbs2EEtWrQgHx8f6tu3LyUnJ5PRaCStVktFRUWUlpZG2dnZVFZWRkuXLiUXFxerPI4effRROnPmDJnNZjp58iT5+fmRRCIhNzc3GjduHO3cuZO0Wq3l+Dp8+DD5+flVO6DR6IWvXbt2dOTIETKZTLRixQqSyWTk5eVFp06dIoPBQFVVVTRnzhxycHAgxhitWbOGTCYTabVa+vnnn6lFixZW94EBoI4dO9KJEyduuzOjqqqKdDod6XQ6qqiooJUrV1JsbCxptVrLz3U6HWVnZ9PMmTNv+SYX+j02Rl7+vZSWllJFRQUtXryYlEol9erVi+Li4shkMtHx48fJw8ODAFDz5s1p8eLFtHjx4tv2GaHfY2PkJTIy0tLb27RpE8lkMmKMka2tLYWHh9OsWbPo/PnzlrMnk8lEZWVltGDBAqs9Q5g0aZLlEllMTAzNmDGDzpw5Q5cvX7bcqWEwGCy941OnTpG7u3uNcx8btfCFhobSwYMHLddjpk6dSqdPn7b09ObOnUvOzs6W9WUy2S2F4PPPP7+tQgud/MbYkd977z3L6W1dln/T6/WUlZVFERERVpWXm5eQkBAqKyujgwcPkp2dHbVo0YK+/fZbqqyspJKSEnrppZduuU6jUChIrVZb3QEOXJumceO4iY2NpYkTJ9KECRNo/vz5t+0rN4qBVquljz/+2GoL3+DBgy0Tum9+/0ajkdLS0mjJkiXUr18/mjt3LuXk5JDJZKI9e/ZQ//79ydbWtt7X+Oo1nSUlJQVbtmyBt7c37Ozs8N///hdEhNzcXGzZsgWLFy9GYWHhLa9JSkqCm5sb7O3tERERgR49euDgwYP1G3q+z8XGxuLSpUvw9fUFAJSUlKC4uLjOrz9+/Dj27NmDS5cu3asQBRcdHQ0bGxsolUp07NgRTz31FIYPHw7GGL777jusXr36lik+VVVVqKqqEjDie+fIkSM4evQogoKC0L59eyxbtgyMMctBaTKZoNVqUVFRgZycHBAR9Ho9MjIyhA79ntm+fTsMBgOGDh2K9u3bQ6lUgoiQmZmJ5cuXY+vWrTCZTPj7779x9epVjB8/Ht27d8dPP/2EFStW4OjRo0hKSkJmZuZtNag6d/VYqrFjx8LV1dXyYRUWFuLPP/9Ednb2beuGhobi+eefx6uvvgoiwsqVK/Huu+8iNzcXAEBW8DgdT09PPP744/D09ARwbcf+559/6rw9g8Fw23P5rCEvN+vUqRMOHDgAGxsby8+Kiopw7NgxfPTRRzhy5Eid5jZaS17atWuHDz/8EM2aNQNjDGFhYbh06RLKy8tRUlKCU6dO4dSpU4iJiYFOp7tjG9aSF5lMBhsbG8t8TpPJdNv7l0qlUCqVWLhwIUJDQ+Hm5oaqqioYDAbMmzcPGzZssKxbU14a9Dy+uvLy8sIrr7wC4FpR2LVrF8rLy2sN7H7G/4ZC9e6Ul7KyMqhUKhgMBly+fBk7duzADz/8gLNnz9Z5Qre15EWlUsHHxwdyuRwymQxPPfUUDhw4gOzsbFRWViIzM7NetzNaS17qQy6Xo0WLFmjVqpXlZ2fPnkV6errl/4IWvtqI8QOrC2vMy/jx4yGXy2EwGJCSkoKkpCTk5OTU65TWGvPSGHheqscLXyPieakez0v1eF6qx//KGsdxXBPihY/jONGp9VSX4zjOGvEeH8dxosMLH8dxosMLH8dxosMLH8dxosMLH8dxosMLH8dxosMLH8dxosMLH8dxosMLH8dxosMLH8dxosMLH8dxosMLH8dxosMLH8dxosMLH8dxosMLH8dxosMLH8dxosMLH8dxosMLH9fkGGNrGWNZjLFSxlgiY2yC0DFx4sIfPc81OcZYGwBJRKRnjAUD+AvA40R0UtjIOLHgPT6uyRHROSLS3/jv9aWlgCFxIsMLHycIxthSxpgWwAUAWQC2CRwSJyL8VJcTDGNMCqAbgEcBfExEBmEj4sSC9/g4wRCRiYj+BuANYKrQ8XDiwQsfdz+QgV/j45oQL3xck2KMuTPGRjDGNIwxKWMsCsBIAHuFjo0TD36Nj2tSjDE3AL8BaI9rX7ypABYR0f8EDYwTFV74OI4THX6qy3Gc6PDCx3Gc6PDCx3Gc6PDCx3Gc6Mhq+yVj7J6PfBARu9dtNDael+rxvFSP56V6QuaF9/g4jhMdXvg4jhOdWk9160KpVKJ3794wmUw4evQoioqKGiMujuO4GjHGLItGo4FcLkdpaSmqqqrq9PpGKXzDhg1Dhw4dcPjwYezZsweHDx9GVlYWxDo5WqlUYsyYMXBzc8PWrVsRHx8Ps9ksdFiCk0gk6Nu3L7p06YLExES4u7tDJpMhNjYWhw8frvNOa60kEgm8vb3Rtm1beHp6ory8HIcPH0Z6errQoQlOqVSiQ4cOaNWqFdzd3aFSqeDg4ADGGOzt7VFZWYl169bh77//rtP2Glz4ysvLsXDhQoSGhmLQoEF47733UFBQgNjYWOzfvx+bN29uaBMPnJCQEMydOxcajQZ9+vTB+vXrsWHDBuTn5wsdmqAYY+jUqRNeeOEF7N69Gy1btkRQUBDy8vIQGxuLnTt3Yv369Vb/heno6AiTyYTy8nIA1/aXoKAgtG3bFl27dkVQUBCUSiUMBgMuXbqETZs2YdmyZQJHLSwnJyc899xzePLJJ6HRaCCTycAYg1KpBACUlJTg0qVLdS58td6yVp9RF6lUCldXV7i4uGDMmDGYMGECdDodoqKicP78+RpfZ42jUc2bN8dnn32GdevWYdq0aWjTpg1++uknfPzxx8jLy6tTG9aYF8YYAgIC4O3tjcuXL0Mmk6Fly5bo378/XnjhBVRWVuL555/H9u3ba9yGNeQlJCQEM2fOREVFBdRqNR555BGoVCro9XocP34cp06dwqlTp8AYw9tvvw0PDw906dIFJSUlNbZhDXmpTa9evbBy5UoYjUasXbsWV65cQUFBAb755ht4e3sjNjYW06ZNw+HDh295XY15IaIaF/z/Y8HrtSxdupQqKytp+fLlJJPJal23tvbv1+VO758xRhqNhqRSKbVr147++OMPysnJoejo6Dvmw5rzciM3EonE8n+JREIqlYpefvll0uv1tGvXLqvPi42NDU2ePJlOnDhBp06doq+++oqmTp1KwcHBZGdnRzY2NiSRSEgmk9GcOXNIq9VSRESE1eeltqVFixY0ZcoU6tevH3l4eNCkSZMoLi6ODAYDJSQkUK9evUgqldY5L40WGGOMXF1d6dNPP6X8/Hx6//33SaFQiPYAv3kZPHgwJSYm0t69e8nFxUXUha+mfWf8+PGWL0sx5IUxRlKplKRSKUkkErre+yHGGMlkMrKxsaHg4GDKzMykK1euWH5v7XmpaZFIJGRra0tdu3alnTt3UmVlJRUXF9Mbb7xB7u7u9c5Lg6/xAYBKpUKnTp3w9ttvQ6PR4Pnnn8eff/4Jg4E/SRwALl26hOzsbAQHB0OhUAgdzn1FLpejffv2GDFiBHQ6Hb788kuhQ2oSRASTyWT5v52dHdzc3BASEoLOnTtj4MCBaN26NTQaDd58802o1WrLNUExYYzByckJXbp0waRJk9C7d2/Y29sjJSUFo0ePxqFDh24U0XppUOGzt7eHv78/oqOjMWLECJw8eRJvvfUWUlNT4eDggLKyMuj1+jtvyIrJZDI89NBD8PT0REZGBoxGo9Ah3Tc0Gg0iIyMxa9YstG7dGosXL8aFCxeEDqvJSaVSTJgwAc8//zyaN2+O/Px8lJaW4ty5c7C3t8cLL7wAV1dXfPrpp3W+RmwtAgIC8Pbbb2Pw4MFwcXGByWSCXq+HSqXCk08+iby8PFy8eLHe273rwteuXTsMHToUQ4YMQWhoKE6cOIELFy6gV69eAK5d4L9y5QqOHTuGI0eO3G0zDzSlUonw8HCMGTMGtra2mD9/PkpLS4UOS3BqtRpt27bFo48+ij59+gAA5s+fj++++07gyIRBREhLS8Off/6JnJwcHD16FImJiSgrK0Pr1q3x7LPP4plnnkFJSQm++uor0fT8JBIJunfvjieeeAK5ubk4duwY8vPzUVRUhJYtW2Ly5Mnw8PDAtGnTah34qc5dj+qOHDkSTzzxBK5evYr8/HxkZGRAp9NZft+qVSsMHDgQCoUCr7zySo3Fj6x0NMrLywvDhw/Hs88+i/bt2+PXX3/Fm2++iYyMjDq1Ya15UalUeOGFFzB27Fio1Wps3LgRv/32G86cOVOn3rC15qU2tra2ePXVVzFlyhS88847WLt27W2nd9aYF8YY2rRpg27duiE1NRUJCQkoKiqCVqtFs2bN8Mcff6B58+aIjo6ucRpLjXm524uPzs7O5OPjQ7a2tjWu07FjRzp37hx98MEHorko27JlS5o7dy5t2LCB8vLyyGw2k8FgoB07dlB4eHi1I09iyAuuX7jv2LEjpaenU0ZGBs2ePbvWC9NiyUtdFm9vb0pMTKTNmzdXO0Am9Hts6ry4uLjQ5cuXqbi4mB599NF67y93fa9uYWEh0tPTb+nl/dvp06dRWFgIW1vbu23mgTNx4kRMnz4dnTp1wq5du/Diiy9i9+7dCA8PxxdffAEvLy+hQxSUXC5HdnY25HI5xowZg9mzZ8Pb21vosO57+fn5WL9+PXx8fODv7y90OIJ7/PHH0bx5c8THxyMhIaH+G7iX31SLFy+m/Px8Gjx4sGi+wUeMGEFDhw6lkJAQcnR0JIVCQSEhIRQTE0OVlZX0zDPP3DKPTSx5ubEoFAry9PSkfv36UUxMDBUWFtIHH3xAarWa9/jukLcZM2bQoUOHKCgoyKry4uHhQRMnTqQBAwaQSqW6Yy5CQkIoPj6ejEYjvfvuu7VO9amx7XvxgYWGhtLmzZupoqKCPvzww1oPdKGT39g7skQiqfb9jho1ijIyMmjz5s0kl8tFf4Azxujpp5+mlJQUWrRoEdnZ2fHCV8vi5OREZ86coS1btlQ7P1bo99iQvERERFBWVhalpaXRkCFDapzkzxijsLAw2rNnDxkMBjp//vwdL5XU1HajzOMDrl2ItLGxQZs2bfDRRx/By8sLzz33HDZu3NhYTTwQanoYQWpqKioqKiz3FoodEaGsrAw6nQ5Go/HGgSAqcrkcAOo039XFxQX29vbYuXOn1c2PNZlMMBgM8Pb2xn//+19IpVLExMTAaDRCKpVCLpfD1dUV0dHRePXVV+Ht7Y34+HiMGTMGubm5d9Vmgwufk5MT1Go13N3d0bNnT0RGRmLHjh34/fffkZqa2tDNWwWNRoPw8HDY2dkhJSVFlAf5v9nZ2aFbt26QyWQ4evSoqOZ7KhQKeHt745FHHrnjjfWMMajVarzyyivIzc3F9u3brW7/0Wq1yMrKglqtxqFDh1BeXo42bdoAAFq0aIE2bdrgySefxEMPPYTs7GysWrUKCxYsQFpa2l23WafpLDKZDE5OTlAoFNBoNNBoNGjevDmkUikiIiIgl8tRXFyMsrIy7Nq1C+fOnavzRF2ywmH4m6lUKkRHR2POnDmQy+Xo379/nSZcWlNeZDIZFAoFtFotFAoFfHx8EB0djfHjx2PLli345JNP6vwcR2vIS2BgIJYvX46IiAjMnj0bGzduRFZW1m2P5boxnWPgwIGYPHky3nnnHfz888/VnlU8yHmxsbFBVFQUXn75ZTz88MPQaDQ3r4OioiJkZmZiz5492LlzJw4fPoyKioo6tVFTXurU4/Py8sLEiRPh5+cHNzc3MMbg4+OD7OxsxMfHY+fOndi3bx+0Wm2dghELuVyORx55BK+88gpsbW2xYsUKXL16VeiwmpxcLsfYsWNhNpshkUgQFRWFHj16YP/+/fjtt99EN6m7rKwMZ8+eRUBAAF588UV0794d8fHxKCsrg1arRVFREZydnaHRaNCzZ0+0bdsWc+fOxW+//WaVz3XU6/XYsWMHkpOT0atXL/j4+MDZ2Rnu7u5ITk5GfHw84uLicOrUqUZ7/3Xq8bm4uKBnz55wd3cHABQUFKCkpARZWVlITk5uUMF7kL+paiOVShEeHo758+fD398fK1aswLJly1BQUFCnNqwpLxKJBN988w2GDx8OrVaLwsJC/P777/jtt99w9uzZW+5ZvRNryAtjDL6+vggJCYGXlxd69+4Nb29vuLi4wNfXF8XFxXB0dMSVK1ewefNmHDt2DFu3bq31FNca8nLTz2Fvbw8XFxdkZGQ06DJITXlptOfx3S1r+sD+tQ48PDwQHByMyspKJCYmorCwsM5tWFteWrZsCT8/P5jNZhQVFeHKlSv1vs0IsL68yGQyeHp6QqPRQKVSwdnZ2fK7/Px8JCcn16lHbG15aSy88DUinpfq8bxUj+elevzPS3IcxzWhWnt8HMdx1oj3+DiOEx1e+DiOEx1e+DiOEx1e+DiOEx1e+DiOEx1e+DiOEx1e+DiOEx1e+DiOEx1e+DiOEx1e+DiOEx1e+DiOEx1e+DiOEx1e+DiOEx1e+DiOEx1e+DiOEx1e+DiOEx1e+DjBMMZaMcYqGWNrhY6FExde+DghLQFwXOggOPHhhY8TBGNsBIBiAHuEjoUTH174uCbHGLMH8CGAmULHwokTL3ycEOYCWEVE6UIHwomTTOgAOHFhjIUB6APgYaFj4cSLFz6uqT0KwA9AGmMMADQApIyxECLqIGBcnIjwv6vLNSnGmAqA/U0/eh3XCuFUIsoTJChOdHiPj2tSRKQFoL3xf8ZYOYBKXvS4psR7fBzHiQ4f1eU4TnR44eM4TnR44eM4TnR44eM4TnRqHdVljN3zkQ8iYve6jcbG81I9npfq8bxUT8i88B4fx3Giwwsfx3Giwwsfx3Gi0yiFT6lU4pFHHsHrr7+Ojh07QiLh9bQ27u7uePPNN9G9e3ehQ+EEpFarcf1+5RqFh4ejU6dOTRSReDS4QsnlckRFRWHhwoUYNmwYNBoN+N0gtZs2bRpef/11tG/fXuhQ7inGGMaMGYNFixahefPmta6rVqshl8ubKDLhubi44MMPP7zjPvDoo49i0aJFCAkJaaLI7g8qlQrz58/Ht99+C0dHxxrXCw8Px8qVK9GnT596bb/Bhc/NzQ0zZ85EaGgodu3ahWPHjvHCVwtbW1uMHj0aNjY2Vp8nW1tbdO/eHWPHjsWsWbMglUqrXc/GxgabN2/GRx99BBsbmyaOUhg+Pj54/PHH8cwzz9R6hqTX6xEYGIji4uImjE54w4YNw4svvoi+ffvWuE9oNBr897//xVNPPYXOnTvXa/sNKnwKhQIrVqxAly5dsG/fPvzwww/Q6XSW3zs5OWHmzJlQqVQNaeaBZGNjg7CwsNt+/txzz8HLywsGgwFHjhwRILKmo9PpUFBQAIlEgokTJyI6Orra9SQSCYKCgvDSSy/hzTffbOIohSGTySCXy+Hg4FDrejY2NigoKICrq2sTRSY8BwcHTJ06FSqVCmfOnEF5eXm1682aNQs9e/YEAJhMpvo1QkQ1LgCopoUxRlOmTKHKykq6cuUKDR069JbfOzs70549e0ir1dLSpUtJo9FUu53a2r9fl9ryAoCkUilNnjyZsrOz6dVXX70lJ0ePHiWj0UgrV64kiURS4zaEfo+NlZeXXnqJ8vPzyWw205kzZ0gqld72vgcPHkwFBQVkMplow4YNteZW6PfYWHkJDg6m8+fPU3p6Osnl8hrfb2xsLP3www+17ivWlBcANHv2bCorK6Pk5GRydXWtdp2RI0dSWVkZGY1GOnjwIDk6OtYrL3fV45NIJOjevTs+//xz6HQ6LFq0CJs2bbplnZCQEERERECn08HZ2RmjRo2Cra3t3TT3wPH398e8efNgZ2cHLy8vy7Wrtm3bIiAgACaTCWvWrIHZbBY40ntv//79yM/PBwC0atUKU6dOxZAhQ+Dp6QknJyfY2dmhqqpKFLm42YULF3Dx4kW4urriscceq3E9xhieeuopPPywOB5Y7ePjgxEjRqCqqgqTJk2y7Ds3k0gkePrpp2Fra4uysjKsWLGi3pcC7up5fCEhIVi+fDkUCgU2b96MtWvX3tLVVKlUGDx4MBhjWLVqFRYuXIiXX34Zzz77LH766adbToet0ZQpU+Ds7IyCggLk5OTAaDRCKpVi4MCBUKlUKC4uRkZGBhQKBaqqqoQO9566cOECduzYgebNm0OtVmPRokXQ6/VIT09HfHw8MjMzERwcDDs7O6FDbXJLly5F27ZtMWrUKBw+fBhlZWXVrkdE0Ov1TRydMF566SUEBATg1KlTuHr1KlQqFSorK2/5Ynz44YfRtm1bANf2r40bN9a/ofp2RdVqNS1fvpwqKyvp2LFj5Ovre9s6vXr1opSUFDIYDBQdHU2hoaH0+eef019//UUdOnSw2i46AAoKCqILFy6Q2WymU6dOUZs2bYgxRqGhoRQfH09ms5mysrJo5syZNHLkSHJycrL6UxcvLy/auHEj6fV6MpvNNS5iOtXF9ctFkydPptLSUnrllVdIrVbf8ns/Pz9KTEykvLw8cnZ2FkVeZsyYQQUFBVRZWUmHDx+mWbNm0ciRIykiIoKCg4PJ3d2dfvzxR6qsrKTi4mIaP378XeWl3j2+yMhI9O3bF2azGQcPHoREIoGjoyMMBgNMJhMcHR0xcuRIeHh4gDGGtm3bYsqUKejcuTMOHToErVZ750YeYBMnTkTLli0t/+/YsSOGDRuGhx9+GC1atABwbSR83rx5yMzMBACsW7dOkFibSmZmJt59912kpaWhV69e8PLygkqlgkKhgMlkgtFohFKpFDrMJkdE+P777zFgwAC88cYb0Gq1WLNmDQwGA7p27YqpU6eiefPmOH36NAoLC4UOt0nExMSgZ8+eGDBgALp27YquXbuioqICKSkpyMzMRFZWFh577DHI5XKUlpZCpVLhpZdegtlsRnp6Ov7+++86nfbW+gTm6m4ijoyMxHfffQdfX19cvXoVSUlJyMnJgU6nQ1VVFXx9fREREQFnZ2cAQHl5OZRKJY4cOYI5c+bgwIEDt5wWkxXdXG1jY4M//vjDcs1Gp9OhpKQE7u7u1U5ZSE1NxfTp0xETE3Pb76wpLzc4OTkhJCQE7u7uCA4Oho+PD7Kzs+Hj44Nnn30WSqUSmzZtwtChQ2vchjXmpU2bNli8eDF8fHwQGxsLk8mEoKAghISEwMbGBuvWrcOoUaNqbcOa8hIaGooePXqgW7du6NevH9zc3Kp9PRGhtLQUpaWlSE9Px48//ogNGzYgLy/v5nWqzUu9C5+NjQ2efPJJPP7443j88cfh6OgIk8l0yze3VCqFyWRCWVkZjh07hu+//x5nzpxBUlLSbde0rOkDc3d3x+HDhxEQEFDja3U6HZYsWYItW7ZAq9Xi4sWLqKiouG09a8pLNetBrVbD1tYWlZWVGDVqFBYsWAA7OztRFj7GGFq3bo3Ro0dj/PjxUKvVOHbsGPLz8zFo0CBcunQJHTrU/gforC0vCoUCLi4uaN26Nfz8/NCsWTMMGDAADz/8MFQqFU6ePIm9e/ciISEBubm5NBI8TQAAIABJREFUyMnJwZUrV1BaWlqnjlW9T3X1ej1+//13bN++HW+//Tb69esHhUIB4Nqgx/Dhw+Hi4oKFCxdi2bJlKC8vR2lpKYxGY32beuDk5+djzpw5GDZsmOVWpBs7dVBQEHQ6HWbOnImffvoJpaWlAkcrHCJCeXm5ZX5WvedgWRkiwoULF/DJJ59gy5YtICJcuXIFL7/8MgBY/XzP6lRVVSErKws5OTk4dOgQ1Go1pFKpZVBj6dKl2LBhA/R6PTw8PBAYGIiioqK6N3A3F2VvXiQSCUkkErK1taWPP/6YzGYz/frrrzXOv/n3IvQF1sa8WH0jH3K53LIolUqaN28emc1mSk5OJi8vL1HmpbZl4sSJVFxcLLrBjeoWxhhd7wlRz549KTc3lxYuXCj6/SUoKIhiYmLIZDLR0aNHKTAw0JIvLy8v6tKly70d3Pg3s9lsGcTo2bMnDAYDdu/eXePQvLUzm823DL23bt0aXbt2BQD8/fff1Z7WctwN1wuCZf6eRqPBiRMnBI5KeLa2trC3v/bnmJcuXYrU1FQA1/KVmZlpGSisq0Z5jIqDgwOGDh2Kjh07YseOHdi1a5do5h3VRiqVon379ujYsSMqKytx9uxZq5+3d7fu9JQSsZHL5ejatStsbGyQlJQkdDiCkkgkCA4ORmhoKADg8uXLMBgMDdtmYwTm6uqKzp07Iy8vD2vXrkVaWlpjbPaBZ29vjw4dOsDe3h4JCQnYsmWL1U/evht6vV50d25wdefq6opevXrBwcEBWq0Wjo6Od7zH+U4aXPgkEgl8fHwQEhKC5ORk2Nraok2bNpDJGnwW/cCzt7dHUFAQACA5ObnGm63FLi4ujp8hcDVydHRE+/btwRgDEaF58+a1PqqqLhpc+IgIBoMBBoMBLVu2xOuvv45evXqJ6tlqNWGMQSqVIisrC9u3b0dBQYHQId2XLly4gKtXr1qub3HXRrrXr1+PuLg4UX9hMsbg6uoKf39/AEBubq7lVseGaHC3jIgQGxuL4cOHQ61WAwCSkpL4NziArKwsvPHGG9D8H3v3HR9FtfYB/He2b+qGJKSTAAEChBCKEgmCiDQRIUpVAaPCFZV7RUXItSDyWrhYEERUlPoKlw6CVOnSQigBQgppJIT0uqRsmX3eP5B9jSQhJCEDO+f7+cxHye7OPPvs7DNnzjkz6+CAxMREfppbC6PRiBkzZuDHH38UO5T7hiAIWLZsGQ4ePGjtyJcqmUwGpVKJ0tJSfPvtt7hw4UKj+/juegJzUyMbm3jZVKSWF7lcDh8fH5hMJmRnZ9f6PKnlpb5sOS8ajQaenp4AgPz8/LuaGVFbXnjhawCel5rxvNSM56VmYualzsLHcRxni/jPoXEcJzm88HEcJzm88HEcJzm88HEcJzm88HEcJzm88HEcJzm88HEcJzm88HEcJzm88HEcJzm88HEcJzm88HEcJzm88HEcJzm88HEcJzm88HEcJzm88HEcJzm88HEcJzm88HEcJzm88HHNjjF2iDFWxRi78eeSKHZMnLTwwseJ5Q0icvhz6SB2MJy08MLHcZzk8MLHieUzxlgBY+wYY+wxsYPhpIX/yhrX7BhjvQBcBmAEMA7AtwBCiShF1MA4yeCFjxMdY2w3gN+IaJHYsXDSwE91ufsBAXjgfhCbe3Dxwsc1K8aYjjE2mDGmYYwpGGPPA+gLYI/YsXHSoRA7AE5ylAD+B0AQAAFAAoCRRMTn8nHNhvfxcRwnOfxUl+M4yeGFj+M4yeGFj+M4yeGFj+M4yalzVJcxds9HPojogZu/xfNSM56XmvG81EzMvPAWH8dxksMLH8dxksMnMHPcA8DV1RWtW7eGr68vrly5gri4OLFDeqDxwtdM3N3dYW9vD2dnZ7Ro0QLBwcHQarX4/vvvUVZWJnZ43H1ILpejRYsWGDBgAB5//HF07twZbm5uWLBggWQKn1wuR1BQEEJCQlBRUYHY2Fhcu3YNZrO5Uevlha+J6XQ6vPXWW8jLy4ODgwN8fHwAAP7+/nB2doZWq4VWq4W3tzeKioqwc+dOXLp0SeSom8eCBQuwZ88eHDp0CN7e3sjNzcWNGzfEDuu+IZfL4eXlBW9vbwQFBSE4OBghISEICgqCl5cXioqKsHXrVpw9e1bsUJsFYwyPP/445s6dCy8vL5hMJqSlpeHSpUvYuHEjoqOjYTKZGrbuui5Za+yoS3h4OEaNGgU7OzsAwIEDB7Bx40YIgmB9jq2NRv38888YNWoUzGYzZDIZ1Go1AEClUkEmk+FWvmNiYnD8+HHMmTMHJSUlt63H1vICACUlJSguLkZaWho8PDyQn5+PxMREXL58GVVVVQAArVaLQYMGobCwEFeuXMGBAwdw4sQJ6z5ji3kBbu4fw4YNw2uvvYa2bdvCzs4O9vb2qKqqwpkzZ7B7924cOXIEGRkZKC0tve0Lb6t5CQwMxI4dO9CuXTsAABHBYDAgLy8Pf/zxB7799lucOnWq1tfXlpd7Uvi6du2KcePGYfLkyXBwcABjN7ddVlaG9PR0vPrqqzhz5kydgd3P6srLxYsX0bFjR+Tl5aGgoABxcXFISkrCsWPH4OPjg+zsbKSlpaGwsBAVFRWoqKiocT22lpewsDAcPHgQBw4cQEFBAW7cuIHWrVujS5cu0Ol01Z6r1WphsVhQVlaGqKgoLF++3HpqY2t5AW629KZNm4aZM2fCzc0NMtnNMcctW7bg888/x5UrV1BZWQmTyYTavq+2mBcAsLOzw/Tp0zF58mTMmzcPBw4cgE6nw+TJkzFixAhUVFTg/fffx6ZNm2r8LtWaFyKqdcHN+6TdcZHJZKRQKCgoKIi+/vpryszMpKNHj9L+/fvp66+/prlz59LUqVOpffv2pFarq722ru3fr0tdubh8+TKVl5eTt7c3yeVykslkJJPJCAD9+UHXaxH7PTZ1XkaNGkV6vZ4CAwOtOVEoFKRWq0mj0VBAQABFRkbS+PHjSaPRkEajIbVabc2dreYFAGk0Gtq4cSNZLBbKycmhVatWUc+ePUmhUNT5uhYtWtA777xDarXaJvNy6z1u27aNDh48SB06dKhWczp37kxLliyhpKQkWrt2bbXH77S/NLqPz83NDf369UPfvn0RERGB0tJSfPDBB9i2bRuKi4sbu/oHjslkglwux6OPPop169ZVe+zPD1vSiAh2dnbQarUAgKqqKpSXlyM9PR3Lly8XObrmxxjDpEmTMHToUKSmpuK9997Dhg0bYLFY7vhaOzs7ODo6Ws+obBFjzLqv/PX7Y7FYEBcXh6lTp0Kn02HixIkYPHgwioqKkJ+ff8f1NqrwMcbQvn17fPvtt3BycsKvv/6KxYsX4/Tp0zAYDI1Z9QNr165daNeunfVDUqvV8PDwsPZzGo1GZGdno7KyUswwm51CoYBcLkffvn0RHByM0NBQADe7BjZv3ozo6GhrP5+UdO7cGXPmzEF5eTmWLFly28GyLnq9Hlu2bGlwB/+DQKvVIjw8HCdOnKj1YFBSUoKFCxfe1Xob3eITBAGXL1+Gv78/4uLiEB8fL9miBwALFy7EhAkTYG9vj6CgIISHh+Opp56Cn58fiAjFxcVYtmwZtmzZIqk8dejQASqVCrNnz8bFixcRGxsLk8mEkJAQfP/991i0aBH+93//F3q9XuxQm1VUVBRcXV1x/PhxbNq06a5eW1paivPnz9+jyO4PMpkMSqUS2dnZKCwsbLL1NqrwERFOnz6Nf/7znxg7diy6deuGkSNHYuPGjSgtLW2qGB8oZrMZgiBg/PjxGDNmDHr27Iny8nKcOXMGpaWlGDx4MD744AM4ODjgp59+EjvcZqHRaPDEE09AEATMmzcP27dvR25uLiwWCwIDA/Hyyy/jww8/REBAAD7//HNJdZFcv34dVVVV8PPzw8yZM3H16lXExsbi5MmTkspDbVxcXKDX63H27NmmzUdjOx9vLc7OztS3b19atGgRvfLKK6TRaCTZie/o6EgnT54kQRAoMzOTli1bRkOGDKGAgABq2bIljR8/nmJiYig5OZn69esniby4u7tTVlYWnTp1ihwdHW973M3NjV5//XUqLCyk119/XVL7i5+fH0VFRVFCQgKZzWYyGAyUkJBAS5cupUGDBpFSqZTk9wh/DmCMHj2a9Ho9/fDDDxQYGEgqlareNamuvNxVYEFBQRQQEHDbSNutRS6XU//+/enQoUPk5eUl2Q+sZ8+e9Pnnn1OfPn3I09Oz2mMqlYqmTZtGer2eFi1aVOtIr9jvsSnzotPpaOnSpRQWFlZrzm6NbCYlJVFwcLCk9hdHR0cKDQ2lqVOn0ldffUXJyclUUVFBKSkpNHjw4Fr3EbVabT2QiP0e70VeAFBgYCAdPXqU8vLy6NSpUzRjxgxycXFp3sL36quv0ueff37bl/mvy4wZM+j8+fN1PsfWd2SZTEb29va17rAODg60ZMkSOn/+PHl4eNh8Xhhj5OzsXOsB89YyevRoKisro/Hjx0tqf/l7IQsICKAvvviCSkpKaNeuXbW2+nx8fKhHjx4kl8ttNi8ymYx8fHzo2Wefpb1791JxcTEtXLiwxjOHu9lf7qqP79y5c3jppZcwaNAg7N+/H6mpqbh48SJkMhnc3d0RFBSEYcOGWUcwpcpisaC8vBzAzZHvPz9kq/LyciQmJqJfv35QKGz/qkEiqlef744dO5CamornnnsOW7Zskdwor8FggMFggF6vx/LlyzF48GA88sgjaNWqFVJSUm57vkwmw4ABA2zyksdb3xuLxYKsrCxs2bIFycnJ+PLLL/Hyyy8jPj4eP/zwQ72m/dTkrr51ycnJ2LBhA6ZOnYpp06ZBqVSCiCAIAiorK2GxWHDy5ElMnjwZubm5DQrIlnh6eqJv3764fPlytZ1TLpdDqVTC0dER3bp1Q1ZWlohRNo+aDgB/ZzAYEBcXh5CQEGg0GskVvr/y8/ODvb094uPjkZmZWevz/nqlhy0ZNGgQjhw5Yp32ZbFYEBsbi2+++QaBgYEIDg6GTCZrnsJXWFiI+fPnY/v27XjnnXfQpk0bADcvRTt8+DCio6MRGxvLLzzHzcmlkydPxvPPP4/p06fj0qVLUCgU0Ol06N69O4YNGwa9Xo/09HSxQ73n7OzsEBISgrNnz8JoNNb53CtXriAkJKSZIrs/ubq6Yty4cfD29sZvv/1W65fbYrFAEIQ7HlAeRM899xyysrKqNRgYY7C3t4dcLm9wwbulQedZCQkJeOWVVxq1YVunUCjg6ekJhUKBNm3aoE+fPvD397de4SKXy/Hdd9/VeApjazQaDb766issXboUv/76a63zsRhj6NatG0pKShp926EHgY+PDzp37gyDwYD8/HzIZDLodDoMHz4cQ4cORWZmJjZv3lztph5/dWuaR22PP8gKCgoQFRWFF1980TpB28/PD0OHDoVWq0VSUlLjCn5TdMo2ZhG7g/Vedco6OTnR4sWLyWQyUVZWFiUlJVFFRQUJgkD5+fn04Ycf1tlBK/Z7bMq8yOVyioiIoKNHj9Knn35KAQEBtz3H1dWVRo4cSUlJSfSvf/2r1oEQsd9jU+blhRdeoJycHMrIyKADBw7Q4cOHKTExkQwGA6WmplJkZORt17bXlFtbywsAGjhwIF2+fJmmT59Ow4YNo6FDh9LixYupoKCAzpw5Qz4+Po2qL/f0tlT1QTZ6Vwm1Wo2xY8firbfeQps2bVBeXo7z58/j+vXriI6Oxi+//FJnl4At5qVPnz4YO3YsfHx8EBsbi9TUVJSUlCAwMBChoaEIDw9HdnY2nnnmmVqvt7SlvHTo0AG9e/eGUqm87bHMzMy7msRsS3kBAEdHR4wcORLjxo1D+/btIQgCWrZsiYSEBCxatAibN2+u15VPteWFF74GqG9enJ2dERAQAGdnZxiNRuvtmIqLi+/4odlqXgICAvDmm28iIiICMpkMRqMRLi4uyMzMxOHDh7Fq1SrExMTU+npbzUtj2WJe5HI5AgMD4evrC+Bmn2Zubi5SUlLqfbknL3xNiOelZvXJC2MMTk5OcHFxqTaVp6qqCnq9HmVlZahrn7TVvDQWz0vNeOFrQjwvNeN5qRnPS83u28LHcRxni2xv5iPHcdwd8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHiYIxNo4xFs8YK2eMpTDGHhU7Jk46GvS7uhzXGIyxgQDmARgLIBqAl7gRcVLDbz3PNTvG2HEAPxPRz2LHwkkTP9XlmhVjTA6gJwB3xlgyY+waY+xbxphW7Ng46eCFj2tuHgCUAEYBeBRAKIBuAN4XMyhOWnjh45pb5Z//XURE2URUAOArAE+KGBMnMbzwcc2KiIoBXAPAO5c50fDCx4lhOYBpjLGWjDEXAG8C2CFyTJyE8OksnBjmAnADkASgCsB6AJ+IGhEnKXw6C8dxksNPdTmOkxxe+DiOkxxe+DiOkxxe+DiOkxxe+DiOk5w6p7Mwxu75kC8RsXu9jabG81Iznpea8bzUTMy83LN5fJ06dYK3tzeuXbuGhISEe7WZB4q7uzv8/f2h0Whw8uRJmM1msUO6Lzg6OqJHjx5ITk7GtWvXxA7nvuPu7o62bdsiIyMD169fFzscUTHG4OXlBW9vbxiNRly4cKFhKyKiWhfcvKzorhalUkldu3al/fv3k8FgoE8//bTO59e1/ft1aUhePD096eOPP6bjx4/TmjVryMHBgecFIFdXV/rXv/5FpaWlNGPGjDs+X+z32Fx5ubW4ubnRxx9/TKmpqTR58mRJ50WpVFKfPn3op59+oq1bt9KaNWsavL80aYtPqVQiIiICr7zyCsLDw8EYQ3p6elNu4oGk1WoxceJETJo0CRs3bsSSJUtQUVEhdliic3Nzw0svvYRp06bhypUrDT9626hOnTph6tSpGDNmDAoLCxEbGyt2SKKRyWTo27cv3n//fZSVleHnn3+GyWRq+AqbsiL37NmTDh8+TEajkUpLS2nOnDkUFBQk+SP4wIED6cKFC7Rx40Zq27ZtvV4j9nu813lxdnamt99+m65evUomk4kGDx5MCoVC8nm5tQQHB9OmTZuooqKCzpw5Q+PGjSOZTCbJvDDGKDg4mH799Vfavn079ezZs177Sl15abIPrHv37vT7779TZWUlWSwWevHFF8nJyYnkcrmkd2SNRkOffPIJ5ebm0ujRo0mpVPLCB1BgYCBt3bqV9Ho9ffTRR2Rvb8/z8ufSqlUrWrt2LVVWVlJycjINGzaM1Gq1ZPPSsmVLWrx4MZ07d46GDh1a76J3zwufl5cXrVy5koxGIwmCQGvWrCFXV1fJ78iMMRowYAAlJSXRvHnzqEWLFo3+wO7n5W4OBq+++ioVFRXR9evXqWfPnjwvfy7u7u40Z84cyszMpP/+978UFBQk6ZawTCaj/v3709WrV+nLL78kJyenuzqI1LbtRs/js7Ozw7hx4zBixAjIZDKsX78e7777LgoLCxu76geeVqvFQw89BJ1Oh5iYGJSWlood0n2hTZs2eO655+Ds7Izff/8dWVlZYod0X1Cr1Xj66afx2muvIT09HbNnz0ZCQoKkR/8dHBwwcuRIFBYWYsuWLSgrK2uaFTemIms0GhozZgwJgkBVVVV09OjRuzp6w4aPVAAoKCiIjhw5QosXLyYXFxeeF4AUCoV1nzEYDDR79uy7OoqL/R7vVV5kMhn169ePcnJyqKCggP7xj3+QRqORdF4YYxQWFkbp6ek0f/78encT1ScvDW7xaTQajB8/Hj/++COICGfOnMGMGTMQExPT0FXaFKVSiV69eqFt27aIiYlBcXGx2CHdFxhj0Gg01hH/Q4cONd1R/AHm4OCA5557DjqdDr/99ht27dqFqqoqscMSlUwmw6OPPgoiwoEDB6yjuPb29nBxcQFjDZ+z3eDpLGFhYfjyyy9hb2+PpKQkLF26FGfPnm1wILZGq9WiV69eiI+Px7Fjx8QO576hUCjg6ekJIoJWq0VwcDAuXLgg6QODQqHAY489hmeffRZlZWVISUmBg4MDgoKCkJqaCqPRKHaIonB0dMTIkSNx7do1xMfHw9/fHx07dkRoaCjc3Nzwxx9/YMeOHQ3qCmhQ4dNoNHjxxRfh7OyM4uJifPzxx9i8eXO1Dyg4OBhqtRoAYDAYkJGRIakju52dHTp27IiCggIUFRWJHc59QxAElJSUAABatmyJN998E25ubvjiiy9QXl4ucnTiGDhwID755BO0aNECRqMRL7zwAsLDw2GxWPDbb79h3bp1yM/PFzvMZufp6YkuXbrgv//9L3x9fTFlyhSEhYWhvLwcOTk5ePbZZ+Hi4oLly5ff9bobVPgeeeQRDBw4EBaLBZs3b8amTZuszVB3d3c8++yzGDduHDQaDYCbhW/fvn347rvvJFMEPD090blzZ6xbtw6CIKBXr15wc3PD6dOnkZeXJ3Z4opHL5XB2dgZjDGVlZXBzc8O0adNQVFSE7777DoIgiB1is3vmmWfQqVMnZGVlYdOmTWjbti2Cg4Ph5eWFHj16wNHREZ999pnYYTY7V1dX6/7y0UcfQafTYcmSJYiOjkZBQQHeeOMNTJo0qUGFr0GdjzNmzCC9Xk85OTkUEhJi/XtoaCgtW7aMsrKyyGw2kyAIZLFYyGw2U3R0dI0DH2J3sN7LTtnMzEyaNGkSvfzyy3T48GFavXo1rV27ljw8PCTZWQ3cnLi8bNkyEgSBfvnlF4qMjKTk5GRKTU0lPz8/SeZlzZo1ZDabad++fdSiRQsKCAig7t2707Rp0yg9PZ2ys7PrvFzNVvPy+uuvk9lspsLCQrpy5QpFRkZWm+85fPhwSkxMJJ1O1zyDG2q1GjKZDOnp6YiLiwNw8/Kajz/+GOPHj4ednR3Wrl2LN954A+np6RAEAcnJyUhOTm7I5h5IWq0WeXl5uHHjBiZNmoTt27fjyy+/RHBwMHr37i12eKK5NbgBAPn5+Th+/DhKS0vRqlUrvPTSSyJHJ54bN25g+/btKCoqQnp6Os6ePYulS5ciJSUFbm5uePJJ6f3ssFwuh0wmg1KpxNatW7Fx48Zq3SEeHh5wcHCAnZ3dXa+7wYMbFosFgiBAEAS0aNECL730Eh566CEsWrQI69evR5cuXTB58mS4u7vjxx9/xBdffGHt27F1crkcw4cPh8lkgiAIuHz5MrZv344rV67gzJkzUCqVYocoGqVSCT8/v9v+zhhDt27dRIhIXGq1Gn5+fjAajUhKSqr2mMFggMVigcViwd69e0WKUDxHjhwBAKSnp2PFihXQ6/XWxxhjmDJlClJTUxt0x5oGFb7c3FxYLBa0bt0affr0QWZm5s2VKRQwmUyYOXMmnnjiCQiCgE8++QSff/55QzbzwLJYLLhw4QIeffRRVFVVITMzE4wxTJ06FUOGDMH8+fPFDlF0jDE89NBD6NKlC4KCgmAymXDo0CGxw2p2jo6OeOihh5CXl4fo6GgA/98qnjhxIrp3747KykqkpKSIHGnzu9VwqKqqQklJCeRyORQKBRwdHREREQE3Nze8+OKLDVt5Q87BPTw86MKFCyQIAuXl5VFiYiIVFRWRIAhkNBrp3LlzNHfuXOrdu/cdJ6eK3c9wL/omGGPUq1cvyszMpA8//JAmTJhACxYsoIyMDDpz5gypVCpJ9mUBIEdHR/r6669Jr9eTIAhkNpupsrKSFi9eXK8Ju2K/x6bOi5OTE6WmplJOTg7179+f2rZtS2FhYRQVFUUZGRl09epVmjlzpuTygj/7g6Ojo0mv19PKlStpxIgR9O6779Lp06cpLS2NHnvssQbnpUGB2dvb06effkrHjx+nwsJCMplMlJeXR/Hx8bRt2zaaPHkyabXaOwZlqx8YAAoICKCVK1dScnIyZWZmUnJyMq1YsYL8/f0lnRfg5g0KNmzYQMnJyXT69Glavnw5derUSbJ5+eabb6i8vJwyMzMpNjaWMjMzqbCwkHbu3EmDBw++440+bDUvMpmMJk6cSFlZWVRZWUlXr16l06dP03//+18aMmRIo/JS5w+K13VraDs7O7Rq1QpPPPEEvLy8kJGRYR3suH79OiwWS63r/Suy4Vtmt2jRAv369YOTkxOKi4tx8uRJ5Ofno66c32LLeVGr1QgNDYW3tzeys7Nx8eLFes/hs8W89OzZE8OGDYO9vb31b1lZWdi6dSsyMjIkvb+4u7vjySefhL+/PxITE5GYmIiUlJRq/X11qS0vDS58TcVWP7DG4nmpGc9LzXhealZbXvivrHEcJzm88HEcJzm88HEcJzl19vFxHMfZIt7i4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh4zhOcnjh45oVY+zG3xaBMbZI7Lg4aVGIHQAnLUTkcOv/GWP2AHIBbBAvIk6KeIuPE9MoAHkAjoodCCctvPBxYpoEYBXxX7zimhn/lTVOFIyxVgDSAAQSUZrY8XDSwlt8nFgmAviDFz1ODLzwcWKZCGCl2EFw0sRPdblmxxjrDWAfAE8i0osdDyc9vMXHiWESgM286HFi4S0+juMkh7f4OI6THF74OI6THF74OI6THF74OI6TnDpvUsAYu+cjH0TE7vU2mhrPS814XmrG81IzMfPCW3wcx0kOL3wcx0kOL3wcx0kOL3wcdx/z9vZGVFQUunbtKnYoNoXfgZnjRKZUKqFSqVBeXl7t7z4+Pli0aBEGDBgABwcHxMbGihRh82CMQS6Xw2w2Q6VSwc7ODgCgVqvh4+MDX19f+Pj4oFevXoiLi8OJEydw+vRpGAyGu97WXRe+li0O1daiAAAgAElEQVRbokuXLlCpVNX+rtPp8OSTT8Le3h5btmxBQUEB5HI5hg4dii5dumDJkiVYt24dLBbLXQf5IAgKCsJ//vMf2NvbY8+ePSAiGI1GXLp0CQkJCQCAqqoqFBUVgV8m+P/8/f3h5+eHLl26oHv37mjVqhW+/vpr7N69W+zQmoWvry/+85//oLCwEL/++qv17127dsWIESPw8MMPQyaTwd3dXcQo7z2NRoN+/fph0KBBMJlM8PPzQ0hICABAJpNBrVZDrVZDo9HAyckJ5eXlKCoqwgcffIC1a9fe9fbuqvD1798fr7/+Ovr06QONRlPtMcYYNBoNGGN44oknrAVOrVZDqVSiqKgIO3bsgF5vm9elP/300wgPD4eTkxMefvhh699NJhPMZjMAIDc3FxcvXoTRaMTmzZsRExOD69evixVys/Lw8EBgYCDCw8Ph7e2N9u3bo2PHjtBqtdBoNFAqlVAqlZDL5fDw8EBmZibi4uLEDvueGzhwICIiIkBEeP7558HYzdkXCoUCGo0GMpkMZrMZGzduFDnSeys4OBizZs1Cr169QESQy+VQKpXVGgm3ckNEcHJygoODA3Q6XYO2d1eFLyAgAL17977j0cfe3v62vz300EN49tlnsWLFirsK8EGxZMkSyOVy+Pr6AgCcnJwwcuTIal9oR0dHBAYGAgDGjRuHCxcuYNq0aTh16pSYod9TISEheP755zFx4kS4u7uDMWbdgYGbO/Gtg6RCcXN39PPzw2OPPSaJwnfmzBkQkfU0Lz8/HxaLBRqNBi1btoRcLsfXX3+Nffv2iR3qPZWUlIStW7eidevWcHd3h8lkgiAIsFgsMBqNuHr1KjIyMuDq6orQ0FBotdpGbe+uCp9MJqu209akoqLC2sKxt7eHXC4HAFgsFuvfbZFer8dnn31m/TdjDGq12vrv4OBghIaGQq1WIyAgANOnT4e7uzscHR3FCLfZvPnmm5gwYYK1qBGR9TQlLy8PV69exfnz52EymTBnzhyo1WpotVq0adNG5Mibx4ULF/Dss88CAMrKyhAdHQ1BEBAREYEvv/wSPj4+OHXqlM13j5SVlWHRokW4cOECiouLkZWVBSKyHhiNRiNMJhOGDh2KBQsWoFWrVgAAo9HYoO3dVeErKipCRUUFgJuncEajEWazGenp6bBYLLBYLNi/fz9SUlKg1Wrx1ltvoVWrViAipKWl4dy5cw0K8kFERKiqqrL+OyYmBjExMWCM4ZlnngEA6wdqy0pKSmAymSCXy1FUVIS4uDhs3boV27dvR1ZWFiorKwEAQ4YMsX65s7OzsWfPHjHDbla7du2q9m9PT088//zz8Pb2RkJCAo4elcaP0FksFhw8eLDWx9u0aYNx48bBw8MDAJCWlobDhw83aFt3VfgOHjyIPXv24KGHHkJ6ejoSEhJw48YNrFu3zjqyUlxcDG9vb4wYMQIymQxEhOzsbGzZsgWJiYkNCtKWdO7cGW+99RYsFgsyMzORmpoqdkj31Pr16+Hi4gK1Wo1Tp05h/fr1yM7OrvYcV1dXfPTRR1Cr1TCbzTh+/HidXwBbc2v08hYHBwc4ODhALpfjm2++QVFRkUiR3T8UCgXGjBmDp556CiqVClVVVZgxYwaSk5Mbtr67eXJZWRkWLlwInU6H7OxsXL169bbnaDQaTJw4EW+//Tbs7OxgMpmwZ88eLF++3KZPdevDw8MDr732Gh566CEYDAbs2bMH165dEzuse+rkyZNISEiAwWCwtu7+Si6XY/To0ejYsSMYY7h+/TpWrFgBk8kkQrTNr23btpg6dWq1LqS/9gUrFArrqXBJSYnN9/X9nUKhgK+vL3x9fTFo0CDY2dmhoqICq1evxoEDBxq+3rt5ssVisU7NqAljDCNHjsSYMWOsnY8VFRVYsWIFSkpKGhykrRg1ahSGDRsGxhj279+P9evXix1Ss6jrs+/YsSMmTpwIBwcHEBF+/fXXBp++PIjatGmDf/7zn5DJbl5L8NeRSyLCe++9B7PZDIvFgj/++ENShe/tt99G+/bt4evrCxcXF3Tr1g3Azf0pLi4O/fr1g0KhQGhoKFq2bInY2Fhs2rQJhYWFd1x3k01gVqvVGDlyJKKiotCuXTswxnDjxg3MnTsX0dHRTbWZB5KnpyfGjh2LKVOmoEWLFli6dCm++uormz/NvRMHBweMHz8enTt3hkwmQ2xsLObNmyepM4Po6GisW7fOWvhycnLQtm1bPPXUU8jIyMBHH32EkpISEBEyMzNFjrZ5zZo1C05OTtYB0ls5cnNzw7vvvguLxQKZTAadTgelUokjR47g5MmT9Sp81iNLTQsAqu/i5+dHq1evJpPJRIIgkCAINGvWLNJqtXW+rq7t36/L3eTF09OTlixZQqWlpVRWVkaRkZHk7Ox8x9eJ/R7vdV4YYzRy5EhKTEwkIqKqqip65513JJkXZ2dn0ul0pNPpyN3dnb7++msSBIE2bNhArq6uJJfLSS6X05+3cZJMXn7//XdrLalr0ev1tH//furTpw8plcp65aVJWnxOTk6IjIzEmDFjrFWZiHDw4MEa+3Wkws3NDW+//TYmTZoEpVKJ999/H+vXr7/t0iQp6tSpE1577TW0a9cOALB3714sXrxY5KjEUVpaav1/V1dX6zzY69evw2g0QhAEsUIT1XvvvYdPPvkE3bt3h5OTEwRBwKVLl5Cbm4uUlBTs2bMHgiCguLgYmZmZuH79+q2CemeNPYLLZDJ6+OGHKTo62lqBjUYjff/996TT6SR5BAdAOp2O5syZQxUVFaTX6+mjjz4id3f3ereIxH6P9yov+LOFM3v2bKqsrCQioqSkJOrcubPk8wKAunfvTseOHSOz2UwREREkl8slnRdPT0+Kjo4ms9lMhw8fpq5du9Y7l3XlpdF3Z3Fzc8MHH3yAHj16ALg5vy8xMRGzZ8+W7ICGTCbD0KFDERUVBQA4dOgQNm7ciPz8fJEjEx9jDO3atcPYsWOh0WhQVFSE+fPnN3hagq3RaDSwt7cHYwx9+/Zt9BUKDzLGmPXGBIwxFBQUVGsdN0ajCl9gYCCGDRuGxx57DMD/n96+9957KC4ubor4Hkienp548sknYbFYcOLECcydOxeXLl0SO6z7gr29PUaPHo0OHTrAZDJhzZo12Lp1a4PusGGLPDw8rFet/POf/4SLi4vIEYnHwcEB48aNg6enJywWC2JiYpCent40K29IU9TOzo4mTpxIW7ZsoczMTOspbkJCAvXr169JmqL381LX+wkICKD58+eTwWCgK1eu0KRJk+rslJZKXoCbAxpDhw6l9PR0IiKKj4+nwYMH1/t0zlbz8telR48edOLECTKbzWQ2m8nPz0+yefH397fmISsriyZMmNBk36MGDW44Oztjzpw58PX1tQ5mVFRUYOfOnbh8+XJDVmkTdDod3n33XYwdOxYlJSX49NNPsXv37lsfsuRpNBq8+eab8Pf3R2VlJXbu3ImYmBjJdt7XJC0tDcePH0dISAhKSkokNbWnLklJSTh9+nTTrbAhFdnHx4fy8/NJEAQym80UFxdHU6ZMobZt297V0Rs2dqR6/fXXKS8vj4xGI61atYo0Gs1dH6FsMS+3lvHjx1N5eTkREZ09e5bCw8N5XmpoFbdq1Yr69etHYWFhpFKpJJsXd3d3OnjwIBUVFdHs2bPrnYv65KXR01lKSkrw/vvvY+fOnZLvpykqKoJKpUJycjI+/fTTajcp4ID27dvDzs4OZWVl+O233yR104r6IiJkZGQgIyND7FBEV1hYiPHjx0Oj0aC4uLhJb+jB/qy8NT9Yy+9eMsbg4OAAxhgsFgsqKysbfLpCNvR7oAqFAlqtFhaLBRUVFagrt3diS3m55dZddIkIBoOhQTuyLealKfC81Ky2vNRZ+DiO42wR/5U1juMkhxc+juMkhxc+juMkhxc+juMkhxc+juMkhxc+juMkhxc+juMkhxc+juMkhxc+juMkhxc+juMkhxc+juMkhxc+juMkhxc+juMkhxc+juMkhxc+juMkhxc+juMkhxc+rtkxxgIYYzsZY8WMsRzG2LeMsUb/DALH1RcvfJwYvgOQB8ALQCiAfgBeEzUiTlJ44ePE0BrAeiKqIqIcALsBdBY5Jk5CeOHjxPANgHGMMTvGmA+AobhZ/DiuWfDCx4nhMG628MoAXAMQA2CrqBFxksILH9esGGMyAHsAbAZgD8ANgAuAeWLGxUkL/3lJrlkxxtwA5APQEVHpn38bCeB/iChY1OA4yeAtPq5ZEVEBgDQAUxljCsaYDsAkALHiRsZJCS98nBieATAEN1t+yQDMAKaLGhEnKfxUl+M4yeEtPo7jJIcXPo7jJIcXPo7jJIcXPo7jJIcXPo7jJKfOWwExxu75kC8RsXu9jabG81Iznpea8bzUTMy88BYfx3GSw2/+yHH3GUdHR3Tt2hXt2rWDWq0GAFgsFqSmpuLs2bMoKioSOcIHX5MWPicnJ4SHh6N3797QarXYv38/du3a1ZSbeGCoVCoEBgaiQ4cOUCgUcHJyQkBAALRaLVJSUrB69WrcuHFD7DCbHWMMdnZ2cHJyQocOHRAWFgZXV1fk5ORgz549SExMhMlkEjtMUbVr1w5RUVF44oknoFKpANwsfFeuXMHatWvx7bfforCwUOQom5dKpUJ4eDj69+8POzs7FBYWoqSkBOnp6bhw4QKysrLuboVEVOsCgOq7BAQE0BdffEGJiYmUmppKer2e4uPjacaMGeTm5lbr6+ra/v263CkXCoWCxo8fTydOnKDU1FRKT0+nzMxM0uv1ZDKZKD8/nyIiIupch9jv8V7kRS6X0wsvvEBr1qyh/fv308WLF6msrIwEQaDi4mI6fvw4hYWFkUqlorZt25Krq6sk8gKAlEolPfzwwzRr1iw6evQolZWVUU3S0tJq3HfEfo/3Ki8AyM/PjxYsWEDnzp0jg8FAFouF9Ho95eXlUUJCAq1cuZL8/f3v6nvUJIF5enrSd999R8XFxbRp0ybq378/ffTRR2QwGGjbtm3k5eUlqS94eHg4ZWdnk16vp927d9PcuXMpIiKCQkNDKSoqiiorKyk0NLRaQbC3tye5XG7TedFoNJSdnU0Gg4HMZjMJgkCCIJDZbCaz2Uwmk4nmzp1LP/74I509e5amTp0qif1FqVTS6tWrKS0tjUpLS+n06dM0Y8YM6t27N4WEhFD37t3pFrPZTHPnzpVEXgBQy5YtaeXKlaTX68lsNpPFYiGLxUKCIJDJZCKj0UgVFRW0adMmCgoKar7C5+joSPPnz6fi4mJaunQp+fn5kUwmI61WSxcvXqTdu3eTTCaTVOEbPHgwVVZW0nfffUc6nY40Go21qL377rv0ww8/kEajIYVCQX369KHVq1dTZGQkaTQam85LZGQkCYJAer2etm3bRpGRkdSvXz+KiIigzZs3k8lkIr1eT5WVlZSVlUWRkZGS2F9mzZpFJpOJysrKaOnSpRQcHExqtZr+HPUkuVxO0dHRdMuaNWvIw8PD5vMCgDp16kSZmZnWg+RfC9/y5ctp4sSJpNfryWAw0MKFC8nR0bF5Cp+7uzsdP36ctm/fTj4+PtUe+/333yk+Pr7aF1oKO/KgQYOovLycPv/8c1KpVNUek8lkpFKpKDQ0lFauXEl5eXn0008/3dZUF/s93ou8pKSkEBFRXl4eTZs2zfrFtre3p1mzZpHRaCRBEKigoIBmzZplfdyW8xIWFkZGo5GqqqpoyZIl1Vr9AEilUtGePXvIYrFYczdlyhSbz8utJSAggE6dOmUtfLdyVVVVRWPHjiW5XE7h4eGUkpJCSUlJNHjw4HrlpdHTWQRBQHZ2NvLz85GTkwPg/zv2AwICYG9vD4VCWoPHZWVlqKqqwvDhw9GpUycolUoANzv27e3tMWHCBPz666949NFH8fXXX+Pf//43rl69KnLU954gCLBYLGCMQaVSQS6Xw8HBAc899xzeffddyOVylJeXY+HChfjyyy9vfTlsmslkgkwmg1qtRp8+fdCuXTsANwcKfX19sWnTJgwcOBCMMRARLly4gN27pfHzJIwxtG7dGq1btwYAJCcnY968eRg7dixef/117NmzB4Ig4Ny5c/j+++/h7u4OT09PMFaPKY2NrcharZZmzJhBBw4coK5du5KDgwONHTuWEhISKD8/n+bMmVPn68U+6tyLI5VcLqdXX32VMjMz6fTp0zRkyBBydXWl3r1706JFi+jixYu0evVq6t+/PykUCsnkZfny5SQIAhkMBtq/fz9FRETQ/PnzKSsri0wmE5nNZtq3b99tp3G2npe0tDQiIkpNTaUxY8ZQcHAwLV68mPR6Pd1SXFxMFy9epFdeeUUyeQFAwcHBZDQaqaSkhKZPn05KpbLG540cOZLKy8tp9uzZ5ODgcMe8NDowxhh17dqV9u7dS/PmzaO3336bMjIyKCkpiWbOnFnrF9vWPzCtVktTpkyh+Ph4Onr0KH3zzTeUlJREqamp9M4775CTk5Pk8hIcHExxcXFkNBrJaDRSWVkZVVVVkSAIlJWVRatXr6awsDDJ5WXatGl0S2lpKV2+fJlKS0uJiOjo0aO0evVqmjlzJvXv319SeQFAbdq0ofT0dLp+/TrNmjWLAgICyMHBgRQKBcnlcrKzsyNPT0+aPXs26fV6mj59er36yhsdGABycHCgWbNmUUZGBuXn59OePXsoIiLitv4tKX1gt/LyxhtvUHZ2NhERxcXF0QsvvHBbB6xU8iKTyWjUqFGUnp5u7bMRBIGKiorozTffvK1/Syp5ad++PWVmZtLf/fbbbxQSElJjX6cU8uLj40OjR4+mxMREEgSBMjIyaPPmzfTFF1/Q22+/TdOmTaOvvvqKfv75Z0pPT6eTJ09Sr1696pWXJut8Y4zBw8MDZWVlWLBgAfbu3QtBEJpq9Q8krVaLgIAAqNVqEBGqqqqwZ88e6PV6sUMTxa0+G3t7exCRtS9GoVCgbdu29eubsSEBAQHo2rUrxo4dC41Gc9vj3333HS5dunSrSEiGk5MT+vXrh8mTJ6NDhw4ICAgAYwy+vr7w8/ODIAgoLy+HIAjQ6XQAgJycHKxZswZxcXH120hjKjJwsz/rscceo9jYWDp//jwlJyfTJ598Qvb29vVqFYl91LkXRyrg5mjcxIkTKTU1lVatWkUbN26kixcvUkBAgGTzEhwcTHl5edZ5e3+dx6fX62nmzJmSysvmzZspIyODiIiMRiMlJibSX93ptN8W8xIYGEjff/89paSkkMlkouLiYkpKSqJz587R9evXqaqqqtqUlhMnTtCIESOoX79+1KJFi9tax7VuuzE7MgDy9vamPXv20OHDhyksLIzmzZtHV65cqXEyoS1/YH9fOnToQNevX6fDhw9TSEgIDR48mJKSkmj+/Pl37Pe0xby0atWK4uPjrTtsVlYWLVu2jHbt2kV6vd76t/bt20siL08//TRdv36diIgqKyvJ19eXPDw8aNCgQXTixAkiIsrNzaUPP/xQMl0AjDH617/+Rbm5uWQymej8+fPUo0cP8vDwoJYtW1Lbtm1p7Nix9MMPP1ivgrrVRVLbXOFat93QHRm4eWnWG2+8QRkZGTR+/HhijNGoUaMoIyODJk+eXOfEZVv6wGorfNnZ2XThwgXq378/ubu7086dO6myspIGDBggubz07NmTSktLSRAEKikpoblz55JWq6VJkyZZJ6gajUZ66qmnbD4vU6ZMoezsbDKbzTR69GjSarXVvvyRkZGUk5NDRETHjx+v9rgt5+WJJ56gixcvUkZGBn322Wc1zv+VyWTk5OREhw4dIpPJRMuWLWtQXho1jy84OBhffvkl/vjjD2zduhVEhMzMTFRWVkIulzdm1Q80mUyGRx55BE5OTti3bx9Onjxpnecok8ng7e0tdojNztvbG1qtFkSElJQUbNy4EZWVlcjKykJVVZV1h7T1fmEvLy8MGDAAnp6eePPNN7Fz505UVlZaHyciLF++HPv37wcAPPLII9iyZYtY4TYrV1dXODo64pNPPsHs2bNRVVV123MsFgvKyspw4MABWCwW+Pv7N2hbjRrcmD59OgoKCrBixQprsevatSscHBxQXl7emFU/0GQyGTp27IiCggL88ccfqKyshFarRVBQEADg8OHDIkconsrKSuzcuROZmZkICgrCCy+8YJ10mpWVhaSkJLFDvKfMZjMKCwthMBgwdepUMMZw9OhR5OTkQKlUwtXVFQDg4uJifU29O+xtxLBhw3Dq1CnEx8fDYDBUe0wmk8HR0REmkwlEBKPR2KBtNKrwderUCZcuXcK+ffugUCjQs2dPPP/888jKysKxY8dgsVgas/oHFhEhPz8f9vb26Ny5MzIyMhAWFgZfX1+cOnXKeoWLlBiNRlgsFmi1WgwfPhxyuRzDhw9H586dQUS4du0a5s+fj5SUFLFDvafy8/Px448/IigoCH379sXChQshCAI2btwINzc3DBgwoNrzi4uLYWdnJ1K0zevGjRvQ6/V48skn4e3tjZ9++glpaWnWg4KLiwtatGiB0NBQvPjiiygvL8eRI0catrGG9tkAoPPnz9Nvv/1GAQEBNGrUKNq3bx/FxMTQ8OHDa51h/fdF7H6GpuibqGnp2rUrpaSkUEZGBkVHR5Ner6eLFy9Snz59JDkvKyAggJKSkqrdjeXWqG5CQgK9/PLLkuoTHjJkCM2dO5cOHTpU4xw+opuDGwsWLKDAwEBJ5MXHx4eioqIoJSWFqqqqyGg0UnFxMR07doxOnjxJmZmZZDQayWAwUG5uLn3xxRd1XuVTV17YnwHU6E73xI+MjMSHH36IS5cuoV27dsjPz8dXX32F3bt3V+u3qAvZ6G8FaLVajBkzBpGRkWjTpg0OHz6MX375BQcOHKhX89wW8zJmzBi8/fbb6NGjBwAgLS0NR44cwY4dO+q9z9haXrp27Qp/f394eXnd9lhBQQFiYmLqdR23reTF1dUVQ4YMwcCBA2FnZwc/Pz94eHjAYrEgPj4e5eXlyM/Px5UrV7BhwwZkZ2fXuY3a8tKowqfRaBAWFga5XA6LxYLc3FykpKTcdl7ekMDuZ/X9kRStVotWrVrB2dkZ165dQ25ubr07720xLwqFAkFBQfDw8AAAlJaW4tq1aygsLKz3XZdtMS9NwZbyolKp0LJlS2t/nlarhUwmQ1FREQwGA6qqqlBRUVGvcYR7Uviagi19YE2J56VmPC8143mpWW154b+yxnGc5NTZ4uM4jrNFvMXHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHcZzk8MLHNTvGWEfG2AHGWCljLJkxFiF2TJy08MLHNSvGmALANgA7ALQAMAXA/zLG2osaGCcp/Dc3uGbFGAsGcBKAI/258zHG9gI4RUQfiBocJxm8xcc1t5p+7o8BCG7uQDjp4oWPa24JAPIAzGCMKRljgwD0A2AnbliclPBTXa7ZMcZCACzCzVZeDIB8AAYielnUwDjJ4IWPEx1j7DiAlUT0g9ixcNLAT3W5ZscYC2GMaRhjdoyxdwB4AVghclichPDCx4lhAoBs3OzrGwBgIBEZxA2JkxJ+qstxnOTwFh/HcZLDCx/HcZLDCx/HcZLDCx/HcZKjqOtBxtg9H/kgopouYbqv8bzUjOelZjwvNRMzL7zFx3Gc5PDCx3Gc5PDCx3Gc5NTZx8c1jZCQEAwfPhwVFRVYtWoVCgsLxQ6J4yStwYVPp9Ohf//+kMvl8PT0RM+ePRETE4Ndu3YhLS0NFoulKeN8IMnlcnTr1g0zZsxAmzZtkJOTg1OnTuHEiROQ4hUzoaGheO2119CyZUuUlpZiw4YN2Lt3L4xGo9ihcRJT5yVrtY26zJ49G0OGDIGvry9kMhlUKhWcnJxQVlaG7OxsJCQkYNWqVdi5c+cdC6CtjkapVCo8+eSTmD59Os6dO4fvv/8eFRUVKCoqwo0bN+64DVvLi4ODA1asWIHhw4dDpVLBbDYjNjYW8+bNw44dO1BZWVmvbdhaXv5Kq9Xi0UcfxYQJEyCTyfDee+8hPT29Xtuw1bzIZDK0b98eEyZMQN++feHk5GR9zGKx4N///jd27dpV6+trzQsR1boAoL8uLi4uNHDgQDp8+DAZDAY6cuQIrVmzxrqsXbuW4uPjqaKignJzc2n79u3k5eVFf1/PX5e6tn+/LnW9n7/mau3atbR9+3YKCgq64/NtPS9RUVFUXl5OBoOBNm3aRJGRkXTu3Dk6e/YshYaG0p9fAsnl5dbi5OREX331FeXn59PRo0cpMTGRJk+eTJ6enuTo6CjJvCgUCpo4cSKtX7+eKioqyGw2k8VisS6CIFBqaiq5uLjcdV7qHRhjjIYPH07x8fF0/fp1Gjt2LOl0OpLJZNUWLy8vGjJkCMXFxZHRaKT9+/eTr6+v5D6wuXPnUmpqKj3//PMkk8lIpVKRVqsltVotuS+4Tqejffv2kclkonXr1pGHhwcpFAoaNWoUJSUl1esAaYt5ubXI5XKaMGECFRYW0meffUadOnWibdu20cKFC2nmzJm0e/duyeVFoVBQVFQUmUwmMpvNJAgCmUwmqqqqsv5bEATKy8sjV1fXe1f4Hn74YTp27BhVVFTQP/7xD1KpVHV+EBEREVRZWUlVVVW0evXqWo/oYif/XuzIvr6+FBMTQ59//jl5e3tTREQEHTx4kMrLy+nixYvk7u4uqR25T58+lJ+fT4WFhfTEE09Y/x4QEECrV6+myspKmjJlCslkMknl5dYSHBxMhYWFtHLlSuuBceDAgTR58mTavHkz7d+/X3J56dmzJxmNRhIEgcrLyykjI4N++eUXev311+n48ePWxzIzM6lFixZ3nZd6T2fp2Am2bMwAAAd7SURBVLEjQkNDkZSUhKNHj96xQ7qoqAhEBLlcjsDAQPj6+tZ3Uw+89957D506dUL37t2xdu1afPHFF2CM4cSJE2jbti2eeuopsUNsVs888wwcHR0RGxuLkydPAgDUajWcnZ2h1WrBGMPjjz8OuVwucqTNT6VSYc6cOUhOTsbs2bNhMNy8LeG+ffvw66+/okuXLpDJpDXrTKVS4dVXX4VMJkNZWRlWrVqFSZMmYfbs2SgoKICDgwNkMhmICDt27KhXn/nf1Tuj2dnZuHbtGhhj6NChAzw8POp8fvfu3SGTycAYg7e3Nzp37nzXwT2oPD09UVJSAn9/f+Tm5mLBggWYOHEi1q1bB4PBgO7du4sdYrMSBAEWiwWVlZUwGo1wcHDA448/jm+//RYjRoyASqUSO0TRtG/fHgMGDMDChQtvG8hwdXWFo6MjGHvgxi0aJSwsDH379gVjDKWlpYiLi0P37t0xY8YMLFiwAJ07dwZjDJWVlfj5558bNCug3tNZzpw5g/Xr12PSpEn4+OOPceTIEaxbtw4nTpyAyWS67fmPPfaYJI/gAPDOO+8gPDwcRUVFOHnyJAoKCgBAslN89u7di2HDhqFz586YO3cuNBoNwsLC0LNnT+Tn58PZ2VlyX+5bxo4di5SUFGzYsKHa3+3t7dG3b19otdpbp4WSER4eDh8fHwCAj48P5syZA51OV+05BoMBa9asQXx8fIO2Ue/CV1hYiHXr1iE8PBx9+vRBmzZt0Lt3b5w4cQLr1q3DuXPn0L17d7Rs2RIA4O3t3aCAbEFKSgpSUlKq/U2lUqF3796Qy+X4/fffRYpMHIcOHcKiRYswZ84czJgxAwBQVlaGTZs2IT4+Hi+99BI8PDyg0WhqPIjask6dOuH8+fPVWi1+fn6IjIxESEgIoqOj4e7uDpVKJZn5jnq93tpIkMlktxU9ADh//jz+53/+B+Xl5Q3byN10PiqVSurQoQM9/fTT9MMPP1B6ejpVVlZSUlISRUdH05UrVygjI4MyMjKovLycBEGg7Oxs+vDDD0mn00miU7a2ZfTo0ZSWlkbnz58nJycnyXVWOzk5UY8ePeiFF16gmTNn0ogRI8jX15f8/f1p9+7dVFxcTFFRUZLLy/Tp0yk6OpqCgoJIp9NR3759admyZXTs2DEaOnQoDRs2jNLS0sjBwUEyefH29qZVq1aR2Wwmo9FIiYmJFBsbS/n5+dbR3OnTp5NcLm/w/tKgCcxyuRz29vZo0aIFWrdujQkTJtz2nMcffxw+Pj44ffo0Bg4cWGtlJhudePl3UVFRePfdd3Hu3DkMGTLkjkdvW82LUqmEQqGAyWSC2WwGYwyzZs3Cv//9b5hMJgwfPhzHjh2r9fW2lhc3Nzd8//33ePjhh1FYWAjGGA4cOIAVK1YgPj4eXbt2xYEDBxAQEICioqJat2FreWnZsiXCw8MhCAIuX74MlUqFDz74AGPGjAEADB06FPv27UNd9QuoPS8NumRNEASUlZWhrKwMV69exdGjR297zpYtW+Dt7Q2z2dzw5qiNUCqV0Ol0UCgU2LBhA8xms9ghicZkMlU7nSUinDx5EiUlJfDy8sKQIUPqLHy2pqCgABMnTkRISAhcXV1x/vx55ObmVttH5HI5vLy86ix8tiYvLw/btm2zttA8PDysI96FhYWNzkWjb1JARDV+kW+dozPGwBi7Y2W2ZUQEi8UCi8WCqqoqscO575w9exaXLl2Cl5cXevfuDRcXFxQXF4sdVrOpqKiwTvOpicViQU5OTjNGdH/462BgYGAgunXrBgDYunUrkpKSGlVT7vkEIZ1OBz8/v3u9mftaixYt8H/t3b2q4kAYBuB30IwRiWAEwUpBQiD+FFro3WjhZXgtWnkRXoCFldoFRQUhqIWgQvxBNKda9xRmd89qiDrfcwEzw1e8YTLfJKlUCrvdDp1OR9jTXTfb7RamacJxHBQKBei67veSXoaiKML18d3DOUckEgEAdLvdf77b7cbTijLGoKqq8MGXzWaRy+UwnU6FOZn7CUmSEAqFwBiDLMsolUp+L+llVCoV4YMvEAhAVVWEw+GnjelpRR3HgWVZGA6HXk7z8mRZBuccx+MRxWIR6XTa7yW9lGAwiHw+D8YYOOdCNbv/jSRJWCwWQr8iiUajKJfLSCQSWK/XWK1WD++aPAm+eDyOWCwGxhhOpxP2+70X07yN+XyOfr8PwzBgGIawjd1uDocDer0erter6ztjkc1mM6EPCGOx2O0m2GAwwHg8xuVyeWhMT77AbNv2bQ+uaRqq1SparZYXU72F0WiERqOBZDKJyWSC1Wrl95JeTrPZhKZp0HVd+Afld5lMxu8l+I5zDlVVsVwu0W63YVnWw2N6EnzH4xG1Wg2RSASO42Cz2Xgxzds4n893b3OQ30zTRL1ehyzLFHzfWJYl/K8KbNvGbDaDoiiYz+dP2fb/VwPzM31a4+WzUF3uE60uoVAIAG49bG4+uS6/Dr0CgQAOh8OPtrludflj8BFCyCcS+5ycECIkCj5CiHAo+AghwqHgI4QIh4KPECIcCj5CiHC+ACMxG+ZhxaQGAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 360x576 with 50 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "num_of_samples = []\n",
    "\n",
    "cols = 5\n",
    "num_classes = 10\n",
    "\n",
    "fig, axs = plt.subplots(nrows=num_classes, ncols = cols, figsize=(5, 8))\n",
    "fig.tight_layout()\n",
    "for i in range(cols):\n",
    "    for j in range(num_classes):\n",
    "        x_selected = x_train[y_train == j]\n",
    "        axs[j][i].imshow(x_selected[random.randint(0, len(x_selected - 1)), :, :], cmap=plt.get_cmap(\"gray\"))\n",
    "        axs[j][i].axis(\"off\")\n",
    "        if i == 2:\n",
    "            axs[j][i].set_title(str(j))\n",
    "            num_of_samples.append(len(x_selected))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of original training examples: 60000\n",
      "Number of original test examples: 10000\n"
     ]
    }
   ],
   "source": [
    "x_train, x_test = x_train[..., np.newaxis]/255.0, x_test[..., np.newaxis]/255.0\n",
    "\n",
    "print(\"Number of original training examples:\", len(x_train))\n",
    "print(\"Number of original test examples:\", len(x_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Filter the dataset to keep just the 1s and 5s, remove the other classes. At the same time convert the label, `y`, to boolean: `True` for 1 and `False` for 5."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def filter_1_5(x, y):\n",
    "    keep = (y == 1) | (y == 5)\n",
    "    x, y = x[keep], y[keep]\n",
    "    y = y == 1\n",
    "    return x,y"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of filtered training examples: 12163\n",
      "Number of filtered test examples: 2027\n"
     ]
    }
   ],
   "source": [
    "x_train, y_train = filter_1_5(x_train, y_train)\n",
    "x_test, y_test = filter_1_5(x_test, y_test)\n",
    "\n",
    "print(\"Number of filtered training examples:\", len(x_train))\n",
    "print(\"Number of filtered test examples:\", len(x_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Display a random image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.colorbar.Colorbar at 0x7f9b2f9f4210>"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAS4AAAD8CAYAAADJwUnTAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAUlElEQVR4nO3df4xlZX3H8feHYQH5pYsrdLusgmSr3baKuIIGa7EEXUgUSURZGkWiXTFso4l/QE1TSZqmtP5Ei2xH2AANQg2grGbritQWTIHuggi7bIEtIgy7YcuPAGL5sTOf/nHuyp07d+49M3NnzjnD55WczD3nOfc531zxu8/znOc8R7aJiGiSvaoOICJiqpK4IqJxkrgionGSuCKicZK4IqJxkrgionGSuCJi1khaJ2mXpC2TlEvSNyRtl3S3pGPK1JvEFRGz6XJgZY/yk4FlrW01cEmZSpO4ImLW2L4ZeLLHKacCV7pwG/AaSYv71bv3oAIsYx/t6/04YC4vGfGK8jzP8aJf0EzqeP97D/ATT46WOveOu1/YCjzfdmjY9vAULrcEeKRtf6R1bGevL80ocUlaCVwEDAGX2r6w1/n7cQDH6cSZXDIierjdN824jieeHOW/Nr6+1LlDix943vaKGVyuW5Lt+xzitBOXpCHgYuAkiiy5SdJ62/dOt86IqJ6BMcbm6nIjwNK2/cOBHf2+NJMxrmOB7bYftP0icA1FfzUiGsyYlzxaahuA9cDHW3cX3wk8bbtnNxFm1lXs1jc9rvMkSasp7hawH/vP4HIRMVcG1eKSdDVwArBI0gjwRWABgO21wAbgFGA78Bvg7DL1ziRxleqbtgbqhgEO1iFZQyei5owZHdByV7ZX9Sk3cO5U651J4ppW3zQi6m+s//h4pWaSuDYByyQdCTwKnAGcOZCoIqIyBkbna+KyvVvSGmAjxXSIdba3DiyyiKjMfG5xYXsDxeBaRMwTBl6q+ZLuczpzPiLqz3j+dhUjYp4yjNY7byVxRcR4xcz5ekviiogOYrTrNM36SOKKiHGKwfkkrohokGIeVxJXRDTMWFpcEdEkaXFFROMYMVrzVd2TuCJignQVI6JRjHjRQ1WH0VMSV0SMU0xATVcxIhomg/MR0Si2GHVaXBHRMGNpcUVEkxSD8/VODfWOLiLmXAbnI6KRRjOPKyKaJDPnI6KRxnJXMSKapHjIOokrKrT3kt/tWb78Bzt7ll942B09y99zz4d7lh98+v9OWjb27LM9vxvVMOKlPPITEU1ikwmoEdE0ygTUiGgWkxZXRDRQBucjolGMspBgRDRL8XqyeqeGekcXERXIC2GjYr/65sKe5d8/7Ac9y/u9iv3mP7q2Z/mb/+rcScveeN6tfWqPKph5PnNe0kPAs8AosNv2ikEEFRHVqnuLaxBp9b22j07SipgfbDHmvUptZUhaKek+Sdslnd+l/NWSfiDpF5K2Sjq7X53pKkbEOMXg/GAe+ZE0BFwMnASMAJskrbd9b9tp5wL32v6ApNcB90m6yvaLk9U70xaXgR9LukPS6kkCXy1ps6TNL/HCDC8XEbOvWHO+zFbCscB22w+2EtE1wKkd5xg4SJKAA4Engd29Kp1pi+t42zskHQrcKOm/bd88LiJ7GBgGOFiHeIbXi4hZVgzOlx7jWiRpc9v+cOv/83ssAR5p2x8Bjuuo4x+B9cAO4CDgo7Z73heaUeKyvaP1d5ek71Fk15t7fysi6m4KM+cf7zO+3S0DdjZg3g/cBfwpcBRFI+gW289MVum0u4qSDpB00J7PwPuALdOtLyLqYc/M+TJbCSPA0rb9wylaVu3OBq53YTvwS+DNvSqdSYvrMOB7RbeUvYHv2P7RDOqLWfDc0/tVev3dB41Wev2YngG+LGMTsEzSkcCjwBnAmR3nPAycCNwi6TDgTcCDvSqdduKy/SDw1ul+PyLqyYaXxgaTuGzvlrQG2AgMAetsb5V0Tqt8LfA3wOWS7qHoWp5n+/Fe9WY6RESMU3QVBzdz3vYGYEPHsbVtn3dQDDWVlsQVERPUfeZ8EldEjDPF6RCVSOKKiA6D7SrOhiSuiJgga85Hpd78ma09y+/e1nu6wlv2qfdrqmLwiruK9f7fPYkrIsbJ0s0R0UjpKkZEo+SuYkQ0Uu4qRkSj2GJ3EldENE26ihHRKBnjisqNPf98z/K6P5MW1UjiiohGyTyuiGikzOOKiEaxYfeAFhKcLUlcETFBuooR0SgZ44qIRnISV0Q0TQbno9aGJrybc7y9+vwHPKQ+g7j1/u8/urAzxhURjSNGc1cxIpomY1wR0Sh5VjEimsfFOFedJXFFxAS5qxgRjeIMzkdEE6WrGLXWbz2usT7zvPBYn/IpBhS1UPe7in3bg5LWSdolaUvbsUMk3SjpgdbfhbMbZkTMFbtIXGW2qpTpyF4OrOw4dj5wk+1lwE2t/YiYJ8asUltV+iYu2zcDT3YcPhW4ovX5CuBDA44rIipkl9uqMt0xrsNs7wSwvVPSoZOdKGk1sBpgP/af5uUiYq4YMVbzu4qzHp3tYdsrbK9YwL6zfbmIGACX3Koy3cT1mKTFAK2/uwYXUkRUasCD85JWSrpP0nZJXcfDJZ0g6S5JWyX9R786p5u41gNntT6fBdwwzXoioo4G1OSSNARcDJwMLAdWSVrecc5rgG8BH7T9B8Dp/eotMx3iauBW4E2SRiR9ErgQOEnSA8BJrf2ImCcG2OI6Fthu+0HbLwLXUNzca3cmcL3th4tru28Pru/gvO1VkxSd2O+7EdE8BsbGSk91WCRpc9v+sO3htv0lwCNt+yPAcR11/B6wQNK/AwcBF9m+stdFM3M+IsYzUH6O1uO2V/Qo71ZRZydzb+DtFI2hVwG3SrrN9v2TVZrEFRETDHCO1giwtG3/cGBHl3Met/0c8Jykm4G3ApMmrnpP1oiIagxuPsQmYJmkIyXtA5xBcXOv3Q3AH0vaW9L+FF3Jbb0qTYsrIjoM7jlE27slrQE2AkPAOttbJZ3TKl9re5ukHwF3A2PApba3TF5rEldEdDPA2aW2NwAbOo6t7dj/EvClsnUmcUXEeAaXv6tYiSSuiOgiiSsimqbmC0AmcUXERElcEdEoU5uAWokkroiYIC/LiIjmyV3FiGgapcUVEY1S9fKmJSRxRUQHZXA+IhooLa6IaJw+LyivWhJXRIyXeVwR0US5qxgRzVPzxJUVUCOicdLieoUb6vNP6159ljcZUp9/++o9VBKTSFcxIprF5JGfiGigtLgiomnSVYyI5kniiojGSeKKiCaR01WMiCbKXcWos9E+E63G+vUZ3Odp3Jr/yx3d1b3F1XfmvKR1knZJ2tJ27AJJj0q6q7WdMrthRsSccsmtImUe+bkcWNnl+NdsH93aNnQpj4gm8svjXP22qvRNXLZvBp6cg1gioi7mQYtrMmsk3d3qSi6c7CRJqyVtlrT5JV6YweUiYq5orNxWlekmrkuAo4CjgZ3AVyY70faw7RW2Vyxg32leLiLiZdNKXLYfsz1qewz4NnDsYMOKiErNx66ipMVtu6cBWyY7NyIapgGD833ncUm6GjgBWCRpBPgicIKkoyly7kPAp2cxxmiwVe+6bdKyO7KOZX3VfB5X38Rle1WXw5fNQiwRURdNT1wR8coiqr1jWEba6hEx3oDHuCStlHSfpO2Szu9x3jskjUr6cL86k7giYqIB3VWUNARcDJwMLAdWSVo+yXl/D2wsE14SV0RMNLjpEMcC220/aPtF4Brg1C7n/QVwHbCrTKVJXBExwRS6iov2PBnT2lZ3VLUEeKRtf6R17OVrSUsoplWtLRtfBudf4T76b5/pWX7/yn+aUf0ffPXPJy27g7fPqO6YReXvKj5ue0WP8m7rJnXW/nXgPNujUrl1wJK4ImI8D/Su4giwtG3/cGBHxzkrgGtaSWsRcIqk3ba/P1mlSVwRMdHg5nFtApZJOhJ4FDgDOHPcpewj93yWdDnww15JC5K4IqKLQT3OY3u3pDUUdwuHgHW2t0o6p1VeelyrXRJXREw0wJnzrYVGN3Qc65qwbH+iTJ1JXBExXsUrP5SRxBUR44j6vywjiSsiJkjiilpbevgTVYcQdZTEFRGNk8QVEY1S8eqmZSRxRcRESVwR0TR1X0gwiSsiJkhXMSKaJRNQI6KRkriiSqMnHNOz/Nrl3+xZvhev6lk+pN5rUb5j38kHS5466109v7vwilt7lsfsyMz5iGgkjdU7cyVxRcR4GeOKiCZKVzEimieJKyKaJi2uiGieJK6IaJTBvuVnVvRNXJKWAlcCvwOMAcO2L5J0CPAvwBHAQ8BHbD81e6HGdDxzxL49yxfutV/P8rF+//S693/hfb8ftdOEeVxl3mS9G/i87d8H3gmcK2k5cD5wk+1lwE2t/YiYD+xyW0X6Ji7bO23f2fr8LLCN4hXapwJXtE67AvjQbAUZEXNLLrdVZUpjXJKOAN4G3A4cZnsnFMlN0qEDjy4i5t58moAq6UDgOuBztp9pvS67zPdWA6sB9mP/6cQYEXOs7oPzZca4kLSAImldZfv61uHHJC1ulS8GdnX7ru1h2ytsr1hA74HiiKgHjZXbqtI3caloWl0GbLP91bai9cBZrc9nATcMPryImHOm9oPzZbqKxwMfA+6RdFfr2BeAC4HvSvok8DBw+uyEGDNxwM6XepY/PfZ8z/JX95kuEfNT3adD9E1ctn9GMbWjmxMHG05E1ELTE1dEvLI0YQJqEldEjGdnIcGIaKB6560kroiYKF3FiGgWA+kqRkTj1DtvJXHNd/ts3NyzfMNzb+hZvuqgx2Z0/Zv+b/LHvF77i6d7frfmT53Ma4PsKkpaCVwEDAGX2r6wo/zPgPNau78GPmP7F73qTOKKiAkGdVdR0hBwMXASMAJskrTe9r1tp/0S+BPbT0k6GRgGjutVb6lnFSPiFcRT2Po7Fthu+0HbLwLXUCyJ9fLl7P9sW4T0NuDwfpWmxRUR4xQTUEu3uBZJah+PGLY93La/BHikbX+E3q2pTwL/2u+iSVwRMVH5AcbHba/oUd7tccGuWVHSeykS17v7XTSJKyImmEKLq58RYGnb/uHAjgnXk94CXAqcbPuJfpVmjCsixhvsGNcmYJmkIyXtA5xBsSTWb0l6PXA98DHb95epNC2uiOgwuGcVbe+WtAbYSDEdYp3trZLOaZWvBf4aeC3wrdbKyrv7dD+TuF7p/vnPP9Cz/PWXX9Gz/C/vP61n+cLPD01aNnbvvZOWRcUGuEig7Q3Aho5ja9s+fwr41FTqTOKKiPHmwwthI+IVqMJlmctI4oqIieqdt5K4ImIijdW7r5jEFRHjmdo/4Z7EFRHjCA9yAuqsSOKKiImSuKLO9rrl5z3L/+6ot/QsP5j/6Vk+OuWIohaSuCKiUTLGFRFNlLuKEdEwTlcxIhrGJHFFRAPVu6eYxBURE2UeV0Q0T80TV98VUCUtlfRTSdskbZX02dbxCyQ9Kumu1nbK7IcbEbPOhtGxcltFyrS4dgOft32npIOAOyTd2Cr7mu0vz154EVGJmre4+iYu2zuBna3Pz0raRvHKoYiYr2qeuKb0sgxJRwBvA25vHVoj6W5J6yQtnOQ7qyVtlrT5JV6YUbARMQcMjLncVpHSiUvSgcB1wOdsPwNcAhwFHE3RIvtKt+/ZHra9wvaKBew7gJAjYnYZPFZuq0ipu4qSFlAkratsXw9g+7G28m8DP5yVCCNibplKB97LKHNXUcBlwDbbX207vrjttNOALYMPLyIqYZfbKlKmxXU88DHgHkl3tY59AVgl6WiK/PwQ8OlZiTAi5l7NB+fL3FX8GaAuRRu6HIuIxstD1hHRNAayrE1ENE5aXBHRLK79XcUkrogYz+AK52iVkcQVERNVOCu+jCSuiJgoY1wR0Sh27ipGRAOlxRURzWI8Wu9X+SZxRcR4e5a1qbEkroiYqObTIaa0kGBEzH8GPOZSWxmSVkq6T9J2Sed3KZekb7TK75Z0TL86k7giYjwPbiFBSUPAxcDJwHKKVWWWd5x2MrCsta2mWKS0pySuiJjAo6OlthKOBbbbftD2i8A1wKkd55wKXOnCbcBrOtb7m2BOx7ie5anHf+Jrf9V2aBHw+FzGMAV1ja2ucUFim65BxvaGmVbwLE9t/ImvXVTy9P0kbW7bH7Y93La/BHikbX8EOK6jjm7nLKH1kp5u5jRx2X5d+76kzbZXzGUMZdU1trrGBYltuuoWm+2VA6yu21p+nYNjZc4ZJ13FiJhNI8DStv3DgR3TOGecJK6ImE2bgGWSjpS0D3AGsL7jnPXAx1t3F98JPN16n+ukqp7HNdz/lMrUNba6xgWJbbrqHNuM2N4taQ2wERgC1tneKumcVvlaimXgTwG2A78Bzu5Xr1zzZ5IiIjqlqxgRjZPEFRGNU0ni6vcIQJUkPSTpHkl3dcxPqSKWdZJ2SdrSduwQSTdKeqD1d2GNYrtA0qOt3+4uSadUFNtSST+VtE3SVkmfbR2v9LfrEVctfrcmmfMxrtYjAPcDJ1HcBt0ErLJ975wGMglJDwErbFc+WVHSe4BfU8wq/sPWsX8AnrR9YSvpL7R9Xk1iuwD4te0vz3U8HbEtBhbbvlPSQcAdwIeAT1Dhb9cjro9Qg9+tSapocZV5BCAA2zcDT3YcPhW4ovX5Cor/8OfcJLHVgu2dtu9sfX4W2EYxE7vS365HXDFFVSSuyab314WBH0u6Q9LqqoPp4rA9c1xafw+tOJ5Oa1pP+K+rqhvbTtIRwNuA26nRb9cRF9Tsd6u7KhLXlKf3z7HjbR9D8cT6ua0uUZRzCXAUcDTFc2ZfqTIYSQcC1wGfs/1MlbG06xJXrX63JqgicU15ev9csr2j9XcX8D2Krm2dPLbnyfnW310Vx/Nbth+zPeripXzfpsLfTtICiuRwle3rW4cr/+26xVWn360pqkhcZR4BqISkA1qDpkg6AHgfsKX3t+bceuCs1uezgBsqjGWcjqVITqOi306SgMuAbba/2lZU6W83WVx1+d2apJKZ863bvV/n5UcA/nbOg+hC0hspWllQPA71nSpjk3Q1cALFsiePAV8Evg98F3g98DBwuu05HySfJLYTKLo7Bh4CPt3vmbNZiu3dwC3APcCe1e6+QDGeVNlv1yOuVdTgd2uSPPITEY2TmfMR0ThJXBHROElcEdE4SVwR0ThJXBHROElcEdE4SVwR0Tj/D68JFnpkNF+nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.imshow(x_train[random.randint(0,12000), :, :, 0])\n",
    "plt.colorbar()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Downscale the images\n",
    "\n",
    "An image size of 28x28 is much too large for current quantum computers. Resize the image down to 4x4:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train_small = tf.image.resize(x_train, (4,4)).numpy()\n",
    "x_test_small = tf.image.resize(x_test, (4,4)).numpy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Contradictory examples\n",
    "\n",
    "\"Learning to Distinguish Digits\", filter the dataset to remove images that are labeled as belonging to both classes mentioned in [this paper](https://arxiv.org/pdf/1802.06002.pdf).\n",
    "\n",
    "This is not a standard machine-learning procedure, but is included in the interest of following the paper."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "def remove_contradicting(xs, ys):\n",
    "    mapping = collections.defaultdict(set)\n",
    "    for x,y in zip(xs,ys):\n",
    "       mapping[tuple(x.flatten())].add(y)\n",
    "    \n",
    "    new_x = []\n",
    "    new_y = []\n",
    "    for x,y in zip(xs, ys):\n",
    "      labels = mapping[tuple(x.flatten())]\n",
    "      if len(labels) == 1:\n",
    "          new_x.append(x)\n",
    "          new_y.append(list(labels)[0])\n",
    "      else:\n",
    "          # Do not include images that match more than one label.\n",
    "          pass\n",
    "    \n",
    "    num_1 = sum(1 for value in mapping.values() if True in value)\n",
    "    num_5 = sum(1 for value in mapping.values() if False in value)\n",
    "    num_both = sum(1 for value in mapping.values() if len(value) == 2)\n",
    "\n",
    "    print(\"Number of unique images:\", len(mapping.values()))\n",
    "    print(\"Number of 1s: \", num_1)\n",
    "    print(\"Number of 5s: \", num_5)\n",
    "    print(\"Number of contradictory images: \", num_both)\n",
    "    print()\n",
    "    print(\"Initial number of examples: \", len(xs))\n",
    "    print(\"Remaining non-contradictory examples: \", len(new_x))\n",
    "    \n",
    "    return np.array(new_x), np.array(new_y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Number of unique images: 6588\n",
      "Number of 1s:  2176\n",
      "Number of 5s:  4626\n",
      "Number of contradictory images:  214\n",
      "\n",
      "Initial number of examples:  12163\n",
      "Remaining non-contradictory examples:  7949\n"
     ]
    }
   ],
   "source": [
    "x_train_nocon, y_train_nocon = remove_contradicting(x_train_small, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Data as quantum circuits\n",
    "\n",
    "We will represent each pixel with a qubit, with the state depending on the value of the pixel. The first step is to convert to a binary encoding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "THRESHOLD = 0.5\n",
    "\n",
    "x_train_bin = np.array(x_train_nocon > THRESHOLD, dtype=np.float32)\n",
    "x_test_bin = np.array(x_test_small > THRESHOLD, dtype=np.float32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The qubits at pixel indices with values that exceed a threshold, are rotated through an `X gate`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "def convert_to_circuit(image):\n",
    "    values = np.ndarray.flatten(image)\n",
    "    qubits = cirq.GridQubit.rect(4, 4)\n",
    "    circuit = cirq.Circuit()\n",
    "    for i, value in enumerate(values):\n",
    "        if value:\n",
    "            circuit.append(cirq.X(qubits[i]))\n",
    "    return circuit\n",
    "\n",
    "x_train_circ = [convert_to_circuit(x) for x in x_train_bin]\n",
    "x_test_circ = [convert_to_circuit(x) for x in x_test_bin]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets now visualize the circuit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"169.517734375\" height=\"100.0\"><line x1=\"34.7588671875\" x2=\"139.517734375\" y1=\"25.0\" y2=\"25.0\" stroke=\"#1967d2\" stroke-width=\"1\" /><line x1=\"34.7588671875\" x2=\"139.517734375\" y1=\"75.0\" y2=\"75.0\" stroke=\"#1967d2\" stroke-width=\"1\" /><rect x=\"10.0\" y=\"5.0\" width=\"49.517734375\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"0\" /><text x=\"34.7588671875\" y=\"25.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">(1, 2): </text><rect x=\"10.0\" y=\"55.0\" width=\"49.517734375\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"0\" /><text x=\"34.7588671875\" y=\"75.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">(3, 1): </text><rect x=\"79.517734375\" y=\"5.0\" width=\"40\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"1\" /><text x=\"99.517734375\" y=\"25.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"18px\">X</text><rect x=\"79.517734375\" y=\"55.0\" width=\"40\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"1\" /><text x=\"99.517734375\" y=\"75.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"18px\">X</text></svg>"
      ],
      "text/plain": [
       "<cirq.contrib.svg.svg.SVGCircuit at 0x7f9a61fdda50>"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "SVGCircuit(x_train_circ[random.randint(0,7949)])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert these Cirq circuits to tensors for `tfq`. This is made very easy with `convert_to_tensor()` in `tfq`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "x_train_tfcirc = tfq.convert_to_tensor(x_train_circ)\n",
    "x_test_tfcirc = tfq.convert_to_tensor(x_test_circ)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Quantum neural Network\n",
    "\n",
    "[The paper](https://arxiv.org/pdf/1802.06002.pdf) propose using two qubit gates, with the readout qubit always acted upon. This is in a way similar to running a [Unitary RNN](https://arxiv.org/abs/1511.06464) in some ways."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Build the model circuit\n",
    "\n",
    "Each layer uses `n` instances of the same gate, with each of the data qubits acting on the readout qubit.\n",
    "\n",
    "Start with a simple class that will add a layer of these gates to a circuit:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "class CircuitLayerBuilder():\n",
    "    def __init__(self, data_qubits, readout):\n",
    "        self.data_qubits = data_qubits\n",
    "        self.readout = readout\n",
    "    \n",
    "    def add_layer(self, circuit, gate, prefix):\n",
    "        for i, qubit in enumerate(self.data_qubits):\n",
    "            symbol = sympy.Symbol(prefix + '-' + str(i))\n",
    "            circuit.append(gate(qubit, self.readout)**symbol)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Build an example circuit layer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/svg+xml": [
       "<svg xmlns=\"http://www.w3.org/2000/svg\" width=\"522.59953125\" height=\"250.0\"><line x1=\"39.810625\" x2=\"492.59953125000004\" y1=\"25.0\" y2=\"25.0\" stroke=\"#1967d2\" stroke-width=\"1\" /><line x1=\"39.810625\" x2=\"492.59953125000004\" y1=\"75.0\" y2=\"75.0\" stroke=\"#1967d2\" stroke-width=\"1\" /><line x1=\"39.810625\" x2=\"492.59953125000004\" y1=\"125.0\" y2=\"125.0\" stroke=\"#1967d2\" stroke-width=\"1\" /><line x1=\"39.810625\" x2=\"492.59953125000004\" y1=\"175.0\" y2=\"175.0\" stroke=\"#1967d2\" stroke-width=\"1\" /><line x1=\"39.810625\" x2=\"492.59953125000004\" y1=\"225.0\" y2=\"225.0\" stroke=\"#1967d2\" stroke-width=\"1\" /><line x1=\"129.99353515625\" x2=\"129.99353515625\" y1=\"25.0\" y2=\"75.0\" stroke=\"black\" stroke-width=\"3\" /><line x1=\"230.73810546875004\" x2=\"230.73810546875004\" y1=\"25.0\" y2=\"125.0\" stroke=\"black\" stroke-width=\"3\" /><line x1=\"331.48267578125007\" x2=\"331.48267578125007\" y1=\"25.0\" y2=\"175.0\" stroke=\"black\" stroke-width=\"3\" /><line x1=\"432.22724609375007\" x2=\"432.22724609375007\" y1=\"25.0\" y2=\"225.0\" stroke=\"black\" stroke-width=\"3\" /><rect x=\"10.0\" y=\"5.0\" width=\"59.62125\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"0\" /><text x=\"39.810625\" y=\"25.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">(-1, -1): </text><rect x=\"10.0\" y=\"55.0\" width=\"59.62125\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"0\" /><text x=\"39.810625\" y=\"75.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">(0, 0): </text><rect x=\"10.0\" y=\"105.0\" width=\"59.62125\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"0\" /><text x=\"39.810625\" y=\"125.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">(1, 0): </text><rect x=\"10.0\" y=\"155.0\" width=\"59.62125\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"0\" /><text x=\"39.810625\" y=\"175.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">(2, 0): </text><rect x=\"10.0\" y=\"205.0\" width=\"59.62125\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"0\" /><text x=\"39.810625\" y=\"225.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">(3, 0): </text><rect x=\"89.62125\" y=\"55.0\" width=\"80.74457031250002\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"1\" /><text x=\"129.99353515625\" y=\"75.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">XX^(xx-0)</text><rect x=\"89.62125\" y=\"5.0\" width=\"80.74457031250002\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"1\" /><text x=\"129.99353515625\" y=\"25.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">XX</text><rect x=\"190.36582031250003\" y=\"105.0\" width=\"80.74457031250002\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"1\" /><text x=\"230.73810546875004\" y=\"125.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">XX^(xx-1)</text><rect x=\"190.36582031250003\" y=\"5.0\" width=\"80.74457031250002\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"1\" /><text x=\"230.73810546875004\" y=\"25.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">XX</text><rect x=\"291.11039062500004\" y=\"155.0\" width=\"80.74457031250002\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"1\" /><text x=\"331.48267578125007\" y=\"175.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">XX^(xx-2)</text><rect x=\"291.11039062500004\" y=\"5.0\" width=\"80.74457031250002\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"1\" /><text x=\"331.48267578125007\" y=\"25.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">XX</text><rect x=\"391.85496093750004\" y=\"205.0\" width=\"80.74457031250002\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"1\" /><text x=\"432.22724609375007\" y=\"225.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">XX^(xx-3)</text><rect x=\"391.85496093750004\" y=\"5.0\" width=\"80.74457031250002\" height=\"40\" stroke=\"black\" fill=\"white\" stroke-width=\"1\" /><text x=\"432.22724609375007\" y=\"25.0\" dominant-baseline=\"middle\" text-anchor=\"middle\" font-size=\"14px\">XX</text></svg>"
      ],
      "text/plain": [
       "<cirq.contrib.svg.svg.SVGCircuit at 0x7f9a777978d0>"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "demo_builder = CircuitLayerBuilder(data_qubits = cirq.GridQubit.rect(4,1),\n",
    "                                   readout=cirq.GridQubit(-1,-1))\n",
    "\n",
    "circuit = cirq.Circuit()\n",
    "demo_builder.add_layer(circuit, gate = cirq.XX, prefix='xx')\n",
    "SVGCircuit(circuit)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now build a two-layered model, matching the data-circuit size, and include the preparation and readout operations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "def create_quantum_model():\n",
    "    \n",
    "    # Place data qunits on the grid\n",
    "    data_qubits = cirq.GridQubit.rect(4, 4)  \n",
    "    readout = cirq.GridQubit(-1, -1)\n",
    "    circuit = cirq.Circuit()\n",
    "    \n",
    "    # Prepare the readout qubit.\n",
    "    circuit.append(cirq.X(readout))\n",
    "    circuit.append(cirq.H(readout))\n",
    "    \n",
    "    builder = CircuitLayerBuilder(\n",
    "        data_qubits = data_qubits,\n",
    "        readout=readout)\n",
    "\n",
    "    # Then add layers (experiment by adding more).\n",
    "    builder.add_layer(circuit, cirq.XX, \"xx1\")\n",
    "    builder.add_layer(circuit, cirq.ZZ, \"zz1\")\n",
    "\n",
    "    # Finally, prepare the readout qubit.\n",
    "    circuit.append(cirq.H(readout))\n",
    "\n",
    "    return circuit, cirq.Z(readout)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_circuit, model_readout = create_quantum_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Wrap the model in Keras layer\n",
    "\n",
    "Build the Keras model with the quantum components. This model is fed the \"quantum data\", from x_train_circ, that encodes the classical data. It uses a Parametrized Quantum Circuit layer, `tfq.layers.PQC`, to train the model circuit, on the quantum data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = tf.keras.Sequential([\n",
    "    # The input is the data-circuit, encoded as a tf.string\n",
    "    tf.keras.layers.Input(shape=(), dtype=tf.string),\n",
    "    # The PQC layer returns the expected value of the readout gate, range [-1,1].\n",
    "    tfq.layers.PQC(model_circuit, model_readout),\n",
    "])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we will compile the model. Since this is a binary classification problem we here use `BinaryCrossentropy`. We shift the output range to $[0,1]$, and treat it as the probability the model assigns to class 3. This could be used with a standard a tf.losses.BinaryCrossentropy loss.\n",
    "\n",
    "> Note: Another valid approach would be to use the Hinge Loss. Since the the expected readout is in the range $[-1,1]$, optimizing the hinge loss is also valid. To use the hinge loss here you need to make two small adjustments. First convert the `labels`, `y_train_nocon`, from boolean to $[-1,1]$, as expected by the hinge loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.compile(\n",
    "    loss= tf.keras.losses.BinaryCrossentropy(),\n",
    "    optimizer=tf.keras.optimizers.Adam(),)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "EPOCHS = 3\n",
    "BATCH_SIZE = 32\n",
    "\n",
    "NUM_EXAMPLES = 500#len(x_train_tfcirc)\n",
    "x_train_tfcirc_sub = x_train_tfcirc[:NUM_EXAMPLES]\n",
    "y_train_hinge_sub = y_train_nocon[:NUM_EXAMPLES]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Model: \"sequential\"\n",
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "pqc (PQC)                    (None, 1)                 32        \n",
      "=================================================================\n",
      "Total params: 32\n",
      "Trainable params: 32\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "None\n"
     ]
    }
   ],
   "source": [
    "print(model.summary())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 500 samples, validate on 2027 samples\n",
      "Epoch 1/3\n",
      "500/500 [==============================] - 140s 281ms/sample - loss: 3.7780 - val_loss: 6.6636\n",
      "Epoch 2/3\n",
      "500/500 [==============================] - 140s 280ms/sample - loss: 3.6805 - val_loss: 6.6183\n",
      "Epoch 3/3\n",
      "500/500 [==============================] - 140s 281ms/sample - loss: 3.6135 - val_loss: 6.5709\n",
      "2027/2027 [==============================] - 16s 8ms/sample - loss: 6.5709\n"
     ]
    }
   ],
   "source": [
    "qnn_history = model.fit(\n",
    "      x_train_tfcirc_sub, y_train_hinge_sub,\n",
    "      batch_size=32,\n",
    "      epochs=EPOCHS,\n",
    "      verbose=1,\n",
    "      validation_data=(x_test_tfcirc, y_test))\n",
    "\n",
    "qnn_results = model.evaluate(x_test_tfcirc, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## References"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* [Classification with Quantum Neural Networks on Near Term Processors](https://arxiv.org/pdf/1802.06002.pdf)\n",
    "* [TensorFlow Quantum Whitepaper](https://arxiv.org/abs/2003.02989)\n",
    "* [TensorFlow Quantum Docs](https://www.tensorflow.org/quantum/api_docs/python/tfq)\n",
    "* [Announcing TensorFlow Quantum Dev Summit 2020](https://youtu.be/-o9AhIz1uvo)\n",
    "* [Programming a Quantum Computer with Cirq](https://youtu.be/16ZfkPRVf2w)\n",
    "* [Announcing TensorFlow Quantum](https://ai.googleblog.com/2020/03/announcing-tensorflow-quantum-open.html)\n",
    "* [Unitary RNNs](https://arxiv.org/abs/1511.06464)"
   ]
  }
 ],
 "metadata": {
  "environment": {
   "name": "tf2-gpu.2-1.m47",
   "type": "gcloud",
   "uri": "gcr.io/deeplearning-platform-release/tf2-gpu.2-1:m47"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}