demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deep learning using Lasagne in Nengo\n",
    "\n",
    "Lasagne is a popular deep learning package.  It uses Theano as the underlying computational framework, but provides a higher level syntax to make it easier to construct neural networks.  The `nengo_lasagne` package is a backend for Nengo that compiles a Nengo network into a Lasagne implementation.  The network can then be trained using any of Lasagne's built-in optimization methods, or any other training scheme implemented in Theano.\n",
    "\n",
    "From the user's perspective, constructing a network looks the same as any other Nengo model.  There are a few additional limitations (so the set of possible `nengo_lasagne` models is a subset of the overall space of Nengo models).  Some of these are due to features that I just haven't implemented yet, some may never be implemented (certain network structures are just not amenable to deep learning optimization, and it seems better to just be upfront about that rather than trying to shoe-horn them in)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import pickle\n",
    "\n",
    "import lasagne\n",
    "import matplotlib.pyplot as plt\n",
    "import nengo # note: requires the `nengo_lasagne_compat` branch currently\n",
    "import nengo_lasagne\n",
    "import numpy as np\n",
    "import theano\n",
    "\n",
    "from lasagne import nonlinearities as nl\n",
    "from nengo.processes import PresentInput\n",
    "\n",
    "theano.config.floatX = \"float32\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We need to define the overall inputs and outputs for the network, which will drive the training. The inputs and outputs should be 3D arrays, with shape `(batch_size, sequence_length, input d)` and `(batch_size, sequence_length, output d)` respectively (input and output `d` do not have to be the same). This means that we're going to be processing `d` dimensional inputs and outputs, and processing `batch_size` vectors at a time. The `sequence_length` dimension defines the inputs that will be presented at each timestep (so each timestep will process one `(batch_size, d)` set of inputs).\n",
    "\n",
    "Note that if there are no recurrent connections in the model then the sequential presentation of inputs makes no difference (since there is no internal state persisting between timesteps).  So in those cases the `batch_size` and `sequence_length` dimensions will be flattened (effectively processing all the timesteps at once).  But to keep things simple the inputs and outputs are still always specified as 3D arrays.\n",
    "\n",
    "In this example, we are splitting the overall input into 3 one dimensional inputs, though it would be possible to do the same thing with one 3 dimensional input."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# in this demo we'll just use uniform random inputs\n",
    "# we could just create one 3D node, but we'll use 3 1D\n",
    "# nodes for demonstration purposes, resulting in a 4D array.\n",
    "inputs = np.random.uniform(-1, 1, size=(3, 1000, 1, 1)).astype(np.float32)\n",
    "\n",
    "# an arbitrary output function\n",
    "def target_func(inputs):\n",
    "    return (inputs[0] + 1) * inputs[1] ** 2 + np.sin(inputs[2])\n",
    "outputs = target_func(inputs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can build the network itself.  `nengo_lasagne.default_config()` specifies a set of defaults that are useful for constructing these kinds of networks (used to override the standard Nengo defaults).  Specifically, this removes synaptic filtering, sets all biases to be the same (zero), limits the initial range of gains (0.9-1.1) and sets the default neuron to be a rectified linear unit (ReLU)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parameters configured for Ensemble:\n",
      "  bias: Choice(options=array([ 0.]))\n",
      "  gain: Uniform(low=0.9, high=1.1)\n",
      "  neuron_type: RectifiedLinear()\n",
      "Parameters configured for Connection:\n",
      "  synapse: None\n"
     ]
    }
   ],
   "source": [
    "print(nengo_lasagne.default_config())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "with nengo.Network() as net, nengo_lasagne.default_config():\n",
    "    # create nodes to define the input for the network (the same as\n",
    "    # any other nengo model). note that the output of these nodes is \n",
    "    # just used when running the network, not during the training \n",
    "    # process (during training we'll use the input array defined above,\n",
    "    # since we're not training online)\n",
    "    # here we'll use the PresentInput process to feed in the inputs we\n",
    "    # defined above, one at a time. \n",
    "    input_nodes = [nengo.Node(output=PresentInput(x.squeeze(axis=1), 0.001))\n",
    "                   for x in inputs]\n",
    "\n",
    "    # create ensembles \n",
    "    ens0 = nengo.Ensemble(100, 2)\n",
    "    ens1 = nengo.Ensemble(50, 1)\n",
    "    \n",
    "    # create connections (note that slicing works in the same\n",
    "    # way as default nengo)\n",
    "    nengo.Connection(input_nodes[0], ens0[0])\n",
    "    nengo.Connection(input_nodes[1], ens0[1])\n",
    "    nengo.Connection(input_nodes[2], ens1)\n",
    "\n",
    "    # network output\n",
    "    output_node = nengo.Node(size_in=1)\n",
    "    \n",
    "    # we can specify a function/transform on a connection. these\n",
    "    # will be used to initialize the weights (using the standard\n",
    "    # NEF nengo solvers)\n",
    "    nengo.Connection(ens0, output_node,\n",
    "                     function=lambda x: (x[0] + 1) * x[1] ** 2)\n",
    "    nengo.Connection(ens1, output_node, function=np.sin)\n",
    "\n",
    "    # add probes to nodes or neurons to collect their output\n",
    "    p = nengo.Probe(output_node)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This network looks like:\n",
    "<img src=\"3Dnetwork.png\" width=400>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To access the `nengo_lasagne` backend we just pass the network we constructed to `nengo_lasagne.Simulator`. This will build  a Lasagne network with the same structure as the Nengo network defined above.  The weights in this network are untrained (set to whatever initial values are defined in the Nengo network).  If we run the untrained network we can see the initial performance.\n",
    "\n",
    "The probed output will be stored in the `sim.data` dictionary, as in standard Nengo.  Note that the probed outputs will have the same shape as described above, `(batch_size, sequence_len, d)`.  Batch size is just 1 when we run it this way, but we'll see an example where batch size is >1 later on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "checking network\n",
      "building network\n",
      "[[[ 0.422203  ]]\n",
      "\n",
      " [[ 0.76344299]]\n",
      "\n",
      " [[ 0.23597831]]]\n",
      "[[[ 0.53315175]\n",
      "  [ 0.85680389]\n",
      "  [ 0.35134622]]]\n",
      "0.0384067296982\n"
     ]
    }
   ],
   "source": [
    "sim = nengo_lasagne.Simulator(net)\n",
    "\n",
    "sim.run_steps(1000)\n",
    "\n",
    "# example target/actual outputs\n",
    "print(outputs[:3])\n",
    "print(sim.data[p][:, :3])\n",
    "\n",
    "# root mean squared error\n",
    "print(np.sqrt(np.mean(outputs - sim.data[p]) ** 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To train the network we call the `train` function on the constructed model.  The first parameter is a dict that maps `nengo.Nodes` to the input arrays we defined above (so the backend knows where to insert those inputs in the network).  The second parameter does the same for the targets.  `n_epochs` defines how many times the training process will iterate through the training data, and `minibatch_size` defines the size of the minibatches used within an epoch (once through the dataset). Typically the batch size will be the size of the entire data set, whereas the minibatch size will be a subsample of that data used per update during training.  `optimizer` specifies one of the \n",
    "optimization methods defined in `lasagne.updates` (or this could be any Theano optimizer that implements the same interface).  The `optimizer_kwargs` will be passed to the optimizer function (in this case this is setting the learning rate)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "training network\n",
      "Simulation finished in 0:00:05.                                                 \n",
      "training complete\n"
     ]
    }
   ],
   "source": [
    "sim.model.train(dict(zip(input_nodes, inputs)), {output_node: outputs},\n",
    "                n_epochs=1000, minibatch_size=100, \n",
    "                optimizer=lasagne.updates.adagrad,\n",
    "                optimizer_kwargs={\"learning_rate\": 1e-2})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can run the trained network:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[[ 0.422203  ]]\n",
      "\n",
      " [[ 0.76344299]]\n",
      "\n",
      " [[ 0.23597831]]]\n",
      "[[[ 0.43956223]\n",
      "  [ 0.75437605]\n",
      "  [ 0.23053226]]]\n",
      "0.000272936274996\n"
     ]
    }
   ],
   "source": [
    "sim.run_steps(1000)\n",
    "\n",
    "print(outputs[:3])\n",
    "print(sim.data[p][:, :3])\n",
    "\n",
    "print(np.sqrt(np.mean(outputs - sim.data[p]) ** 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You'll notice that we get a pretty small error after training. For comparison, we can run the same network with the reference simulator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Simulation finished in 0:00:01.                                                 \n",
      "[[[ 0.422203  ]]\n",
      "\n",
      " [[ 0.76344299]]\n",
      "\n",
      " [[ 0.23597831]]]\n",
      "[[[ 0.85368089]\n",
      "  [ 0.37356134]\n",
      "  [ 1.01460264]]]\n",
      "0.031002271685\n"
     ]
    }
   ],
   "source": [
    "sim = nengo.Simulator(net)\n",
    "sim.run_steps(1000)\n",
    "\n",
    "print(outputs[:3])\n",
    "print(sim.data[p][None, :3])\n",
    "\n",
    "print(np.sqrt(np.mean(outputs - sim.data[p][None, ...]) ** 2))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Recurrent networks\n",
    "\n",
    "Recurrent networks work in exactly the same way. One new feature being demonstrated here is the option to pass a distribution for the transform parameter.  This will cause the initial weights to be sampled from that distribution.  Any of the distributions in `nengo.dists` can be used, or any of the distributions in `lasagne.init` (access them via `nengo_lasagne.init` to wrap them in a Nengo distribution).\n",
    "\n",
    "Here we're implementing a 1D integrator:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "checking network\n",
      "building network\n",
      "training network\n",
      "Simulation finished in 0:00:30.                                                 \n",
      "training complete\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.legend.Legend at 0x10b525b90>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEKCAYAAADzQPVvAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHmhJREFUeJzt3Xe8JFWd9/FPT2IGhjikITlIEAQRkPgAemXWZRAVkSUY\nUEBdV1TWgIIgy4AkCeqCrglQQIUlKyKspBF4CEqYIeOQk2QfwHV3cfX7/HHO9fad6b63Q1Wdqq7v\n+/Xq1/Ttrjrny3Dn19XnVJ0CMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzGxSPArNz7mMucHbOfZjl\nakLqAGZ9UnyYmdkAe4RwZL8vcANwIvAS8DAwp2m7ecBxwC3Ay8AlwPLxvSHgiUXafTS2Owf4H+A1\n4FXgjqz/A8yK4CN7GwTDR/ZbAfcDM4ATgNMX2W4fYD9gJvC/wCnjtCngCuBY4FxgaWCzzFKbFcjF\n3gbJY4QCL+AsQlFfOb43/Nq9wJ+Aw4E9gUYH7TY63M6stFzsbZA80/T8T/HP6U2vNQ/VPA5MBlbM\nO5RZGbjYW52stcjzPwMvAP8JLNn03kRgpaafPQFsledib4Og06GYDwEbEgr7UcD5hEL+O2Aq8E7C\n0f5XgCWa9n0GmNVhP2al5GJvg0C0PgVTizw/G/gR8HtgCnBgfO9l4ADgNOBJ4I+MHvI5P/75InBr\nhrnNKmMq4VS2+YSJr+PSxjFr61pg/9QhzFKZ1Of+/w28nTAZNolwnvP28U+zsvEwjNVWFsM4w2c9\nTCFMbL2UQZtmefBEq1kfJhCGcV4lXMhiZmYDbFngZsKl52ZmViL9jtk3exm4DNiCsA7JsAeBdTLs\nx8ysDh4C1k0dYtiKwHLx+TTgOhZfbraM46RzUwdoYW7qAG3MTR2ghbmpA7QwN3WAFuamDtDC3NQB\n2pibOkALmdbOfo/sZwJnEsbtJxDOY76631BmZpatfov9XcDmWQQxM7P81PUK2nmpA7QwL3WANual\nDtDCvNQBWpiXOkAL81IHaGFe6gBtzEsdYBCUcczezKzsMq2ddT2yNzOrFRd7M7MacLE3M6sBF3sz\nsxpwsTczqwEXezOzGnCxNzOrARd7M7MacLE3M6sBF3szsxpwsTczqwEXezOzGnCxNzOrARd7M7Ma\ncLE3M6sBF3szsxpwsTczqwEXezOzGnCxNzOrARd7M7MacLE3M6sBF3szsxpwsTczqwEXezOzGnCx\nNzOrARd7M7Ma6LfYrwlcC9wD3A0c2HciMzMrnVWBTePz6cADwIaLbKNCE5mZDYZS185LgNmLvFbq\nwGZmJVXa2jkLeIxwhN+stIHNzEqslLVzOnAr8N4W75UysJlZyWVaOydl0MZk4ELgx4RhnBa2/jHc\n8mD8YV58mJnZiKH4KKUGcBbwjTG2Eehh0JIFZTIzGwSlGhXZHvgrMB+4Iz7mLLKNQOeAji86nJlZ\nhZWq2HdCoFVAz4E2SR3GzKwiqljsAfRx0M0gX7VrZja+yhb7CaAbQAekjWNmVglVLfYAeiPoedBq\n6eKYmVVClYs9gI4GnZ8miplZZVS+2E8DLQS9K00cM7Nmmpw6QRtVL/YAmg16FLTo0gpmZgXS+qAX\nQbNSJ2lhEIo9gM4CnVxsFDOzYZoIuhH0FOifU6dpYWCK/UqgZ0GbFRvHzAxAXwJdA9oVdG3qNC0M\nSrEH0H6g34ZPWDOzomijeGbgrDiP+DJoxdSpFjFQxb4BmgfyHa7MrCCaDLo1XOj5t9cuBO2bLFJr\ng1TsAfQG0AugNYuJY2b1psNBl4eDzb+9tg+ozaq9yQxasQfQEaCL849iZvWmTeM6XWss8vryoFdK\ntjrvQBb7JUD3g1rd/MTMLAOaAloA+nCb968qWQ0axGIPoLeBngAtnW8cM6snHQ362ejhm1Hvfxp0\nZrGZxjSoxR5AZ4D+Nb8oZlZP2hL0DGjVMbZZM15glcUd/LIw0MV+RvwfsmV+ccysXjQVdC9orw62\nvRX09vwzdWSQiz3EWfE7SvTpamaVphNB53W47WElGl0Y+GLfiBMlX8gnjpnVh7YHPd35BVPaCPRY\n+3H9Qg16sQfQuvHc+9dlH8fM6kFLEVbY7eIMGzXiPpvnl6tjdSj2EL9O/aIkn7BmVjk6FXR2D/ud\nCDoq+zxdq02xnwK6B/QP2cYxs8GnHUFPhoulut53O9Cd2WfqWl2KPcTxtqdAy2YXx8wGm5Yh3C9j\n5x73nxjPClwn01g9BEncf9f6DKzvgb6dTRQzG3z6Pui0DNr4fDZ5eg+RuP+u9Vvsl4+z6dtmE8fM\nBpfmxKP6Zfps552g6zOJ1EeIxP13LYPA2juMoZX2XpFmlpyWi0uuzM6gramg/wdauf+2eg+RsO+e\nZFHsG4QlSQ/uvy0zG0w6M9shX/076KPZtdd9gIR99ySjwFo7nnv/+mzaM7PBofeAHgRNz7DN94Mu\nza697gMk7LsnGQbWl0BX+Nx7MxuhGXFeb4eM212WsMZ9qpV4a13sJxPWo947uzbNrNp0LujrObX9\nHwmv9SldsT8DeBa4q837GQfWNvFTvIeLJcxssGgPwo2PpuXU/idBP86n7fE7T9RvWzsAm1FYsYcw\nCaPvZ9+umVWHVokXP22dYx+rgV5KdCZg6Yo9wCyKLfbLxkuht8++bTMrPzVAF4OOK6Cvm0F/l38/\ni3ecoM9xzaLQYg9hHE33hDV0zKxe9CHQXaAlCujrkERX8WdaO7M6q2UWcCnwphbvCTiy6ed58dEn\nNYCfAzdD45j+2zOzatDqwB3AHGjcXkB/GwBXAWtB4685djQUH8OOILsanZlZFH5kD6C14rn36+XX\nh5mVhxqgX4LmFtzv/RR/u9RMa+eELBsrXuNx4FjgOz733qwW9gdWBYr+Nn8x0MVNUAbTOcDTwP8A\nTwD7LfJ+zpMMmgS6HbRPvv2YWVp6Heh5UKvh4rz73jrMERbbacH99a2AwNoinoI1I/++zKx4mkC4\nN/WXE/b/NGj9IjstsK9MFBRY3wSdUUxfZlYsHQC6JXyTT5bhO2HJluI6LLCvTBRV7JcGPQ4aKqY/\nMyuG1oknYmyQOMdOoBuL7LDAvjJRYGDtGmfNCzj31szypwmg60h/1yjCNT36A2hmUR0W1E9mCg6s\ni4s/LcvM8qHPga4HTUydJNBPQf9YVGcF9ZOZoov9GnHGPvFXPjPrj94Qh29S3/i7ifYEXV5UZwX1\nk5kEgfUZ0Dyfe29WVZoU16T5VOoko2kZwhr3fd7jtrPOCugjUymK/UTQb0CLnvNvZpWgQ0BXhzH7\nstEvQXsV0VEBfWQqUWBtCnoWtFKa/s2sN9o4DsW+LnWS1vRx0DlFdFRAH5lKGFgngc5K17+ZdUeT\n4xXxH0udpD2tGs/KyfusPxf7LrpeCvQoadaiNrOu6Yg4TFLy+Tb9X9CcvDvJuf3MJQ6sXUALye22\nZWaWDW0Oeo6whHHJ6Yug7+bdSc7tZ64EgXUe6OjUKcysHS1BuBlJRRY01Hqg3+c8gVyC2tmdEgTW\nzDjhs1HqJGbWio4FXVL+4Ztmuhu0TZ4d5Nh2LkoSWJ8E3VDOU7nM6kxbE1atXSV1ku7oaNDxeXaQ\nY9u5KElgTQDdFE6bMrNy0LS4ntWeqZN0T1uAHsjx20hJamfnShRYb4oTQKumTmJmADoZ9O+pU/RG\nDdAToA3z6iCndnNTssA6DnR26hRmph0INwRZMXWS3ulU8ruhSslq5/hKFljT4yz6m1MnMasvLQV6\nELRr6iT90WzQLXk1nlO7uSlhYH0adFnqFGb1pW8NxtXtmgx6KadrA0pYO8dWwsCaAnoY9NbUSczq\nR7PjWPfyqZNkQ2eDDsij4RzazFVJA+tDoBurdV6vWdVpGdBjBSw1UCDtDvpVHg3n0GauShpYE0AL\nQO9JncSsPnQa6AepU2RLSxHWuF8u64Yzbi93JQ6sXeJVcCW55ZnZINMXQA9RzI0/Cqafgz6QdaMZ\nt5e7EgdWg3Az44+kTmI2uNQAnQC6F7Rm6jT50P6g87NuNOP2clfywNoujiFOTZ3EbPBoEuiHhFsM\nzkidJj9aCfRyxnWk5LVzcRUIrJ+DPps6hdlg0bT4b+uKMK496HRdGBrOrsEM2ypEBQJrY8ItDAdw\nLNEsBS0Huh70k3Cqcx3ocxlPPlegdo5WkcA6E3Rk6hRm1aeZoDtB/0qtVpnV6+NBY1YnfJSuds4B\n7gcWAge3eL90gVvTLNCLoJVTJzGrLq0bL1g8tJ7XsGgBaPusGsuonUxMBB4EZgGTgfnAoivAlSrw\n2PRN0CmpU1jVaEfQbqlTpKfNCAub1XgZcR0JOimrxjJqJxPbAlc0/XxIfDQrVeCxaWXQC6C1Uyex\nqtAEwprmT4POAC2dOlEaGiIsH/6+1EnS0maEBd6y+FaTae3sdzxtdeCJpp+fjK9VVOM54FTAY/fW\nqXcBrwDrE/5x3g7aKm2koul9wHnAXtC4KHWaxOYDk4DS3QJ1Up/7d/rJM7fp+bz4KKuvAwtBm0Dj\nztRhrPS+AJwMjT8CHwX9A3BpmJzka9D4S9p4edPHCQdHO0HjjtRp0msIdAnwXuDuLnceio9S2obR\nwzhfZvFJ2goN4wzTgaBLU6ewstOWoEfDhUOjXl8TdC3o16C1kkTLnRpxEvbhMClrIzQEui2LhjJo\nIzOTgIcIE7RTqPwE7TAtEf8RZzWrbgNJ54Zzq1u+NxF0SBzHruD9VceiCfFkhgXhNEsbTZPi3F+/\nH/Slq507Aw8QzsppdXuu0gXujD4MuqGep4/Z+P52qu44F+JpS9DvCEsGDMDkrabEC6WuI/tVHgeI\nfgT6TL+NZJGkSJULHGgi6C7Qu1MnsTLSN0AndLjtdMLSvg9We/JWS4EuJyyBMC11mnLTrqCr+20k\nkygFqlzgEXp3LPheAtmaaDnCrejW6HK/3eMVlodV73dKMwiLmf1w8TkKW5yWJCyMtkI/jWQWpyCV\nCzxCjTiUs0/qJFYm+hLo7B73reDkrdYkLE98goc1u6GLw3Bw7w1kFqUglQs8mrYHPRImbc00BfQk\naNM+2pgIOrgak7fagLAE+BdTJ6kefQTUz3UHlaudlQu8OP0inI5ppg9lMBY73NYWcfL2R+WcvNVW\noGdA+6ZOUk2aEYdyluy1gUzjFKBygRenTeIvfQn/QVpx1ADNB+2cYZvTQT+Ik7dbZ9duv/T3oOd9\ngkK/dA293+e6crWzcoFb09mgI1KnsJQ0G3RPPuPWZZq81d5xiMnXmfRNB4ZJ7d52zjRKASoXuDWt\nTbhQYqXUSSwVXQ7aP8f214hHgtelm7zVp+OcxCZp+h80el2sG72cwVS52lm5wO3pFNA3U6ewFLQx\n6Pf5T9RrYjzb5znQXvn2NarfBmF53oV41deM6XbQ23rZMfMoOatc4Pa0CuGqyVmpk1jRdEYYYims\nv7cQlk4+M/+5Ik0EfQd0G755Tw50OOgbveyYeZScVS7w2HRk+Ado9aFVCRdRzSi436VA3wc9lN/k\nrZYAnReHj3wP5lzoTYTTt7ud66lc7axc4LFpmTiRtnHqJFYUHQ36dsL+3xd/576S7eStlgZdBboA\nNDW7dm00NeIHdrfXZlSudlYu8Pj0WdDPUqewImipeApi4mV8tTroatD1YdKv7/ZWAt0K+m76s3/q\nQCeD5na7Ux5J8lS5wOPTVMISyNulTmJ506fo7yrIDGkC6Itx8nbvPtqZFecDjsrnNFJbnHYAze92\np1yi5KhygTujj8SjLP9jGViaGL9+/5/USUbrZ/JWG4OeoP/ld60rmhg/pLs506lytbNygTujiYQL\nbHZJncTyoveBbkqdorVRk7fbdLjPdnHs//35ZrPWdFoYAu58h9yi5KRygTunXUF3hq/XNnh0I2j3\n1CnGpt0IS3kcPvbYu3aJR5Y7FZfNRtO7QL/uZofcouSkcoE7p0YsCB9MncSypm3jUXMFJi+1ejyr\nps3krT4cPxBKtPZOHWkaYWG0Tq/Cr1ztrFzg7uithJsuT0mdxLKkC6o1rt1u8lafJyxRvOi9oS0J\nXQDar9ONc42Sg8oF7p4uA306dQrLitaJp1tOT52ke9ocdD/oLNCJoPtAa6ZOZcP0QTo/bbtytbNy\ngbunNxPWTalgcbDF6VTQsalT9E5LxfPnf03hV/3a2LQ86JXw/2j8jXOPk7HKBe6NfhImyazatAJh\naYSZqZPYoNKV4Uyv8TfMPUrGKhe4N3o9YSnTFVMnsX7oUHpff9ysA/pUGGYbf8Pco2SscoF7p2+B\nvp46hfVKS4CeBr0pdRIbZFqDsHru5PE2LCROhioXuHd/WwI50Y0nrD/aD3RF6hRWB/oNaMfxNiok\nSoYqF7g/+qqHAapIDdDdoHekTmJ1oENBp4y3USFRMlS5wP3RsoRL0jdKncS6oTmgBXitIyuE3gh6\nfJzft8rVzsoF7p8+D7o4dQrrhq4MV5qaFUEN0O9Abxlro8LiZKRygfunqfGqxW1TJ7FOaFPCTbZ9\nFbQVSCeEYd/2GxQWZRx7APcAfwE2H2O70gQulvaLF7V4WKD0dBbo4NQprG60LeiusTYoLMo4NgDW\nB67Fxb4FTQLdC9o5dRIbi9aIF1EtnzqJ1Y0mxCvv290FrXS108W+Le0Gmo+XQC4xfQ30zdQprK70\nPdBB7d4sNEoHXOzbUgN0M75ZRElp6XjVczd3DzLLkHYG3dDuzSx7mjTO+1cCq7Z4/VDg0i76mdv0\nfF581EBDoEOA00EXQuO11IlslI8CV0PjkdRBrLauAc4JF2Q2NgSGEucZk4/sx6XLQQekTmHNNIlw\n0/itUiexutO5oI+1eqPwKOO4Fhioc0Wzp83imiteArk0tBfo+tQpzEB7gy5r9UbhUdrYDXgC+C/g\nGeDyNtuVJnBaOgd0WOoUBnEu5TegXVMnMYtX3b8S5pBGv5EkTh8qFzgfWjdOBvpmEslph3j1os+S\nspLQFaA9Fn0xSZQ+VC5wfvRvoJNSpzD9DPRPqVOYjdAnQD9Z9MUkUfpQucD50UzCEsi+J2gyegPh\nptxLpk5iNkIz48V9zUt2VK52Vi5wvnQM6PTUKepL3wUdmTqF2eJ0E+jvm19IFqVHlQucLy0Xjyw3\nTJ2kfrQS6A+glVMnMVucDg5DvSMvJIvSo8oFzp8OAl2UOkX96AjQD1KnMGtNbwA91XTiQOVqZ+UC\n50/TQE+Atk6dpD40DfSMv1FZuenepgv9Klc7Kxe4GPoo6Fq8BHJB9HHQL1KnMBubjg2P8EPSKD2o\nXOBiaBLoftBOqZMMPk2If9dDqZOYjU1bge4b/iFplB5ULnBxtDvodl/ckze9C3Sbv0VZ+WlCHLff\ngIJXvbR8XQQcDOwJnFtMl2oA04EZHTyWAA6Dxo3FZMvNQcBJYRVSszJr/DVc9EfmS3kUcaSjgvqp\nKO0IfB/YEBp/7nLficDyhMK8Ip0V8BnA/wIvjvF4If45AzgZOAY4tZrFUm8BLgbW6f7v1ywFvQM4\nChrbkGHtdLEvBf0HcBnhSL+5KI9XwJcBXmGkOI/3iNs1/ruLbGsDFwILgY9B49V+/kuLp58Ct0Hj\n5NRJzDqjKcAz0FgeF/tBo80IN4p5jfZH2a0ef4DGXwrINxU4Fdge2B0a9+bfZxa0FnAHsDY0Xkmd\nxqxzOhgax1Ox2lnBr/7WmvYDPU9lbrOok73wnFVY5Wpn5QLbWLQp6EHQqYss2lQyWtaLzlnFVa52\nVi6wjUfLxWWCbypvMdVBLZaMNauSytXOygW2TmhCXLjpmXj2QIloclyOYqx7I5uVXeVqZ+UCWzc0\nRLi/7lfKc3GYPgi6JnUKsz5VrnZWLrB1S6uBrgddBlohcZZGvCp5l7Q5zPpWudpZucDWC00OZ77o\nEdAWCXPsGFcOLMm3DLOeVa52Vi6w9UO7E27O8ok0a9HoMtDHiu/XLHOVq52VC2z90vqgu0BnUui9\nXvXGOGE8tbg+zXJTudpZucCWBS0FOhu0ALReQX2eBjq8mL7Mcle52lm5wJYVNUD/FId1dsu5r1UJ\n95ddMd9+zApTudpZucCWNW0JehR0IiinZbX1VUbfrNms6ipXOysX2PKgGaDLQb8Gzcy47SXjt4eC\nhovMClG52lm5wJYXTQD9C+FOPG/NsN0DQBdn155ZKVSudlYusOVNO8WzZg7q//RMTQQtBG2fTTaz\n0ihN7TwRuA9YQLjpxrJttitNYCsTrQW6BXRRWKGy53Z2i+1Uat1vsw6Upna+Axi+SvH4+GilNIGt\nbLQE6NvxyHyTHtu4AbRHtrnMSqGUtXM34Mdt3itlYCsTfYBwU5QPd7nfNnF5hpzO8DFLqpS181Lg\nA23eK2VgKxttBLof9L3Or4DV+aAD881llkyhtfNK4K4Wj3c3bXMY4YbU7bjYW4e0dCzgt4JmjbPt\n60EvgKYXEs2seJnWzvG+/o53U4p9gXcCs8fZbm7T83nxYbaIxqugPYF/Bm4B7QeNX7bZ+LPAD6Dx\nx+LymeVqKD5KZw5wDzDe5ek+srceaDvC3aaOCqdXjnpvBdBLoNXSZDMrRGlq50LgMeCO+Gh3qXpp\nAlvVaBXQNaBfgVZqev3LYUVNs4FWudpZucBWJpoEOhb0eDz7Zkq8ArfHUzXNKqNytbNyga2M9J64\n/s2F4UjfbOBVrnZWLrCVldaJC6m9LXUSswJUrnZWLrCZWQlkWjt9U2YzsxpwsTczqwEXezOzGnCx\nNzOrARd7M7MacLE3M6sBF3szsxpwsTczqwEXezOzGnCxNzOrARd7M7MacLE3M6sBF3szsxpwsTcz\nqwEXezOzGnCxNzOrARd7M7MacLE3M6sBF3szsxpwsTczqwEXezOzGnCxNzOrARd7M7MacLE3M6sB\nF3szsxrop9h/FVgAzAeuBtbMJJGZmZXK0k3PPwOc1mY7FZClW0OpA7QwlDpAG0OpA7QwlDpAC0Op\nA7QwlDpAC0OpA7QxlDpAC5nWzn6O7F9tej4deKHPLEUaSh2ghaHUAdoYSh2ghaHUAVoYSh2ghaHU\nAVoYSh2gjaHUAfI2qc/9jwH2Af4EbNN/HDMzy8N4R/ZXAne1eLw7vn8YsBbwI+Ab+UQ0M7N+NTJq\nZy3gl8DGLd57EFgno37MzOriIWDdrBrrZxhnPWBhfL4rcEeb7TILa2ZmxbuAMKQzH7gQWDltHDMz\nMzMz69gc4H7C8M3BbbY5Jb6/ANisg31XIEwE/w74FbBcSXLtAdwD/AXYvCSZTgTui9tfBCxbgkz9\nXlyXR6ZhXwD+SvgdS51pLvAkYbjzjrhd6kwQrpG5D7gb+FqXmfLKdS4jf0+P0H6IuMhMWwG/iVl+\nC2xZgkxvBm4C7gR+zuhrn/oykTDZOguYTPjHveEi27yTMFELsDVwcwf7ngB8KT4/GDi+JLk2ANYH\nrqX7Yp9XpncwcgbV8XT3d5VXpk4vrisyE4QPnSsIxaKbYp9XpiOAz3eRo4hMbyccaE2OP69UklzN\nTgK+UoJM84Cd4vOdCXUhdabfAjvE5/sBR40VopuLqraKnT4K/Jnw6bvrItu8BzgzPr+FcJS+6jj7\nNu9zJvDeLjLlmet+wreNXuSV6UrCkerwPmuUIFM/F9fllQng64wcRHQjz0y9nv2WV6ZPAsfF1wGe\nL0muYQ1gT+CcEmT6PSPfpJcDnipBpvWA6+Pzq4DdxwrRTbFfHXii6ecn42udbLPaGPuuAjwbnz8b\nf+5GXrn6UUSm/Rk5Ekid6RjgceAjdPdtI69Mu8af7+wiS96ZIHzzWQCcTnfDlXllWg94K+Eoch6w\nRReZ8sw1bAdCTXioBJkOAU4m/J6fCHy5BJnuYaTw78E4Q6jdFPtO12no5Oil0aY9ddFP8z6dyOqa\ngk7knekw4DXgp13sk2emXi+uyyPTNOBQwrBJL/vn9ff0HWBtYFPCUeLJJcg0CViecPX7F4Hzutw/\n79/z99Pd7zjkl+l04EDC7/nngDNKkGl/4ADgVsK36tfG2rib8+yfYvQnx5qET5mxtlkjbjO5xevD\nX4OeJXxdeQaYCTzXRaasc7Xatxd5ZtqXML43u0SZhv2U7r5t5JFpHcL45oKm7W8jfB3u5Hcrr7+n\n5r5PAy7tIEvemZ4kTPRDGP/9KzADeDFxLgi1aTe6ny/LK9NWwN/F5xfQ3dxUXpkeYGQeYX1gly4y\njWkS4evULGAK408ybMPIJMNY+57AyAzzIXQ/QZtXrmHXAm8pSaY5hK9uK3aZJ89M6zXt/xng7BJk\natbtBG1emWY27f85ujtizSvTJ4Aj4/P1CUMU3cjz/98cupsEzTvT7cDb4vPZhA/H1JmGJ9QnAGcR\nDgQzszPh0+RBRsasPhEfw74V31/A6E/lVvtC+Id4Ff2deplHrt0IY2X/RfjWcXkJMi0EHmPktLR/\nK0Gmfi+uyyNTs4fp/tTLPDKdRZhDWABcQvdzU3lkmkz4cL6L8O1nqMtMeeUC+CHwjz3kySvTFoSJ\n0/mE0x2bT41MlenA+PoDwLFd5jEzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzMzG2z/H7sq9hJLN8ot\nAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10b813110>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZIAAAEKCAYAAAA4t9PUAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJztnXe4FNX5xz8D9yIINkBpCtg19o5R9CoW1Nh7izX2mvwU\niUSP0VgwJkaNvZsodiN2LFhib6gREFFUVLDHhiXy/v54z3X3zp3tszuzu+/neeZhd/acM9972D3v\nzDnveV8wDMMwDMMwDMMwDMMwDMMwDMMwDMMwDMMwDMMwDMMwDMMwjIblHmDvBK57GvAx8EHEZ8OB\nKbWVYxiGkRwzgBFFlp0IHFA1JYVxwHUJXr+dwcC3QJ+khQBzgSWSFmGkhy5JCzCaEvFHsWWrRUsV\n246bwcCn/kgDQdICDMNobt4GNvav9wWeAM4GPgPeAkb6z/4E/A+YA3wFnOfPLwdMQAfVKcDOWW33\nAcYD/wWeRaeDHs/6fC5wGDANmO7P/Q1419d5Hljfnx8JfA/84K//kj8/kcxTUgCMQZ+yZgPXAPP7\nz4b66/0aeAedlvp9nn5ZALgW+Mi3d6JvfxP0aeQnr+PKiLptwHtZ72cAvwMmAV8A44B5ssrOBEZ7\nTW8De2TVzf77QP+P2vvwMf83fe21ZPe9YRhGzQgbkh/QgSsADgHezyr7CLB/1vue6IC5D/pEvSo6\nGC7vPx8HXA909+feRQe/duYC9wMLkhlY9wQW8u39FvgQ6OY/Oxkd3LPJ1rQ/apSGem23ZpUf6q93\nib/WysB3qCGM4lrgdt/OEGBq1nU2pKOhCNMW+vxt4Gmgv//bXgcOzir7I/BnoBXYADUMS0f8fdDR\nkIBNbRkhbGrLSAPvAFeg01jXAgOARbI+z55G+RU6SF6DDmgvA7ehd8ZdgR3Qwf87YLIvF56GOQO9\nS//ev/8n8Llv7y/ooL9s1rXzTePsCZyDPgF8g97l70bH39Yp/lqvoE8Iq0S00xXY1df/Bu2Tc8gs\n6pczlXQeMAv928ajRjebP6AG5THgbn99wygZMyRGGpiV9fpb/2+vrHPZ6yRDgHXQwbH92APoB/RF\n1z2y78xnRlwvfGf/f+gd+xe+vQV8W8UwAB3023nXa+iXdS789/WMaKcv+nQQbmtQkTqiyL7uHDr2\n6ef+XDvvoH+LYZSMGRIj7YQX298FHkWna9qP+YDDgU/QNZXFssovRmey2xwOHIc+0Szo2/svmSeA\nQov9H6BTWO0M9hpmF6gX5hP06SDcVpQhjIOFgHmz3g8h41b8DR2NXf8qaTAaBDMkRtqZDSyZ9f4u\nYBlgL/QOvhVYC113+Amd5nJAD39ub/Ibg/nQgf8TdF3kJDKL5aB39UPJPbV0A3CsL9MLOB1dp5mb\n55pRbf0E3IQ6GPRCB/ZjgX/kaadSTkH7bziwFXCzP/8yOkXYA1iKzu7X4f8To8kxQ2IkTZQrcPb7\nvwE7oR5d56KLwpuh6xDvowvjZ5BZHD8CnZqaha6P3IAu5ke1DXCfP95A1znmoE897bQPrp+iHl1h\nrkT3mTyGepx9CxyZ53q5zuHrfePbeRxdu7mqiHrFfB7u5/a1kw9Q/QejfQDwV7TPZvvr/yNU16F9\n+zn6f2MYFTMSdcGcBozKUeY8//kkYLUi6p7qy74MPET09IRhFMNZdByMjc4eXoaRKF2BN9HH+lZ0\n4F8+VGZLNKQE6CLp00XUnS+r/pHA5fHKNhqYZVE32wBYG3UN3iZRRemjDTMkRoxUOrW1NmoMZqAL\nheOAbUNltkEfgwGeQRc0+xeo+1VW/V7o/LVhFMN86F6Or9Hv1J+BOxNVlE6qGTHAaDIqDRExiM6u\nlusUUWYQMLBA3T+hC6XfAsMq1Gk0D8+T2VhnRDMR9QgzjFio9Imk2LuacjZTnYh+2a9GF/8MwzCM\nFFLpE8n7dPbZD/u9h8ss6su0FlEXNNzFPRHnQafGzA3RMAyjeKajbt2poQUVNRR1vyy02D6MzGJ7\nvrrZUxNHkjuMdxrneV3SAiJwSQuIwCUtIAKXtIAcuKQFROCSFhCBS1pABC5pARHEPm5W+kTyP9Rv\n/37UC+sKNL5Re3C4S1AjsiX69PANsF+BuqD7ApZFN2lNBw6tUKdhGIZRJeLIx3CvP7K5JPT+iBLq\nQt1ucpLhsGB3jbBhGIZh1AMpmdqSVpCzQebATRNB0pb0py1pARG0JS0ggrakBeSgLWkBEbQlLSCC\ntqQFRNCWtIAIUjJupocUdIgMBHkc5B6QQSCvg+yWtCrDMIwcpGDcTBcJd4hsDPIByBgQ70ota4LM\nBrGQ3IZRHp+RiQ1mR/nHZzn61wxJiIQ6RLqA/B7kQ5BNIj4/BeSuFE5xGUY9YANdPOTqR+vfEAl0\niPQGuRvkCZ3KiizTDeRFkP2jPzcMIw820MWDGZIiqXGHyFogM0DOAWktUHYlkI9BhtZCmWE0EDbQ\nxYMZkiKpUYdIAHIYyEcgO5RQbxTIw5n1E8MwisAGungwQ1IkNegQ6QXyT5BJICUGA5SuIE+CHFm4\nrGEYHhvo4sEMSZFUuUNkedSd90qQHmW2sbSf4lomXm2G0bA06kA3F1iihu2ZISmSKnaI7O4NQOEF\nc8dSOI7AcTOOLSLaOgLkKX1CMQyjAI060M0l3iCzhdqrmSGxuftOyDwgf0fT/W4KwZWdijjmw7Et\njgtxTEfzda8BPApcjWP7UI0L0bwq/1dd7YZh1IDl0ZwunwOvAVv78xOBA7LK7Qs87l8/5v+dhCbu\n2xnd9T4TGI1m8nwb2COrfqntJUYcsbYaCBkC3Iz+564BgQbNcnQBVgU298caaLbH+4DtgNdw3so7\nngTuwdGK4yZtN5jrn2yeB7kHgldr+EcZhhEfrcB4NP33JsBw4F/AmmQ2AkaxAfoEsTLwlj/XBvQD\n+qCJ/tZFg9w+B0wro73EMEPyM7IlcBUwFvgLLlgY2As1HJsBX6CRiscCj+L4JrIZx4s4NgPux9EN\nxz/0g+Ad9eLiGpBhEPxQ5T/IMBoYiWl6Jih10/AwoCdwpn//CHAXsHt7g2WI+AOabvwx4G5gV+C0\nMtpJDDMkum5xCl1+2JeRx57K2hcORNO1Lol+Se4D/oBjRtFNOl7BMQKY4I1J+/TYVcAOwBjgpBj/\nCMNoMko2AHERThEO8A6aPhxKX3/4HJgTaqvuwis1tyHZfs+1+PS061j8oT4MfbQbgeyLGo5jgKdx\n/Fh2247XcWwEPIRjHhwXQSAgvwFeBhkPwXOx/B2GYdSKD9BsrgEZozEEmIoamZ5ZZfsX0d5CwLzo\nGmp7W6/419+U0Z5RBqVZf0cvHL/CcT6/n/ddjlv4J/ZfbxJ/aN0Tx8JVUehYEscMHEdnTspu3q24\nTJdiw2ho0uy11Yom2xvlX7cBXwLLoNNRjwA90FS208gsjgN8CGya9b4NndI627c1HPjat0UZ7YWp\nK/ffkcAU9I8claPMef7zScBqRdQ9G82WOAm4DVggR7v5O8QR4FgFx/E4HsbxFSfzMDvtchcDnvuU\nrt9tlf9PiwnHEBzTcRyvJyQAuQnkzzW5vmHUF6kc6LL4BepR9QXqtbWtP98HXUf9Eh3wTybjXQWa\nOfYDdDprJ9SQvAf8HvXamgHsmVW+1PbC1I0h6Yqm0B2KWtRCOdvXIZOzPV/dTcm4Jp9JZmErTOcO\ncfTFsTuOq3F8iONNHBfg2Jpfjh0IchvIs95Dq3Y4BuGYiuMPXnpfNAT98JrqMIz0k7qBrkq00Xm9\nJU5qZkgqXSNZGzUGM/z7cah1npxVZhvgGv/6GWBBdK5v8Tx1J2TVfwbYMacCRwvqSbE5+oSzDHq3\ncD9wqt/nAciq6D6P+4HdIfi+hL+zchzv49gQXTPpBsFJODkEuBpkFQi+rqkewzCMmKjUkAyio0Wd\niT51FCoziM7eD1F1AfYHbsijof2R8D7gOOBJHCHXWtkfOAs4CoJ8bVUXxywcbcCDwDy4YBROtkdd\nig9LTJdhGEnREE9flRqSYjuhXFe9E4EfgOtzlhjLFXxL9t18lhGReYEL0CeWDSCYTNI4PsaxMfAA\ncC7dvziG7xZ8BWQzCB5IWp5hGDVjIjC4BtdpI525439mGPok0M5oOi+4Xwxk5zCfgu7mLFR3X+Df\nQPc8189jyGRpH7H3nxrBN2U4FsTxNI6L6TpnU5B3QRZMWpZhpICGuEtPAXWz2N6CusINBbpReLF9\nGJnF9nx1RwL/AfoWuH6ODpEdfO6QQ0lzulvH/Dgex3ElXb6/EOSawpUMo+FJ3UBXp9SNIQHYAt2M\n8yb6VAHqlnZwVpkL/OeTgNUL1AV1B34HeMkfF+a4dqhDpNVnL3xbsxnWAY6eOB5mTLdxdPnhTZDt\nkpZkGAmTyoGuDqmZIUnv3XpxCD//DTIIuBH17f41BJ8lJ6tEHPMCt/PloFbOfWt55nZbGYKPk5Zl\nGAmR9bs2KiBXP8bevw0SRl5GoPGx7ga2qSsjAuD4FtiW+d//lgN/+RWt31yS6ik5wzCMBkJAxviN\nfRsnLaZiHN04qesd7LPRV8w7+9dJyzGMhGjGqa2JdMw9Egd1tUaSJALyOMjApIXEhqOV3/W/n33a\nvmepe+JMy2kY9ULaB7oZQCU3rg64LnTuEXTPXJxYhsQS2BiCD5IWERuOH5lv1la0fD+FESc+y/G9\n0+e6bBjNTb41huaOqF6npP3OpXxWvXIedtrlE347cBqO+ZOWYxg1JM2/6+uAn9Cw71+h0TTmok8T\n76BTVBvSOYbWDGAEurXhe3Tj9FeoVyroE8kfgSfQII33o0EbK8GmtoqksTuk9auV2PrAOZzY/SUc\nCyUtxzBqRNp/12+TmdoaghqSq9Fw7935ORijjAS5X7cldKhzMnBtqM2J6DaIpXwbjwBnVKizboI2\nGtXkx/le5a7/nUzXH45klWsfwrEpjk+TlmUYieNiGgxdxW6w7fUdHTMdgj6lrI4ajnCd8HUFuBI1\nJgA3oQFv6wIzJGlHup7DHVdtR69Z37DUA4/g2ATHR0nLMoxEqdwAxE1oKisI0Ijkw4CHoEfQ2c50\nYlbW6zlA3ayPNsJie4MT/ARd9uEf967G54s/BkzE1V9OZ8NoIKKehrLPfQOtCwBPaqDYb/aHoB+s\nPH+e+nWNGZK6IJgGXRx/e2sNfmq5AXgUx6JJqzKMJmU2sGSez9+AoAeMmQy0Qq914DuBMcf7jcaz\n0BiD4aeqtD1lFY0ZkvrhQuAbTv3xR+AS1JgMTVaSYTQlZwBjgM/QpHvhmH9z4e8/wOm7o3mWvoa5\n78A8CwFHAjf7gp+iETl+rhh63XBPLmmlyTpaBoN8DLISjiNxzMDlvTMyjHqkzn/XsgfIXRHnl/S/\n31VrJaTE801LE3aI7AfyEkg3HAfheA/HskmrMowYqfPftdwBkiPEkewBMgWkZy2ElHi+aWnCDpEA\nZDzIqQA49vP54FdIWJhhxEUd/65lfpD/giyQp8zVIFfUQkyJ55uWJu0QGQAyG2RtABx74vgQxyoJ\nCzOMOKjj37XsDfKvAmV6gbwBslv+cpWLKfF809LEHSK7gkwG6QGAY2ccs3GsmbAww6iUOv5dy3iQ\nPYsot7rP4rp4NcWUeD5RRqJ52KfROV97O+f5zycBqxVRd2c01e5PdMyoGCaVHVI75EbNCOlxbIvj\nIxzDEhRlGJVSp79rWdBPaxUZG0+OBXnah1CpiqASzydGV3RL/1CglcI529chk7M9X93lgGXQeDNm\nSHIifUHeB9ng51OOLbwxGZ6gMMOohDr9Xcs+ILeVUL4LyD0gp1dLUInnE2Nd4L6s9yf4I5uLgV2z\n3k8B+hdZ1wxJQWRrkLd03tWjYVQ+xlWUM8EwkuIzMvso7Cj/yJUpNvZxs9INiYPoGGNmpj9XTJmB\nRdQ1ChKMRyOHnv3zKceDwE7AOBybJ6PLMMqmN5nAhnVySB+QL0HmK6PupmiW10Vi1tW7vO4vnUqD\nNhZr2aq59d9lvZ7oj2bjWOAVkM0huB8Ax6M4tgdux3EgjjsTVWgYjc12wAQIvi69avAgyHXA1SC/\ngiDuJ4Y2f6SWYXScnhpN5wX3i4FsN7cpQL8i69rUVtHIJiDvgXTMW+JYC8csHJYD3jCqhtwPsnMF\n9VtBngE5Jj5NuS9Wg2uURAswHV0w70bhxfZhZBbbi6n7CLBGnuunrkOSRS4ACSfMAcfyON7FUYsv\nqWE0GdIX9daqcLe6LIGGUMl38xwHqRw3twCmoh5Yo/25g/3RzgX+80l0fMKIqguwPbp+MgeNlHlv\njmunskOSQ3qCvAmyfaePHINxTMVxagpzORhGHSO/AbkxprZ2B5nawXkmfmzcDGEd0glZD+RDkIU7\nfeRYGMfzOC7C0TUBcYbRgMgEkB1jbO9KkKvia6/zBarYdl1iHRKJjAW5Fc190BHH/DgexnEjjm4J\niDOMBkIWAfmC9ggT8bTZyz+V7BFfmx0vEHeDlo+kMTkJWBbYvdMnji/RdatuwHhc/aTzNIwUsgNw\nDwQF8+gWT/A16qD0N103MaqNPZHkRNZAY/kMjPzY0YLjShxP4Wrnb24YjYU8DLJdldo+2ntyxR1C\nJfZxs94XXYX6/xuqiJyMhqXZKtI3XRfdx6JOD5vjeL+2+gyjnpH+wGRgAATfVaH9ABgPvAZBOOpH\nRQ1j42YH7IkkL9IK8gLIIXmLOUbheBvH0jUSZhgNgBzuNxJW8xoLg8zU3e/xNRpjWw2BdUhBZFnv\nm54/vLzjNzg+wHWIzmwYRk7kUZCta3CdEWhw1kXiajCmdhoG65CikJ1A3gbJvxbi2NFHDt4gbznD\naHpkIMhnIPPU6Hqng9wLEoeDlI2bIaxDikbOQUNW5/8iZiIH1+BOyzDqFTkK5OoaXq8V5CmQ38bR\nWAxtNBTWIUUjrSCPgZxUsKhjbYvPZRj5kCdAtqzxNRf309T5wkYV1VAschoI65CSkAF+rrVwaHmL\nz2UYOZBFQT4FSWBDr+yK5nufr5JGYpPTIFiHlIxsADILZEjBopn4XKdZfC7DaEeO0TAmiV3/cpBr\nKmkgNikNgnVIWcjvQJ4raqHQ4nMZRgh5qqin+updvyfIFJC9ym0gVjnU/6YU21hTFhIAtwAfQXBo\nweKO+YE7gI+BvXH8UF19hpFWZAjwAroJ8ccEdawKTACGQTC91MrEPG5arK2mJBBgP2BjkMIL6haf\nyzDa2Qm4PVkjAhC8DPwRGJfMWk1jYVNbFSErei+QVYoqbvG5jKZHnol5l3kFSAAyXqN9l1axKnLq\nGOuQipE9QaaBLFhUcUeA42wcr+EYVGVxhpEiZHEfCLUlaSUZpC+aYnuzUipVTU4FjETzsE+jc871\nds7zn0+CDiE4ctXtjc7/vQE8AOQa5FLZIfWHXAByR0m7Zi0+l9F0yPEgFyetojOyEcgHIP2KrVBV\nOWXQFU2TOxRopXDO9nXI5GzPV3cscLx/PQo4M8f1U9ch9Yl0A3kaJNeNQDQWn8toKuR5kI2TVhGN\nnAZyX5E3g6kbN9cF7st6f4I/srkY2DXr/RSgf4G6U4B269rfv48idR1Sv8hiaIrejUqqZvG5jKZA\nlvL7r1LqAi+tIE+qa3/hwnFfvVKvrUHAe1nvZ/pzxZQZmKduP2C2fz2bjFExqkbwHrA38E+Q4tc+\nHLcCewC3Wnwuo4HZGbgVgp+SFhJN8CP6OxxVMNJ3Fah00ahYy1aMz3KQoz0pcB2X9XqiP4yyCB7U\n9RJuBmmDoLj9Io4HcWwF3InjeBzXVlOlYSTALpD2cEHBDJDDgRtAVofgK/9Bmz9SyzA6Tk+NpvOC\n+8Vo/uF22qet8tVtn/4CGIBNbdUQ6eJdCs8tuWomPtexVRBmGAkhy/jF7JROa4WRS8mfcCt142YL\nMB1dMO9G4cX2YWQW2/PVHUvGqJyALbbXGFkIZDrIboXLhrD4XEbDIWNAzktaRfHIvCCvg+ydq0BN\n5RTJFsBU1ANrtD93sD/aucB/PglYvUBdUPffBzH33wSRVf1mxV+UXDUTn+tii89l1D/yCsj6Saso\nDVnZ/36j3PNt3AxhHVJVZD+QyWWFrHbMj+NhHDfisBAORp0iy6OpF+ownJQc7l2Ww7+/1HltGQ1N\ncBXwOHCFD/RYPBafy2gMdgFuhmBu0kLK4ELgfeD0pIWkHXsiqTrSHeQFkPI8VjLxuZ7G0SdmcYZR\nZeQ/IOsmraJ8pI8PoTIy+2RiclKKdUhNkMVBZpc9T5yJz/Ufi89l1A+yAsi79TmtlY1s6L3O2j1h\nbdwMYR1SM2QLkJlZX8bSsfhcRl0hp4Cck7SKeJA/gjzgjaKNmyGsQ2qKnAIysaLopxafy6gLJECz\nEK6TtJJ4kBaQJ0COw8bNTliH1BTpCnI/yFkVNePYGcdMy2lipBdZGWRGyU4mqUaGgHyEjZudsA6p\nOdLX/8C2r6gZx7neNbiBfqhG4yCnUXrCqDpAdsTGzU5YhySCrKV3NrJM2U04euB4HcceMQozjBiQ\nAOSNJIIf1ggbN0NYhySGHALyKkjPsptwrO5D0C8WozDDqBBZzYcIatSnZduQaKSGS4AXgYvL/sE5\nXgTOBa7G2XfRSA27ADdBYDeqRWI/XqNMAgEOBVYBDqmgobFAd+CoOFQZRmVIwM+GxGgW7I4hcWRp\nv16ydtlNOJbA8TGOFWMUZhhlIGuCTGvgaS2wcbMT1iGpQLYFeUc9usrEcQCOl3HME6MwwygRGase\nWw2NrZEYaST4F3ADcD3lJ/+5EniHjhkvDaOG2LRWs2JPJKlBWkAe1lAMZeJYBMeHOOos94PRGMja\nfjd7I09rQQrHzd7ABAonoBqJpsudRsdUvLnq9wYeAb4Czs9z/dR1SHMj/Xyk0a3KbsKxDY63cMwf\nozDDKAI5p6IbofohdePmWOB4/3oU0Slxu6IZEIcCrXROqRtVf15gPTTLohmSukLW85GCFy+7Ccdl\nOK6IUZRhFEC6+Ei/KyStpAakbtycAvTzr/v792HWBe7Len+CP4qpvy9mSOoQORrkRZAeZVV3zIdj\nOo7tYhZmGDmQdTX3SFOQusX2fsBs/3o2GaOQzSDgvaz3M/25YuqboahPzkOnK/PdBOTG8RXwa+Bi\nXOR3yjDixhbZK6AYQzIBeDXi2CZUToge+MPngjzlzHA0BIEABwK/BDmgrCYc/wauAC63wI5GdZEu\nwM6YISmbYvJKbJrns9nolNQsYADwUUSZ96FDLKVF/bli6xfCZb2e6A8jcYKvfaTRx0BeguDFMho5\nBXgaNUqXxSrPMDL8EvgMgslJC6kSbf5ILWPJeGGdQPRiewswHV1s70bnxfZ89ffF1kjqHNkF5C2Q\n8nKPOH7hd70vFbMww/DIeSBjklZRQ1I3bvYGHqSz++5A4O6sclsAU1HvrdFF1AeYAXyKugC/CywX\ncf3UdYgRhfwV5C7KzX3tOArHk7iinqANowSkK8iHFaVEqD9s3AxhHVIXSKtP81neXZ+jC44JOJrp\nrtGoCbKhTr1WAcf2OK7B0b0q7ZdP6ry2DKMIgh9Rr5jDQPKtuUXjmAvsBxyFo1GTDRnJUB1vLccw\n4FJgEWA8jnljv0aKMENi1IjgA2AP4DqQwSVXd8xEQ81f1+g/SqNWSFdgR+DmWJt1LAHcDuwDbA18\nCNyLY75Yr5MizJAYNSSYCPwFuBmk9Ci/jnHAS0Q7dRhGqWwIvA/Bm7G16FgIXR8+Fcc9OP6HOg1N\nBR7A5QwjVdeYITFqzdnAB8Cfyqx/OLAdjs3ik2Q0KbsAN8bWmqMbcBv69HFh1vm5aLin54CHcPSJ\n7Zopod43egn1/zc0IdIfmAwMgeDLkqs7NgGuBlbG8Vm82ozmQFrQG5p1IHi74uZ00+zVwALAjjh+\nylHmLDSI7aa4n6N61JrYx017IjESIJgFPAzsXlZ1x4PALcBFtuvdKJONgBmxGBFlDPALYM9IIwLg\nEHTf3O3ARNzPoaLqHjMkRlJcCvymgvqjgZUo1xgZzU583lqOPYEDgK1xfFOgrOA4GbgWeBTHkFg0\nGBVh+0jqFukCMgNkjbKbcKyO4yNchxA8hlEAaQX5BKTyQdwx3H8HVyyj7jE4ZuBYsmIdpWH7SIxG\nIZiLxs8q/6nE8SJwLnA1zr7LRtFsDEyD4J2KWnEsg7oO74njtTLqn4uumUzERUbuqBvsx2ckyVXA\nriC9KmhjLNAd3WNiGMWwK5VOazn6om6+Y3BMqKCdi4CTgIfLeqoxYsGmtuoeuQPkwIqacCzhAzva\nD9EogHQD+RRk0bKbcHTH8QSOM2KT5dgdxywcq8fWZm5sastoOC4FDqqoBcdbaPTof+AofaOj0Uxs\nAkyGYGZZtXUK9So0FcaJsaly3AAchu5BWSe2dmuEGRIjae4H+oOsVmE7VwLv0DE/jWGEqdRb64/A\nEGBfv9EwPhy3AfujsbmGx9q2kReb2moI5CSQCwuXK4BjERwf4lg/BlFGwyHzgHwGMrCs6o79cUzH\nsXDMwsLX2dRP1Y6o0hVs3AxhHdIQyKL+B96z4qYc2+B4C8f8MQgzGgrZGuTRsqo6RuCYjWPZmEXl\nut6G3phsWYXWbY3EaESCmcAT6LRDZTjuBB4C/lpxW0ajUd60luMXwA3Arjimxi0qxzUfBbZBXdu3\nr8k1E6Q3MIHoDIfZjASmANPIpNbNV39T4HngFf/vRjnatSeShkF+BfJULE055vNTENvF0p7RAEh3\nkM99nLficfTD8TaOX1dJWKHrr+G9uXaLsdXUjZtjgeP961FEh/fuiqbYHQq00jlne1T9VYH2//AV\ngFweFqnrEKNcpAVkJsjKsTTnWM//APvF0p5R58h2II+UVMUxL45ncAk7cDhWxvEBjn1iajF14+YU\n+PmH2t+/D7MucF/W+xP8UWz9AM3d3hrxWeo6xKgEOQXk/Niac/wJx3gL7GiAXA9ySNHFNb3zrTiu\nS8X3x7E8jpm4Cl3lldStkfSDn0Mhz4bIu79BwHtZ72f6c8XW3xF4AfixQq1G+rkC2AMkrgyIp6Df\ntco2PBp1jvQAtkRzhRTLWUBf4EAftTdZHJOBNuD3uPRFcSjGkEwAXo04tgmVE6ItXfhckKdc+PwK\n6HTXwUVEBNrDAAAXd0lEQVToNOqe4F3gaWDnWJpz/ADsBZyBY6lY2jTqkS2B5yH4qKjSjkPQ8W17\nHN9XU1hJON5Eszoejeuw1pw4LUWU2TTPZ7PRKalZwAAg6j/qfegQnXVRf65Q/UXRO4i9gXw5A1zW\n64n+MOqXS9F1s2tiac3xOo5TgWtxbICmPjWai+K9tRxboGPKeqQxaZrjHRwboJkWuwN/LOKJqc0f\nqWUsGS+sE4hebG8BpqOL7d3ovNgeVX9BYBIU9LpJ/pHTiBlpAXkfZIXYmtT57gk4xsTWplEnSE+Q\n/4L0LVjUsYrfu7FeDYRVhnqTvYrjzDLWcFI3bvYGHqSz++5ANDJmO1sAU1HvrdFF1B8DfA28lHVE\nfRFS1yFGHMipIOfG2qRjUTRvxJqxtmukHNkZ5P6CxRyDcLyLY9caiIoHR18cL+L4W4nGJPZxM3lv\nhMqwnO0NiQxF9w8tBsGc2JpVX/yTgTVwfBtbu0aKkVuAeyG4ImcRRy/gMeDmWCP61gLHQsC96EzP\nYUXG/7Kc7UYzEMwAnkM99uLDMQ59uo2agjUaDumFrvHenrOIoyswDniRevxeOD4HNkMdk670f0/N\nMUNipJXLqDS8fDSHA9vh2KwKbRvp4lfAvyHIt2j+VzQx2qGpcPMtB8eXaPSQRdFUClF77qqKGRIj\nrYwHlgZZvmDJUtA7uP3Ru7fesbZtpI38mRAdRwMjgJ1wdb5PzfENsDUwP3AjNc7LU+/rC7ZG0tDI\n6UB3CH4be9OaL3sAsFvd3okaeZD50Y3QQyH4vNPHjm2Bi4Bf4phRU2nVRA3IONRDdkcc30WUsjUS\no6m4HNhbA+7FzmhgJWD3KrRtJM/WwGM5jMga6Hdru4YyIoDfQLkL8CVwF47KUzMUgRkSI8UEb6GL\n4zvE3rRjDrrr/Vxchw2zRmMQvQnRMRi4EzgIx7O1FlUTdJpuLzQc1b3UIDePGRIj7VSe0z0XjheB\nc9GcD/ZbaBhkQTT1xJ0dTjsWQPe3nYPL48nVCDh+QtcCJwMP4HKm+IiFel9fsDWShke6Ae8CG0Dw\nRuzNO1qAR9E9BPFugixeQxegBzAv0DP0bwvw7xxz3UYk8mtgRwi2/fmUejLdjW6KPrxp1sV0o+K5\nwHBgMxyfUIVxs94HYTMkTYGcCXSF4LiqNO9YEg0W2YbjPxGftxA9yBfzbzFlugPfAd8A34b+7QYs\nDlwCXIRjVsx/fQMidwHXQ3A90D6YXoK6x27TdPHW9O8/A9gK2MR/h8yQZGGGpCmQpYAn0Z3u1YnG\n6jgAOA0NIBoe6FvoPMAX+reUsnPy7kh2LA8ciToG3IWu67wQ15/eWMhCwAxgUQi+AsBxPLAnsD6O\nr5LTliBqTE4CdkfzzpshycIMSdMgDwGXQnBjVZrXH9pq/l14oP8+FVMhGg7jQOAI1LX1b8DtTXeH\nnRfZD9gaAnXQcOwM/AVYF5cz02rz4BiF40zMkHTADEnTILsCB0EwImkliaNTbdsCxwCDgb8Dl6cy\n7HnNkXuBayAYh2NddMF9MxwvJSwsTdgaSQgzJE2DzIPeha8HwbSk1aQGx+rA0WgiphuB83C8nqyo\npJA+wFvAIFywCPBvNMPh3fnrNR1mSEKYIWkq5GxgLgSpyg6XChz9gUP8MQn11Lm/yGiwDYIcCGyO\nCw5C19QuwPH3hEWlETMkIcyQNBWyDPA4uuj+Q9JqUomGyNgNfUrpCZwHXIPj60R11QR5gO6fXcUJ\nfQ4CXsZxbNKKUooZkhBmSJoOeQT4OwS3JK0k1ajzwPqoQdkIuAq9Q5+RpKzqIQvD3Df5Q/fxdP2x\nFxpn6qekVaWUVI2bvYEJdM5uGGYkMAWYBh0S1ueqvzaZrIivQN6MZcl70hg1RnYHeSBpFXWFYyiO\ns3F8guM2HBuWkZ415cjhbHXwKzier1V8qTomVRkSxwKfkMm7vhCadz2brmiK3U2A99FkRbuj2/Zz\n1e8BfA/MBfoDrwH9IPLuIlWW1agF0h1ddF/Hx+IyikUzAf4aOAqYg7oPj6vbXfOOHsAv+Hi5DXl3\nvdNY5R9f0PL9Gjg+TFpayknV1NYUYENgNjrgTwSWC5VZF01tOtK/bzc0ZxZZf3E0p/uSOTSYIWlK\n5C/AdxD8PmkldYmGZNkMdR9eFY1ndlFqB2DN+rckGq05+1gM4Q2mbdmP7xd4mZVuOAyH3VwUJlWG\n5HP0KaK9nc+y3rezE7A58Bv/fi9gHXSXbr76a6NzuoujTzD/yqHBDElTIssDDwODIajvhERJk6Zd\n8zrdNgBYkY4GYzn0hvNVf7zm/30DJ8egIeM3gsDWRIoj9nGzpcDnE9CnhTAnht4L0fNu4XNBnnLZ\n559FcxAvB9yHPq38N4dGl/V6oj+MhiaYDDINHUBuS1pNXeOYDByG40R01/xtuBrsmtfQ5ivS2WjM\nJWMwnkCTT70eHdpEVgeOA9YyI5KXNn9UjUqnttrQ2EQDgEfoPDU1DB3o26e2RqNflLOKrA/wEHA8\nRN4l2RNJ0yJ7AXtBMLJgUaN4dNf8dqi3V+W75h3dgGVRI5FtNBYGXidjNNqPj4oLRyPzomPCqT8H\nZzSKJVVTW2OBT1GjcALqdRVebG9BF9tHAB+gTxrZi+1R9YeiCVn+BwxB9w2siGb8CmOGpGmRHuii\n+1oQvJ20moZEMwkejT755d81r9NSQ8gYinajsRQaRLF9Oqr9eLsy91y5EJgfgr3Kb6NpSZUh6Y1m\nIBuMflF2Ab4ABgKXoSGLAbZAd9l2Ba5Awxnnq78XalB+9MdJ6PRWFGZImho5F/gagjFJK2loOu+a\nPw8NZpltNNpv9sLrGJPj9wqTrYHzgVUgyDXlbeQmVYYkDZghaWpkBXQP0hAILAJutcnsmj8EdcfP\nNhqv1SZopPRH95jtAsHj1b9eQ2LjZgjbkNj0yBMg2xYuZ9Q/Emh0Xzk1aSV1jo2bIaxDmh7ZB8Si\nuzYFcgTIMyCtSSupc2zcDGEd0vTIvCCfggxOWkl1kMX0TrzZkRVAPgZZOmklDYCNmyGsQwxAzgc5\nJWkV8SObg/wPZHTSSpJF5gGZBHJA0koaBBs3Q1iHGICsBDITpNAG2zpCVgD5CGRnkLdB9k9aUXLI\nOSC32pNZbNi4GcI6xPDIU94ttAGQRUDe8psuAVkWZFbj/H2lIJv6m4Q+SStpIGzcDGEdYnhkP5Dx\nSauoHOkO8u/Onkmytl8j+GUyupJA+ngjsknSShoMGzdDWIcYHukJ8hnIokkrKR8JQP4BchNIl4jP\nR4LM9vtnGhwJQG7TaS0jZmzcDGEdYmQhfwc5KWkV5SN/AHnWx5HKVWZvkHfVm6uRkQP8Avs8SStp\nQGzcDGEdYmQhq/hBtmvSSkpHdgV5x+/cLlT2dyCvg/Suvq4kkKX9NF4TPHklgo2bIaxDjBDyLMiW\nSasoDVnHD5yrlFDnbJAn8z+91CPS6v8Pj0haSQNj42YI6xAjhBwIckfSKopHhoB8ULpHlnQBuU4d\nDBrK7fk0kLvN1beq2LgZwjrECCG9QD4HGZS0ksLIfCCvgBxbZv1WNPbUlY0x8MpwkA9B+iWtpMGx\ncTOEdYgRgVwMkvLQ8tIV5C6QSyozAtILjT91enzakkAWBJkB8quklTQBNm6GsA4xIpDV/aCU4kV3\n+SvIg8QSgFD6gkwBOarytpJC/okmqzKqj42bIaxDjBzI87rvIo3IIX7gXyjGNof4zXu7xddmrZA9\nQSY3nuNAaknVuNkbmAC8gSYXWjBHuZFofvZpwKgS6g8GvgZ+l0dDqjrESBNyEMitSavojGyKhjtZ\nqgptr4zG56qjneAy1HusrZa0kiYiVePmWOB4/3oUcGZEma7Am2ge9lbgZWD5IuvfguaJNkNilIHM\n5xfdByStJIMs7wf6Dap4jQ38Ndao3jXiQrqCPA5yXNJKmoxUjZtTgHbviv7+fZh16Zhv/QR/FKq/\nHWpoTsYMiVE2chmpCcEufUGmg+xTg2tt512Kq/DUEydyIshDRIaDMapIqsbNz7NeB6H37ewEXJb1\nfi/g/AL1ewFPAvNihsSoCFkLjaKb8EAl8/g77xp6VslB3nAVsVM+CWRtNG5YHcdGq1tiHzcLbWSa\ngD4thDkx9F6IFhc+F+Qp137eAX8FvqW4BPUu6/VEfxgGwPPAf4ER6Hc5ASQALgU+Amrokhxc6o3I\nvSAbQvBl7a5dCOkF/BM4AoKZSatpAtr8kUqmkDEyA4ie2hpGx6mt0WQW3HPVfwx42x+fA58Ch+XQ\nYE8kRgHkUJCbE7z+770HWQIeSRKoS608RKqCH8rlIFclraKJSdW4OZaMUTiB6MX2FmA6utjejc6L\n7YXqnwz8No+GVHWIkUZkAZAvktktLTuhQSQH1v7aP2voCnILGpo+BftqZAeQN9UZwkiIVI2bvYEH\n6ey+OxC4O6vcFsBU1HtrdBH1szFDYsSAXAEyqnC5WK+5VnrcWqU7yESQC0g0lIoM8usiw5LTYGDj\nZiesQ4wikHVAptVu0V0WA3kfZNvaXK8YZAGQl9VTKpHrdwGZQF3ni2kYbNwMYR1iFIEEaJKkjWtw\nrV5+wP6/6l+rVGSA92I7MIFr/xZNIdxAkYrrFhs3Q1iHGEUih4OMq/I1uoLc6ReTUxqNV5ZGI+xu\nU8NrruKn+Zao3TWNPNi4GcI6xCgSWdAvui9cxWv8GeRhkG7Vu0YcyJp+YF+/BtfqAfIfkL2rfy2j\nSGzcDGEdYpSAXF29KSf5Dcgb1E36W9nML3yvWOXrnK9Pgml9QmtKbNwMYR1ilID80g/2MQ9qMsIP\nykvH2261kT1A3gMZXKX2t0Tz0McY5diIARs3Q1iHGCUgAchrIG0xtrmsNyIxtllL5Bg0hHufmNtd\nBI33tWG87RoxYONmCOsQo0TkKJDrY2qrj3cr3j+e9pJCzgJ5GqRnTO0FaC75M+Jpz4gZGzdDWIcY\nJSK9/aJ73wrb6QbyqA7C9Y4Efv3obuLJ2HgoyAvpdzpoWmzcDGEdYpSBXAtybAX1A5CrQG6v3SbH\naiOt3pBcXdkakizvPcKWjU2aETc2boawDjHKQNb36wJlDpgyCuTF+KaC0oL0BHkKJCruXTH1u/l+\nOTheXUbM2LgZwjrEKAMJQF4HGV5G3R3Q3OiD4teVBqSPN7LHlFH3LJB/matv6rFxM4R1iFEmcizI\ndSXWWcNP29RBGttKkMHeLXj3EupsjMYXq+KGTyMmbNwMYR1ilIn08YvuRW4glEH+SWSH6upKC7Ki\nd2verIiyvb3hGVl9XUYM2LgZwjrEqAD5J8jRRZTr6ef+axyKPmlkfZCPQNbKUyYAuRnk3NrpMirE\nxs0Q1iFGBciGaByoPHP60sV7Z13ZnHP/sg0a5DHHrn3ZF+RVkO41lWVUgo2bIaxDjAqQAGQqyHp5\nypzl94s08Z4IORANPz8gdH5Jv2a0UjK6jDJJ1bjZG5hA/gyHACPRfOzTyKTWzVd/KDAHeMkfF+bR\nkKoOMeoR+Z3unYj87AB053rM4UPqETkRzbOygH/f4l2Fy/DuMhImVePmWOB4/3oU0TnXu6IpdocC\nrXTO2R5VfyjwapEaUtUhRj0iff2ieyiwoLT5xWbbWAf4p7fzQR7RaSxxIPfTMBsym4pUjZtTgH7+\ndX//Psy6wH1Z70/wR776QzFDYtQUGQdyRNb7pb0RGZGcpjQiXUFu9FN9szpPdRl1QqrGzc+zXgeh\n9+3sBFyW9X4v4PwC9YcCX6PTWhOBfMl3UtUhRr0iG4O84u+6e/t1k98krSqdyDzemPwqaSVG2cQ+\nbhbKnzwBfVoIc2LovRAtLnwuyFOu/fwHwGKoYVkduANYAfgqh0aX9XqiPwyjFCYCPYDh6PdpPASX\n5avQvATfA7smrcIoiTZ/pJIpZIzMAKKntobRcWprNJkF92LqAzyCGpQo7InEiAk53q+V3KlTOIbR\nsKRq3BxLxiicQPRiewswHZ2u6kbnxfao+n3RRXqAJYCZ5PYIS1WHGPWMLAxyDUivpJUYRpVJ1bjZ\nG3iQzu67A4G7s8ptAUxFvbdGF1F/B+A1dI3kBWCrPBpS1SGGYRh1gI2bIaxDDMMwSiP2cdN8wA3D\nMIyKMENiGIZhVIQZEsMwDKMizJAYhmEYFWGGxDAMw6gIMySGYRhGRZghMQzDMCrCDIlhGIZREWZI\nDMMwjIowQ2IYhmFUhBkSwzAMoyLMkBiGYRgVYYbEMAzDqAgzJIZhGEZFmCExDMMwKqISQ9Ibzeke\nTkwVZiSaRncamYyIheqvDDyFJrh6BZinAp2GYRhGShkLHO9fjyI61W5XNDPiUKCVzql2o+q3AJOA\nlfz7hcht8NKY2KotaQERtCUtIIK2pAVE0Ja0gBy0JS0ggrakBUTQlrSACNqSFhBBqhJbbQNc419f\nA2wXUWZt1JDMAH4ExgHbFqi/GfoU8qp//zkwtwKdtaYtaQERtCUtIIK2pAVE0Ja0gBy0JS0ggrak\nBUTQlrSACNqSFlALKjEk/YDZ/vVs/z7MIOC9rPcz/bl89ZdBLeZ9aM724yrQaBiGYVSZlgKfTwD6\nR5w/MfReiH5cCp8L8pRrP98CrA+sCcwBHkINysMFtBqGYRh1xhQyRmaAfx9mGPpk0c5oMgvuuerv\nClydVWcM8H85NLxJxgjZYYcddthR+HiTFDGWjFE4gejF9hZgOrrY3o3Oi+1R9RdCn0B6+PoTgC3i\nlW4YhmGkgd7Ag3R23x0I3J1VbgtgKmoFRxdRH2BP1PX3VaINlGEYhmEYhmEYRrzk2riYzXn+80nA\nakXULXbTZC017Qz8B/gJWL1EPdXSdDYw2Ze/DVggJbpO9WVfRp0uFkuBpnZ+h7ql906BJod6RL7k\nj5Ep0ARwJPq9eg04KwWaxpHpo7f9v6VSDV1rA896Pc8Ba6VA0yropvBXgDuB+UrUlAj5Ni62syVw\nj3+9DvB0EXWL2TRZa03LoS7Oj1C6IamWpk3JuIKfSenTidXSlf3lPRK4PAWaQA3afehgVIohqZam\nk4HflqCjFpo2Qm/iWv37hVOgKZs/o448pVAtXROBzf3rLdCxIWlNzwHD/ev9gD/mE5GWWFv5Ni62\nk72B8Rn06aJ/gbrFbJqstaYp6BNSOVRL0wQymz6fARZNia6vsur3Aj5JgSaAv5C5QSmFamoKytBT\nTU2HAmf48wAfp0BTOwGwC3BDCZqqqetDMrMACwLvp0DT0sDj/vWDwI75RKTFkOTbuFiozMA8dYvZ\nNFlrTZVQC037k7l7SYOuPwHvAvtQ2pNStTRt69+/UoKWamsCfWKbBFxBaVO41dK0NLABevc7Ed0X\nlrSmdoaj48H0EjRVU9cJwDno9/xsOjolJaXpP2SMys4UmFZOiyGRIssVc9dVzKbHYohTU1xUW9OJ\nwA/A9SXWq6auE4HB6N6iv5ZQrxqaegC/R6eSyqlfrX66CFgcWBW9uz0nBZpaUFf+YWh0iptSoKmd\n3Sn9Ow7V03UFcBT6PT8WuDIFmvYHDgOeR2cDfshXuNDO9lrxPh0t3mKodcxXZlFfpjXifPuj4Wz0\nEW4Wuunxo4Q0RdUth2pq2hedSx2RMl3tXE9pT0rV0LQkOp88Kav8C+gUQTHfrWr1U/a1LwfGF6Gl\n2ppmoo4boPPtc4E+wKcJagId87anPEeXaulaG9jEv76F0tYCq6VpKpl1m2WArUrQlBj5Ni62k71g\nNIzMglE5mx6T1NTOI8AaJeippqaR6KNs3xL1VFvX0ln1jwSuS4GmbEpdbK+WpgFZ9Y+ltLvtamk6\nGDjFv14GnbZJWhPod72Uxexa6HoR2NC/HoEa3qQ1tTtHdAGuRW8064KojYsH+6OdC/znk+h4R1HO\npsekNG2PzkvOQZ+U7k2BpmnAO2RcIy8sUVO1dN2Cbkp9GbgVWCQFmrJ5i9Ldf6uh6Vp0zWYScAel\nrQVWS1MravhfRZ/a2lKgCeAq4KAStVRb15roIvjLqMtttntuUpqO8uenAqeXqMcwDMMwDMMwDMMw\nDMMwDMMwDMMwDMMwDMMwDMMwDMMwDMMwDMMwDMMwDKMz/w+/L+1hDKh2sAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10af43990>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "batch_size = 1000\n",
    "sig_len = 10\n",
    "dt = 1e-3\n",
    "inputs = np.random.randn(batch_size, sig_len, 1).astype(np.float32)\n",
    "targets = np.cumsum(inputs, axis=1) * dt\n",
    "\n",
    "with nengo.Network() as net, nengo_lasagne.default_config():\n",
    "    # note that here we're using a different test input than the training input\n",
    "    input_node = nengo.Node(output=nengo.processes.WhiteNoise(scale=False))\n",
    "    ens = nengo.Ensemble(50, 1)\n",
    "    output_node = nengo.Node(size_in=1)\n",
    "\n",
    "    nengo.Connection(input_node, ens)\n",
    "    # initialize weights from distribution\n",
    "    nengo.Connection(ens.neurons, ens.neurons,\n",
    "                     transform=nengo_lasagne.init.GlorotUniform())\n",
    "    nengo.Connection(ens, output_node)\n",
    "\n",
    "    input_p = nengo.Probe(input_node)\n",
    "    output_p = nengo.Probe(output_node)\n",
    "\n",
    "sim = nengo_lasagne.Simulator(net, dt=dt)\n",
    "sim.model.train({input_node: inputs}, {output_node: targets},\n",
    "                n_epochs=1000, minibatch_size=100,\n",
    "                optimizer_kwargs={\"learning_rate\": 1e-2})\n",
    "sim.run_steps(sig_len)\n",
    "\n",
    "truth = np.cumsum(sim.data[input_p], axis=1)*dt\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(sim.trange(), sim.data[input_p][0])\n",
    "plt.title('Input')\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(sim.trange(), sim.data[output_p][0], label=\"output\")\n",
    "plt.plot(sim.trange(), truth[0], label=\"truth\")\n",
    "plt.title('Integration of input')\n",
    "plt.legend()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we run the network as above, `nengo_lasagne` generates the inputs by checking the output of the input Nodes.  However, the inputs can also be specified directly by passing a `dict` mapping nodes to input arrays (the same format used when specifying the training inputs).  This can be useful to run a batch of inputs simultaneously:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(1000, 10, 1)\n",
      "0.0015365\n"
     ]
    }
   ],
   "source": [
    "sim.run_steps(sig_len, {input_node: np.random.randn(batch_size, sig_len, 1).astype(np.float32)})\n",
    "\n",
    "truth = np.cumsum(sim.data[input_p], axis=1)*dt\n",
    "\n",
    "print(sim.data[output_p].shape)\n",
    "print(np.sqrt(np.mean((truth - sim.data[output_p])**2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As above, we can use the NEF to initialize the weights on the recurrent connection."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "checking network\n",
      "building network\n",
      "training network\n",
      "Simulation finished in 0:00:28.                                                 \n",
      "training complete\n",
      "(1000, 10, 1)\n",
      "0.00153751\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXsAAAEACAYAAABS29YJAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAHGRJREFUeJzt3XmYXFWdxvHvTdIBkiAQlkAIAsMiAkIgEGCE0LImKkJU\nUAQUmHEcRWFckG2UuIC4IIqM+jggiBsqKCMq+9DCgwkEhbCZsIUl7CTsqDjkN3+c06Roq7rrVp1z\nz71138/z9JPqqnvPfZ9O969unXvOuSAiIiIiIiIiIiIiIiIiIiIiIiIiIm3KArRxP/Ac8Arwd2B6\ngDZFRKRkFgMTU4cQEZHWRgVqJ8QnBBERiSREsTfgKuAm4IMB2hMRkRJaz/+7NnALsFvCLCIi0sSY\nAG086v99EvgV7gLtdQ2v3wNsEuA4IiJ1ci+waeoQg8YBq/rH44HrgX2GbGOFJmrPnNQBmpiTOkAL\nc1IHaGJO6gBNzEkdoIk5qQM0MSd1gBbmpA7QRNDa2e2Z/STc2fxgWz8GruiyTRERCazbYr8YmBoi\niIiIxBNq6GXVDKQO0MRA6gAtDKQO0MRA6gBNDKQO0MRA6gBNDKQO0MJA6gC9oIx99iIiZRe0dtb1\nzF5EpFZU7EVEakDFXkSkBlTsRURqQMVeRKQGVOxFRGpAxV5EpAZU7EVEakDFXkSkBlTsRURqQMVe\nRKQGVOxFRGpAxV5EpAZU7EVEakDFXkSkBlTsRURqQMVeRKQGVOxFRGpAxV5EpAZU7EVEakDFXkSk\nBkIV+9HAzcAlgdoTEZGAQhX7Y4A7AQvUnoiIBBSi2E8B3gqcDWQB2hMRkcBCFPszgGOB5QHaEhEp\nmG2ZOkERxnS5/9uBJ3D99f3DbDen4fGA/xIRSczWB+4AmwTZE4nD9DN8HU3qVOAhYDHwKPAicP6Q\nbdSPLyIlZUeAGdh+qZM0UdrauTvNR+OUNrCI1J39FOx+sFNSJ2kiaO0MPc5ehV1EKsJGA3vjeih2\nSRymJ+gNQERKyHYAuxNsIthzvviXSanP7EVEqmIf4ArIlgGPAFsnzhOVir2I1NW+wBX+8Txg54RZ\nolOxF5EaslWBacDv/RNz6fFiXwT12YtIydh+YFc3fL8t2MJ0eZpSn72ISJcau3AA7gDWdxdrpVM6\nsxeRkrG7wKYOee5/wWamydOUzuxFRDpnGwOrAbcOeWEePTzeXsVeROpmcMjl0MUbe35ETmzqxhGR\nErGLwA5r8vw6YM+AleUkuHK1s3KBRaRX2Riwp8HWbfH6vSVa8lh99iIiHZoOPADZYy1e79muHBV7\nEakT31/fkiZXdUHdOCJSEvYHsL2GeX0HsNuKyzOsytXOygUWkV5ka/jVLVceZpuxYC+Ava64XC2p\nz15EpAN7ANdD9tfWm2Qv426zumNBmQqjYi8idTF0iYRWenpyVUzqxhGRxCzztx9sY1ilvQvsN9Ej\njaxytbNygUWk19jmYEtc0R9x28lgT7W3bVTqsxcRyWlf4HLI2iig2SPAS8CmkTMVSsVeROpgpPH1\nQ/XceHsVexHpcTYWmAFclWOnnptJq2IvIr3un4FFkC3NsU/PjchRsReRXpe3CwfgT8AbwMZHyFNJ\nKwM3ALcAdwJfarKNRuOISEJ2E9iMDvab19l+wZSudo7z/47BffTZdcjrpQssInVha4M9C9bXwb7f\nADsufKb2A4RsLEQ3zkv+37HAaGBZgDZFRELYCxiA7O8d7NtTI3JCFPtRuG6cx4FrcN05IiJl0El/\n/SA/Iif55KogxgRoYzkwFXcD38uBfmBgyDZzGh4PNHldRCQwy3DF/pQOG3jQ/7shcH+IRCPo91+V\n8BngU0OeU5+9iCRgW4Pd192Zuf0S7L3hMuU7eMjGuu3GWQtY3T9eBdgbtzyoiEhqvgunnSUSWuq5\nyVWdehNuPOotwK3AsU220Zm9iCRgl4PN7rKNGWA3hMmT/+CJjtuxygUWkaqzVcCeB1uty3bGgb04\n/N2toilVN46ISBntBiyA7NnumsleAhYC2wfIlJSKvYj0om6GXA7VE/32KvYi0otCFvuemlwVk/rs\nRaRANhlsGdjoQO1tCvZQmLbyHThkYzqzF5FeszdwNWSvBGrvXmBlsCmB2ktCxV5Eek3ILhz8OP15\nwE7h2iyeir2I9BAbhTuzD1jsgR64mYmKvYj0kqnAMsgeCNxu5UfkqNiLSC8J3IXzqhuBqf5+tpWk\nYi8ivWRf3Oq7gWXPA/cB24Zvuxgq9iLSI2wCsCPw+0gHqPR4exV7EekVuwPzIXshUvuV7rdXsReR\nXhGpC+dVlR+RE5tm0IpIAWwhWMQFy2wU2NNgk+Id47UHDNmYzuxFpAfYhsBE3L01IsmWAzdQ0clV\nKvYi0gv2Aa70BTmmynblqNiLSC+INb5+qEqPyIlNffYiEpGN8atcrlfAsdbwd8AaE/9Y6rMXEWm0\nA7AEskfjHyp72h2LreMfKywVexGpuqK6cAZVsitHxV5Eqi72+PqhKj25Kib12YtIJLa670NfpcBj\nbgO2qIgDFXCMoCoXWESqwmaDFXlWj7vdoT0HtmbsA4VsrNtunA2Aa4A7gNuBo7tOJCLSvqK7cPC3\nO5wPTC/2uGmti7tZAMAEYBHwxiHb6MxeRCKwDGwxWIKRMXYK2OdjHyRkY92e2T/GiunJLwB/BiZ3\n2aaISDs2AcbiehaKVskROaFsBDyAO8NvpDN7EYnAjgI7N9Gx1wZ7xt/zNtpBQjYWahbYBOBC4Bjc\nGf5QcxoeD/gvEZFu7ANckObQ2ZNgT+G6rUN9suj3X6XVh7tA8h8tXteZvYgEZn1gz4KtlTDDD8H+\nJeYBQjbW7UeQDDgHuBP4RvdxRETasgtwN2RPJcxQ2RUwO7ErsBx3kfZm/zVzyDY6sxeRwOyLYKcm\nzjAN7PaYB4jYdhSVCywiZWc3gu2eOEMf2Atgq8U6QKR2o6lcYBEpM1vL99ePTZ0E7FqwvWM1HrIx\nLYQmIlWzJ3AtZC+nDkKFxtur2ItI1SRYIqElrYDZQN04IhKIZWBLwDZPncSx9cCWulzhG4/QZlSV\nCywiZWVbgt0fqbh2yO6P9OajPnsRqS3fhZOV6SSyEl05KvYiUiVF34KwHbWaXDWcMr0Di0hl2cr+\npiGrp07yWrYT2M0xGo7QZlSVCywiZWR7gf0hdYp/ZCuBvQg2PnTDIRtTN46IVEUZu3CA7G/ArcCO\nqZMMR8VeRKpiH8ozvn6oykyuikndOCLSJVsX7GmwUPfgCMwOArs4dKOB24uucoFFpGzsMLCLUqdo\nzV4P9njg8f/qsxeR2ilzFw7AQ8D/4W7PWls6sxeRLtgof9a8ceokw7OLwA4O2WDAtnRmLyKltw3w\nLGSLUwcZQaknV6nYi0jZlb0LZ1DtR+SoG0dEumBXg+2XOsXIbJyfXLVKqAYDtVOYygUWkbKw8WDP\ng62aOkl77CawN4dqLFA7gLpxRKTcdgf+CNnzqYO0qbRdOSr2IlJmJV0ioaVKLHcci7pxRKRDdifY\ntNQp2mebuDtphWksUDuFqVxgESkD2wDsSTfOviosA3sCbEqIxgK08aoQP8TvA48DtwVoS0Rk0D7A\nVZAtTx2kfZlR0q6cEMX+XGBmgHZERBpVZXz9UHMp8eSqbm1E6zN7deOISE42Gmwp2Pqpk+RnbwG7\nPkRDAdp4VYX6wkSkRqYBj0L2cOogHZgPbAs2NnWQRkWtDT2n4fGA/xIRaaWqXThA9gLYPcBU4MYc\nO/b7r1LbCHXjiEgwdh3YvqlTdM6+C3Z0t40EieKpG0dESsZehzsrvi51ki6UegXMTv0UeAT4G24B\n/yOGvK4zexHJwfYHq9Ks2SbsDWDdLslcudpZucAikpJ9G+xTqVN0x0aBLXP3zu28kWBxClK5wCKS\nkt0Dtk3qFN2zS8EO6KaBYFFQn72IlIptAoynN2bkl2oFTBV7ESkTv8pl1gs9AqVcNiGmXvhPE5FC\n2K/A3pc6RRi2ur/xSqfzmSpXOysXWERSsD6wZ8DWSZ0kHLsDbLtOdw6ZRN04IlIWOwH3QfZE6iAB\nlaYrR8VeRMqiwksktNSTk6taUTeOiLTBbnArRvYS2xrsrk53DhqlAAZ2qruDi4hIMzYR7DmwlVIn\nCctGgz0LtmYnO4dMUlQ3zh7Aee4CjIjEZxu7KfuVsSdwLWR/Sx0krOwV3JLHyfvtiyz2E4HfgK1a\n0DFFasYysBlu+CLzgevATgFbOXWyNuwLVHw9nJZKNbkqJv9RxMaAfQ/sj12uFyEir2Ergb0f7E9g\nC8E+DDbe/Z3ZL8HuBCtxsbEM7EGwLVInicPeBnZlJzsGjxJZQ2DLwD4Ldh/Y5ukiifQCW8f/PT0K\ndjnYLLcA12u2ycAO9Nt8HWxcmqzDsS3AHujd63q2lu+3H513xyhxImoS2P7V//LtVHwckaqzbcDO\nAXvaf1reqo191gL7sV9krD96xFzsGLD/Tp0iLrvLjczJt1OUKBG1CGxvB3vS/Ssiw7NRYPuBXQ32\nMNiJroDnbucdYEv8MsIluX5mvwV7d+oUcdn57iQ3305RokQ0TGDbyZ/h5/0hiNSETQD7KNjdYPPd\nujHd3sjaVgc723edJL71n63kh1xOTJsjNvuw+zSWb6coUSIaIbBtBnYv2Mm922cnkpdtBPY1sKVg\nvwB7c/i/D9sH7H6wc8HWCNt22xn2AJuX5thFsu3A7si7U5QoEbUR2CaB3eT67TpeIU6k4izzRf1C\nX+S/CrZh5GOuCnaW7xraP+6xmh7/NLDPFX/cotkY3AqYq+fZKVqcSNoMbBPALgP7Ddj4uJFEysTG\ngh3iu2nu9t02Bfen2wx/7AvA1i7wuDe7N7g6sAH3aar9HWIliSVHYOsDOw+3RkaBv3AiKdhaYCf5\ns+qr/aCFhIsT2jj/aeIxsPfG71a1SbgRRTWZWW+ngX02zw7RokSSM7BluFl/d4H9U5xIIinZVrgh\nk0+7i3Zlu9+q7eT6l+1isMkRj3MI2C/jtV82dgDY7/LsEC1KJB0Gto+APQI2LWwckRRsFNhbwa7w\nI9A+Q6lv0mErgX0B7Amww+Oc5dv5YP8evt2ysnXBluX49FaXYg9gs3Fj8RMPDxPplI3HDbtbiFvO\n4DAqtbKjbef71S8Le7HYMt9dVLNP77aY9heoK12xnwksBO4GjmvyepeB7c1gj4O9v7t2RIpkrwf7\nMthTrqvCZsTvA4/F+nCTuJ7yb1wBrivYNu6CcN3YT8E+0O7GUaPkNBq4B9gI6ANuAd44ZJsAge2N\nuPHAJ1b3D0bqwXYB+5n/uH5Gb5252pZg8/yokk27bOtYsP8KEqtS7Giw77a7cdQoOe0CXNbw/fH+\nq1GgwDYZbIH7Bcm9oJBIRNbnR6/Mw00QPAbsdalTxWGjwT7hz/I/3vnfol1JknH9qdl0sFva3Thq\nlJzeDTQuYHQo8K0h2wQMbKvhhqj9CmyVcO2KdMrWwU0IvNYVr7qciNim/gx/rjvjz7XvONwEox59\nQxyOjQV7kfbmUQQt9t3OVm03zJyGxwP+qwPZs2CzgPOAq8D2g2xZZ22VgfXhbrJ8KLAr8BfghSFf\nzzd5rtXzg8+9BFmpzgp6k22Mu0H2BcDJ9fqZZ/eA7QF8CLgW7OvAVyH7exs7zwBuhuy5qBFLKXvZ\nn9nvAFwz5MV+/xXnyF3uvzOukM/0358ALAe+3LCNBTjOEDbKH+Pt7tjZA2Hbj8ky3M/tEOAg3IXt\nH+GKxhhgArCq/7fZV6vXGp9fGXiJ7t4wXgCehOzhSD+IirNtgd8Cp0F2Vuo0admGwPeAtYEjIRuh\nm8K+DiyF7JT42crITgeeguxLI21IwNrZ7Zn9TcBmuAu0jwDvAQ7uss02ZMuBY8EeBq4HextkC+If\ntxv2BlyBPwR4GVfgd4JscYRjjQbG0d4bx+uAyU2enwC8Huw03Blbjc5aR2IzgAuBj0L289Rp0sse\nAJsJHA5cCfYd4JRh7ie7D3BEUelKaB5wWOoQnZgFLMKNyjmhyeuRi4QdhJv4sUfc43TCJvmLdfNZ\ncaegadUZUWRTcLeR/AGVGhsekx2Am/uxV+ok5WSTwf4HNwO3yc2JbIq/uFuTaxvN2BRfs0aqA5U7\nwSogsPX7H14BnypGYhNwE2cuw02H/wHY3tX95bbxuCV2r6fUMz6L8Ood1nZInaTcLPOjkx7DrbXT\ncCtEOwLsgnTZysKWMPKwXBX7Fod5E9hDYJ8s/szZ+nBT4X8C9gzYJf6XvYT3++yEjQL7PG6uQ8nW\ncSmCZbg5HovRvZNzsHVcYbe7fdcXuElFR6bNVQZ2IdghI21USJSACgxsG/iPj2cQffVAy8B2BvuW\n/1TxB9x6Ph3cKq4q7L2+C6NG46NtFNg3wW4l6qJgvcxm49a5Osv//kxJnSg9+6SrHcNvVEiUgAoO\nbGvgxjz/DGzlCO1vDvY53I2bF+IWtNok/HHKyqb7j6DHVefaQ6dsrP+0di35bjoh/8DWwN0R66bU\nScrB3gw2f6SNCokSUILAtrLvZx4I80dqk3DTnG/0/ZBnuH7bXi92rbzmwm2EN9QysAlgl+OW+dUE\nvmDq+jczlK2Cm1w13O+Win2bhx0NdibYbZ19bLQJYIeCXer74c/H3bNTt00E3PUI+4XvvpqUOk1Y\ntrZ/Yz9H/98Sj90ItutwGxQWJZCEgS3DLbj0INhWbWw/BmwW2I99gf8N2MHoNokt2CjfpXU/bpJR\nD7ANwRaBnaqzUInLzgT71HAbFBYlkBIEtkNxyyTv1uS1DHdnnjP9NnPBjkK3RczB3uMvvB2QOkl3\nbGvciK5jUieROrCDwS4aboPCogRSksC2N27UzLv995uBzcENC1sE9lm6Xra1zmxHf+H2hGqeEb96\n34T3pU4idWEb40Yptfp7KUntbF+JAtt2uJs73+IvtH7TF6kKFqcysvXdaAv7YbUu3Np+6I5oUjjL\n/AnGBq02KDROACULbFPA9tSFt1hsHNjPfXdYBS7c2hH+jX966iRSR3Yx2EGtXiw0SgCVCyzdslG+\ni+wBsKmp0zRnGdin/cXlLVKnkbqy43GrgDZ9sdAoAVQusIRiB5Xzwq2NAjsd7HY0m1OSsn43fLn5\ni0UmCaFygSUk28GPcCnJhVvr89cUrgebmDqN1J2Nx02uaraqbOVqZ+UCS2i2Pm6Z5x+lvXBr48F+\nh1uorkcWqZPqs5tpuhx02NoZebEwEfB3u9od6AOuAVu3+Ay2JnAV8ATwTsheKj6DSFPzcHevqzyd\n2YtnGdjJxV+4tQ3A7gT7cjm6kkQa2ftpvsZ/5Wpn5QJLbHagv3D7zgKOtSVuuYxPxj+WSCdsczcq\n7B9fKDpJtyoXWIpg0/yF25PinW3bLn4MfSXv9yl1YRnYUrD1hr6QJE4XKhdYimKTcSv/Rbhwa7P8\np4dZYdsVicF+CzZ76JNJonShcoGlSLaK66+0eeEu3Nqh/ox+lzDticRmn3HXlF77ZJIoXahcYCma\nZbiF6B4E267Ltj7h29kyTDaRItjeYL8f+mSSKF2oXGBJpZsLt5aBfcWPumm1sJRISdlqYM+7SX8r\nnkwWp0OVCywpdXLh1sbg7m8614+nF6kgux1s+8YnkkUZ4kDgDuAVYPthtitNYKmKVy/c/oQR7/9q\n4/yM2N+hO4pJpdnZYEc1PhGy9W5m0N4GzAauDZRFxMsewc24zYCBJkPSPFsDuAJ4BtgfshcLCigS\nw1xKPpP2GnRmL1FY5kcpNLlwa+v7j72ng2nZD+kBthXY3Y1PJIvSgoq9RGbv8hdu3+W/38LNOLRP\na/kD6R02CuwZVtz/OmjtHOluTVcCzcY+nwhckuM4cxoeD/gvkTZlF4EtBi4G68ddLzoesvOSxhIJ\nKpsBRy2Fm78J3JU6TTM6s5eC2HpgvwbbL3USkTjs82BfHPwmaZQmrgGmDfN66QKLiJSTzQK7evCb\npFEazAYeAv4CPAZc2mK70gQWESk3WxPsObDRVLB2Vi6wiEg6tgjsTZRonL2IiIQ3Fwi+iJ+KvYhI\nuVT2NoXqxhERaZtNdQv6Va92Vi6wiEg6NsatgKk+exGRHpb9H3Bu6hSd0Jm9iEh+OrMXEZF8VOxF\nRGpAxV5EpAZU7EVEakDFXkSkBlTsRURqQMVeRKQGVOxFRGpAxV5EpAZU7EVEakDFXkSkBlTsRURq\nQMVeRKQGVOxFRGpAxV5EpAZU7EVEaqCbYv9V4M/AAuCXwGpBEomISKnszYo3i9P8VzNlvFNVf+oA\nTfSnDtBCf+oATfSnDtBEf+oATfSnDtBEf+oALfSnDtBEae5UdSWw3D++AZjSfZzC9KcO0ER/6gAt\n9KcO0ER/6gBN9KcO0ER/6gBN9KcO0EJ/6gCxheqzPxL4XaC2REQksDEjvH4lsG6T508ELvGPTwJe\nBn4SMJeIiASUdbn/4cAHgT2Bv7bY5h5gky6PIyJSN/cCm6YOATATuANYK3UQEREZXjdn9ncDY4Fl\n/vu5wEe6TiQiIiIiIvHNBBbizuqPa7HNmf71BcB2bew7EXch+C7gCmD1kuQ6ENdN9QqwfUkydTuR\nLUamL/htbwGuBjYoQaZBn8QND55YgkxzgCXAzf5rZgkyAXwM9zt1O/DlnJli5bqAFT+nxf7f1Jmm\nAzf6LPOBHUuQaVtcj8qtwK+BVXNmamk07mLrRkAf7o/7jUO2eSsrhmDuBMxrY9+vAJ/2j4+j9eSs\nonNtAWwOXEP+Yh8rU7sT2YrM1PgL9jHg7BJkAvemcxmuWOQp9rEynQx8IkeOIjK9BXei1ee/X7sk\nuRp9DfjPEmQaAPb1j2fh6kLqTPOB3fzjI4DPDxcizzj76f6g9wN/x7377j9km3cAP/CPb8Cdpa87\nwr6N+/wAOCBHppi5FuI+bXQiVqZuJrLFyvR8w/4TgKdKkAng66w4icgjZqZOr5HFyvRh4Ev+eYAn\nS5JrUAYcBPy0BJkeZcUn6dWBh0uQaTPgOv/4KuBdw4XIU+zXBx5q+H6Jf66dbSYPs+8k4HH/+HH/\nfR6xcnWjiEx5J7LFzHQK8CDwAfJ92oiVaX///a05ssTOBO6TzwLgHPJ1V8bKtBkwA3cWOQDskCNT\nzFyDdsPVhHtLkOl44HTc7/lXgRNKkOkOVhT+AxmhCzVPsW93nYZ2zl6yFu1ZjuM07tOObucU5BE7\nUycT2WJmOgl4PXAecEaO/WJkWgU36e/kDveP9XP6DrAxMBV3lnh6CTKNAdYAdgaOBX6ec//Yv+cH\nk3+yZqxM5wBH437PPw58vwSZjsSNgLwJ96n65eE2HmkGbaOHee07xwa4d5nhtpnit+lr8vzgx6DH\ncR9XHgPWA57IkSl0rmb7diJmpsNx/Xt7lijToJ+Q79NGjEyb4Po3FzRs/0fcx+F2frdi/Zwaj302\nK2agtyNWpiW4C/3g+n+XA2sCSxPnAlebZpP/elmsTNOBvfzjC8l3bSpWpkWsuI6wOfC2HJmGNQb3\ncWoj3Pj6kS4y7MyKiwzD7fsVVlxhPp78F2hj5Rp0DTCtJJm6mcgWK9NmDft/DPhhCTI1ynuBNlam\n9Rr2/zj5zlhjZfoQ8Dn/eHNcF0UeMf//ZpLvImjsTH8CdveP98S9OabONHhBfRRwPu5EMJhZuHeT\ne1jRZ/Uh/zXoLP/6Al77rtxsX3B/iFfR3dDLGLlm4/rK/oL71HFpCTLdDTzAimFp3y5BpguB23C/\nhBcB65QgU6P7yD/0Mkam83HXEBYAF5P/2lSMTH24N+fbcJ9++nNmipUL4Fzg3zrIEyvTDrgLp7fg\nhjs2Do1Mlelo//wi4NSceUREREREREREREREREREREREREREREREREREetv/A/zEglEbE2DJAAAA\nAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10d763b10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZEAAAEACAYAAAByG0uxAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8VHX9x/HXAS6ioiAKIouhuWGUK4jgct0SsQQzszJN\nrV8kP2lXLH/lt8U0TSu0zNzSytwTt1xz3BEXQERRUBFBRcUdNbbv74/PGe7cYfZzZs6Ze9/Px2Me\nzD1zvt/5ON57PnO+K4iIiIiIiIiIiIiIiIiIiIiIiIiIiEgHMwaYC8wDJhc5Z0r4+ixgpwrLTgKe\nAZ4CfhNjvCIikhJdgfnAEKAFmAkMzTtnLHBr+Hw3YFoFZfcB7gyPA/SNPXIREYmsS8TyI7BEsABY\nAVwJjMs75xDgsvD5I0BvoH+ZsscDp4fHAd6IGKeIiNRB1CQyEHg55+dF4bFKzhlQouzWwF7YXUsG\n2DVinCIiUgfdIpb3FZ4XVFlvN2AjYCQwHLga2LLKOkREpM6iJpHFwOCcnwdjdxSlzhkUntNSouwi\n4Prw+aPAamBjYGnO+fOBT0aIXUSkM3oe2CrpILK6YQENAbpTvmN9JG0d66XKTgB+Hj7fBlhY4L0r\nvQtqJJd0AEW4pAMowCUdQAEu6QAKcEkHUIBLOoAiXNIBFOCSDqCAWK+dUe9EVgInALdjo60uxobl\nTghfvwBLIGOxO4dlwLFlygJcEj5mA8uBoyPGKSIidRA1iQD8O3zkuiDv5xOqKAs2KuuoiHGJiEid\nRR2dJe1lkg6giEzSARSQSTqAAjJJB1BAJukACsgkHUARmaQDKCCTdABSXBr7RERE0i5VfSIiInF7\nCxviL9G8DfRJOog0052ISMekv+14FPscY/181SciIiI1UxIREZGaKYmIiNSFPwD8NklHIcWp3VSk\nY+ogf9v+AfB/TjKAKo93OvogRDqmDvC37dcBvwz86+Czo2BXE+9CsuXqU8e6iEiT2gl4FngR2Dfn\neLUrmpcTd31VUxIREanOUGwm+tvY9t2fD49ngG+Ez0fDz96CIX2BI4D7wuOzgPeBw4FWbMXyH2Mb\n770IfDXnfXLrAzgGuD98Xqi+RGiyoYhI5VqAm4CLgP2BPYGp2MZ5nramolHw0nxYsgEwHvxmEHwM\nfAZ4ITynFdgU2+ZiALA7tmDto8C8vPry7YU1Z+XWlwjdiYhIE/I++qMmI4H1gTOwlcjvAW4GvhK+\nHoAPgNHw2Dz4eDm2OvkBJer8Kbbo7H3ALdidS9PQnYiINKEgqb6A/G29AV6ibWtvD2wBrIJnspvo\nXQ18qUh9bwMf5dW1WTyhNobuREREKvcKtgtrbhL7BLaD6zLsLmUU8BD4/uHr19LWb5JvI2C9vLpe\nCZ9n68vqTwopiYiIVG4a8CFwEtY/0gp8DvgntjvrF+DdveCOZ1nTKR68AjwJPd6h8JbePw/r2hM4\nGLgmPB7Wx7rYdrbfyCu3pEh9UqEOMJZcRApI+9/29tjIqXew0VnjwuMbA7fD+qug1wzgVNaMovLH\nw0+nY3cZbwNfxBLQy8BPsNFZC4Ajc94nrI/3sFFZOfUBtoNsbn35NNmwDH0QIh1TE/9t+17gPwDf\nkne8H/h3wOc2XbWydv9KrMFUebwmas4SEYnPbsDjEKxofzh4HRu6OzaBmOoqjiQyBpiLjWueXOSc\nKeHrs7CZnOXKOmwSzozwMSaGOEVE6i3sVC/oKtYevtvEd13x6ArMB4ZgHUMzsdmcucZiE2jAsvS0\nCsqeCvygzHt3+g9fpINq4r9tfyf4zxV5rQ/4d8H3bFQwVR6vSdQ7kRFYIliATZa5krZOpqxDgMvC\n548AvbGhauXKJr4mjIhI5XxX7Ivyw4VfD94CHqT4cN+mFDWJDKR9x9Ai2ibdlDsnf9JOftlJWPPX\nxVjiERFJs2HAYgiWljin1MTDphQ1iVR6W1TtXcX52KzPHYFXgbOrLC8i0mijKd4fknUDsC/4DRsQ\nT0NEXfZkMTZ7M2swdkdR6pxB4TktJcq+nnP8ImzBs0JczvNM+BARScIobC2tEoJ3wN+LNd3/rQEx\ngQ0lbm3Qe1WtG/A81jnenfId6yNp61gvVTZ37ZjvA1cUeO8m7nwTkRKa9G/bvwh+uwrOOxL8zfWP\np3kmGx6Ebb4yH1sXH2wm5YScc84LX58F7FymLMDlwJPh+TdgyyXnS90HISKxaMK/bT8A/FLwFXQR\n+A3CUVob5RzMsPayJpGDqvJ4p6MPQqRjSvvf9gLa71YI+MOquLtw8NmXwB+Xc+we4Lgi59eqKYb4\nioh0Np61BwuNxobvVtjP/PJLNNm+IR1R2r+tiEht0vy3/TdgFbaS7/vAicBqOOd56PEa1iy1N2uv\nibUA2A9bfeO/wHLo6aHb7PD1e4BfAA9gCy7eji3AGIWas8rQByHSMaX9b/tF2pqzPgGshqNWwC/7\nAD0ovLBibplTgcvBXwk+23ecwfqGtwrruAc4PWKcDUki2tlQRJqPi+FC6GJZFSOs4ydzYOhbVZQJ\nsLW0JgEXYBf2S7BEAjYp8ZAY4qs7JRERaT7xJIAYbXlvDYVuAy61HRADgNdyXvsIaNQaW5GoY11E\npDp5d0FBgHWqZy2j/Za3XYG+a5cPPgJuBg6LP8TGURIREalOzra03cNDH+Qud/Ic1q8xFluZ4/+A\ndXJefw2bZJ1t0sqO0krZ3VVllERERKpzOpYY3oItJti1f+NXcl5/F5iILdm0CPiA9h3t2T3Ul0IX\nBwyDHt1pf4fjSf8Ag6anD1ikY2qiv21/LPhCyzJVU8el4L8XTzztK67yeE10JyIiUrtSOxlWqtCO\nh9IATfRtRUSq0ER/2/5p8DuVP69kHS3g3wD/iXhiaqu4yuOdjj4IkY6pSf62fR/w74GPYaqE/wv4\nE6PX077SKo93OvogRDqmJvnb9mPB3xVTXfuBfyyeutoqrfJ4TdQnIiJSm0p2MqzUvcBg8FvFVF/D\nKImIiNQmjk71ULASuA44PJ76GqcpJ7eECi3HLCLN7y1go7JnSTlvA30KHNe1M9Qk7aYi0vH4XcHP\nLn9eVXV2Bf8K+G3jrXftN4qzMjVniYhUL8amrKxgFTabXXNGGkR3IiKSEH8V+KPrUO8o8HPir7f9\nm9S5/qqNAeYC84DJRc6ZEr4+C8idmFOu7A+B1RRv1xMRSYB/uT4jqXwX8AvBD4u/7rY3qWPdVeuK\nbaIyBFutciYwNO+cscCt4fPdgGkVlh2Mrbf/IkoiIpIafjD4JeDr1Dntfwv+l/Wp294gzsqi9omM\nwBLBAmAFcCUwLu+cQ4DLwuePAL2B/hWUPQc4KWJ8IiJxC+eHBPX6IhuupVWvJBWvqElkIO2XOF4U\nHqvknAElyo4Lf34yYnwiInGrrFPdsTWuYCtKOY9hu87uWEPZhou65kulmbiajLou8BPggArKu5zn\nmfAhIlJPo7G90cu5GLgD+FV11Qce/NXAl4AZ1QZXQGv4qIuoSWQx1neRNRi7gyh1zqDwnJYiZT+J\n9ZPMyjn/caz56/W8ul3NkYuIVM33BLYDnih5mqMfsAewnKqTCGBNWteB/0kMzWYZ2n/BPjVifbHq\nBjyPXfS7U75jfSRtHeuVlAV1rItIavh9wD9Y9jTHN3DcjuN9XLutcSt9nwD8c+CH1xBk2crjrCzq\nnchK4ATgdmy01cXAM8CE8PULsAQyFutEXwYcW6ZsPiULEUmLShddHA/8FdgYa0W5v7q3CbzNReEI\n4NHqyjZWU/T+F6H1X0SkwfytwIUQ/KvoKY4NaGvG/ynwLo4ahuz6YdiX8CEQrK4h2KIVE+O1U8ue\niIhUxHcBdqf8nciBwEM43gXuoeZO7eAp4H2sGyC1lERERCozFFgKwZIy5x0K3BA+fwDYrbZ+EaAJ\n9l9XEhERqUz5+SGO7sBBwNTw53exvt7danzPq4DDbYXfdFISERGpzGig3MisVuBZHK/mHLsH2Ke2\ntwyexaY27FFb+fpTEhERqUwlM9XHA/md7hmiTfZLdZOWkoiISFm+L9APKL5Mu6MLtmTTDXmvPAAM\nx9Gjxje/GjgMfNQpGXWhJCIiUt4oYFqZobbDseG8z7U76ngPSz41jrIKnsfWGdy7tvL1pSQiIlJe\nrU1ZWRk6aJOWkoiISHmVdKrnDu3NF2G+CGBNWoeCb4lQR10oiYiIlOTXwZZln170FMd2QE9sGfdC\nHgR2xbFubTEEL2FLR+1XW/n6URIRESltJ+A5CN4vcc544AZckbX+HO8Ds4k2+zyVTVpKIiIipVWy\n6GKppqysDDXPFwHgGmBceGeUGkoiIiKlle5UdwwEtgLuLVNPhkj9IsFi4Cngs7XXET8lERGRonxA\n+U71ccCtOFaUqexBYGcc60UIKHVNWkoiIiLFbQGsAhaWOKfU0N42jg+AJ7GVgGt1HfA58DV20MdP\nSUREpLhRwINFt6h1bIR1lt9eYX0Rh/oGr2Fb846pvY54KYmIiBRXrlN9LJDBsazC+jJE61yHlDVp\nKYmIiBRXbqZ6ZU1ZbR4CdsSxfoSYrgPGgI9SR2yURERECvIbAp8EZhR82SYOHgDcXHGVdscyk0j9\nIsGbwCPAwbXXEZ84ksgYYC4wD5hc5Jwp4euzsIk75cr+Mjx3JnA3tlexiEgjjQQeh6DYqKv9gJk4\n3qiy3gwdrEkriq7YVPwhQAt20R+ad85YbLN5sN29plVQdoOc8pOAiwq8d+GOLhGRWHgH/vSiLzsu\nxvG9qqt17Icruw5XGX4j8O+C36D8uWsXjvbe7UW9ExmBJYIFwArgSmzMdK5DgMvC548AvYH+Zcrm\nLi/QE3gzYpwiItUqPj/E0RX4POVnqRfyMLADjp61hxa8DdyPXV8TFTWJDMTWuc9aFB6r5JwBZcqe\nho3N/jpwRsQ4RUSq4LtiLScPFzlhFLAYx4Kqq3Z8iPWzjKo1ulAqmrSi7pRV6W1RUEPdp4SPk4Hf\nAccWOMflPM+EDxGRqIYBiyFYWuT1StbKKiU7X+SOCHVMBf4IvjcE75Q4r5Voy9CXFDWJLKZ9p/dg\n7I6i1DmDwnNaKigLcAVtfSr5XBWxiohUqvj8EEeADe0dH6H+DNbaEkHwHvj/YN0Al5U4MUP7L9in\nRnvf9qI2Zz0GbI11jnfHbq1uzDvnRuDo8PlI4B1gSZmyW+eUH0exIXYiIvVRan7IZ8J/Z0eo/2Hg\n09H6RYCUNGlFdRDwLNZJ/uPw2ITwkXVe+PosYOcyZQGuxf4HzcQm1vQr8L4anSUideJfBL9dwZcc\np+I4O/JbOO7FRV2+xPcMR2ltXE2haO/ZceiDEJE68APALwVfuKXGMQPHnpHfxvFzXByDhvw14L9Z\nTYHo79lGM9ZFRNrbHXgIgtVrveLYAhtFWm6TqkpE3Xc9K9EmLSUREZH2Si26OA64EceqGN5nGjAM\nRy0TBnPdCgwHX6jZv+6URERE2ivVqR51aG8bx8fAo8Ae0SoKPsQSyWHRg6qekoiIyBp+XeDT2MW9\nPUdfYEfgrhjfMEP0dbQgwSYtJRERkTa7AnPCb/f5Pg/cEd5BxCVDPP0itwM72KCAxlISERFpE+5k\nWNB44mrKavMIsD2ODaNVE3wM3EQCTVpKIiIibQp3qtukwFbglljfze5qpkMMQ4YTatJSEhERAcAH\nFO9UPxCYhqPUGlW1imuo753AUPAN3X9JSURExGwNfADB4gKvVbsNbjUyxJJEguVYc9vh0euqnJKI\niIgp1pTVgm1Fm78uYFymA9vh6BVDXQ1v0lISERExxZqy9gbm4Sh0hxKd479YB3sc/SL/AbYAv0UM\ndVVESURExBTbybCeTVlZGWKZLxKsBK4HvhS9rsooiYiI4Ptgex21X97d0YX6DO3NF1fnOjS4SUtJ\nRETE9jqaHn6Tz7UL8D6OuXV+/0eBbXD0jqGu+4DNwG9d9swYKImIiBRfdDG+tbJKcSzHFmTcK3pl\nwSpsT6aG3I0oiYiIFO9Ub0RTVlacTVpX06B+ESUREenkfAu2Zta0docd2wK9KLQYY31kiGcxRrAB\nAn3AD42pvqKURESks9sBWABB/mz08cBUHGtvTlUfjwFb4egTvapgNXANDWjSUhIRkc6u2KKLjRja\n28b6RR4invkisGaUlg9iqq+gOJLIGGAuMA+YXOScKeHrs4CdKih7FvBMeP71EMtMThGRQtbuVHcM\nALYF7m1wLBnia9J6BMjuj5JaXYH5wBCgBZgJ5LfBjcV23QLYjbZ2x1JlD6AtwZ0RPvLFutm8iHRW\n/mXwW7U75Pg2jn80PBTHSBwz46vQnwn+tPyD8dUf/U5kBJYIFgArgCuxPYhzHQJcFj5/BOgN9C9T\n9k5Y0w75CDYJSEQkZn4w0B14Pu+FQ2lkU1abx4EtcWwcU311b9KKmkQGAi/n/LwoPFbJOQMqKAtw\nHG13MiIicQqbsoK2b+c24W934LaGR+NYYfHEMV8EgCfCf3eOqb61dItYvtLbolqz4CnAcuCKIq+7\nnOeZ8CEiUqlC80MOAu7F8UEC8UDbfJEY7oQCDxc+DP+ZgrXwxC5qElkM5G6AMhi7oyh1zqDwnJYy\nZY/B+lP2K/H+rqpoRUTaGwV8J+9YY2apF5cBLoyvum+eBd+8Ea74eXjHdWp8dUfXDWtLHIK1K5br\nWB9JW8d6qbJjgDnAJiXeWx3rIhKB7wn+A/A91hxy9MDxDo5+iYXl6IbjXVzJ618VfAB+Lvjdsgfi\nqddE7RNZCZwA3A48jXXiPANMCB9gCeQFrBP9AmBimbIA5wI9sduvGcCfIsYpIpJvODALgo9zju0H\nPInj9YRiAsdKbN7K3vFUGHgS2n897XQnIiIR+P8Df1a7Q44LcfwgoYBy4zgJx7nxVei3B78IfBdS\ndiciItKs2neqO7piUxKS7A/JinMxRiB4Gngb+2+OlZKIiHRCvgs2jDd3ZNbuwGs4XkgmpnZmAINx\n9I2xzro0aSmJiEhnNBRYCsGSnGONXPa9NOsXeYDY+kUASyJfjLE+QElERDqn9osuOgKSm6VeTIb4\n1tECgnnAq/HVZ5RERKQzyl90cRi2nt+sZMIpKEOs/SKA3Y3ESklERDqj/JnqdhfiUjXqcwYwMOY5\nKzGO+DJKIiLSyfi+QD9sQnNWevpDshyrgPuJd5TWh/HVZZRERKSzGQVMC3f/A8cnsGWXCm1MlbSY\nh/rGT0lERDqb/J0MxwM3hSOi0iZDrJ3r8VMSEZHOJr8/JH1NWW1mAf1x9E86kGKURESkE/HrYFt0\nTwcIFzncmTotkx6Z9YvcR7zzRWKlJCIinclOwHMQvB/+/DngLhwfJRhTORlS3KSlJCIinUn+/JDx\npGuCYSGp7lxXEhGRzqStP8SxPrAvcEuSAVXgSaAfjs2SDqQQJRER6SR8QPuRWZ8FpuN4O7mYKuBY\njfWLtCYcSUFKIiLSWWwBrAYWhj+nba2sUlLbpKUkIiKdRXgXEngcLcDBwNSEY6pUhpR2riuJiEhn\nkdupvhfwPI5FCcZTjdnAxjgGJh1IPiUREekscicZpnmC4dqsX+ReUjhfJI4kMgaYC8wDJhc5Z0r4\n+ixsnHa5sodji6OtwiYCiYhE4DcEPgnMCPcOaa4kYjKksEkrahLpCpyHJYPtga9gO4blGgtsBWwN\nfAs4v4Kys7FOr/sixiciAjASeByCFcAuwIfAM8mGVLVUdq5HTSIjgPnAAmAFcCUwLu+cQ4DLwueP\nAL2B/mXKzgWeixibiEhW7tBeuwtJ194hlZgD9MYxKOlAckVNIgOBl3N+XhQeq+ScARWUFRGJQ25/\nSDMN7W3T1i/SmnAk7XSLWL7STB5EfJ9iXM7zTPgQEcnhuwK7AQ/j2AboQ3YBxuaTwZLI36so00od\nE0/UJLIY28wlazCsNWQu/5xB4TktFZQtx1V5voh0PsOAVyBYChwHTA2/1TejDPD9Gspkcn4+NaZY\ngOjNWY9hHeZDgO7AEcCNeefcCBwdPh8JvAMsqbAs1O8uRkQ6h9z5Ic3ZlNVmDrABjs2TDiQrahJZ\nCZwA3A48DVyFjXiYED4AbgVewDrRLwAmlikL9j/6ZSzp3AL8u/Db++4R4xeRjs861W0Bw6HYKKfm\nZIMBUtcv0qw8+EfBb5V0ICKSZv4F8NvhmIDjiqSjicwxEcclEWqIdVRas89Yvxx4GPzR4QqdIiI5\n/GZAL2zKQDNOMCwkQ4ruRJo8iQTnAvsBJwN/D2eliohk2dBeF2yA9Y3clnA8cXgGWB/HJ5IOBJo+\niQAETwK7Au8DM8CPTDggEUmPbKf6QcD9ON5LOJ7orF8kQ0ruRjpAEgEIPoTg28CJwFTwp4Rjw0Wk\nc8tOMuwoTVlZGVKyjlYHSSJZwfXYXckBwF3gU7U8gIg0kl8X+DTD/zgLW6Ov0BSCZpVBdyL1EryM\n9ZPcBTwOfnzCAYlIMnYF5nDwCbsDT+FYknRAMZoL9MAxJOlAmjuJOKbgKDBXJFgFwWnYLew54M8H\nv16jwxORRGUXXTyUjtWUldsvkniTVnMnEZvtfg+OAYVfDh7G9i/pBTwK/jMNi0ykI3J0wzEs3JMj\n7UbR7aNp2EriHSuJmFQsDd/sSWQ8NqP9MVyxHb+Cd4EjgTOBu8FP0pwSkSo5tsPxG2wlif9gX952\nTDiqEnwAjOKrn/8IeAPH/KQjqoMMsE/SCb25k4hjNY5fA8cAV+H4YeEPNPAQXIbd3h4N3Ai+byND\nFWk6jg1wfAPHg7QtFbIPto3DVcDtOP6CY9PEYixua2AZW969J829VlYpz2EL2W6RZBDNnUSyHHdg\nSz1/GbgaxwaFTwzmYePGnwZmgt+/USGKNAVHgGOPcFmNhcDngDOAzXFMxjEXx0oc5wPbAh8Ac3Cc\niGOdBCPPNxpWP0RH7A/Jsn6RxJu0OkYSAXC8BOwJvAVMx621TW8oWA7BZODrwGXgz9RCjtLpOQbg\nOBl4FrgQ+6K1HY5DcdyEY0WBMu/g+AH2xWxvLJmMS7p5JTSK7a97AfumPiPpYOooQ8Kd62n4n10r\nT7H4HcdifSATcVxTooq+wKXApsBXwzsVkc7BRjYeDHwDSwTXAJcAj9S0dazjQOB3wCvA93HMji/Y\navk5TByWod+clTi+m1wcdebYGuuj2ryK/2fFr5016JhJBMCxM3AtcD1wMo6VRaoJsCXpfwb8CLjc\n+lBEOijHp7DNmb6GzTe4BLgWx7IY6m7BtoH4GXAd8DMcb0Sutyq+D7CAU7s8R+BPxDXx0u/l2F3f\nImAvHM9XWEpJJFT+g3D0Af4BrAccgeO1EtV9BvgnMAs4PhzVJdIxOHphfYbHYbuL/hX4K4763H3b\n354DvgL8GvgjjuV1ea+1+LFsPPfHTBo6FOhf/AtkB+H4BzZa7qIKS8SaRDpOn0ghjrew2/V7sGHA\no4ufHDwJDAfexRZy3L0RIYrUjaMLjn1w/A14Cdgfu7BvjuOUuiUQe++3cHwH6ys5EJiN4+AG9ZeM\nZuTvPwBu7vAJxCTaud6x70RyOcZi/R+nAeeWbj/0hwJ/Bs4FTrcZ8CJNwjEYGzhyLPAhcDHwdxxv\nJhRPAIwFzgEWYP0lT9fvDf09/GjTjej5usN10JFZuRxbYbsdDqqwX0TNWaHqPwjHllg77TPA/5Ru\nA/aDgL+HPxwVrsklkk42vHYc1lw1HJvHcQnweE2d5EX5Htjk3enAU1X1H1pH/kTgFOBK4NSwtSBG\nvoV133yLk/p6Avrj+DDe+lPIkvRCYN8K7y7VnFUzxwvYhMP/AtNwbFP85GARtpDjHcBj4d2JSLo4\ndsDxB6xz9VvYbp+Dwi1UH4s5gWwC3I1N2L0JeBb86eCHV7QKhGM5jt9j+5x3BebimBR2xsdlB4Zd\n9Q4Bd3eKBAKJ7y8SRxIZg43wmAdMLnLOlPD1WdhaVuXK9gHuxGZk3gH0jiFO4/gI+7Z2LvAAjhLJ\nIVgFwa+xb3hng/+zFnKUxDk2wvG/OB7HLubvACNw7I/jivB3PGZ+G+Bh2uYlbIF1mnvsjn0B+N+B\n3wN86euK400cE7EvaeOAWeHw4DiMYofLV9JxZ6kXkyEFizHWoiswH1sIsQWYCWtN8hsL3Bo+3w2Y\nVkHZM4GTwueTsRmz+aJ/w3IMx/ESjjNwdCt9su8F/h/gnwa/Q+T3FqmGdZIfgOOf4SS/f4Y/N2Dz\nNb8X+NfAH1fk9QD8MPCngp8F/tVw5ez9wZe+y7AZ8uNwzMdxC45tI4W6zjvX8NOWD8PRYZ2HY0sc\nr1Q4cCFVUxh2p/2exSeHj1x/Bo7I+Xku0L9M2bmwZj2e/uHP+eL5IByb4LgTx904+pU+2Qfgjwb/\nBvjvaCFHqTvHEBwu/LLzBI4TGnuB9EeCX1LdEkF+a/CTwU8H/yb4S8AfDL74siiOdXD8KLxL+R2O\njWoK91NXvsmPez5UU9lmZsl4YYVJONYkErU5ayC2qmfWovBYJecMKFF2U1izgcwSqOMCbzZiZQx2\nh/QYjt2Knxx4CC7HEuBRwE2pX8gxHUtQSLUcXXH8CngMa94dj2NnHOfF3xldiA/A/wz4FbAvBHdV\nXjaYB8FvIBgB7Aw8ibUovAb+CvCHgV+/XRHHf3H8FtgeWB/rLzm+fAtBu5gHs+2N69N92T8rL9NB\nJLiOVhX/gwqqNKNVciELitTnS7yPy3meCR/Vc6wCTsHxCHATjp8BFxTvlAzmgx8N/AJbyPEYCO6s\n6b3rwSaWjcWWyj8oHLnzIbAsfJR7Xs25yzrJWPzGsW/h2Umy2+N4vbEB+O7AX7AL+u4QlJikW06w\nEPi9PXx/7Hfy28Al4O/CVpS4ec3kXvtv/RaOP4XlJuL4Po7ySazlgz3Z5qYuBH5q7fE2tQzwWeCC\nvOOt1DG5RP2WOhK7kI8Jf/4xsBr4Tc45f8b+464Mf56LTUDaokTZudh/9GvAZliG3S7vvWMdpraG\nrUVzHbZo2/HlR3j4/YDLsNnup9gCj9XyAdZH1AJ0D/+t5tGdLe7uxy5/2Y2B04ez4eKteHuLeby4\nzxwemziX9wa+Q89Xl9Nr4Qp6L1xNr5dW0Wuhp9dC2OAV6PlaF7p/2AO7aK2f82+lz1dRffL5F45n\nqv+sOjjEAfRCAAASBUlEQVTHMKxT+GbgJAotfFhXfiPswv428DUI6jTCyW8MfB44DLse3I/93U2F\nYCmQvYs+FPgtMBv4IaX2Bdnh0uvY75QRnPPK4PrEnHKOLbDBD5uVGZWXqnki3bBVP/fDFl2bjo3Y\nyL04jMXWphqLJZ3fh/+WKnsmsBRLKCdjo7Py+1rqk0QAHOtj2XwYcBhl16TxfbEJXdtizXJtF/fK\nE8FqYAWwPPy3zGP1CgY83p1PX9GHrW7rx4aLerJo5GKe+9wCnjzyJT7a5KPw3JVAD2ADoGeRfzfA\nPs/3saW9K/83WPkBPZd8TL/Zq9j0yZUMeMKz6ZOw8XMBXVYVSzwbY+s2nQJcGO8w1CbmOBz4E/A9\nbCmLBvNbYoNgbgYmN26Srd8QW1niMOAArAnvOuBfELyKowfwPWxtu0uBX+FYe1micccsof+sG7hg\nxoTGxJ1CjgXAGFzBfuSsVCURgIOwxNAVu5Ceji3ABm23VedhdxzLsFm0T5QoC9YGfDWwOTbD9UvY\nMMZc9UsikP0WNBE4FTgOx82lC/gAW4q+O+0v+BUmhWB1BTF1wUa4jQ8f62N7JdwA3Fv7t1YfhHGX\nSzTFXit0bg/s/3ehBPQ+u599Mwf+6CRsGPe3cLxdW+wdgI2wOg0bgPIFXBJLl/uR2B3ILyE4v/Hv\nvyaO9bBlUg7DEsscLKFcjwuWY5/TQdgCj5eETdGw7hs9+fbO77GqZRemvNCRl34vzfFXbBXmUv8P\nU5dEklLfJJLlGIXN/r0U+PmaX9pGsf6MfbGkcQh2h5ZNHDHPRo6T74ollUKJZnPgVDZ+9mAmbXcU\n9t/1VRydcVTNxsAV2BepL5PI0iT+cOCPwDEQ3Frm5Aby62AtFYdhvyMvAdcx/pi57HjZD7Dfp+/h\nyLDXL77FLhf+gV4vr5fev4kGcHwdGItrNyI2n5JIqDFJBMC2/7wSm+l+JI6ldX6/Xti3rUOxb2VP\nYUljKvVcNK+h/JewvSf2wAXDsI2QzgXOaHiiTopjB6z/41rgJ40foOAD4ESsufkQCGY29v2r4bsB\ne2EJ5VBYvZTWX8xlj9NH0XX5NF4e/QmWbbKaq6aOSDjQZDmGAI9gqxcXS6ZKIqHGJREgHGr4a+Bw\n4IvYbOE46x+Afdsajy3Nch+WOG7CrRnu3MH4ScAkYDQu6I6NSPLA13AsTjS0enN8BVvJYRJuzaCT\nBvIt2N3HCOBz4TI/TcJ3wYbZH0bLssPY87T1GTmlD/f+bDIPTj4r6egS53gROJjii1wqiYQam0Sy\nHIcB52PfHCtdv79YXdvR1r+xLdapeQNwG473I0baJPxpWGfqvrjgI2yU3gnYApk3JRpaPdiXkd9g\n/8+/gGNW44PwvbA+x5XAlyFo4t81HwA70+2jMaxcd0pz/7fExHEJNuftT0XOUBIJJZNEIHvxvx54\nCDgBx8cVluuCffPLJo4NaN8x3qBNe9LEB8BF2ETTQyBYju378g9gKjC54s837Rx9sWbRlcBXGjNp\nMJ/fHLgFu9P9LgSa49PROI4GPh+O9itESSSUXBIBcPTERpR9EmveWlDkvHWwhdHGY4vNvUX7jvHy\no7I6PN8N6xt4B/g6BKvDCXcXAlthHc6lhiymn23XfD3Wif7TZPp9/C5YYj4b+L22ge6gHJtjw6Q3\nLdIvoiQSSjaJQHYY8PewJR2+juP28PiGtM0YH4MNU8x2jD+XTLBp59cD7gIeguBHQPbz/R9sWOdk\n4NKmHHnjOArbkOl4HNcmE4Q/BLvj+xYEHX+jps7O5rYdgmNOgVeVRELJJ5Esx55YM8VUbCb+aGwG\nbrZjPMKyEZ2J7wM8AFwMwdlrDjs+hX2+c4AJBSeapZHtk/Fb7AvFoTieSiYQ/11sVezxEDyaTAzS\nUI6LgRk4zivwqjalSh3H/cCu2OS6S7FNgQ7GcaESSDWCt7Ahzd8F/7U1h+3b1AhsjswMSi6SmRK2\nIvRdWHPc8GQSiO8Kfgq2WdVoJZBOpWGLMabjm3xt0nMnIjHznwL+g/WP3NbuJdtE7M/YHJMzU9mn\n5BiOzf24DHDJxOh7Yuu59QAOhyB/xQfpyByDsD2a+hX4/dO1M9R8beNSBT863Ldl+FovOTbHcR+O\nu3BslkBwxTmOxfE6JXfMrDc/APwT4C8uuymUdFyOeTg+XeCVVO0nIlInwYPAN4Absa1Z2zgWYkvB\n3A88gWNs4+PL4+iO44/YAIC9cUltz+p3wPbGuQb4JgQNXgVYUiRDQvuuNwvdiXQK/pvgXwRf+I7D\nsRe2o9s54XDqxnP0x/EAjqnhkjUJ8WPAvw6+1LpJ0lk4jsRxfYFXdCcinUlwETYf59/hTOv2HPcB\nO2Gj4h7Csc1a59STYyTwKHAHNgIroZFjfgI2qONQCK5KJgZJmQx2V6zrfBG6E+k0fAD+PPD3gO9R\n8BTbY3oijjdwHE0jtgV2/E/Y//H5ur9XUb4L+LPAPwt+q+TikFRyPIfjM3lHY712NnMPvUYYdCq+\nK227Y3656IZJ1pF4JbYz5UQc78UeijWbTcH2jxmf3ARSvx5wOdAPuwOp7+rS0nwcFwBP4/hDzlHN\nE5HOKFgFHAVsAkwJ19xam2M2MBybs/NEONw2PrbacgboC+yWYALZFJsL8DFwgBKIFJGhzp3rSiLS\nRIKPsaVkRmNb6xbm+BDHBGxL5VtwnBhLu7AtDPkocBO2XlpCK8b67bERWLcBR0Hw32TikCaQQf0i\nRalPpNPym4F/wUZuleH4BI4HcdyGbS5WPetvOR7HEhwH1VRHbPy+4JeAPzrZOKRpOObi2DHnSGpG\nZ/UB7sT2yL4D6F3kvDHAXGAeNoa+XPk+2G36+9hOdyJ5glex36tfgh9X8lTHS8De2B3EDBwHVvVW\njh7YwoUTgdE4/l1LxPHwx2Cz0I+A4PLk4pAmkyGl80XOxBZ1A0sOZxQ4pyswHxgCtGDT8IeWKb8e\n1lwxgdJJRHcinZ4fHs5qH13R6Y59cLyM4ywc3Ss4fzCO6TiuDpf+T4gPwP8S/PPgt0suDmlKjiNw\nTM05kpo7kUOwtYEI/x1f4JwRWBJZAKzARs1kvzkWK/8h8CC2n7lICcGjwNeA68P1tkpz3IPNKdkW\neBBH8SGxjr2wvaqvBY7A8UEcEVfP98A26Nof2B2C5t5XRZJwL7AXjq71qDxKEtkU1uz9vST8Od9A\n4OWcnxeFxyoprzsNqUBwO/AD4LZw177SHG9iX2QuBx7GcWTe6wGO72DLhhyD48zk9jDxW2JNvi3A\nvhC8nkwc0tRsJfHXgB3qUX23Mq/fCfQvcDx/ZIyn8EU//1hQ4jwlDalR8A/w/bFEske4pHxxlhTO\nDWe7X4Xjs8D/AquAC7A/tt1xvFDnwIvwQ7G95sdim1mdAUH6ViuWZpJdGv6JuCsul0QOKPHaEizB\nvAZsBhT6lrQYGJzz86DwWKXly3E5zzPhQzql4OwwkdwMfn8IPixbxDELxy7YxMEngA+wQSCjcCyr\na7gF+R2xL2h7AX8AJkHQHBtwSZq18lf6sgn7ARsmHUyuM2kbbXUyhTvWuwHPYx3r3Vm7Y71U+WNQ\nx7pUxXcB/zfwN4X7tlfOcXi4bEoCqyD4kWHMi8H/APz6jY9BOjTHpjjeDvtFUnPt7IPt3JY/RHcA\ncEvOeQcBz2Id7D+uoDxYR/xSbJjvQqDQiJTUfBCSJr4F/G3hXhopXhbHB+Bbwd8FfgH444uuCyYS\nB8ec8M5b186QPggpwvcEPx38aUlHsjYfgD8I/IPgn7O5H9o4ShrA8UccP0LXzjX0QUgJvm+4su2k\npCMxvgv4L4B/HPxs8F8OF5UUaQzHF3HcjK6da+iDkDL8FmE/w5cSjKEb+CPBzwH/qM2w91rHSBrP\n0S/c70bXzpA+CKmA3yHc7W/fBr9vd/DfAD8f/P3gD0x3H410Co6n0LVzDX0QUiHfGiaSnRrwXuuC\n/1/wL4G/A/xe9X9PkQo5zkXXzjX0QUgV/BfBvxLOAq9H/T3B/wj8q+Cngh9Rn/cRicAxCl0719AH\nIVXyE8HPA98vxjp7g/9peKdzJfj8rUhF0kbXzpA+CKmB/wX4x8BvELGevuB/DX4p+L+C3zaW8ETq\nT9fOkD4IqYEPwP8F/J3W+V11+QHgzwH/FvjzbQSYSFPRtTOkD0Jq5LuF/RZXVD7c1g8Jk8Zb4H8H\nfmDZIiLppGtnSB+ERODXDYfenlN66K3fBvylYbPVr+PtTxFJhK6dIX0QEpHfCPxT4E8s8Npnwo7y\n18OO840aH59IXejaGdIHITHwg8I5HUeHPw8Hf0M4VPfE6B3wIqmja2dIH4TExA8FvwT8veAXgj/B\nmrtEOiRdO0P6ICRGfgfwX6ttxJZIU9G1M6QPQkSkerFeO7WaqIiI1ExJREREaqYkIiIiNVMSERGR\nmkVJIn2AO4HngDuA3kXOGwPMBeYBkysofwDwGPBk+O8+EWIUEZGUOhM4KXw+GTijwDldgfnAEKAF\nmAkMLVN+R6B/+PxTwKIi75/G0VmtSQdQRGvSARTQmnQABbQmHUABrUkHUEBr0gEU0Zp0AAW0Jh1A\nAakZnXUIcFn4/DJgfIFzRmBJZAGwArgSGFem/EzgtfD508C6WAJqBq1JB1BEa9IBFNCadAAFtCYd\nQAGtSQdQQGvSARTRmnQABbQmHUC9RUkimwJLwudLwp/zDQRezvl5UXis0vKHAY9jCUhERFKmW5nX\n76StaSnXKXk/ewrfIuUfC0qcl3/8U1gT1wFlYhQRkSY0l7YEs1n4c76RwG05P/+Yts71UuUHAc8C\nu5d4//m0JR899NBDDz0qe8wnJc6kLSGcTOGO9W7A81jHenfW7lgvVL43MIvCfSwiItJB9AHuYu0h\nugOAW3LOOwi7q5iP3YmUK/9/wAfAjJzHJnX5LxARERERESk2KTHXlPD1WcBOFZStdEJkI2M6HJgD\nrAJ2rjKeesV0FvBMeP71QK8UxPTL8NyZwN3A4BTElPVDYDX2+1WtesTlsJGP2Tv3MSmICWAS9nv1\nFPCbFMR0JW2f0Yvhv0nHNAKYHsbyKDA8BTHtADyMTfi+EWiKjdlKTUrMGgvcGj7fDZhWQdlKJkQ2\nOqbtgG2Ae6g+idQrpgNoG+59Bun4nHJ/cScBF6UgJrBkdht2Eao2idQrrlOBH1QZS71j2gf7Aped\n49U3BTHl+i3WdJ50TBngwPD5Qdh1IemYHgX2DJ8fC/yiVBBpWTur1KTErNzJiY9gdxX9y5StZEJk\no2Oai90Z1aJeMd2JfbPOlhmUgpjezynfE3gzBTEBnEPbF5Nq1TOuIGUxHQ+cTtscrzdSEFNWAHwJ\n+GcKYnqVtjv/3sDiFMS0NXB/+PwubL5eUWlJIqUmJZY7Z0CJspVMaGx0TFE0IqbjaPvmknRMpwEL\nga9T3d1RvWIaF/78ZBWxNCIusLu1WcDFVNdsW6+Ytgb2wr75ZoBdUxBT1p7Y9eD5FMR0MnA29nt+\nFu0HHyUV0xzaEsrhlGlKTksS8RWeV8m3rWomNJYSZ0xxqXdMpwDLgSuqKFPPmE4BNgf+CvyuinL1\niGld4CdY01Et5aF+n9X5wBbYunOvYhelpGPqBmyEzRU7Ebg6BTFlfYXqfsehfjFdDHwH+z3/PnBJ\nCmI6DpiILYDbE7smFFVuxnqjLKZ9thvM2gsv5p8zKDynpcDx7C3hEuzW7TVsQuPrCcVUqGwt6hnT\nMVj76X4piinrCqq7O6pHTJ/E2o9n5Zz/ONYsUOnvVb0+q9z3vwi4qcJ46hnTImyQBlgb+2pgY2Bp\ngjGBXfMOpfr+yHrFNALYP3x+LdX1/dUrpmdp66fZBji4ipgSU2pSYlZuB9FI2jqIapnQmGRMWfcA\nu1QRTz1jGoPdwtYyH6deMW2dU34S8LcUxJSrlo71esW1WU7571Pdt+x6xTQB+Hn4fBusuSbpmMB+\n16vpvK53TE8Ae4fP98MSbtIxZQdBdAEux75gNoVCkxInhI+s88LXZ9H+m0S1ExqTjOlQrC3yI+wO\n6d8piGke8BJtwx//lIKYrgVmY7/c1wH9UhBTrheobYhvPeK6HOunmQXcQHV9f/WKqQVL/LOxO7bW\nFMQEcCnwrSpjqWdMu2Id3jOxYbW5Q3CTiuk74fFngV9XGY+IiIiIiIiIiIiIiIiIiIiIiIiIiIiI\niIiIiIh0Nv8PdL7Inwgh9MwAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x10d55b7d0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "with nengo.Network() as net, nengo_lasagne.default_config():\n",
    "    # note that here we're using a different test input than the training input\n",
    "    input_node = nengo.Node(output=nengo.processes.WhiteNoise(scale=False))\n",
    "    ens = nengo.Ensemble(50, 1)\n",
    "    output_node = nengo.Node(size_in=1)\n",
    "\n",
    "    nengo.Connection(input_node, ens)\n",
    "    nengo.Connection(ens, ens)\n",
    "    nengo.Connection(ens, output_node)\n",
    "\n",
    "    input_p = nengo.Probe(input_node)\n",
    "    output_p = nengo.Probe(output_node)\n",
    "\n",
    "sim = nengo_lasagne.Simulator(net, dt=dt)\n",
    "sim.model.train({input_node: inputs}, {output_node: targets},\n",
    "                n_epochs=1000, minibatch_size=100,\n",
    "                optimizer_kwargs={\"learning_rate\": 1e-2})\n",
    "sim.run_steps(sig_len)\n",
    "\n",
    "truth = np.cumsum(sim.data[input_p], axis=0)*dt\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(sim.trange(), sim.data[input_p][0])\n",
    "\n",
    "plt.figure()\n",
    "plt.plot(sim.trange(), sim.data[output_p][0], label=\"output\")\n",
    "plt.plot(sim.trange(), truth[0], label=\"truth\")\n",
    "plt.legend()\n",
    "\n",
    "sim.run_steps(sig_len, {input_node: np.random.randn(batch_size, sig_len, 1).astype(np.float32)})\n",
    "\n",
    "truth = np.cumsum(sim.data[input_p], axis=1)*dt\n",
    "\n",
    "print(sim.data[output_p].shape)\n",
    "print(np.sqrt(np.mean((truth - sim.data[output_p])**2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Inserting Lasagne code directly\n",
    "\n",
    "Another thing we might want to do is write some code directly in Lasagne, and then insert that into a `nengo`/`nengo_lasagne` model.  To do that, we can use `nengo_lasagne.LasagneNode`.  To start, just create the desired Lasagne network (in this case we'll do a 2D convnet MNIST example):"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# load the inputs/targets (download the dataset at\n",
    "# http://deeplearning.net/data/mnist/mnist.pkl.gz)\n",
    "with open(\"mnist.pkl\", \"rb\") as f:\n",
    "    train, _, test = pickle.load(f)\n",
    "targets = np.zeros((train[1].shape[0], 1, 10), dtype=np.float32)\n",
    "targets[np.arange(train[1].shape[0]), :, train[1]] = 1.0\n",
    "\n",
    "# input layer\n",
    "l = lasagne.layers.InputLayer(shape=(None,))\n",
    "\n",
    "# reshape it into the (batch_size, n_channels, 28, 28) image shape\n",
    "l = lasagne.layers.ReshapeLayer(l, shape=(-1, 1, 28, 28))\n",
    "\n",
    "# 2 convolution/pooling layers\n",
    "for _ in range(2):\n",
    "    l = lasagne.layers.Conv2DLayer(l, num_filters=32, filter_size=(5, 5),\n",
    "                                   nonlinearity=nl.rectify,\n",
    "                                   W=lasagne.init.HeNormal(gain=\"relu\"))\n",
    "    l = lasagne.layers.MaxPool2DLayer(l, pool_size=(2, 2))\n",
    "\n",
    "# dense layer\n",
    "l = lasagne.layers.DenseLayer(l, num_units=256, nonlinearity=nl.rectify,\n",
    "                              W=lasagne.init.HeNormal(gain='relu'))\n",
    "\n",
    "# dropout\n",
    "l = lasagne.layers.DropoutLayer(l, p=0.5)\n",
    "\n",
    "# output layer\n",
    "l = lasagne.layers.DenseLayer(l, num_units=10, nonlinearity=nl.softmax)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the variable `l` refers to only the most recently constructed layer. However, each layer contains a reference to the previously constructed layer as well, so the entire network can be accessed through `l`.  Next we create a `LasagneNode`, and pass the output layer as the `output` parameter:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "with nengo.Network() as net, nengo_lasagne.default_config():\n",
    "    # 'insert_weights' is an optional config setting we can use\n",
    "    # to control whether a connection weight matrix is inserted\n",
    "    # for each connection, or whether we just directly connect\n",
    "    # the output of the pre to the post. in this case we already\n",
    "    # created all the weight matrices we want in the above\n",
    "    # Lasagne network.\n",
    "    net.config[nengo.Connection].set_param(\"insert_weights\",\n",
    "                                           nengo.params.BoolParam(False))\n",
    "\n",
    "    # input node will just present one input image per timestep\n",
    "    input_node = nengo.Node(output=PresentInput(test[0], 0.001))\n",
    "    \n",
    "    # insert the convolutional network we defined above\n",
    "    conv_layers = nengo_lasagne.layers.LasagneNode(output=l, size_in=784)\n",
    "    \n",
    "    # output node\n",
    "    output_node = nengo.Node(size_in=10)\n",
    "\n",
    "    nengo.Connection(input_node, conv_layers)\n",
    "    nengo.Connection(conv_layers, output_node)\n",
    "\n",
    "    p = nengo.Probe(output_node)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When we build the network, `nengo_lasagne` will compile the contents of the `LasagneNode` directly into the network graph.  This means that we can train the parameters inside the `LasagneNode` in the same way as we did previously:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "checking network\n",
      "building network\n",
      "training network\n",
      "Simulation finished in 0:09:01.                                                 \n",
      "training complete\n"
     ]
    }
   ],
   "source": [
    "sim = nengo_lasagne.Simulator(net)\n",
    "\n",
    "# we'll use a different optimizer function here, nesterov_momentum.\n",
    "# we'll also use a different objective, categorical_crossentropy,\n",
    "# as opposed to the default squared error we were using previously.\n",
    "sim.train({input_node: train[0][:, None]}, {output_node: targets},\n",
    "          n_epochs=10, minibatch_size=500,\n",
    "          optimizer=lasagne.updates.adagrad,\n",
    "          optimizer_kwargs={\"learning_rate\": 0.01},\n",
    "          objective=lasagne.objectives.categorical_crossentropy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If we run the trained network, we can see it getting ~98% accuracy on the test set.  Note that we're only training for 10 epochs, which is quite small.  If you have more patience you can change the `n_epochs` parameter above to 100/200, which should get >99% accuracy."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.9845\n"
     ]
    }
   ],
   "source": [
    "sim.run_steps(test[0].shape[0])\n",
    "\n",
    "output = sim.data[p].squeeze()\n",
    "\n",
    "print(np.mean(np.argmax(output, axis=1) == test[1]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "But that's not all!  One of the nice things about `LasagneNode` is that it's just a thin wrapper around a regular `nengo.Node`.  That means that we can take the same MNIST network (which contains our trained `LasagneNode`), and build it using reference `nengo` instead:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Simulation finished in 0:00:06.                                                 \n",
      "0.984498449845\n"
     ]
    }
   ],
   "source": [
    "sim = nengo.Simulator(net)\n",
    "sim.run_steps(test[0].shape[0])\n",
    "\n",
    "output = sim.data[p]\n",
    "\n",
    "print(np.mean(np.argmax(output[:-1], axis=1) == test[1][1:]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And there we go, the same Lasagne network is now running within regular Nengo."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(10000, 784)\n",
      "(784,)\n"
     ]
    }
   ],
   "source": [
    "print(test[0].shape)\n",
    "# how is this sent in the demo\n",
    "print(test[0][0].shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(50000, 1, 784)\n",
      "(50000, 1, 10)\n"
     ]
    }
   ],
   "source": [
    "print(train[0][:, None].shape)\n",
    "print(targets.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.,  0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.],\n",
       "       [ 1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  1.],\n",
       "       [ 0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  1.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.],\n",
       "       [ 0.,  0.,  0.,  0.,  1.,  0.,  0.,  0.,  0.,  0.]], dtype=float32)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "targets[:10, 0, :]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}