26. Assignment 10 .ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Solutions to Assignment 10_prateek.ipynb",
      "provenance": [],
      "collapsed_sections": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mHjEwJl01xE_"
      },
      "source": [
        "### Training a single recurrent neuron"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "whuS7vSS12MP"
      },
      "source": [
        "We define a recurrent neuron as follows: <br>\n",
        "$y_{t+1} = 0.5y_t + 0.5x_t$"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "1QZuQZ8Y2MGr"
      },
      "source": [
        "### Generate Data\n",
        "Start with $y_0$ = $0$ <br>\n",
        "Generate a series of random numbers $+1/-1$ with $50/50$ probability. This will constitute our input at each time step ($x_t$)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qByezjAEvIks"
      },
      "source": [
        "import numpy as np"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "4KxA7SCO062E"
      },
      "source": [
        "def generate_inputs(n):\n",
        "  \"\"\"\n",
        "  Inputs:\n",
        "    n: int\n",
        "\n",
        "  Outputs: \n",
        "    x_t : list of length n with either +1 or -1 generated randomly with 50/50 probability\n",
        "  \"\"\"\n",
        "  ## x_t = np.array([1 if np.random.rand(1) > 0.5 else -1 for i in range(n)]) ## list comprehension\n",
        "  x_t = []\n",
        "  for i in range(n):\n",
        "    if(np.random.uniform() > 0.5):\n",
        "      x_t.append(1)\n",
        "    else:\n",
        "      x_t.append(-1)\n",
        "  \n",
        "  return np.array(x_t)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uOLLhByKcR7o"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "_9Jo5tPm7S0G"
      },
      "source": [
        "def generate_outputs(x_t):\n",
        "  \"\"\"\n",
        "  Inputs:\n",
        "    x_t: list of length n with inputs to recurrent neuron\n",
        "\n",
        "  Outputs:\n",
        "    y_t : list of length n+1 with outputs generated using our definition of recurrent neuron \n",
        "  \"\"\"\n",
        "  ### Write your code here\n",
        "  y_t = [0]\n",
        "  for i in range(1, x_t.shape[0] + 1):\n",
        "    y_t.append(0.5*y_t[i-1] + 0.5*x_t[i-1])\n",
        "  return np.array(y_t)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nyDMlVkK7yMR"
      },
      "source": [
        "### Training using BPTT\n",
        "Write the train function that takes $y_t$ and $x_t$, starts with random values of $w$ and $g$ where \n",
        "$$\n",
        "y_{t+1} = wx_{t} + gy_t \\\\\n",
        "$$\n",
        "and trains using BPTT to find values of $w$ and $g$\n",
        "$$\n",
        "\n",
        "Backpropagation of gradients\n",
        "$$\n",
        "dy_{hat_t} = \\frac{\\partial{E}}{\\partial{y_{t+1}}} * \\frac{\\partial{y_{t+1}}}{\\partial{y_t}} \n",
        "$$\n",
        "$$\n",
        "dw = \\frac{\\partial{y_{t+1}}}{\\partial{w}} * \\frac{\\partial{E}}{\\partial{y_{t+1}}} \n",
        "$$"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "37MeIOak80R3"
      },
      "source": [
        "def train_bptt(x_t, y_t, time_steps, epochs, w_his = None, g_his = None):\n",
        "  \"\"\"\n",
        "  Inputs:\n",
        "    x_t: list of length n with inputs to recurrent neuron\n",
        "    y_t: list of length n+1 with outputs\n",
        "    time_steps: int, number of timesteps to roll the rnn and do bptt\n",
        "    epochs: number of epochs to train the rnn for\n",
        "  \n",
        "  Outputs:\n",
        "    w, g: trained parameters of the model\n",
        "  \"\"\"\n",
        "  ### Write your code here\n",
        "  if(w_his == None):\n",
        "    w_his = [np.random.rand(1)*0.01]\n",
        "    g_his = [np.random.rand(1)*0.01] \n",
        "\n",
        "  y = [0]\n",
        "  eta = 0.1\n",
        "\n",
        "  for e in range(epochs):\n",
        "    y = [0]\n",
        "    for i in range(0, x_t.shape[0], time_steps):\n",
        "    \n",
        "      ts = min(x_t.shape[0] - i, time_steps)\n",
        "      w = w_his[-1]\n",
        "      g = g_his[-1]\n",
        "      dy = [0]\n",
        "      dyhat = np.zeros(ts + 1)\n",
        "      dw = 0\n",
        "      dg = 0\n",
        "\n",
        "      ## Forward pass\n",
        "      for t in range(i, i+ts):  ## I had used range(i+ts) earlier, but this works as well\n",
        "        y.append(w*x_t[t] + g*y[t])\n",
        "      \n",
        "\n",
        "      ## Backward pass\n",
        "      for j in range(ts):\n",
        "        ## dy is gradient directly coming from error at a particular timestep\n",
        "        ## E = 1/2(y - yd)^2 , so dy = dE/dy = y - yd\n",
        "        dy.append(y[i+j+1]-y_t[i+j+1])\n",
        "      \n",
        "      for j in range(ts - 1, 0, -1):\n",
        "        ## dyhat is gradient to y_t coming from dy_t+1 \n",
        "        dyhat[j] = g*(dyhat[j+1] + dy[j+1])\n",
        "        dw += x_t[i+j]*(dyhat[j+1] + dy[j+1])\n",
        "        dg += y[i+j]*(dyhat[j+1] + dy[j+1]) \n",
        "      \n",
        "      w_his.append(w - eta*dw)\n",
        "      g_his.append(g - eta*dg)\n",
        "  \n",
        "  return w_his, g_his\n",
        "  "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xGrwYEWXxLyU"
      },
      "source": [
        "x_t = generate_inputs(100)\n",
        "y_t = generate_outputs(x_t)\n",
        "w_his, g_his = train_bptt(x_t, y_t, 5, 100)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "m1VNju9GaRZo"
      },
      "source": [
        "#w_his, g_his = train_bptt(x_t, y_t, 5, 500, w_his = w_his, g_his = g_his)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kwPsxG0BxoK_",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 283
        },
        "outputId": "8bb1a267-9f00-4e23-fb10-c06319622893"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "plt.plot(w_his)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f74b71097f0>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 158
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAATk0lEQVR4nO3df5BdZX3H8fc3icHKL8GsyITEBI2OUTpKt8BMFR1LMVGb2KptsB2htZM6I1Mt7bQ4WMahf4FT/+iYqabKVB01oK3TdBoHLbV2OlNoFoxigMCaoiTlRxQKjJSQZL/9456bnHv3bPZucn/ss75fMzvn3Oeeveebc3c/efY5zz0nMhNJUvkWjboASVJ/GOiStEAY6JK0QBjokrRAGOiStEAsGdWOly1blqtWrRrV7iWpSHfddddPMnOs6bmRBfqqVauYmJgY1e4lqUgR8aOZnnPIRZIWCANdkhYIA12SFggDXZIWCANdkhYIA12SFggDXZIWiKID/alnD7H9e/8z6jIkaV4Y2QeL+uGaW3dx+/2Pc8HyM1m97NRRlyNJI1V0D/2Rp54D4GcHD4+4EkkavaID/QVLWuU/f2RqxJVI0uiVHeiLAoDDR7yNniQVHehLFrcD3R66JBUd6HfsfQKAQ1P20CWp2EA/ePjI0fVDh+2hS1KxgZ61TrknRSWp4ECfqiX6c4eOHGdLSfr5UGyg13vozkOXpB4DPSLWRcSeiJiMiGsbnr8qIg5ExK7q6w/6X2qneg/9sacPDnp3kjTvzRroEbEY2AKsB9YCV0TE2oZNb8nM11dfn+1zndPUJ7Z86tuTPPPcIZ557tCgdytJ81Yv13K5CJjMzL0AEbEN2AjcO8jCZpPZOVXxgo9/E4A3v2qM0164BKqns1qpb57HeU6SBu19F6/kLa9+ad9ft5dAXw48XHu8D7i4Ybt3R8SlwAPAH2fmw90bRMRmYDPAypUr515tzUwh/PgzB9n35LPt/bWWR/dfq6VqrbdJ0jD87OBgJnL062qL/wR8JTMPRsQfAp8H3tq9UWZuBbYCjI+Pn1S/uD2G/ppzz+C+R55m2WmnMPGxy07mJSWpaL0E+n5gRe3xeVXbUZn509rDzwI3nXxpx9ceQ3/fRSt4y6tfyjlnvHDQu5Skea2XWS47gTURsToilgKbgO31DSLi3NrDDcB9/SuxWXv8OyJYcfaLWLqk2BmYktQXs/bQM/NwRFwN3AYsBm7OzN0RcQMwkZnbgT+KiA3AYeAJ4KoB1lzV1VouchBckoAex9Azcwewo6vt+tr6R4GP9re042uPoS8yzyUJKPiTou0xdDvoktRSbKC356GHiS5JQMGB/mh1P1HH0CWppdhAf8+n/xNwDF2S2ooN9DY76JLUUnygO+QiSS3FB7onRSWppfhAdwxdklqKD3RJUkvxge61zCWppdhAv3j12QBc9ppzRlyJJM0PxQb6aacs4XXLz+AXli4edSmSNC8UG+hHMp2yKEk1xQb6VDoHXZLqig30zHTKoiTVFBvoUw65SFKHcgN9yiEXSaorNtCPZHphLkmqKTbQH33qOXvoklTT0z1F55vvPHCAHz/xLD9+4tlRlyJJ80aRPfR79v3vqEuQpHmnyED3krmSNF2RgS5Jmq7IQLeDLknTlRnomOiS1K3MQDfPJWmaMgN91AVI0jxUZqCb6JI0TZGBfufeJ0ZdgiTNOz0FekSsi4g9ETEZEdceZ7t3R0RGxHj/Spzu9vsfH+TLS1KRZg30iFgMbAHWA2uBKyJibcN2pwMfBu7sd5GSpNn10kO/CJjMzL2Z+TywDdjYsN1fAjcCz/WxvuP6lVe+ZFi7kqR5r5dAXw48XHu8r2o7KiIuBFZk5j8f74UiYnNETETExIEDB+ZcbLe/eOe0PxQk6efWSZ8UjYhFwCeBP5lt28zcmpnjmTk+NjZ2srv2A0aSVNNLoO8HVtQen1e1tZ0OvA74t4h4CLgE2D7oE6MAh6emBr0LSSpGL4G+E1gTEasjYimwCdjefjIzn8rMZZm5KjNXAXcAGzJzYiAV1xyZykHvQpKKMWugZ+Zh4GrgNuA+4NbM3B0RN0TEhkEXeDwvO/OFo9y9JM0rPd2xKDN3ADu62q6fYdu3nHxZxzf+8rOYyuSlpxvoktRW5CdFE3jR0iLvnidJA1NkoE9lej0XSepSaKDDIhNdkjqUGehTySLzXJI6lBnomfbQJalLoYEOi+yiS1KHIgM90yEXSepWZKA75CJJ0xUa6M5ykaRuhQa689AlqVuRgZ720CVpmiIDfcqTopI0TcGBbqJLUl2ZgT4FYaBLUociA9156JI0XZGB7rRFSZqu0EBPFhVZuSQNTpGxOJWOoUtStyID3TF0SZquyEB32qIkTVdooHtSVJK6FRroXstFkroVGehey0WSpisy0L2WiyRNV3Cgm+iSVFdooDsPXZK6FRnozkOXpOmKDHSnLUrSdD0FekSsi4g9ETEZEdc2PP/BiLgnInZFxH9ExNr+l3qMJ0UlabpZAz0iFgNbgPXAWuCKhsD+cmZekJmvB24CPtn3SiuZSTqGLknT9NJDvwiYzMy9mfk8sA3YWN8gM5+uPTwVyP6V2CmrV3bIRZI6Lelhm+XAw7XH+4CLuzeKiA8B1wBLgbc2vVBEbAY2A6xcuXKutQKt4RbAIRdJ6tK3k6KZuSUzXwH8OfCxGbbZmpnjmTk+NjZ2QvuZavfQTXRJ6tBLoO8HVtQen1e1zWQb8K6TKep42j10R1wkqVMvgb4TWBMRqyNiKbAJ2F7fICLW1B6+A3iwfyV2cgxdkprNOoaemYcj4mrgNmAxcHNm7o6IG4CJzNwOXB0RlwGHgCeBKwdVsGPoktSsl5OiZOYOYEdX2/W19Q/3ua4ZHQt0E12S6or7pGj7pKjz0CWpU3GBng65SFKj4gJ9ypOiktSowEC3hy5JTYoNdMfQJalTcYHuPHRJalZcoDvkIknNCgz01tIeuiR1Ki/Qp7yWiyQ1KS7QHUOXpGbFBfrRMfTiKpekwSouFr2WiyQ1KzDQR12BJM1PxQV6+3alfrBIkjoVF+jtk6LGuSR1Ki/Qq6UddEnqVF6gH+2hm+iSVFdeoOMHiySpSXGB3maeS1Kn4gI9nbYoSY2KDXSHXCSpU3mBfmyey0jrkKT5prxAt4cuSY2KC/Q281ySOhUX6Md66Ea6JNUVF+htxrkkdSou0I+dFJUk1ZUX6J4UlaRG5QV6tTTQJalTT4EeEesiYk9ETEbEtQ3PXxMR90bE9yPi9oh4ef9Lbcmqi+7FuSSp06yBHhGLgS3AemAtcEVErO3a7LvAeGb+IvA14KZ+F9p2dATdPJekDr300C8CJjNzb2Y+D2wDNtY3yMxvZ+az1cM7gPP6W2Z9X62leS5JnXoJ9OXAw7XH+6q2mXwA+EbTExGxOSImImLiwIEDvVfZ/Fon9f2StND09aRoRPwuMA58oun5zNyameOZOT42NnaCe3HaoiQ1WdLDNvuBFbXH51VtHSLiMuA64M2ZebA/5U3nkIskNeulh74TWBMRqyNiKbAJ2F7fICLeAHwG2JCZj/e/zGOctihJzWYN9Mw8DFwN3AbcB9yambsj4oaI2FBt9gngNOCrEbErIrbP8HInzXuKSlKzXoZcyMwdwI6ututr65f1ua7j1QLYQ5ekbuV+UnSkVUjS/FNcoB9loktSh+IC3ZtES1Kz8gIdr+UiSU2KC3S8fK4kNSou0D0pKknNygt07ykqSY2KC/Q281ySOhUX6N5TVJKalRfoXpxLkhqVF+jV0iEXSepUXqCn81wkqUl5gV4t7aFLUqfiAh3H0CWpUXmBXnEeuiR1Ki7QnbYoSc3KC3SHXCSpUbmBbqJLUofyAr1aevlcSepUXqB7T1FJalReoI+6AEmap4oL9DZ76JLUqbhA956iktSsuEDHe4pKUqPiAt1pi5LUrLxAr5YGuiR1Ki/Qj35S1ESXpLryAh3noUtSk+ICvc08l6ROPQV6RKyLiD0RMRkR1zY8f2lE3B0RhyPiPf0v8xinLUpSs1kDPSIWA1uA9cBa4IqIWNu12Y+Bq4Av97vAbp4UlaRmS3rY5iJgMjP3AkTENmAjcG97g8x8qHpuagA1dvCeopLUrJchl+XAw7XH+6q2OYuIzRExERETBw4cOJGXqL3WSX27JC04Qz0pmplbM3M8M8fHxsZO8DVaS/Nckjr1Euj7gRW1x+dVbSNxbNqikS5Jdb0E+k5gTUSsjoilwCZg+2DLkiTN1ayBnpmHgauB24D7gFszc3dE3BARGwAi4pcjYh/wXuAzEbF7UAU75CJJzXqZ5UJm7gB2dLVdX1vfSWsoZuC8OJckNSvuk6LeU1SSmpUX6N5TVJIalRfooy5Akuap4gK9zR66JHUqL9DtoktSo+IC3Q8WSVKz8gLdeeiS1Ki8QK+WdtAlqVN5ge49RSWpUXmB7j1FJalRcYHeZp5LUqfiAt17ikpSs/ICvb1iF12SOhQX6O0uuidFJalTcYHutEVJalZeoPvBIklqVGCg+9F/SWpSXKC3GeeS1Km4QHfWoiQ1Ky/QvaeoJDUqL9CrpdMWJalTeYHuNBdJalRcoLc55CJJnYoLdDvoktSsuEBvcx66JHUqLtDTiYuS1Ki8QHfIRZIalRfo1dIRF0nq1FOgR8S6iNgTEZMRcW3D86dExC3V83dGxKp+F9p2/rJTeccF57J4kYkuSXVLZtsgIhYDW4BfA/YBOyNie2beW9vsA8CTmfnKiNgE3Aj89iAKvvy1L+Py175sEC8tSUXrpYd+ETCZmXsz83lgG7Cxa5uNwOer9a8BvxpOQ5Gkoeol0JcDD9ce76vaGrfJzMPAU8BL+lGgJKk3Qz0pGhGbI2IiIiYOHDgwzF1L0oLXS6DvB1bUHp9XtTVuExFLgDOBn3a/UGZuzczxzBwfGxs7sYolSY16CfSdwJqIWB0RS4FNwPaubbYDV1br7wH+NY9eRUuSNAyzznLJzMMRcTVwG7AYuDkzd0fEDcBEZm4HPgd8MSImgSdohb4kaYhmDXSAzNwB7Ohqu762/hzw3v6WJkmai+I+KSpJahajGuqOiAPAj07w25cBP+ljOf1iXXNjXXM3X2uzrrk5mbpenpmNs0pGFugnIyImMnN81HV0s665sa65m6+1WdfcDKouh1wkaYEw0CVpgSg10LeOuoAZWNfcWNfczdfarGtuBlJXkWPokqTpSu2hS5K6GOiStEAUF+iz3T1pwPteERHfjoh7I2J3RHy4av94ROyPiF3V19tr3/PRqtY9EfG2Adb2UETcU+1/omo7OyK+FREPVsuzqvaIiL+u6vp+RFw4oJpeXTsmuyLi6Yj4yCiOV0TcHBGPR8QPam1zPj4RcWW1/YMRcWXTvvpQ1yci4v5q31+PiBdX7asi4v9qx+3Tte/5per9n6xqP6n7EcxQ15zft37/vs5Q1y21mh6KiF1V+zCP10zZMNyfscws5ovWtWR+CJwPLAW+B6wd4v7PBS6s1k8HHgDWAh8H/rRh+7VVjacAq6vaFw+otoeAZV1tNwHXVuvXAjdW628HvkHrXtuXAHcO6b17FHj5KI4XcClwIfCDEz0+wNnA3mp5VrV+1gDquhxYUq3fWKtrVX27rtf5r6rWqGpfP4C65vS+DeL3tamuruf/Crh+BMdrpmwY6s9YaT30Xu6eNDCZ+Uhm3l2tPwPcx/SbfdRtBLZl5sHM/G9gkta/YVjqd5L6PPCuWvsXsuUO4MURce6Aa/lV4IeZebxPBw/seGXmv9O6cFz3/uZyfN4GfCszn8jMJ4FvAev6XVdmfjNbN4oBuIPWJatnVNV2Rmbeka1U+ELt39K3uo5jpvet77+vx6ur6mX/FvCV473GgI7XTNkw1J+x0gK9l7snDUW0boT9BuDOqunq6k+nm9t/VjHcehP4ZkTcFRGbq7ZzMvORav1R4JwR1NW2ic5ftFEfL5j78RnFcft9Wj25ttUR8d2I+E5EvKlqW17VMoy65vK+Dft4vQl4LDMfrLUN/Xh1ZcNQf8ZKC/R5ISJOA/4e+EhmPg38DfAK4PXAI7T+7Bu2N2bmhcB64EMRcWn9yaonMpI5qtG6jv4G4KtV03w4Xh1GeXxmEhHXAYeBL1VNjwArM/MNwDXAlyPijCGWNO/ety5X0NlpGPrxasiGo4bxM1ZaoPdy96SBiogX0HrDvpSZ/wCQmY9l5pHMnAL+lmPDBEOrNzP3V8vHga9XNTzWHkqplo8Pu67KeuDuzHysqnHkx6sy1+MztPoi4irgncDvVEFANaTx02r9Llrj06+qaqgPywykrhN434Z5vJYAvwncUqt3qMerKRsY8s9YaYHey92TBqYao/sccF9mfrLWXh9//g2gfQZ+O7ApIk6JiNXAGlonY/pd16kRcXp7ndZJtR/QeSepK4F/rNX1/upM+yXAU7U/Cweho+c06uNVM9fjcxtweUScVQ03XF619VVErAP+DNiQmc/W2sciYnG1fj6t47O3qu3piLik+hl9f+3f0s+65vq+DfP39TLg/sw8OpQyzOM1UzYw7J+xkzmzO4ovWmeHH6D1v+11Q973G2n9yfR9YFf19Xbgi8A9Vft24Nza91xX1bqHkzyTfpy6zqc1g+B7wO72cQFeAtwOPAj8C3B21R7Alqque4DxAR6zU2ndX/bMWtvQjxet/1AeAQ7RGpf8wIkcH1pj2pPV1+8NqK5JWuOo7Z+xT1fbvrt6f3cBdwO/XnudcVoB+0PgU1SfAu9zXXN+3/r9+9pUV9X+d8AHu7Yd5vGaKRuG+jPmR/8laYEobchFkjQDA12SFggDXZIWCANdkhYIA12SFggDXZIWCANdkhaI/wcTa3HaqhU9sQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "-mz2qgb_yJW2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 191
        },
        "outputId": "b16d4220-4afa-4134-ac49-8ec2c481dfbd"
      },
      "source": [
        "w_his[-10:]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[array([0.5]),\n",
              " array([0.5]),\n",
              " array([0.5]),\n",
              " array([0.5]),\n",
              " array([0.5]),\n",
              " array([0.5]),\n",
              " array([0.5]),\n",
              " array([0.5]),\n",
              " array([0.5]),\n",
              " array([0.5])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 150
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Q61I3HknzCWw",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 283
        },
        "outputId": "28fd1455-87c4-4733-9126-42e5aa257012"
      },
      "source": [
        "plt.plot(g_his)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[<matplotlib.lines.Line2D at 0x7f74b758d1d0>]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 151
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAUJElEQVR4nO3df4xlZX3H8fd3Z3b4/VOmlLALu1ps3bRVcUT+UNqIP8C2i63aQNoIrc3WxE211rQYGmJo0gRMbdKUVLGSolFBbU236Rq01rZpUigDorggMlKU3SCsgqDuLjtz77d/3DM75/6Y3Tu798c84/uVTObc55695zvn3vnsM895zjmRmUiSyrdu3AVIkgbDQJekNcJAl6Q1wkCXpDXCQJekNWJyXBs+66yzctOmTePavCQV6d577/1+Zk73em5sgb5p0yZmZ2fHtXlJKlJEfGe55xxykaQ1wkCXpDXCQJekNcJAl6Q1wkCXpDWir0CPiMsi4uGImIuIa3s8f01E7I2I+6uvPxh8qZKkwznitMWImABuBl4P7AbuiYgdmflgx6p3ZOb2IdQoSepDP/PQLwLmMvNRgIi4HbgC6Az0sbr3O0/zyJM/Zv98g/3zDQ7MN8FLA0tahS59ydm8dOPpA3/dfgL9XODx2uPdwKt6rPeWiLgE+Bbwx5n5eOcKEbEN2AZw3nnnrbzaZXzu3t2877Nf62qPGNgmJGlgfubU48cW6P34F+DTmfl8RPwhcBvw2s6VMvMW4BaAmZmZgXSfv/fsAf5y50O8+OyT+djVr+SU4yc5fv0Ex02uI0x0ST9F+jkougfYWHu8oWo7JDN/kJnPVw//HnjFYMo7stv+5zGe/slBbnrrS9l45omcfuIUx6+fMMwl/dTpJ9DvAS6IiM0RMQVcCeyorxAR59QebgUeGlyJh/fdp/dx/gtO5GVD+PNFkkpyxCGXzFyIiO3AncAEcGtm7oqIG4DZzNwB/FFEbAUWgKeBa4ZYc5vdz+znvDNPHNXmJGnV6msMPTN3Ajs72q6vLb8feP9gS+vPnmf2sWXL2ePYtCStKkWfKbr/YIPv//ggG86why5JRQf6nh/uA2DDGSeMuRJJGr+iA/2JZw8AcM5pBrokFR3oP9w3D8AZJ64fcyWSNH5FB/pzB1qBfuoJBrokFR3oz+5vBfppBroklR3oz+1fYGpiHcdNFv1jSNJAFJ2ET/3oAGedPOVp/pJE6YH+3POcfdrx4y5DklaFogP9JwcXOPm4QV0wUpLKVnSg7z/Y4IT1E+MuQ5JWhaIDfd/BBidOGeiSBMUH+gInTDnkIklQfKA3OMkeuiQBBQd6ZrJ/3iEXSVpUbKAfmG+SiUMuklQpNtD3HVwAsIcuSZWCA70BGOiStKjYQN8/3wr0Ewx0SQIKDvT5RhOAyXXF/giSNFDFpmGzledMrPPCXJIEBQd6IxOAiWJ/AkkarGLjsFkF+jovnStJQMmB3jTQJamu2EBvNBeHXAx0SYKCA73Kc3voklQpONDtoUtSXbGB3jg0hj7mQiRplSg30BdnuZjokgQUHOi5OOTiGLokAQUHesMzRSWpTV+BHhGXRcTDETEXEdceZr23RERGxMzgSuxtcQzdDroktRwx0CNiArgZuBzYAlwVEVt6rHcK8G7g7kEX2YuzXCSpXT899IuAucx8NDMPArcDV/RY7y+AG4EDA6xvWU3H0CWpTT+Bfi7weO3x7qrtkIi4ENiYmf86wNoO69C0RXvokgQM4KBoRKwDPgT8SR/rbouI2YiY3bt37zFt1x66JLXrJ9D3ABtrjzdUbYtOAX4R+I+IeAy4GNjR68BoZt6SmTOZOTM9PX30VbN0PXRP/Zekln4C/R7ggojYHBFTwJXAjsUnM/PZzDwrMzdl5ibgLmBrZs4OpeLK0olFw9yKJJXjiHGYmQvAduBO4CHgM5m5KyJuiIitwy5wOU2vtihJbSb7WSkzdwI7O9quX2bdXz32so6s4Ri6JLUpdsBi8fK5YaBLElByoDvkIkltig30Q3cssocuSUDBgd50losktSk2Dg8Fuj10SQIKDnQvnytJ7YoNdHvoktSu2ECfr7rok/bQJQkoONCfX2gyNbHOqy1KUqXYQD+40GRqstjyJWngik3E5xcaBrok1RSbiAerIRdJUkuxiTjfSNZPOn4uSYuKDfRGMz3tX5Jqyg30TGe4SFJNsYHetIcuSW3KDfRMT/uXpJpiA73R9OYWklRXbKC3eujjrkKSVo9iI9FZLpLUrthAbzrLRZLaFBvo9tAlqV3RgW4PXZKWFBvozbSHLkl1BQe6N4iWpLpiI7HRTG8/J0k1xQa6Z4pKUrtiA91ZLpLUruhAd5aLJC0pNtCd5SJJ7YoN9EbTMXRJqis20FvTFg10SVrUV6BHxGUR8XBEzEXEtT2ef2dEPBAR90fEf0fElsGX2q6ZiXkuSUuOGOgRMQHcDFwObAGu6hHYn8rMX8rMlwE3AR8aeKUdnOUiSe366aFfBMxl5qOZeRC4HbiivkJmPld7eBKQgyuxt6azXCSpzWQf65wLPF57vBt4VedKEfEu4L3AFPDaXi8UEduAbQDnnXfeSmtt03CWiyS1GdhB0cy8OTNfBPwZ8OfLrHNLZs5k5sz09PQxba/R9KCoJNX1E+h7gI21xxuqtuXcDrz5WIrqh7egk6R2/UTiPcAFEbE5IqaAK4Ed9RUi4oLaw18DHhlcib15UFSS2h1xDD0zFyJiO3AnMAHcmpm7IuIGYDYzdwDbI+J1wDzwDHD1MIsGb0EnSZ36OShKZu4Edna0XV9bfveA6zqippfPlaQ2xY5CN7x8riS1KTbQm03soUtSTbGB3nCWiyS1KTYSneUiSe2KDPRms3VlAWe5SNKSIgO9ka1At4cuSUuKDPRm2kOXpE5lBnqz9d1ZLpK0pMhAPzTkUmT1kjQcRUZiY/GgqD10STqkyEBfnOXimaKStKTIQF8acjHQJWlRkYHedMhFkrqUGejVHUvtoUvSkiIDfXHIxTyXpCVFBrpDLpLUrchAbzjLRZK6lBnoznKRpC5FBrpDLpLUrchAt4cuSd3KDHR76JLUpchArzroTluUpJoiA91ZLpLUrcxA9wYXktSlyEA/dLVFx9Al6ZAiA90hF0nqVmSgL1SBPmmgS9IhRQb6fKN1U9H1k0WWL0lDUWQizjdaPfT164osX5KGoshEXKh66JMTDrlI0qIiA32+GkNfb6BL0iF9BXpEXBYRD0fEXERc2+P590bEgxHx9Yj4ckScP/hSl8wvVGPoE0X+fyRJQ3HERIyICeBm4HJgC3BVRGzpWO2rwExm/jLwOeCmQRdat9BcHHIx0CVpUT+JeBEwl5mPZuZB4HbgivoKmfmVzNxXPbwL2DDYMtstHRR1yEWSFvUT6OcCj9ce767alvMO4Au9noiIbRExGxGze/fu7b/KDosHRR1ykaQlA03EiPhdYAb4YK/nM/OWzJzJzJnp6emj3s5iD91ZLpK0ZLKPdfYAG2uPN1RtbSLidcB1wK9k5vODKa+3+aY9dEnq1E8i3gNcEBGbI2IKuBLYUV8hIl4OfATYmplPDb7MdgsNT/2XpE5HDPTMXAC2A3cCDwGfycxdEXFDRGytVvsgcDLw2Yi4PyJ2LPNyAzHfaBLhxbkkqa6fIRcycyews6Pt+try6wZc12HNN5L169YRXj5Xkg4pchC60WzaO5ekDkUGejO9n6gkdSoy0DNxuEWSOpQZ6CTmuSS1KzPQE8xzSWpXaKCnQy6S1KHMQMeDopLUqchAb9pDl6QuRQZ6Om1RkroUGeitO9CZ6JJUV2Sg47RFSepSZKA75CJJ3YoM9GYm4ZCLJLUpMtDtoUtStyIDvem1XCSpS5GBnuS4S5CkVafIQCdhXZmVS9LQFBmLHhSVpG5FBrrXcpGkbkUGugdFJalbkYGemQ64SFKHMgMdPPVfkjqUGehePleSuhQa6F5rUZI6FRnozUzW2UOXpDZFBnqmY+iS1KnMQMdpi5LUqcxAd9qiJHUpMtCbXstFkroUGYuNZjLhkIsktSky0JvOQ5ekLn0FekRcFhEPR8RcRFzb4/lLIuK+iFiIiLcOvsx2zUwmvDqXJLU5YqBHxARwM3A5sAW4KiK2dKz2XeAa4FODLrAXh1wkqdtkH+tcBMxl5qMAEXE7cAXw4OIKmflY9VxzCDV2aTY9KCpJnfqJxXOBx2uPd1dtY9NwyEWSuoy0nxsR2yJiNiJm9+7de9Sv02h66r8kdeon0PcAG2uPN1RtK5aZt2TmTGbOTE9PH81LAB4UlaRe+gn0e4ALImJzREwBVwI7hlvW4XlQVJK6HTHQM3MB2A7cCTwEfCYzd0XEDRGxFSAiXhkRu4G3AR+JiF3DLLrRTNbZQ5ekNv3MciEzdwI7O9qury3fQ2soZiSaaQ9dkjoVOfmv0XQMXZI6FRnorYtzGeiSVFdkoLcOio67CklaXYoNdHvoktSuyED3oKgkdSsy0D0oKkndigz0ZjrkIkmdigx0zxSVpG7lBro9dElqU2SgNxOvtihJHYoM9FYPfdxVSNLqUmQsNjwoKkldigz0pgdFJalLkYHeSO9YJEmdigv0zCS9OJckdSku0JvZ+u6QiyS1Ky7QG1WiO8tFktoVF4vNbAW6Qy6S1K64QD/UQ3fIRZLalBfouTjkYqBLUl1xgd6seuhOW5SkdsUF+tJBUQNdkurKC3QPikpST8UFerPZ+u5BUUlqV1ygLx0UHXMhkrTKFBeLHhSVpN6KC3QPikpSb+UFuvPQJamn4gJ9/8EGAMevnxhzJZK0uhQX6D86sADAKcdNjrkSSVpdigv0x5/ZB8DpJ06NuRJJWl2KC/QXnDTF67eczS/87CnjLkWSVpW+Aj0iLouIhyNiLiKu7fH8cRFxR/X83RGxadCFLrr0JWfz0bfPeKaoJHU4YqBHxARwM3A5sAW4KiK2dKz2DuCZzPw54K+BGwddqCTp8PrpoV8EzGXmo5l5ELgduKJjnSuA26rlzwGXRnjmjySNUj+Bfi7weO3x7qqt5zqZuQA8C7yg84UiYltEzEbE7N69e4+uYklSTyM9KJqZt2TmTGbOTE9Pj3LTkrTm9RPoe4CNtccbqrae60TEJHAa8INBFChJ6k8/gX4PcEFEbI6IKeBKYEfHOjuAq6vltwL/nlmdoy9JGokjnm6ZmQsRsR24E5gAbs3MXRFxAzCbmTuAjwGfiIg54GlaoS9JGqG+zp/PzJ3Azo6262vLB4C3DbY0SdJKxLhGRiJiL/Cdo/znZwHfH2A5g2JdK2NdK7daa7OulTmWus7PzJ6zSsYW6MciImYzc2bcdXSyrpWxrpVbrbVZ18oMq67iruUiSerNQJekNaLUQL9l3AUsw7pWxrpWbrXWZl0rM5S6ihxDlyR1K7WHLknqYKBL0hpRXKAf6WYbQ972xoj4SkQ8GBG7IuLdVfsHImJPRNxffb2p9m/eX9X6cES8cYi1PRYRD1Tbn63azoyIL0XEI9X3M6r2iIi/qer6ekRcOKSafr62T+6PiOci4j3j2F8RcWtEPBUR36i1rXj/RMTV1fqPRMTVvbY1gLo+GBHfrLb9+Yg4vWrfFBH7a/vtw7V/84rq/Z+raj+my1cvU9eK37dB/74uU9cdtZoei4j7q/ZR7q/lsmG0n7HMLOaL1qUHvg28EJgCvgZsGeH2zwEurJZPAb5F66YfHwDe12P9LVWNxwGbq9onhlTbY8BZHW03AddWy9cCN1bLbwK+AARwMXD3iN677wHnj2N/AZcAFwLfONr9A5wJPFp9P6NaPmMIdb0BmKyWb6zVtam+Xsfr/G9Va1S1Xz6Eulb0vg3j97VXXR3P/xVw/Rj213LZMNLPWGk99H5utjE0mflEZt5XLf8IeIjua8PXXQHcnpnPZ+b/AXO0foZRqd945DbgzbX2j2fLXcDpEXHOkGu5FPh2Zh7u7OCh7a/M/C9a1xnq3N5K9s8bgS9l5tOZ+QzwJeCyQdeVmV/M1n0FAO6idYXTZVW1nZqZd2UrFT5e+1kGVtdhLPe+Dfz39XB1Vb3s3wY+fbjXGNL+Wi4bRvoZKy3Q+7nZxkhE676pLwfurpq2V3863br4ZxWjrTeBL0bEvRGxrWo7OzOfqJa/B5w9hroWXUn7L9q49xesfP+MY7/9Pq2e3KLNEfHViPjPiHhN1XZuVcso6lrJ+zbq/fUa4MnMfKTWNvL91ZENI/2MlRboq0JEnAz8I/CezHwO+DvgRcDLgCdo/dk3aq/OzAtp3fv1XRFxSf3Jqicyljmq0brs8lbgs1XTathfbca5f5YTEdcBC8Anq6YngPMy8+XAe4FPRcSpIyxp1b1vHa6ivdMw8v3VIxsOGcVnrLRA7+dmG0MVEetpvWGfzMx/AsjMJzOzkZlN4KMsDROMrN7M3FN9fwr4fFXDk4tDKdX3p0ZdV+Vy4L7MfLKqcez7q7LS/TOy+iLiGuDXgd+pgoBqSOMH1fK9tManX1zVUB+WGUpdR/G+jXJ/TQK/BdxRq3ek+6tXNjDiz1hpgd7PzTaGphqj+xjwUGZ+qNZeH3/+TWDxCPwO4MqIOC4iNgMX0DoYM+i6ToqIUxaXaR1U+wbtNx65GvjnWl1vr460Xww8W/uzcBjaek7j3l81K90/dwJviIgzquGGN1RtAxURlwF/CmzNzH219umImKiWX0hr/zxa1fZcRFxcfUbfXvtZBlnXSt+3Uf6+vg74ZmYeGkoZ5f5aLhsY9WfsWI7sjuOL1tHhb9H63/a6EW/71bT+ZPo6cH/19SbgE8ADVfsO4Jzav7muqvVhjvFI+mHqeiGtGQRfA3Yt7hdaN+r+MvAI8G/AmVV7ADdXdT0AzAxxn51E63aEp9XaRr6/aP2H8gQwT2tc8h1Hs39ojWnPVV+/N6S65miNoy5+xj5crfuW6v29H7gP+I3a68zQCthvA39LdRb4gOta8fs26N/XXnVV7f8AvLNj3VHur+WyYaSfMU/9l6Q1orQhF0nSMgx0SVojDHRJWiMMdElaIwx0SVojDHRJWiMMdElaI/4fsygGWQ97EtcAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eXpkOBIM0HVl",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 191
        },
        "outputId": "f9c1fbc2-4792-404e-9b65-c97bb793b1bc"
      },
      "source": [
        "g_his[-10:]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[array([-0.05074037]),\n",
              " array([-0.05074037]),\n",
              " array([-0.05074037]),\n",
              " array([-0.05074037]),\n",
              " array([-0.05074037]),\n",
              " array([-0.05074037]),\n",
              " array([-0.05074037]),\n",
              " array([-0.05074037]),\n",
              " array([-0.05074037]),\n",
              " array([-0.05074037])]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 75
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bzoHje7W0Rgq"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}