heatmap_handpicked.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## LSTM Activation- Hand picked cases\n",
    "\n",
    "After studying the [Effectiveness of RNNs and LSTMs](http://karpathy.github.io/2015/05/21/rnn-effectiveness/), we were eager to duplicate and enhance the results showcased in this blog. After trying character prediction using LSTM model on [Shakespeare data](http://www.gutenberg.org/ebooks/100), we wanted to visualize the activations of the LSTM cells to find some pattern in the way the cells are storing/processing information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 233
    },
    "colab_type": "code",
    "id": "E3V4V-Jxmuv3",
    "outputId": "3190f6c3-b742-424f-8ec6-7ccecf8a3568"
   },
   "outputs": [],
   "source": [
    "# Import stuff\n",
    "\n",
    "import numpy as np\n",
    "import six\n",
    "import tensorflow as tf\n",
    "import time\n",
    "import os\n",
    "import seaborn as sns\n",
    "from matplotlib import pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The main purpose of this notebook is to visualize the LSTM cell activations of language model. The training part of the model and the prediction results can be found [here](https://github.com/InFoCusp/lstm_deepdive/blob/master/shakespeare_writing_style_lstm.ipynb). Here, we are directly loading the saved weights and showcasing some of the trends that we obsereved. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Function to convert text into integers (also dropping any non-ascii characters)\n",
    "\n",
    "def transform(txt, pad_to=None):\n",
    "    output = np.asarray([ord(c) for c in txt if ord(c) < 255], dtype=np.int32)\n",
    "    if pad_to is not None:\n",
    "        output = output[:pad_to]\n",
    "        output = np.concatenate([\n",
    "            np.zeros([pad_to - len(txt)], dtype=np.int32),\n",
    "            output,\n",
    "        ])\n",
    "    return output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "def lstm_model(seq_len=100, batch_size=None, stateful=True):\n",
    "    \"\"\"Language model: predict the next word given the current word.\"\"\"\n",
    "    source = tf.keras.Input(\n",
    "      name='seed', shape=(seq_len,), batch_size=batch_size, dtype=tf.int32)\n",
    "\n",
    "    embedding = tf.keras.layers.Embedding(input_dim=256, output_dim=EMBEDDING_DIM)(source)\n",
    "    lstm_1 = tf.keras.layers.LSTM(EMBEDDING_DIM, stateful=stateful, return_sequences=True)(embedding)\n",
    "    lstm_2 = tf.keras.layers.LSTM(EMBEDDING_DIM, stateful=stateful, return_sequences=True)(lstm_1)\n",
    "    predicted_char = tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(256, activation='softmax'))(lstm_2)\n",
    "    model = tf.keras.Model(inputs=[source], outputs=[predicted_char,lstm_1,lstm_2])\n",
    "    # Here we are taking LSTM cell states of both the layers alongwith the output predicted character\n",
    "    \n",
    "    model.compile(\n",
    "      optimizer=tf.train.RMSPropOptimizer(learning_rate=0.01),\n",
    "      loss='sparse_categorical_crossentropy',\n",
    "      metrics=['sparse_categorical_accuracy'])\n",
    "    return model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "BATCH_SIZE = 1\n",
    "PREDICT_LEN = 250\n",
    "EMBEDDING_DIM = 512"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 2987
    },
    "colab_type": "code",
    "id": "tU7M-EGGxR3E",
    "outputId": "33eeff13-12f8-4e75-89a6-6e6b08655ac0"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "WARNING:tensorflow:From /home/nisarg/.local/lib/python3.6/site-packages/tensorflow/python/ops/resource_variable_ops.py:435: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.\n",
      "Instructions for updating:\n",
      "Colocations handled automatically by placer.\n"
     ]
    }
   ],
   "source": [
    "# Defining the prediction model \n",
    "\n",
    "BATCH_SIZE = 1\n",
    "prediction_model = lstm_model(seq_len=1, batch_size=BATCH_SIZE, stateful=True)\n",
    "\n",
    "# A explained in the training notebook, Keras requires the batch size be specified ahead of time for stateful models.\n",
    "# We use a sequence length of 1, as we will be feeding in one character at a \n",
    "# time and predicting the next character."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 2987
    },
    "colab_type": "code",
    "id": "tU7M-EGGxR3E",
    "outputId": "33eeff13-12f8-4e75-89a6-6e6b08655ac0"
   },
   "outputs": [],
   "source": [
    "prediction_model.load_weights('model_weights/shakespeare_handpicked_heatmap.h5')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Prediction Function:\n",
    "\n",
    "This function takes a test text sequence and the trained model as inputs. <br>\n",
    "It predicts the next character for each input character from the input text sequence. Alongwith the generated output, we are also storing the LSTM cell states at each character level. The LSTM layer has 512 cells. Thus, we have [512 x len(`seed_text`)] number of states at each layer. This model has two such LSTM layers. <br>\n",
    "After storing the LSTM cell values, the function plots the heatmap of the cell values of layer number given by `node_num` and plots the activations of the cells between `start_node` and `end_node` at each input step of seed text. The cell values range from -1 to 1 and the colors of the cells indicate this range. Both positive and negative extremities are considered as useful activations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def predict(seed_text, prediction_model, start_node, end_node, node_num):\n",
    "    seed = transform(seed_txt)\n",
    "    seed = np.repeat(np.expand_dims(seed, 0), BATCH_SIZE, axis=0)\n",
    "    PREDICT_LEN = len(seed_txt)-1\n",
    "    print(\"Length of input text: \" + str(PREDICT_LEN))\n",
    "    \n",
    "    predicted_char_o = []\n",
    "    lstm_1_o = []\n",
    "    lstm_2_o = []\n",
    "\n",
    "    # First, run the seed forward to prime the state of the model.\n",
    "    prediction_model.reset_states()\n",
    "    \n",
    "    for i in range(len(seed_txt) - 1):\n",
    "        [predicted_char_temp,lstm_1_temp,lstm_2_temp] = prediction_model.predict(seed[:, i:i + 1])\n",
    "        predicted_char_o.append(predicted_char_temp[0,0,:])\n",
    "        lstm_1_o.append(lstm_1_temp[0,0,:])\n",
    "        lstm_2_o.append(lstm_2_temp[0,0,:])\n",
    "    \n",
    "    print(\"LSTM Layer\" + str(node_num))\n",
    "    plt.figure(figsize=[24,4])\n",
    "    if node_num == 1:\n",
    "        ax2 = sns.heatmap(np.array(lstm_1_o).T[start_node:end_node,:] ,xticklabels=list(seed_txt), )\n",
    "    else:\n",
    "        ax2 = sns.heatmap(np.array(lstm_2_o).T[start_node:end_node,:] ,xticklabels=list(seed_txt), )\n",
    "    ax2.tick_params(labelsize=18)\n",
    "    plt.show()\n",
    "    \n",
    "    return"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Analysis:\n",
    "\n",
    "Now, we analyse some special cases by providing the input text accordingly. We plot the heat-map of the cell which shows high or low activation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1. Full Stops:\n",
    "\n",
    "Observe how the cells give negative activation (black, dark purple in heatmap) at full-stops. There is no activation at the larger space provided at the end of the input text. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of input text: 99\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABMMAAAEECAYAAAAlPPOQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XucdVddH/7PNwlJQC6CsYABDEpKidaXQkStghiCBVsMaLVcKsSKqVBrvVAbCxXEnxZUiiKpErwkiDeqaIINck3wUkQiSDGhQAhBwBAIt4ghl+eZ9ftj76HDZC7rzMyZc2b2+/28zmues8939l577bXXXmfN2mtXay0AAAAAMAXHLDoBAAAAALBfdIYBAAAAMBk6wwAAAACYDJ1hAAAAAEyGzjAAAAAAJkNnGAAAAACToTMMAAAAgMnQGQYAAADAZOgMAwAAAGAyjlt0At5677NaT1xV/zpb1xrn64k3X9sVd+PRm7rX+Qd3uO9Ok7Ol3rS+74YPd6/zE3/87O7Yd37XRd2xvXr3KUk+fOPHu2Nff9cv20lytnXcsStdcf/6xuvmsv1ZvO3iH+mKu/I7fmsu25/l2Lb0VQa/dcIX7TQ5S22WvHrPJz/UHfuXdz99J8lZiFnyoNdvnnDPPV9nkhxzTP/F6/Gf6auPb7j1H7rX+co73q87dhan/c8ndMfOo96YpQxcf/OnumNPOuEuXXG9dWayHPXmPPzdjR/rjr30rl8+x5Rsb5a8esDt79Ed+8yVGRqyc3Di8Ue6Y2+6Ze+/HsySr9d95hPdsa/7/NN2kpw9M8t+ffzmG7pjX3Xn++8kOXvmn77tBd2xn/mx7+uKu+qPb9+9zlPPurU7dhZHrruxK+59f9FXv89qluvhV33r87tje9tws7Tfjq70j5k59pi+7zFJ8oMrn+6K+/lj7ti9znl64Ac2/J662Ap9idx6/dVdjdfbnfQlS5lnRoYBAAAAMBkLHxkGAAAAwAGycnTRKdgVnWEAAAAA9Dvafwv+MtIZBgAAAEC31vrni1tGOsMAAAAA6LeiMwwAAACAqTAyDAAAAIDJMIE+AAAAAJNhZBgAAAAAU9E8TRIAAACAyTCBPgAAAACT4TZJAAAAACbjgE+gf8yiEwAAAADAAdJW+l4dquqRVfWuqrqqqs7d4PP7VNWlVfW2qvo/VfUtu02+kWEAAAAA9NujCfSr6tgk5yV5RJIPJnlLVV3cWrtyTdgzk7y8tfZLVXVakkuSnLKb7RoZBgAAAEC/lZW+1/YenOSq1trVrbVbkvxOkrPWxbQkdx7/f5ckf7fb5BsZBgAAAEC31vZszrCTk3xgzfsPJvmadTHPTvKaqvoPST4vyZm73aiRYQAAAAD065wzrKrOqarL17zO2cHWHp/kgtbavZJ8S5LfqKpd9WcZGQYAAABAv75bINNaOz/J+VuEfCjJvde8v9e4bK3vSfLIcX1vqqoTk5yU5CO9yV3PyDAAAAAA+u3d0yTfkuTUqrpvVR2f5HFJLl4X87dJHp4kVfWAJCcm+ehukm9kGAAAAAD9jt66J6tprR2pqu9P8uokxyb5tdbaFVX1nCSXt9YuTvIjSV5SVT+UYTL9s1trbTfb1RkGAAAAQL/O2yR7tNYuSXLJumU/vub/Vyb5+j3bYHSGAQAAADCLvlsgl5bOMAAAAAD67eHIsEXQGQYAAABAP51hAAAAAExF26MJ9BdFZxgAAAAA/cwZBgAAAMBkuE0SAAAAgMkwMgwAAACAyTAyDAAAAIDJMDIMAAAAgMk4cmTRKdgVnWEAAAAA9DMyDAAAAIDJMGcYAAAAAJNhZBgAAAAAk2FkGAAAAACTYWQYAAAAAJPhaZIAAAAATEZri07BrugMAwAAAKCfOcMAAAAAmAydYQAAAABMhgn0AQAAAJiMo0cXnYJd0RkGAAAAQD+3SQIAAAAwGTrDAAAAAJgMc4YBAAAAMBVtpS06CbuiMwwAAACAfm6TBAAAAGAyPE0SAAAAgMkwMgwAAACAydAZBgAAAMBkNBPoAwAAADAVRoYBAAAAMBkrRoYBAAAAMBWeJgkAAADAVDS3SQIAAAAwGW6TBAAAAGAympFhAAAAAEzFAR8ZdsyiEwAAAADAAXLkaN+rQ1U9sqreVVVXVdW5W8R9e1W1qjp9t8nXGQYAAABAv7bS99pGVR2b5Lwkj0pyWpLHV9VpG8TdKcl/TPLmvUi+zjAAAAAA+q20vtf2Hpzkqtba1a21W5L8TpKzNoj7ySTPS3LTXiRfZxgAAAAA3drKSterw8lJPrDm/QfHZZ9VVQ9Mcu/W2v/aq/SbQB8AAACAfp0T6FfVOUnOWbPo/Nba+b2bqapjkvz3JGfPkrzt6AwDAAAAoF9nZ9jY8bVV59eHktx7zft7jctW3SnJlye5rKqS5B5JLq6qb22tXT5LktfSGQYAAABAv6N9T4rs8JYkp1bVfTN0gj0uyRNWP2ytfSrJSavvq+qyJE/fTUdYojMMAAAAgBm0zpFh266ntSNV9f1JXp3k2CS/1lq7oqqek+Ty1trFe7KhdXSGAQAAANBvjzrDkqS1dkmSS9Yt+/FNYh+2F9vUGQYAAABAv74nRS4tnWEAAAAA9NvDkWGLoDMMAAAAgH46wwAAAACYinbUbZIAAAAATIWRYQAAAABMRdMZBgAAAMBk6AwDAAAAYDIO9pRhOsMAAAAA6NeOHOzeMJ1hAAAAAPQ72H1hOsMAAAAA6GcCfQAAAACmw8gwAAAAAKbCyDAAAAAApsPIMAAAAACmoh1ZdAp2R2cYAAAAAN2akWEAAAAATIbOMAAAAACmwsgwAAAAACZDZxgAAAAAk9GO1qKTsCs6wwAAAADoZmQYAAAAAJPRVowMAwAAAGAijAwDAAAAYDJaMzIMAAAAgIkwMgwAAACAyVjxNEkAAAAApsIE+gAAAABMhs4wAAAAACajtUWnYHd0hgEAAADQzcgwAAAAACajNZ1hAAAAAEzEUU+TBAAAAGAqjAwDAAAAYDLMGQYAAADAZHiaJAAAAACTcdBHhh2z6AQAAAAAcHAcXTmm69Wjqh5ZVe+qqquq6twNPj+hqn53/PzNVXXKbtO/o86wqjququ5cVUaWAQAAAExIa32v7VTVsUnOS/KoJKcleXxVnbYu7HuSfKK1dr8kL0jyvN2mv7szrKoeV1V/VFXXJbk5ySeS3FxV143LH7/bxAAAAACw3FZadb06PDjJVa21q1trtyT5nSRnrYs5K8mF4/9/L8nDq2pX92luO7Krqu6Q5OIkZyS5MclfJ7ksyU1JTkxycpKHJXlUVT0lyaNbazfuJlEAAAAALKfW19HV4+QkH1jz/oNJvmazmNbakar6VJIvSHL9Tjfac5vjc5I8NMkPJHlJa+3m9QFVdUKSc5I8P8lPJPlPO00QAAAAAMur92mSVXVOhv6iVee31s6fR5pm0XOb5HcmeWFr7UUbdYQlSWvt5tbaLyb5xSSP226FVXVOVV1eVZe/4tPXzJRgAAAAABan9zbJ1tr5rbXT17zWd4R9KMm917y/17hsw5hx7vq7JPnYbtLf0xn2hUne2bm+K5OctF3Q2sz4tjue0rlqAAAAABZtD58m+ZYkp1bVfavq+AwDrC5eF3NxkieP//9XSd7QWu/YtI31pOyaJI/sXN+3jPEAAAAAHEKt87Xtelo7kuT7k7w6w0Csl7fWrqiq51TVt45hv5rkC6rqqiQ/nOTc3aa/Z86w85M8v6penuTnk7yltXbr6odVdbsMs///YJLHJHn6bhMFAAAAwHLqfFJkl9baJUkuWbfsx9f8/6Yk37FnG0xfZ9jPJzklQ0/dtydZqarrk9yc5IQMt0WujjA7b4wHAAAA4BDaw6dJLsS2nWHjfZj/sapenOQJSU5P8kVJ7pDkhiRvz3CP5++21v5mjmkFAAAAYMFWFp2AXeoZGZYkaa1dmeSZc0wLAAAAAEuu5ZCPDAMAAACAVUcO+22SAAAAALDKyDAAAAAAJmMyc4YBAAAAgJFhAAAAAEyGkWEAAAAATMZRI8MAAAAAmIqVg90XpjMMAAAAgH4rRoYBAAAAMBVt0QnYJZ1hAAAAAHQzgT4AAAAAk7FSbpMEAAAAYCKOLjoBu6QzDAAAAIBuniYJAAAAwGR4miQAAAAAk+FpkgAAAABMhtskAQAAAJiMlUUnYJd0hgEAAADQ7aiRYQAAAABMhZFhAAAAAEyGzjAAAAAAJqO5TRIAAACAqTAyDAAAAIDJOLroBOySzjAAAAAAuq24TRIAAACAqXCbJAAAAACToTMMAAAAgMloi07ALukMAwAAAKCbOcMAAAAAmAxPkwQAAABgMlYO+I2SOsMAAAAA6GYCfQAAAAAm42CPC9MZBgAAAMAMjAwDAAAAYDKO1MEeG3bMohMAAAAAwMHROl+7VVV3q6rXVtV7xp933SL2zlX1wap60Xbr1RkGAAAAQLeVztceODfJ61trpyZ5/fh+Mz+Z5E96VqozDAAAAIBuK2ldrz1wVpILx/9fmOQxGwVV1YOS3D3Ja3pWqjMMAAAAgG77dZtkkru31q4d///hDB1en6Oqjkny/CRP712pCfQBAAAA6NZ7C2RVnZPknDWLzm+tnb8u5nVJ7rHBrz9j7ZvWWqvacOb+pyW5pLX2warqSpfOMAAAAAC6He0c9zV2fJ2/TcyZm31WVddV1T1ba9dW1T2TfGSDsK9L8pCqelqSOyY5vqo+3VrbdH4xnWEAAAAAdNujyfF7XJzkyUmeO/68aH1Aa+2Jq/+vqrOTnL5VR1hizjAAAAAAZtA6/+2B5yZ5RFW9J8mZ4/tU1elV9Ss7XamRYQAAAAB026+RYa21jyV5+AbLL0/ylA2WX5Dkgu3WqzMMAAAAgG4re/WsyAXRGQYAAABAt4PdFaYzDAAAAIAZHDng3WE6wwAAAADotkeT4y+MzjAAAAAAuu3XBPrzojMMAAAAgG5GhgEAAAAwGUaGAQAAADAZR5uRYQAAAABMxIrbJAEAAACYCnOGAQAAADAZ5gwDAAAAYDLcJgkAAADAZLhNEgAAAIDJ8DRJAAAAACbDbZIAAAAATIYJ9AEAAACYDHOGAQAAADAZbpMEAAAAYDKaCfQBAAAAmIqjRoYBAAAAMBVukwQAAABgMtwmCQAAAMBkGBkGAAAAwGQ0nWEAAAAATMVRt0kCAAAAMBVukwQAAABgMnSGAQAAADAZniYJAAAAwGQYGQYAAADAZHiaJAAAAACTcbStLDoJu6IzDAAAAIBu5gwDAAAAYDLMGQYAAADAZJgzDAAAAIDJWHGbJAAAAABTcdBHhh2z6AQAAAAAcHAcbStdr92qqrtV1Wur6j3jz7tuEvczVXVFVb2zql5YVbXVenWGAQAAANBtpbWu1x44N8nrW2unJnn9+P5zVNU/S/L1Sb4iyZcn+eok37jVSnWGAQAAANCtdf7bA2cluXD8/4VJHrNhcpITkxyf5IQkt0ty3VYrNWcYAAAAAN32cQL9u7fWrh3//+Ekd18f0Fp7U1VdmuTaJJXkRa21d261Up1hAAAAAHTrHfVVVeckOWfNovNba+evi3ldknts8OvP+Jxtttaq6jYbrqr7JXlAknuNi15bVQ9prf3pZunSGQYAAABAt6PtaFfc2PF1/jYxZ272WVVdV1X3bK1dW1X3TPKRDcIem+QvWmufHn/nVUm+LsmmnWHmDAMAAACgW2ut67UHLk7y5PH/T05y0QYxf5vkG6vquKq6XYbJ87e8TVJnGAAAAADdVtK6XnvguUkeUVXvSXLm+D5VdXpV/coY83tJ3pvkHUnenuTtrbVXbrVSt0kCAAAA0G2PRn31bOdjSR6+wfLLkzxl/P/RJP9ulvXqDAMAAACg2z4+TXIudIYBAAAA0K33aZLLSmcYAAAAAN2OtpVFJ2FXdIYBAAAA0G2/5gybF51hAAAAAHQzZxgAAAAAk2FkGAAAAACTsWICfQAAAACmwsgwAAAAACbD0yQBAAAAmAwT6AMAAAAwGW6TBAAAAGAymgn0AQAAAJgKI8MAAAAAmIyDPmdYWmtL+Upyzl7GLUPsord/WNN6WPdr0ds/rGk9rPu16O0f1rQe1v1a9Pal9WDFLnr70nqwYhe9fWk9WLGL3r60Lj520dufZ6zXcr8WnoBNE5ZcvpdxyxC76O0f1rQe1v1a9PYPa1oP634tevuHNa2Hdb8WvX1pPVixi96+tB6s2EVvX1oPVuyity+ti49d9PbnGeu13K9jAgAAAAAToTMMAAAAgMlY5s6w8/c4bhliF739WWIXvf15xS56+7PELnr7s8Quevvzil309meJXfT2Z4ld9PbnFbvo7c8Su+jtzxK76O3PErvo7c8rdtHbnyV20dufJXbR259X7KK3P0vsorc/S+yitz+v2EVvf5bYRW9/lthFb3+W2EVvf56xLLEa73sFAAAAgENvmUeGAQAAAMCeWsrOsKp6WFW1qjp70WmBqaqq+1bVH1bVR8fz8YI9WGf3ua0eOFgWcbyq6uxxmw+b4zaWphxW1TVVddmi07HWMh2DZThW86g3Z9z+wvPgIFrGc2s/KC+H00E6rgcprcDhc9yiE8Bsxi8cl65ZtJLkhiQfSvJXSX47yaub+1/ZvQuSfEWSn0ry4STvXWhq9klVfUmSc5M8NMl9ktycYf//MskFrbVL18V/ZZLHjJ9ds7+pBZbMBZlgvclsXDdgNs4ZYB50hh1cv53kkiSV5E5J7p/hIvGkJK+rqu9orX1ygenjAKuqE5I8JMmLWms/t+j07JeqOj3JG5PcmuSlSa5Icvskpyb55iR/n8/tjE6Sr0zyrCSXJblmn5IKLJmp1pvsyLJcN/4kwzXu1gWmgb13kI5rb1qX5ZwBDhGdYR2q6tgkJ7TWblx0WtZ4a2vtZWsXVNUPJ/mZJD+cobPsUYtIGIfC3TN0tH580QnZZ89KcockX9lae/v6D6vqHvufJA67Jb3GMLup1pvss72qM1prK0lu2ptUsSwO0nE9SGkFDp+lnDNsO1V1QlX9l6q6oqpuqqpPVtUrq+qr1sXdqar+v6p6c1VdX1U3V9VVVfXcqrrDJutenf/kzKr6r1X13gyV9HduEHNGVT29qt47rvvdVfXk+e795lprR1trP5Lkz5I8sqq+oapOqqrzquoDVXXL+PO8qvqCeaenqh475tP3bvL5FePxqPF9V1qr6tnjek/ZYJ37Nu9HVZ04puVdVXXjWA7fUVU/uy6uq7zOGjvH/bogyfvHt88a8/pz5gRalmMwB6cm+dhGHWFJ0lr78Nr3VfXsJL8+vr10TV5dsC6ut2yvrX+eXVXvH+uW/1NVj9soTb3lcJFmLS9VdXxV/WhV/fW4T5+qqsur6vs7t/eMcXu/WFVzvc7NWsd2XmPuXVUvH/f7hrEO+NJ57sceOGa76+Ey1G/z0lNvjnG97ZdDmVdVdUpV/f5Yrm+oqotqmGdto3qg+9yaoY5d+LnVe91YE79lnVEztrXWfbbpfE01Y1t3lmM7D7Okd7vyUlVfPK7rJ9b93qvH5T+0bvmbq+qdc9qvmfN1m+P6qPGzH9jkd99Uw5yHtxvfz7Uu2iqta2KenRnOmRm23duO37S8rCl3D6+qH6+h3faZsUx87fj731hVf1ZV/1BV147n8aGs42fVcwxqhrZxzfjdvzrbm47X4XXgRoaNlfMfJ/lnSX4jyYuS3CXJ9yb586p6aGvt8jH85CRPSfL7SX4ryZEk35jkR5N8VZJ/vsWmfi7J7ZK8JMOcXO/aIOanMwztfXGGeYWemuSCqrqqtfbnu9jN3frVJN+Q5NuS/Msk90vya0nemmG/n5rkjKp6cGvt7+eYjldmmDPl32bIx88aLxCnJXlGa61V1V2S/O8FpnUnzsuwby9N8t8znE+nJjljNWCW8jpj2Z6nFyf56yQvSPIHSV4xLp9LQ2/JvDfJ/avq21prr9g2esibeyY5J0N9sJpHn50naIdl+3lJPi/J/xjff3eS366qE1trF6yL3bYcHiRVdXySVyd5WJLXJHlZhi9+/zRDnfaiLX732PHz70vyY6215845rbuptza8xlTV52e4beTeSX45yZUZrluXZrjeLKstr4dLVL/Ny7b1Zm8eHNa8qqGj4U8zjKD75Qx585AMZfvz1sV2n1u9sUt0bm173djEZu3S7rbWDtO7bVt3lmO7D7ari3rKy/ur6uoM19FnJZ+9Nn1Dhrl6z8hwrqeq7pzkQeP29tSc8vU1GcrLk5K8cN32Tk3ytUle2Fq7dYnqop2eM9vpacdvWV6S/MIY+twkx47vj0/yI0leU1VPyvC97Pwkv5mhE/s5SR6f5EtziOr4HZqlDdvTNu7+7t/b3lyi84B5aK0t3StDoWxJzt7gsx8aP/vn65bfOcnfJrlszbLjk9xug3X85LiOB2/w2dnjZ+9KcodN0rca87Ykx69ZfnKGC+9v70PePH2LmAeOMf93/Pm0dZ//+3H5T+7DsfzpcVunrVv+kgwV1BeN73+qN61Jnj0uO2WD7V2ztgzMed8+nuSSbWJmKa/dsfuwb6eMaXn2Jp/v6BhsdW7vJnYP9/vrktwybvfdGRo+T03ygC1+Z7U+eNgmn89StlfX9f4kd1mz/C7jso8nuf2s5XCf8m7T4zVLecnQYGlJfnqD2GM2y/sMX37+YDx+T9qn/eo+thukecNrTP5fnfnd65b//Lj8sp3uy5yO++r+bHk9zC7qt966YBF1xrrtn5Kt682uPNiPvFpQ/vzMmLYnbrL8sjXLZqk3u2KX6dzKNteNTWK3apd2tbVmKS+95/asx3Yf8nS7uqi3vLwkw/XkDuP7h46f/0aGzsjjxuWPHpd/+yLPmd7jOn7+s5uUl9XvRw8c38+9XbpdWndyzsyw7Z52/Hbl5aLx51vXlbtvHZffmuT0NcuPH8vPXPP1oLw6j8Hqsd+2bZwZvvuns725H+eB1+JeB/E2yX+ToZPnr8ZhqydV1UkZCv9rk3xDVd0+SVprt7TWbk2Sqjququ46xr5uXNfXbLGdX2rbz8XwP1prt6y+aa19KMMX6FN3tGd754bx572SfDTDXyPWevG4/LH7kJaXZKhAvmd1QVV9XpJ/neRVrbW/Gxc/NotP66w+leTLqurLt4jpLq8zxjIHrbU3ZfgL74UZLrLfneEvUFdW1Z/U8KTJWe2kbP9Sa+1Ta9L1qQx/Fb5rhobjWj3l8CB5YpJPZPjL6edow9wiG7lbhnPkzCSPbq29dH7J+xy7qbc2u8Y8Jsl1Gf5KutbzdprIfbLd9VD91p8HhzWvHp3k2gxzmq610cMGZjm3emMP6rm1aqt2aW9bayd62rqzHNt52y69veXlDRlG4j1kfH9Gko9kGPlzpyRfPS7/pgx5f+me7sVgXvl64fjzSasLqqoy1D1/01p767j4sNZFq3raT9uVlweN739pbbnLMKIvSd7c1owaGmNWkhzN4c3XWczSht22bTzjd//e9uZhPw8m7cDdJpnkARlGAHx0i5iTknwgSarqaRlumfmy3HaOtLtusY53d6Tl6g2WfSzJF3f87jzdefx5+yRva60dWftha+1IVb07wwiyuWqtva+qXpfku6rq3LGC+s4MDYlfWRN63ySXLzKtO/CDGf5C+I5xOP2lGW5XeOWaSnSW8jpT2WY+WmvvyPBXqFTVF2cYXv2UDA3ii6rqQesaPNvZSdne6JbUK8ef6zvkesrhQXJqkr9urc0yoe4FSe6Y5KGttT+bS6o2tpt6a7NrzJckeUtr7ei6dV5bVcv8hODtrofqt/48OKx5dd8kf7m+XmqtfWSDsj3LudUbe1DPrVWbtktnaGvtRE9bd5ZjO2/bpbe3vLxh/HlGhlupzshwfX1rhi/QZyR50/jz7a21eTw4Yy752lr7m6p6a5InVtV/Gdf/0AyjW390TehhrYtW9bSftisvq52iV6/7/BND/2Let8F275jhlsrDmq+zmKUN29U2nuG7f29787CfB5N2EDvDKsk7MjwxcTMfTT77dMXnZ7gP+IVJ/i7DkOeTM3x52mpkXM8Teo5usvw2k5Tus68Yfy7LF+Hzk/zPDEOGfz/DXy4/nOR/7XB9bYvP9q1Mt9YuqmFC8G/J0GFyZoZ9+9OqOnPsMOkurzPGLtpSHIN5a629P8lLq+o3MvyV7+uTPDjDQyqWQmc5XLR5l5ffzTCK779W1WNaa5/Zg3XO22F7cuR218ODVL/NS28eyCs2sl2dsddtrVXL2tbdzJ6kt7V2XVVdmWEesTtkGFHyH1prK1X1xiQPr6pfztDmfsGuUrwYL81wi/AZGUbNPClD3q19Uv2hrov2uP20WbnbbHmSPGKLzw5svs5ir9uwu/zuv+lqc4jPg6k7iF9a35PkC5O8oWPUw3dlmI/mUWtjq+qR80veUlgdJv+BDJOBH7f2LxpVdVySf5yN/3o2DxdlGFr+PVX1Nxk6FJ637q8sV8+Q1tW/vt0tw/FdjT0xwwSbV81jJzYy/iXwZUleNg4xf26Gv6qdlaFROkt5nSV20ZbmGOyH1lqrqjdnKLsnr/94m1+fpWyvekCG82at09asb336tiuHizZLeXl3kn9SVSe01m7uXP9vJnl9hr8u/lFVPbrjNve9sJNj27POU6vq2LUjWKrqnkk+f7cJXqCDVL/NS1ceVNVhzatrktyvqo5Z1yb7R7lt2Z7l3OqNXaZza7vrxk70tLXm5Zr0H9tFm6VsvSHJ0zLcrnh8hutMxp8/l+RRGb4ovyHzcU3ml6+/lWHusCdV1Z8n+VdJXttau3ZNzDLVRfM4Z3raT9uVl4/mtu3C7dyQYYTSMuTrws3Qhu1pG8/y3b+3vblM5wF77CDOGfbSJPfIJr2zVXX3NW+PZqg8a83nxyU5d54J3CRdt6uqf1JV95njNo6tqp/L8LSbSzLMMfCFGW7xWut7x+V/sB9pHYfrX5DhCR7PGhf/6rqwP5whrau3Cpy5LvaHskWZ3sv9GvP6cxoirbXViVuT4Ut/Mlt5nSV27fK5l60N7OgYzMte5UFVPWKsI9Yvv32Sbx7fXrnu40+PP++Wjc1Stlc9tYYnGK1u/y4Zhnx/Mskb1yzvLYdr92XZy8tvZmgkPnP9SsZG0oauEsqgAAAFs0lEQVRaa7+T4elMD0nyqqq642axe5gHOzm227kow5PDnrRu+X/e7hcXdGx77ah+m5cF5VVvHuxLXvXmwSx5tU3sKzN0fj9+3fKnbxA7y7nVG7tM59Z2142Zdba15mWWY/tZCzoPZylbb8hwjXpWkr9trb13zfITkvxYhgcU/MnaFe3hfu0oX3u01j6a5FUZnpr3xAxTrFy4LmyZ6u09PWdmaD9tV17+agebXy1HM+frkl/nZ7KDNmxP23iW7/697c1lOg/YYwdxZNgvZBhW+rNVdUaGC9INSe6T5OEZHon6TWPs7yX5bxm+GL0iQ0X/hAxP9thvJ2e41/mNue0E2DvxwKr6N+P/75Tk/hkmh/3iDENDnzB+9h1JzquqB2aoXL4qw8ixd2V4Gs1+pDUZJnf9Txku6G9srb1n3ec/M0NaXzcue04Nj51+X4YOwK9Ncv0WadjL/bpTkmur6uIxrR/JMK/AUzPMJfHKMW6W8jpL7Lz2q9dOj8G87FUevCDJF4zH9R0Zbku5d4bz6R8neek4p9hab8lwS/IzququSf4hyftaa28eP5+lbK+6Psmbq+rXx/ffnaEcPGXdiKfecrjWspeXX8jwV/hnVtVXZ6jPbsow98P9c9sOtc9qrf1eVd2a5OVJXl1Vj2qt3bBB6F7lwU6Obc86n5DkJVX1oCRXjGn8umx/bi3i2Pbaaf02L4vIq9482K+86s2DWfJqq9jnZSjbv15VD84wIfFDMjyu/vp87siPWc6t3thlOre2u27s1HZtrXmZ5diutYjzcJaydVmG4/SADB2NSZLW2pVV9eEMo1L+orX29+u2sVf7tdN87XVhhttqn59hIvM/XPf5MtXbe33O9Laftisvr8qQh7O4Msnp2Vm+LvN1flaztmF72sazfPfvbW8u03nAXmtL8EjL9a9s/0jg45L8QIaK8R/G13sy9PB+85q4YzP81eaqDI9Vfn+GSu0B2eTx5+l4dO9WMRkunNdssPyU7MHjpdfkzerraIYL2BUZLmqPXBf/hRmehvfBDBXBB5Ocl+SkLbaxJ2ndYL2vH9f7XZt83p3WDB0Tf5yhs+KTGb78npxhaOyG6d7L/cowXP6/JfnLDBOz3jxu+9eSnLqT8jpr7DyP15p13uYc2eUxWC2/Z89Q1nti9+r8+uaxzL09w0X3yHh8L03yb7PmUcvrfu/JGRo3t4zpuGAnZTv/r245M8lPZHhk880ZOuaesJtyOM/y0nO8ZikvSU5M8owM9dpNY/xbcttHm6/m18PWLf8X4+/9RdY8hnsnedCxXzPVsZuleV3MfTI06G4YX69M8qUb5dV+HNtt8mfT/cm662F2UL/1HINZ4+aVV+mrN3vbL3PNq1nyYMbzZcvYDF90XpHk78eyfdG47Pokl6yLnaVN0FvHLs25lW2uGz3n2CbxW7a1esvLVtvNBm3dWY7tPPJ1lvTOWLb+aqP8HM/FluSn5rxfO8nXTY/rurjjM7QdWpKXbBKzo7pohv3rSuss50zndmdpx29aXrYpdxumMUPHattJvu5l2Vr0q/cYZIa2cWb87p/+9uZczwOvxb1qPMAwd1V1SYa/wH5ROxgTXMO+qqqzk/x6km9qrV222NQAzN84WvT6JC9urX3fotNz0C1TW8uxnQ/5ypRoGzNPB3HOMA6gqrpfhnksXrboxhkAsP/G+RfXW53L5bX7mZbDaJFtLcd2PuQrwPwcxDnDOECq6msyDE39gQzDmp+/2BQBAAtySVW9P8lbM/xB9uFJ/mWS/53bzldEpyVpazm28yFfAeZEZxjz9tQMT266OskTW2vXLDY5AMCC/FGGNsFjk9w+w9w7z0/yE621o4tM2AG3DG0tx3Y+5CvAnJgzDAAAAIDJMGcYAAAAAJOhMwwAAACAydAZBgAAAMBk6AwDAAAAYDJ0hgEAAAAwGTrDAAAAAJiM/x8+U9J5wC/diwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1728x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of input text: 99\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABMMAAAEECAYAAAAlPPOQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XmYNFddL/DvLwkJhE0wCBiWoAYIKhcQQa+AGAIGFQMuXBYFVMwjynVB1Cgqi1cFBBc0Km9ckoiCXEQSvFH2oLjERARDgkAIgYBhCQJBkSTvO+f+UTXYdGapnpme7pn6fN6nn5mu/r11Tp06VXX6zKlT1VoLAAAAAIzBYYvOAAAAAADsFp1hAAAAAIyGzjAAAAAARkNnGAAAAACjoTMMAAAAgNHQGQYAAADAaOgMAwAAAGA0dIYBAAAAMBo6wwAAAAAYjSMWnYF/vtMpbafX+SUP+c/BsZe/4aY7nXyS5GGf+tdBca+95d3nkv4sHnD12wfFveWY/zF4nQ/+93cMjj3/1l8xOHaooduUzLZd83L3s08ZFHfcKc8fvM6/uuXdtpqdDX3FhcPy8I6v/snB67zN7T89OPbbrjw0OPbFR3zB4NhFm0e5LsNxcItbfHZQ3DXX3Hgu6S9DGQx19NHXDY69zwfeOShu0du0DBZdB250xPBz1t3+8UWDY99xnx8bHHvxoZsPivvej71p8DpPvt29Bsf+wpyam3f/hRMGxf3rzw07XpLF15d5+fPDbjY49lEr/zEo7tbHDG9vf/tV1w6OPeNGtx4cu2j7tb6c8NJHD45952NfPsecjNd+rVt7zb0/cM5ai2u387Gsrr/68kF9OTc65kuWssyMDAMAAABgNBY+MgwAAACAPWRl+Oj3ZaQzDAAAAIDhDh1cdA62RWcYAAAAAIO1trLoLGyLzjAAAAAAhlvRGQYAAADAWBgZBgAAAMBomEAfAAAAgNEwMgwAAACAsWieJgkAAADAaJhAHwAAAIDRcJskAAAAAKNhAn0AAAAARsPIMAAAAABGwwT6AAAAAIyGCfQBAAAAGIvWzBkGAAAAwFiYMwwAAACA0XCbJAAAAACjYWQYAAAAAKNx6PpF52BbDlt0BgAAAADYQ1ZWhr0GqKqTq+pdVXVZVZ22Tsyjq+rSqrqkqv5ku9k3MgwAAACA4XboNsmqOjzJ6UkemuSDSS6sqnNba5dOxByf5KeTfF1r7RNV9UXbTdfIMAAAAACG27mRYfdLcllr7fLW2nVJXpbklKmY709yemvtE0nSWvvodrOvMwwAAACA4XauM+zYJFdOvP9gv2zSXZPctar+tqr+oapO3m723SYJAAAAwGBt4AT6VXVqklMnFh1orR2YMbkjkhyf5MFJ7pDkr6vqK1trn5xxPZ+3QgAAAAAYZuCcYX3H10adXx9KcseJ93fol036YJILWmvXJ3lfVb07XefYhYPzO8VtkgAAAAAMt3O3SV6Y5PiquktVHZnkMUnOnYp5VbpRYamqY9LdNnn5drJvZBgAAAAAw+3Q0yRbawer6qlJXpPk8CR/0Fq7pKqek+Si1tq5/WcPq6pLkxxK8hOttY9vJ12dYQAAAAAMN2zU1yCttfOSnDe17Ocnfm9Jnta/doTOMAAAAACG26GRYYuiMwwAAACA4Q4eXHQOtkVnGAAAAADDGRkGAAAAwGjs4Jxhi6AzDAAAAIDhjAwDAAAAYDSMDAMAAABgNIwMAwAAAGA0PE0SAAAAgNFobdE52BadYQAAAAAMZ84wAAAAAEZDZxgAAAAAo2ECfQAAAABG49ChRedgW3SGAQAAADCc2yQBAAAAGA2dYQAAAACMhjnDAAAAABiLttIWnYVt0RkGAAAAwHBukwQAAABgNDxNEgAAAIDRMDIMAAAAgNHQGQYAAADAaDQT6AMAAAAwFkaGAQAAADAaK0aGAQAAADAWniYJAAAAwFg0t0kCAAAAMBpukwQAAABgNJqRYQAAAACMhZFhAAAAAIzGQRPoAwAAADAWbpMEAAAAYDT2+G2Shy06AwAAAADsHW1lZdBriKo6uareVVWXVdVpa3z+A1V1cVW9rareUlX32G7+dYYBAAAAMNxKG/baRFUdnuT0JA9Pco8kj12js+tPWmtf2Vq7V5LnJ/nV7WZfZxgAAAAAw+1QZ1iS+yW5rLV2eWvtuiQvS3LKZEBr7ZqJtzdNsu17NM0ZBgAAAMBwh3bsaZLHJrly4v0Hk9x/OqiqfijJ05IcmeTE7SZqZBgAAAAAg7WVNuhVVadW1UUTr1O3lF5rp7fWvjTJTyX52e3m38gwAAAAAIYb+DTJ1tqBJAc2CPlQkjtOvL9Dv2w9L0vyO4MS34CRYQAAAAAMt7Iy7LW5C5McX1V3qaojkzwmybmTAVV1/MTbb07ynu1m38gwAAAAAIYbODJsM621g1X11CSvSXJ4kj9orV1SVc9JclFr7dwkT62qk5Jcn+QTSZ643XR1hgEAAAAw3A51hiVJa+28JOdNLfv5id9/ZMcS6+kMAwAAAGCwdmjQLZBLS2cYAAAAAMPt4MiwRdAZBgAAAMBgTWcYAAAAAKOhMwwAAACA0djbU4bpDAMAAABguHZwb/eG6QwDAAAAYLi93RemMwwAAACA4UygDwAAAMB4GBkGAAAAwFgYGQYAAADAeBgZBgAAAMBYtIOLzsH26AwDAAAAYLBmZBgAAAAAo6EzDAAAAICxMDIMAAAAgNHQGQYAAADAaLRDtegsbIvOMAAAAAAGMzIMAAAAgNFoK0aGAQAAADASRoYBAAAAMBqtGRkGAAAAwEgYGQYAAADAaKx4miQAAAAAY2ECfQAAAABGQ2cYAAAAAKPR2qJzsD06wwAAAAAYzMgwAAAAAEajNZ1hAAAAAIzEIU+TBAAAAGAsjAwDAAAAYDTMGQYAAADAaHiaJAAAAACjsddHhh226AwAAAAAsHccWjls0GuIqjq5qt5VVZdV1WlrfH5UVf1p//kFVXXcdvO/pc6wqjqiqm5RVUaWAQAAAIxIa8Nem6mqw5OcnuThSe6R5LFVdY+psO9L8onW2pcl+bUkz9tu/gd3hlXVY6rqL6rqI0muTfKJJNdW1Uf65Y/dbmYAAAAAWG4rrQa9Brhfkstaa5e31q5L8rIkp0zFnJLkrP73VyR5SFVt6z7NTUd2VdXRSc5NcmKSzyR5W5Lzk3w2yY2THJvkwUkeXlVPTvKI1tpntpMpAAAAAJZTG9bRlao6NcmpE4sOtNYOTLw/NsmVE+8/mOT+U6v5XExr7WBVfSrJFya5esZsf86Q2xyfk+RBSX44yRmttWunA6rqqHQb98Ikz07yE1vNEAAAAADLa+jTJPuOrwObBu6yIbdJPjrJi1prv7VWR1iStNauba39ZpLfTPKYzVZYVadW1UVVddGf/ccVM2UYAAAAgMXZwdskP5TkjhPv79AvWzOmn7v+lkk+vp38D+kMu02Sdw5c36VJjtksqLV2oLV239bafb/9ZscNXDUAAAAAi7aDT5O8MMnxVXWXqjoy3QCrc6dizk3yxP7370jyxtaGjk1b25CcXZHk5IHr+6Y+HgAAAIB9qA18bbqe1g4meWqS16QbiPXy1tolVfWcqvrWPuz3k3xhVV2W5GlJTttu/ofMGXYgyQur6uVJfj3Jha2161c/rKobpZv9/0eTPDLJ07ebKQAAAACW08BbIAdprZ2X5LypZT8/8ftnk3znjiWYYZ1hv57kuHQ9dd+eZKWqrk5ybZKj0t0WuTrC7PQ+HgAAAIB9aOjTJJfVpp1h/X2YP1JVL07yuCT3TfLFSY5Ock2St6e7x/NPW2vvmGNeAQAAAFiwlUVnYJuGjAxLkrTWLk3ys3PMCwAAAABLrmWfjwwDAAAAgFUH9/ttkgAAAACwysgwAAAAAEZjNHOGAQAAAICRYQAAAACMhpFhAAAAAIzGISPDAAAAABiLlb3dF6YzDAAAAIDhVowMAwAAAGAs2qIzsE06wwAAAAAYzAT6AAAAAIzGSrlNEgAAAICROLToDGyTzjAAAAAABvM0SQAAAABGw9MkAQAAABgNT5MEAAAAYDTcJgkAAADAaKwsOgPbpDMMAAAAgMEOGRkGAAAAwFgYGQYAAADAaOgMAwAAAGA0mtskAQAAABgLI8MAAAAAGI1Di87ANukMAwAAAGCwFbdJAgAAADAWbpMEAAAAYDR0hgEAAAAwGm3RGdgmnWEAAAAADGbOMAAAAABGY68/TfKwRWcAAAAAgL1jJW3Qa7uq6tZV9bqqek//81ZrxNy5qt5aVW+rqkuq6gc2W6/OMAAAAAAGWxn42gGnJXlDa+34JG/o30+7KsnXttbuleT+SU6rqi/eaKU6wwAAAAAYrA187YBTkpzV/35WkkfeIC+tXddau7Z/e1QG9HXpDAMAAABgsF0cGXbb1tpV/e8fTnLbtYKq6o5V9S9JrkzyvNbav220UhPoAwAAADDYwRo27quqTk1y6sSiA621A1Mxr09yuzX++zMm37TWWtXaCbfWrkxyz/72yFdV1Staax9ZL186wwAAAAAYbOgtkH3H14FNYk5a77Oq+khV3b61dlVV3T7JRzdZ179V1TuSPDDJK9aLc5skAAAAAIPt4m2S5yZ5Yv/7E5OcMx1QVXeoqpv0v98qyQOSvGujleoMAwAAAGCwlbRBrx3w3CQPrar3JDmpf5+qum9V/V4fc0KSC6rq7UnenOQFrbWLN1qp2yQBAAAAGGyHnhS5eTqtfTzJQ9ZYflGSJ/e/vy7JPWdZr84wAAAAAAbboVsgF0ZnGAAAAACDHdq1sWHzoTMMAAAAgMGMDAMAAABgNJqRYQAAAACMhZFhAAAAAIzGipFhAAAAAIzF3u4K0xkGAAAAwAwO7vHuMJ1hAAAAAAxmAn0AAAAARsME+gAAAACMhpFhAAAAAIyGkWEAAAAAjMahZmQYAAAAACOx4jZJAAAAAMbCnGEAAAAAjIY5wwAAAAAYDbdJAgAAADAabpMEAAAAYDQ8TRIAAACA0XCbJAAAAACjYQJ9AAAAAEbDnGEAAAAAjIbbJAEAAAAYjWYCfQAAAADG4pCRYQAAAACMhdskAQAAABgNt0kCAAAAMBpGhgEAAAAwGk1nGAAAAABjcchtkgAAAACMhdskAQAAABiNvd4ZdtiiMwAAAADA3tFaG/Tarqq6dVW9rqre0/+81Tpxd6qq11bVO6vq0qo6bqP16gwDAAAAYLCVtEGvHXBakje01o5P8ob+/VrOTvIrrbUTktwvyUc3WqnOMAAAAAAGawP/7YBTkpzV/35WkkdOB1TVPZIc0Vp7XZK01v6jtfaZjVZqzjAAAAAABjvUVnYrqdu21q7qf/9wktuuEXPXJJ+sqlcmuUuS1yc5rbV2aL2V6gwDAAAAYLCh84FV1alJTp1YdKC1dmAq5vVJbrfGf3/GVJqtqtZK+IgkD0xy7yQfSPKnSZ6U5PfXy5fOMAAAAAAGGzofWN/xdWCTmJPW+6yqPlJVt2+tXVVVt8/ac4F9MMnbWmuX9//nVUm+Jht0hpkzDAAAAIDBdnHOsHOTPLH//YlJzlkj5sIkX1BVt+nfn5jk0o1WqjMMAAAAgMFWWhv02gHPTfLQqnpPkpP696mq+1bV7yVJPzfY05O8oaouTlJJzthopW6TBAAAAGCwHRr1tXk6rX08yUPWWH5RkidPvH9dknsOXa/OMAAAAAAG28WnSc6FzjAAAAAABtuhWyAXRmcYAAAAAIPt1m2S86IzDAAAAIDBjAwDAAAAYDSMDAMAAABgNA61Q4vOwrboDAMAAABgsOY2SQAAAADGYsVtkgAAAACMhZFhAAAAAIyGp0kCAAAAMBqeJgkAAADAaBxqK4vOwrboDAMAAABgMHOGAQAAADAa5gwDAAAAYDSMDAMAAABgNFZMoA8AAADAWBgZBgAAAMBoeJokAAAAAKNhAn0AAAAARsNtkgAAAACMRjOBPgAAAABjYWQYAAAAAKOx1+cMS2ttKV9JTt3JuGWIXXT6+zWv+3W7Fp3+fs3rft2uRae/X/O6X7dr0enL696KXXT68rq3YhedvrzurdhFpy+vi49ddPrzjPVa7tfCM7BuxpKLdjJuGWIXnf5+zet+3a5Fp79f87pft2vR6e/XvO7X7Vp0+vK6t2IXnb687q3YRacvr3srdtHpy+viYxed/jxjvZb7dVgAAAAAYCR0hgEAAAAwGsvcGXZgh+OWIXbR6c8Su+j05xW76PRniV10+rPELjr9ecUuOv1ZYhed/iyxi05/XrGLTn+W2EWnP0vsotOfJXbR6c8rdtHpzxK76PRniV10+vOKXXT6s8QuOv1ZYhed/rxiF53+LLGLTn+W2EWnP0vsotOfZyxLrPr7XgEAAABg31vmkWEAAAAAsKOWsjOsqh5cVa2qnrTovMBYVdVdqupVVfWx/ng8cwfWOfjYdh7YWxaxv6rqSX2aD55jGktTD6vqiqo6f9H5mLRM+2AZ9tU8zpszpr/wMtiLlvHY2g3qy/60l/brXsorsP8csegMMJv+C8ebJhatJLkmyYeS/FOSlyZ5TXP/K9t3ZpJ7JvnFJB9O8t6F5maXVNWXJDktyYOS3CnJtem2/x+TnNlae9NU/L2SPLL/7IrdzS2wZM7MCM+bzMZ1A2bjmAHmQWfY3vXSJOclqSQ3T3K3dBeJJyR5fVV9Z2vtkwvMH3tYVR2V5IFJfqu19oJF52e3VNV9k7w5yfVJzk5ySZKbJDk+ycOSfDqf3xmdJPdK8swk5ye5YpeyCiyZsZ432ZJluW78dbpr3PULzAM7by/t16F5XZZjBthHdIYNUFWHJzmqtfaZRedlwltbay+ZXFBVT0vy/CRPS9dZ9vBFZIx94bbpOlr/fdEZ2WXPTHJ0knu11t4+/WFV3W73s8R+t6TXGGY31vMmu2ynzhmttZUkn92ZXLEs9tJ+3Ut5BfafpZwzbDNVdVRV/UxVXVJVn62qT1bVq6vq3lNxN6+q/1NVF1TV1VV1bVVdVlXPraqj11n36vwnJ1XVz1XVe9OdpB+9RsyJVfX0qnpvv+53V9UT57v162utHWqt/XiStyQ5uaoeUFXHVNXpVXVlVV3X/zy9qr5w3vmpqkf15fT963x+Sb8/qn8/KK9V9ax+vcetsc5dm/ejqm7c5+VdVfWZvh5eXFW/MhU3qL7OGjvH7Tozyfv7t8/sy/rz5gRaln0wB8cn+fhaHWFJ0lr78OT7qnpWkj/s375poqzOnIobWrcnzz/Pqqr39+eWf6mqx6yVp6H1cJFmrS9VdWRV/WRVva3fpk9V1UVV9dSB6T2jT+83q2qu17lZz7EDrzF3rKqX99t9TX8O+NJ5bscOOGyz6+EynN/mZch5s48b2n7Zl2VVVcdV1Z/19fqaqjqnunnW1joPDD62ZjjHLvzYGnrdmIjf8JxRM7a1pj5bd76mmrGtO8u+nYdZ8rtZfamqO/frevbU/3tNv/zHppZfUFXvnNN2zVyum+zXh/ef/fA6//fvq5vz8Eb9+7meizbK60TMszLDMTND2kPb8evWl4l695Cq+vnq2m3/1deJr+n//9dX1Vuq6j+r6qr+ON6X5/hZDdkHNUPbuGb87l8D25v21/6150aG9Sfnv0ryP5P8UZLfSnLLJN+f5G+r6kGttYv68GOTPDnJnyX5kyQHk3x9kp9Mcu8k37hBUi9IcqMkZ6Sbk+tda8T8UrqhvS9ON6/QU5KcWVWXtdb+dhubuV2/n+QBSb4tybck+bIkf5Dkrem2+ylJTqyq+7XWPj3HfLw63Zwp35uuHD+nv0DcI8kzWmutqm6Z5O8WmNetOD3dtp2d5FfTHU/HJzlxNWCW+jpj3Z6nFyd5W5JfS/LnSV7ZL59LQ2/JvDfJ3arq21prr9w0uiub2yc5Nd35YLWMPjdP0Bbr9vOS3DTJb/fvvyfJS6vqxq21M6diN62He0lVHZnkNUkenOS1SV6S7ovfV6Y7p/3WBv/38P7zH0jy06215845r9s5b615jamqL0h328gdk/xukkvTXbfelO56s6w2vB4u0fltXjY9bw4tg/1aVtV1NPxNuhF0v5uubB6Yrm7fdCp28LE1NHaJjq1NrxvrWK9dOrittcX8btrWnWXf7oLNzkVD6sv7q+rydNfRZyafuzY9IN1cvSemO9ZTVbdI8lV9ejtqTuX62nT15QlJXjSV3vFJvibJi1pr1y/RuWirx8xmhrTjN6wvSX6jD31uksP790cm+fEkr62qJ6T7XnYgyR+n68R+TpLHJvnS7KNz/BbN0oYd0jYe/N1/aHtziY4D5qG1tnSvdJWyJXnSGp/9WP/ZN04tv0WSDyQ5f2LZkUlutMY6fqFfx/3W+OxJ/WfvSnL0OvlbjfnnJEdOLD823YX3pbtQNk/fIOY+fcy/9j9/cOrzH+qX/8Iu7Mtf6tO6x9TyM9KdoL64f/+LQ/Oa5Fn9suPWSO+KyTow52379yTnbRIzS30dHLsL23Zcn5dnrfP5lvbBRsf2dmJ3cLu/Nsl1fbrvTtfweUqSEzb4P6vngwev8/ksdXt1Xe9PcsuJ5bfsl/17kpvMWg93qezW3V+z1Jd0DZaW5JfWiD1svbJP9+Xnz/v994Rd2q7B+3aNPK95jcl/nzO/Z2r5r/fLz9/qtsxpv69uz4bXw2zj/Db0XLCIc8ZU+sdl4/PmoDLYjbJaUPk8v8/b49dZfv7EslnOm4Nil+nYyibXjXViN2qXDmprzVJfhh7bs+7bXSjTzc5FQ+vLGemuJ0f37x/Uf/5H6Tojj+iXP6Jf/u2LPGaG7tf+819Zp76sfj+6T/9+7u3SzfK6lWNmhrSHtOM3qy/n9D/fOlXvvrVffn2S+04sP7KvP3Mt173yGrgPVvf9pm3jzPDdPwPbm7txHHgt7rUXb5P8rnSdPP/UD1s9pqqOSVf5X5fkAVV1kyRprV3XWrs+SarqiKq6VR/7+n5d998gnd9pm8/F8NuttetW37TWPpTuC/TxW9qynXNN//MOST6W7q8Rk17cL3/ULuTljHQnkO9bXVBVN03yv5L8ZWvt3/rFj8ri8zqrTyX58qr6ig1iBtfXGWOZg9ba36f7C+9Z6S6y35PuL1CXVtVfV/ekyVltpW7/TmvtUxP5+lS6vwrfKl3DcdKQeriXPD7JJ9L95fTztG5ukbXcOt0xclKSR7TWzp5f9j7Pds5b611jHpnkI+n+SjrpeVvN5C7Z7Hro/Da8DPZrWT0iyVXp5jSdtNbDBmY5tobG7tVja9VG7dKhba2tGNLWnWXfzttm+R1aX96YbiTeA/v3Jyb5aLqRPzdP8tX98m9IV/Zv2tGt6MyrXM/qfz5hdUFVVbpzzztaa2/tF+/Xc9GqIe2nzerLV/Xvf2ey3qUb0ZckF7SJUUN9zEqSQ9m/5TqLWdqwm7aNZ/zuP7S9ud+Pg1Hbc7dJJjkh3QiAj20Qc0ySK5Okqn4w3S0zX54bzpF2qw3W8e4Bebl8jWUfT3LnAf93nm7R/7xJkn9urR2c/LC1drCq3p1uBNlctdbeV1WvT/LdVXVaf4J6dLqGxO9NhN4lyUWLzOsW/Gi6vxBe3A+nf1O62xVePXESnaW+zlS3mY/W2sXp/gqVqrpzuuHVT07XID6nqr5qqsGzma3U7bVuSb20/zndITekHu4lxyd5W2ttlgl1z0xysyQPaq29ZS65Wtt2zlvrXWO+JMmFrbVDU+u8qqqW+QnBm10Pnd+Gl8F+Lau7JPnH6fNSa+2ja9TtWY6tobF79dhatW67dIa21lYMaevOsm/nbbP8Dq0vb+x/npjuVqoT011f35ruC/SJSf6+//n21to8Hpwxl3Jtrb2jqt6a5PFV9TP9+h+UbnTrT06E7tdz0aoh7afN6stqp+jlU59/outfzPvWSPdm6W6p3K/lOotZ2rCD2sYzfPcf2t7c78fBqO3FzrBKcnG6Jyau52PJ556u+MJ09wG/KMm/pRvyfGy6L08bjYwb8oSeQ+ssv8Ekpbvsnv3PZfkifCDJ/003ZPjP0v3l8sNJ/t8W19c2+GzX6nRr7ZzqJgT/pnQdJiel27a/qaqT+g6TwfV1xthFW4p9MG+ttfcnObuq/ijdX/m+Lsn90j2kYikMrIeLNu/68qfpRvH9XFU9srX2Xzuwznnbb0+O3Ox6uJfOb/MytAyUFWvZ7Jyx022tVcva1l3PjuS3tfaRqro03TxiR6cbUfK/W2srVfXmJA+pqt9N1+b+tW3leDHOTneL8InpRs08IV3ZTT6pfl+fi3a4/bRevVtveZI8dIPP9my5zmKn27Db/O6/7mqzj4+DsduLX1rfk+Q2Sd44YNTDd6ebj+bhk7FVdfL8srcUVofJX5luMvAjJv+iUVVHJLlr1v7r2Tyck25o+fdV1TvSdSg8b+qvLJfPkNfVv77dOt3+XY29cboJNi+bx0aspf9L4EuSvKQfYv7cdH9VOyVdo3SW+jpL7KItzT7YDa21VlUXpKu7x05/vMl/n6Vurzoh3XEz6R4T65vO32b1cNFmqS/vTnL3qjqqtXbtwPX/cZI3pPvr4l9U1SMG3Oa+E7ayb4es8/iqOnxyBEtV3T7JF2w3wwu0l85v8zKoDKpqv5bVFUm+rKoOm2qTfVFuWLdnObaGxi7TsbXZdWMrhrS15uWKDN+3izZL3Xpjkh9Md7vikemuM+l/viDJw9N9UX5j5uOKzK9c/yTd3GFPqKq/TfIdSV7XWrtqImaZzkXzOGaGtJ82qy8fyw3bhZu5Jt0IpWUo14WboQ07pG08y3f/oe3NZToO2GF7cc6ws5PcLuv0zlbVbSfeHkp38qyJz49Icto8M7hOvm5UVXevqjvNMY3Dq+oF6Z52c166OQZuk+4Wr0nf3y//893Iaz9c/8x0T/B4Zr/496fCXjVDXldvFThpKvbHskGd3snt6sv68xoirbXViVuT7kt/Mlt9nSV2cvnc69YatrQP5mWnyqCqHtqfI6aX3yTJw/q3l059/B/9z1tnbbPU7VVPqe4JRqvp3zLdkO9PJnnzxPKh9XByW5a9vvxxukbiz06vpG8kram19rJ0T2d6YJK/rKqbrRe7g2WwlX27mXPSPTnsCVPLf2qz/7igfTvUls5v87KgshpaBrtBUlskAAAFf0lEQVRSVkPLYJay2iT21ek6vx87tfzpa8TOcmwNjV2mY2uz68bMBra15mWWffs5CzoOZ6lbb0x3jXpmkg+01t47sfyoJD+d7gEFfz25oh3cri2V6xCttY8l+ct0T817fLopVs6aClum8/aOHjMztJ82qy//tIXkV+vRzOW65Nf5mWyhDTukbTzLd/+h7c1lOg7YYXtxZNhvpBtW+itVdWK6C9I1Se6U5CHpHon6DX3sK5L8crovRq9Md6J/XLone+y2Y9Pd6/zm3HAC7K24T1V9V//7zZPcLd3ksHdONzT0cf1n35nk9Kq6T7qTy73TjRx7V7qn0exGXpNuctefSHdBf3Nr7T1Tnz9/hry+vl/2nOoeO/2+dB2AX5Pk6g3ysJPbdfMkV1XVuX1eP5puXoGnpJtL4tV93Cz1dZbYeW3XUFvdB/OyU2Xwa0m+sN+vF6e7LeWO6Y6nuyY5u59TbNKF6W5JfkZV3SrJfyZ5X2vtgv7zWer2qquTXFBVf9i//5509eDJUyOehtbDScteX34j3V/hf7aqvjrd+eyz6eZ+uFtu2KH2Oa21V1TV9UlenuQ1VfXw1to1a4TuVBlsZd8OWefjkpxRVV+V5JI+j1+bzY+tRezbobZ6fpuXRZTV0DLYrbIaWgazlNVGsc9LV7f/sKrul25C4geme1z91fn8kR+zHFtDY5fp2NrsurFVm7W15mWWfTtpEcfhLHXr/HT76YR0HY1JktbapVX14XSjUv6htfbpqTR2aru2Wq5DnZXuttoXppvI/FVTny/TeXunj5mh7afN6stfpivDWVya5L7ZWrku83V+VrO2YYe0jWf57j+0vblMxwE7rS3BIy2nX9n8kcBHJPnhdCfG/+xf70nXw/uwibjD0/3V5rJ0j1V+f7qT2glZ5/HnGfDo3o1i0l04r1hj+XHZgcdLT5TN6utQugvYJekuaidPxd8m3dPwPpjuRPDBJKcnOWaDNHYkr2us9w39er97nc8H5zVdx8Rfpeus+GS6L7/Hphsau2a+d3K70g2X/+Uk/5huYtZr+7T/IMnxW6mvs8bOc39NrPMGx8g298Fq/X3SDHV9SOxOHV8P6+vc29NddA/2+/dNSb43E49anvp/T0zXuLmuz8eZW6nb+e9zy0lJnp3ukc3XpuuYe9x26uE868uQ/TVLfUly4yTPSHde+2wff2Fu+Gjz1fJ68NTyb+7/3z9k4jHcWymDAds10zl2vTxPxdwpXYPumv716iRfulZZ7ca+3aR81t2eTF0Ps4Xz25B9MGvcvMoqw86bQ9svcy2rWcpgxuNlw9h0X3RemeTTfd0+p192dZLzpmJnaRMMPccuzbGVTa4bQ46xdeI3bGsNrS8bpZs12rqz7Nt5lOss+Z2xbv3TWuXZH4styS/Oebu2Uq7r7tepuCPTtR1akjPWidnSuWiG7RuU11mOmYHpztKOX7e+bFLv1sxjuo7VtpVy3cm6tejX0H2QGdrGmfG7f4a3N+d6HHgt7lX9Doa5q6rz0v0F9ovb3pjgGnZVVT0pyR8m+YbW2vmLzQ3A/PWjRa9O8uLW2g8sOj973TK1tezb+VCujIm2MfO0F+cMYw+qqi9LN4/FSxbdOAMAdl8//+K01blcXrebedmPFtnWsm/nQ7kCzM9enDOMPaSq7p9uaOoPpxvW/MLF5ggAWJDzqur9Sd6a7g+yD0nyLUn+Ljecr4iBlqStZd/Oh3IFmBOdYczbU9I9uenyJI9vrV2x2OwAAAvyF+naBI9KcpN0c++8MMmzW2uHFpmxPW4Z2lr27XwoV4A5MWcYAAAAAKNhzjAAAAAARkNnGAAAAACjoTMMAAAAgNHQGQYAAADAaOgMAwAAAGA0dIYBAAAAMBr/H+c46d23Mt5ZAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1728x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "seed_txt = 'learn. Do. you. see. full. Stops. luck.lord.  hell. feel...good. trying. now. give it some.    space'\n",
    "predict(seed_txt,prediction_model,133,134,1)\n",
    "predict(seed_txt,prediction_model,131,132,2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2. 'a' 'Ar':\n",
    "\n",
    "Here, one can observe that this particular node/cell of LSTM Layer 2 gives negative activation followed by a positive activation whenever sub-string `ar` is encountered after a space (There is no activation when `ar` is in between a word like in 'learn'). Also interestingly, there is strong negative activation when the string `an` is encountered. Capital letter makes the activations a little less. One can try for other input strings of more variety of words containing `an, a, artist... ` and soon and observe different patterns. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of input text: 102\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABMMAAAEECAYAAAAlPPOQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xm0bFddJ/DvLyPEJWMwhIAGNSBRWDgFnEMIGHRpUFsElA4aeEuU1tblEBtbEIcG5ykqj8GAtCMoBDvKEAZtBzpRUUw0AwEkEsIMKmZ47+7+o86Ty829t3bdW/Xq1j2fz1tnvapTu/bZ++wz3V/ts0+11gIAAAAAY3DMsgsAAAAAAEeLYBgAAAAAoyEYBgAAAMBoCIYBAAAAMBqCYQAAAACMhmAYAAAAAKMhGAYAAADAaAiGAQAAADAagmEAAAAAjMZxyy7A7e+/ofWke/ejD8x92X/xnlO6097z8OGudA887f1d6R5+w7u7l/3+j32kK91n3u207jz/5L536U7b64FXX9uV7l9vfGN3noto91Vw3o0f7U775vPv1pXuQ3+9ttPi7Em921vSv82NdXtblIde+46udDf/0x9253nT137PDktzdH3be/t+a3rRpyx3v5z3cds+tByLOB6+72sv7M7z9luO7Ur3DTfd2p3ny089sTvtvO2388vrb753d9qHn/TBrnQnffJtOy3OXPS20aG1vuv3JHnb5zxop8XZ1Flvu7E77Y3XvrIr3bsf8507Lc6WZtkvfzCf2pXurFNv7s7zXhd+Vle6973wn7rzZDnu85qDXekWcTx88DU3dKX7wDtfN1O+x5/86ZvNrpky2cd6YznHn/zpe3Kd6RkGAAAAwGgsvWcYAAAAACtkht63e5FgGAAAAAD9Dh9adgl2RTAMAAAAgG6trfaY1IJhAAAAAPRbEwwDAAAAYCz0DAMAAABgNAygDwAAAMBo6BkGAAAAwFg0T5MEAAAAYDQMoA8AAADAaLhNEgAAAIDRMIA+AAAAAKOhZxgAAAAAo2EAfQAAAABGwwD6AAAAAIxFa6s9Ztgxyy4AAAAAACukrfVNHarqvKq6pqqur6qLtkjzuKq6uqquqqrf2m3x9QwDAAAAoN+cbpOsqmOTXJzkUUluTHJFVV3aWrt6XZozkvxQki9prX2oqj5lt8vVMwwAAACAfvPrGXZWkutbaze01m5L8jtJzt+Q5qlJLm6tfShJWmvv3W3xBcMAAAAA6Hf49q6pqg5U1ZXrpgMbcjotybvWvb9xmLfeA5I8oKr+vKr+qqrO223x3SYJAAAAQL/O2yRbaweTHNzl0o5LckaSs5PcN8mfVtWDW2sf3mmGeoYBAAAA0G9+t0n+S5L7rXt/32HeejcmubS1dntr7e1Jrs0kOLZjgmEAAAAA9Ftb65umuyLJGVV1/6o6Icnjk1y6Ic0rMukVlqo6OZPbJm/YTfHdJgkAAABAvzk9TbK1dqiqnp7k1UmOTfKi1tpVVfXsJFe21i4dPnt0VV2d5HCS72+tfWA3yxUMAwAAAKBbO3z7/PJq7bIkl22Y9yPrXrck3ztMcyEYBgAAAEC/vvHA9izBMAAAAAD6zek2yWURDAMAAACgn55hAAAAAIyGnmEAAAAAjIaeYQAAAACMxqFDyy7BrgiGAQAAANBPzzAAAAAARsOYYQAAAACMhp5hAAAAAIyGnmEAAAAAjIaeYQAAAACMhqdJAgAAADAarS27BLsiGAYAAABAP2OGAQAAADAagmEAAAAAjIYB9AEAAAAYjcOHl12CXREMAwAAAKCf2yQBAAAAGA3BMAAAAABGw5hhAAAAAIxFW2vLLsKuCIYBAAAA0M9tkgAAAACMhqdJAgAAADAaeoYBAAAAMBqCYQAAAACMRjOAPgAAAABjoWcYAAAAAKOxtto9w45ZdgEAAAAAWCGHD/dNHarqvKq6pqqur6qLNvn826vqrVX1lqr6v1V15m6LLxgGAAAAQLe2ttY1TVNVxya5OMljkpyZ5AmbBLt+q7X24NbaQ5P8VJKf2235BcMAAAAA6LfW+qbpzkpyfWvthtbabUl+J8n56xO01j667u0nJdn1PZrGDAMAAACgX+sbQL+qDiQ5sG7WwdbawXXvT0vyrnXvb0zysE3y+c4k35vkhCTnzFrcjQTDAAAAAOjXOYD+EPg6ODXh9HwuTnJxVT0xyQ8nuWA3+QmGAQAAANDvUN/g+B3+Jcn91r2/7zBvK7+T5Nd2u1BjhgEAAADQr631TdNdkeSMqrp/VZ2Q5PFJLl2foKrOWPf2q5Nct9vi6xkGAAAAQL/O2ySnaa0dqqqnJ3l1kmOTvKi1dlVVPTvJla21S5M8varOTXJ7kg9ll7dIJoJhAAAAAMygrfUNoN+VV2uXJblsw7wfWff6u+e2sIFgGAAAAAD95tQzbFkEwwAAAADoJxgGAAAAwGgcntvTJJdCMAwAAACAbk3PMAAAAABGQzAMAAAAgNGY49Mkl0EwDAAAAIB+eoYBAAAAMBqCYQAAAACMRTvsNkkAAAAAxkLPMAAAAADGogmGAQAAADAagmEAAAAAjMZqDxkmGAYAAABAv3ZotaNhgmEAAAAA9FvtWJhgGAAAAAD9DKAPAAAAwHjoGQYAAADAWOgZBgAAAMB46BkGAAAAwFi0Q8suwe4IhgEAAADQrekZBgAAAMBoCIYBAAAAMBZ6hgEAAAAwGoJhAAAAAIxGO1zLLsKuCIYBAAAA0E3PMAAAAABGo63pGQYAAADASKx6z7Bjll0AAAAAAFZHa9U19aiq86rqmqq6vqou2uTzE6vqd4fP31xVp++2/IJhAAAAAHRra33TNFV1bJKLkzwmyZlJnlBVZ25IdmGSD7XWPjPJzyd57m7LLxgGAAAAQLe1w9U1dTgryfWttRtaa7cl+Z0k529Ic36SFw+vX5bkkVW1q0HLBMMAAAAA6NbWqmvqcFqSd617f+Mwb9M0rbVDST6S5J67Kb9gGAAAAADdeoNhVXWgqq5cNx1YdtkTT5MEAAAAYAat9aZrB5Mc3CbJvyS537r39x3mbZbmxqo6Lsldk3ygt6yb0TMMAAAAgG5zvE3yiiRnVNX9q+qEJI9PcumGNJcmuWB4/V+SvL613nDc5vQMAwAAAKBba7sav35dPu1QVT09yauTHJvkRa21q6rq2UmubK1dmuSFSX6zqq5P8sFMAma7IhgGAAAAQLfDfU+K7NJauyzJZRvm/ci617ck+ca5LTCCYQAAAADMYF49w5ZFMAwAAACAbp3jge1ZgmEAAAAAdNvd8PXLJxgGAAAAQDc9wwAAAAAYjcNrxyy7CLuyo2BYVR2X5KQkH2utHZpvkQAAAADYq1b9NsnuUF5VPb6q/qiqbk5ya5IPJbm1qm4e5j9hYaUEAAAAYE9Ya9U17VVTe4ZV1UlJLk1yTpKPJXlLkjcmuSXJnZKcluTsJI+pqqck+ZrW2scWVF4AAAAAlqjt4UBXj57bJJ+d5MuTfFeS57fWbt2YoKpOTHIgyc8m+dEk3z/PQgIAAACwN4zhNsnHJfml1tqvbBYIS5LW2q2ttV9O8stJHj8tw6o6UFVXVtWVL3jJb89WYgAAAACWZt/fJpnkXkn+sTO/q5OcPC1Ra+1gkoNJcvv7b1jxeCIAAADAeKz60yR7Sv+OJOd15vdVQ3oAAAAA9qHWOe1VPcGwg0m+oap+r6q+uKqOX/9hVR1fVV9SVb+f5LFDegAAAAD2oTHcJvkLSU5P8vQk35Bkraren+TWJCdmclvkkaDaxUN6AAAAAPahff80ydZaS/LdVfW8JE9M8gVJ7pPkpCQfTfJ3Sa5I8ruttX9YYFkBAAAAWLK1ZRdgl3p6hiVJWmtXJ/nhBZYFAAAAgD2uZZ/3DAMAAACAIw7t99skAQAAAOAIPcMAAAAAGI3RjBkGAAAAAHqGAQAAADAaeoYBAAAAMBqH9QwDAAAAYCzWVjsWJhgGAAAAQL81PcMAAAAAGIu27ALskmAYAAAAAN0MoA8AAADAaKyV2yQBAAAAGInDyy7ALh2z7AIAAAAAsDrWqm/araq6R1W9tqquG/6/+yZpPq2q/qaq3lJVV1XVt0/LVzAMAAAAgG5rqa5pDi5Kcnlr7Ywklw/vN7opyRe11h6a5GFJLqqq+2yXqWAYAAAAAN1a5zQH5yd58fD6xUkee4eytHZba+3W4e2J6Yh1CYYBAAAA0O1o3SaZ5JTW2k3D6/ckOWWzRFV1v6r6+yTvSvLc1tq7t8vUAPoAAAAAdFvrTFdVB5IcWDfrYGvt4IY0r0ty702+/oz1b1prrao27XDWWntXkocMt0e+oqpe1lq7eatyCYYBAAAA0O1wZ6+vIfB1cEqac7f6rKpurqpTW2s3VdWpSd47Ja93V9U/JPmyJC/bKp3bJAEAAADottY5zcGlSS4YXl+Q5JUbE1TVfavqzsPruyf50iTXbJepYBgAAAAA3Y5iMOw5SR5VVdclOXd4n6r6gqp6wZDmQUneXFV/l+RNSX6mtfbW7TJ1myQAAAAA3dp8BsefvpzWPpDkkZvMvzLJU4bXr03ykFnyFQwDAAAAoNucen0tjWAYAAAAAN0OL7sAuyQYBgAAAEC3taN0m+SiCIYBAAAA0M1tkgAAAACMhmAYAAAAAKPRll2AXRIMAwAAAKCbMcMAAAAAGA1PkwQAAABgNNZW/EZJwTAAAAAAuhlAHwAAAIDRWO1+YYJhAAAAAMxAzzAAAAAARuNQrXbfMMEwAAAAALqtdihMMAwAAACAGbhNEgAAAIDRWFvxvmGCYQAAAAB0W+1QmGAYAAAAADNwmyQAAAAAo3F4xfuGCYYBAAAA0E3PMAAAAABGo+kZBgAAAMBY6BkGAAAAwGis6RkGAAAAwFisdihMMAwAAACAGRxa8XCYYBgAAAAA3QygDwAAAMBoGEAfAAAAgNFY9Z5hxyy7AAAAAACsjrXOabeq6h5V9dqqum74/+5bpPvUqnpNVf1jVV1dVadvl69gGAAAAADdDrfWNc3BRUkub62dkeTy4f1mXpLkp1trD0pyVpL3bpepYBgAAAAA3dbSuqY5OD/Ji4fXL07y2I0JqurMJMe11l6bJK21f2utfWy7TAXDAAAAAOjWOv/NwSmttZuG1+9JcsomaR6Q5MNV9QdV9bdV9dNVdex2mRpAHwAAAIBuveOBVdWBJAfWzTrYWju4Ic3rktx7k68/Y/2b1lqrqs0ibMcl+bIkn5vkn5P8bpInJ3nhVuUSDAMAAACgW+8tkEPg6+CUNOdu9VlV3VxVp7bWbqqqU7P5WGA3JnlLa+2G4TuvSPLwbBMMc5skAAAAAN2O4m2Slya5YHh9QZJXbpLmiiR3q6p7De/PSXL1dpkKhgEAAADQ7Sg+TfI5SR5VVdclOXd4n6r6gqp6QZK01g4n+b4kl1fVW5NUkudvl6nbJAEAAADoNqcnRU7VWvtAkkduMv/KJE9Z9/61SR7Sm69gGAAAAADdegfQ36sEwwAAAADoNqfxwJZGMAwAAACAbkfrNslFEQwDAAAAoFubz+D4SyMYBgAAAEC3w3qGAQAAADAWbpMEAAAAYDTcJgkAAADAaOgZBgAAAMBoNMEwAAAAAMbisNskAQAAABgLt0kCAAAAMBqCYQAAAACMhqdJAgAAADAaeoYBAAAAMBqeJgkAAADAaBxua8suwq4IhgEAAADQzZhhAAAAAIyGMcMAAAAAGA1jhgEAAAAwGmtukwQAAABgLPQMAwAAAGA0PE0SAAAAgNFwmyQAAAAAo+E2SQAAAABGQ88wAAAAAEZDzzAAAAAARuNwO7zsIuyKYBgAAAAA3dqK3yZ5zLILAAAAAMDqWEvrmnarqu5RVa+tquuG/+++RbrnVtU/DNM3TctXMAwAAACAbq21rmkOLkpyeWvtjCSXD+8/QVV9dZLPS/LQJA9L8n1VdZftMhUMAwAAAKDbWmtd0xycn+TFw+sXJ3nsJmnOTPKnrbVDrbV/T/L3Sc7bLlPBMAAAAAC6tc5/c3BKa+2m4fV7kpyySZq/S3JeVZ1UVScneUSS+22XqQH0AQAAAOh2uK11pauqA0kOrJt1sLV2cEOa1yW59yZff8b6N621VlV3iLC11l5TVV+Y5C+SvC/JXybZ9nGXgmEAAAAAdOsdD2wIfB2ckubcrT6rqpur6tTW2k1VdWqS926Rx08k+YnhO7+V5Nrtluk2SQAAAAC6HcUxwy5NcsHw+oIkr9yYoKqOrap7Dq8fkuQhSV6zXaZ6hgEAAADQbU5PiuzxnCS/V1UXJnlnksclSVV9QZJvb609JcnxSf6sqpLko0m+pbV2aLtMBcMAAAAA6LY2n8Hxp2qtfSDJIzeZf2WSpwyvb8nkiZLdBMMAAAAA6HYUe4YthGAYAAAAAN16nya5VwmGAQAAANBtToPjL41gGAAAAADd3CYJAAAAwGi0ozSA/qIIhgEAAADQTc8wAAAAAEZj1ccMS2ttz01JDsw7rTz39rLlqS3lufeWLU9tuR/y3G/1GXOe+60+Y85zv9VnzHnut/qsSp77rT77MU/T3p+WXoBNC5VcOe+08tzby5antpTn3lu2PLXlfshzv9VnzHnut/qMOc/9Vp8x57nf6rMqee63+uzHPE17fzomAAAAADASgmEAAAAAjMZeDYYdXEBaee7tZctz/yxbnvtn2fLcP8sec577rT5jznO/1WfMee63+ow5z/1Wn1XJc7/VZz/myR5Xw32vAAAAALDv7dWeYQAAAAAwd3syGFZVZ1dVq6onL7ss+0lVvaOq3rjscgDsdXvlPOS4vXtVdf+qekVVvW9o00uO4rL3xHa0H9k3ONrGtM0t+9jVu/wZ0i3tPADsXXsyGAZ8oqq6e1X9x3ACf9Kyy8PiVdVDq+pZVXX6sssCK+6SJF+R5LlJnpTkeUstDTBXY75Gcq3Q7ZI4D7BL9rf957hlFwDo8s1JTkzy9iTfluQ3l1scjoKHJnlmkjcmecdSSwIrqqpOTPJlSX6ltfYzyy4PsBBjvkZa1LXCnya5c5Lb55jnIkwtp/MAc+TafJ/RM2yKqjq2qk5adjnGxnq/gwuTvCHJLyT5iqr69N4vWpd9rKe9bb+1z36rT7Jn63RKkkrywWUXZJXs0bZcSdblUbHja6RVcbS3o9baWmvtltba4aO1zJ3oLKfzADNx3B6PlQuGVdWJVfU/quqqqrqlqj5cVa+qqs/dkO6Tq+rHq+rNVfX+qrq1qq6vqudstXFX1ZOHLtbnVtX/rKq3JbklyeM2fH5OVX1fVb1tyPfaqrpgh/XpLmfv8qvqflX1e1X1kar66LB+PmMn5VtkfTap16brfUjT1e6dZfy6YXlP3eLzq4Yy1/D+5Kq6uKreVVW3Df9fXFX33PC9Zw35nr5JnjseZ6KqPi+TXyJenOS3khzK5JfPzdJOXZdDujsN5b2mqj42rM+3VtVPb0jXW/e57xu9VmGbW5dn73p/VpLfGN6+YSjrHca42EH7nDss/53Devr7qnr8TsvZWeeVaZ8Z6tS13mesj+P2/Ot1SZJ3Dm+fuW4/OnsnZVx2fXossC2nbvM1wzmwZjhnLGvf2I/7xYzrfcs2r6pPG/L50Q3fefUw/3s2zH9zVf3jgurUdY20zG1u1m2pZzsa0j0rHdcKOyzz1LG4ao7XCosqZ3WcB4Z0e/74Pque9qnO68MdbMMnVNUPVNVbhmV/pKqurKqnb0g31/U+azk3WQdL2d9YnpW6TbKqjk/yJ0m+OJMu0L+S5K5Jnprkz6vqy1trVw7JT0vylCQvz8dPjl+R5AeSfG6Sr9xmUT+T5Pgkz0/y0STXbPj8JzPpkvu8JLcmeVqSS6rq+tban89YrZ2Uc8vlJ7kqky7D90vy60muHvJ7w/CdRZv7ep+x3Xu8Ksl7MrlYev76D6rq4UnOTPKM1lqrqrsm+Yskn5nkRUn+ZqjH05KcU1Vntdb+dYZl78SFSf4tyctba/9eVX+U5IKq+pHW2toW35m2DV+cSf1fkuTnMjkWnJHknCMJdlj3ee4bvVZhmzti6nof/EGSU5McyGSdHvkj4m1HEuywfZ6b5JOS/Orw/luT/HZV3am1dskOytljldpnql0eE7aqz93iuL0Iz0vyliQ/n+QPM9mvkmF/mqWMe6Q+PRbRlos8D257zljyvrFf94tk+nrfts2TnJXkhuH1M4e6nZDkS5OsDfN/fph/lySfn8WN0TTrNdIytrmdbkvTruWmXiss2DyvFRZl2/NAsqf2y3mbpX2mXR92b8PDseDVSc5O8pokL80kuPTgJF+fyfpd1Hpf5N//y97fWITW2p6bMtl5WpInb5j/PcP8r9ww/y5J/jnJG9fNOyHJ8Zvk/WNDHmdt8tmTh8+uSXLSNp//bZIT1s0/LZMT6m/voK7d5exZfiY7Z0vyrRvy+4Vh/htnLeOi6jPDeu9u9xnKeWQ9nblh/vMzOXDeZ3j/E0O679iQ7juH+T+2bt6zhnmnb7K8d+ywnHdK8qEkl6ybd/6wnMfMui7XpftgksumLHuWus9939hv21zvet+kjGfPsX3emeSu6+bfdZj3wSR33kk591P7bMjr7Gx+Hupe7zPUx3F7cXU7fSjHs3ZTxp3WZ6vtaMXasmubzwznwHSeM5a5b+zH/WKG9T61zTO5XrrtSD2TfPnw2W9m8ofkccP8rxnmf8MC6tN9jbTMbW7WbWnadrRF2rPnvG7PzpRjV+Z4rbCT5c9QztOzxXlg+HxPnK8WsA57rvWPbD/bXh/Osg1nEnhqSX5yk/THLHK9z7qvbVgHS9vfTMubVu02yW9J8k9J/rom3bdPrqqTM9nwX5vkS6vqzknSWruttXZ7klTVcTV50szJSV435PWwbZbza621j23z+a+21m478qa19i9Jrs0k2j6THZZzu+U/NsnNmfwKsN5zZy3bTixovXe3+wyen8nB7MIjM6rqk5J8U5I/bq29e5j9dUnel+Tghu8/b5j/dTMud1Zfn+RumXT/P+KyYdmb3io5mLYNfyTJZ1fV52yTZid1n9u+0WuFtrmkb7332kn7/Fpr7SNH3gyvfz3J3TO5oJx7OVesfXrs5piwVX0ct5djljKuQn0W1ZaLPA9OO2csbd/Y5/vFtPXe0+avz6QXxZcNn52T5L1JfjHJJyf5wmH+IzK53nrD3Guxs2uko77N7WJbmnYtt2zzvKZZpr2yX87bLO2z7fXhjNvwN2cSpH72xoW0T+ytOff1vuC//9mHVuo2ySQPyiQ6/b5t0pyc5F1JUlXfkeTbk3x27jg+2t23yePaKeW4YZN5H0jyaVO+t6kdlHO75Z+a5Iq2YSDJ1tpNVfXhnZRvVgtY7zO1e4/W2tur6nVJnlRVFw0HzsdlcgH3gnVJ75/kytbaoQ3fP1RV1yb5vN5l7tCFmdT7xqr6zHXzX5PkG6vq5Nba+zf53rRt+L9n8uvtW6vqhkwuUl+V5FXrTlQ7qftc941eq7DNDXrWe6+dtM9mY7ZcPfy/fsDheZZzldqnx26OCVvV59PjuL0Ms5RxFeqTZCFtucjz4LRzxlL3jX28X0xb7z1t/vph9jmZ3BJ1Tibnir/J5A/hc5L85fD/37XWFjF4+U6ukZayze1wW5p2Lbdsc71WWKK9sl/O2yztM/X6cIZt+Iwkb2mt3TKlfAtZ7wv8+599aNWCYZXkrUm+d5s070uSqvreJD+byQnxl5K8O5Pu3KcluSTbPzxgWlR4qyeW1JTv3fELOyvn3JY/bwta793tPqODSX4/yddmcm/5hZmMJfZ/dpBXMvnlcysz72tVdf9MflGtbH2A/pZMuu5vtO023Fp7ZU0GOf6qTO6lPzeT+v9ZVZ27/lfTGR31bXOVtrkFrve5mmc5V6l9joI9+YvjiNtoljKuQn0Wfe0zzU7Oga5n7uhobEe7Xu+ttZur6upMxo07KZMeF/+ttbZWVW9K8siq+vUkD8kwftg87eIaaZWuU/bkOeOIVbmm6bBX9su52kPXcltmmzmv9yWfA1lBqxYMuy7JvZK8vuMXhydlMkbFY9anrarzFle8HZl3OW9IckZVHbv+V62qOjWTruSLtoj1Pku7z+KVmXTpv7Cq/iHJlyR57oZfQm9I8sCqOm79/Ko6LskD8om/MB751fMemayDI2nvlEmPvetnLN+3ZnKieGqSzX6N/PFMbgPYLBg21fAr7UuTvLSqKslzMrnP//xMgoSz1H2ZVmmb61nv/5l0SlY7aZ8HZbLdr3fmuvx2Us5pVqp9Oixiv3DcXo5ZyrgK9UkW05a92/y8z4FHlr2sfWOs+0XS3+avT/IdmYwLdkKSy4f5l2cyGPVjMrmOOdKLbJ4WdY20iG1ukX+TTLtWWKg5Xiss06rslzOboX2mXR/+ZPq34WuTfFZVndhau3Wb4i1ivS/67/+l7m/M36qNGfaSJPfOFhHkqjpl3dvDmWywte7z45JctMgC7sC8y/nKJKck+a8b5v/gdl+qquOr6rOq6lN3uNwjFrHeZ2n3bsOtkZdk8mSRZw6zX7gh2SsyOVA/ZcP8pw7z/3DdvCO/TJ67Ie33ZMZ9raqOyWSQxre21l7QWnvZximTByY8uKq+cNvM7pj3sTV5WtJ/aq0dGVQ2mfwhk8xW95mNbZubYb0f8W9bzD9iJ+3ztJo8JexIme6aSVfyDyd50w7LOc1KtM8MFrFfOG5/fP686tRjljIudJvb423Zu83P7Ry4zjL3jbHuF0l/m78+k7Z9ZpJ/bq29bd38E5P8UCYPJfrT9Znstj6LvEbKDre5KRb5N8m0a4WFWMC1wjIt85piIXbQPtOuD2fZhv93Jrcj/vAm5VrfA3MR633Rf/8vZX9jcVatZ9gvJnlUkp+uqnMyOdl+NMmnJnlkJo9tfcSQ9mVJ/leSP66qP8jkyRRPTHL70S70FPMu508N339+VX1+kqsyGRj7i5JsNrbUEadlcr/4m/KJA2nPahHrfZZ2n9Xzk3x/kickeVNr7boNn/9Ukm9McnFVfV4mJ5HPzaSb8TXD50e8bpj37Kq6Z5K3Z/KY8Ydn+3W/mUdn8ljvjcG59V6eydO7LkxyxQx5f3KSm6rq0kzq895Mxgd5WibjfLxqSDdL3XdibNtc73o/4oocGXlkAAAC5UlEQVRMHlH/jKq6e5J/T/L21tqbh8930j7vT/LmqvqN4f23DnV6yrpBQ2ct5zSr0j69FrFfOG5/3LzqNO8yLnqb28tt2bvNz/McuH7Zy9o3xrpfJP1t/sZMzlMPyuTHxSRJa+3qqnpPJj1L/qq19q8b8t9tfRZ5jbTTbW47i/ybZNq1wqLM+1phmZZ5TbEos7bPtteHVTXLNvyLmfQW/eEhGP2aTNbhZyd5YD7+g8ki1vui//5f1v7GorQ98EjLjVO2eUxuJgG878pkY/z3Yboukyj0o9elOzaTX6Suz+SRye/M5AT3oGz9mPUnZ5vHpW73eSYXBO/YQV27y9m7/EwOIi/L5IDy0UwOeJ+RDY823/D90zOHx5QvYr3P0u47LPPlw/KftMXn90ryq0luzORgemOSi5OcvEnaByT5k0zuO/9wkt/L5KJvy3W/xTJ/fyjTg6eku2ZYzp1nWJcnZHKi+H+ZDBp761C+FyU5Yyd138m+MbZtbpb1vu47F2QygOltQ3kv2WX7nJvkRzN5ZPWtmYzV8MTdlnM/tM8m+Z+drc9D3ceEGerjuD3HOm2S3x3qMGsZd1ifLbejFWvL3mNN1zlwu2VmwzljWfvGftwvZlzvvW3+19nkGmqoT0vyE/OuT3ZwjbTobW6e21LPdrQh/bbXCjtcx2dnm2NX5nytMOvyZ0mXKeeBIc1CrymO9tTbPum8PtzBNnynJM/IJJh8y7AfXpHkOxa53mct54Z1cHbnMua+v5mWN9XQqDBaVXVZJr/43ae19h/LLg/MW1U9OclvJHlEa+2Nyy0NAADL5vqQsVu1McNgrmryKO6vTPJSgTAAAADY/1ZtzDCYi6p6WCZdZr8rk26uP7vcEgEAAABHg55hjNXTMrlv/i5Jvrm19o7lFgcAAAA4GowZBgAAAMBo6BkGAAAAwGgIhgEAAAAwGoJhAAAAAIyGYBgAAAAAoyEYBgAAAMBoCIYBAAAAMBr/H5QV0wf1UW5kAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 1728x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "seed_txt = 'learn and. are you Arson. are stops are lord. are. feel. around. are. now. And art is life    space art'\n",
    "predict(seed_txt,prediction_model,386,387,2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3. 'the'  and 'he':\n",
    "\n",
    "This is yet another interesting observation where the particular node of LSTM triggers and gives a strong positive activation whenever the string `the` is encountered. We tried to observe whether the same kind of activation can be observed for strings succeeded by `he` (can be observed in words `she, shelter, check`). The node showed no such activation at such words (slight purple). However, the node showed strong activation for variety of strings having sub-string `the` (such as then, they, their, there)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of input text: 122\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABLsAAAEECAYAAADAojb8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3XuULVddJ/DvLwnhGRAMo5AEghoewXEpg6CDQiZETJiByEIZgiCw0DhgRGFQozA84qAi6ggakQsiDx15zahBg3EEAo5LMEEQSDBwCUgS3vKIEJJwb+/549R1Dk33OXW6q/ucrvv53FWr76mqs/dv79qnqnr3rn2qtRYAAAAAGIMjlh0AAAAAAAxFZxcAAAAAo6GzCwAAAIDR0NkFAAAAwGjo7AIAAABgNHR2AQAAADAaOrsAAAAAWIqqellVfaqq3rfJ9qqqF1bV/qp6T1Xda16aOrsAAAAAWJaXJzl9xvYzkpzULWcnedG8BHV2AQAAALAUrbW3JfnsjF3OTPLKNvH2JF9XVXeYleZRQwa4FUcdfVybtf3aXzpjbhq3/oU3DhLLWXe478ztf/TxdwySzxCuu/Iv5u5zi2+a1THKdn35Y389c/vN7/i9uxTJMB52h3vP3eePP37pLkSyWu597Elz97n0Mx/cdj5f/JsXzt3nVvd78rbzOVxdcNv5n8eHfm72Z7qPL7711+buc6sHPG3b+Qxlt85j8/IZMq+95F/+7Olz9znmPz13FyLp519e8aMztx/z2JfuUiSr5UuXvW7m9lve84fmpvHFN/3K3H1u9cBze8e0F3z5qjfP3efmJ5y6C5HsvXPUKsW7SrEcjp56x/vP3ec3Pva2XYhk99rCB+52z7n73PWKy7adzyEHbrxm/aoaLPE97iufuXJmX84hR9/+m388k9FYh+xrre1bMLvjklw19frqbt3HN3vD0ju7AAAAABifrmNr0c6tbdPZBQAAAEB/awd3M7drkpww9fr4bt2mzNkFAAAAQH8HD/RbhnFBkh/pvpXxu5J8obW26SOMiZFdAAAAACygtbXB0qqqP0pySpJjq+rqJM9KcpNJPu13k1yY5MFJ9ie5Lsnj56WpswsAAACA/taG6+xqrZ01Z3tL8hOLpKmzCwAAAID+BhzZtRN0dgEAAADQ3+5OUL8wnV0AAAAA9GdkFwAAAABj0Yb7psUdobMLAAAAgP4GnKB+J+jsAgAAAKA/jzECAAAAMBomqAcAAABgNIzsAgAAAGA0TFAPAAAAwGiYoB4AAACAsWjNnF0AAAAAjIU5uwAAAAAYDY8xAgAAADAaRnYBAAAAMBoHv7LsCGbS2QUAAABAfx5jBAAAAGA0PMYIAAAAwGgY2QUAAADAaOjsAgAAAGAsmgnqAQAAABgNc3YBAAAAMBoeYwQAAABgNIzsAgAAAGA0jOwCAAAAYDSM7AIAAABgNA4cWHYEM+nsAgAAAKA/I7sAAAAAGA1zdgEAAAAwGkZ2AQAAADAaRnYBAAAAMBpGdgEAAAAwGr6NEQAAAIDRaG3ZEcykswsAAACA/szZBQAAAMBorHhn1xHLDgAAAACAPaSt9Vt6qKrTq+qKqtpfVedusP1OVfWWqnpXVb2nqh48L00juwAAAADo7+DBQZKpqiOTnJ/k+5JcneSSqrqgtXb51G7PSPLa1tqLqurkJBcmOXFWukZ2AQAAANDf2lq/Zb77JNnfWruytXZjklcnOXPdPi3Jrbv/3ybJx+YlamQXAAAAAP31nLOrqs5OcvbUqn2ttX1Tr49LctXU66uT3HddMs9O8pdV9ZNJbpnktHn56uwCAAAAoL+e83F1HVv75u4421lJXt5a+/Wq+u4kr6qqb21t8yB0dgEAAADQW1trQyV1TZITpl4f362b9oQkpydJa+1vq+pmSY5N8qnNEjVnFwAAAAD9DTdn1yVJTqqqu1TV0UkemeSCdft8NMkDk6Sq7pHkZkk+PStRI7sAAAAA6G+gb2NsrR2oqnOSXJTkyCQva61dVlXnJbm0tXZBkv+a5CVV9ZRMJqt/XGtt5tAynV0AAAAA9Ndzgvo+WmsXJrlw3bpnTv3/8iT3WyRNnV0AAAAA9DdgZ9dO0NkFAAAAQH+znyJcOp1dAAAAAPRnZBcAAAAAo7FmZBcAAAAAYzHQtzHuFJ1dAAAAAPTWPMYIAAAAwGh4jBEAAACA0WhGdgEAAAAwFkZ2AQAAADAaB0xQDwAAAMBYeIwRAAAAgNHwGCMAAAAAY9HWjOwCAAAAYCyM7AIAAABgNHR2AQAAADAaB30bIwAAAAAj0YzsAgAAAGA0dHYBAAAAMBq+jREAAACA0TCyCwAAAIDR0NkFAAAAwFi0gx5jBAAAAGAsjOwCAAAAYCyazi4AAAAARkNnFwAAAACjsdpTdunsAgAAAKC/dmC1e7t0dgEAAADQ32r3densAgAAAKA/E9QDAAAAMB5GdgEAAAAwFkZ2AQAAADAeRnYBAAAAMBbtwLIjmE1nFwAAAAC9tRUf2XXEsgMAAAAAYA9Z67n0UFWnV9UVVbW/qs7dZJ9HVNXlVXVZVf3PeWka2QUAAABAb0ON7KqqI5Ocn+T7klyd5JKquqC1dvnUPicl+fkk92utfa6q/s28dI3sAgAAAKC3ttZv6eE+Sfa31q5srd2Y5NVJzly3z48lOb+19rkkaa19al6iOrsAAAAA6K0drF5LVZ1dVZdOLWevS+q4JFdNvb66WzftrknuWlV/U1Vvr6rT58XnMUYAAAAAeuv7GGNrbV+SfdvM7qgkJyU5JcnxSd5WVf+2tfb5WW8AAAAAgF7aWg2V1DVJTph6fXy3btrVSd7RWvtKkg9X1Qcy6fy6ZLNEPcYIAAAAQG8Dztl1SZKTquouVXV0kkcmuWDdPn+SyaiuVNWxmTzWeOWsRI3sAgAAAKC31oYZ2dVaO1BV5yS5KMmRSV7WWrusqs5Lcmlr7YJu24Oq6vIkB5P8TGvtn2elq7MLAAAAgN76ztnVK63WLkxy4bp1z5z6f0vy1G7pRWcXAAAAAL2tHRxszq4dobMLAAAAgN4GnKB+R+jsAgAAAKA3nV0AAAAAjEZry45gNp1dAAAAAPRmZBcAAAAAo9Gazi4AAAAARuKgb2MEAAAAYCyM7AIAAABgNMzZBQAAAMBo+DZGAAAAAEbDyC4AAAAARuPg2hHLDmGmLXV2VdVRSW6R5LrW2oFhQwIAAABgVa36Y4y9u+Kq6pFV9WdV9ckkNyT5XJIbquqT3fqzdixKAAAAAFbCWqtey7LMHdlVVbdIckGSU5Ncl+TdSS5Ocn2SmyU5LskpSc6oqh9N8pDW2nU7FC8AAAAAS9SW2JHVR5/HGM9Lcv8kT07yktbaDet3qKqbJjk7ya8neU6SnxkySAAAAABWwxgeY3xEkhe21n57o46uJGmt3dBa+60kv5XkkfMSrKqzq+rSqrp0be1Li0UMAAAAwNLs+ccYk9w+yft7pnd5kmPn7dRa25dkX5IcdfRxK94fCAAAAMAhq/5tjH2i+0iS03um9+BufwAAAABGqPVclqVPZ9e+JA+vqtdW1b+vqptMb6yqm1TV/arqdUl+oNsfAAAAgBEaw2OMv5nkxCTnJHl4krWq+kySG5LcNJPHFg91mp3f7Q8AAADACO35b2NsrbUkP1VVL07yqCT3TnLHJLdIcm2Sf0hySZLXtNbet4OxAgAAALBka8sOYI4+I7uSJK21y5M8YwdjAQAAAGDFtezxkV0AAAAAcMiBvf4YIwAAAAAcYmQXAAAAAKMxmjm7AAAAAMDILgAAAABGw8guAAAAAEbjoJFdAAAAAIzF2mr3densAgAAAKC/NSO7AAAAABiLtuwA5tDZBQAAAEBvJqgHAAAAYDTWymOMAAAAAIzEwWUHMMcRyw4AAAAAgL1jrfotfVTV6VV1RVXtr6pzZ+z38KpqVXXveWnq7AIAAACgt7VUr2WeqjoyyflJzkhycpKzqurkDfY7JslPJXlHn/h0dgEAAADQW+u59HCfJPtba1e21m5M8uokZ26w3y8meV6S6/skqrMLAAAAgN76PsZYVWdX1aVTy9nrkjouyVVTr6/u1v2rqrpXkhNaa3/eNz4T1AMAAADQ21rP/Vpr+5Ls22o+VXVEkt9I8rhF3qezCwAAAIDeDvacfL6Ha5KcMPX6+G7dIcck+dYkF1dVknxjkguq6qGttUs3S1RnFwAAAAC99R3Z1cMlSU6qqrtk0sn1yCSPOrSxtfaFJMceel1VFyd52qyOrsScXQAAAAAsYK3nMk9r7UCSc5JclOT9SV7bWrusqs6rqoduNT4juwAAAADorQ33GGNaaxcmuXDdumdusu8pfdLU2QUAAABAbwM+xrgjdHYBAAAA0NvBZQcwh84uAAAAAHpbG/Axxp2gswsAAACA3jzGCAAAAMBo6OwCAAAAYDTasgOYQ2cXAAAAAL2ZswsAAACA0fBtjAAAAACMxtqKP8ioswsAAACA3kxQDwAAAMBorPa4Lp1dAAAAACzAyC4AAAAARuNArfbYLp1dAAAAAPS22l1dOrsAAAAAWIDHGAEAAAAYjbUVH9ulswsAAACA3la7q0tnFwAAAAAL8BgjAAAAAKNxcMXHdunsAgAAAKA3I7sAAAAAGI1mZBcAAAAAY2FkFwAAAACjsWZkFwAAAABjsdpdXTq7AAAAAFjAgRXv7tLZBQAAAEBvJqgHAAAAYDRMUA8AAADAaBjZBQAAAMBoGNkFAAAAwGgcbEZ2AQAAADASax5jBAAAAGAszNkFAAAAwGiYswsAAACA0Vj1xxiPWHYAAAAAAOwdree/Pqrq9Kq6oqr2V9W5G2x/alVdXlXvqao3VdWd56WpswsAAACA3g621muZp6qOTHJ+kjOSnJzkrKo6ed1u70py79batyV5fZJfnZeuzi4AAAAAeltL67X0cJ8k+1trV7bWbkzy6iRnTu/QWntLa+267uXbkxw/L1GdXQAAAAD0ttZzqaqzq+rSqeXsdUkdl+SqqddXd+s284Qkb5wXnwnqAQAAAOit73xcrbV9SfYNkWdVPTrJvZM8YN6+OrsAAAAA6G3Ab2O8JskJU6+P79Z9lao6LcnTkzygtXbDvER1dgEAAADQW+sx+XxPlyQ5qarukkkn1yOTPGp6h6r6jiQvTnJ6a+1TfRLV2QUAAABAbwcHGtnVWjtQVeckuSjJkUle1lq7rKrOS3Jpa+2CJM9Pcqskr6uqJPloa+2hs9LV2QUAAABAbwM+xpjW2oVJLly37plT/z9t0TR1dgEAAADQ24CPMe4InV0AAAAA9DbkyK6doLMLAAAAgN6azi4AAAAAxuKgxxgBAAAAGAuPMQIAAAAwGjq7AAAAABgN38YIAAAAwGgY2QUAAADAaPg2RgAAAABG42BbW3YIM+nsAgAAAKA3c3YBAAAAMBrm7AIAAABgNMzZBQAAAMBorHmMEQAAAICxMLILAAAAgNHwbYwAAAAAjIbHGAEAAAAYDY8xAgAAADAaRnYBAAAAMBpGdgEAAAAwGgfbwWWHMJPOLgAAAAB6ax5jBAAAAGAs1jzGCAAAAMBYGNkFAAAAwGj4NkYAAAAARsO3MQIAAAAwGgfb2rJDmElnFwAAAAC9mbMLAAAAgNEwZxcAAAAAo2FkFwAAAACjsWaCegAAAADGwsguAAAAAEbDtzECAAAAMBomqAcAAABgNFb9McYjlh0AAAAAAHtH6/mvj6o6vaquqKr9VXXuBttvWlWv6ba/o6pOnJemzi4AAAAAemut9Vrmqaojk5yf5IwkJyc5q6pOXrfbE5J8rrX2LUn+R5LnzUtXZxcAAAAAva211mvp4T5J9rfWrmyt3Zjk1UnOXLfPmUle0f3/9UkeWFU1M9W+vXG7uSQ5ezvbd3MfsYjlcC6zWMQiFrGIRSyrkM8qxXI4llksYhGLWMRimVVnSS6dWs5et/0Hk7x06vVjkvz2un3el+T4qdcfSnLszHyXXfBNKuPS7WzfzX3EIpbDucxiEYtYxCIWsaxCPqsUy+FYZrGIRSxiEYtlq0t2qLPLY4wAAAAALMM1SU6Yen18t27DfarqqCS3SfLPsxLV2QUAAADAMlyS5KSquktVHZ3kkUkuWLfPBUke2/3/B5O8uXVDvDZz1OBhDmPfNrfv5j5iEctu5yMWsYhFLGIRyyrkI5bl5iMWsYhFLGJZvVhYUGvtQFWdk+SiJEcmeVlr7bKqOi+Tx0YvSPJ7SV5VVfuTfDaTDrGZak5nGAAAAADsGR5jBAAAAGA0dHYBAAAAMBo6u1ZUVT2uqlpVnbLsWA5XYzsGO1GeqjqlS/NxQ6W528Z2nPsYY5l3q0x7re72Wrxjs9fqf6/Fu1ucX7ZmlcqzSrH0sUrxjvH+cS99psfeFlYhr62qqo9U1cXLjoPN6ezaoqr69qp6dlWduArpsDXqf+eoW/aqvdZ291q8Y7PX6n+vxdvHEGUaY73MM8Z72VWKZZ5VinWVYgEYis6urfv2JM9KcuKKpMPWqP+do27Zq/Za291r8Y7NXqv/vRZvH0OUaYz1Ms8Y72VXKZZ5VinW3Y7lbUlunuRVu5QfcBjS2XUYq6ojq+oWy47jcKX+t07drT7HaGvU28bUy8bUC7AXtdbWWmvXt9YO9tnfuY5l0O72vqV3dlXVw7rncX9sk+2XVdX+qjq2qs6vqquq6sbu5/lV9fXdfs/u0jlxgzR6PU9bVcdU1X+vqndU1Weq6oYu71+ZbuhV9ewkv9+9fEuXb6uqly9Y9j7pHFFVT6uqD3XxfKCqHjuVxk2r6he6erq+qj5fVW+oqu9Yl9eh555Pq6r/VlUfSnJ9kkf0TWMIVXWz7lhdUVXXdXm9t6qe37c8Q8W7wHHc9BisWv3Pq9+hytMjjmdnTt0OUXd7qV5qhc51fcq8SLl3qn1X1Z27dJ+zbv1F3fqnbPC2V2znOPZpu/Pqbpfb9o7HOxXnqbPaS49YZ7bt7dTLom2lqv6uqj41o8xD3w8cNu2lZxxz29QC7aVvmVa+XqYcVZNz/T91+bynqh65SD41/36rb5ln6pnOINeaZcdS/a/jNVCsM+Pd5XobNJaaMWdXDXOuO7qqfraq3t21/y9U1aVVdc6QZVogn/XxPb0r429VVd/fx3f0c9SnfSd53m7E0rfc8/Kpntfybt9Z97InVNVru+N7bZfHN2+hLOy21tpSlyRHJfl4kr/dYNt3JWlJnpPkA0nWkrw0yZOSvKR7/f4kxyR5drfviRuk85EkF/eI5e5JPpHk/CQ/leQnkry2y+eiqf2+LcmLu/yem+TR3fLdC5Z903SSPK5b//Yk/5DkZ7uY/rFbf78kN0nyliQ3dPXyxCTnJvlQkuuS3Hsqr0PpvTvJZd1+T0ryPX3TGOh4/14XxyuS/HhXx7+Z5J19yrNImbdT/+vqbLNjcP9Vq/859TtYeabyO6V77+MWrNsh2m7vz9tu18seONfNK/PKnF+6ff966vXRSb6U5GCSN0yt/y9dDJ/YznHMapwXFmnbOx5vn/bSI87bZE7b3m69LNBWbtfle3BGmQe5Hzgc20vPOObl86AF2su278lWsF7e2aX9c91yKJ8nLNBeNr3uDVVv89LpUW8LXWtWIJb7Zv51/Bf2UptbVizZ5P5xiHNdJuf+t3RpXJTkaUnO6cr45qHKtGA+p3Svj0zyom7duQOdLwf5HKXfferrdyOWAY9Rr2v5nHb3fUk+nORAkt/u1r0myUeTfDo97rsty1uWHkBrLUl+qWtcJ69b/5KuYb2g2/6kddt/olv/ixnmF8Cjk9xkg/W/2KV9n6l1X3Xy2kbZN0xnav27khw9tf647kP9R0me0u3z/evee+vuA3jxBuldkeQWU+t7pzHQsf5skgs32TY3lqHjnXUcexyDv1+1+p9Tv4OVZ2rbKZl/s7JR3W677a5yvcyIY1XOdfPKvDLnl65ubjyUbiY3Ny2TeT6uTXJUt/43u/VXbvc4zmm7g7WXIdr2bsTbp730iPG5mdO2t1svC7SVl3brz5tR5kHuBw7H9rJgDJvlc1nf9jKvTHu0Xv4pyW2m1t+mW/elBdrLpte9IettVjo96m2ha80qxJL51/E77qU2t6xY0u/+cUvnukw6RVqSX9pg2xFDlWnBfE7JZI6yP87kOvUjWzg2O/456tG+f3oXYxniGG3lWr7+XvZQnTx+XRqH7jsX+qxZdndZ+mOMnZdk0liecGhFVd0yyX9O8sZMelQ/nWTfuve9uFv/sCGCaK3d2Fr7Spf/UVV126o6Nslfdbvcd4h8FvQ7rbUbp2K8JpO/dp6UyV9d/jHJO2sy3P/YLt6jk/yfJN9TVTdfl96LWmvXTb3eShrb8YUk96yqb91g29xYkjxml+NNNj8Gd9tCLDtd/7PqdyfKs1VDtN1FrEq9rMS5bspeOL+8OZO/En5v9/rUJJ/KpGPwmCTf2a2/e/fzBbvUvnfyvLATthxvJvU/K42TeuT/sCzethetl75t5SHdz1fOKPORA98PHDbtZaDP0Tdld8+Ffe1WvbyotfaFqXy+kOR3k9wiyVU98+lz3dstQ19rlhZLkldmxnW8tfaxAWOdFe9u3rMtK5atnut+OMnnkpy3fkNrbW3dqu1cGx+9QD636953WpKHtNZeuYVy7cbnaN596ud3MZZ55e5zjLZyLV/f7n4gyScz+exPe15YeUctO4Akaa19uKr+KsljqurcrlE+IpOb05cmeXWSS1trB9a970BVfSDJvYaKpaqelMnjMPfM185pdtuh8lnAlRus++ckd07yDZn8leDTM95/bCY3Rod8YN32e2whje346Uz+yv7eqroyk+Gnb+iWPrHcI8nN5uwzZLzJ5sfgnpn8gr1K9T+rfg8ZsjxbtZVyr6+7RaxEvazSua6zF84vb+5+nprJYwKnZnL8/j6Tm8xTk/xtkpO7/d63QRo70b538rywE7YT7zFz0rhzj/zvksXb9qL10ret3L7b74Mz0jq2qh6S4e4HDqf2MtTnaDfPhX3tVr28f4N1l3c/T+iZz6bXvQ1+Ed9pQ19rlhnLlzL5RXmz6/jQVuGebVmxbPVcd1KSd7fWru+x73bKdFKSd/XM5+VJbpXk/q21/9tj/43s+Oeox33qoWvfbn6mt9XutnAtX9/uvinJJW3dlym01j5eVZ8PK20lOrs6+5K8LslDk/yvTHqUP5Hkz3u+v83Y1qucVfXUJL+e5C+TvDDJxzIZanpcJiepZYyE2+xbSqpb3pvkqTPev/4EsP4vJFtJY8taa39ak4m1H5zkAZn8heMJSf66Zyxv6LHPYPF2Zn1TzErV/5z6fUW325Dl2aoh2m5vK1YvSz/XTVn580tr7ZNVdXmSU2symeh9k/xka22tqt6a5IFV9btJju/eslvHcSfPCzthO/HeZU4acydj3qKF6mWBtlJJPpPkrBnJnZXJX22Huh84nNrLUJ+jVbQK18+PJXnsvHxmXfeq6rTpkRK7YOhrzbJj2e51fBGr0OaWFcuqn+suWCCf1yR5fJL/VlU/0Fr78gLvPWS3Pkez2vejdzmWWXmlRz5buZbvRrtjl6xSZ9efZvK4wROq6n2ZTHD3vO6veFcmuVtVHTX9V76qOirJXTPp8f1st/p2mcxbc2ifmyW5Q5L9PWJ4TPfeM6b/6lVVp2+w76xfOBexnXQ+mMlfqN+8jb/SDZHGQlprn03yB0n+oKoqya9k8tz7R+fFUlVDx7ud+j8wQCyD1/+M+n3vnLcOUZ6vCmXGtlVqd7tdL6twrutjlc4vb85kQtCHZDI8/U3d+jcl+bUkZ2R+h8six3HZ54VF7Wi8tcG3ZW1Bn7Y9hD5tJZlMEjyrzM/P7twPjK69DORAFmsvQ9yTrVK93COTa8W0Q6NXb943nxnXvTMz+WV2LPeyuxpLVW16HV8wv1Vqc6sUy3Z9IMndq+qmrbUbtphGn2vjIvn8YSbXoVcl+bOqesjAj6MP+TmadZ+627HM0ucYLXIt38yVSU6qqiOnR3dV1R2SfN1WAmf3rMqcXemGSb48yfcneVa3+ve6n3+SSWP+0XVv+7Fu/R/n/w85PG3dPk9J/3IezORk/6+f5O7G6twN9v1i9/N2PdPezHbSeWWSb8wmPdpV9Q07kUZV3aSq7l5Vd1og1lTVkVX1VSeF1tqhiQeT5JIesQxR5mnbqf8vDhDLYOXpUb+3nJPEEOVZn16ycd0OfRw3tWr1siLnuj6Wcn7ZxJszKduzkny0tfahqfU3TfLzmT9CZJHjuOzzwlbyTFY73j5tewh92spaJo8uzCrzbt0PaC8buy6LtZch7slWqV6eWFW3mXr/bTJ5DOfLmd92+1z3DpVxLPeyuxrLnOv4Ilapza1SLNv1h5l8Tp6xQSx9RyL3KdNC+bTWXp3JSKPvTfLGqrpVz1j6GOxzNED73q17/D7HaJFr+Wb+NJNHM39k3fqfWyRYlmOVRnYlk0nxfiaTE8FbW2uH5tP41SQ/lOT8qrpXJhfr78hkWOUV3fbruv+fV1Vfn8lXhH5PJl+V+pme+b8+yS9ncgL635l8m8Ojknxlg30vyeSG+elVddtMnuH/cGvtHQuVeJN0er73BZlMaP38qjo1k5v5a5PcKckDk1yf5D/sQBrHZTKfxFsz+XaRvo5J8vGquiCTY/ipTB6PeWIm86k8tdtnViwPGqDM07ZzHK/t3r/b9b+ZefX77sx+9GGI8kybVbdDlnueVauXZPnnuj6WdX7ZyMWZtKV7ZHIDliRprV1eVZ/IZMTDh5J884w0FjmOyz4vLGqn433F5m/vrU/bHsLFmd9W3pHJ5N2zyrxb9wNjbC9DxPuFTM7VfdvLEPdkq1Qvn0nyjqr6/e7147u0fjzJw3vkM++6d2i+yrHcyy4jls2u44tYpTa3SrFs1wsyGd37jKr6zkweYbs+kzmb7pav/WPhRvqU6fsXzae19vqq+kqS1ya5qKrOaK1du42yHjL052g77Xu37vH7HKNFruWb+dXuPS+pqn+XybcFn5LkuzPsfTc7oa3AV0JOL5kM8WxJHrNu/e2T/E6SqzNpoFcnOT/JsVP73DXJX2Tyy+DnMzmRHJfJ8MWLe+R9ZCZ/9d2fyVea/lMmDfweXUzPXrcP9V49AAAB8ElEQVT/YzOZMPTGbvvLt1jmr0kns78C+OIkH+n+f1SSJ2fyYf9St3wwk782PGjqPbPS65XG1P4nZgtftZrJIyW/nOTvMplY8Ibu2LwsyUl9Y1k03q3Uf486u7iLfdfrf6v1O2R5pt53Sjb56uh5n5Eh6m5V66VnXMs8180t87LOLzNifucm9fWH3fo3DHkcN2u7Q7aXWWltoT3tWLx920uPGOe27YE+8/PaynN7lHmQ+4HDsb30zL9PPr3ay7wy7dF6OS3JczKZ3uGGTB63f1S3T5/7pLn3W0PW22bp9Km3vmVaxViyyXV8L7W5ZcWSGfePs/JboCw3S/L0TDomrs/kXumSJE8askxbzSfJf+z2f3uS2/Q8L+z25+hr2vduxjLEMcoC1/I5+d0pk46za7vlDZn8gfUjWfD3YcvuLtUdwJVRVRdm0lN6x7a1yfsAVp5zHQDsXa7jjJn2zRiszJxdSVJV35LJkNA/8KECxsq5DgD2Ltdxxkz7ZixWYmRXVd03k+GET+5+3qO19pGlBgUwMOc6ANi7XMcZM+2bsVmVkV1PzGQegVsn+WEfKmCknOsAYO9yHWfMtG9GZSVGdgEAAADAEFZlZBcAAAAAbJvOLgAAAABGQ2cXAAAAAKOhswsAAACA0dDZBQAAAMBo/D8z7ZvJm02FLAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1728x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "seed_txt = 'you at there. see there shelter she you here whether then or weather the not the best the they the their check he yhe heard'\n",
    "predict(seed_txt,prediction_model,99,100,2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4. Capital Letters:\n",
    "\n",
    "Here, we can clearly see activations at capital letters even though the activations are a bit delayed (starting from the second or third capital letter)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of input text: 97\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABMMAAAEECAYAAAAlPPOQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3X+8fdd8J/7XO4lfaVGECFKhQvFlVAla+vUjNMZ0Qk2VahuayJfWo1rlOxlaVZ2qiE59OxOdfvxoov0qLUZC0yKKaqkmo4hQEkGFCElDEJJ87l3zxz63rpNz7937/Lg/cp7Pz2M/Pvfs8z5rr7PO3muvs87aa1drLQAAAACwDA7Y6QwAAAAAwHbRGQYAAADA0tAZBgAAAMDS0BkGAAAAwNLQGQYAAADA0tAZBgAAAMDS0BkGAAAAwNLQGQYAAADA0tAZBgAAAMDSOGinM/C5+x7ddjoPi3KbZz+gd+xDnvu3vWPP/+q/9I69dmV/79hvffF9vWOveu6JvWO/+pHeoQvz+Euu7h17+TVX9o691Q1v1jv2A+ed3ivui4/qX7a3/unDe8d+5Q2f7x17u3fs6x07JL9D3OxO/ffdKz+zmKrssDe+uHds3eyQ3rHPud/zesc++5aX9Y7da4Ycl4uq94b4wK2P6h17z+fdvnfss065tHfsW674aO/Yq67tX75DyuybX+h/vlr5xN/3jr30Wf3qyGR31FG3+cV794798iv6f26n/Outesf+8Zc/2Dv2wOr/++e3Buw7Vw1oO3zzWSf0jr3xr/1K79hLT/iD3rG7wZD990H3Oq537JsOu1Hv2CPP/+fesattbzXVH3Dru/WOfeRBh/WOfeqt+tfVe82Nb35t79jHfqp/XXLhN77YO/aOB9+md+yQNsGVn39379hLH9O/jvq+H/3e3rErV1zVO/YRf92//v301y/pHfvAWxzZO/Yfrrigd+zdbnaH3rFfGfAd62Hfe+fesad98QO9Y4faf80XJq2uhW1wj7n2sot6nSBucMidd2WZGRkGAAAAwNLY8ZFhAAAAAOwhqys7nYOZ6AwDAAAAoL8FTU2yXXSGAQAAANBba6s7nYWZ6AwDAAAAoL9VnWEAAAAALAsjwwAAAABYGibQBwAAAGBpGBkGAAAAwLJo7iYJAAAAwNIwgT4AAAAAS8NlkgAAAAAsDRPoAwAAALA0jAwDAAAAYGmYQB8AAACApWECfQAAAACWRWt7e86wA3Y6AwAAAADsIW2139JDVR1TVZ+sqgur6qQNYp5QVR+vqvOr6nWzZt/IMAAAAAD6m9NlklV1YJJTkzwyycVJzqmqM1trH18Xc2SS/5LkR1trV1TVbWbdrpFhAAAAAPQ3v5FhRyW5sLV2UWvtmiSvT3LsWMzTkpzaWrsiSVprX541+zrDAAAAAOhv5dpeS1WdWFXnrltOHEvp9kk+v+7xxaN16901yV2r6u+r6h+q6phZs+8ySQAAAAD663mZZGttX5J9M27toCRHJnlokjsk+duquldr7avTJmhkGAAAAAD9ze8yyS8kOXzd4zuM1q13cZIzW2vXttY+k+RT6TrHpqYzDAAAAID+Vlf7LVs7J8mRVXWnqrphkicmOXMs5i3pRoWlqg5Jd9nkRbNk32WSAAAAAPQ3p7tJttb2V9Uzk7w9yYFJXtNaO7+qXpTk3NbamaPnHlVVH0+ykuS5rbXLZ9muzjAAAAAAemsr184vrdbOSnLW2LoXrPu7JXn2aJkLnWEAAAAA9NdvPrBdS2cYAAAAAP3N6TLJnaIzDAAAAID+jAwDAAAAYGkYGQYAAADA0jAyDAAAAIClsX//TudgJjrDAAAAAOjPyDAAAAAAloY5wwAAAABYGkaGAQAAALA0jAwDAAAAYGkYGQYAAADA0nA3SQAAAACWRms7nYOZ6AwDAAAAoD9zhgEAAACwNHSGAQAAALA0TKAPAAAAwNJYWdnpHMxEZxgAAAAA/blMEgAAAICloTMMAAAAgKVhzjAAAAAAlkVbbTudhZnoDAMAAACgP5dJAgAAALA03E0SAAAAgKVhZBgAAAAAS0NnGAAAAABLo5lAHwAAAIBlYWQYAAAAAEtjdW+PDDtgpzMAAAAAwB6ystJv6aGqjqmqT1bVhVV10oTnn15V51XVh6vq76rqHrNmX2cYAAAAAL211dVey1aq6sAkpyZ5dJJ7JHnShM6u17XW7tVau0+Slyb5b7PmX2cYAAAAAP2ttn7L1o5KcmFr7aLW2jVJXp/k2PUBrbUr1z38niQzX6NpzjAAAAAA+mv9JtCvqhOTnLhu1b7W2r51j2+f5PPrHl+c5AET0vmlJM9OcsMkDx+a3XE6wwAAAADor+cE+qOOr31bBm6dzqlJTq2qn0ny60mOmyU9nWEAAAAA9Le/3+T4PXwhyeHrHt9htG4jr0/yh7Nu1JxhAAAAAPTXVvstWzsnyZFVdaequmGSJyY5c31AVR257uFjklwwa/aNDAMAAACgv56XSW6ltba/qp6Z5O1JDkzymtba+VX1oiTnttbOTPLMqjo6ybVJrsiMl0gmOsMAAAAAGKCt9ptAv1darZ2V5KyxdS9Y9/ez5raxEZ1hAAAAAPQ3p5FhO0VnGAAAAAD96QwDAAAAYGmszO1ukjtCZxgAAAAAvTUjwwAAAABYGjrDAAAAAFgac7yb5E7QGQYAAABAf0aGAQAAALA0dIYBAAAAsCzaisskAQAAAFgWRoYBAAAAsCyazjAAAAAAlobOMAAAAACWxt6eMkxnGAAAAAD9tf17uzdMZxgAAAAA/e3tvjCdYQAAAAD0ZwJ9AAAAAJaHkWEAAAAALAsjwwAAAABYHkaGAQAAALAs2v6dzsFsdIYBAAAA0FszMgwAAACApaEzDAAAAIBlYWQYAAAAAEtDZxgAAAAAS6Ot1E5nYSY6wwAAAADozcgwAAAAAJZGWzUyDAAAAIAlsddHhh2w0xkAAAAAYO9orXotfVTVMVX1yaq6sKpOmvD8jarqDaPnP1hVR8yaf51hAAAAAPTWVvstW6mqA5OcmuTRSe6R5ElVdY+xsOOTXNFau0uS309y8qz51xkGAAAAQG+rK9Vr6eGoJBe21i5qrV2T5PVJjh2LOTbJ6aO/35jkEVU106RlOsMAAAAA6K2tVq+lqk6sqnPXLSeOJXX7JJ9f9/ji0bqJMa21/Um+luRWs+TfBPoAAAAA9Nb3bpKttX1J9i02N8MZGQYAAABAb631W3r4QpLD1z2+w2jdxJiqOijJzZNcPkv+dYYBAAAA0FvfyyR7OCfJkVV1p6q6YZInJjlzLObMJMeN/v5PSf6mtZ5dbRtwmSQAAAAAvbU20/z169Jp+6vqmUnenuTAJK9prZ1fVS9Kcm5r7cwkr07yJ1V1YZJ/TddhNhOdYQAAAAD0ttLvTpG9tNbOSnLW2LoXrPv720l+am4bjM4wAAAAAAaY18iwnaIzDAAAAIDe+t5NcrfSGQYAAABAb7NNX7/zdIYBAAAA0JuRYQAAAAAsjZXVA3Y6CzOZqjOsqg5KcnCSq1pr++ebJQAAAAB2q71+mWTvrryqemJVva2qLk1ydZIrklxdVZeO1j9pYbkEAAAAYFdYbdVr2a22HBlWVQcnOTPJw5NcleTDSd6T5NtJbpzk9kkemuTRVXVCkp9orV21oPwCAAAAsIPaLu7o6qPPZZIvSvJjSX45yStba1ePB1TVjZKcmOT3kvxWkufOM5MAAAAA7A7LcJnkE5L8QWvtf0zqCEuS1trVrbX/nuS/J3niVglW1YlVdW5Vnfu6y74wLMcAAAAA7Jjr/WWSSW6d5BM90/t4kkO2Cmqt7UuyL0k+d9+j93h/IgAAAMDy2Ot3k+yT+88mOaZnev9+FA8AAADA9VDruexWfTrD9iV5fFX9eVX9SFXdYP2TVXWDqvrRqvqLJI8dxQMAAABwPbQMl0m+PMkRSZ6Z5PFJVqvqsiRXJ7lRussi1zrVTh3FAwAAAHA9dL2/m2RrrSV5VlX9UZKfSXK/JLdLcnCSK5N8JMk5Sd7QWvvYAvMKAAAAwA5b3ekMzKjPyLAkSWvt40l+fYF5AQAAAGCXa7mejwwDAAAAgDX7r++XSQIAAADAGiPDAAAAAFgaSzNnGAAAAAAYGQYAAADA0jAyDAAAAIClsWJkGAAAAADLYnVv94XpDAMAAACgv1UjwwAAAABYFm2nMzAjnWEAAAAA9GYCfQAAAACWxmq5TBIAAACAJbGy0xmY0QE7nQEAAAAA9o7V6rfMqqpuWVXvrKoLRv/fYkLMHavqQ1X14ao6v6qevlW6OsMAAAAA6G011WuZg5OSvKu1dmSSd40ej7skyYNaa/dJ8oAkJ1XV7TZLVGcYAAAAAL21nsscHJvk9NHfpyd57HXy0to1rbWrRw9vlB59XTrDAAAAAOit72WSVXViVZ27bjlx4KYOba1dMvr7S0kOnRRUVYdX1UeTfD7Jya21L26WqAn0AQAAAOhttWdca21fkn2bxVTV2UluO+Gp54+l1apq4oCz1trnk9x7dHnkW6rqja21Szfaps4wAAAAAHpbmct0YJ3W2tEbPVdVl1bVYa21S6rqsCRf3iKtL1bVx5I8JMkbN4pzmSQAAAAAva32XObgzCTHjf4+LskZ4wFVdYequsno71skeXCST26WqM4wAAAAAHrbxs6wlyR5ZFVdkOTo0eNU1f2q6lWjmLsn+WBVfSTJe5O8rLV23maJukwSAAAAgN7aHC+T3HQ7rV2e5BET1p+b5ITR3+9Mcu8h6eoMAwAAAKC3OY362jE6wwAAAADobWWnMzAjnWEAAAAA9La6TZdJLorOMAAAAAB6c5kkAAAAAEtDZxgAAAAAS6PtdAZmpDMMAAAAgN7MGQYAAADA0nA3SQAAAACWxuoev1BSZxgAAAAAvZlAHwAAAIClsbfHhekMAwAAAGAAI8MAAAAAWBr7a2+PDdMZBgAAAEBve7srTGcYAAAAAAO4TBIAAACApbG6x8eG6QwDAAAAoLe93RWmMwwAAACAAVwmCQAAAMDSWNnjY8N0hgEAAADQm5FhAAAAACyNZmQYAAAAAMvCyDAAAAAAlsaqkWEAAAAALIu93RWmMwwAAACAAfbv8e4wnWEAAAAA9GYCfQAAAACWhgn0AQAAAFgae31k2AE7nQEAAAAA9o7VnsusquqWVfXOqrpg9P8tNoj7/qp6R1V9oqo+XlVHbJauzjAAAAAAeltprdcyBycleVdr7cgk7xo9nuS1SU5prd09yVFJvrxZojrDAAAAAOhtNa3XMgfHJjl99PfpSR47HlBV90hyUGvtnUnSWvtGa+2qzRLVGQYAAABAb63nv6o6sarOXbecOHBTh7bWLhn9/aUkh06IuWuSr1bVm6vqn6rqlKo6cLNETaAPAAAAQG995wNrre1Lsm+zmKo6O8ltJzz1/LG0WlVNGm52UJKHJPmhJP+S5A1JnpLk1RttU2cYAAAAAL3N6RLIJElr7eiNnquqS6vqsNbaJVV1WCbPBXZxkg+31i4aveYtSR6YTTrDXCYJAAAAQG99L5OcgzOTHDf6+7gkZ0yIOSfJ91XVrUePH57k45slqjMMAAAAgN628W6SL0nyyKq6IMnRo8epqvtV1auSpLW2kuQ5Sd5VVeclqSSv3CxRl0kCAAAA0Ns8L5PcTGvt8iSPmLD+3CQnrHv8ziT37puuzjAAAAAAeus7gf5upTMMAAAAgN7mNB/YjtEZBgAAAEBv23WZ5KLoDAMAAACgtzafyfF3jM4wAAAAAHpbMTIMAAAAgGXhMkkAAAAAlobLJAEAAABYGkaGAQAAALA0ms4wAAAAAJbFisskAQAAAFgWLpMEAAAAYGnoDAMAAABgabibJAAAAABLw8gwAAAAAJaGu0kCAAAAsDRW2upOZ2EmOsMAAAAA6M2cYQAAAAAsDXOGAQAAALA0zBkGAAAAwNJYdZkkAAAAAMvCyDAAAAAAloa7SQIAAACwNFwmCQAAAMDScJkkAAAAAEvDyDAAAAAAloaRYQAAAAAsjZW2stNZmInOMAAAAAB6a3v8MskDdjoDAAAAAOwdq2m9lllV1S2r6p1VdcHo/1tsEHdyVX1stPz0VunqDAMAAACgt9Zar2UOTkryrtbakUneNXr8XarqMUnum+Q+SR6Q5DlVdbPNEtUZBgAAAEBvq631Wubg2CSnj/4+PcljJ8TcI8nfttb2t9a+meSjSY7ZLFGdYQAAAAD01nr+q6oTq+rcdcuJAzd1aGvtktHfX0py6ISYjyQ5pqoOrqpDkjwsyeGbJWoCfQAAAAB6W2mrveJaa/uS7NsspqrOTnLbCU89fyytVlXXGW7WWntHVd0/yfuTfCXJB5JsertLnWEAAAAA9DbPu0m21o7e6LmqurSqDmutXVJVhyX58gZp/E6S3xm95nVJPrXZNl0mCQAAAEBv2zhn2JlJjhv9fVySM8YDqurAqrrV6O97J7l3kndslqiRYQAAAAD0Ns+RYVt4SZI/r6rjk3wuyROSpKrul+TprbUTktwgyfuqKkmuTPKzrbX9myWqMwwAAACA3lazPZ1hrbXLkzxiwvpzk5ww+vvb6e4o2ZvOMAAAAAB628aRYQuhMwwAAACA3vreTXK30hkGAAAAQG9zmhx/x+gMAwAAAKA3l0kCAAAAsDTaNk2gvyg6wwAAAADozcgwAAAAAJbGXp8zLK21XbkkOVGs2Ot77E5vX6xYsWLFihUrVqxYsWLFzj/WsruXHc/AhhlLzhUr9voeu9PbFytWrFixYsWKFStWrFix84+17O7lgAAAAADAktAZBgAAAMDS2M2dYfvEil2C2J3evlixYsWKFStWrFixYsWKnX8su1iNrnsFAAAAgOu93TwyDAAAAADmatd0hlXVU6qqVdVDdzovLIbPeH6q6rNV9Z6dzseyqqqHjvblp+x0XnYL+yR9DTl+HGvLRT0Cw6lT52/a7yzKF/aWXdMZxndVoM/ZJKZV1dvG4jda9o/i3lhVK1X14A3SfPDo+TfOkPf7VNULq+qIadNYtCHlUFU3q6rfqKoPVdXXq+qqqvp4VZ1SVYeOvW7thPmUDdI9YvT8afN/V3vbovabvbA/0llUvbdJ7DdGx/WvVtVBW8ROTHeK97a2rFTVFVX1sao6vaqOqaqaphzWrbtFVX1r9NzP9czHhuUwJN29ai/UD0P2nVH8kHPWkGPozlW1r6r+eZTmFVX1iVEeHradZbKstvisxpcjpjne121r03qoqm5cVb9YVX9TVV+pqmur6qtVdU5VnVxVPzhlbN/27NTvbRH2Ql2yaFV1cFX9SlW9r6r+dfQ5X1pVZ1XXPj5oLH6Wc9b/rqpnVdWB62Kvl3XUVsfiKGaa74SDjqG+6Q54X9PWZ9ercmD32NaTBgvzZ0nOmrB+dfT/M5I8JMlpVfXvWmvfXAuoqoOTnJbksiRPnyEP90nym0nek+SzM6SzSH3L4ZQkH0lyxyRvTvLqJNcmeWCSZyV5alX9RGvtA9ub/eulRe03i94f/zbJTdLtF+yMreq9SbGV5LZJfj7Jf0ty9yQnzpBuX+u3f9Mkd0vy2FE+zq6qn5oh7ScnuVGSzyT5hSR/0jMfW5XDkHQXaRHH2l44X63ps+/cJsnbM/yctem+XlX3S/LeUVqvTXJ+us/iyCSPSvL1JO+e1xtlQ+MdBg9Jd7zuS/K+see+kuSI0d9DjvctVdWdk7xt9Pr3Jvn9JJck+d50x9QvJHlOVX1/urqjV2xr7QvrNtO3/p3re5vBXqpL5q6q7pLkL5PcNcnZSX43XTv6NkmOTvLHSe6R5P9d97Jpz1m3S/KUJC9Pcs8kJ+7SOmqn24eLOobm1Taatj4bareXA7uEzrDrhw+11v50oydba1+pqv8nyf9K8tIkv7Tu6ZOT/ECSx7bWLltsNndWz3J4QroT6u2T/ERr7S/XxeyrqlekO+GfUVX3aq1duj25ZzuNfnW8UWvtqknPt9ZWk3x7e3M1H1u9tz1k03pvs9jRcfzPSU6oque31r4yZbp9XSfNqnp2unro2ekaVydPmfbx6Rr7ZyR5eVXdubV2UZ98bFEOQ9JdmL18rM3JVvvOG9J9WZjmnLXVvv6bSQ5Ocp/W2kfGn6yq207zhhhmwud/ULovaR+Y9PnVdwYMDjneN1VVN0nX6fEDSX6ytfa/JsTcOMmvpuvo6Bs7fhevvvXv3N4b0xntE29Lcuckj2+tvXks5OSqun+S+4+tn+Wc9YdJPpHuc/6N7MI6ahecsxZ1DM2lbTRDfTbUri4Hdo/deJnkAVX1nKr6dFVdXVWfqqrj1gdU1Y2q6nlVdX5Vfbu6YddvraofGot73Gjo4tMmbWj0+gurc9Oq+q9V9cGqumy07Qur6iWjUUPrX9c7drdorb0l3a8vz6iqRyTdkM90HUKvba2dMW3aVfXCdL/+JMm71w0ZPW2Dl8ztMx5qq3JI96vAXZO8fOxLxdrrz03yvCS3TvLcWfLSR1UdXlV/XlVfq6orR2XwAxvEHlJVp1bV56vqmtH/p1bVrWbMwxFV9abR9q+sqjOq6k41YW6XIZ/bFPtN3/wOSre+c5nr0dVdZvTpdA2ZJ2yyjbXh0k/Z4PkhZbaQ8h3y3gZ+br33yd1qNCr0H9L9ErgjeW+trbTWfi3J3yU5Jsm9hqZRVfdNNzLh9CSvS7I/3S/tffMwsRxmTXeetjrWRjE3ru5SpU9Wd5nMV6vqvKo6ZULsC9Ozfuh7XKw7zh6+1bltHsb2nUdlceesI5NcPulL5ijtLw1M798MrUfmfX6rAe3Dodsf7YutJlw6N6le3y4z1nsnJPnBJKdM6twapf/t1trvJnlM39jW2hcH5mOiWev0mqJdP7StsYg87LAT0o1U/b0JHWFJktbaOa21V6w9nsM568okH0j3Od85C6yj1tnyO8t6fc5Zu9FuaBftBsph+ezGkWEvTje89I+SXJ3u0rbTqurC1trfV9UNkvx1kh9J16nxP5LcPMnTkvx9Vf3YqPGXJG9N8qV0Fe0r12+kqh6Ybuju81trrapun65if1O+U0H/3+mG9v5Qkh9f9/IhsdM4uKoOmUP8NaMTx5pfTvKwJK+pqgcleU2SL4zWz+LNSQ5L17P/4nS/2iTJpzeIn+dnPI3NyuHMUcy+TV5/Wrph2o9PsuE17LOqqu9LN9z68CT/M8nH0+1n705Xfutjb57k/Unuku79fCjdvviMJA+vqqNaa1+fIg+3Sjds+dBRHj6Rbkjzu5N8z1js0M9t6H7T17TpvizJDdLVFVcm+eQ0Gx9YZoss317vbUi6Q/bJKSyq3tvIWiPnX+ec7lCvTvLgdJe0bbb9SY5P8o0kb2qtfbO6OTOOq6oXjH6d7mNSOcwj3e10arrz/GvTXd5wULovSQ+fENurfpjyeNv03Dbje5xkbd9JpjtnbbWvfzrJ3arqJzf6sjuNofXIgs5vQ9qHCzm/7pCN6r2t/KfR/6+ac+y4Werfad9bMl27ft5tmEV/t5i3tc95s7pn3EznllHn9F1GDy/LguqoMdtdr4/bzrbRZsfQdreN+m5/aPxeLwfmrbW2K5Z014G3JP+U5Ibr1t8+XeXzZ6PHa8Oqf3zs9TdL8i9J3jO2/sWj+HuMrX9luhPN7UaPb5jkBhPy9duj1x+1bl3v2IFl8NDR67da3tYz/m0TtvGo0XNfSXd98yPn/Pk9dLs/4ynzO7Ecklye5Moer//o6PXfu+59PWWD2CNGz582MI9r++5Tx9a/fLT+PevW/c5o3S+Oxf7SaP1vT1lOLx29/skbrF+fh8GfW5/9ZlH744TYTyY5uGf6a8fedT7zgWW2XeW74Xsbku6QfXLAZ7VWlnOv99bFviDJIelGx9wrXedJS/LBAfm4Tn064L09Z5OY+45i3juwHG6c5Iqsq1eSHDuKefQG+ehTDr3TneH4XMvPdY6faWLTNVjPGrD9teNiw/ph4HGxlt6m57YF7jvX9khv/Tmr176e5EFJrhmt+1S6jqBnJLn7jJ//oHokizu/9W0fDtp+kheO1h8xYZufHX9/U+R7bX+beExkwPHed79L1zb62oT4A0fbWL8Mib3J2Hb71v+D39sWZTpVuz5zbMNMm4excpm4T0wbu0U6Ez/nTeJnPWfdO92x2dJdUpcsqI4a+2wH1etzLN+tjomNjo25HkN9051DOU8sr2UpB8vOLbtxZNgrWmvXrD1orX2hqj6V7lfeJPnZdNfy/u8JPbPvTPcLw01aa98arXtlkpPS/Rrxa0lSVd+T5KeT/FUbDdFev83qrl++aboT99lJfj3JA5L849DYKe1L8hcbPPfOAfHXmTOhtfaOqtqX0WSFrbVJ6S3avD/jwTYph5ul+7V4K2u9/zefNg89PDbJpelGO6x3crpJkdd7XLrPe/wXuj9KN6fC45L8xhR5+Il0k97+2dj6l+W6l9ws/HNbsD9s85lHa0iZbVf5bvbehqQ7ZJ8camH1XpLfGi3rvTnfPW/gNOnOw1pdsjYSsG85/GSS70t3ucmas9Ll8xeS/NWE1/cph2nS3WlfS3LPqvq/Wmsfm1Oa0xxvW53b5m1t31kZELv+nLXpvt5a+0BV/XC6ttOjkzx1tKSq3pfuy8s088gNrUcWdX7r1T5c4PYXbUi9t5WN2kZ3T3Le2LrVJJMuf5wU+9x057s1fevfeb637WjX74k8DHSzdMdxX/M4Z62mu4LjxGShddR6212vj1tU22joMbTdbaO+20+WqxyYs93YGTap0ro83V2Sku5kepNsvtMdkuTzSdJa+0xVnZ3k56rqpNbatenmy7lpxoZwV9Uvpruj4j1z3fnUbjFt7BQuaK2dPemJmjyR4IbxG/hARpMVTpG3eZjrZzyDSeVwZboT/FbWYr42YHttQGzSzYdwTmvtu77otNYuqaqvjsXeKcm5rbX9Y7H7Ryft+w7c9vp0/7GNDV9vrX15Qh6263NblE/NKZ0hZbZd5bvZexuS7pB9cqhF1ntrjZcbpPvl7z8nuUMmT3I7tD6d1VpdsnZ3277lcHy6z+zi6u7oteYdSX6qqg5p170pSp9ymCbdnfYr6S5lPK+qLkp3ud1bk7x1/NgaYJrjbatz27yt7TsHDoicycPUAAAJp0lEQVRdf87acl9vrZ2X7pf7VNUd0122dUK6y7nPqKofXv9Fsaeh9chCzm8D2oeLOr8u2pB6bysbtY0+k+SRo7//XbqOrW8PiB3Xt/6d53tLsvB2/Z7JwwBXpjtW+prlnNXSnSM/1Vr7rkvXFlRHrbfd9fq4RbWNhh5D29026r39JSsH5mw3doZt9Atnrfv/vHR3UdrIeON1bUf/j+muxT8+3S9c/zbZbHV3Z/q9dJXyH6T7VeuadMNhT8u6k9KQWCZaxGc8Lx9L8mNVdZfW2oWTAqqbyPQHk3y2tfaNqlobGbDRBKdrIz5264ioednJz20edvvdFWcp383e217/3PpY33j5q6r6u3STj//PJE/cuWwl6S79SAZ0ElfVndLNe1jZuKPzZ9NddrbepuUwQ7o7qrV2RnWTlf/7dF+Ejk53nn9fVR095RehaY6Lrc5t87a27xw08Jw11cZaa59L8tqq+pN0cx3+aJKj0u1De9WW7cMpbPbD13a2u+dZ7621je7UWvvM2srWTTZ9dpJU1Vpn4ZeS3Lln7LTmWqfvhnb9bsjDQGv7xJZ3Gp7TOWtLC6qjtrte3y67uV20nZTDktuNnWFbuSDddb1/M+AX3zOSfDnJ8VX1sXSV48ljv/L9XLq5HB69Pt2qOmZCekNil8XQUU+bmeYznpc3J/mxdL8qnbRBzM+n+wVhbbLOtcbe3TeIv/tYXF8XJTmyqg5c/wt6VR2Wbqj5eOzdquqg9fv1aKj9XTP5l60+PpvkLlV1wNi+fpsJeZjmc5vnfrMd6fbx2fQvsyGxizouhqQ7ZJ/ctVpr7x81ln++qv6gtfb+HczO8aP//yHJk3u+5qnpGuJPSzJpJM1/TXfZyaadVuPlkO6uljOnuxNGowX+NMmfjiZZfkm6iaePzXUvaehTP+zkeaiv49f9PeScNZPWWquqD6ZrS91+iiSG1iOLOr8l/dqHQ7e/NnLllunq+LX4G6ebcH1ip+WizVjvvTHfaRs9f4vYj6Yb/dcndi7mUKdP266fZ1tjr323eFO+s088b4vYuZyz+ppDHbV0dlm7aMcoh+Wz235l6OO1SW6bDX6trapDx9eNhr6flu5OLL85Wv3qsbCVdCe1f+vpHzV0JjUuh8Qui2+M/r/lHNIa/BnP0avSNVSfPakBUt1toX833YiAU0arP5RuVMcTq+p2Y/E3TPLMdPvLmRnmjHR3Gfz5sfX/eULsW9J9cTthbP3TRusn3t68h7ema7w/aWz9pLtoTvO5zXO/2Y50+xhSZosu3z6GpDtkn9ztfjtdXf6indh4VR1YVS9LdzfAs9L9yt7XU5Kc11p7VWvtjeNLujno7lVV9++R1vpymGe622JUjt/VgdJaW5v0OJlcB/SpH3byPLSpsX3n7Rl+zuq7nUeO2jbj62+S7iY0SXcnyKGG1iOLOr/1bR8O3f7ayJejx9b/ana+3T1tvfeqdHPoPbeqHrdBzFqb+IMDYudpljp92nb9PNsae+27xavS3ZznOVV17KSAqvrhqvqlLOjcssA6alntaLtoF1EOS2Qvjgz7/9LNOXBKVT08yd+ku279+5M8It01vg+b8LpXppuo80lJ3ttau2Ds+TemazD+VVW9Od18Bz+T5NoJaQ2J3Q73raqf3eC5t7TWvrHBc/N0TrqJLZ9fVbdId23/Z1prH5wirWk/45m17lbP/zHJXyf5y6p6U5L3pLuz1FHpfrn7RpLHtta+NHrN/qp6RroG8Uer6tXpbvd8aLqJeO+Z5MWttU8OzM5L0+1Xr6xugtDz093N5EHpbik9HvtTSU4dffn5p3S34j4+XWPlpQO3vebkUR7+uKqOStfAfUiSHxnlYf2votN8bvPcb7Yj3T6GlNmiy7ePIekO2ScXbaZ6r7V2YVW9PsmTq+oh80q3R15vmuRu6SYRv2O6S2J+Jt3x2setkxye635hX+9N6e5od3y6Y2FD68thtGou6W6jmya5pKrOTFfvfTndHE/PSHfnsrdOeE2f+mHHzkNjttp3fjrJ7TLgnLVB2uPekuT3k9xqVLbnpbvc+vB0++tdk7x2NF/PUEPrkUWd39Zs1T4cuv2zR+tfVFW3Sjcq/MFJHrjB+9s24/Vea+19PV/3rap6TJK3JXlzVb0n3f73pXRt4B9Mty+upHu/fWPHLw/fap+c+3sbmbZdP8+2xm77brGp1tpVVfUf0l1S/Jaqeke6icwvT3eeeli6Tua/yBzPWWMWVUftZYs6hnbDd80hlAP9tF1wS8vuR9yNb0+crmH32XWPD0ryy+kqzG+OlguS/P9JHrXJNt412sbPTXjuwCT/Jd0vrFcn+Vy6Bs7dR6954TSxA8vgoaPXb3Yr9SG3j21J7rJJWT9lzp/hcel+gVm7zfFp2/0ZT7nPXacc0t1x6wVJPpzui8S30nVUvCzJbTdI7/7pTvpfStd4+Wq6iZyfMEMevz9dA+nK0fLWJD+QCbdnT9f4eEWSi0fbvzjdLYIPmbGc7pTu8pqvj/JwxmjdZUnOGosd/Llttd8san/ss19ukvbasTfxGBpYZgsr377vbUi6Q/bJgWU593pvq7TT1dkr6Y7T3ulO8d7WlpV0E5ifn+6OWsdMUQ5fHP1/ry22/cl0ddBNepZDG5ruDMfmWn4mHj8Dj7UbpvsC+Y/pvoRdPdoXX5PkyE3S3bJ+6HtcZMC5bRH7zii+9zmr776ebmTFqUk+kq4+2j8q43enu6TpgBn2gUH1SBZ0fluX/obtw2m2n+6L+F+n+3L+1SR/nu5yrYnvb2Be1/a3jY6Jtc93y3pvitfeJN1d1t492ifW2jrnpmsH323K2L775NTvbYsynbpdnzm1YWbMw1q5TNwnpo3tme+D0416/Lt0P0Bcm+4uk3+ZrjN+bQL8uZ2z1r1mkXXU2nH20AnPvScb1OvzKt8+ZZAp2kZDj6G+6c6hnCeW17KUg2Xnlhp9wEuhqs5K98vj7dp33w4dGGD0a/dlSf6otfb0nc7PXjCkzJQvwPbRPgSA5bPTcxdsm+pu4/vjSf5UQwf6G829MG5tDot3bmde9oohZaZ8AXaO9iEALKfr/ciwqnpAuqGOvzz6/+6ttc/uaKZgD6mqd6cbrv+hdB3oj0jyH5K8P8mPtXV3A6MzpMyUL8D20z4EgOW2FyfQH+oZ6e5YdFGSJ2vowGBvS3cMPS7dXA4XJ/m9JL+lo2ZDQ8pM+QJsP+1DAFhi1/uRYQAAAACwZmnmDAMAAAAAnWEAAAAALA2dYQAAAAAsDZ1hAAAAACwNnWEAAAAALA2dYQAAAAAsjf8D2ppT2JjUy9IAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 1728x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "seed_txt = 'heyaa HEY the YOU do get triggered HERE DHARA listen DOES do you TRIGGER at all CAPS hi HERE THERE'\n",
    "#predict(seed_txt,prediction_model,266,267,1) #266 267\n",
    "predict(seed_txt,prediction_model,156,157,2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5. Common words starting with 'a', 'o' and 'i' \n",
    "\n",
    "This cell of LSTM layer 2 is an indication of the fact that LSTM nodes can store information of more than one manner. Here, the cell gives activations for words having common sub-strings such as `an, is, and, in, at, I`. This cell can be explored for more input strings and more interesting observations can be found out. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Length of input text: 128\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAABMMAAAEECAYAAAAlPPOQAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4zLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvnQurowAAIABJREFUeJzt3Xm4LVddJ/zvL6OAiIQhBBIFmzAE5QFksEU0QkBoxEjbAiqaSDAPKHYrL/rGRhEQuxkUcUgjF8QEbXAAheAbXwhh0EaEpGkUiEBCmEJCQiCAGDPce1b/UXVk52QPtc/eZ7r787nPfu7ZVbVrraq1alXt3161qlprAQAAAIBVcMhOZwAAAAAAtotgGAAAAAArQzAMAAAAgJUhGAYAAADAyhAMAwAAAGBlCIYBAAAAsDIEwwAAAABYGYJhAAAAAKwMwTAAAAAAVsZhO52BTz3gpDZt/j0v+tjUzx96yOx43kX3uvvU+YcduTZ1/v7rtz5meOpVNXOZfz5w3dT5bzjmyIXz8btfvMPU+T971OcXTmM7/NoXv3Hq/F856ktT5192+fTPJ8mxd56+jmWYVf/3rx2YOv+bv+HomWm8/ZuOmjr/sZ/96tT5Rx12q5lp/NHR0+v3HX/uQTPXcdXLLpi5zMHgmL988dT5Vzz+F7cpJ1vrhI9cMnOZ6/ffOHX+rY+4xdT5H7jHXWem8UNXXD91/lMO++ap8x97h8/NTGPR4/jj33rvmWnMcqc/f8HU+Z/74V+euY77X/ypqfOfeLv7T51/xlFfnJnGweI+H/n41PnX7b9h6vxllPl2WDswvW0/5NCpl3hJZh8f9/zGY6fOP+fOt5yZxiz3+qeLp84/7uunXxslyXnHzb5uOBjM2ldDfOTexy8hJ7vfom1/knzsnveZOv/Qw6d/hxliN5yjHvzxy6bOf9+/m94OJMkdf+GhU+df9ZJ3T50/5LrkO466x9T5r77j7PKYdd3xoWumn2s/esL0PBxM7vzWfVPnX/6o0xdOY1bdS5Irv/yRjZNmf3FfETdefensE32Sw2//Lbtyn+kZBux6qxIIAwAAYOvteM8wAAAAAPaQAT1cdzPBMAAAAACGO7B/p3OwEMEwAAAAAAZrbfFxC3eSYBgAAAAAw60JhgEAAACwKvQMAwAAAGBlGEAfAAAAgJWhZxgAAAAAq6J5miQAAAAAK8MA+gAAAACsDLdJAgAAALAyDKAPAAAAwMrQMwwAAACAlWEAfQAAAABWhgH0AQAAAFgVre3tMcMO2ekMAAAAALCHtLVhrwGq6tFV9dGquqSqzpiwzBOq6qKq+nBVvXbR7OsZBgAAAMBwS7pNsqoOTXJmkkcmuSzJBVV1TmvtopFljk/yS0ke2lq7pqruuGi6eoYBAAAAMNzyeoY9OMklrbVLW2s3JPmTJCdvWOankpzZWrsmSVprVy2afcEwAAAAAIY7cOOgV1WdXlUXjrxO37CmuyT5zMj7y/ppo+6R5B5V9e6q+vuqevSi2XebJAAAAADDDbxNsrW2L8m+BVM7LMnxSU5McmySv6mqb2utfWmzK9QzDAAAAIDhlneb5GeTHDfy/th+2qjLkpzTWruxtfaJJB9LFxzbNMEwAAAAAIZbWxv2mu2CJMdX1d2q6ogkT0pyzoZl3piuV1iq6vbpbpu8dJHsu00SAAAAgOGW9DTJ1tr+qnpGkrckOTTJq1trH66q5ye5sLV2Tj/vUVV1UZIDSX6htfaFRdIVDAMAAABgsHbgxuWtq7Vzk5y7YdpzRv5uSZ7Zv5ZCMAwAAACA4YaNB7ZrCYYBAAAAMNySbpPcKYJhAAAAAAynZxgAAAAAK0PPMAAAAABWhp5hAAAAAKyM/ft3OgcLEQwDAAAAYDg9wwAAAABYGcYMAwAAAGBl6BkGAAAAwMrQMwwAAACAlaFnGAAAAAArw9MkAQAAAFgZre10DhYiGAYAAADAcMYMAwAAAGBlCIYBAAAAsDIMoA8AAADAyjhwYKdzsBDBMAAAAACGc5skAAAAACtDMAwAAACAlWHMMAAAAABWRVtrO52FhQiGAQAAADCc2yQBAAAAWBmeJgkAAADAytAzDAAAAICVIRgGAAAAwMpoBtAHAAAAYFXoGQYAAADAyljb2z3DDtnpDAAAAACwhxw4MOw1QFU9uqo+WlWXVNUZY+Y/rao+WFUfqKr/VVUnLJp9wTAAAAAABmtra4Nes1TVoUnOTPKYJCck+ZExwa7Xtta+rbV2vyQvTvLSRfMvGAYAAADAcGtt2Gu2Bye5pLV2aWvthiR/kuTk0QVaa18ZeXurJAvfo2nMMAAAAACGa8MG0K+q05OcPjJpX2tt38j7uyT5zMj7y5I8ZMx6fibJM5MckeTh82Z3I8EwAAAAAIYbOIB+H/jaN3PB2es5M8mZVfWjSX45ySmLrE8wDAAAAIDh9g8bHH+AzyY5buT9sf20Sf4kycsXTdSYYQAAAAAM19aGvWa7IMnxVXW3qjoiyZOSnDO6QFUdP/L2sUkuXjT7eoYBAAAAMNzA2yRnaa3tr6pnJHlLkkOTvLq19uGqen6SC1tr5yR5RlWdlOTGJNdkwVskE8EwAAAAAObQ1oYNoD9oXa2dm+TcDdOeM/L3f1laYj3BMAAAAACGW1LPsJ0iGAYAAADAcIJhAAAAAKyMA0t7muSOEAwDAAAAYLCmZxgAAAAAK0MwDAAAAICVscSnSe4EwTAAAAAAhtMzDAAAAICVIRgGAAAAwKpoB9wmCQAAAMCq0DMMAAAAgFXRBMMAAAAAWBmCYQAAAACsjL09ZJhgGAAAAADDtf17OxomGAYAAADAcHs7FiYYBgAAAMBwBtAHAAAAYHXoGQYAAADAqtAzDAAAAIDVoWcYAAAAAKui7d/pHCxGMAwAAACAwZqeYQAAAACsDMEwAAAAAFaFnmEAAAAArAzBMAAAAABWRjtQO52FhQiGAQAAADCYnmEAAAAArIy2pmcYAAAAACtir/cMO2SnMwAAAADA3tFaDXoNUVWPrqqPVtUlVXXGmPlHVtWf9vPfW1V3XTT/gmEAAAAADNbWhr1mqapDk5yZ5DFJTkjyI1V1wobFTktyTWvt7kl+K8mLFs2/YBgAAAAAg60dqEGvAR6c5JLW2qWttRuS/EmSkzcsc3KSs/u/X5/kEVW10KBlgmEAAAAADNbWatBrgLsk+czI+8v6aWOXaa3tT/LlJLdbJP+CYQAAAAAMNjQYVlWnV9WFI6/TdzrviadJAgAAADCH1oYu1/Yl2Tdlkc8mOW7k/bH9tHHLXFZVhyW5TZIvDM3rOHqGAQAAADDYEm+TvCDJ8VV1t6o6IsmTkpyzYZlzkpzS//2fkry9taHhuPH0DAMAAABgsNYWGr9+ZD1tf1U9I8lbkhya5NWttQ9X1fOTXNhaOyfJHyT5o6q6JMkX0wXMFiIYBgAAAMBgB4Y9KXKQ1tq5Sc7dMO05I39fl+SHl5ZgBMMAAAAAmMOyeobtFMEwAAAAAAYbOB7YriUYBgAAAMBgiw1fv/MEwwAAAAAYTM8wAAAAAFbGgbVDdjoLC9lUMKyqDktyyyTXttb2LzdLAAAAAOxWe/02ycGhvKp6UlX9VVVdmeT6JNckub6qruyn/8iW5RIAAACAXWGt1aDXbjWzZ1hV3TLJOUkenuTaJB9I8s4k1yX5uiR3SXJiksdU1VOTPK61du0W5RcAAACAHdR2caBriCG3ST4/yXcn+c9JXtlau37jAlV1ZJLTk/xmkucl+YVlZhIAAACA3WEVbpN8QpLfaa393rhAWJK01q5vrf1ukt9N8qRZK6yq06vqwqq68LVXf3a+HAMAAACwYw762yST3CHJPw1c30VJbj9rodbaviT7kuRTDzhpj8cTAQAAAFbHXn+a5JDcfzLJoweu7z/0ywMAAABwEGoDX7vVkGDYviQ/VFV/VlXfWVWHj86sqsOr6qFV9edJfrBfHgAAAICD0CrcJvmyJHdN8owkP5RkraquTnJ9kiPT3Ra5HlQ7s18eAAAAgIPQQf80ydZaS/JfquoVSX40yQOT3DnJLZN8Jck/JLkgyZ+21j60hXkFAAAAYIet7XQGFjSkZ1iSpLV2UZJf3sK8AAAAALDLtRzkPcMAAAAAYN3+g/02SQAAAABYp2cYAAAAACtjZcYMAwAAAAA9wwAAAABYGXqGAQAAALAyDugZBgAAAMCqWNvbsTDBMAAAAACGW9MzDAAAAIBV0XY6AwsSDAMAAABgMAPoAwAAALAy1sptkgAAAACsiAM7nYEFHbLTGQAAAABg71irYa9FVdVRVXVeVV3c/3/bMct8c1W9v6o+UFUfrqqnzVqvYBgAAAAAg62lBr2W4Iwk57fWjk9yfv9+oyuS/PvW2v2SPCTJGVV152krFQwDAAAAYLA28LUEJyc5u//77CQ/eLO8tHZDa+36/u2RGRDrEgwDAAAAYLDtuk0yydGttSv6vz+X5OhxC1XVcVX1j0k+k+RFrbXLp63UAPoAAAAADLY2cLmqOj3J6SOT9rXW9m1Y5m1J7jTm488efdNaa1U1tsNZa+0zSe7b3x75xqp6fWvtykn5EgwDAAAAYLADA3t99YGvfTOWOWnSvKq6sqqOaa1dUVXHJLlqxrour6oPJXlYktdPWs5tkgAAAAAMtjbwtQTnJDml//uUJG/auEBVHVtVt+j/vm2S70ry0WkrFQwDAAAAYLBtDIa9MMkjq+riJCf171NVD6yqV/XL3DvJe6vqH5K8K8lvtNY+OG2lbpMEAAAAYLC2nMHxZ6fT2heSPGLM9AuTPLX/+7wk951nvYJhAAAAAAy2pF5fO0YwDAAAAIDBDux0BhYkGAYAAADAYGvbdJvkVhEMAwAAAGAwt0kCAAAAsDIEwwAAAABYGW2nM7AgwTAAAAAABjNmGAAAAAArw9MkAQAAAFgZa3v8RknBMAAAAAAGM4A+AAAAACtjb/cLEwwDAAAAYA56hgEAAACwMvbX3u4bJhgGAAAAwGB7OxQmGAYAAADAHNwmCQAAAMDKWNvjfcMEwwAAAAAYbG+HwgTDAAAAAJiD2yQBAAAAWBkH9njfMMEwAAAAAAbTMwwAAACAldH0DAMAAABgVegZBgAAAMDKWNMzDAAAAIBVsbdDYYJhAAAAAMxh/x4PhwmGAQAAADCYAfQBAAAAWBkG0AcAAABgZez1nmGH7HQGAAAAANg71ga+FlVVR1XVeVV1cf//bScs901V9daq+qequqiq7jptvYJhAAAAAAx2oLVBryU4I8n5rbXjk5zfvx/nNUle0lq7d5IHJ7lq2koFwwAAAAAYbC1t0GsJTk5ydv/32Ul+cOMCVXVCksNaa+clSWvtq621a6etVDAMAAAAgMHawH9LcHRr7Yr+788lOXrMMvdI8qWq+ouq+j9V9ZKqOnTaSg2gDwAAAMBgQ8cDq6rTk5w+Mmlfa23fhmXeluROYz7+7NE3rbVWVeMibIcleViS+yf5dJI/TXJqkj+YlC/BMAAAAAAGG3oLZB/42jdjmZMmzauqK6vqmNbaFVV1TMaPBXZZkg+01i7tP/PGJN+RKcEwt0kCAAAAMNg23iZ5TpJT+r9PSfKmMctckOQbq+oO/fuHJ7lo2koFwwAAAAAYbBufJvnCJI+sqouTnNS/T1U9sKpelSSttQNJnpXk/Kr6YJJK8sppK3WbJAAAAACDLelJkTO11r6Q5BFjpl+Y5Kkj789Lct+h6xUMAwAAAGCwoQPo71aCYQAAAAAMtqTxwHaMYBgAAAAAg23XbZJbRTAMAAAAgMHacgbH3zGCYQAAAAAMdkDPMAAAAABWhdskAQAAAFgZbpMEAAAAYGXoGQYAAADAymiCYQAAAACsigNukwQAAABgVbhNEgAAAICVIRgGAAAAwMrwNEkAAAAAVoaeYQAAAACsDE+TBAAAAGBlHGhrO52FhQiGAQAAADCYMcMAAAAAWBnGDAMAAABgZRgzDAAAAICVseY2SQAAAABWhZ5hAAAAAKwMT5MEAAAAYGW4TRIAAACAleE2SQAAAABWhp5hAAAAAKwMPcMAAAAAWBkH2oGdzsJCBMMAAAAAGKzt8dskD9npDAAAAACwd6ylDXotqqqOqqrzquri/v/bTljuRVX1of71xFnrFQwDAAAAYLDW2qDXEpyR5PzW2vFJzu/f30RVPTbJA5LcL8lDkjyrqr5h2koFwwAAAAAYbK21Qa8lODnJ2f3fZyf5wTHLnJDkb1pr+1tr/5LkH5M8etpKBcMAAAAAGKwN/LcER7fWruj//lySo8cs8w9JHl1Vt6yq2yf53iTHTVupAfQBAAAAGOxAWxu0XFWdnuT0kUn7Wmv7NizztiR3GvPxZ4++aa21qrpZhK219taqelCSv0vy+STvSTL1cZeCYQAAAAAMNnQ8sD7wtW/GMidNmldVV1bVMa21K6rqmCRXTVjHryf59f4zr03ysWlpuk0SAAAAgMG2ccywc5Kc0v99SpI3bVygqg6tqtv1f983yX2TvHXaSvUMAwAAAGCwJT0pcogXJvmzqjotyaeSPCFJquqBSZ7WWntqksOT/G1VJclXkjy5tbZ/2koFwwAAAAAYbG05g+PP1Fr7QpJHjJl+YZKn9n9fl+6JkoMJhgEAAAAw2Db2DNsSgmEAAAAADDb0aZK7lWAYAAAAAIMtaXD8HSMYBgAAAMBgbpMEAAAAYGW0bRpAf6sIhgEAAAAw2F7vGXbITmcAYJY7/tyDdjoLAAAA9NZaG/TatVpru+qV5PRFl1l0vjSkIY29n8Zeyac0pCENaUhDGtKQhjSksZfzKQ2vvfja8QzcLEPJhYsus+h8aUhDGns/jb2ST2lIQxrSkIY0pCENaUhjL+dTGl578eU2SQAAAABWhmAYAAAAACtjNwbD9i1hmUXnS0Ma0tj7aSxjHdKQhjSkIQ1pSEMa0pDGTqWxjHVIY3vTYI+o/r5XAAAAADjo7caeYQAAAACwJQTDxqiqE6uqVdWpO52XvaSqPllV79zpfCxLVZ3a14MTdzovW+FgKy+2zqw2saruVlVvrKrP98udtb05HJunuev3Xmz7Hcd7w3bUrQHH6Z47jg9m27G/92KbxnB7of3fC3mcZDe026vEvmAnCIax8qrqflX13Kq6607nBfaos5J8T5IXJfnxJK/Y0dzsUlV126r61/5i78d3OC/rF53P2sl87HYrdn44K7v8ON6u8ph2fCwxD2dll+9vVstuOUctmo9Varf3yrbulXyyeg7b6QzALnC/JL+a5J1JPrmjOYE9pqqOTPKwJL/XWvuNnc7Pgv4myS2S3LhF6/+xJEcm+USSpyT5oy1Kh+VZ1vlhq+vWQnnYQ8fxbjhfL5yHPbS/WS275Ry1aD72Uru9aBq7oU0cYkg+d8N5khWjZ9hBoqoOrapb7nQ+GE6ZcZA4Okkl+eJOZ2RRrbW11tp1rbUDW5TEaUnekeRlSb6nqr5li9JZWbu1Xd2GurVoHg6a43iPsL/ZjXbLOWrufGxF278d7fZuODdstFPn0d24L2bZrdcczKG1tqtfSW6d5AVJ3pvk6iTXJ7kkyQuT3LJf5tQkLclJSZ6b5FP9cv+Y5Emz5o9J88R++VNHph2Z5L8m+XCS65J8Kcmbk9x/w2e/rk/jo0mu7Zf7YJKXDNze2yc5M8lnktzQ/39mktuNLDO6Pb+S5OPpouinDkxj4raMrPvhSZ7Vr/v6JB9Lckr/+eOS/FmSLyf5Sv/Zf5cu0v/OCWmur/cRSZ7Tl8G/9uX6Hf0y35PkfyX5lyRXJDlv1v5ewr58bp+vja+zBu6LofViU2WW+er/2HxuprwG5uH/m7NM/7kvg7Hb0S8/rixGy2RimQ4pr022Qevr3bK6O2ca1/XLvnHCuj7c79upbVEG1t1JbWI//awJZXXiktv2pbZHU/I0djuX8UrygH7dP9HX4xuSvGBMHdiS43jKtj5rjs+MK7Mr+/X8hwF1d/34+JUZ6dw1yRv67fxKkjcludvGbc3kdvWp0+r2kLrXL/fcMXW7JTlrgf196pj8L6XNmlV/J83PjOM4c7QVmzw2tr08Nnt8LCMP0/b3kH09T3lMKfNNX6tOKa//0af1vA3Lv6Wf/vMbpr8vyVUztnXWeWzjNcEX+rQen2261ly0bmebz3NT8rorzlGz8rEhLze7ps4Wt9vLOIZmpTGkTgxpjzKsPZm2L4d8fuZ+GFomm90XGXjNMKMctvz7t9fufO2F2yTvku7C9g1JXptkf7oT2S+mC95838iyL0pyq3Qn5CT5ySSvS/IH0+ZX1de11s6alIGqOjzJ/5/kO9N11f29JLdJ8lNJ3l1V391au7Bf/Mx0XXpfk+Sl6W5FPT7dQTxVVd0myd8luXuSVyd5f7+NT0/y8Kp6cGvtn0c+8htJDk/yynQH/0cHpDF1W5K8uF/0v6XrqvqKdI3O05OcVVWfS/L76U56v5/konTl8Y5++VlemOTQJL+d5Igk/0+St1bVT6Qrp33pyuwF6Rqbd27M44b9PWk7h+7Lv0hyTJLT+23+p34VH09yzxn74hNJnjdpX07I57xlNk/9H5vPdBd7d8zmymtaHv5TP39Imb4tyS8lOTbJHyf5+wnbMW5siMemC2pfkyll2qc/cT9U1SWttXcP3OZxtqPuDk3jpUlOrqont9b+eP3DVfUdSU5I8uxMaYvmbNOmeUWSDyT5rSR/me54Sr52HE2zcN1eQnu0nU5L8tUkb2it/UtV/VWSU6rqOa21tZHltuo4XoZxZXZKn6/XpWs3p9Xd/5nkCUmeX1WfGK2766rqdkn+Nl3Pmd9PV5celm5bbzUhX6Pt6lfT7bNvzYS63S8zpO5NOz8s01a1WUNNPI6X2FZMM7Qt2K7ymGYZeZi0vy/OjH2d5B9mLTOwPDZ9rZrJ5fW0dIGnh6e7HSpVdUSS70qy1k//rX76UUkemO6L5TlTtmPaeWzcdd4T+ry8Nt311ZZeay6wr3bjeW63nKOG5iMZf039/mxPO7HIMTTUrHPDxPZoE233xn15ycDPD9kPy2g3J+2Lq9Idz/NcM9zEdnz/Zhfb6WjcrFe6k9jhY6b/WrqT6IPztUjtp5LcZmSZ2/TTvjpj/heT3GJk+okZiUwn+fn+/fdtyMM3JPl0bvpL9ReTnLvJbf31Pp2f3jD9Z/rpv9a/X9/ej2bkV9OBaczalo/08/9PkiNG5t8lXePz4X7+T274/Mv66e+ckO56nt+/Yb0/0E+/MckDN+TxC0neM21/L7ovN+TtxAl5nrQv3j9HvdhUmWW++j8un/s3U14D8/Cmfh0fGVCmRyS5ZbqLpfeM244JaT8w3a+370kXKJpWpuv5mVRer9vkcbnldXeeNPrp6/viAxvW88q+zO+cKW1R5mjT+uknZvKvo3ft5z13zv26aN3edHs0JU8Tt3ORV7pfTq/JyK+gSU7u03rMhjqwJcfxlG2dp2fYzcpsJN8tyXcOqLtHZEM7sGF9L+4/92MTpr9zTNr/1q4OqdtD6t6YNE5csA7crG4NqN9ztVmz6u+0+ZlwHA/Zn0s4Pra9PBY5PpaRh3H7e2DdXUrbncWuVWeV140jx+N399P+KN2XxsP66a/qpz9/2nZMy2fGXOflpu3Rr49M35JrzWXU7WzzeW5CPnfFOWpIPjbkZew1dbaw3V7GMTQrjQF14nVjlj1xw3oHtRWT9uUcnx+0H4aUySb3xUUZeM0wJd0t//7ttXtfu37MsNbaDa21G5Okqg7rnzBy+3Q9TZLkISOLv7y19uWRz345XZT4VjPm3zbdATjJk9N94f/fVXX79Ve6k9x5Sb6rqtZ/+fhykvtU1bduYnMfn+Tz6SLco17RT3/8hukvb61dO2cas7bl+H65/9Fau2H9Q621z6brkvot6W6Lec2G9b5oYPovH11vuh4ASfLe9rVfKNbz+L4k95ixvyeZd19OM2lf3DPD68W6ucpszvo/Lp/7+9dmy2taHi7qF3n/rDLt13FtujI9fsZ2pE/ruHS/GF+V7mLoBzK9TL+9fz+pvI7PYraj7g5JI/naU8fWey+mqm6V5IlJ/rq1dnmmt0XztGlbYgl1exnt0Xb5j0m+McnZI9POTVdvn7Jh2S05jpdhXJkl+fqRRb595O+xdbfftvdl8vH4uHTBstdtmD5tkPHRdnVm3U5y6Bx1bztsVZu1DFveVszZFhzMhtTdH5+1zMDy2PS16oDyOixdz4yk6x1yVboeWrdO8qB++uP6/18zYzum5XPSdV7SDctw8sj7rbrWnGoPned2yzlqnnwkm/setCyLfN8bapFzw7xt98Z9OfTz27Efksn74m6Z/5pho+34/s0utRduk0xV/XS67tf3yc0H/b9tupNeMv7WnItG/p42f9rgjPdO1zXz81OWuX26+4t/Lt0vYB+sqkvTddN8c5I3t5t3793obkkubK3tH53YWttfVR9Ldx/9qI/NWN84Q7YlSS4dM+0L6crg4rZhcMPW2hVV9aUB6d9kva21a6oq6Z4YszGP9+rfj8vr+v6eZN59OTjPvfV9ca8J+ZuUz7nLbI76Py6fRyS5YYHyGpKH0eDNpDJdX8fD+jxvHDT4thuWvXWSv0oXyH5ka+2qqppVpusX2ZPK65unbeMA21F3h6SR1tonquryJHeuqsP7i+0npPuy8ap+sYltUeZr07bMgnV7Ge3Rdjkt3b6+rKruPjL9rUl+uL/AXLdlx/EyDCiz9byNrbu9a9KN8zfO3ZK8b+P5sm8DJm3raLs6qG5X1eMyYDu2yVa1WcuwLW3F0Hp1kBuyr++drvfMouWxyLXqrPJKuiDYW/r/35Gu1/M1/fv3JLlDv9zFM7Zj2nls7DVB71PpxqtKsqXXmjPtkfPcbjlHzcxHa+3qkemb+R60LAsdQwMtcm4Y2nav27gvh35+O/ZDMv34uGTOa4aNtuP7N7vUrg+GVdUzk/xmuobwd5Jcnu7L913S3aO+Hb3bKt1ggM+cssznk6S19qaqumu6wYS/J91YBKcl+duqOmlDr49FbSYqPWtbHpXkF5JMepJHbSLNUZPWOzp9PY9X9Pl55JjlZwXzlmnaU00G1YsRc5XZnPV/S56+MiUPT0w3FsYk/5afkXVcnu7i77GZsB3sxLX0AAAIs0lEQVRVdWiSP0039tX3t9Y+PGeW93LdHZLGuo+lux3yB9KNR3Jaks+le7DB1LYoc7RpW2VJdXvRMt1yfRD3e9PlddIF1JPTDTqbbNFxvAwTyuzEdOeMZPz5eDu2Z7RdHVK3fyRdj4WdvK4YtZvr95a3FbvkOm83GLKv3zxgmZnlsci16oDy+ny6cXZuma7n08+21taq6l1JHlFVv99v69XpjsWJ2zHjPLYZ23qtuRfOc7vlHDVHPl428n7Heuds0/e9RerEvG33xn056PPb+L13N50n9Qo7iOz6YFi6LuGfTHev+L9Ffavq0WOWvXe6cYNGnTBw/riI87qL0/2K9fYhUe7W2hfTDRL+x9X9FPXCdINlnpzkz6d89NIk96yqw0aj01V1WJJ7zMjjUFO3paqOnfH5G9Pd5nbo6C9AVXVMuq7Ny7Cex/cnSWvtbdMXH2uefdk2mc/9maNebNI89X+cf03ydQuW16Q8PGbg50fX8a50T385t1/HuO34nSSPSXfv/ltGps8q08+nu8DcScuou0N9uv//tKr6UJKHJnnR6L6Z0hZ9Oltfd2dZtG4n29MeLeon012s/VS+9mVi1AvS3f7x0inrWMZxvAw3K7OquuOS0/hkkrtX1SEb6sUdM2xbZ56vq+olGV73Nnt+OFjMdf2zSfO0BbuhPLYqD0Pq7tLKY4Fr1Vnl9Yl0430+Ll2PofP76eenu3Vp/drh0CHbMSWfl2XMNUHv2My+Zt6O8/VeOM/tlnPU0Hy8bMy8jbalnVjgGFpqNiZMH9RW9D0mN/35ZPB+2Koy2Z/FrhmS7fn+zS61F35tO5DuAPq3o7WvnGeMWfbp1T0RYn2526TrmnztjPlfSvclfZLXJLlTJkTHq+ro/v9Dq+omB15rbX3QvyQ5akoaSfLGdA3Pxt42P9VP/8sZnx9i6rakGxhxmmvTPbHjJzZM/38XzNeo9TzeZ9zM9f09wzz78qv9/7PKZ6OvZkC9WNA89X+cq9Md54uU16Q8PHYJ67jJdlTVzyX56SS/3Vp7+YZ1zCrT/z1HflJVh1fVvarqm+b53AzLqLtDrV9YfF/6J3elf3LugLbogmx93Z1l0bqdbE97dBPz1JuqOiTdgKsfbK29qrX2+o2vdONcfFu6bvqTLOM4XoablVmWfx3x5nRPndrYY+RZAz8/5Hw9T93b7PnhYDHo+mfk/Wba1aWWxxa17XPlYZOG7Ou5ymPCMoteq84qr0+kaxd+NcmnW2vrT4x7e5Ij0z1Zei1dL/GJ2zEgnx/M+GuCpPsSPOuaea7z9TbU7UnmOs/txXPUPPmoqgdNXVlnS9vtJRxDyzRpWxdtK2Z+fs79sFVlcm0Wu2ZItuf7N7vUXugZ9vok/z3JX1fVX6QL1vxoul9KNro6yXur6g/79z+Z5JvSPSb1KVPmP7VNHwjvt9N1n35JVT083Qn9K/1nH5HkunRde2+d5IqqOiddQ3BVupPH09ONlfDmGdv64iQ/nOTMqnpAv477p+tu+tF+/qJmbcusKPpX0g2W+Mqq+vZ0T7k5Mcm/T7d/l2E9j49Kkqp6Vsbv72nm2ZcXpLswe3Z1A0L/S8aPc7PRV/rPzqoXi5in/o/zmXS9pRYpr0l5OGLg50fXcWSSVNUvZsN2VDf45m+mu9Xv/VX15A3reEeml+lfp7tlcKi7pBtH8F2Z/gCNeSyj7s6r0l0EvKu1tj7+yqy26Jn9MltZd2dZtG4n29MebTRPvXlUkuPSBykneEOS56Z76tokyziOl2FcmT1jyWm8KF09+MOqenC6AXwflu7x7ldn9q/LQ87X89S9seeH1tp7F9nIPWTo9c+6zbSryy6PrWjb583DZgzZ148asMystnvRa9VZ5fXJdPvn3uluBUyStNYuqqrPpbsj473p2u5p2/H4Gfn8+SR3z02vCZ7YJ/eZzL5mnvd8vdV1e5J5z3N78Rw1Tz5OS/L3M9a31e32osfQMk36DjNv273RkM/POkZH98NWlcmX0wXaNnvNkGzP9292qb0QDHtJui97p6U7MD+XbjyhP8xNB8dPul8gHpbuUahHp7vn/MfSfWl/yqT5rbXXTstAa+3Gqnpsuh4rP57kef2sy9M9hWb9qSfXpuu++4h090x/fbqxCM5J8t9b94S3ael8uaoe2q//B9IF665M98TLX22t/fO0zw8xYFsuz9cGEx1nLd0J8aX52i9A70rXoJ4/6UObzOO70z12etL+nraOwfuytfbpqlqvHy9PcnifxjsHZHdIvVjEPPV/nP3p6uB7s/nympSHa5L8tznX8Uv9+5/Jzbfj9ul+WbxTxu+7s9PdCji2TJN8/8C8bJll1N1NeEe6QYlHLyJntUWfHtimbaVF63ayDe3Rgk7r//+LSQu01j5U3QCt056Yt4zjeBnGldmH0n2JWYrW2tVV9V3pAuNPSXch+45023pButtxpn1+yPn6/DHbMbbuTTk/rEQwbI7rn0UMbgt2Q3lsVR6G7OsllcdC16qZXV7XJflAukGn377hs29PFww6P915e9p2zMrnxWOu89av7X521jXzNp2vd/t5breco+bJx5My406AbWgnFj2GlmbStrbWTl2krRjY1gzeD1tYJmvpxirb1DVDn7ct//7N7lVdb8a9rapOTXdi+d7W2jvnnQ+wV1XVuel+fb1za23mSR/2oqq6XbpfeV/RWnvaTucHANidXDMw1F4YMwyAMap79Pj3JfljgTAOFlV1izGT18fYOW878wIA7F6uGVjEXrhNEoARVfWQdOOx/Od0j2n/zZ3NESzVuVX1qXRPeTsk3S0Y35/k79INdAsAkLhmYAGCYQB7z9PTjctxabpxDz+5s9mBpfqrdPX78UlukeSydAHf57XWDuxkxgCAXcU1A5t2UIwZBgAAAABDGDMMAAAAgJUhGAYAAADAyhAMAwAAAGBlCIYBAAAAsDIEwwAAAABYGYJhAAAAAKyM/wvRpQwOhxZU6AAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 1728x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "seed_txt = ' apples or end me other and amazon. me myself and i. And I am great in life. at. It feels awesome. and. And Art is interesting or'\n",
    "predict(seed_txt,prediction_model,386,387,2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus, we can observe how the LSTM is able to learn patterns. Feel free to play around and explore more! :)"
   ]
  }
 ],
 "metadata": {
  "accelerator": "TPU",
  "colab": {
   "collapsed_sections": [
    "N6ZDpd9XzFeN"
   ],
   "name": "Predict Shakespeare with Cloud TPUs and Keras",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}