notebooks/1_train_model.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Train a PyTorch Classifier\n",
    "In this notebook we will train a binary classifier to detect plagiarism using a [dataset created by researchers at the University of Sheffield](udacity/Solutions/1_Data_Exploration.ipynb) in which plagiarism has been simulated\n",
    "\n",
    "In the [Feature Engineering](udacity/Solutions/2_Plagiarism_Feature_Engineering.ipynb) stage I used two similarity metrics to compare the student answers to the Wikepedia source text.\n",
    "\n",
    "1. [Containment](https://youtu.be/FwmT_7fICn0) - How many n-grams do the answer and source text have in common. Useful for detecting copy-paste style plagiarism. \n",
    "2. [Longest Commmon Subsequence](https://youtu.be/yxXXwBKeYvU) - The longest sequence of words that appear left to right in both texts. The words don't have to be exactly continuous. Useful for detecting plagiarism that uses paraphrasing.\n",
    "\n",
    "Rather than go through feature engineering again we will load in the feature matrices produced earlier. We'll explore this data then build, validate and save a PyTorch binary classifier which will [later be used to detect plagiarism in Medium articles](notebooks/2_results.ipynb)\n",
    "\n",
    "Lets get started."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Initialise"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.insert(1, '../') \n",
    "\n",
    "# where to save torch model output\n",
    "MODEL_DIR = '../model'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Data\n",
    "Lets load in the training and test data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_keys(['train_x', 'train_y', 'test_x', 'test_y'])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pickle\n",
    "plagiarism_data = pickle.load(open('plagiarism_data.p', 'rb'))\n",
    "plagiarism_data.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "train_x has shape (70, 2)\n",
      "train_y has shape (70,)\n",
      "test_x has shape (25, 2)\n",
      "test_y has shape (25,)\n"
     ]
    }
   ],
   "source": [
    "# unpack data\n",
    "train_x, train_y, test_x, test_y = plagiarism_data.values()\n",
    "\n",
    "print(f\"train_x has shape {train_x.shape}\")\n",
    "print(f\"train_y has shape {train_y.shape}\")\n",
    "print(f\"test_x has shape {test_x.shape}\")\n",
    "print(f\"test_y has shape {test_y.shape}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is quite a small dataset with only 70 training examples and 25 test examples.\n",
    "There are two input features. \n",
    "1. Containment - stored in column 0\n",
    "2. Longest Common Subsequence - stored in column 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Visualise Training Data\n",
    "Visualising our training data may give an indication as to the relationship between the inputs and the targets, and guide us to the level of non-linearity present in the data.\n",
    "\n",
    "## Longest Common Sequence\n",
    "We'll plot longest common sequence on the horizontal axis and plagiarism (yes/no) on the y axis. We would expect that higher values be associated with plagiarism."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAFNCAYAAAAQOlZzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3de7gkdX3n8feHw0EHQQZkTMIwA14QBS+LGQVjHm/RiCZBYzCKSxQ1uMbH3HRJdJONeElMJEYTw0YxGqIbwcsmZDS4s1kDGi8gQwgkILOOeJlhMAxy8cKow/DdP6rO0NPT53SfS59DnXm/nqef01X1q19961d16ttV9euuVBWSJKl79lvqACRJ0tyYxCVJ6iiTuCRJHWUSlySpo0zikiR1lElckqSOMolL+6gkZyT57FLHsa9JcnaS/7lAdX03yYPnOO/adv6JhYhFS8Mkrt2SfC3J05c6jlEkqSQPHVLmx5K8L8lNSb6T5Pokb0xyv8WKc9ySrEzy/iTfbNfx/yX57aWOa1+X5PwkP2yT5K1J/jHJwxd6OVV1UFXdMMd5v9HOv2uh49LiMYlrWUpyGPAFYAXwhKo6GHgGsBJ4yFLGtsDeARwEPAI4BDgF+MqSRqQpb6uqg4AjgZuB85c2nHsk2X+pY9DCMIlrJEnOTLK5PatYn+SInmmV5JVJvpzktiTnJkk7bSLJ25PckuSrSV7dlt+/nX5Iz9nyjUneMnV5L8lDk3w6yR3t/B9ux3+mXfTV7ZnOCwaE/BrgO8DpVfU1gKraUlW/XlXXtPX8RJIr2vqvSPITPet0aRvL59tlfDzJA5L8TZJvt+WP7muDV7Vt8J0kb07ykCRfaMt/JMkB823PAR4HfKiqbququ6vq+qr6WFvP0b1t3bNev7znps272ja4PslP9Uw4I8kN7fp8Ncl/7pn2siRfauPbkOSonmnPaOu6I8mft9vwl9tpe1xK7o9xyP5wRpLPJvnjdrlfTfKsnroOS/JXSba10y/qmfazSf41ye3tNn30oMZM8u4kf9w37u+TvKZ9/9ttXN9Jsqm3vaZTVXcCHwIeOc0yP5rmSsodST6T5PieaQ9o972pfe4t6bkFkp4rUkl+JslVbdktSc4e0M4vT/IN4J8GtP3A7d2O/1ySd7Ttd0Oa/50z2uXcnOQlw9pBY1JVvnxRVQBfA54+YPzTgFuAxwL3Ad4FfKZnegGfoDnLXQtsB05up70SuI7mbORQ4P+25fdvp18EvAe4H/BA4IvAf2mnXQD8Ds2HzfsCP9m3zIfOsC6XAW+cYfphwG3ALwH7A6e1ww9op18KbKY5az+kXYf/Bzy9Lf8B4K/64lkP3B84HvgB8CngwT3zv2S+7TlgPf4SuBZ4KXBM37Sje9u6Z71+uX1/BnAX8JvAJPAC4I62be4HfBs4ti37Y8Dx7fvntm3ziLYtfhf4fDvt8Ha+U9s6f7NdxtQyzwb+53QxDtkfzgB2AmcCE8CvANuAtNP/AfgwzX42CTy5Hf9YmjPhE9v5XkKzr99nQHs+CdjSU+ehwA7gCODYdtoRPbE/ZJrtcj7wlvb9QTRJ/J+naYOXAQe3+8I7gX/tmXZh+zoQOK5d/mcH/R8ATwEeRfP/8mjgP4Dn9rXzB9q2XdHb9kO29xntNnxp235vAb4BnNvG/NM0H5gPWupj2L74WvIAfN17XkyfxN9Hc2lwavig9mB6dDtc7JlgPwK8rn3/T1MH4Xb46T0Hjh+hSXYreqafBlzSvv8AcB5w5ICYhiXxLwOvnGH6LwFf7Bv3BeCM9v2lwO/0THs78Mme4Z/rO9gW8MSe4SuB3+6b/53zbc8B67EC+G/t8nbSJNdntdN2H6R7yl/Knkl8dxJsx32xbZv7AbcDv9C7fdoynwRe3jO8H3AncBTwYuCynmkBtjJCEh9hfzgD2Nwz7cB23h+lSTp3A4cOaKO/AN7cN24TbZLvGx+aBPWkdvhM4J/a9w+l+TDwdGByyP/S+cD32zb8Js0HvIcMaoO++Va263QITcLcSZtY2+lvYZokPqCudwLv6GvnB0/T9jNt7zOAL/cMP6qd70d6xn0L+E8ztYmv8by8nK5RHAF8fWqgqr5L80+7uqfMN3ve30mTmKbm3dIzrff9UTRnTDe1l+lupzkLe2A7/bdoDqpfTHJtkpfNIuZv0RzYp7PHOrW+zp7r9B8973cMGD6IPY1afj7tuYeq2lFVf1BVPw48gCbhfzRNn4BR3FjtUbj1dZozze/RnJm/kmb7/EPu6Zh1FPCnPdvsVprttJq+7d3W3bvNZzJsf4CedqnmMjU0bbMGuLWqbpum3tdO1dnWu6aNdQ9tvBfSfHgAeBHwN+20zcBv0CThm5Nc2HsbZIA/rqqVVfWjVXVKVe3VVyHN7aY/TPKVJN+m+SANzRWNVTQJdrr/n/66TkxySZLtSe6g2XaH9xUbOP+Q7Q1778tU1bD/By0Ck7hGsY3mQAhAmt7dDwBuHGHem2gupU9Z0/N+C82Z1+HtwW5lVd2/qo4HqKpvVtWZVXUE8F+A/5EhPdJ7/F/g55NMt4/vsU6ttYy2TvM1n/acVlV9G/gDmrOqBwHfaycd2FPsR/tmW53scb99bRsfVbWhqp5B82HoeuC9bZktNFdXVva8VlTV52m29+5t3Nbdu82/N0M8M+4PQ2wBDkuycpppv98X74FVdcE0dV0AnNre5z8R+F9TE6rqQ1X1kzTbr4A/GiG2mbwIeA7N2f0hNGfH0Hwo2k5zGXu6/59+H6I5419TVYcA727r6VV7zTU1YfrtrXsxk7j6TSa5b89rf5qDw0uT/Kck96FJFJdX22FsiI8Av55kdXuA3f31p6q6Cfg/wNuT3D/Jfmk6gz0ZIMnzk0wdwG6jOQBNfR3mP2juN0/nT2juT/91ezCmjeFP2k5NFwMPS/KiJPun6Rx3HM296HGbT3vuIcl/T/K4JAckuS/w6zSXRTdV1XaaDwant2d8L2PvnvkPBH4tyWSS59Pc5744yY8kOaX9gPED4Lvc0/bvBl4/1QErTWe057fT/gE4Psnz2n3n19gzUf8r8KQ031E+BHj91IRh+8NM2nk/SfNB79B2fZ7UTn4v8Mr2TDVJ7td2Ajt4mrquokmgfwlsqKrb2/U8NsnT2m32fZqzz/l+Petgmvb9Fs2Hmz/oiWMX8LfA2UkObM+MXzykrlur6vtJHk/zAWEkQ7a37sVM4up3Mc3Baep1dlV9CvjvNGckN9EkgheOWN97aQ7M1wBXtfXfxT0HiBcDB9B0/LoN+Bj3XAZ/HHB5ku/SnGH8elV9tZ12Nk2Cvj3JL/YvtKpuBX6C5p7i5Um+Q9PR7A6a+6rfAn4WeC3NAfS3gJ+tqltGXK85m2d77lUd8Fc0HeW20XyN7mfaS/TQ3NM9i2Ydjwc+3zf/5cAx7fy/D5zats1+NG2zjeZy+ZOBV7Xx/x3NGeiF7SXgfwee1U67BXg+8IftMo8BPtez7v9I0/nsGpr7+P0fmmbaH4b5JZrtfT3NvevfaJe5sW2HP2/r3Exzn3cmF9CcHX+oZ9x92vW6heay/gNp+iPMxwdobmHcSLPOl/VNfzXNGfo3gQ+2cf1gmrpeBbyp3dd/j+YD9Kim3d66d5vqgSktijRfCXp3VfVfytYyleRSmo5cf7nUsXRdkj8CfrSqXrLUsejewTNxjVWSFUme3V6yXg28Afi7pY5L6oIkD0/y6PY2wOOBl+P/j3qYxDVuAd5IcxnzKuBLNJf6JA13MM198e/RXB5/O/D3SxqR7lW8nC5JUkd5Ji5JUkeZxCVJ6qjOPcnm8MMPr6OPPnqpw5AkaVFceeWVt1TVqkHTOpfEjz76aDZu3LjUYUiStCiS9P9E9G5eTpckqaNM4pIkdZRJXJKkjjKJS5LUUSZxSZI6yiQuSVJHmcQlSeook7gkSR1lEpckqaPGlsSTvD/JzUn+fZrpSfJnSTYnuSbJY8cViyRJy9E4f3b1fODPgQ9MM/1ZwDHt60TgL9q/nXbRVTdyzoZNbLt9B0esXMFZzzyW556wemz1P/Xhq7jk+u1zWt6gWIGR4p/Lek7Nc+PtO0hg6im4K1dMcvYpx+81/2yXMZvy891OvesykbCravff1T31TbecPdoCmHog8KEHTvKGnzt+pHlnin0+67dQ+/BC19Pb1qv76utd1soDJ6mCO3bsHMv/4GKZafufvf5abt+xE9hznxm1zlHbcuoY84mrb5rT8kZdv0HbDIYfi8Z9vB0W93yPwQthrM8TT3I08ImqeuSAae8BLq2qC9rhTcBTquqmmepct25d3Vt/O/2iq27k9X/7b+zYuWv3uBWTE7z1eY9akI06qP5+oy5vUF2TE4GCnXffs08Mqm8u6zks9sn9wjnPf8weB5LZLGM25ee7nUbdDr/w46v5X1feuNdyBo3vNTkRXvC4NSPP2x/7fNZvofbhcdbTXx8w4/ZYyP/BxTJd+/3Cj6/mw1/cssf/KDT7zDmnPmbGdVyItpzN8mYyyvGAwM5d0x+Lxn28nUvc44ohyZVVtW7QtKW8J74a2NIzvLUd11nnbNi018bdsXMX52zYNLb6+426vEF17dxVex0cBtU3l/UcFvvOu2uP+We7jNmUn+92GnU7XHD5loHLGTS+185dNat5+2Ofz/ot1D48znr66xu2PRbyf3CxTNd+F1y+dwKHZp8Zto4L0ZazWd5cY4HmeNCbwHtjnKmOcW/rhTwGL5SlfIpZBowbeFkgySuAVwCsXbt2nDHNy7bbd8xq/ELVP5dys4mpv+xc1nO2Mc12GbMZP9/tNGq5XdNc5Zpu/HzmnU/bjVJmtvvwuOuZbX0L9T+4WKaLd6Z9Z75ttdDbZhzzLtR+Plf3xv1tKc/EtwJreoaPBLYNKlhV51XVuqpat2rVwEeq3iscsXLFrMYvVP1zKTebmPrLzmU9ZxvTbJcxm/Hz3U6jlpvIoM+p04+fz7zzabtRysx2Hx53Pb3TF3p/vzeYLt6Z9p1R2mrY9PkcF2ZjrvMu1H4+Vwt9jFgIS5nE1wMvbnupnwTcMex++L3dWc88lhWTE3uMWzE5sbuTxjjq7zfq8gbVNTmR5l7UkPrmsp7DYp/cL3vMP9tlzKb8fLfTqNvhtBPXDFzOoPG9Jicyq3n7Y5/P+i3UPjzOevrrG7Y9FvJ/cLFM136nnbhmr/9RaPaZYeu4EG05m+XNNRZojgeTEzMfi8Z9vB1kIY/BC2Vsl9OTXAA8BTg8yVbgDcAkQFW9G7gYeDawGbgTeOm4YlksUx0ZxtVbclD9c+0ZOV2so8Q/l/XsnWeU3umzXcZsys93O/Wvy0y909cdddjA5UyNn6l3+rB5p4t9Puu3UPvwOOqZqUd177KWQ+/0mdpv3VGHzal3+lzacly90/vXby6908d9vB0l7mXfO30c7s290yVJWmj31t7pkiRpHkzikiR1lElckqSOMolLktRRJnFJkjrKJC5JUkeZxCVJ6iiTuCRJHWUSlySpo0zikiR1lElckqSOMolLktRRJnFJkjrKJC5JUkeZxCVJ6iiTuCRJHWUSlySpo0zikiR1lElckqSOMolLktRRJnFJkjrKJC5JUkeZxCVJ6iiTuCRJHWUSlySpo0zikiR1lElckqSOMolLktRRJnFJkjrKJC5JUkeZxCVJ6iiTuCRJHWUSlySpo0zikiR1lElckqSOMolLktRRJnFJkjrKJC5JUkeZxCVJ6iiTuCRJHTXWJJ7k5CSbkmxO8roB09cmuSTJVUmuSfLsccYjSdJyMrYknmQCOBd4FnAccFqS4/qK/S7wkao6AXgh8D/GFY8kScvNOM/EHw9srqobquqHwIXAc/rKFHD/9v0hwLYxxiNJ0rIyziS+GtjSM7y1HdfrbOD0JFuBi4FfHVRRklck2Zhk4/bt28cRqyRJnTPOJJ4B46pv+DTg/Ko6Eng28MEke8VUVedV1bqqWrdq1aoxhCpJUveMM4lvBdb0DB/J3pfLXw58BKCqvgDcFzh8jDFJkrRsjDOJXwEck+RBSQ6g6bi2vq/MN4CfAkjyCJok7vVySZJGMLYkXlV3Aa8GNgBfoumFfm2SNyU5pS32WuDMJFcDFwBnVFX/JXdJkjTA/uOsvKoupumw1jvu93reXwc8cZwxSJK0XPmLbZIkdZRJXJKkjjKJS5LUUSZxSZI6yiQuSVJHmcQlSeook7gkSR1lEpckqaNM4pIkdZRJXJKkjjKJS5LUUSZxSZI6yiQuSVJHmcQlSeook7gkSR1lEpckqaNM4pIkdZRJXJKkjjKJS5LUUSZxSZI6yiQuSVJHmcQlSeook7gkSR1lEpckqaNM4pIkdZRJXJKkjjKJS5LUUSZxSZI6yiQuSVJHmcQlSeook7gkSR1lEpckqaNM4pIkdZRJXJKkjjKJS5LUUSZxSZI6yiQuSVJHmcQlSeook7gkSR011iSe5OQkm5JsTvK6acr8YpLrklyb5EPjjEeSpOVk/3FVnGQCOBd4BrAVuCLJ+qq6rqfMMcDrgSdW1W1JHjiueCRJWm7GeSb+eGBzVd1QVT8ELgSe01fmTODcqroNoKpuHmM8kiQtK+NM4quBLT3DW9txvR4GPCzJ55JcluTkMcYjSdKyMtLl9CQPAn4VOLp3nqo6ZabZBoyrAcs/BngKcCTwz0keWVW39y3/FcArANauXTtKyJIkLXuj3hO/CHgf8HHg7hHn2Qqs6Rk+Etg2oMxlVbUT+GqSTTRJ/YreQlV1HnAewLp16/o/CEiStE8aNYl/v6r+bJZ1XwEc057F3wi8EHhRX5mLgNOA85McTnN5/YZZLkeSpH3SqEn8T5O8Afg/wA+mRlbVv0w3Q1XdleTVwAZgAnh/VV2b5E3Axqpa30776STXAbuAs6rqW3NcF0mS9impGn51OslbgV8CvsI9l9Orqp42xtgGWrduXW3cuHGxFytJ0pJIcmVVrRs0bdQz8Z8HHtx+VUySJN0LjPoVs6uBleMMRJIkzc6oZ+I/Alyf5Ar2vCc+01fMJEnSGI2axN8w1igkSdKsjZrENwI7quruJA8DHg58cnxhSZKkYUa9J/4Z4L5JVgOfAl4KnD+uoCRJ0nCjJvFU1Z3A84B3VdXPA8ePLyxJkjTMyEk8yROA/wz8QztuYjwhSZKkUYyaxH+D5rnff9f+6tqDgUvGF5YkSRpmpI5tVfVp4NM9wzcAvzauoCRJ0nAzJvEk76yq30jycfZ+jKjfE5ckaQkNOxP/YPv3j8cdiCRJmp0Zk3hVXZlkAjizqk5fpJgkSdIIhnZsq6pdwKokByxCPJIkaUSj/mLb14DPJVkPfG9qZFX9yTiCkiRJw42axLe1r/2Ag8cXjiRJGtWoXzF747gDkSRJszNSEk+yCvgtmp9ave/U+Kp62pjikiRJQ4z6i21/A1wPPAh4I8098ivGFJMkSRrBqEn8AVX1PmBnVX26ql4GnDTGuCRJ0hCjdmzb2f69KcnP0HRyO3I8IUmSpFGMmsTfkuQQ4LXAu4D7A785tqgkSdJQo/ZO/0T79g7gqeMLR5IkjWrYA1B+q6reluRdDH4Aik8ykyRpiQw7E/9S+3fjuAORJEmzM+wBKB9v//714oQjSZJGNeqPvQx6nvgdNGfo76mq7y90YJIkaWajfk/8BuC7wHvb17eB/wAe1g5LkqRFNupXzE6oqif1DH88yWeq6klJrh1HYJIkaWajnomvSrJ2aqB9f3g7+MMFj0qSJA016pn4a4HPJvkKEJrfUH9VkvsBdnqTJGkJjPpjLxcnOQZ4OE0Sv76nM9s7xxWcJEma3qhn4gDHAMfSPIr00Umoqg+MJyxJkjTMqF8xewPwFOA44GLgWcBnAZO4JElLZNSObacCPwV8s6peCjwGuM/YopIkSUONmsR3VNXdwF1J7g/cDDx4fGFJkqRhRr0nvjHJSpofdrmS5odfvji2qCRJ0lCj9k5/Vfv23Un+N3D/qrpmfGFJkqRhhj2K9LEzTauqf1n4kCRJ0iiGnYm/ved97wNQ0g4/baaZk5wM/CkwAfxlVf3hNOVOBT4KPK6qfOypJEkjGPYo0qcCJFkBvAr4SZrk/c/AX8w0b5IJ4FzgGcBW4Iok66vqur5yBwO/Blw+x3WQJGmfNGrv9L8GHgH8GfCu9v2w74g/HthcVTdU1Q+BC4HnDCj3ZuBtgI8zlSRpFkbtnX5sVT2mZ/iSJFcPmWc1sKVneCtwYm+BJCcAa6rqE0n+64ixSJIkRj8TvyrJSVMDSU4EPjdkngwYt/u+epL9gHfQPFxl5oqSVyTZmGTj9u3bRwxZkqTlbdQkfiLw+SRfS/I14AvAk5P8W5Lpvmq2FVjTM3wksK1n+GDgkcClbZ0nAeuTrOuvqKrOq6p1VbVu1apVI4YsSdLyNurl9JPnUPcVwDFJHgTcCLwQeNHUxKq6g3ueSU6SS4H/au90SZJGM+qPvXx9thVX1V1JXg1soPmK2fur6tokbwI2VtX62dYpSZLuMZtHkc5aVV1M89Sz3nG/N03Zp4wzFkmSlptR74lLkqR7GZO4JEkdZRKXJKmjTOKSJHWUSVySpI4yiUuS1FEmcUmSOsokLklSR5nEJUnqKJO4JEkdZRKXJKmjTOKSJHWUSVySpI4yiUuS1FEmcUmSOsokLklSR5nEJUnqKJO4JEkdZRKXJKmjTOKSJHWUSVySpI4yiUuS1FEmcUmSOsokLklSR5nEJUnqKJO4JEkdZRKXJKmjTOKSJHWUSVySpI4yiUuS1FEmcUmSOsokLklSR5nEJUnqKJO4JEkdZRKXJKmjTOKSJHWUSVySpI4yiUuS1FEmcUmSOsokLklSR401iSc5OcmmJJuTvG7A9NckuS7JNUk+leSoccYjSdJyMrYknmQCOBd4FnAccFqS4/qKXQWsq6pHAx8D3jaueCRJWm7GeSb+eGBzVd1QVT8ELgSe01ugqi6pqjvbwcuAI8cYjyRJy8o4k/hqYEvP8NZ23HReDnxy0IQkr0iyMcnG7du3L2CIkiR11ziTeAaMq4EFk9OBdcA5g6ZX1XlVta6q1q1atWoBQ5Qkqbv2H2PdW4E1PcNHAtv6CyV5OvA7wJOr6gdjjEeSpGVlnGfiVwDHJHlQkgOAFwLrewskOQF4D3BKVd08xlgkSVp2xpbEq+ou4NXABuBLwEeq6tokb0pySlvsHOAg4KNJ/jXJ+mmqkyRJfcZ5OZ2quhi4uG/c7/W8f/o4ly9J0nLmL7ZJktRRJnFJkjrKJC5JUkeZxCVJ6iiTuCRJHWUSlySpo0zikiR1lElckqSOMolLktRRJnFJkjrKJC5JUkeZxCVJ6iiTuCRJHWUSlySpo0zikiR1lElckqSOMolLktRRJnFJkjrKJC5JUkeZxCVJ6iiTuCRJHWUSlySpo0zikiR1lElckqSOMolLktRRJnFJkjrKJC5JUkeZxCVJ6iiTuCRJHWUSlySpo0zikiR1lElckqSOMolLktRRJnFJkjrKJC5JUkeZxCVJ6iiTuCRJHWUSlySpo0zikiR11FiTeJKTk2xKsjnJ6wZMv0+SD7fTL09y9DjjkSRpOdl/XBUnmQDOBZ4BbAWuSLK+qq7rKfZy4LaqemiSFwJ/BLxgXDHN5KKrbuScDZvYdvsOVh44yfd37mLHzrt3T1+5YpKffcyPccn129l2+w6OWLmCs555LM89YfW09UxXpr/8Gz9+LbfduXOkOA+YCAcesD937Ng5sP6LrrqRs9dfy+07mvr2C9xdMJGwq4rVPfP87kX/xgWXb2FXFRMJp524hrc891F7xddb36EHTvKGnzt+3uu9UPNK0r4sVTWeipMnAGdX1TPb4dcDVNVbe8psaMt8Icn+wDeBVTVDUOvWrauNGzcuaKwXXXUjr//bf2PHzl2zmm/F5ARvfd6jdiecQfX0l+lf7lkfu5qdu+a+DXrrv+iqGznro1ez8+6Z61sxOcFj1x7C575y617TTj9p7e5EPl19kxPhnFMfM+f17jWfeSVpX5DkyqpaN2jaOC+nrwa29AxvbccNLFNVdwF3AA8YY0wDnbNh06wTOMCOnbs4Z8OmGevpL9O/3Pkk8P76z9mwaWgCn5pnUAIHuODyezbZdPXt3FXzWu9e85lXkvZ140ziGTCuPyOMUoYkr0iyMcnG7du3L0hwvbbdvmNB5p2untmOn2sMC1Hfrp6LIDPVN5/1HqXMQrWNJC1n40ziW4E1PcNHAtumK9NeTj8E2OsUsarOq6p1VbVu1apVCx7oEStXLMi809Uz2/FzjWEh6pvIPZ+rZqpvPus9SpmFahtJWs7GmcSvAI5J8qAkBwAvBNb3lVkPvKR9fyrwTzPdDx+Xs555LCsmJ2Y934rJCc565rEz1tNfpn+5kxODLkbMLYaznnksk/sNr2/F5ARPfMhhA6edduI9n7umq29yIvNa717zmVeS9nVjS+LtPe5XAxuALwEfqaprk7wpySltsfcBD0iyGXgNsNfX0BbDc09YzVuf9yhWr1xBaHpgr5jcs2lWrpjk9JPW7i6zeuWKvTpf9dczqEz/cs859TEceuDkyLEeMBFWrpgcWP9zT1jNOc9/DCtX3FPfVA6eOsOemudvznwCp5+0dvf4iWSPTm3T1XfogZN7dGqby3r3t8Fc55Wkfd3YeqePyzh6p0uSdG+1VL3TJUnSGJnEJUnqKJO4JEkdZRKXJKmjTOKSJHWUSVySpI4yiUuS1FEmcUmSOsokLklSR5nEJUnqqM797GqS7cDXlzqOOTgcuGWpg9gH2e5Lw3ZfOrb90hhnux9VVQMf4dm5JN5VSTZO99u3Gh/bfWnY7kvHtl8aS9XuXk6XJKmjTOKSJHWUSXzxnLfUAeyjbPelYbsvHdt+aSxJu3tPXJKkjvJMXJKkjjKJL6AkJyfZlGRzktcNmP6aJNcluSbJp5IctRRxLkfD2r6n3KlJKom9dxfAKO2e5Bfb/f7aJB9a7BiXoxGONWuTXJLkqvZ48+yliHO5SfL+JDcn+fdppifJn7Xb5Zokjx17UFXlawFewATwFeDBwAHA1cBxfWWeChzYvv8V4MNLHfdyeI3S9m25g4HPAJcB65Y67q6/RtznjwGuAg5thx+41HF3/TViu58H/FxK+AMAAAOpSURBVEr7/jjga0sd93J4AU8CHgv8+zTTnw18EghwEnD5uGPyTHzhPB7YXFU3VNUPgQuB5/QWqKpLqurOdvAy4MhFjnG5Gtr2rTcDbwO+v5jBLWOjtPuZwLlVdRtAVd28yDEuR6O0ewH3b98fAmxbxPiWrar6DHDrDEWeA3ygGpcBK5P82DhjMokvnNXAlp7hre246byc5hOb5m9o2yc5AVhTVZ9YzMCWuVH2+YcBD0vyuSSXJTl50aJbvkZp97OB05NsBS4GfnVxQtvnzTYPzNv+46x8H5MB4wZ2/U9yOrAOePJYI9p3zNj2SfYD3gGcsVgB7SNG2ef3p7mk/hSaK0//nOSRVXX7mGNbzkZp99OA86vq7UmeAHywbfe7xx/ePm3kPLBQPBNfOFuBNT3DRzLgElaSpwO/A5xSVT9YpNiWu2FtfzDwSODSJF+juVe13s5t8zbKPr8V+Puq2llVXwU20SR1zd0o7f5y4CMAVfUF4L40v+2t8RopDywkk/jCuQI4JsmDkhwAvBBY31ugvaT7HpoE7r3BhTNj21fVHVV1eFUdXVVH0/RHOKWqNi5NuMvG0H0euIimQydJDqe5vH7Doka5/IzS7t8AfgogySNokvj2RY1y37QeeHHbS/0k4I6qummcC/Ry+gKpqruSvBrYQNN79P1VdW2SNwEbq2o9cA5wEPDRJADfqKpTlizoZWLEttcCG7HdNwA/neQ6YBdwVlV9a+mi7r4R2/21wHuT/CbN5dwzqu0+rblLcgHNraHD2/4GbwAmAarq3TT9D54NbAbuBF469pjcrpIkdZOX0yVJ6iiTuCRJHWUSlySpo0zikiR1lElckqSOMolLmtZMT95L8pIkX25fL1nKOKV9lV8xkzStJE+leRLTnUl+BXhKVb0gyWHARpqfDy7gSuDHpx50ImlxeCYuabckL27Puq9O8sEZnrz3TOAfq+rWNnH/I+DDTaRF5i+2SQIgyfE0v+v/xKq6pT3b7tX75L1Ff1qTpL2ZxCVNeRrwsaq6BaCqdj83ecCT9xb9aU2S9ubldElTwoBEPM2T9xb9aU2S9mbHNknA7svpfwc8oaq+1V5OPwr4GHByVX25p+xhNJ3ZHtuO+heajm23ImnReDldEgDtk7B+H/h0kl3AVTRn2Hs9ea+qbk3yZprHYgK8yQQuLT7PxCVJ6ijviUuS1FEmcUmSOsokLklSR5nEJUnqKJO4JEkdZRKXJKmjTOKSJHWUSVySpI76/zM9BUj1s+oEAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.figure(figsize = (8, 5))\n",
    "plt.scatter(train_x[:,1], train_y)\n",
    "plt.title('Longest Common Subsequence vs Plagiarism')\n",
    "plt.xlabel('c20')\n",
    "plt.ylabel('plagiarism')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Every answer with a longest common subsequence greater than 0.38 contains plagiarism.\n",
    "\n",
    "## Containment\n",
    "Next we'll plot containment (with an ngram choice of 20) on the horizontal axis and plagiarism (yes/no) on the y axis. We would expect that higher values be associated with plagiarism."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAFNCAYAAAAQOlZzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAdkElEQVR4nO3de5RlZX3m8e9Dg0gEwdBtotDQmACKRsekBHKZiLeImgGTMApqvIwjKxqSldGYkNGliGZCJIlGQ6IkOqgR8BJDGsUQE1GjI4RCFAJKbBDttlVaETSCXH/zx9mNx6Iuu6prV9Vb/f2sVavP3vvd7/m9Z1fXc/blnJ2qQpIktWeX5S5AkiQtjCEuSVKjDHFJkhpliEuS1ChDXJKkRhnikiQ1yhCXlkGSZyf5p+WuYyVKckqSv12kvv4zyUMWuO4B3fprFqMWaQiGuHYqSZ6VZLL74/y1JB9O8guL0O9ZSV7Xt31VvbuqfmlHn3exzXccO/g8t3fb4cYkH0ny0MV+nqras6quW+C6X+nWv2ux65IWiyGunUaSlwJvBP4P8GPAAcBfAscuZ107sddX1Z7A/sANwFnLW84PJNl1uWuQ+jDEtVNIsjdwKvCbVfWBqvpeVd1RVedX1cu7NrsneWOSrd3PG5Ps3i07KsmWJC9LckO3F/+CbtmJwLOB3+v2LM/v5p+c5Nok301ydZJfGavn+Uk+OTZdSX4jyReTfDvJGUky1vZTSd6Q5KYk1yX5uW7+5q6e5431dVa3/oe6574kyU+MLX9ot+d7Y5JrkjxjtnFMeR3fkuRPpsz7h+4NEkl+P8lXu+e9JskT5to2VXULcDbwiBm23fuSfD3JzUk+keThY8v2TXJ+ku8kuTTJ66Z5XX+ye/y0JJd3bTcnOWWs3Yau7QuTfAX46Ni8Xce2w3Xd2L6U5NkL2T7SYjLEtbP4WeC+wN/P0uYVwJHAfwEeBRwOvHJs+Y8DewP7AS8EzkjygKo6E3g33Z5lVf23rv21wH/t1nkN8LdJHjTL8/8y8JjuuZ8BPHls2RHAFcC+jALv3K7tTwLPAf4iyZ5j7U/onvMBwCbgDwGS3A/4SNfHA7t2f5nk4bOMY9zZwDPH3mA8APgl4NwkhwInAY+pqr26+q+fZbx0fezJ6M3D5TM0+TBwcFfvZ7oatzsD+B6jbfO87mcm3wOeC+wDPA14cZKnT2nzWOBh/PBrv/11exPwlG5sPwd8dqzJfLePtCgMce0s9gW+WVV3ztLm2cCpVXVDVW1jFIK/Prb8jm75HVV1AfCfwKEzdVZV76uqrVV1d1W9B/giozcGMzmtqm6qqq8AFzF6M7Hdl6rq/3bnZ98DrO9qua2q/gm4nVFgbPeBqvq3brzvHuvrl4Hru77urKrPAH8HHDdLXeP+FShGb07o1vt0VW0F7gJ2Bw5LsltVXV9V187S1+8muYnRm4w9gedP16iq3l5V362q24BTgEcl2TujC85+DXh1Vd1SVVcD75jpyarqY1V1Zbc9rgDOYRTa407pjtLcOk0XdwOPSLJHVX2tqq4aWzbf7SMtCkNcO4tvAWvnONf5YODLY9Nf7ubd08eUNwG3MAqfaSV5bpLPdodYb2J0uHjtLM//9Vn6/sbY41sBqmrqvPH2M/V1IHDE9pq6up7NaE92TjW6Y9K5jPbgAZ5Ft2dcVZuA32EUtDckOTfJg6frp/MnVbVPVf14VR0zXeAnWZPktO60xHf4wZ79WmAdsCuweWyVzcwgyRFJLkqyLcnNwG9w7+0x7fpV9T3gmd06X+tOVYxfiDff7SMtCkNcO4tPA98Hph4+HbeVUchtd0A3r48fuh1gkgOBv2Z0eHnfqtoH+HcgfQseyGbg4114bv/Zs6pe3C3vc1vDc4DjujEewWhPfrRy1dlV9QuMXscC/ngH630WowsPn8jotMSGbn6AbcCdjC6M2279LH2dDWwE1lfV3sBbuPf2mHH8VXVhVT0JeBDwBUbbV1pWhrh2ClV1M/AqRuexn57kR5LsluQpSV7fNTsHeGWSdUnWdu37fl75G8D455HvxygQtgFkdBHctBduLbEPAock+fVu/LsleUySh3XLp47jXqrqckbj+hvgwqq6CSDJoUken9HFgN9ntPe5ox/P2gu4jdGRlB9h9MmC7XXcBXwAOKXbng9ldM57tr5urKrvJzmc0RuEXpL8WJJjunPjtzE6leJHz7TsDHHtNKrqz4CXMrpYbRujvdKTgPO6Jq8DJhldoHQlo4uo+n5m+m2MzgXflOS87vzsnzI6AvAN4KeATy3SUBasqr7L6EK04xkdZfg6o73l3bsmPzSOWbo6h9He8dlj83YHTgO+2fX7QOB/72DJ72R0WuOrwNXAxVOWn8RoD/3rwLu6um6boa+XAKcm+S6jN2jvnUcduwAvY/Sa3cjoXPpL5rG+NIiMTnFJUvuS/DHw41XlR7q0U3BPXFKzus+8PzIjhzP66N9sHyOUVhW/lUhSy/ZidAj9wYy+9e1PgX9Y1oqkJeThdEmSGuXhdEmSGmWIS5LUqObOia9du7Y2bNiw3GVIkrQkLrvssm9W1brpljUX4hs2bGBycnK5y5AkaUkk+fJMyzycLklSowxxSZIaZYhLktQoQ1ySpEYZ4pIkNcoQlySpUYa4JEmNMsQlSWqUIS5JUqMGC/Ekb09yQ5J/n2F5krwpyaYkVyT56aFqkSRpNRrya1fPAv4CeOcMy58CHNz9HAH8Vffvsthw8oemnb/PHrvxy496EBd9YRtbb7qVB++zBy9/8qE8/dH7zdrfeZd/ldMvvGbWdfq0WUi/81kX4DXnX8W3b7njnvGecszDe/c53zp3pP65+l7oOkPWulhWUi0L0Xr9c3nleVdyziWbuauKNQknHLGe1z39pxbc30p6vVZSLSvRcr8+g95PPMkG4INV9Yhplr0V+FhVndNNXwMcVVVfm63PiYmJWuzvTp8pwGeyx25r+KNf/akZN9R5l3+VP/jAldx6x10zrtOnzUL6ncl06+62Jtx1d3H3lF+B3XYJp//3Ry34F3GmOn/tZ/bj7y776oLqn6vvhb5uwGC1LpYd2e4rQev1z+WV513J3178lXvNf86RBywoyFfS67WSalmJlur1SXJZVU1Mt2w5z4nvB2wem97SzVvxbr3jLk6/8JoZl59+4TU/tFGnW6dPm4X0O59177jr3gEOcMfd1avP+dZ5ziWbF1z/XH0v9HUbstbFsiPbfSVovf65nHPJ5nnNn8tKer1WUi0r0Up4fZbzLmaZZt60hwWSnAicCHDAAQcMWVNvW2+6dd7Lxuf3abOQfue77mK177PuXTMc9ZnPcy3V67YYtS6WHdnuK0Hr9c9lpt+VmebPZSW9XiuplpVoJbw+y7knvgVYPza9P7B1uoZVdWZVTVTVxLp1095Sdck9eJ895r1sfH6fNgvpd77rLlb7PuuuyXTv2+b3XIv9ug1Z62LZke2+ErRe/1xm+l2Zaf5cVtLrtZJqWYlWwuuznCG+EXhud5X6kcDNc50PXyn22G3NPReFTeflTz6UPXZbM+s6fdospN/5rLvbmrDLNH9ndtslvfqcb50nHLF+wfXP1fdCX7cha10sO7LdV4LW65/LCUesn9f8uayk12sl1bISrYTXZ7DD6UnOAY4C1ibZArwa2A2gqt4CXAA8FdgE3AK8YKha5nL9aU9b1KvTty+b7YrFPm0W0u9814XFvzp9tjonDvzRHbqSc6jXbYhaF8uObPeVoPX657L94rXFujp9Jb1eK6mWlWglvD6DXp0+hCGuTpckaaVaqVenS5KkHWCIS5LUKENckqRGGeKSJDXKEJckqVGGuCRJjTLEJUlqlCEuSVKjDHFJkhpliEuS1ChDXJKkRhnikiQ1yhCXJKlRhrgkSY0yxCVJapQhLklSowxxSZIaZYhLktQoQ1ySpEYZ4pIkNcoQlySpUYa4JEmNMsQlSWqUIS5JUqMMcUmSGmWIS5LUKENckqRGGeKSJDXKEJckqVGGuCRJjTLEJUlqlCEuSVKjDHFJkhpliEuS1ChDXJKkRhnikiQ1yhCXJKlRhrgkSY0yxCVJapQhLklSowYN8SRHJ7kmyaYkJ0+z/IAkFyW5PMkVSZ46ZD2SJK0mg4V4kjXAGcBTgMOAE5IcNqXZK4H3VtWjgeOBvxyqHkmSVpsh98QPBzZV1XVVdTtwLnDslDYF3L97vDewdcB6JElaVYYM8f2AzWPTW7p5404BnpNkC3AB8FvTdZTkxCSTSSa3bds2RK2SJDVnyBDPNPNqyvQJwFlVtT/wVOBdSe5VU1WdWVUTVTWxbt26AUqVJKk9Q4b4FmD92PT+3Ptw+QuB9wJU1aeB+wJrB6xJkqRVY8gQvxQ4OMlBSe7D6MK1jVPafAV4AkCShzEKcY+XS5LUw2AhXlV3AicBFwKfZ3QV+lVJTk1yTNfsZcCLknwOOAd4flVNPeQuSZKmseuQnVfVBYwuWBuf96qxx1cDPz9kDZIkrVZ+Y5skSY0yxCVJapQhLklSowxxSZIaZYhLktQoQ1ySpEYZ4pIkNcoQlySpUYa4JEmNMsQlSWqUIS5JUqMMcUmSGmWIS5LUKENckqRGGeKSJDXKEJckqVGGuCRJjTLEJUlqlCEuSVKjDHFJkhpliEuS1ChDXJKkRhnikiQ1yhCXJKlRhrgkSY0yxCVJapQhLklSowxxSZIaZYhLktQoQ1ySpEYZ4pIkNcoQlySpUYa4JEmNMsQlSWqUIS5JUqMMcUmSGmWIS5LUKENckqRGGeKSJDVq0BBPcnSSa5JsSnLyDG2ekeTqJFclOXvIeiRJWk12HarjJGuAM4AnAVuAS5NsrKqrx9ocDPwB8PNV9e0kDxyqHkmSVpsh98QPBzZV1XVVdTtwLnDslDYvAs6oqm8DVNUNA9YjSdKqMmSI7wdsHpve0s0bdwhwSJJPJbk4ydED1iNJ0qrS63B6koOA3wI2jK9TVcfMtto082qa5z8YOArYH/jXJI+oqpumPP+JwIkABxxwQJ+SJUla9fqeEz8PeBtwPnB3z3W2AOvHpvcHtk7T5uKqugP4UpJrGIX6peONqupM4EyAiYmJqW8EJEnaKfUN8e9X1Zvm2felwMHdXvxXgeOBZ01pcx5wAnBWkrWMDq9fN8/nkSRpp9Q3xP88yauBfwJu2z6zqj4z0wpVdWeSk4ALgTXA26vqqiSnApNVtbFb9ktJrgbuAl5eVd9a4FgkSdqppGruo9NJ/gj4deBafnA4varq8QPWNq2JiYmanJxc6qeVJGlZJLmsqiamW9Z3T/xXgId0HxWTJEkrQN+PmH0O2GfIQiRJ0vz03RP/MeALSS7lh8+Jz/YRM0mSNKC+If7qQauQJEnz1jfEJ4Fbq+ruJIcADwU+PFxZkiRpLn3PiX8CuG+S/YB/AV4AnDVUUZIkaW59QzxVdQvwq8Cbq+pXgIcPV5YkSZpL7xBP8rPAs4EPdfPWDFOSJEnqo2+I/w6j+37/ffetaw8BLhquLEmSNJdeF7ZV1ceBj49NXwf89lBFSZKkuc0a4kneWFW/k+R87n0bUT8nLknSMpprT/xd3b9/MnQhkiRpfmYN8aq6LMka4EVV9ZwlqkmSJPUw54VtVXUXsC7JfZagHkmS1FPfb2y7HvhUko3A97bPrKo/G6IoSZI0t74hvrX72QXYa7hyJElSX30/YvaaoQuRJEnz0yvEk6wDfo/RV63ed/v8qnr8QHVJkqQ59P3GtncDXwAOAl7D6Bz5pQPVJEmSeugb4vtW1duAO6rq41X1P4AjB6xLkiTNoe+FbXd0/34tydMYXeS2/zAlSZKkPvqG+OuS7A28DHgzcH/gfw1WlSRJmlPfq9M/2D28GXjccOVIkqS+5roByu9V1euTvJnpb4DincwkSVomc+2Jf777d3LoQiRJ0vzMdQOU87t/37E05UiSpL76ftnLdPcTv5nRHvpbq+r7i12YJEmaXd/PiV8H/Cfw193Pd4BvAId005IkaYn1/YjZo6vqF8emz0/yiar6xSRXDVGYJEmaXd898XVJDtg+0T1e203evuhVSZKkOfXdE38Z8Mkk1wJh9B3qL0lyP8CL3iRJWgZ9v+zlgiQHAw9lFOJfGLuY7Y1DFSdJkmbWd08c4GDgUEa3In1kEqrqncOUJUmS5tL3I2avBo4CDgMuAJ4CfBIwxCVJWiZ9L2w7DngC8PWqegHwKGD3waqSJElz6hvit1bV3cCdSe4P3AA8ZLiyJEnSXPqeE59Msg+jL3a5jNEXv/zbYFVJkqQ59b06/SXdw7ck+Ufg/lV1xXBlSZKkucx1K9Kfnm1ZVX1m8UuSJEl9zLUn/qdjj8dvgJJu+vGzrZzkaODPgTXA31TVaTO0Ow54H/CYqvK2p5Ik9TDXrUgfB5BkD+AlwC8wCu9/Bf5qtnWTrAHOAJ4EbAEuTbKxqq6e0m4v4LeBSxY4BkmSdkp9r05/B/Aw4E3Am7vHc31G/HBgU1VdV1W3A+cCx07T7rXA6wFvZypJ0jz0vTr90Kp61Nj0RUk+N8c6+wGbx6a3AEeMN0jyaGB9VX0wye/2rEWSJNF/T/zyJEdun0hyBPCpOdbJNPPuOa+eZBfgDYxurjJ7R8mJSSaTTG7btq1nyZIkrW59Q/wI4P8luT7J9cCngccmuTLJTB812wKsH5veH9g6Nr0X8AjgY12fRwIbk0xM7aiqzqyqiaqaWLduXc+SJUla3foeTj96AX1fChyc5CDgq8DxwLO2L6yqm/nBPclJ8jHgd706XZKkfvp+2cuX59txVd2Z5CTgQkYfMXt7VV2V5FRgsqo2zrdPSZL0A/O5Fem8VdUFjO56Nj7vVTO0PWrIWiRJWm36nhOXJEkrjCEuSVKjDHFJkhpliEuS1ChDXJKkRhnikiQ1yhCXJKlRhrgkSY0yxCVJapQhLklSowxxSZIaZYhLktQoQ1ySpEYZ4pIkNcoQlySpUYa4JEmNMsQlSWqUIS5JUqMMcUmSGmWIS5LUKENckqRGGeKSJDXKEJckqVGGuCRJjTLEJUlqlCEuSVKjDHFJkhpliEuS1ChDXJKkRhnikiQ1yhCXJKlRhrgkSY0yxCVJapQhLklSowxxSZIaZYhLktQoQ1ySpEYZ4pIkNcoQlySpUYa4JEmNGjTEkxyd5Jokm5KcPM3ylya5OskVSf4lyYFD1iNJ0moyWIgnWQOcATwFOAw4IclhU5pdDkxU1SOB9wOvH6oeSZJWmyH3xA8HNlXVdVV1O3AucOx4g6q6qKpu6SYvBvYfsB5JklaVIUN8P2Dz2PSWbt5MXgh8eLoFSU5MMplkctu2bYtYoiRJ7RoyxDPNvJq2YfIcYAI4fbrlVXVmVU1U1cS6desWsURJktq164B9bwHWj03vD2yd2ijJE4FXAI+tqtsGrEeSpFVlyD3xS4GDkxyU5D7A8cDG8QZJHg28FTimqm4YsBZJkladwUK8qu4ETgIuBD4PvLeqrkpyapJjumanA3sC70vy2SQbZ+hOkiRNMeThdKrqAuCCKfNeNfb4iUM+vyRJq5nf2CZJUqMMcUmSGmWIS5LUKENckqRGGeKSJDXKEJckqVGGuCRJjTLEJUlqlCEuSVKjDHFJkhpliEuS1ChDXJKkRhnikiQ1yhCXJKlRhrgkSY0yxCVJapQhLklSowxxSZIaZYhLktQoQ1ySpEYZ4pIkNcoQlySpUYa4JEmNMsQlSWqUIS5JUqMMcUmSGmWIS5LUKENckqRGGeKSJDXKEJckqVGGuCRJjTLEJUlqlCEuSVKjDHFJkhpliEuS1ChDXJKkRhnikiQ1yhCXJKlRhrgkSY0aNMSTHJ3kmiSbkpw8zfLdk7ynW35Jkg1D1iNJ0mqy61AdJ1kDnAE8CdgCXJpkY1VdPdbshcC3q+onkxwP/DHwzKFqms2Gkz90r3nXn/a0ZahEkqR+htwTPxzYVFXXVdXtwLnAsVPaHAu8o3v8fuAJSTJgTdOaLsBnmy9J0kowZIjvB2wem97SzZu2TVXdCdwM7DtgTZIkrRpDhvh0e9S1gDYkOTHJZJLJbdu2LUpxkiS1bsgQ3wKsH5veH9g6U5skuwJ7AzdO7aiqzqyqiaqaWLdu3UDlSpLUliFD/FLg4CQHJbkPcDywcUqbjcDzusfHAR+tqnvtiUuSpHsbLMS7c9wnARcCnwfeW1VXJTk1yTFds7cB+ybZBLwUuNfH0JbCTFehe3W6JGklS2s7vhMTEzU5ObncZUiStCSSXFZVE9Mt8xvbJElqlCEuSVKjDHFJkhpliEuS1ChDXJKkRhnikiQ1yhCXJKlRhrgkSY0yxCVJapQhLklSo5r72tUk24AvD/gUa4FvDtj/cnBMbXBMbVhtY1pt44HVN6YDq2raW3g2F+JDSzI503fUtsoxtcExtWG1jWm1jQdW55hm4uF0SZIaZYhLktQoQ/zezlzuAgbgmNrgmNqw2sa02sYDq3NM0/KcuCRJjXJPXJKkRu20IZ7k6CTXJNmU5ORplu+e5D3d8kuSbFj6Kuenx5h+MclnktyZ5LjlqHE+eoznpUmuTnJFkn9JcuBy1DkfPcb0G0muTPLZJJ9Mcthy1Dkfc41prN1xSSrJir9quMd2en6Sbd12+myS/7kcdc5Hn+2U5Bnd/6mrkpy91DXOV4/t9IaxbfQfSW5ajjoHVVU73Q+wBrgWeAhwH+BzwGFT2rwEeEv3+HjgPctd9yKMaQPwSOCdwHHLXfMijOdxwI90j1+8SrbR/cceHwP843LXvaNj6trtBXwCuBiYWO66F2E7PR/4i+WudZHHdDBwOfCAbvqBy133jo5pSvvfAt6+3HUv9s/Ouid+OLCpqq6rqtuBc4Fjp7Q5FnhH9/j9wBOSZAlrnK85x1RV11fVFcDdy1HgPPUZz0VVdUs3eTGw/xLXOF99xvSdscn7ASv9opU+/5cAXgu8Hvj+Uha3QH3H1JI+Y3oRcEZVfRugqm5Y4hrna77b6QTgnCWpbAntrCG+H7B5bHpLN2/aNlV1J3AzsO+SVLcwfcbUkvmO54XAhwetaMf1GlOS30xyLaPQ++0lqm2h5hxTkkcD66vqg0tZ2A7o+7v3a92pnPcnWb80pS1YnzEdAhyS5FNJLk5y9JJVtzC9/0Z0p9oOAj66BHUtqZ01xKfbo566x9OnzUrSWr1z6T2eJM8BJoDTB61ox/UaU1WdUVU/Afw+8MrBq9oxs44pyS7AG4CXLVlFO67Pdjof2FBVjwT+mR8ctVup+oxpV0aH1I9itNf6N0n2GbiuHTGfv3nHA++vqrsGrGdZ7KwhvgUYf+e8P7B1pjZJdgX2Bm5ckuoWps+YWtJrPEmeCLwCOKaqblui2hZqvtvoXODpg1a04+Ya017AI4CPJbkeOBLYuMIvbptzO1XVt8Z+3/4a+Jklqm2h+v7N+4equqOqvgRcwyjUV6r5/H86nlV4KB123hC/FDg4yUFJ7sNoA2+c0mYj8Lzu8XHAR6u7OmKF6jOmlsw5nu4w7VsZBfhKP38H/cY0/kfzacAXl7C+hZh1TFV1c1WtraoNVbWB0bULx1TV5PKU20uf7fSgscljgM8vYX0L0efvw3mMLhYlyVpGh9evW9Iq56fX37wkhwIPAD69xPUtjeW+sm65foCnAv/B6OrGV3TzTmX0BwbgvsD7gE3AvwEPWe6aF2FMj2H07vV7wLeAq5a75h0czz8D3wA+2/1sXO6aF2FMfw5c1Y3nIuDhy13zjo5pStuPscKvTu+5nf6o206f67bTQ5e75kUYU4A/A64GrgSOX+6ad3RM3fQpwGnLXetQP35jmyRJjdpZD6dLktQ8Q1ySpEYZ4pIkNcoQlySpUYa4JEmNMsQlzWi2O8UleV6SL3Y/z5utH0nD8CNmkmaU5HHAJVV1S5IXA0dV1TOT/Cgwyejrbgu4DPiZ6m6eIWlpuCcu6R5JntvtdX8uybtq5jvFPRn4SFXd2AX3R4CVfsMMadXZdbkLkLQyJHk4o++h//mq+ma3tz1u/E5xq+2ueVKTDHFJ2z2e0Z2evglQVffc8GfsTnGP3T5rmvU9NyctMQ+nS9ouTBPEM9wpbrXdNU9qkhe2SQLuOZz+98DPVtW3usPpBwLvB46uqi+Otf1RRhez/XQ36zOMLmxbybfrlVYdD6dLAqCqrkryh8DHk9wFXM5oD3tP4H1JAL5SVcdU1Y1JXsvodpAApxrg0tJzT1ySpEZ5TlySpEYZ4pIkNcoQlySpUYa4JEmNMsQlSWqUIS5JUqMMcUmSGmWIS5LUqP8Pv8h7Q6fQD7sAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (8, 5))\n",
    "plt.scatter(train_x[:,0], train_y)\n",
    "plt.title('Containmnet vs Plagiarism')\n",
    "plt.xlabel('c20')\n",
    "plt.ylabel('plagiarism')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Containmnet looks to be a much better predictor of plagiarism, with everything above 0 indicating plagiarism\n",
    "\n",
    "## Two Way Scatter Plot\n",
    "Now lets plot containmnet and longest common subsequence together in a scatter plot, and label the points as plagiarised (yes/no)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3deZxU5ZX/8c8XaGQRgQgmyqoZFGURsDUoKBgQcCOuqD+JRo0gUYka1zE/o8w4Y5TfmDCjJmhQYtzQuECikQlBlijRZrEBFTWI2rghSoMC0tDn98e93VQX1d23l1tL13m/XvWi6qlbt04X3ffUfZ7nnkdmhnPOufzVLNMBOOecyyxPBM45l+c8ETjnXJ7zROCcc3nOE4FzzuU5TwTOOZfnPBE456qQtETS+EzH4dLHE4FrNJK+SriVS9qW8Pj8GN7vQEm7JHVIaPu3atqejeH9J0l6O/z5PpE0R1LrBu7zcUk/T2r7RNLQhkW7x/t8ImlrQuwPSGpTx330lrSzMeNymeGJwDUaM9u74gZ8AJya0PZIDO/3HrAeODah+Tjg7RRtCxvzvSWNBn4OnBn+vH2AZxrzPRqDpGaSqvs7HxXGfhTB53V9+iJz2cQTgUsLSe0kbZe0T/j43yV9U/ENWtJUSXeE978l6VFJGyS9J+l6Sapm14sIDvRIagkcBtyb1HYUYSKQdLqk1yVtlvS+pH9NiHGepEuT4n5b0pgU73sksMjMVgKY2UYzm2Fm28LXtZU0TdKHkkolLZDUIrz9UdKnkjZJmi/pkPA1k4Ezgf8bflN/UtKTwH7A3LBtcrjtsZL+Ee5jmaQhCTEvkTRF0j+ArcABNf3fmNkHwFygb/JzkppLuk3SB2HMMyS1C59eCDRPOOsbWNP7uCxmZn7zW6PfgHXAyKS2V4GTw/sLgX8Cxyc8d2J4fxbwJLA38C/Ae8D51bzPROAf4f2hBAe0fkltm4Hm4eMRBN/emwGDgC+AMeFzFwALEvb9PeCTitcmve9IgoPsLcDRQMuk538XxvIdoDnBN+7mQAvgwvBnawXcByxJeN3jwM+T9vUJMDThcU9gYxhDM+AkYAPQMXx+CbAWOAQoAFqkiL9yn+H+3gZuTnj9+PD+T4A3gR7APsCfgPvD53oDOzP9u+a3ht/8jMCl0wJgmKS9gF4EB8Fh4TfM/sDfw+fOBG4ws6/M7F3gV8APa9jnIEltCQ62i4BVQM+Etr+b2S4AM5tnZqvNrNzMlhEknWHhvv4IDJTUPXz8Q+DRitcmMrO/AucSJIsXgc8l/TLsiikgSCpXmtknZrbLzBaF/+40s5nhz7YduA04SlKrOnyOFwJPm9lfw5/jeeANYFTCNg+Y2RozKzOz6vrxX5C0CXgJ+AswNcU25wN3mdn7ZrYZuBk4v4YzNJeDPBG4dFoADCc4eBYBfyM4CA8BVoYHmu8Q/F5+kPC694EuqXZoZm8RfKs/hqA7aJGZGfBaQlvl+ICkIWE3zQZJpcCPgE7hvr4GniY40BUA5wAPV/fDmNlsMzsZ6ACcDUwiSB77E3zzX5v8mrBraKqktZI2A28BAvat7n1S6AGMD7uFNoUH80KqdgF9GGE/J5pZBzPraWaTzeybFNscQPD5V3gfaA18qw7xuiznicCl0yLgcOBkgqSwgqB7YVT4GIIui3Kge8LruhMMCte03+EEB8N/JLUdQ9WB4lnAE0A3M2sPPERwIK4wExgPjAE+NbPltf1Q4bfyF8P36Qt8DOwEDkqx+UUEP+/xQHuCn5+EGFKVA05u+5DgG3+HhFtbM7u7htfU10cEiadCd2AbQfL10sVNhCcClzZmVgqsJvjmvMDMygnODH5MmAjCb6XPAP8RDrh+F/gp8Icadr0w3Mc7Fg7WAovDtgKCswPC7oy9gY1mtl3SMQTf5BO9FG5zO/D76t5Q0lmSzpbUQYFjCM5slphZWfjaX0v6djjgOlRSc6AdsJ2gj78t8O9Ju/6UPRNIcttM4GxJI8J9tw7vf6eGz6i+HgOuldQ97ML7d4LuMgM+Ixgs7l7jHlzW80Tg0m0BwbffZQmP2xIcuCtMDP99n6D76AGgpumnCwhm1iTu4zWCwc1/mNkOgPDgdRkwVdIWgumSTybuKNzmYYIB5UdreM8vCQZS/0kwGD0DuM3M/hg+Pzl8bjnBQf/fwp/7dwQDu58AK5NiBpgOHBl2+Twett0O3B62XWFmawnGUW4DPif4nH5KPH/P9xF0l70c/jxfANcAmNmXwJ3A0jC2ATG8v0sDBb/3zrkKkiYA48xsZKZjcS4d/IzAuQThTKNJBN/MncsLngicC0kaS9Dv/S7wVIbDcS5tvGvIOefynJ8ROOdcnmuR6QDqqlOnTtazZ89Mh+Gcczll6dKln5tZ51TP5Vwi6NmzJ0VFRZkOwznncoqk96t7zruGnHMuz3kicM65POeJwDnn8pwnAuecy3OeCJxzLs95InDOuTwXWyII1zb9TNKqap5XuKbru5KKJQ2KKxbnnMtpxbPg7r5wa4fg3+JZjbr7OM8IHiJY3KM6JxIsV9gLmEBQ7tY551yi4lkwZzKUfghY8O+cyY2aDGJLBGa2kKB2eXV+APzeAkuADpL2jyse51wDxPyN1NVg3hQo21a1rWxb0N5IMjlG0IWq66qWUM26tJImSCqSVLRhw4a0BOecC6XhG6mrQWlJ3drrIZOJQCnaUpZCNbPpZlZoZoWdO6csleGci0savpG6GrTvWrf2eshkIigBuiU87kqwULZzLpuk4Rupq8GIW6CgddW2gtZBeyPJZCKYDVwQzh4aDJSa2ccZjMc5l0oavpG6GvQfB6dOg/bdAAX/njotaG8ksVUflfQYMBzoJKkE+AVQAGBmvwGeB04iWA1qK3BRXLE45xpgxC3BmEBi91AjfyN1teg/rlEP/MliSwRmdl4tzxtweVzv75xrJBUHoHlTgu6g9l2DJBDjgcmlV86tR+Ccy4CYv5G6zPISE845l+c8ETjnXJ7zROCcc3nOE4FzzuU5TwTOOZfnPBE451ye80TgnGsavEJqvfl1BM653FdRIbXi6ueKCqng1z9E4GcEzrnc5xVSG8QTgXMu93mF1AbxRODSz/tyXWPzCqkN4onApZevduXikIaa/U2ZJwKXXt6X6+KQhpr9TZnPGnLpVW1f7odBV5GXOHb15RVS683PCDIhn/vIa+yz9a4i5zKh1kQg6duSfifphfDxYZIuiT+0Jirf+8hT9eUm864i59IqyhnBQ8CLwAHh47eBq+IKqMnL9z7y5L7c6vi0P+fSJkoi6GRms4ByADPbCeyKNaqmzOc7B8ng6lVw66YwIaTg0/6cS5soieBrSfsCBiBpMFAaa1RNmc93rsqn/TmXcVESwTXAbOC7kv4O/B64MtaomjI/8FXl0/6cy7hap4+a2TJJw4BDCDp115hZWeyRNVUVB7h5U4LuIJ8u6dP+nMuwWhOBpMuBR8xsdfi4o6TzzOze2KNrqvzA55zLIlG6hi41s00VD8zsS+DS+EJyzjmXTlESQTNJlfP8JDUHWsYXknPOuXSKUmLiRWCWpN8QzBy6DPhLrFHFrXiW99E751woSiK4AZgITCIYLJ4LPBBnULGKayUjTy7OuRwVZdZQOXBfeMt9NV3ZW98Dty+T55zLYVFqDQ2R9L+S3pa0VtJ7ktamI7hYxHFlb76XjXBNXz4XSswDUbqGfgdcDSylKZSWaN81LPiWor2+vGyEa8r8jLfJizJrqNTMXjCzz8xsY8Ut9sjiEseVvV42wjVlfsbb5EVJBPMl3SXpaEmDKm6xRxaXOEoa5FLZCD/Fd3XlZ7xNXpSuoe+F/xYmtBnw/cYPJ00a+8redJaNaMjsJD/Fd/URR3eqyyoys0zHUCeFhYVWVFSU6TAyI/lADsGZR9Qzmrv7VvMH3S0oC+1cKg39vXNZQdJSMytM9ZyvUJZLGtpX66f4rj68QmyTF6Vr6CHgQeDm8PHbwBMEs4lcOjX0QO6n+K6+vFBik+YrlOWShs5OyqVBbedc2sS6QpmkMZLWSHpX0o0pnu8uab6k5ZKKJZ1Up+jzTUMP5H6K75xLIUrXUPIKZZ2Bs2p7UVil9B7gBKAEeE3SbDN7I2GznwOzzOw+SYcBzwM96/Yj5JHGmJ3kp/jOuSRxrlB2FPCuma0FkPQ48AMgMREYsE94vz3wUR1ir79cLhDnB3KXa3L57y1PRFmh7IKkpkGSMLPf1/LSLkDiyGQJu69JqHArMFfSlUBbYGQ1MUwAJgB07969tpBr5nPpnUsf/3vLCVHGCI5MuB1LcPAeG+F1StGWfNHCecBDZtYVOAl4WNIeMZnZdDMrNLPCzp07R3jrGvjl8s6lj/+95YQoXUNXJj6W1B54OMK+S4BuCY+7smfXzyXAmPB9XpHUCugEfBZh//Xjc+mdSx//e8sJUc4Ikm0FekXY7jWgl6QDJbUEziUYdE70ATACQNKhQCtgQz1iis4LxDmXPv73lhOiXFk8R9Ls8PYnYA3wXG2vC683uIJgqcs3CWYHrZY0RVJF19LPgEslvQ48BvzI4q550WtU3dqdc/Xn167khCjTR6cm3N8JvG9mkc7rzOx5gimhiW23JNx/AxgSZV+N5p25dWt3ztVfOgsyunqLMkawIB2BpI33WTqXXj7lOetFmT66hT1n+0AwK8jMbJ8Uz2Uvr7fjnHNVRBksvhu4keC6gK7ADcC/m1m7nEsC4H2WzjmXJEoiGG1m95rZFjPbbGb3AWfGHVhsvN6Oc85VEWWweJek84HHCbqIziPXq496n6VzzlWKckbwf4BxwKfh7eywzTnnXBMQZdbQOoJicc4555qgKBeUHSxpnqRV4eP+kn4ef2jOOefSIUrX0P3ATUAZgJkVE5SLcM451wRESQRtzOzVpLadcQTjnHMu/aIkgs8lfZfdS1WeBXwca1TOOefSJsr00cuB6UBvSeuB94DxsUblnHMubaLMGloLjJTUFmhmZlviD8s551y6RJk19FNJ+xCsQ3C3pGWSvGazc841EVHGCC42s83AKGA/4CLgjlijcs45lzZREkHF2sMnAQ+a2eukXo/YOedcDoqSCJZKmkuQCF6U1A4ojzcs55xz6RJl1tAlwABgrZltlbQvQfeQc865JiDKrKFyST2B8ZIMWGxmz8QdmHPOufSIMmvoXuAyYCWwCpgo6Z64A3POOZceUbqGhgF9zaziyuKZBEnBOedcExBlsHgN0D3hcTegOJ5wnMszxbPg7r5wa4fg3+JZmY7I5aFqzwgkzSGoL9QeeFPSq+Hj7wEvpye8LFQ8C+ZNgdKSYMH7Ebf4ameufopnwZzJULYteFz6YfAY/HfKpVVNXUNT0xZFrvA/XNeY5k3Z/btUoWxb0O6/Ty6Nqk0EZrYgnYHkBP/DdY2ptKRu7c7FJMqsoS2SNoe37ZJ2SdqcjuCyjv/husbUvmvd2p2LSa2JwMzamdk+4a0VcCbwP/GHFqP6DtD5H65rTCNugYLWVdsKWgftzqVRlFlDVZjZs8D3Y4glPSr6+Us/BGx3P3+UZJANf7g+y6Tp6D8OTp0G7bsBCv49dZp3M7q0q/U6AklnJDxsBhQSrlaWkxrSz1/xfKZmDflgddPTf5z/37mMi3JB2akJ93cC64AfxBJNOjS0nz+Tf7g+WO2ci0GUWkNNq8Bc+65ht1CK9mzng9XOuRhUO0Yg6VJJvcL7kjRDUqmkYkmD0hdiI8uGfv768sFq51wMahos/ilBNxDAecDhwEHANcCv4w0rRrk8QJfLScw5l7Vq6hraaWZl4f1TgN+b2Ubgr5LujD+0GOXqAF2mB6udc01STYmgXNL+wJfACOD2hOdap36Ji12uJjHnXNaqKRHcAhQBzYHZZrYaQNIwYG0aYnOu8XnRQOf2UO0YgZn9CegBHGpmlyY8VQScE2XnksZIWiPpXUk3VrPNOElvSFot6dG6BO9cnTTkYkLnmrAaryw2s51m9mVS29dm9lVtO5bUHLgHOBE4DDhP0mFJ2/QCbgKGmFkf4Ko6xu9cdDVdh+FcHqtziYk6OAp418zWmtkO4HH2vBDtUuCeimRjZp/FGI/Ld34dhnMpxZkIugCJV26VhG2JDgYOlvR3SUskjYkxHpfv/DoM51KKUmICSV0IxgsqtzezhbW9LEVbco2iFkAvYDjQFVgkqa+ZbUp6/wnABIDu3bvjXL2MuKVqrSbw6zCcI1rRuV8SDA6/AewKmw2oLRGUEKxvXKEr8FGKbZaE1yu8J2kNQWJ4LXEjM5sOTAcoLCzM3YJ3LrP8OgznUopyRnAacIiZfVPHfb8G9JJ0ILAeOBf4P0nbPEtw1fJDkjoRdBX51FQXH78Ow7k9RBkjWAsU1HXHZrYTuAJ4EXgTmGVmqyVNkTQ23OxFYKOkN4D5wHXh1cvOOefSRGY197RI+iNBnaF5QOVZgZlNjje01AoLC62oqKjhO/ILi5xzeUTSUjMrTPVclK6h2eGt6fAFXpxzrlKU9QhmSmpJ0H8PsCahGF1u8gVenHOuUpRZQ8OBmQQlqQV0k3RhhOmj2csvLHLOuUpRuob+HzDKzNYASDoYeAw4Is7AYpXLq5Q551wjizJrqKAiCQCY2dvUYxZRVvEFXpxzrlKUM4IiSb8DHg4fnw8sjS+kNPALi5xzrlKURDAJuByYTDBGsBC4N86gYudTR51zrlKUWUPfAP8V3nKfTx11zrkqah0jkHSKpOWSvpC0WdIWSZvTEVwsvCa9c85VEaVr6FfAGcBKq+0y5FzgU0edc66KKLOGPgRWNYkkAF6T3jnnkkQ5I7geeF7SAqrWGsrNMYMRt8Bzl8OuHbvbmrf0qaPOubwVJRHcDnwFtAJaxhtOmiSf3DSRkx3nnKuPKIngW2Y2KvZI0mXeFChPKpVUXuZ1hpxzeSvKGMFfJTWdROCDxc45V0WURHA58BdJ25rE9NHWHevW7pxzTVyUC8rapSMQ55xzmRFljABJ/YGeidub2dMxxRSvbV/Wrd3tyUt0ONekRFmPYAbQH1gNlIfNBuRmImjdEbZ9kbrd1c5LdDjX5EQ5IxhsZofFHkm67Pqmbu2uKl/dzbkmJ8pg8SuSmk4i2PF13dpdVT7ryrkmJ8oZwUyCZPAJwZXFAszM+scamctOvrqbc01OlEQwA/ghsJLdYwS5q6AtlKX49l/QNv2x5KIRt1QdIwBf3c25HBclEXxgZrNjjyRdWuyVOhG02Cv9seQiX93NuSYnSiJ4S9KjwByqFp3LzVlD1U4f/QLu7usHtyj6j/PPxrkmJEoiaE2QABLLTDS96aOwu+/bp0Q65/JIlCuLL0pHIFnHp0Q2nF945lxOiLJUZVdJz0j6TNKnkv4oKXeniNTlCmKfEll/FReelX4I2O6zrOJZmY7MOZckynUEDwKzgQOALgRjBQ/GGVSs6jLN0adE1p+vDe1czoiSCDqb2YNmtjO8PQR0jjmu+Iy4JZjumKh5S2hWULXNp0Q2jF945lzOiJIIPpc0XlLz8DYe2Bh3YLHpPw5OnQbtuwEK/v3BPXDavVXbTp3m/dkN4WtDO5czoswauhj4H+BugtlCL4dtuau66Y9+4G88fuGZczkjyqyhD4CxaYjFNSV+4ZlzOaPaRCDpTmCtmf0mqf1q4DtmdkPcwcXGpzWmh1945lxOqGmM4BRgeor2XwMnxxNOGvi0Ruecq6KmRGBmtkeRubBN8YUUM5/W6JxzVdSUCLZK6pXcGLZtS7F9bvBpjc45V0VNieAW4AVJP5LUL7xdBPw5fK5WksZIWiPpXUk31rDdWZJMUmHdwq8Hn9bonHNVVJsIzOwF4DTgeOCh8DYcONPMnq9tx5KaA/cAJwKHAeelWulMUjtgMvCPOkdfH6kuKPNpjc65PFbj9FEzWwVcWM99HwW8a2ZrASQ9DvwAeCNpu38D7gSuref71I1Pa9yTz6JyLq9FuaCsvroAiWsalgDfS9xA0kCgm5n9SVK1iUDSBGACQPfu3RsemU9r3K1iFlXFALqX4HYu70QpMVFfqWYWWeWTUjOCq5V/VtuOzGy6mRWaWWHnzrlb5igr+Swq5/JelDLUQ6K0pVACdEt43BX4KOFxO6Av8JKkdcBgYHZaBozdbj6Lyrm8F+WM4L8jtiV7Degl6UBJLYFzCcpZA2BmpWbWycx6mllPYAkw1syKIuzbNRafReVc3qupxMTRwDFAZ0nXJDy1D9C8th2b2U5JVwAvhtvPMLPVkqYARWY2u+Y9xMgHR3fz4nDO5b2aBotbAnuH27RLaN8MnBVl5+E00+eT2lIeYcxseJR9NpgPjlbls6icy3sys5o3kHqY2fvh/WbA3ma2OR3BpVJYWGhFRQ3oPbq77+5F6hO17wZXr6r/fp1zLotJWmpmKcdgo4wR/KekfSS1JbgGYI2k6xo1wnTywVHnnKsiSiI4LDwDOI2gm6c78MNYo4qTD44651wVURJBgaQCgkTwnJmVkXA9QK557btXssOqjnWbwTdbt3gpaudcXoqSCH4LrAPaAgsl9SAYMM5Jfy7+iGZJeUyCvco2+boEzrm8VGsiMLNpZtbFzE6ywPsEhehy0uSyB2ihPZZZCPgVtc65PBTlyuJvS/qdpBfCx4dR/0J0GddRX9W8gQ8aO+fyTJSuoYcILgo7IHz8NnBVXAFlnA8aO+fyTJRE0MnMZgHlEFwxDOyKNaoYqfW3qn/Sr6h1zuWhKInga0n7Es4UkjQYKI01qjid+Et2qeoF1QbQ+ltw6jS/otY5l3eirEdwDUGxuO9K+jvQmYglJrLRs7uG8PWu4zlH82hOObtoxhM2grajfs1p/btkOjznnEu7KLOGlgHDCArQTQT6mFlx3IHFZcWfp3O6FtBC5UjQQuWcrgWs+PP0TIfmnHMZUesZgaQzkpoOllQKrDSzz+IJKz4/3vEH2jTbUaWtjXbw4x1/AG7LTFDOOZdBUbqGLgGOBuaHj4cTrB1wsKQpZvZwTLHF4oBmG+vU7pxzTV2UweJy4FAzO9PMzgQOA74hWH/4hjiDi0NpwX51anfOuaYuSiLoaWafJjz+DDjYzL4AyuIJKz53lp3DN0m1hr6x5txZdk6GInLOucyK0jW0SNKfgCfDx2cS1BxqC2yKLbKYfL1jJypQlTYhvt6xM0MROedcZkU5I7ic4OriAcBA4PfA5Wb2tZnlXM2h61vMoqWqHvRbaifXt/Bic865/FTrGYEFS5g9Fd5yXs4NFvv6ys65mEUpOneGpHcklUraLGmLpJwtQ72xeec6tWdUxfrKpR8Ctnt9ZS+V7ZxrRFG6hu4ExppZezPbx8zamdk+cQcWl9u3n81Wa1mlbau15PbtZ2coohrMmxKUxk7kpbKdc40sSiL41MzejD2SNHl21xBuLPsxJeWdKDdRUt6JG8t+zLO7hmQ6tD35+srOuTSIMmuoSNITwLME1w8AYGZPxxZVjJoJZpcPZfaOoXu0Z532XcNuoRTtzjnXSKKcEewDbAVGAaeGt1PiDCpOzas54FfXnlEjbglKYyfyUtnOuUYWZdbQRekIJF3Kqlulspr2jKqYHeSzhpxzMYpSdK4r8N/AEILS/YuBn5pZznZUj222mOtbzOIAfc5H1ok7d45jdvnQ2l+YCf3H+YHfORerKGMEDwKPAhXTasaHbSfEFVScxjZbzB0FD9BGQQXSrvqcOwoeCItlnJzR2JzLJmVlZZSUlLB9+/ZMh+LqoFWrVnTt2pWCgoLIr4mSCDqb2YMJjx+SlLNrFl9fMKsyCVRoox1cXzAL+M/MBOVcFiopKaFdu3b07NkTKRsH0VwyM2Pjxo2UlJRw4IEHRn5dlMHizyWNl9Q8vI0HsvQy3Np10ecp2w+opt25fLV9+3b23XdfTwI5RBL77rtvnc/ioiSCi4FxwCfAxwTLVObsAPKuan7k8kgfhXP5xZNA7qnP/1mUpSo/MLOxZtbZzPYzs9OA5FXLckZzSz09qLp255xr6ur7NfiaRo0ijdZbpzq1O+dyx0MPPcRHH31U59dcccUV9X7PY445ps6vOemkk9i0qWFV/NetW0ffvn0btI8K9U0EOXu+eOfOcSlrDd2506doOtcQzy5fz5A7/saBN/6ZIXf8jWeXr097DPVJBA318ssvR97WzCgvL+f555+nQ4cOMUZVN/VNBNaoUaTR7PKhKWsNZe11BM7lgGeXr+emp1eyftM2DFi/aRs3Pb2yQclg3bp1HHrooVx66aX06dOHUaNGsW1bUIRxxYoVDB48mP79+3P66afz5Zdf8tRTT1FUVMT555/PgAEDKretMHz4cK666iqOOeYY+vbty6uvvrrHe86ZM4fvfe97DBw4kJEjR/Lpp8HijBs2bOCEE05g0KBBTJw4kR49evD558EEk7333huAr776ihEjRjBo0CD69evHc889V+Xn+MlPfsKgQYP48MMP6dmzJ59//jlff/01J598Mocffjh9+/bliSeeAGDp0qUMGzaMI444gtGjR/Pxxx9Xth9++OEcffTR3HPPPfX+bJNVmwgqyk2nuG0BDmi0CDJgdvlQhu6YxkHfPMLQHdM8CTjXQHe9uIZtZbuqtG0r28VdL65p0H7feecdLr/8clavXk2HDh344x//CMAFF1zAL3/5S4qLi+nXrx+33XYbZ511FoWFhTzyyCOsWLGC1q1b77G/r7/+mpdffpl7772Xiy++eI/nhw4dypIlS1i+fDnnnnsud955JwC33XYb3//+91m2bBmnn346H3zwwR6vbdWqFc888wzLli1j/vz5/OxnPyNYzgXWrFnDBRdcwPLly+nRo0fla/7yl79wwAEH8Prrr7Nq1SrGjBlDWVkZV155JU899RRLly7l4osv5uabbwbgoosuYtq0abzyyisN+lyTVXsdgZm1a9R3cs41WR9t2lan9qgOPPBABgwYAMARRxzBunXrKC0tZdOmTQwbNgyACy+8kLPPjlZG/rzzzgPguOOOY/PmzXv005eUlHDOOefw8ccfs2PHjsq5+IsXL+aZZ54BYMyYMXTs2HGPfZsZ//qv/8rChQtp1qwZ69evrzyj6NGjB4MHD97jNf369ePaa6/lhhtu4JRTTuHYY49l1apVrFq1ihNOCMgLrfIAABDHSURBVK7Z3bVrF/vvv/8eP/cPf/hDXnjhhUg/d218zqRzrsEO6LDnt++a2qPaa6+9Ku83b96cnTsbtrZ48tTK5MdXXnklV1xxBStXruS3v/1t5Xz8im/2NXnkkUfYsGEDS5cuZcWKFXz729+ufH3btm1Tvubggw9m6dKl9OvXj5tuuokpU6ZgZvTp04cVK1awYsUKVq5cydy5czGz2KbzxpoIJI2RtEbSu5JuTPH8NZLekFQsaZ6kHqn245zLbteNPoTWBc2rtLUuaM51ow9p9Pdq3749HTt2ZNGiRQA8/PDDld+S27Vrx5YtW6p9bUUf/OLFi2nfvj3t27ev8nxpaSldunQBYObMmZXtQ4cOZdasYGXAuXPn8uWXX+6x79LSUvbbbz8KCgqYP38+77//fq0/y0cffUSbNm0YP3481157LcuWLeOQQw5hw4YNld0/ZWVllV1j7du3Z/HixUCQeBpLlBIT9SKpOXAPQU2iEuA1SbPN7I2EzZYDhWa2VdIkgtXQzokrJoDxg7vzhyV79u+NH9w9zrd1rkk7bWBw8LzrxTV8tGkbB3RozXWjD6lsb2wzZ87ksssuY+vWrRx00EE8+GBQBedHP/oRl112Ga1bt+aVV17ZY5ygY8eOHHPMMWzevJkZM2bssd9bb72Vs88+my5dujB48GDee+89AH7xi19w3nnn8cQTTzBs2DD2339/2rWr2nt+/vnnc+qpp1JYWMiAAQPo3bt3rT/HypUrue6662jWrBkFBQXcd999tGzZkqeeeorJkydTWlrKzp07ueqqq+jTpw8PPvggF198MW3atGH06NH1/fj2oCinPPXasXQ0cKuZjQ4f3wRgZikL+kgaCPyPmdW4VFhhYaEVFRU1KLZ7fnU7P/hiRmX10ee+dTGXX3Vzg/bpXFPz5ptvcuihh2Y6jEYzfPhwpk6dSmFhYZ1f+80339C8eXNatGjBK6+8wqRJk1ixYkUMUTaOVP93kpaaWcofPrYzAqALkLi8VgnwvRq2vwRIOfIhaQIwAaB794Z9c39t9m+5+Mtf0brZ7uqjF3/5K16b3Ykjx05s0L6dc03TBx98wLhx4ygvL6dly5bcf//9mQ6pUcWZCFKNaqQ8/QgL2RUCw1I9b2bTgekQnBE0JKhuy+6idVL10dbaQbdld4EnAuearJdeeqner+3VqxfLly9vvGCyTJyJoATolvC4K7DHJX+SRgI3A8PM7Jvk5xvbfrYhZYraz7z6qHMuP8U5a+g1oJekAyW1BM4FZiduEI4L/BYYa2afxRhLpc/UuZp2rzXknMtPsSUCM9sJXAG8CLwJzDKz1ZKmSBobbnYXsDfwpKQVkmZXs7tGs7D7pJS1hhZ2nxT3WzvnXFaKs2sIM3seeD6p7ZaE+yPjfP9Ufv3ZQBaX/Thcs3gjH9m+3LlzHEs/G4iXnXPO5aO8u7J4/aZtKWsNrW/gpfDOufQZPnw49Z1GPnv2bO644446vaaoqIjJkyfX6/0S3XrrrUydOrXB+2lssZ4ROOfySPEsmDcFSkugfVcYcQv0z77z7LFjxzJ27NjaNwzt3LmTwsLCel1/kCvy7ozAOReD4lkwZzKUfghY8O+cyUF7A6xbt47evXtz4YUX0r9/f8466yy2bt1aZZtJkyZRWFhInz59+MUvflHZ/vzzz9O7d2+GDh3K5MmTOeWUU4CqC9FUV3b61ltvZcKECYwaNYoLLriAl156qfL1CxYsYMCAAQwYMICBAwdWlrS46667OPLII+nfv3+VOG6//XYOOeQQRo4cyZo1DavGGhdPBM65hps3BcqSulfLtgXtDbRmzRomTJhAcXEx++yzD/fee2+V52+//XaKioooLi5mwYIFFBcXs337diZOnMgLL7zA4sWL2bBhQ8p9V1d2GoLa/8899xyPPvpolddMnTqVe+65hxUrVrBo0SJat27N3Llzeeedd3j11VdZsWIFS5cuZeHChSxdupTHH3+c5cuX8/TTT/Paa681+POIg3cNOecarrSkbu110K1bN4YMCSrPjB8/nmnTplV5ftasWUyfPp2dO3fy8ccf88Ybb1BeXs5BBx1UWUb6vPPOY/r06Xvsu7qy0xB0IaVa02DIkCFcc801nH/++Zxxxhl07dqVuXPnMnfuXAYOHAgEi9S88847bNmyhdNPP502bdpU7jMb+RmBc67h2netW3sd1FQ6+r333mPq1KnMmzeP4uJiTj75ZLZv3x6pbDRUX3Yaqi8dfeONN/LAAw+wbds2Bg8ezFtvvYWZcdNNN1WWjn733Xe55JJLUsafjfIuEXSppj56de3OuQhG3AIFSX9DBa2D9gb64IMPKksyP/bYYwwduntFwc2bN9O2bVvat2/Pp59+WrlQS+/evVm7di3r1q0DdpefTlZd2ema/POf/6Rfv37ccMMNFBYW8tZbbzF69GhmzJjBV199BcD69ev57LPPOO6443jmmWfYtm0bW7ZsYc6cOfX6DOKWd11D140+hJueXlllWb246qY7lzcqZgfFMGvo0EMPZebMmUycOJFevXoxadKkygPq4YcfzsCBA+nTpw8HHXRQZRdS69atuffeexkzZgydOnXiqKOOSrnv6spO1+RXv/oV8+fPp3nz5hx22GGceOKJ7LXXXrz55pscffTRQLCO8R/+8AcGDRrEOeecw4ABA+jRowfHHntsgz+POMRWhjoujVGG+tnl69NWN925XJUNZajXrVvHKaecwqpVq+r82q+++oq9994bM+Pyyy+nV69eXH311TFEmX2yqQx11jptYBc/8DvXxN1///3MnDmTHTt2MHDgQCZO9OrC1cnLROCcyw09e/as19kAwNVXX503ZwANlXeDxc656HKt69jV7//ME4FzLqVWrVqxceNGTwY5xMzYuHEjrVq1qtPrvGvIOZdS165dKSkpqfaqXJedWrVqRdeudbt+wxOBcy6lgoKCKlfauqbLu4accy7PeSJwzrk854nAOefyXM5dWSxpA/B+I+2uE/B5I+0rDh5fw3h89ZfNsYHHVx89zKxzqidyLhE0JklF1V1ynQ08vobx+Oovm2MDj6+xedeQc87lOU8EzjmX5/I9Eey5ZFF28fgaxuOrv2yODTy+RpXXYwTOOef8jMA55/KeJwLnnMtzeZEIJI2RtEbSu5JuTPH8XpKeCJ//h6SeWRTbcZKWSdop6ax0xVWH+K6R9IakYknzJPXIsvguk7RS0gpJiyUdlk3xJWx3liSTlNYphxE+vx9J2hB+fisk/Tib4gu3GRf+Dq6W9Gg2xSfp7oTP7m1Jm9IZX2Rm1qRvQHPgn8BBQEvgdeCwpG1+AvwmvH8u8EQWxdYT6A/8HjgrCz+744E24f1J6frs6hDfPgn3xwJ/yab4wu3aAQuBJUBhNsUH/Aj4n3T+3tUxvl7AcqBj+Hi/bIovafsrgRmZ+Cxru+XDGcFRwLtmttbMdgCPAz9I2uYHwMzw/lPACEnKhtjMbJ2ZFQPlaYinPvHNN7Ot4cMlQN3q38Yf3+aEh22BdM6OiPK7B/BvwJ3A9jTGBtHjy5Qo8V0K3GNmXwKY2WdZFl+i84DH0hJZHeVDIugCfJjwuCRsS7mNme0ESoF9syS2TKprfJcAL8QaUVWR4pN0uaR/EhxsJ6cpNogQn6SBQDcz+1Ma46oQ9f/3zLDr7ylJ3dITGhAtvoOBgyX9XdISSWPSFl0d/j7CLtMDgb+lIa46y4dEkOqbffK3wijbxCFT7xtV5PgkjQcKgbtijSjpbVO07RGfmd1jZt8FbgB+HntUu9UYn6RmwN3Az9IWUVVRPr85QE8z6w/8ld1nzukQJb4WBN1Dwwm+cT8gqUPMcVWoy9/vucBTZrYrxnjqLR8SQQmQ+C2mK/BRddtIagG0B77IktgyKVJ8kkYCNwNjzeybNMUGdf/8HgdOizWiqmqLrx3QF3hJ0jpgMDA7jQPGtX5+ZrYx4f/0fuCINMUG0f92nzOzMjN7D1hDkBiyJb4K55Kl3UJAXgwWtwDWEpyWVQzo9Ena5nKqDhbPypbYErZ9iPQPFkf57AYSDJj1ytL/214J908FirIpvqTtXyK9g8VRPr/9E+6fDizJsvjGADPD+50Iumr2zZb4wu0OAdYRXsCbjbeMB5Cm/7CTgLfDA9bNYdsUgm+wAK2AJ4F3gVeBg7IotiMJvnl8DWwEVmfZZ/dX4FNgRXibnWXx/RpYHcY2v6YDcSbiS9o2rYkg4uf3n+Hn93r4+fXOsvgE/BfwBrASODeb4gsf3wrckc646nrzEhPOOZfn8mGMwDnnXA08ETjnXJ7zROCcc3nOE4FzzuU5TwTOOZfnPBG4jJD0HUmPS/pnWDnyeUkH13NfV0lqE2G7B9JdfTThvYdLOqaG50+TdEuM719tlVhJF0p6J7xdmND+V0kd44rJZQ+fPurSLizo9zLBhUC/CdsGAO3MbFE99reOYP79540aaCOSdCvwlZlNreb5lwnmnsfyM0g6HviHmW2VNAkYbmbnSPoWUERQHsSApcARZvZlmBS6mtntccTksoefEbhMOB4oq0gCAGa2wswWKXCXpFXhOgLnQOU36pfCwmdvSXok3HYycAAwX9L8cNv7JBWF9elvq3iP8PWF4f2vJN0u6fWwWNm3w/aHwtfPl7RW0jBJMyS9KemhhH2NkvSKgrUinpS0d9i+TtJtYftKSb0VrG9xGXB1WJf+2MQPIzwT+qYiCYQxTJP0chhDg9ehsOqrxI4G/tfMvrCgguf/ElytCzCboH6Pa+I8EbhM6EvwzTOVM4ABwOHASOAuSfuHzw0ErgIOI6gBP8TMphHUdznezI4Pt7vZzAoJ1nEYJql/ivdpS1Au4XCCtQAuTXiuI/B94GqComt3A32AfpIGSOpEULxupJkNIvhGfU3C6z8P2+8DrjWzdcBvgLvNbECKs54hwLKktv2BocApwB2pPihJixIWPUm8jUy1fYLEKrHVVtAME8NektJRiddlUItMB+BckqHAYxZUafxU0gKCMhubgVfNrARA0gqCRXsWp9jHOEkTCH6/9ydIHMVJ2+wAKko/LwVOSHhujpmZpJXAp2a2MnzP1eF7dg33+fdw2YqWwCsJr386Yb9nRPiZ9wc2JLU9a2blwBsVZyvJzOzYVO01SagSO6yiKdWuE+5/RnDGtbGu7+VyhycClwmrgeq6O2paECixsukuUvz+SjoQuBY4MuznfoigllSyMts9QJa8r4r3KU96z/Jwu10E3SnVdZtUvCZljClsI6h4m2ofUM1nImkRQQXTZNea2V9TbF9RJXaY7a4oWkJQwrlCV4KaRxVahfG5Jsy7hlwm/I2gy6GyO0bSkZKGEXTTnCOpuaTOwHEEhQBrsoXdB8R9CAr0lYbfpE9s9OiDPvYhkv4ljL1NhBlPiTEmexP4l7oGYWbHhl1NybdUSWAg8FuCAenEVbxeBEZJ6hjOEBoVtlUM6n+HoHKma8I8Ebi0C7+Jnw6coGD66GqCCo0fAc8QdOO8TpAwrjezT2rZ5XTgBUnzzex1gjVsVwMzgL/HEP8GgrV8H5NUTJAYetfysjnA6akGiwmS38DwwBuXu4C9gSfDGGYDmNkXBEtlvhbepoRtEKw9sMSCVftcE+bTR53LApJ+TTA2sce3+UwJY5ptZvMyHYuLl58ROJcd/gOo9aK4NFvlSSA/+BmBc87lOT8jcM65POeJwDnn8pwnAuecy3OeCJxzLs95InDOuTz3/wFMkep3HfCATQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import numpy as np\n",
    "\n",
    "label_dict = {0:'not plagiarised', 1:'plagiarised'}\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "for is_plagiarism in np.unique(train_y):\n",
    "    ix = np.where(train_y == is_plagiarism)\n",
    "    ax.scatter(train_x[ix,0], train_x[ix,1], label = label_dict[is_plagiarism])\n",
    "ax.legend(loc = 4)\n",
    "plt.title('Two Way Scatter Plot')\n",
    "plt.xlabel('Containment (n = 20)')\n",
    "plt.ylabel('Longest Common Subsequence')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It looks like these features complement each other nicely. Longest Common Subsequence allows us to further partition the cases where Containment is zero. We could probably produce a reasonable accurate set of classificaton rules from this chart alone, but instead we'll train a neural network to do this job for us."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Training the Model\n",
    "We are going to train a neural network to classify the students answers into plagiarised vs non-plagiarised. Although we are training a model on a small dataset, eventually we will be requesting predictions for hundreds of thousands of datapoints. This is because we will be making pairwise comparisons for all downloaded Medium articles. So before we get started with the modelling we need to choose an environment to work in."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Amazon Sagemaker vs Vanilla PyTorch\n",
    "\n",
    "As part of the Udacity ML Engineer Nano Degree they provided some starter code for building and deploying models using Amazon SageMaker. SageMaker allows you to deploy a range of models (sklearn, PyTorch etc) as a Rest API so that it can be consumed via a web app or similar. There is an overhead that comes with this - training model can be a bit tedious since every time you train a model seperate infrastructure needs to be provisionsed. Additionally when you come to deploy the model through the API endpoint there are costs associated with this. Our usecase is simply exploratory in nature - we are not looking to deploy our model. Therefore Amazon Sagemaker is overkill. Instead we'll write a basic classifier ourselves using vanilla PyTorch.\n",
    "\n",
    "## PyTorch Data Loaders\n",
    "We tend to train pytorch models in batches. In order to do this we need to create a data loader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch.utils.data\n",
    "\n",
    "# Turn the numpy arrays into tensors\n",
    "train_x = torch.from_numpy(train_x).float()\n",
    "train_y = torch.from_numpy(train_y).float().squeeze()\n",
    "\n",
    "# build the torch dataset\n",
    "train_ds = torch.utils.data.TensorDataset(train_x, train_y)\n",
    "\n",
    "# build the data loaders\n",
    "train_dl = torch.utils.data.DataLoader(train_ds, batch_size = 10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we specify our train function. Usually to avoid overfitting we would use a validaiton sample and stop training once the validation error no longer decreases. However in our case we only have 70 training examples and therefore we'll train until the training error reaches a plateu. There is a risk here of overfitting - and we'll guard against it in two ways\n",
    "1. Plot the classification boundary to check what the classifier is going\n",
    "2. Limit the number of peeks at the test data, and use that as an independent check of classifier performance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train(model, train_loader, epochs, optimizer, criterion, device):\n",
    "    \n",
    "    for epoch in range(1, epochs + 1):\n",
    "        # monitor training loss\n",
    "        train_loss_total = 0\n",
    "        \n",
    "        ######################\n",
    "        # train the model    #\n",
    "        ######################\n",
    "        model.train() # prep model for training\n",
    "        \n",
    "        for data, target in train_loader:\n",
    "            target = target.contiguous().view(-1, 1)\n",
    "            \n",
    "            data, target = data.to(device), target.to(device)\n",
    "            optimizer.zero_grad() # zero accumulated gradients\n",
    "            output = model(data) # make a forward pass\n",
    "            loss = criterion(output, target)\n",
    "            loss.backward() # make a backward pass\n",
    "            optimizer.step()\n",
    "            \n",
    "            train_loss_total += loss.item()\n",
    "            \n",
    "        # calculate average loss over an epoch\n",
    "        train_loss = train_loss_total / len(train_loader)\n",
    "        \n",
    "        # print loss statistics\n",
    "        print(f\"Epoch: {epoch}, train_loss: {train_loss}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We define a pytorch model class called BinaryClassifer and a predict function which we store in a file for later use"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[37m# torch imports\u001b[39;49;00m\r\n",
      "\u001b[34mimport\u001b[39;49;00m \u001b[04m\u001b[36mtorch\u001b[39;49;00m\u001b[04m\u001b[36m.\u001b[39;49;00m\u001b[04m\u001b[36mnn\u001b[39;49;00m\u001b[04m\u001b[36m.\u001b[39;49;00m\u001b[04m\u001b[36mfunctional\u001b[39;49;00m \u001b[34mas\u001b[39;49;00m \u001b[04m\u001b[36mF\u001b[39;49;00m\r\n",
      "\u001b[34mimport\u001b[39;49;00m \u001b[04m\u001b[36mtorch\u001b[39;49;00m\u001b[04m\u001b[36m.\u001b[39;49;00m\u001b[04m\u001b[36mnn\u001b[39;49;00m \u001b[34mas\u001b[39;49;00m \u001b[04m\u001b[36mnn\u001b[39;49;00m\r\n",
      "\u001b[34mimport\u001b[39;49;00m \u001b[04m\u001b[36mtorch\u001b[39;49;00m\r\n",
      "\r\n",
      "\u001b[34mdef\u001b[39;49;00m \u001b[32mpredict\u001b[39;49;00m(input_data, model):\r\n",
      "    \u001b[33m'''\u001b[39;49;00m\r\n",
      "\u001b[33m    Produce model predictions for some input_data using a saved PyTorch model\u001b[39;49;00m\r\n",
      "\u001b[33m\u001b[39;49;00m\r\n",
      "\u001b[33m    Arguments:\u001b[39;49;00m\r\n",
      "\u001b[33m        input_data (numpy matrix)\u001b[39;49;00m\r\n",
      "\u001b[33m        model (PyTorch model)\u001b[39;49;00m\r\n",
      "\u001b[33m    \u001b[39;49;00m\r\n",
      "\u001b[33m    Returns:\u001b[39;49;00m\r\n",
      "\u001b[33m        numpy array of model predictions\u001b[39;49;00m\r\n",
      "\u001b[33m    '''\u001b[39;49;00m\r\n",
      "    \u001b[36mprint\u001b[39;49;00m(\u001b[33m'\u001b[39;49;00m\u001b[33mPredicting class probabilities for the input data...\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m)\r\n",
      "\r\n",
      "    device = torch.device(\u001b[33m\"\u001b[39;49;00m\u001b[33mcuda\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m \u001b[34mif\u001b[39;49;00m torch.cuda.is_available() \u001b[34melse\u001b[39;49;00m \u001b[33m\"\u001b[39;49;00m\u001b[33mcpu\u001b[39;49;00m\u001b[33m\"\u001b[39;49;00m)\r\n",
      "    \r\n",
      "    \u001b[37m# Process input_data so that it is ready to be sent to our model.\u001b[39;49;00m\r\n",
      "    data = torch.from_numpy(input_data.astype(\u001b[33m'\u001b[39;49;00m\u001b[33mfloat32\u001b[39;49;00m\u001b[33m'\u001b[39;49;00m))\r\n",
      "    data = data.to(device)\r\n",
      "\r\n",
      "    \u001b[37m# Put the model into evaluation mode\u001b[39;49;00m\r\n",
      "    model.eval()\r\n",
      "\r\n",
      "    \u001b[37m# Predicted scores\u001b[39;49;00m\r\n",
      "    probabilities = model(data).cpu().detach().numpy()\r\n",
      "\r\n",
      "    \u001b[34mreturn\u001b[39;49;00m probabilities\r\n",
      "\r\n",
      "\u001b[34mclass\u001b[39;49;00m \u001b[04m\u001b[32mBinaryClassifier\u001b[39;49;00m(nn.Module):\r\n",
      "    \u001b[33m\"\"\"\u001b[39;49;00m\r\n",
      "\u001b[33m    Define a neural network that performs binary classification.\u001b[39;49;00m\r\n",
      "\u001b[33m    The network should accept your number of features as input, and produce \u001b[39;49;00m\r\n",
      "\u001b[33m    a single sigmoid value, that can be rounded to a label: 0 or 1, as output.\u001b[39;49;00m\r\n",
      "\u001b[33m    \u001b[39;49;00m\r\n",
      "\u001b[33m    Notes on training:\u001b[39;49;00m\r\n",
      "\u001b[33m    To train a binary classifier in PyTorch, use BCELoss.\u001b[39;49;00m\r\n",
      "\u001b[33m    BCELoss is binary cross entropy loss, documentation: https://pytorch.org/docs/stable/nn.html#torch.nn.BCELoss\u001b[39;49;00m\r\n",
      "\u001b[33m    \"\"\"\u001b[39;49;00m\r\n",
      "\r\n",
      "    \u001b[37m## TODO: Define the init function, the input params are required (for loading code in train.py to work)\u001b[39;49;00m\r\n",
      "    \u001b[34mdef\u001b[39;49;00m \u001b[32m__init__\u001b[39;49;00m(\u001b[36mself\u001b[39;49;00m, input_features, hidden_dim, output_dim):\r\n",
      "        \u001b[33m\"\"\"\u001b[39;49;00m\r\n",
      "\u001b[33m        Initialize the model by setting up linear layers.\u001b[39;49;00m\r\n",
      "\u001b[33m        Use the input parameters to help define the layers of your model.\u001b[39;49;00m\r\n",
      "\u001b[33m        :param input_features: the number of input features in your training/test data\u001b[39;49;00m\r\n",
      "\u001b[33m        :param hidden_dim: helps define the number of nodes in the hidden layer(s)\u001b[39;49;00m\r\n",
      "\u001b[33m        :param output_dim: the number of outputs you want to produce\u001b[39;49;00m\r\n",
      "\u001b[33m        \"\"\"\u001b[39;49;00m\r\n",
      "        \u001b[36msuper\u001b[39;49;00m(BinaryClassifier, \u001b[36mself\u001b[39;49;00m).\u001b[32m__init__\u001b[39;49;00m()\r\n",
      "\r\n",
      "        \u001b[37m# define any initial layers, here\u001b[39;49;00m\r\n",
      "        \u001b[36mself\u001b[39;49;00m.fc1 = nn.Linear(input_features, hidden_dim)\r\n",
      "        \u001b[36mself\u001b[39;49;00m.dropout = nn.Dropout(\u001b[34m0.3\u001b[39;49;00m)\r\n",
      "        \u001b[36mself\u001b[39;49;00m.fc2 = nn.Linear(hidden_dim, output_dim)\r\n",
      "        \r\n",
      "        \u001b[37m# sigmoid layer\u001b[39;49;00m\r\n",
      "        \u001b[36mself\u001b[39;49;00m.sig = nn.Sigmoid()\r\n",
      "        \r\n",
      "    \u001b[37m## TODO: Define the feedforward behavior of the network\u001b[39;49;00m\r\n",
      "    \u001b[34mdef\u001b[39;49;00m \u001b[32mforward\u001b[39;49;00m(\u001b[36mself\u001b[39;49;00m, x):\r\n",
      "        \u001b[33m\"\"\"\u001b[39;49;00m\r\n",
      "\u001b[33m        Perform a forward pass of our model on input features, x.\u001b[39;49;00m\r\n",
      "\u001b[33m        :param x: A batch of input features of size (batch_size, input_features)\u001b[39;49;00m\r\n",
      "\u001b[33m        :return: A single, sigmoid-activated value as output\u001b[39;49;00m\r\n",
      "\u001b[33m        \"\"\"\u001b[39;49;00m\r\n",
      "        \r\n",
      "        \u001b[37m# define the feedforward behavior\u001b[39;49;00m\r\n",
      "        x = \u001b[36mself\u001b[39;49;00m.fc1(x)\r\n",
      "        x = F.relu(x)\r\n",
      "        x = \u001b[36mself\u001b[39;49;00m.dropout(x)\r\n",
      "        x = \u001b[36mself\u001b[39;49;00m.fc2(x)\r\n",
      "        x = \u001b[36mself\u001b[39;49;00m.sig(x)\r\n",
      "        \r\n",
      "        \u001b[34mreturn\u001b[39;49;00m x\r\n",
      "    \r\n"
     ]
    }
   ],
   "source": [
    "from src.utils.model import BinaryClassifier, predict\n",
    "!pygmentize '../src/utils/model.py'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we train our model. A small number of layers is chosen since our input dimension is small."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 1, train_loss: 0.7066477366856166\n",
      "Epoch: 2, train_loss: 0.7075955016272408\n",
      "Epoch: 3, train_loss: 0.7036196163722447\n",
      "Epoch: 4, train_loss: 0.7022808960505894\n",
      "Epoch: 5, train_loss: 0.7039280704089573\n",
      "Epoch: 6, train_loss: 0.7012208444731576\n",
      "Epoch: 7, train_loss: 0.6994143554142543\n",
      "Epoch: 8, train_loss: 0.7002227987561908\n",
      "Epoch: 9, train_loss: 0.6986893330301557\n",
      "Epoch: 10, train_loss: 0.6963434049061367\n",
      "Epoch: 11, train_loss: 0.6967335428510394\n",
      "Epoch: 12, train_loss: 0.6963287081037249\n",
      "Epoch: 13, train_loss: 0.694948537009103\n",
      "Epoch: 14, train_loss: 0.6930008445467267\n",
      "Epoch: 15, train_loss: 0.6937265566417149\n",
      "Epoch: 16, train_loss: 0.6929550426346915\n",
      "Epoch: 17, train_loss: 0.6926676801272801\n",
      "Epoch: 18, train_loss: 0.6898664917264666\n",
      "Epoch: 19, train_loss: 0.6890133449009487\n",
      "Epoch: 20, train_loss: 0.6885411994797843\n",
      "Epoch: 21, train_loss: 0.6878343309674945\n",
      "Epoch: 22, train_loss: 0.688482906137194\n",
      "Epoch: 23, train_loss: 0.6875549895422799\n",
      "Epoch: 24, train_loss: 0.6858681184904916\n",
      "Epoch: 25, train_loss: 0.6854349630219596\n",
      "Epoch: 26, train_loss: 0.6850882853780474\n",
      "Epoch: 27, train_loss: 0.68381610086986\n",
      "Epoch: 28, train_loss: 0.6827153393200466\n",
      "Epoch: 29, train_loss: 0.6828190514019558\n",
      "Epoch: 30, train_loss: 0.6809933355876377\n",
      "Epoch: 31, train_loss: 0.6800429991313389\n",
      "Epoch: 32, train_loss: 0.6793430703026908\n",
      "Epoch: 33, train_loss: 0.6746668815612793\n",
      "Epoch: 34, train_loss: 0.6776165281023298\n",
      "Epoch: 35, train_loss: 0.6786598988941738\n",
      "Epoch: 36, train_loss: 0.6743305580956596\n",
      "Epoch: 37, train_loss: 0.6747087155069623\n",
      "Epoch: 38, train_loss: 0.6734846404620579\n",
      "Epoch: 39, train_loss: 0.6743411336626325\n",
      "Epoch: 40, train_loss: 0.669326731136867\n",
      "Epoch: 41, train_loss: 0.6692943572998047\n",
      "Epoch: 42, train_loss: 0.6730014681816101\n",
      "Epoch: 43, train_loss: 0.6655786888939994\n",
      "Epoch: 44, train_loss: 0.6720118011747088\n",
      "Epoch: 45, train_loss: 0.6705750397273472\n",
      "Epoch: 46, train_loss: 0.6691714610372271\n",
      "Epoch: 47, train_loss: 0.6648821064404079\n",
      "Epoch: 48, train_loss: 0.6628319195338658\n",
      "Epoch: 49, train_loss: 0.6578512958117894\n",
      "Epoch: 50, train_loss: 0.6674837895802089\n",
      "Epoch: 51, train_loss: 0.6688434141022819\n",
      "Epoch: 52, train_loss: 0.6610288194247654\n",
      "Epoch: 53, train_loss: 0.6572527800287519\n",
      "Epoch: 54, train_loss: 0.6627772109849113\n",
      "Epoch: 55, train_loss: 0.6552483694893974\n",
      "Epoch: 56, train_loss: 0.6577945947647095\n",
      "Epoch: 57, train_loss: 0.6567311372075763\n",
      "Epoch: 58, train_loss: 0.6523480755942208\n",
      "Epoch: 59, train_loss: 0.6546104550361633\n",
      "Epoch: 60, train_loss: 0.6625211324010577\n",
      "Epoch: 61, train_loss: 0.6488624470574516\n",
      "Epoch: 62, train_loss: 0.650219167981829\n",
      "Epoch: 63, train_loss: 0.6486446516854423\n",
      "Epoch: 64, train_loss: 0.6566880430494036\n",
      "Epoch: 65, train_loss: 0.6541023424693516\n",
      "Epoch: 66, train_loss: 0.6510869520051139\n",
      "Epoch: 67, train_loss: 0.634715906211308\n",
      "Epoch: 68, train_loss: 0.644249609538487\n",
      "Epoch: 69, train_loss: 0.6305969102042062\n",
      "Epoch: 70, train_loss: 0.6454809733799526\n",
      "Epoch: 71, train_loss: 0.6390276551246643\n",
      "Epoch: 72, train_loss: 0.6380171179771423\n",
      "Epoch: 73, train_loss: 0.6322032724108014\n",
      "Epoch: 74, train_loss: 0.6405370831489563\n",
      "Epoch: 75, train_loss: 0.6226859007562909\n",
      "Epoch: 76, train_loss: 0.6253347396850586\n",
      "Epoch: 77, train_loss: 0.6420159084456307\n",
      "Epoch: 78, train_loss: 0.636121119771685\n",
      "Epoch: 79, train_loss: 0.654941201210022\n",
      "Epoch: 80, train_loss: 0.6486233302525112\n",
      "Epoch: 81, train_loss: 0.6445428133010864\n",
      "Epoch: 82, train_loss: 0.6243322832243783\n",
      "Epoch: 83, train_loss: 0.621263861656189\n",
      "Epoch: 84, train_loss: 0.6146723798343113\n",
      "Epoch: 85, train_loss: 0.6449670706476484\n",
      "Epoch: 86, train_loss: 0.6516124095235553\n",
      "Epoch: 87, train_loss: 0.6209771718297686\n",
      "Epoch: 88, train_loss: 0.6304341384342739\n",
      "Epoch: 89, train_loss: 0.6388834289142064\n",
      "Epoch: 90, train_loss: 0.6189961433410645\n",
      "Epoch: 91, train_loss: 0.6344508954456874\n",
      "Epoch: 92, train_loss: 0.639908526624952\n",
      "Epoch: 93, train_loss: 0.6395352738244193\n",
      "Epoch: 94, train_loss: 0.6145294223512922\n",
      "Epoch: 95, train_loss: 0.6200470583779472\n",
      "Epoch: 96, train_loss: 0.5998520510537284\n",
      "Epoch: 97, train_loss: 0.5905123438153949\n",
      "Epoch: 98, train_loss: 0.6327066251209804\n",
      "Epoch: 99, train_loss: 0.6245362418038505\n",
      "Epoch: 100, train_loss: 0.6444341455187116\n",
      "Epoch: 101, train_loss: 0.6104051811354501\n",
      "Epoch: 102, train_loss: 0.6239234634808132\n",
      "Epoch: 103, train_loss: 0.6024137905665806\n",
      "Epoch: 104, train_loss: 0.6114331568990435\n",
      "Epoch: 105, train_loss: 0.610156067780086\n",
      "Epoch: 106, train_loss: 0.608531015259879\n",
      "Epoch: 107, train_loss: 0.6023514355931964\n",
      "Epoch: 108, train_loss: 0.6440937008176532\n",
      "Epoch: 109, train_loss: 0.6054911358015878\n",
      "Epoch: 110, train_loss: 0.6168840782982963\n",
      "Epoch: 111, train_loss: 0.6206809963498797\n",
      "Epoch: 112, train_loss: 0.5877884541239057\n",
      "Epoch: 113, train_loss: 0.599537696157183\n",
      "Epoch: 114, train_loss: 0.5925307869911194\n",
      "Epoch: 115, train_loss: 0.5978709885052272\n",
      "Epoch: 116, train_loss: 0.576933239187513\n",
      "Epoch: 117, train_loss: 0.6110691045011792\n",
      "Epoch: 118, train_loss: 0.6291719249316624\n",
      "Epoch: 119, train_loss: 0.6318403823035104\n",
      "Epoch: 120, train_loss: 0.5941944973809379\n",
      "Epoch: 121, train_loss: 0.6034653782844543\n",
      "Epoch: 122, train_loss: 0.5912418280329023\n",
      "Epoch: 123, train_loss: 0.5879633171217782\n",
      "Epoch: 124, train_loss: 0.5663297474384308\n",
      "Epoch: 125, train_loss: 0.5943065796579633\n",
      "Epoch: 126, train_loss: 0.5876800843647548\n",
      "Epoch: 127, train_loss: 0.5888404335294452\n",
      "Epoch: 128, train_loss: 0.5816729153905597\n",
      "Epoch: 129, train_loss: 0.5717911975724357\n",
      "Epoch: 130, train_loss: 0.5831446690218789\n",
      "Epoch: 131, train_loss: 0.6125929015023368\n",
      "Epoch: 132, train_loss: 0.5771014562674931\n",
      "Epoch: 133, train_loss: 0.5770418473652431\n",
      "Epoch: 134, train_loss: 0.5971933858735221\n",
      "Epoch: 135, train_loss: 0.6004563527447837\n",
      "Epoch: 136, train_loss: 0.6236485413142613\n",
      "Epoch: 137, train_loss: 0.5809252985886165\n",
      "Epoch: 138, train_loss: 0.5875373312405178\n",
      "Epoch: 139, train_loss: 0.5890059811728341\n",
      "Epoch: 140, train_loss: 0.5694697286401477\n",
      "Epoch: 141, train_loss: 0.5944038799830845\n",
      "Epoch: 142, train_loss: 0.583417249577386\n",
      "Epoch: 143, train_loss: 0.5866504779883793\n",
      "Epoch: 144, train_loss: 0.57420095375606\n",
      "Epoch: 145, train_loss: 0.566400442804609\n",
      "Epoch: 146, train_loss: 0.5455868031297412\n",
      "Epoch: 147, train_loss: 0.5925724932125637\n",
      "Epoch: 148, train_loss: 0.6136838453156608\n",
      "Epoch: 149, train_loss: 0.5642551055976323\n",
      "Epoch: 150, train_loss: 0.5585808796542031\n",
      "Epoch: 151, train_loss: 0.5938996161733355\n",
      "Epoch: 152, train_loss: 0.5937803983688354\n",
      "Epoch: 153, train_loss: 0.5571934963975634\n",
      "Epoch: 154, train_loss: 0.5860378061022077\n",
      "Epoch: 155, train_loss: 0.5727565373693194\n",
      "Epoch: 156, train_loss: 0.551376496042524\n",
      "Epoch: 157, train_loss: 0.5950501177992139\n",
      "Epoch: 158, train_loss: 0.5710433125495911\n",
      "Epoch: 159, train_loss: 0.5608693957328796\n",
      "Epoch: 160, train_loss: 0.5441226490906307\n",
      "Epoch: 161, train_loss: 0.5486945595060077\n",
      "Epoch: 162, train_loss: 0.5796055666037968\n",
      "Epoch: 163, train_loss: 0.5223589326654162\n",
      "Epoch: 164, train_loss: 0.579732324395861\n",
      "Epoch: 165, train_loss: 0.5535181249891009\n",
      "Epoch: 166, train_loss: 0.5721480335508075\n",
      "Epoch: 167, train_loss: 0.5560345351696014\n",
      "Epoch: 168, train_loss: 0.5917794108390808\n",
      "Epoch: 169, train_loss: 0.5463546173913139\n",
      "Epoch: 170, train_loss: 0.6045891387122018\n",
      "Epoch: 171, train_loss: 0.528961045401437\n",
      "Epoch: 172, train_loss: 0.5663583661828723\n",
      "Epoch: 173, train_loss: 0.5453590708119529\n",
      "Epoch: 174, train_loss: 0.5160062781402043\n",
      "Epoch: 175, train_loss: 0.5159112427915845\n",
      "Epoch: 176, train_loss: 0.5796046682766506\n",
      "Epoch: 177, train_loss: 0.5539806016853878\n",
      "Epoch: 178, train_loss: 0.5504654092448098\n",
      "Epoch: 179, train_loss: 0.5144126032079969\n",
      "Epoch: 180, train_loss: 0.5565098226070404\n",
      "Epoch: 181, train_loss: 0.5244647562503815\n",
      "Epoch: 182, train_loss: 0.5485826645578656\n",
      "Epoch: 183, train_loss: 0.5924939555781228\n",
      "Epoch: 184, train_loss: 0.5980914831161499\n",
      "Epoch: 185, train_loss: 0.5578377715178898\n",
      "Epoch: 186, train_loss: 0.5471737512520382\n",
      "Epoch: 187, train_loss: 0.5353058661733355\n",
      "Epoch: 188, train_loss: 0.5378037222794124\n",
      "Epoch: 189, train_loss: 0.5534372585160392\n",
      "Epoch: 190, train_loss: 0.5051261144024985\n",
      "Epoch: 191, train_loss: 0.5539480916091374\n",
      "Epoch: 192, train_loss: 0.5547896836485181\n",
      "Epoch: 193, train_loss: 0.5458381303719112\n",
      "Epoch: 194, train_loss: 0.5656005101544517\n",
      "Epoch: 195, train_loss: 0.5420844640050616\n",
      "Epoch: 196, train_loss: 0.5207873710564205\n",
      "Epoch: 197, train_loss: 0.5245975852012634\n",
      "Epoch: 198, train_loss: 0.5515156132834298\n",
      "Epoch: 199, train_loss: 0.5377596233572278\n",
      "Epoch: 200, train_loss: 0.5573704242706299\n",
      "Epoch: 201, train_loss: 0.5545925923756191\n",
      "Epoch: 202, train_loss: 0.5436286415372577\n",
      "Epoch: 203, train_loss: 0.5111160959516253\n",
      "Epoch: 204, train_loss: 0.5474759638309479\n",
      "Epoch: 205, train_loss: 0.511098393372127\n",
      "Epoch: 206, train_loss: 0.5451207033225468\n",
      "Epoch: 207, train_loss: 0.4920972926276071\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch: 208, train_loss: 0.5412098126752036\n",
      "Epoch: 209, train_loss: 0.5474626209054675\n",
      "Epoch: 210, train_loss: 0.579317991222654\n",
      "Epoch: 211, train_loss: 0.568027023758207\n",
      "Epoch: 212, train_loss: 0.49963065556117464\n",
      "Epoch: 213, train_loss: 0.5204983694212777\n",
      "Epoch: 214, train_loss: 0.5319481832640511\n",
      "Epoch: 215, train_loss: 0.5017608531883785\n",
      "Epoch: 216, train_loss: 0.5391326631818499\n",
      "Epoch: 217, train_loss: 0.5230586443628583\n",
      "Epoch: 218, train_loss: 0.5274069394384112\n",
      "Epoch: 219, train_loss: 0.48301070077078684\n",
      "Epoch: 220, train_loss: 0.5077752854142871\n",
      "Epoch: 221, train_loss: 0.521927569593702\n",
      "Epoch: 222, train_loss: 0.4932613968849182\n",
      "Epoch: 223, train_loss: 0.5278824865818024\n",
      "Epoch: 224, train_loss: 0.5297253131866455\n",
      "Epoch: 225, train_loss: 0.5014594282422747\n",
      "Epoch: 226, train_loss: 0.5177151688507625\n",
      "Epoch: 227, train_loss: 0.4770152909415109\n",
      "Epoch: 228, train_loss: 0.527503605399813\n",
      "Epoch: 229, train_loss: 0.4857472649642399\n",
      "Epoch: 230, train_loss: 0.5228347906044551\n",
      "Epoch: 231, train_loss: 0.4858179858752659\n",
      "Epoch: 232, train_loss: 0.5049561645303454\n",
      "Epoch: 233, train_loss: 0.5108223600046975\n",
      "Epoch: 234, train_loss: 0.5028068338121686\n",
      "Epoch: 235, train_loss: 0.5785657891205379\n",
      "Epoch: 236, train_loss: 0.5379981057984489\n",
      "Epoch: 237, train_loss: 0.550253859588078\n",
      "Epoch: 238, train_loss: 0.4708875034536634\n",
      "Epoch: 239, train_loss: 0.5087989313261849\n",
      "Epoch: 240, train_loss: 0.5026406943798065\n",
      "Epoch: 241, train_loss: 0.5302309138434274\n",
      "Epoch: 242, train_loss: 0.5172080142157418\n",
      "Epoch: 243, train_loss: 0.49792990939957754\n",
      "Epoch: 244, train_loss: 0.49238590257508413\n",
      "Epoch: 245, train_loss: 0.5778591462544033\n",
      "Epoch: 246, train_loss: 0.5099919353212629\n",
      "Epoch: 247, train_loss: 0.5197299293109349\n",
      "Epoch: 248, train_loss: 0.4843269671712603\n",
      "Epoch: 249, train_loss: 0.5055563236985888\n",
      "Epoch: 250, train_loss: 0.4917655757495335\n",
      "Epoch: 251, train_loss: 0.49619546106883455\n",
      "Epoch: 252, train_loss: 0.4834559602396829\n",
      "Epoch: 253, train_loss: 0.5218811844076429\n",
      "Epoch: 254, train_loss: 0.5006611262048993\n",
      "Epoch: 255, train_loss: 0.4863197420324598\n",
      "Epoch: 256, train_loss: 0.5588411731379372\n",
      "Epoch: 257, train_loss: 0.5057415068149567\n",
      "Epoch: 258, train_loss: 0.5285536391394479\n",
      "Epoch: 259, train_loss: 0.46258785469191416\n",
      "Epoch: 260, train_loss: 0.5350590561117444\n",
      "Epoch: 261, train_loss: 0.4980054497718811\n",
      "Epoch: 262, train_loss: 0.5097063907555172\n",
      "Epoch: 263, train_loss: 0.5226568281650543\n",
      "Epoch: 264, train_loss: 0.517521824155535\n",
      "Epoch: 265, train_loss: 0.5023231591497149\n",
      "Epoch: 266, train_loss: 0.45921052779470173\n",
      "Epoch: 267, train_loss: 0.49793451172964914\n",
      "Epoch: 268, train_loss: 0.5219088409628186\n",
      "Epoch: 269, train_loss: 0.4904984789235251\n",
      "Epoch: 270, train_loss: 0.5316506709371295\n",
      "Epoch: 271, train_loss: 0.4817761480808258\n",
      "Epoch: 272, train_loss: 0.4745319996561323\n",
      "Epoch: 273, train_loss: 0.4962396366255624\n",
      "Epoch: 274, train_loss: 0.5117199846676418\n",
      "Epoch: 275, train_loss: 0.47247057727405\n",
      "Epoch: 276, train_loss: 0.493446090391704\n",
      "Epoch: 277, train_loss: 0.48020066959517343\n",
      "Epoch: 278, train_loss: 0.498538008758\n",
      "Epoch: 279, train_loss: 0.4307369717529842\n",
      "Epoch: 280, train_loss: 0.4888132171971457\n",
      "Epoch: 281, train_loss: 0.42710033484867643\n",
      "Epoch: 282, train_loss: 0.4624211149556296\n",
      "Epoch: 283, train_loss: 0.4838579552514212\n",
      "Epoch: 284, train_loss: 0.4918649026325771\n",
      "Epoch: 285, train_loss: 0.4794561650071825\n",
      "Epoch: 286, train_loss: 0.521037003823689\n",
      "Epoch: 287, train_loss: 0.5048523843288422\n",
      "Epoch: 288, train_loss: 0.4784691333770752\n",
      "Epoch: 289, train_loss: 0.5149314786706652\n",
      "Epoch: 290, train_loss: 0.4661079985754831\n",
      "Epoch: 291, train_loss: 0.49064643468175617\n",
      "Epoch: 292, train_loss: 0.47319712383406504\n",
      "Epoch: 293, train_loss: 0.48425441128867014\n",
      "Epoch: 294, train_loss: 0.5322557134287698\n",
      "Epoch: 295, train_loss: 0.4423361207757677\n",
      "Epoch: 296, train_loss: 0.4590211255209787\n",
      "Epoch: 297, train_loss: 0.4824049600533077\n",
      "Epoch: 298, train_loss: 0.47351754776069094\n",
      "Epoch: 299, train_loss: 0.47653958201408386\n",
      "Epoch: 300, train_loss: 0.4717054750238146\n"
     ]
    }
   ],
   "source": [
    "import torch.optim as optim\n",
    "\n",
    "input_features = 2\n",
    "hidden_dim = 7\n",
    "output_dim = 1\n",
    "learning_rate = 0.001\n",
    "epochs = 300\n",
    "\n",
    "device = torch.device(\"cuda\" if torch.cuda.is_available() else \"cpu\")\n",
    "model = BinaryClassifier(input_features, hidden_dim, output_dim)\n",
    "optimizer = optim.Adam(model.parameters(), lr = learning_rate)\n",
    "loss_fn = torch.nn.BCELoss()\n",
    "\n",
    "train(model, train_dl, epochs, optimizer, loss_fn, device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predict Test Data and Evaluate Classifier\n",
    "Now we'll predict the test data and convert the predictions to a binary classification"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicting class probabilities for the input data...\n"
     ]
    }
   ],
   "source": [
    "probabilities = predict(test_x, model)\n",
    "labels = probabilities.round()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To evaluate the model we'll use the helper function provided by Udacity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# returns a variety of model metrics\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "def evaluate(test_preds, test_labels, verbose=True):\n",
    "    \"\"\"\n",
    "    Evaluate a model on a test set given the prediction endpoint.  \n",
    "    Return binary classification metrics.\n",
    "    :param predictor: A prediction endpoint\n",
    "    :param test_features: Test features\n",
    "    :param test_labels: Class labels for test data\n",
    "    :param verbose: If True, prints a table of all performance metrics\n",
    "    :return: A dictionary of performance metrics.\n",
    "    \"\"\"\n",
    "    test_preds = np.squeeze(test_preds)\n",
    "    \n",
    "    # calculate true positives, false positives, true negatives, false negatives\n",
    "    tp = np.logical_and(test_labels, test_preds).sum()\n",
    "    fp = np.logical_and(1-test_labels, test_preds).sum()\n",
    "    tn = np.logical_and(1-test_labels, 1-test_preds).sum()\n",
    "    fn = np.logical_and(test_labels, 1-test_preds).sum()\n",
    "    \n",
    "    # calculate binary classification metrics\n",
    "    recall = tp / (tp + fn)\n",
    "    precision = tp / (tp + fp)\n",
    "    accuracy = (tp + tn) / (tp + fp + tn + fn)\n",
    "    \n",
    "    # print metrics\n",
    "    if verbose:\n",
    "        print(pd.crosstab(test_labels, test_preds, rownames=['actuals'], colnames=['predictions']))\n",
    "        print(\"\\n{:<11} {:.3f}\".format('Recall:', recall))\n",
    "        print(\"{:<11} {:.3f}\".format('Precision:', precision))\n",
    "        print(\"{:<11} {:.3f}\".format('Accuracy:', accuracy))\n",
    "        print()\n",
    "        \n",
    "    return {'TP': tp, 'FP': fp, 'FN': fn, 'TN': tn, \n",
    "            'Precision': precision, 'Recall': recall, 'Accuracy': accuracy}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "predictions  0.0  1.0\n",
      "actuals              \n",
      "0              9    1\n",
      "1              0   15\n",
      "\n",
      "Recall:     1.000\n",
      "Precision:  0.938\n",
      "Accuracy:   0.960\n",
      "\n"
     ]
    }
   ],
   "source": [
    "metrics = evaluate(labels, test_y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That is a pretty good model!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Classification Boundary\n",
    "With a little help from the [scikit-learn documentation](https://scikit-learn.org/stable/auto_examples/classification/plot_classifier_comparison.html)\n",
    "we will display the classification boundary for our neural net."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# unpack data again\n",
    "train_x, train_y, test_x, test_y = plagiarism_data.values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "h = 0.02 # step size in the mesh\n",
    "\n",
    "xx, yy = np.meshgrid(np.arange(0, 1, h),\n",
    "                    np.arange(0, 1, h))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Predicting class probabilities for the input data...\n"
     ]
    }
   ],
   "source": [
    "Z = predict(np.c_[xx.ravel(), yy.ravel()], model)\n",
    "Z = Z.reshape(xx.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAfEAAAFNCAYAAAAQOlZzAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjMsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+AADFEAAAgAElEQVR4nO3debwddX3/8debYIpAgsiiQsKiBCVSNiNg0QKCEa0CFVwQFCwPEHdFbO1PH4hYK4Val5aqVDZpZbFFDRQKKqDIooSdhGIxJBCQPYQlsoR8fn/MnDA5OffeuXNnzpmZ834+HvfBPXPmnvPNkJz3/XznuygiMDMzs+ZZY9ANMDMzs2Ic4mZmZg3lEDczM2soh7iZmVlDOcTNzMwayiFuZmbWUA5xs3GSdJykf6/w9edJ2iP9XpJOl7RE0m8lvUnSHRW852aSnpQ0qezXrhNJIWmrQbfDrCwOcbMeJL1f0tw02P4g6WJJb+zHe0fEayPiivThG4G3ANMiYueIuDIiXj3R95C0UNLemfe8OyLWjYjnJ/raPd4rJD2VXsuHJZ0t6SVlv4/ZMHKIm3WRdDTwTeDvgZcBmwH/Cuw3gOZsDiyMiKcG8N5l2j4i1gVeCawPHDfY5oyPpDUH3QazXhziZhmS1gOOBz4WEedHxFMR8VxEXBARnxvhZ34k6X5JSyX9StJrM8+9XdJ8SU9IulfSMenxDSVdKOkxSY9KulLSGulzCyXtLelw4PvAG9Iq9suS9pC0OPP60yWdL+khSY9I+pf0+KskXZYee1jSf3SqX0lnkfxickH6un8taYu0Yl4zPWcTSXPStt0p6YjMex4n6TxJP0j/XPMkzcpzfSPicWAOMDPzeqO91xmS/i7zuPvPv1DSMZJuSa//uZLWyjz/ubQn5T5Jf9X1/+0vJN0o6XFJ90g6LvNc53ocLulu4DJJ/y3pE12vcYuk/fP82c2q4BA3W9UbgLWAH4/jZy4GZgAbAzcA/5F57lTgwxExBdgWuCw9/llgMbARSbX//4BV1kCOiFOBo4Br0q7uL2WfT+9fXwgsArYANgXO6TwNfA3YBNgGmE5a/UbEB4C7gXemr3tijz/T2Wn7NgEOBP5e0l6Z5/dN3+slJKH8L6Ncn2yb1wf2B64dx3uN5T3APsCWwHbAYel77QMcQ3I7Ygawd9fPPQV8MP0z/AXwkR6BvDvJ9XsrcCZwSObPsj3JNb9oHG01K5VD3GxVGwAPR8TyvD8QEadFxBMR8QxJUG6fVvQAzwEzJU2NiCURcUPm+CuAzdNK/8oY/0YGO5ME3+fSHoOnI+LXaZvujIifRcQzEfEQ8E8kgTQmSdNJ7sX/TfqaN5H0CHwgc9qvI+Ki9B76WcD2Y7zsDZIeAx4m6QX43jjeayzfjoj7IuJR4AJgh/T4e4DTI+K29HbEcdkfiogrIuLWiFgREbeQ/DLRfY2OS6/tH4GfAjMkzUif+wBwbkQ8O462mpXKIW62qkeADfPeA5U0SdIJkn4v6XFgYfrUhul/DwDeDiyS9EtJb0iPnwTcCVwqaYGkzxdo63RgUa9fOCRtLOmctAv/ceDfM20ayybAoxHxRObYIpKqs+P+zPfLgLXGuGY7RcRLSHo5vgNcmXZ753mvsXS3Zd3Mn+OertddSdIuki5Pb0UsJen16L5GK38+/SXtPOCQ9NbHQSS/wJgNjEPcbFXXAE+TdPnm8X6SAW97A+uRdGtD0p1NRFwXEfuRdLX/hCQESCv3z0bEK4F3AkePswsZkoDZbITw/BpJ9/x2ETGVpBtYmedHq/rvA14qaUrm2GbAveNs32oi4jmSSntLktsLY73XU8DamedePo63+wPJLzrZ1836IcmtgOkRsR7wXVa9RrD6dToTOBjYC1gWEdeMoz1mpXOIm2VExFLgWOBkSftLWlvSiyS9TVKve8dTgGdIKvi1SUa0AyBpsqSDJa2XhtfjwPPpc++QtJUkZY6Pd3rXb0mC6gRJ60haS9JumXY9CTwmaVOge1DeAyQjxXtdg3uAq4Gvpa+5HXA4q97rLyS9j/8h4I/AghzvdRPwdkkvlfRy4NPjeLvzgMMkzZS0NvClruenkPQCPC1pZ5JfyEaVhvYK4Ou4CrcacIibdYmIfwKOBr4IPERS8X6cpJLu9gOSbtp7gfmsOmALkvumC9Mu7aN4YWDUDODnJEF7DfCvmbnhedv5PEkVvxXJQLXFwHvTp78M7AQsBf4bOL/rx78GfFHJ6Phjerz8QSS9CveRDPL7UkT8bDzt63KzpCeBJcChwF+m97DHeq+zgJtJblNcCpyb9w0j4mKSqYKXkdy6uKzrlI8Cx0t6guQXt/NyvvQPgD8luUVhNlAa/1gaM7PhJemDwJER0ZfFf8xG40rczCyntFv+o8Apg26LGTjEzcxykfRWktsrD5AMijMbOHenm5mZNZQrcTMzs4ZyiJuZmTVU43bmefHU9WPqxpsMuhnj8vjSZSu/n7ymf28yM7P8Hr9/wcMRsVGv5xoX4lM33oT3n3j2oJtRyKWX3rTy++nrrz3KmWZmZolLTnj3opGec1nYR7Nn78Ds2cneDPcsWcY9S5aN8RNmZmYjc4gPgMPczMzK4BAfIIe5mZlNhEO8BhzmZmZWhEO8RhzmZmY2Ho0bnT4MOkEOL4xo92h2MzPr5hCvuU6ge3qamZl1c3d6Q7ir3czMujnEG8ZhbmZmHQ7xhnKYm5mZQ7zhHOZmZsPLId4SDnMzs+HjEG8Zh7mZ2fDwFLOW8lxzM7P2c4gPAc81NzNrJ3enDxF3tZuZtYtDfAg5zM3M2sEhPsQc5mZmzeYQN4e5mVlDOcRtJYe5mVmzOMRtNQ5zM7Nm8BQzG5HnmpuZ1ZtD3HLxXHMzs/pxd7qNi7vazczqwyFuhTjMzcwGzyFuE+IwNzMbHIe4lcJhbmbWfw5xK5XD3Mysfzw63Srh6WlmZtVziFvluqenOczNzMrhELe+8VxzM7Ny+Z649Z3vm5uZlcMhbgPjMDczmxiHuA2cw9zMrBiHuNWGw9zMbHwc4lY7DnMzs3wc4lZbDnMzs9F5ipnVnheOMTPrzSFujeK55mZmL3B3ujWSu9rNzBzi1nAOczMbZg5xawWHuZkNI4e4tYrD3MyGiUPcWslhbmbDoNLR6ZL2Ab4FTAK+HxEndD2/GXAm8JL0nM9HxEVVtsmGi6enmVmbVRbikiYBJwNvARYD10maExHzM6d9ETgvIr4jaSZwEbBFVW2y4ebpaWbWNlV2p+8M3BkRCyLiWeAcYL+ucwKYmn6/HnBfhe0xA9zVbmbtUWV3+qbAPZnHi4Fdus45DrhU0ieAdYC9K2yP2SqylXknyF2Zm1mTVFmJq8ex6Hp8EHBGREwD3g6cJWm1Nkk6UtJcSXP/uHRJBU21YebK3MyaqsoQXwxMzzyexurd5YcD5wFExDXAWsCG3S8UEadExKyImPXi9davqLk27BzmZtY0VYb4dcAMSVtKmgy8D5jTdc7dwF4AkrYhCfGHKmyT2Zgc5mbWFJWFeEQsBz4OXALcTjIKfZ6k4yXtm572WeAISTcDZwOHRUR3l7vZQDjMzazuKp0nns75vqjr2LGZ7+cDu1XZBrOJ8lxzM6srb0VqNg7dc80d5mY2SA5xswK8cIyZ1YHXTjebAN83N7NBcoiblcBhbmaDMGaIS3qZpFMlXZw+ninp8OqbZtY8DnMz66c8lfgZJNPENkkf/w74dFUNMmsDh7mZ9UOeEN8wIs4DVsDK+d/PV9oqs5ZwmJtZlfKMTn9K0gak655L2hVYWmmrzFrGc83NrAp5QvxokuVSXyXpKmAj4MBKW2XWYp6eZmZlGTPEI+IGSbsDrybZmeyOiHiu8paZtZy3QjWzicozOv1jwLoRMS8ibgPWlfTR6ptmNhx839zMisozsO2IiHis8yAilgBHVNcks+HkMDez8coT4mtIUueBpEnA5OqaZDbcHOZmlleegW2XAOdJ+i7JCPWjgP+ptFVm5nvmZjamPCH+N8CHgY+QDGy7FPh+lY0ysxc4zM1sJHlGp68AvpN+mdmAeK65mXUbM8Ql7QYcB2yeni8gIuKV1TbNzEbiueZmBvm6008FPgNcj5dbNasVd7WbDbc8Ib40Ii6uvCVmVpjD3Gw45QnxyyWdBJwPPNM5GBE3VNYqMyvEYW42XPKE+C7pf2dljgXw5vKbY2ZlcJibDYc8o9P37EdDzKx8DnOzdsuzdvrLJJ0q6eL08UxJh1ffNDMri1eBM2unPN3pZwCnA19IH/8OOJdk1LqZNYjnmpu1S5610zeMiPOAFQARsZwBTjVbsSJ48tnlg3p7s9boVOedytzVuVnz5AnxpyRtQDKYDUm7AksrbdUoNlg32XvlyWeXO8zNSuCudrPmytOdfjQwB3iVpKuAjYADK23VGA7bdXMAzrh20cogX3dynj+KmY3Eg+DMmifP6PQbJO0OvJpkydU7IuK5yluWg8PcrHwOc7PmyLN2+ge7Du0kiYj4QUVtGrdOmEMS6OAwN5soh7lZ/eVJutdnvl8L2Au4AahNiGdlq/MOB7pZcQ5zs/rK053+iexjSesBZ1XWopK4q92sXA5zs/opkmrLgBllN6QqDnOzcnmuuVl95LknfgHp9DKSKWkzgfOqbFQVHOZm5fO+5maDlSfF/jHz/XJgUUQsrqg9lXOYm5XPXe1mg5Hnnvgv+9GQfnOYm5XPYW7WX3m605/ghe70VZ4CIiKmlt6qPnKYm5XPYW7WH3nS6hvA/SQj0gUcDEyJiBOrbFi/ea65Wfkc5mbVypNSb42IXTKPvyPpN0CrQjzLc83NyuUwN6tGng1Qnpd0sKRJktaQdDAD3MWsnw7bdfOVge4NV8wmzputmJUrT3n5fuBb6VcAV6XHhobvm5uVy3PNzcqRZ3T6QmC/6ptSfw5zs/J5rrlZcWN2p0vaWtIvJN2WPt5O0herb1p9uZvdrHzuajcbvzz3xP8N+FvgOYCIuAV4X5WNagqHuVn5HOZm+eUJ8bUj4rddx5xWGQ5zs/I5zM3GlueG7sOSXkW64IukA4E/VNqqhvJcc7PyeXqa2cjyJMzHgFOA10i6F7gLOKTSVrWA55qblcthbra6PKPTFwB7S1oHWCMinqi+We3hEe1m5XKYm70gz+j0T0maSrKP+Dck3SBpdp4Xl7SPpDsk3Snp8yOc8x5J8yXNk/TD8TW/OXzf3KxcnXvms2fv4HvmNrTylIR/FRHfkvRWYGPgQ8DpwKWj/ZCkScDJwFuAxcB1kuZExPzMOTNIRr7vFhFLJG1c8M/RGK7MzcrnueY2rPKMTlf637cDp0fEzZljo9kZuDMiFkTEs8A5rL5ozBHAyRGxBCAiHszX7OZzZW5WPo9ot2GTJ8Svl3QpSYhfImkKsCLHz20K3JN5vDg9lrU1sLWkqyRdK2mfXi8k6UhJcyXNfeyRR3K8dXM4zM3K5zC3YZGnH/dwYAdgQUQsk7QBSZf6WHpV6937kq8JzAD2AKYBV0raNiIeW+WHIk4hGSHPa7bbodfe5o3nbnaz8nkQnLVdntHpKyRtARwiKYBfR8SPc7z2YmB65vE04L4e51wbEc8Bd0m6gyTUr8vx+q3kueZm5XOYW1vlGZ3+r8BRwK3AbcCHJZ2c47WvA2ZI2lLSZJKlWud0nfMTYM/0fTYk6V5fkL/57dbpau90s7ur3Wxi3M1ubZOnxNsd2DYiOiu2nUkS6KOKiOWSPg5cAkwCTouIeZKOB+ZGxJz0udmS5pPsUf65iGjXTe8SuKvdrFyuzK0t8iTBHcBmQGfpsenALXlePCIuAi7qOnZs5vsAjk6/bAwOc7NyeV9za7oRE0DSBSQD0dYDbpf02/TxLsDV/Wme9eIwNyuf55pbE432yf+PfWuFFeIwNyufu9qtSUb8xI+IX/azIVacw9ysfA5za4IxP+klPcEL87snAy8CnoqIqVU2zMbPYW5WPoe51VmeeeJTso8l7U+ypKrVlOeam5XPYW51lGfZ1VVExE+AN1fQFquA55qblctzza1O8nSnvyvzcA1gFqsvn2o15652s3K5Mrc6yPMp/s7M98uBhay+G5k1hMPcrFyea26DlOeeeJ7NTqxhHOZm5fNcc+u3Ee+JSzpC0oz0e0k6TdJSSbdI2ql/TbQqeStUs/L5vrn1y2gD2z5F0nUOcBCwPfBKkiVSv1Vts6zfHOZm5XOYW9VGC/Hl6RahAO8AfhARj0TEz4F1qm+aDYLD3Kx8DnOrymg3QVdIegWwBNgL+GrmuRdX2qrReFx8X3iuuVn5PKLdyjbap/KxwFySbUTnRMQ8AEm7M+A9v2NFoDU0yCYMlewguA4HullxDnMry2hrp18oaXNgSkQsyTw1F3hv5S0bwTqTJwEO8kHwiHazcjnMbaJG/QSOiOUk3enZY09V2qIcZm2arAQ7994nABzmfeYwNyuX55pbUY3+5J216RTm3vuEq/IBcZiblc9zzW08xr12et10qvJYEcQKj3obBI9oNyufR7RbHrnKJkmbAptnz4+IX1XVqPHKdq+7Kh8cV+Zm5fN9cxtNng1Q/oFkINt84Pn0cAC1CfEOd6/Xg8PcrHwOc+slzyfr/sCrI+KZqhtTBg96qw/PNTcrn8PcsvLcE18AvKjqhpQte6/cBs/7mpuVy/fMDfJV4suAmyT9AlhZjUfEJytrVUlcldePu9rNyuXKfLjl+fSck341lu+V14/D3Kxcnms+nPLsJ36mpMnA1umhOzIbozRGNsjBVXldOMzNyue55sMjz+j0PYAzSbYlFTBd0qF1mmKWl6ei1ZfD3Kx87mpvvzwD274OzI6I3SPiz4G3At+otlnV8qC3+vLCMWbl8yC49spT6rwoIu7oPIiI30lq3Gj1bh70Vm+uzM3K58q8ffJ8Ks6VdCpwVvr4YOD66prUXx70Vm+ea25WPod5e+TpTv8IMA/4JPApkpXbjqqyUf3m9debwXPNzcrlbvbmU0SzQut1r3tdXHX11ZW9vrvXm6NTmYOrc7MyeDR7PV1ywruvj4hZvZ7LMzr9HcBXeGEDFAEREVNLbWVNuHu9OXzf3KxcnmvePHk+8b4JvAu4NZpWthfkQW/N4jA3K5/nmjdDnnvi9wC3DUuAZ3kqWrN4eppZ+XzfvN7ylCt/DVwk6Zesunb6P1XWqhrxSm/N48rcrHwe0V5PeT7Zvgo8CawFTK62OfXkld6ayWFuVj6Heb3k+UR7aUTMrrwlDeBBb83kueZm5XOY10Oee+I/l+QQT83adAqzNp3iOeUN5bnmZuXyPfPByhPiHwP+R9IfJT0u6QlJj1fdsLrzoLdm8yA4s3I5zAcjz1akU/rRkCbyoLfm831zs3L1mmsO7mqvSq5PK0nbAVtkz4+I8ytqU6N40Fs7OMzNyuf75tXLs2LbacB2JOunr0gPB+AQz/Cgt3ZwmJuVz2FenTyfTrtGxMzKW9ICXumtPRzmZuVzmJcvz6fSNZJmRsT8ylvTEq7K28NhblY+3zcvT55PozNJgvx+khXbOhugbFdpy0YQK54fxNuOmwe9tYvnmptVw9X5xOT5FDoN+ABwKy/cEx+oWPY4Wrv+m6h50Fs7ZavzDge62cQ4zIvJM0/87oiYExF3RcSizleeF5e0j6Q7JN0p6fOjnHegpJDUc7/UVc59JvmfG8seJ5Y1Y7q655S3k+eam5XP883HJ0/58L+SfghcwKoboIw6Ol3SJOBk4C3AYuA6SXO6761LmgJ8EvhN7kYvSu6hLN98h0ZW5eDu9TbxfXOz8vm+eT55KvEXk4T3bOCd6dc7cvzczsCdEbEgIp4FzgH263HeV4ATgadztTijE+ZNqcjBVXmbuTI3q4ar85HlWbHtQwVfe1OSvcg7FgO7ZE+QtCMwPSIulHRMkTfJVuWAq3IbOFfmZtXwffPVjVmJS5om6ceSHpT0gKT/kjQtx2v3SqaV5aekNYBvAJ/N0YYjJc2VNPfhx5b2PMdVudWNK3OzanQq89mzd1hZmQ9rdZ6nO/10YA6wCUl1fUF6bCyLgemZx9OA+zKPpwDbAldIWgjsCszpNbgtIk6JiFkRMWvDl6w34htmg7wpYZ4Ncod5OznMzaoz7F3tihg9OCTdFBE7jHWsx8+tCfwO2Au4F7gOeH9EzBvh/CuAYyJi7miv+7qZW8e1Pzx51DZDs7rXO9y9Pjw819ysGm0cBHfJCe++PiJ6zt7K8wnysKRDgLPTxwcBj4z1QxGxXNLHgUuAScBpETFP0vHA3IiYk6/5xay56KZGjV4Hr/Q2TDzX3KwawzaqPU8lvhnwL8AbSO5pXw18Ku9c8bLlrcSzXJVb3TnMzarT9DAfrRIf8554RNwdEftGxEYRsXFE7D+oAC/Kg96s7nzf3Kw6bb5vPmKISzpR0lE9jn9G0j9U26zyedCbNYHD3Kw6vcK86YE+Yne6pPnAthGxouv4GsAtEbFtH9q3miLd6d3cvW5N4W52s2o1oau9aHd6dAd4enAFveeAN4a7160pXJmbVavpXe2jhfgySTO6D6bH/lhdk/pjzUU3seaimxrXvT5r0ynuXh9CDnOzajU1zEfrnzsWuFjS3wHXp8dmAX8LfLrqhvWLp6JZk3hfc7NqNW2K2oiVeERcDOwP7AmckX7tARwQERf1oW1940Fv1kSd6rxTmbs6NytXE6rzMeeJ100ZA9tG40Fv1lQeBGdWrUFV5hNdsW2ouHvdmsq7p5lVq45d7f4X3oO3N7Umc5ibVa8u26Lm2Yp0tzzH2shT0azJPKLdrHqDvm+e59fzfwZ2ynGslbLd6+Cq3JrHlblZ9QbV1T7iv2RJbwD+DNhI0tGZp6aS7Eo2NLLd675Xbk3lMDfrj352tY/2L3gysG56zpTM8ceBAytpTc01fdAbuCo3zzU365d+hHmerUg37+xalq6bvm5EDOwmcdVTzPJq0qC3Dnev20g8Pc2setludsgf6BPaihT4mqSpktYB5gN3SPpcrnduMQ96szbxIDiz6nUGwZU5EC5PiM9MK+/9gYuAzYAPTOhdW6KpK715/XUbicPcrD/KCvM8If4iSS8iCfGfRsRzgD/9U52NVKC5VbnD3Lo5zM36Y6Jhnufm1/eAhcDNwK8kbU4yuM0ymj4VzSPYrRePaDfrj6JT1AqtnS5pzYgYyK/mO229RVz57S8x+eXTBvH2uXjQm7WVB8CZ9U8nzOd/79DiA9skvUzSqZIuTh/PBA4ts6HjsXzZ0wA8e//iQTVhTE3vXjcbSaebPbt7mplVI9vVPpI898TPAC4BNkkf/44B7yf+yI23A/UP8jUX3eRBb9ZaDnOzwcsT4htGxHnACoC0G/35SluVwyM33s4jN97Os/cvrn2Yg6tyay/va242OHlC/ClJG5COSJe0K7C00laNQ1OqcvBUNGs3j2g36788IX40MAd4laSrgB8An6i0VeOUDfK6hrmnotmwcJib9c+YIR4RNwC7k2yG8mHgtRFxS9UNG69O9zo0pypvik5VDg5zy89hbla9MeeISHpX16GtJS0Fbo2IB6tpVnGP3Hg7G+y4zcogr+NUtOyuaNCcqWjd88rBU9JsbJ5rbladPN3phwPfBw5Ov/6NpIv9Kkm1XH7VVXm1XJlbEa7MzcqXJ8RXANtExAERcQAwE3gG2AX4myobN1FNC3KHuQ0Dh7lZefL0aW0REQ9kHj8IbB0Rj0p6rqJ2laYT5BvsmDyue/d6k/Yq7+gEOXjlN8vP+5qbTVyeSvxKSRdKOlTSocBPSdZQXwd4rNrmladpVXlTdU9Nc3VueXiuuVkxY66dLknAAcBugIBfA/8VRRZdL8F2014eF37sg4V/foMdt1n5fR2r8o6mDXobSacyB1fnlp/XaDd7wTcP2H7EtdMLbYAySBMN8Y5OmDvI+8NhbkU4zM1GD/E8G6C8S9L/SVoq6XFJT0hqbn9vqmnd603uYgcPgrNiPAjObHR5frU9EXhnRNxedWP6zYPe+s9zza0IzzU36y3PwLYH2hjgWU2rytvAlbkV4crcbFV5fpWdK+lc4Cck88MBiIjzK2vVAHilt8Hw9DQrwtPTzBJ5/tZPBZYBszPHAmhViEO2ez0J8zoGOSRh3pbu9axsV3uHA93Gku1q73Cg27AY8296RHyoHw2pk2xV3oQgh3ZU5R2+b25F+L65DaM8o9OnSfqxpAclPSDpvyTVM9lK1Fl/3dubDo7vm1sRvm9uwyTPwLbTSfYT3wTYFLggPTYUPOht8BzmVoTD3IZBnhDfKCJOj4jl6dcZwEYVt6tWskFe1zDvVOVtmFM+Eoe5FeEwtzbLE+IPSzpE0qT06xDgkaobVjfe3rQ+HOZWhMPc2ihPiP8V8B7gfuAPwIHA0A1262hakDvMzVblMLc2yTM6/W5g3+wxSZ8GvllVo+rOK73Vi+eaWxGea25tkKcS7+XoUlvRUK7K68dboVoR3grVmqpoiLvMSTVp0Bu0+155lrvarQh3tVvTFA3xXJ+IkvaRdIekOyV9vsfzR0uaL+kWSb+QtHmv16k7D3qrL4e5FeEwt6YYMcQ7W472+HqCZM74qCRNAk4G3gbMBA6SNLPrtBuBWRGxHfCfJDumjer5p58Z65SBaUqQt30qWi8OcyvCYW51N+IojoiYMtJzOe0M3BkRCwAknQPsB8zPvMflmfOvBQ7J88KP/e73ALxk61dNsInla8KgN2jv+utj8ZKuVoSXdLW6KtqdnsemwD2Zx4vTYyM5HLi41xOSjpQ0V9Lch558hrvSfQ46YV5HTanKYXgGvWW5MrciXJlb3VQZ4r3Km56flOkCMrOAk3o9HxGnRMSsiJg1dfKfADQuyOsa5sM46C3LYW5FOMytLqrsD1oMTM88ngbc132SpL2BLwC7R8S4bnh3gnxLmtC93pxd0Yape73Dc82tCM81t0GrshK/DpghaUtJk4H3kWykspKkHYHvAftGxINF36hpVXldDeugt26ea25FeK65DUJlIR4Ry4GPA5cAtwPnRcQ8ScdL6qwAdxKwLvAjSTdJmjPCy40pG+R1DfMmbG8KwzkVrRd3tVsR7mq3flJEsz6Ytpq6fnx91ptHPWfLtIerjt3rHRvsuA1Q39HrkCzZ2jGMXby3RvQAAAxWSURBVOzdOt3s4K52y6/TzQ7uardivnnA9tdHxKxez1XZnT4wTeler3tVPuyD3rq5MrciXJlblVoZ4pAE+V2L6t29Ds25Vw4O8g6HuRXhMLcqtDbEO5pSlUMzpqIN+6C3LIe5FeEwtzK1PsShWYPewFV50zjMrQiHuZWhlQPbRuNBb+XwoLfRea65FeG55tbL0A1sG01Tutc96K3ZPNfcivBccxuvoQtx8KC3MjnIR+eudivCXe2W11CGeEdTqnLwoLemc5hbEQ5zG8tQhzg0oyr3oLf2cJhbEQ5zG8nQh3hH06ryuhrm7U3Hw2FuRTjMrZuHQGbctSgZvd4J8jqOYH9hV7TkcR1HsHeCfJh3RcurE+Rz731iZZB7RLuNpRPkZ1y7aGWQe0T7cHIl3qXTvQ6uyifK3ev5uTK3IlyZ29DNEx+PJs0ph3pW5R2deeWuyvPzXHMrwnPN28fzxAvySm/lcVU+fp5rbkV4rvlwcYiPwd3r5fGgt2Lc1W5FuKt9ODjEc2pKkHult/ZymFsRDvN2c4iPQxPmlEPzqnIbH4e5FeEwbyeHeAFNqcrBK721mcPcinCYt4tDvKCmBLmr8vZzmFsRDvN2cIhPQBOCHJrTve6qfGIc5laEw7zZPJFwgjpBviX1XeUNmrHSGyRh7pXeJqYT5OC55pZfJ8jBc82bxJV4SVyVl8dT0crjueZWhOeaN4dDvERNDPK6hrmnopXLXe1WhLva688hXrImTUNrWlVuE+cwtyIc5vXlEK9IE6vyuvKgt/I5zK0Ih3n9OMQr1KQgr/tKb+CqvAoOcyvCYV4fDvGKNSXIoTlVOXjQW9kc5laEw3zwHOJ90LQgr3tV7kFv1XGYWxEO88FxiPdJk4IcmleVW7k6YZ6dnmY2lk6YZ6enWbUc4n3U5CCva5h70Fv1PNfcivBc8/5wiPdZE4PcVbmBu9qtGHe1V8shPgBNC3JoXve6w7w6DnMrwmFeDYf4gDQ1yD3ozToc5laEw7xcDvEBamKQQ/OqcquWw9yKcJiXwyE+YG0I8rqGuQe99ZfD3IpwmE+MQ7wGmhzkrsqtm8PcinCYF+MQr4mmBjk0r3vdYd4fnmtuRfSaa+5AH5lDvEaaHuQe9GYj8VxzK8LV+dgc4jXT5CCH5lXl1l/uarciHOYjc4jXUJuCvK5h7kFvg+UwtyIc5qtziNdUG4LcVbmNxWFuRfi++Qsc4jXW9CCH5nWvO8wHw2FuRQ17de4Qr7m2BLkHvVkeDnMraljD3CHeAG0IcmheVW6D4zC3ooatq90h3hBtDPK6hrkHvdWH55rbRAxDde4Qb5A2BbmrchsvzzW3otoc5pWGuKR9JN0h6U5Jn+/x/J9IOjd9/jeStqiyPW3QliCH5nWvO8zrwV3tVlQbu9orC3FJk4CTgbcBM4GDJM3sOu1wYElEbAV8A/iHqtrTJm0Lcg96syIc5jYRbanOq6zEdwbujIgFEfEscA6wX9c5+wFnpt//J7CXJFXYptZoU5CDq3IrzmFuE9H0MK8yxDcF7sk8Xpwe63lORCwHlgIbVNimVukEeVs0JchdldeTw9wmoqlhvmaFr92rou7+V5XnHCQdCRyZPnxm/8vPv22CbbN8NgQeHnQjhoSvdf/4WvePr3U5Nh/piSpDfDEwPfN4GnDfCOcslrQmsB7waPcLRcQpwCkAkuZGxKxKWmyr8LXuH1/r/vG17h9f6+pV2Z1+HTBD0paSJgPvA+Z0nTMHODT9/kDgsohwH5iZmVkOlVXiEbFc0seBS4BJwGkRMU/S8cDciJgDnAqcJelOkgr8fVW1x8zMrG2q7E4nIi4CLuo6dmzm+6eBd4/zZU8poWmWj691//ha94+vdf/4WldM7r02MzNrJi+7amZm1lC1DXEv2do/Oa710ZLmS7pF0i8kjTjdwUY31rXOnHegpJDkkb0F5bnWkt6T/t2eJ+mH/W5jW+T4DNlM0uWSbkw/R94+iHa2UkTU7otkINzvgVcCk4GbgZld53wU+G76/fuAcwfd7iZ+5bzWewJrp99/xNe6umudnjcF+BVwLTBr0O1u4lfOv9czgBuB9dPHGw+63U38ynmtTwE+kn4/E1g46Ha35auulbiXbO2fMa91RFweEcvSh9eSzPm38cvz9xrgK8CJwNP9bFzL5LnWRwAnR8QSgIh4sM9tbIs81zqAqen367H6miFWUF1D3Eu29k+ea511OHBxpS1qrzGvtaQdgekRcWE/G9ZCef5ebw1sLekqSddK2qdvrWuXPNf6OOAQSYtJZix9oj9Na79Kp5hNQGlLttqYcl9HSYcAs4DdK21Re416rSWtQbKb32H9alCL5fl7vSZJl/oeJL1LV0raNiIeq7htbZPnWh8EnBERX5f0BpL1QbaNiBXVN6/d6lqJj2fJVkZbstXGlOdaI2lv4AvAvhHxTJ/a1jZjXespwLbAFZIWArsCczy4rZC8nyE/jYjnIuIu4A6SULfxyXOtDwfOA4iIa4C1SNZVtwmqa4h7ydb+GfNap1283yMJcN83LG7Uax0RSyNiw4jYIiK2IBl/sG9EzB1Mcxstz2fIT0gGbSJpQ5Lu9QV9bWU75LnWdwN7AUjahiTEH+prK1uqliGe3uPuLNl6O3BepEu2Sto3Pe1UYIN0ydajgRGn69jIcl7rk4B1gR9JuklS9z9QyyHntbYS5LzWlwCPSJoPXA58LiIeGUyLmyvntf4scISkm4GzgcNcdJXDK7aZmZk1VC0rcTMzMxubQ9zMzKyhHOJmZmYN5RA3MzNrKIe4mZlZQznEzczMGsohblYBSS+XdI6k36dbXV4kaeuCr/VpSWvnOO/7kmYWeY+JkrSHpD8b5fn9JR1b4fuPuF2upEMl/V/6dWjm+M8lrV9Vm8z6wfPEzUqW7qZ3NXBmRHw3PbYDMCUirizwegtJtiR9uNSGlkjSccCTEfGPIzx/Ncnqc5X8GSTtCfwmIpZJ+giwR0S8V9JLgbkka/4HcD3wuohYkgb6tIj4ahVtMusHV+Jm5dsTeK4T4AARcVNEXKnESZJuk3SrpPfCykr2Ckn/Kel/Jf1Heu4ngU2AyyVdnp77HUlzJc2T9OXOe6Q/Pyv9/klJX5V0c7pD18vS42ekP3+5pAWSdpd0mqTbJZ2Rea3Zkq6RdIOkH0laNz2+UNKX0+O3SnqNpC2Ao4DPpCv6vSl7MdIeiGc6AZ624duSrk7bcOBEL/go2+W+FfhZRDyabjn6M6CzW9kcko05zBrLIW5Wvm1JKr5e3gXsAGwP7A2cJOkV6XM7Ap8GZgKvBHaLiG+TbCaxZ0TsmZ73hYiYBWwH7C5pux7vsw5wbURsD/yKZO/sjvWBNwOfAS4g2TnttcCfStohXUf8i8DeEbETSSV7dObnH06Pfwc4JiIWAt8FvhERO/TobdgNuKHr2CuANwLvAE7odaEkXZn+UtD9tXev8zOy2+WOuE1mGup/IslbGFtj1XUrUrO2eiNwdkQ8Dzwg6ZfA64HHgd9GxGIASTcBWwC/7vEa75F0JMm/31eQhP4tXec8C3T2JL8eeEvmuQsiIiTdCjwQEbem7zkvfc9p6WteldwZYDJwTebnz8+87rty/JlfweqbXfwk3YZyfqeXoFtEvKnX8dFo9e1yx9om80GSng6vmW6N5BA3K988kp31eukVKh3ZLV6fp8e/T0lbAscAr0/v655BsiNUt+cyG0x0v1bnfVZ0veeK9LznSbqgR+pq7vxMzzb28EeSrYJ7vQaMcE0kXUmyPWu3YyLi5z3O72yXu3tmu9zFJPuFd0wDrsg8Xittn1kjuTvdrHyXkXTTruzClvR6SbuTdG2/V9IkSRsBfw78dozXe4IXwmwq8BSwNK1g31Z665N7yrtJ2ipt+9o5RtZn29jtdmCr8TYiIt6Uds93f/UK8JG2y70EmC1p/XQk+uz0WGcA4suBheNtm1ldOMTNSpZWwH8JvEXJFLN5wHEk97Z/TNL1fTNJ2P91RNw/xkueAlws6fKIuBm4kaTaPw24qoL2PwQcBpwt6RaSUH/NGD92AfCXvQa2kfzismMamlXpuV1uRDwKfIVkz+vrgOPTYwCvIxk3sLzCdplVylPMzKxykr5Fci9+tSp6UNI2zYmIXwy6LWZFuRI3s374e2DMBWv67DYHuDWdK3EzM7OGciVuZmbWUA5xMzOzhnKIm5mZNZRD3MzMrKEc4mZmZg31/wEPzqQEFM0MBgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 576x360 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.figure(figsize = (8, 5))\n",
    "cm = plt.cm.RdBu\n",
    "plt.contourf(xx, yy, Z, cmap=cm, alpha=.8)\n",
    "plt.title('Classification Boundary')\n",
    "plt.xlabel('Containment (n = 20)')\n",
    "plt.ylabel('Longest Common Subsequence')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Red is low probability, Blue is high. We can see that high values of the features are associated with a high probability of plagiarism. This gives us further confidence the model is behaving as expected."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Save Model\n",
    "We'll want to use this PyTorch model later to predict the Medium articles, so we save it here"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "model_info_path = os.path.join(MODEL_DIR, 'model_info.pth')\n",
    "with open(model_info_path, 'wb') as f:\n",
    "    model_info = {\n",
    "        'input_features': input_features,\n",
    "        'hidden_dim': hidden_dim,\n",
    "        'output_dim': output_dim,\n",
    "    }\n",
    "    torch.save(model_info, f)\n",
    "  \n",
    "# Save the model parameters\n",
    "model_path = os.path.join(MODEL_DIR, 'model.pth')\n",
    "with open(model_path, 'wb') as f:\n",
    "    torch.save(model.cpu().state_dict(), f)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}