other_predictors.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "3e7e4f8f",
   "metadata": {},
   "source": [
    "# Other predictors\n",
    "\n",
    "This exercise is a chance to practice working with predictors.\n",
    "\n",
    "First, set up the tests and imports by running the cell below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b1115d3c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell.\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "plt.style.use('fivethirtyeight')\n",
    "\n",
    "# These lines load the tests.\n",
    "from client.api.notebook import Notebook\n",
    "ok = Notebook('other_predictors.ok')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0a1ea4a",
   "metadata": {},
   "source": [
    "We were studying prediction errors [the meaning of the mean](https://lisds.github.io/textbook/mean-slopes/mean_meaning) notebook.\n",
    "\n",
    "In that notebook, you see the assertions that, *for any sequence of numbers*:\n",
    "\n",
    "* The mean gives the lowest sum of prediction errors (and therefore, mean\n",
    "  prediction error);\n",
    "* The mean gives the lowest sum of squared prediction error (and therefore mean\n",
    "  squared prediction error).\n",
    "\n",
    "As you remember, if you have a value $c$ that is a *predictor*, then you get the prediction error for every element in your sequence, by subtracting $c$ from that element.\n",
    "\n",
    "To be more specific, lets look at some [data on chronic kidney disease](https://lisds.github.io/textbook/data/chronic_kidney_disease).\n",
    "\n",
    "This is a data table with one row per patient and one column per test on that patient.  Many of columns are values from blood tests.  Most of the patients have chronic kidney disease."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "415a8a3c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Run this cell\n",
    "ckd = pd.read_csv('ckd_clean.csv')\n",
    "ckd.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7962025d",
   "metadata": {},
   "source": [
    "We are interested in the column 'White Blood Cell Count'.\n",
    "\n",
    "Make a new variable `wbc` that is a Series containing the \"White Blood Cell Count\" data.  Do a histogram of these values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed497e02",
   "metadata": {},
   "outputs": [],
   "source": [
    "wbc = ...\n",
    "wbc.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8abaa1bf",
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = ok.grade('q_1_wbc')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7268ef13",
   "metadata": {},
   "source": [
    "Could these values plausibly have been drawn from a normal distribution?\n",
    "\n",
    "Assign either 1, 2, or 3 to the name `wbc_likely_normal` below.\n",
    "\n",
    "1. Yes, that's plausible.\n",
    "2. There isn't enough evidence to be confident either way.\n",
    "3. No, that's not plausible."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "675ed366",
   "metadata": {},
   "outputs": [],
   "source": [
    "wbc_likely_normal = ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a14f63f7",
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = ok.grade('q_2_wbc_likely_normal')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b191c317",
   "metadata": {},
   "source": [
    "## Mean square error\n",
    "\n",
    "Make a function called `mean_sq_err` that accepts two inputs:\n",
    "\n",
    "1. a sequence of numbers\n",
    "1. a predictor (a single number)\n",
    "\n",
    "It returns the mean of the squared prediction errors.\n",
    "\n",
    "For example, say the sequence of numbers was `np.array([3, 4])`, and your\n",
    "predictor was 5.  Then the sum of squared prediction errors is `(3 - 5) **\n",
    "2 + (4 - 5) ** 2` = `5`, and the mean of the squared prediction errors is `5 / 2` = 2.5.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "35ebf212",
   "metadata": {},
   "outputs": [],
   "source": [
    "def mean_sq_err(seq, p):\n",
    "    # Your code here\n",
    "    ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1119a57",
   "metadata": {},
   "source": [
    "Simple test with the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f02604b5",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(mean_sq_err(np.array([3, 4]), 5))  # Should show 2.5\n",
    "print(mean_sq_err(np.array([3, 5]), 4))  # Should show 1\n",
    "print(mean_sq_err(np.array([2, 3, 5]), 4))  # Should show 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "70c441db",
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = ok.grade('q_6_mse_func')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c82bd5f8",
   "metadata": {},
   "source": [
    "Use this function to calculate the mean squared error of `wbc` for candidate\n",
    "predictors from 7000, up to, but not including 10000, in steps of 0.5.  Your\n",
    "predictors should include 7000, 7000.5, 7001.0 ... 9999.5, and you should\n",
    "calculate a mean squared error for `wbc`, for each predictor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e18c0384",
   "metadata": {},
   "outputs": [],
   "source": [
    "predictors = ...\n",
    "mse_for_predictors = ...\n",
    "# Show the first five mean squared error values.\n",
    "mse_for_predictors[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "83e62c5f",
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = ok.grade('q_7_mse_for_predictors')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a6464e4",
   "metadata": {},
   "source": [
    "Plot the `predictors` on the x axis against `mse_for_predictors` on the y axis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "33b4b67b",
   "metadata": {},
   "outputs": [],
   "source": [
    "#- Plot mse_for_predictors against predictors"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54f2a24c",
   "metadata": {},
   "source": [
    "Now calculate the mean squared error for `wbc` using the mean as a predictor.\n",
    "Subtract this value from the minimum of `mse_for_predictors` and put the result\n",
    "into the variable `best_vs_mean`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10c7a4c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "best_vs_mean = ...\n",
    "best_vs_mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "850c6522",
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = ok.grade('q_8_best_vs_mean')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "157dfc6a",
   "metadata": {},
   "source": [
    "Calculate the median of `wbc`, calculate the mean squared error for `wbc` using the median as predictor, and subtract the mean squared error using the mean as predictor, putting the result into `median_vs_mean`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "acdad9e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "mse_for_median = ...\n",
    "median_vs_mean = ...\n",
    "median_vs_mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7c36c465",
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = ok.grade('q_9_median_vs_mean')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20b81c4e",
   "metadata": {},
   "source": [
    "## Mean absolute error\n",
    "\n",
    "You have dealt with one measure of a predictor - the mean square prediction\n",
    "error.\n",
    "\n",
    "Another measure of a predictor is its ability to reduce the *absolute* error.\n",
    "\n",
    "For example, say we have a sequence `3, 4`, and a predictor `5`.  The absolute\n",
    "errors are `abs(3 - 5), abs(4 - 5)`, and the mean absolute error is then\n",
    "(2 + 1) / 2 = 1.5.\n",
    "\n",
    "Before you continue, take some time to think whether you think the mean or the\n",
    "median will do a better job here.  Write down your answer *on the piece of\n",
    "paper you already had next to you on the desk*!\n",
    "\n",
    "Write a function `mean_abs_err` to do this prediction for a sequence `seq` and\n",
    "a predictor `p`.\n",
    "\n",
    "*Hint*: remember the Numpy function to return the absolute values in an array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42c28b9e",
   "metadata": {},
   "outputs": [],
   "source": [
    "def mean_abs_err(seq, p):\n",
    "    # Your code here\n",
    "    ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "280d4825",
   "metadata": {},
   "source": [
    "Simple test with the following:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3f355606",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(mean_abs_err(np.array([3, 4]), 5))  # Should show 1.5\n",
    "print(mean_abs_err(np.array([3, 5]), 4))  # Should show 1\n",
    "print(mean_abs_err(np.array([2, 3, 5]), 4))  # Should show 1.333 ish"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "67ac65b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = ok.grade('q_10_mae_func')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6fc49728",
   "metadata": {},
   "source": [
    "Use this function to calculate the mean absolute error of `wbc` for the candidate\n",
    "predictors you used before, from 7000 to 10000, in steps of 0.5.  You should\n",
    "calculate a mean absolute error for `wbc`, for each predictor."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66cd2d47",
   "metadata": {},
   "outputs": [],
   "source": [
    "mae_for_predictors = ...\n",
    "# Show the first five mean absolute error values.\n",
    "mae_for_predictors[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8f9a1ef1",
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = ok.grade('q_11_mae_for_predictors')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "934d4c5d",
   "metadata": {},
   "source": [
    "Plot the `predictors` on the x axis against `mae_for_predictors` on the y axis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e9d72706",
   "metadata": {},
   "outputs": [],
   "source": [
    "#- Plot mae_for_predictors against predictors"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "58315375",
   "metadata": {},
   "source": [
    "Now calculate the mean absolute error for `wbc` using the mean as a predictor.\n",
    "Subtract this value from the minimum of `mae_for_predictors` and put the result\n",
    "into the variable `a_best_vs_mean`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "57374ba1",
   "metadata": {},
   "outputs": [],
   "source": [
    "a_best_vs_mean = ...\n",
    "a_best_vs_mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0d2c7c3e",
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = ok.grade('q_12_a_best_vs_mean')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5725af8",
   "metadata": {},
   "source": [
    "Calculate the median of `wbc`, calculate the mean absolute error for `wbc` using\n",
    "the median as predictor, and subtract the mean absolute error using the mean as\n",
    "predictor, putting the result into `a_median_vs_mean`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b74aa976",
   "metadata": {},
   "outputs": [],
   "source": [
    "mae_for_median = ...\n",
    "a_median_vs_mean = ...\n",
    "a_median_vs_mean"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e885e193",
   "metadata": {},
   "outputs": [],
   "source": [
    "_ = ok.grade('q_13_a_median_vs_mean')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e448ca9",
   "metadata": {},
   "source": [
    "Were you right in your speculation as to which of the median or mean would be a better predictor of the absolute value?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "00e0aef7",
   "metadata": {},
   "source": [
    "## Done\n",
    "\n",
    "You're finished with the assignment!  Be sure to...\n",
    "\n",
    "- **run all the tests** (the next cell has a shortcut for that),\n",
    "- **Save and Checkpoint** from the \"File\" menu."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "65ba4f4b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# For your convenience, you can run this cell to run all the tests at once!\n",
    "import os\n",
    "_ = [ok.grade(q[:-3]) for q in sorted(os.listdir(\"tests\")) if q.startswith('q')]"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "split_at_heading": true,
   "text_representation": {
    "extension": ".Rmd",
    "format_name": "rmarkdown",
    "format_version": "1.1",
    "jupytext_version": "1.2.4"
   }
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}