ryostd.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "84b7bf8d",
   "metadata": {},
   "source": [
    "# Roll your own standardize\n",
    "\n",
    "The world nearly ended, and you have lost most of Numpy.\n",
    "\n",
    "All you have left is:\n",
    "\n",
    "* `np.array`.  You can make arrays, and these will add, subtract,\n",
    "  multiply, divide and so on.\n",
    "* `np.sum`. Luckily the internet saved the code for the `sum` function\n",
    "  as well as:\n",
    "* `np.sqrt`;  You can also take the square root.\n",
    "\n",
    "But that's all that's left - you have no other functions in Numpy.\n",
    "\n",
    "As a well-trained data scientist, you need functions to:\n",
    "\n",
    "* Calculate the mean of an array.\n",
    "* Calculate the variance of an array.\n",
    "* Calculate the standard deviation of an array.\n",
    "* Return the array in standard units.\n",
    "\n",
    "This next cell makes you a new reduced version of the Numpy module."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45efd1f1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# This cell makes a tiny fake 'np' module with the things you can use.\n",
    "# Run it to put the guard-rails in place!\n",
    "import numpy as old_np  # Forbidden Numpy with funny name.\n",
    "# Set precision of printing for neatness\n",
    "old_np.set_printoptions(precision=4, suppress=True)\n",
    "class np:  # Fake np module with stuff you can use.\n",
    "    sum = old_np.sum\n",
    "    array = old_np.array\n",
    "    sqrt = old_np.sqrt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec7961a3",
   "metadata": {},
   "source": [
    "Here are some arrays you can test your functions with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b4598c41",
   "metadata": {},
   "outputs": [],
   "source": [
    "test1 = np.array([7.83, 11.33, 13.38, 5.18, 11.75, 9.65,\n",
    "                  7.78, 9.68, 13.08, 10.])\n",
    "test2 = np.array([15.03, -1.79, 8.45, 24.92, -4.52, 9.19,\n",
    "                  -38.60, -14.42])\n",
    "test3 = np.array([11, 13, 18, 7, 12, 8, 1, 3, 5, 13, 11, 1, 15, -2, -3])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c7cac5a5",
   "metadata": {},
   "source": [
    "Now define a function `my_mean` that calculates the mean of its only argument, an array `arr`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "40bfb40f",
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_mean(arr):\n",
    "    ...."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b38412b9",
   "metadata": {},
   "source": [
    "Test your function with these print statements, that show the mean as\n",
    "calculated by you, and by the full Numpy from the past.\n",
    "\n",
    "The should show the same result printed twice."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "47ba0533",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(my_mean(test1), old_np.mean(test1))\n",
    "print(my_mean(test2), old_np.mean(test2))\n",
    "print(my_mean(test3), old_np.mean(test3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "325f972a",
   "metadata": {},
   "source": [
    "Next write a function called `my_var` that returns the *variance* for the input array `arr`.  Remember, the *variance* is the mean squared deviation of the array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f04f8ccb",
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_var(arr):\n",
    "    ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ff1b24ac",
   "metadata": {},
   "source": [
    "Test your function with these print statements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f9c43971",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(my_var(test1), old_np.var(test1))\n",
    "print(my_var(test2), old_np.var(test2))\n",
    "print(my_var(test3), old_np.var(test3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2f476ede",
   "metadata": {},
   "source": [
    "Write a function `my_std` to calculate the *standard deviation* of\n",
    "`arr`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1bfdefd6",
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_std(arr):\n",
    "    ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad82bd5f",
   "metadata": {},
   "source": [
    "Test your function with these print statements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5662c5ac",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(my_std(test1), old_np.std(test1))\n",
    "print(my_std(test2), old_np.std(test2))\n",
    "print(my_std(test3), old_np.std(test3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2a155b11",
   "metadata": {},
   "source": [
    "Now write a function `my_standard_units` that returns an array `arr` in standard units:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fab6f768",
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_standard_units(arr):\n",
    "    ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "14b9ef0f",
   "metadata": {},
   "source": [
    "This prints out the mean and standard deviation of your output arrays.\n",
    "The mean should be very close to 0, and the standard deviation should be\n",
    "very close to 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5cc820d9",
   "metadata": {},
   "outputs": [],
   "source": [
    "su1 = my_standard_units(test1)\n",
    "print(my_mean(su1), my_std(su1))\n",
    "su2 = my_standard_units(test2)\n",
    "print(my_mean(su2), my_std(su2))\n",
    "su3 = my_standard_units(test3)\n",
    "print(my_mean(su3), my_std(su3))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a1a5077",
   "metadata": {},
   "source": [
    "## Sample variance, sample standard deviation\n",
    "\n",
    "Call the sum of squared deviations in an array $SS$.  Call the number of\n",
    "elements in an array $n$.\n",
    "\n",
    "The definition we have used so far for the variance is the mean of the\n",
    "(squared deviations from the mean).  Because of the definition of the\n",
    "mean, could also write this as $SS / n$.\n",
    "\n",
    "In fact there are two common definitions for variance.  The definition\n",
    "above is often called the *population variance*.  Another common\n",
    "definition of variance is the *sample variance*. See [this Wikipedia\n",
    "section on\n",
    "variance](https://en.wikipedia.org/wiki/Variance#Population_variance_and_sample_variance)\n",
    "for more detail.\n",
    "\n",
    "The *sample variance* a very slight tweak to the population variance.\n",
    "It is the (sum of the squared deviation from the mean) divided by (the\n",
    "number of elements minus 1).  This is $SS / (n - 1)$ in the symbols\n",
    "above.\n",
    "\n",
    "The matching *sample standard deviation* is the square root of the\n",
    "sample variance.\n",
    "\n",
    "Make a function `my_sample_var` to calculate the *sample variance*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "45bf7668",
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_sample_var(arr):\n",
    "    ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5dc92a0a",
   "metadata": {},
   "source": [
    "Test your function with these print statements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4ed14170",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(my_sample_var(test1), old_np.var(test1, ddof=1))\n",
    "print(my_sample_var(test2), old_np.var(test2, ddof=1))\n",
    "print(my_sample_var(test3), old_np.var(test3, ddof=1))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ee9c4edf",
   "metadata": {},
   "source": [
    "Make a function `my_sample_std` to calculate the *sample standard\n",
    "deviation*:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c6e66b3a",
   "metadata": {},
   "outputs": [],
   "source": [
    "def my_sample_std(arr):\n",
    "    ..."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31f7c2f1",
   "metadata": {},
   "source": [
    "Test your function with these print statements."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e2b44df8",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(my_sample_std(test1), old_np.std(test1, ddof=1))\n",
    "print(my_sample_std(test2), old_np.std(test2, ddof=1))\n",
    "print(my_sample_std(test3), old_np.std(test3, ddof=1))"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "split_at_heading": true,
   "text_representation": {
    "extension": ".Rmd",
    "format_name": "rmarkdown",
    "format_version": "1.1",
    "jupytext_version": "1.2.4"
   }
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}