Reggie_Linear_Regression_Skeleton.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Project: Linear Regression\n",
    "\n",
    "Reggie is a mad scientist who has been hired by the local fast food joint to build their newest ball pit in the play area. As such, he is working on researching the bounciness of different balls so as to optimize the pit. He is running an experiment to bounce different sizes of bouncy balls, and then fitting lines to the data points he records. He has heard of linear regression, but needs your help to implement a version of linear regression in Python.\n",
    "\n",
    "_Linear Regression_ is when you have a group of points on a graph, and you find a line that approximately resembles that group of points. A good Linear Regression algorithm minimizes the _error_, or the distance from each point to the line. A line with the least error is the line that fits the data the best. We call this a line of _best fit_.\n",
    "\n",
    "We will use loops, lists, and arithmetic to create a function that will find a line of best fit when given a set of data.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1: Calculating Error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "The line we will end up with will have a formula that looks like:\n",
    "```\n",
    "y = m*x + b\n",
    "```\n",
    "`m` is the slope of the line and `b` is the intercept, where the line crosses the y-axis.\n",
    "\n",
    "Fill in the function called `get_y()` that takes in `m`, `b`, and `x`. It should return what the `y` value would be for that `x` on that line!\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "True\n",
      "True\n"
     ]
    }
   ],
   "source": [
    "def get_y(m, b, x):\n",
    "  y = m * x + b\n",
    "  return y\n",
    "\n",
    "print(get_y(1, 0, 7) == 7)\n",
    "print(get_y(5, 10, 3) == 25)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "Reggie wants to try a bunch of different `m` values and `b` values and see which line produces the least error. To calculate error between a point and a line, he wants a function called `calculate_error()`, which will take in `m`, `b`, and an [x, y] point called `point` and return the distance between the line and the point.\n",
    "\n",
    "To find the distance:\n",
    "1. Get the x-value from the point and store it in a variable called `x_point`\n",
    "2. Get the y-value from the point and store it in a variable called `y_point`\n",
    "3. Use `get_y()` to get the y-value that `x_point` would be on the line\n",
    "4. Find the difference between the y from `get_y` and `y_point`\n",
    "5. Return the absolute value of the distance (you can use the built-in function `abs()` to do this)\n",
    "\n",
    "The distance represents the error between the line `y = m*x + b` and the `point` given.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Write your calculate_error() function here\n",
    "\n",
    "def calculate_error(m, b, point):\n",
    "    [x_point, y_point] = point\n",
    "    y = m * x_point + b\n",
    "    distance = abs(y - y_point)\n",
    "    return distance\n",
    "\n",
    "calculate_error(1, 0, [3,3])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's test this function!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "1\n",
      "1\n",
      "5\n"
     ]
    }
   ],
   "source": [
    "#this is a line that looks like y = x, so (3, 3) should lie on it. thus, error should be 0:\n",
    "print(calculate_error(1, 0, (3, 3)))\n",
    "#the point (3, 4) should be 1 unit away from the line y = x:\n",
    "print(calculate_error(1, 0, (3, 4)))\n",
    "#the point (3, 3) should be 1 unit away from the line y = x - 1:\n",
    "print(calculate_error(1, -1, (3, 3)))\n",
    "#the point (3, 3) should be 5 units away from the line y = -x + 1:\n",
    "print(calculate_error(-1, 1, (3, 3)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "Great! Reggie's datasets will be sets of points. For example, he ran an experiment comparing the width of bouncy balls to how high they bounce:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "datapoints = [(1, 2), (2, 0), (3, 4), (4, 4), (5, 3)]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "The first datapoint, `(1, 2)`, means that his 1cm bouncy ball bounced 2 meters. The 4cm bouncy ball bounced 4 meters.\n",
    "\n",
    "As we try to fit a line to this data, we will need a function called `calculate_all_error`, which takes `m` and `b` that describe a line, and `points`, a set of data like the example above.\n",
    "\n",
    "`calculate_all_error` should iterate through each `point` in `points` and calculate the error from that point to the line (using `calculate_error`). It should keep a running total of the error, and then return that total after the loop.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n"
     ]
    }
   ],
   "source": [
    "# Write your calculate_all_error function here\n",
    "\n",
    "def calculate_all_error(m, b, points):\n",
    "    total_error = 0\n",
    "    for point in datapoints:\n",
    "        point_error = calculate_error(m, b, point)\n",
    "        total_error += point_error\n",
    "    return total_error\n",
    "\n",
    "# Try with your own values:\n",
    "\n",
    "datapoints = [(1, 1), (3, 3), (5, 5), (-1, -1)]\n",
    "print(calculate_all_error(1, 0, datapoints))\n",
    "\n",
    "# Notice that the above prints out 0 which is to be expected. This is because all of the points in datapoints lie on the\n",
    "# line y = x (which has an m value of 1 and an intercept value of 0), and so the total error is 0.\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's test this function!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0\n",
      "4\n",
      "4\n",
      "18\n"
     ]
    }
   ],
   "source": [
    "#every point in this dataset lies upon y=x, so the total error should be zero:\n",
    "datapoints = [(1, 1), (3, 3), (5, 5), (-1, -1)]\n",
    "print(calculate_all_error(1, 0, datapoints))\n",
    "\n",
    "#every point in this dataset is 1 unit away from y = x + 1, so the total error should be 4:\n",
    "datapoints = [(1, 1), (3, 3), (5, 5), (-1, -1)]\n",
    "print(calculate_all_error(1, 1, datapoints))\n",
    "\n",
    "#every point in this dataset is 1 unit away from y = x - 1, so the total error should be 4:\n",
    "datapoints = [(1, 1), (3, 3), (5, 5), (-1, -1)]\n",
    "print(calculate_all_error(1, -1, datapoints))\n",
    "\n",
    "\n",
    "#the points in this dataset are 1, 5, 9, and 3 units away from y = -x + 1, respectively, so total error should be\n",
    "# 1 + 5 + 9 + 3 = 18\n",
    "datapoints = [(1, 1), (3, 3), (5, 5), (-1, -1)]\n",
    "print(calculate_all_error(-1, 1, datapoints))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Great! It looks like we now have a function that can take in a line and Reggie's data and return how much error that line produces when we try to fit it to the data.\n",
    "\n",
    "Our next step is to find the `m` and `b` that minimizes this error, and thus fits the data best!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2: Try a bunch of slopes and intercepts!\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The way Reggie wants to find a line of best fit is by trial and error. He wants to try a bunch of different slopes (`m` values) and a bunch of different intercepts (`b` values) and see which one produces the smallest error value for his dataset.\n",
    "\n",
    "Using a list comprehension, let's create a list of possible `m` values to try. Make the list `possible_ms` that goes from -10 to 10 inclusive, in increments of 0.1.\n",
    "\n",
    "Hint (to view this hint, either double-click this cell or highlight the following white space): <font color=\"white\">you can go through the values in range(-100, 100) and multiply each one by 0.1</font>\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-10.0, -9.9, -9.8, -9.700000000000001, -9.600000000000001, -9.5, -9.4, -9.3, -9.200000000000001, -9.1, -9.0, -8.9, -8.8, -8.700000000000001, -8.6, -8.5, -8.4, -8.3, -8.200000000000001, -8.1, -8.0, -7.9, -7.800000000000001, -7.7, -7.6000000000000005, -7.5, -7.4, -7.300000000000001, -7.2, -7.1000000000000005, -7.0, -6.9, -6.800000000000001, -6.7, -6.6000000000000005, -6.5, -6.4, -6.300000000000001, -6.2, -6.1000000000000005, -6.0, -5.9, -5.800000000000001, -5.7, -5.6000000000000005, -5.5, -5.4, -5.300000000000001, -5.2, -5.1000000000000005, -5.0, -4.9, -4.800000000000001, -4.7, -4.6000000000000005, -4.5, -4.4, -4.3, -4.2, -4.1000000000000005, -4.0, -3.9000000000000004, -3.8000000000000003, -3.7, -3.6, -3.5, -3.4000000000000004, -3.3000000000000003, -3.2, -3.1, -3.0, -2.9000000000000004, -2.8000000000000003, -2.7, -2.6, -2.5, -2.4000000000000004, -2.3000000000000003, -2.2, -2.1, -2.0, -1.9000000000000001, -1.8, -1.7000000000000002, -1.6, -1.5, -1.4000000000000001, -1.3, -1.2000000000000002, -1.1, -1.0, -0.9, -0.8, -0.7000000000000001, -0.6000000000000001, -0.5, -0.4, -0.30000000000000004, -0.2, -0.1, 0.0, 0.1, 0.2, 0.30000000000000004, 0.4, 0.5, 0.6000000000000001, 0.7000000000000001, 0.8, 0.9, 1.0, 1.1, 1.2000000000000002, 1.3, 1.4000000000000001, 1.5, 1.6, 1.7000000000000002, 1.8, 1.9000000000000001, 2.0, 2.1, 2.2, 2.3000000000000003, 2.4000000000000004, 2.5, 2.6, 2.7, 2.8000000000000003, 2.9000000000000004, 3.0, 3.1, 3.2, 3.3000000000000003, 3.4000000000000004, 3.5, 3.6, 3.7, 3.8000000000000003, 3.9000000000000004, 4.0, 4.1000000000000005, 4.2, 4.3, 4.4, 4.5, 4.6000000000000005, 4.7, 4.800000000000001, 4.9, 5.0, 5.1000000000000005, 5.2, 5.300000000000001, 5.4, 5.5, 5.6000000000000005, 5.7, 5.800000000000001, 5.9, 6.0, 6.1000000000000005, 6.2, 6.300000000000001, 6.4, 6.5, 6.6000000000000005, 6.7, 6.800000000000001, 6.9, 7.0, 7.1000000000000005, 7.2, 7.300000000000001, 7.4, 7.5, 7.6000000000000005, 7.7, 7.800000000000001, 7.9, 8.0, 8.1, 8.200000000000001, 8.3, 8.4, 8.5, 8.6, 8.700000000000001, 8.8, 8.9, 9.0, 9.1, 9.200000000000001, 9.3, 9.4, 9.5, 9.600000000000001, 9.700000000000001, 9.8, 9.9, 10.0]\n"
     ]
    }
   ],
   "source": [
    "possible_ms = [m * 0.1 for m in range(-100,101)]\n",
    "print(possible_ms)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's make a list of `possible_bs` to check that would be the values from -20 to 20 inclusive, in steps of 0.1:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[-20.0, -19.900000000000002, -19.8, -19.700000000000003, -19.6, -19.5, -19.400000000000002, -19.3, -19.200000000000003, -19.1, -19.0, -18.900000000000002, -18.8, -18.7, -18.6, -18.5, -18.400000000000002, -18.3, -18.2, -18.1, -18.0, -17.900000000000002, -17.8, -17.7, -17.6, -17.5, -17.400000000000002, -17.3, -17.2, -17.1, -17.0, -16.900000000000002, -16.8, -16.7, -16.6, -16.5, -16.400000000000002, -16.3, -16.2, -16.1, -16.0, -15.9, -15.8, -15.700000000000001, -15.600000000000001, -15.5, -15.4, -15.3, -15.200000000000001, -15.100000000000001, -15.0, -14.9, -14.8, -14.700000000000001, -14.600000000000001, -14.5, -14.4, -14.3, -14.200000000000001, -14.100000000000001, -14.0, -13.9, -13.8, -13.700000000000001, -13.600000000000001, -13.5, -13.4, -13.3, -13.200000000000001, -13.100000000000001, -13.0, -12.9, -12.8, -12.700000000000001, -12.600000000000001, -12.5, -12.4, -12.3, -12.200000000000001, -12.100000000000001, -12.0, -11.9, -11.8, -11.700000000000001, -11.600000000000001, -11.5, -11.4, -11.3, -11.200000000000001, -11.100000000000001, -11.0, -10.9, -10.8, -10.700000000000001, -10.600000000000001, -10.5, -10.4, -10.3, -10.200000000000001, -10.100000000000001, -10.0, -9.9, -9.8, -9.700000000000001, -9.600000000000001, -9.5, -9.4, -9.3, -9.200000000000001, -9.1, -9.0, -8.9, -8.8, -8.700000000000001, -8.6, -8.5, -8.4, -8.3, -8.200000000000001, -8.1, -8.0, -7.9, -7.800000000000001, -7.7, -7.6000000000000005, -7.5, -7.4, -7.300000000000001, -7.2, -7.1000000000000005, -7.0, -6.9, -6.800000000000001, -6.7, -6.6000000000000005, -6.5, -6.4, -6.300000000000001, -6.2, -6.1000000000000005, -6.0, -5.9, -5.800000000000001, -5.7, -5.6000000000000005, -5.5, -5.4, -5.300000000000001, -5.2, -5.1000000000000005, -5.0, -4.9, -4.800000000000001, -4.7, -4.6000000000000005, -4.5, -4.4, -4.3, -4.2, -4.1000000000000005, -4.0, -3.9000000000000004, -3.8000000000000003, -3.7, -3.6, -3.5, -3.4000000000000004, -3.3000000000000003, -3.2, -3.1, -3.0, -2.9000000000000004, -2.8000000000000003, -2.7, -2.6, -2.5, -2.4000000000000004, -2.3000000000000003, -2.2, -2.1, -2.0, -1.9000000000000001, -1.8, -1.7000000000000002, -1.6, -1.5, -1.4000000000000001, -1.3, -1.2000000000000002, -1.1, -1.0, -0.9, -0.8, -0.7000000000000001, -0.6000000000000001, -0.5, -0.4, -0.30000000000000004, -0.2, -0.1, 0.0, 0.1, 0.2, 0.30000000000000004, 0.4, 0.5, 0.6000000000000001, 0.7000000000000001, 0.8, 0.9, 1.0, 1.1, 1.2000000000000002, 1.3, 1.4000000000000001, 1.5, 1.6, 1.7000000000000002, 1.8, 1.9000000000000001, 2.0, 2.1, 2.2, 2.3000000000000003, 2.4000000000000004, 2.5, 2.6, 2.7, 2.8000000000000003, 2.9000000000000004, 3.0, 3.1, 3.2, 3.3000000000000003, 3.4000000000000004, 3.5, 3.6, 3.7, 3.8000000000000003, 3.9000000000000004, 4.0, 4.1000000000000005, 4.2, 4.3, 4.4, 4.5, 4.6000000000000005, 4.7, 4.800000000000001, 4.9, 5.0, 5.1000000000000005, 5.2, 5.300000000000001, 5.4, 5.5, 5.6000000000000005, 5.7, 5.800000000000001, 5.9, 6.0, 6.1000000000000005, 6.2, 6.300000000000001, 6.4, 6.5, 6.6000000000000005, 6.7, 6.800000000000001, 6.9, 7.0, 7.1000000000000005, 7.2, 7.300000000000001, 7.4, 7.5, 7.6000000000000005, 7.7, 7.800000000000001, 7.9, 8.0, 8.1, 8.200000000000001, 8.3, 8.4, 8.5, 8.6, 8.700000000000001, 8.8, 8.9, 9.0, 9.1, 9.200000000000001, 9.3, 9.4, 9.5, 9.600000000000001, 9.700000000000001, 9.8, 9.9, 10.0, 10.100000000000001, 10.200000000000001, 10.3, 10.4, 10.5, 10.600000000000001, 10.700000000000001, 10.8, 10.9, 11.0, 11.100000000000001, 11.200000000000001, 11.3, 11.4, 11.5, 11.600000000000001, 11.700000000000001, 11.8, 11.9, 12.0, 12.100000000000001, 12.200000000000001, 12.3, 12.4, 12.5, 12.600000000000001, 12.700000000000001, 12.8, 12.9, 13.0, 13.100000000000001, 13.200000000000001, 13.3, 13.4, 13.5, 13.600000000000001, 13.700000000000001, 13.8, 13.9, 14.0, 14.100000000000001, 14.200000000000001, 14.3, 14.4, 14.5, 14.600000000000001, 14.700000000000001, 14.8, 14.9, 15.0, 15.100000000000001, 15.200000000000001, 15.3, 15.4, 15.5, 15.600000000000001, 15.700000000000001, 15.8, 15.9, 16.0, 16.1, 16.2, 16.3, 16.400000000000002, 16.5, 16.6, 16.7, 16.8, 16.900000000000002, 17.0, 17.1, 17.2, 17.3, 17.400000000000002, 17.5, 17.6, 17.7, 17.8, 17.900000000000002, 18.0, 18.1, 18.2, 18.3, 18.400000000000002, 18.5, 18.6, 18.7, 18.8, 18.900000000000002, 19.0, 19.1, 19.200000000000003, 19.3, 19.400000000000002, 19.5, 19.6, 19.700000000000003, 19.8, 19.900000000000002, 20.0]\n"
     ]
    }
   ],
   "source": [
    "possible_bs = [b * 0.1 for b in range(-200, 201)]\n",
    "print(possible_bs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are going to find the smallest error. First, we will make every possible `y = m*x + b` line by pairing all of the possible `m`s with all of the possible `b`s. Then, we will see which `y = m*x + b` line produces the smallest total error with the set of data stored in `datapoint`.\n",
    "\n",
    "First, create the variables that we will be optimizing:\n",
    "* `smallest_error` &mdash; this should start at infinity (`float(\"inf\")`) so that any error we get at first will be smaller than our value of `smallest_error`\n",
    "* `best_m` &mdash; we can start this at `0`\n",
    "* `best_b` &mdash; we can start this at `0`\n",
    "\n",
    "We want to:\n",
    "* Iterate through each element `m` in `possible_ms`\n",
    "* For every `m` value, take every `b` value in `possible_bs`\n",
    "* If the value returned from `calculate_all_error` on this `m` value, this `b` value, and `datapoints` is less than our current `smallest_error`,\n",
    "* Set `best_m` and `best_b` to be these values, and set `smallest_error` to this error.\n",
    "\n",
    "By the end of these nested loops, the `smallest_error` should hold the smallest error we have found, and `best_m` and `best_b` should be the values that produced that smallest error value.\n",
    "\n",
    "Print out `best_m`, `best_b` and `smallest_error` after the loops.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.30000000000000004 1.7000000000000002 4.999999999999999\n"
     ]
    }
   ],
   "source": [
    "datapoints = [(1, 2), (2, 0), (3, 4), (4, 4), (5, 3)]\n",
    "smallest_error = float(\"inf\")\n",
    "best_m = 0\n",
    "best_b = 0\n",
    "\n",
    "for m in possible_ms:\n",
    "    for b in possible_bs:\n",
    "        error = calculate_all_error(m, b, datapoints)\n",
    "        if error < smallest_error:\n",
    "            best_m = m\n",
    "            best_b = b\n",
    "            smallest_error = error\n",
    "\n",
    "print(best_m, best_b, smallest_error)\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3: What does our model predict?\n",
    "\n",
    "Now we have seen that for this set of observations on the bouncy balls, the line that fits the data best has an `m` of 0.3 and a `b` of 1.7:\n",
    "\n",
    "```\n",
    "y = 0.3x + 1.7\n",
    "```\n",
    "\n",
    "This line produced a total error of 5.\n",
    "\n",
    "Using this `m` and this `b`, what does your line predict the bounce height of a ball with a width of 6 to be?\n",
    "In other words, what is the output of `get_y()` when we call it with:\n",
    "* m = 0.3\n",
    "* b = 1.7\n",
    "* x = 6"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3.5"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "get_y(0.3, 1.7, 6)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Our model predicts that the 6cm ball will bounce 3.5m.\n",
    "\n",
    "Now, Reggie can use this model to predict the bounce of all kinds of sizes of balls he may choose to include in the ball pit!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}