00-linear-regression.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Mechanics of Linear Regression\n",
    "\n",
    "#### Equation of a Simple Linear Regression\n",
    "\n",
    "For a simple linear regression with one independent variable (X) and one dependent variable (Y), the equation takes the form:\n",
    "\n",
    "Y = β₀ + β₁X + ε\n",
    "\n",
    "\n",
    "- Y is the dependent variable.\n",
    "- X is the independent variable.\n",
    "- β₀ is the intercept (where the line intercepts the y-axis when X is zero).\n",
    "- β₁ is the slope (how much Y changes when X changes by one unit).\n",
    "- ε represents the error term (the difference between the observed Y and the predicted Y).\n",
    "\n",
    "### Fitting the Line\n",
    "\n",
    "The goal of linear regression is to find the best-fitting line that minimizes the difference between the actual Y values and the predicted Y values (the line of best fit). This is often done by minimizing the sum of the squared differences between the observed Y values and the values predicted by the linear equation.\n",
    "\n",
    "### Interpreting Coefficients\n",
    "\n",
    "The coefficients (β₀ and β₁) quantify the relationship between the independent and dependent variables.\n",
    "\n",
    "- β₀ represents the expected value of Y when X is zero.\n",
    "- β₁ represents the change in Y for a one-unit change in X.\n",
    "\n",
    "### Assumptions\n",
    "\n",
    "Linear regression assumes:\n",
    "\n",
    "- A linear relationship between the variables.\n",
    "- Normally distributed residuals with constant variance (homoscedasticity).\n",
    "- Independence of observations (no autocorrelation between residuals).\n",
    "\n",
    "## Interpreting Results\n",
    "\n",
    "### Coefficient Sign and Magnitude\n",
    "\n",
    "- A positive β₁ indicates a positive relationship between X and Y, while a negative β₁ indicates an inverse relationship.\n",
    "- The magnitude of β₁ determines the steepness of the relationship.\n",
    "\n",
    "### R-squared (R²) Value\n",
    "\n",
    "- R-squared measures the proportion of variation in the dependent variable explained by the independent variables.\n",
    "- Higher values indicate a better fit of the model to the data (ranges from 0 to 1).\n",
    "\n",
    "### P-values and Confidence Intervals\n",
    "\n",
    "- P-values associated with coefficients help determine their significance (< 0.05 indicates more significance).\n",
    "- Confidence intervals provide a range within which the true population parameter is likely to fall.\n",
    "\n",
    "### Residual Analysis\n",
    "\n",
    "- Examining residuals helps verify assumptions. Patterns in residuals against predicted values or independent variables may indicate violations of assumptions.\n",
    "\n",
    "Understanding the mechanics and interpreting the results of linear regression are crucial for drawing meaningful insights from data and making predictions based on the relationships between variables.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Import packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "import pandas as pd\n",
    "from sklearn import linear_model\n",
    "from sklearn.metrics import mean_absolute_error\n",
    "import matplotlib.pyplot as plt\n",
    "from scipy.stats import pearsonr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "url = 'http://wiki.stat.ucla.edu/socr/index.php/SOCR_Data_Dinov_020108_HeightsWeights'\n",
    "height_weight_df = pd.read_html(url)[1][['Height(Inches)','Weight(Pounds)']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n"
     ]
    }
   ],
   "source": [
    "num_records = len(height_weight_df)\n",
    "print(num_records)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "x = height_weight_df['Height(Inches)'].values.reshape(num_records, 1)\n",
    "y = height_weight_df['Weight(Pounds)'].values.reshape(num_records, 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = linear_model.LinearRegression().fit(x,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "ŷ = -106.02770644878126 + 3.432676129271628 x₁\n"
     ]
    }
   ],
   "source": [
    "print(\"ŷ = \" + str(model.intercept_[0]) + \" + \" + str(model.coef_.T[0][0]) + \" x₁\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "illm",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}