Chapter_05.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Python solutions to Gary Koop's \"Analysis of Economic Data\" (4th ed., 2013)**\n",
    "\n",
    "**Chapter 5: Statistical Aspects of Regression**\n",
    "\n",
    "Jakub Janus, [jakub.janus@uek.krakow.pl](jakub.janus@uek.krakow.pl)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercise 5.1\n",
    "The data sets used to calculate Figures 5.1, 5.2, 5.3 and 5.4 are in FIG51.XLS, FIG52.XLS, FIG53.XLS and FIG54.XLS. \n",
    "\n",
    "(a) Calculate the OLS estimates alpha and beta for these four data sets. How close are they to 0 and 1 (the values we used to artificially simulate the data)?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ],
      "text/html": "<table style=\"text-align:center\"><tr><td colspan=\"5\" style=\"border-bottom: 1px solid black\"></td></tr><tr><td style=\"text-align:left\"></td><td colspan=\"4\"><em>Dependent variable:</em></td></tr><tr><td style=\"text-align:left\"></td><tr><td style=\"text-align:left\"></td><td>(1)</td><td>(2)</td><td>(3)</td><td>(4)</td></tr><tr><td colspan=\"5\" style=\"border-bottom: 1px solid black\"></td></tr><tr><td style=\"text-align:left\">X</td><td>0.911<sup></sup></td><td>1.039<sup>***</sup></td><td>1.003<sup>***</sup></td><td>1.515<sup></sup></td></tr><tr><td style=\"text-align:left\"></td><td>(0.779)</td><td>(0.172)</td><td>(0.009)</td><td>(1.712)</td></tr><tr><td style=\"text-align:left\">const</td><td>-0.942<sup></sup></td><td>-0.132<sup></sup></td><td>-0.008<sup></sup></td><td>-1.552<sup></sup></td></tr><tr><td style=\"text-align:left\"></td><td>(1.928)</td><td>(0.556)</td><td>(0.028)</td><td>(5.147)</td></tr><td colspan=\"5\" style=\"border-bottom: 1px solid black\"></td></tr><tr><td style=\"text-align: left\">Observations</td><td>5.0</td><td>100.0</td><td>100.0</td><td>100.0</td></tr><tr><td style=\"text-align: left\">R<sup>2</sup></td><td>0.313</td><td>0.271</td><td>0.993</td><td>0.008</td></tr><tr><td style=\"text-align: left\">Adjusted R<sup>2</sup></td><td>0.084</td><td>0.264</td><td>0.993</td><td>-0.002</td></tr><tr><td style=\"text-align: left\">Residual Std. Error</td><td>2.289(df = 3.0)</td><td>1.937(df = 98.0)</td><td>0.097(df = 98.0)</td><td>1.94(df = 98.0)</td></tr><tr><td style=\"text-align: left\">F Statistic</td><td>1.366<sup></sup>(df = 1.0; 3.0)</td><td>36.523<sup>***</sup>(df = 1.0; 98.0)</td><td>13721.34<sup>***</sup>(df = 1.0; 98.0)</td><td>0.784<sup></sup>(df = 1.0; 98.0)</td></tr><tr><td colspan=\"5\" style=\"border-bottom: 1px solid black\"></td></tr><tr><td style=\"text-align: left\">Note:</td>\n <td colspan=\"4\" style=\"text-align: right\">\n  <sup>*</sup>p&lt;0.1;\n  <sup>**</sup>p&lt;0.05;\n  <sup>***</sup>p&lt;0.01\n </td></tr></table>"
     },
     "metadata": {},
     "execution_count": 1
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import statsmodels.api as sm\n",
    "from stargazer.stargazer import Stargazer\n",
    "from IPython.core.display import HTML\n",
    "# fig51\n",
    "fig51 = pd.read_excel(r'datasets\\FIG51.xls')\n",
    "formula_fig51 = sm.OLS(fig51['Y'], sm.add_constant(fig51['X']))\n",
    "reg_fig51 = formula_fig51.fit()\n",
    "# fig52\n",
    "fig52 = pd.read_excel(r'datasets\\FIG52.xls')\n",
    "formula_fig52 = sm.OLS(fig52['Y'], sm.add_constant(fig52['X']))\n",
    "reg_fig52 = formula_fig52.fit()\n",
    "# fig53\n",
    "fig53 = pd.read_excel(r'datasets\\FIG53.xls')\n",
    "formula_fig53 = sm.OLS(fig53['Y'], sm.add_constant(fig53['X']))\n",
    "reg_fig53 = formula_fig53.fit()\n",
    "# fig54\n",
    "fig54 = pd.read_excel(r'datasets\\FIG54.xls')\n",
    "formula_fig54 = sm.OLS(fig54['Y'], sm.add_constant(fig54['X']))\n",
    "reg_fig54 = formula_fig54.fit()\n",
    "# Comparison\n",
    "stargazer_fig51_fig52_fig53_fig54 = Stargazer([reg_fig51, reg_fig52, reg_fig53, reg_fig54])\n",
    "HTML(stargazer_fig51_fig52_fig53_fig54.render_html())"
   ]
  },
  {
   "source": [
    "(b) Calculate confidence intervals for $\\hat{\\alpha}$ for the four data sets. Examine how the\n",
    "width of the confidence interval relates to N and the variability of the errors."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                  down        up\n",
       "ci_fig51_95  -7.078510  5.195277\n",
       "ci_fig52_95  -1.235726  0.971559\n",
       "ci_fig53_95  -0.063201  0.047175\n",
       "ci_fig54_95 -11.765251  8.661804"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>down</th>\n      <th>up</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>ci_fig51_95</th>\n      <td>-7.078510</td>\n      <td>5.195277</td>\n    </tr>\n    <tr>\n      <th>ci_fig52_95</th>\n      <td>-1.235726</td>\n      <td>0.971559</td>\n    </tr>\n    <tr>\n      <th>ci_fig53_95</th>\n      <td>-0.063201</td>\n      <td>0.047175</td>\n    </tr>\n    <tr>\n      <th>ci_fig54_95</th>\n      <td>-11.765251</td>\n      <td>8.661804</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 2
    }
   ],
   "source": [
    "ci_fig51_95 = reg_fig51.conf_int(alpha = 0.05).iloc[0:1]\n",
    "ci_fig51_95 = ci_fig51_95.rename(index = {\"const\": \"ci_fig51_95\"})\n",
    "ci_fig52_95 = reg_fig52.conf_int(alpha = 0.05).iloc[0:1]\n",
    "ci_fig52_95 = ci_fig52_95.rename(index = {\"const\": \"ci_fig52_95\"})\n",
    "ci_fig53_95 = reg_fig53.conf_int(alpha = 0.05).iloc[0:1]\n",
    "ci_fig53_95 = ci_fig53_95.rename(index = {\"const\": \"ci_fig53_95\"})\n",
    "ci_fig54_95 = reg_fig54.conf_int(alpha = 0.05).iloc[0:1]\n",
    "ci_fig54_95 = ci_fig54_95.rename(index = {\"const\": \"ci_fig54_95\"})\n",
    "# Concatenate all\n",
    "ci_all_95 = [ci_fig51_95, ci_fig52_95, ci_fig53_95, ci_fig54_95]\n",
    "df_ci_all_95 = pd.concat(ci_all_95)\n",
    "df_ci_all_95.rename(columns = {0: \"down\", 1: \"up\"})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "(c) Calculate 99% and 90% confidence intervals for the data sets. How do these\n",
    "differ from the 95% confidence intervals in part (b)?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                  down         up\n",
       "ci_fig51_99 -12.204977  10.321744\n",
       "ci_fig52_99  -1.593026   1.328858\n",
       "ci_fig53_99  -0.081068   0.065042\n",
       "ci_fig54_99 -15.071841  11.968394"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>down</th>\n      <th>up</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>ci_fig51_99</th>\n      <td>-12.204977</td>\n      <td>10.321744</td>\n    </tr>\n    <tr>\n      <th>ci_fig52_99</th>\n      <td>-1.593026</td>\n      <td>1.328858</td>\n    </tr>\n    <tr>\n      <th>ci_fig53_99</th>\n      <td>-0.081068</td>\n      <td>0.065042</td>\n    </tr>\n    <tr>\n      <th>ci_fig54_99</th>\n      <td>-15.071841</td>\n      <td>11.968394</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 3
    }
   ],
   "source": [
    "# 99% confidence intervals\n",
    "ci_fig51_99 = reg_fig51.conf_int(alpha = 0.01).iloc[0:1]\n",
    "ci_fig51_99 = ci_fig51_99.rename(index = {\"const\": \"ci_fig51_99\"})\n",
    "ci_fig52_99 = reg_fig52.conf_int(alpha = 0.01).iloc[0:1]\n",
    "ci_fig52_99 = ci_fig52_99.rename(index = {\"const\": \"ci_fig52_99\"})\n",
    "ci_fig53_99 = reg_fig53.conf_int(alpha = 0.01).iloc[0:1]\n",
    "ci_fig53_99 = ci_fig53_99.rename(index = {\"const\": \"ci_fig53_99\"})\n",
    "ci_fig54_99 = reg_fig54.conf_int(alpha = 0.01).iloc[0:1]\n",
    "ci_fig54_99 = ci_fig54_99.rename(index = {\"const\": \"ci_fig54_99\"})\n",
    "# Concatenate all\n",
    "ci_all_99 = [ci_fig51_99, ci_fig52_99, ci_fig53_99, ci_fig54_99]\n",
    "df_ci_all_99 = pd.concat(ci_all_99)\n",
    "df_ci_all_99.rename(columns = {0: \"down\", 1: \"up\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                  down        up\n",
       "ci_fig51_90  -5.479742  3.596509\n",
       "ci_fig52_90  -1.055584  0.791416\n",
       "ci_fig53_90  -0.054193  0.038167\n",
       "ci_fig54_90 -10.098140  6.994693"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>down</th>\n      <th>up</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>ci_fig51_90</th>\n      <td>-5.479742</td>\n      <td>3.596509</td>\n    </tr>\n    <tr>\n      <th>ci_fig52_90</th>\n      <td>-1.055584</td>\n      <td>0.791416</td>\n    </tr>\n    <tr>\n      <th>ci_fig53_90</th>\n      <td>-0.054193</td>\n      <td>0.038167</td>\n    </tr>\n    <tr>\n      <th>ci_fig54_90</th>\n      <td>-10.098140</td>\n      <td>6.994693</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 4
    }
   ],
   "source": [
    "#90% confidence intervals\n",
    "ci_fig51_90 = reg_fig51.conf_int(alpha = 0.1).iloc[0:1]\n",
    "ci_fig51_90 = ci_fig51_90.rename(index = {\"const\": \"ci_fig51_90\"})\n",
    "ci_fig52_90 = reg_fig52.conf_int(alpha = 0.1).iloc[0:1]\n",
    "ci_fig52_90 = ci_fig52_90.rename(index = {\"const\": \"ci_fig52_90\"})\n",
    "ci_fig53_90 = reg_fig53.conf_int(alpha = 0.1).iloc[0:1]\n",
    "ci_fig53_90 = ci_fig53_90.rename(index = {\"const\": \"ci_fig53_90\"})\n",
    "ci_fig54_90 = reg_fig54.conf_int(alpha = 0.1).iloc[0:1]\n",
    "ci_fig54_90 = ci_fig54_90.rename(index = {\"const\": \"ci_fig54_90\"})\n",
    "# Concatenate all\n",
    "ci_all_90 = [ci_fig51_90, ci_fig52_90, ci_fig53_90, ci_fig54_90]\n",
    "df_ci_all_90 = pd.concat(ci_all_90)\n",
    "df_ci_all_90.rename(columns = {0: \"down\", 1: \"up\"})"
   ]
  },
  {
   "source": [
    "# Exercise 5.2\n",
    "The file ADVERT.XLS contains data on annual sales (Y) and advertising expenditure\n",
    "(X) (both measured in millions of dollars) for 84 companies in the USA.\n",
    "\n",
    "(a) Run a regression of Y on X and obtain 95% confidence intervals for\n",
    "α and β."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:              Sales$000   R-squared:                       0.089\n",
       "Model:                            OLS   Adj. R-squared:                  0.078\n",
       "Method:                 Least Squares   F-statistic:                     8.012\n",
       "Date:                Thu, 11 Feb 2021   Prob (F-statistic):            0.00584\n",
       "Time:                        14:41:50   Log-Likelihood:                -328.25\n",
       "No. Observations:                  84   AIC:                             660.5\n",
       "Df Residuals:                      82   BIC:                             665.4\n",
       "Df Model:                           1                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "const        502.9167      4.132    121.700      0.000     494.696     511.137\n",
       "Ads$000        0.2183      0.077      2.831      0.006       0.065       0.372\n",
       "==============================================================================\n",
       "Omnibus:                        4.165   Durbin-Watson:                   2.301\n",
       "Prob(Omnibus):                  0.125   Jarque-Bera (JB):                3.674\n",
       "Skew:                           0.326   Prob(JB):                        0.159\n",
       "Kurtosis:                       3.790   Cond. No.                         166.\n",
       "==============================================================================\n",
       "\n",
       "Warnings:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "\"\"\""
      ],
      "text/html": "<table class=\"simpletable\">\n<caption>OLS Regression Results</caption>\n<tr>\n  <th>Dep. Variable:</th>        <td>Sales$000</td>    <th>  R-squared:         </th> <td>   0.089</td>\n</tr>\n<tr>\n  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.078</td>\n</tr>\n<tr>\n  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   8.012</td>\n</tr>\n<tr>\n  <th>Date:</th>             <td>Thu, 11 Feb 2021</td> <th>  Prob (F-statistic):</th>  <td>0.00584</td>\n</tr>\n<tr>\n  <th>Time:</th>                 <td>14:41:50</td>     <th>  Log-Likelihood:    </th> <td> -328.25</td>\n</tr>\n<tr>\n  <th>No. Observations:</th>      <td>    84</td>      <th>  AIC:               </th> <td>   660.5</td>\n</tr>\n<tr>\n  <th>Df Residuals:</th>          <td>    82</td>      <th>  BIC:               </th> <td>   665.4</td>\n</tr>\n<tr>\n  <th>Df Model:</th>              <td>     1</td>      <th>                     </th>     <td> </td>   \n</tr>\n<tr>\n  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n</tr>\n</table>\n<table class=\"simpletable\">\n<tr>\n     <td></td>        <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n</tr>\n<tr>\n  <th>const</th>   <td>  502.9167</td> <td>    4.132</td> <td>  121.700</td> <td> 0.000</td> <td>  494.696</td> <td>  511.137</td>\n</tr>\n<tr>\n  <th>Ads$000</th> <td>    0.2183</td> <td>    0.077</td> <td>    2.831</td> <td> 0.006</td> <td>    0.065</td> <td>    0.372</td>\n</tr>\n</table>\n<table class=\"simpletable\">\n<tr>\n  <th>Omnibus:</th>       <td> 4.165</td> <th>  Durbin-Watson:     </th> <td>   2.301</td>\n</tr>\n<tr>\n  <th>Prob(Omnibus):</th> <td> 0.125</td> <th>  Jarque-Bera (JB):  </th> <td>   3.674</td>\n</tr>\n<tr>\n  <th>Skew:</th>          <td> 0.326</td> <th>  Prob(JB):          </th> <td>   0.159</td>\n</tr>\n<tr>\n  <th>Kurtosis:</th>      <td> 3.790</td> <th>  Cond. No.          </th> <td>    166.</td>\n</tr>\n</table><br/><br/>Warnings:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified."
     },
     "metadata": {},
     "execution_count": 5
    }
   ],
   "source": [
    "advert = pd.read_excel(r'datasets\\ADVERT.xls')\n",
    "formula_advert = sm.OLS(advert['Sales$000'], sm.add_constant(advert['Ads$000']))\n",
    "reg_advert = formula_advert.fit()\n",
    "reg_advert.summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "               down          up\n",
       "const    494.695983  511.137385\n",
       "Ads$000    0.064886    0.371742"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>down</th>\n      <th>up</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>const</th>\n      <td>494.695983</td>\n      <td>511.137385</td>\n    </tr>\n    <tr>\n      <th>Ads$000</th>\n      <td>0.064886</td>\n      <td>0.371742</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 6
    }
   ],
   "source": [
    "ci_advert_const = reg_advert.conf_int(alpha = 0.05)\n",
    "ci_advert_const.rename(columns = {0: \"down\", 1: \"up\"})"
   ]
  },
  {
   "source": [
    "(b) Write a sentence explaining verbally what the 95% confidence interval for β\n",
    "means in terms of the possible range of values that the effect of the explanatory\n",
    "variable on the dependent variable may take."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# An increase in adveritising expenditures by a million dollars raises the annual sales by from 0.065*1000000 = 65000 to 0.372*1000000 = 372000 dollars"
   ]
  },
  {
   "source": [
    "# Exercise 5.3\n",
    "The file ELECTRIC.XLS contains data on the costs of production (Y, measured\n",
    "in millions of dollars) and output (X, measured in thousands of kilowatt hours)\n",
    "for 123 electricity utility companies in the USA. Repeat Exercise 5.2 for this\n",
    "data set."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:                   Cost   R-squared:                       0.916\n",
       "Model:                            OLS   Adj. R-squared:                  0.916\n",
       "Method:                 Least Squares   F-statistic:                     1323.\n",
       "Date:                Thu, 11 Feb 2021   Prob (F-statistic):           5.36e-67\n",
       "Time:                        14:41:50   Log-Likelihood:                -517.88\n",
       "No. Observations:                 123   AIC:                             1040.\n",
       "Df Residuals:                     121   BIC:                             1045.\n",
       "Df Model:                           1                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "const          2.1866      1.879      1.163      0.247      -1.534       5.908\n",
       "Output         0.0048      0.000     36.376      0.000       0.005       0.005\n",
       "==============================================================================\n",
       "Omnibus:                       65.961   Durbin-Watson:                   1.559\n",
       "Prob(Omnibus):                  0.000   Jarque-Bera (JB):              859.400\n",
       "Skew:                           1.392   Prob(JB):                    2.42e-187\n",
       "Kurtosis:                      15.647   Cond. No.                     1.81e+04\n",
       "==============================================================================\n",
       "\n",
       "Warnings:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "[2] The condition number is large, 1.81e+04. This might indicate that there are\n",
       "strong multicollinearity or other numerical problems.\n",
       "\"\"\""
      ],
      "text/html": "<table class=\"simpletable\">\n<caption>OLS Regression Results</caption>\n<tr>\n  <th>Dep. Variable:</th>          <td>Cost</td>       <th>  R-squared:         </th> <td>   0.916</td>\n</tr>\n<tr>\n  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.916</td>\n</tr>\n<tr>\n  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   1323.</td>\n</tr>\n<tr>\n  <th>Date:</th>             <td>Thu, 11 Feb 2021</td> <th>  Prob (F-statistic):</th> <td>5.36e-67</td>\n</tr>\n<tr>\n  <th>Time:</th>                 <td>14:41:50</td>     <th>  Log-Likelihood:    </th> <td> -517.88</td>\n</tr>\n<tr>\n  <th>No. Observations:</th>      <td>   123</td>      <th>  AIC:               </th> <td>   1040.</td>\n</tr>\n<tr>\n  <th>Df Residuals:</th>          <td>   121</td>      <th>  BIC:               </th> <td>   1045.</td>\n</tr>\n<tr>\n  <th>Df Model:</th>              <td>     1</td>      <th>                     </th>     <td> </td>   \n</tr>\n<tr>\n  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n</tr>\n</table>\n<table class=\"simpletable\">\n<tr>\n     <td></td>       <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n</tr>\n<tr>\n  <th>const</th>  <td>    2.1866</td> <td>    1.879</td> <td>    1.163</td> <td> 0.247</td> <td>   -1.534</td> <td>    5.908</td>\n</tr>\n<tr>\n  <th>Output</th> <td>    0.0048</td> <td>    0.000</td> <td>   36.376</td> <td> 0.000</td> <td>    0.005</td> <td>    0.005</td>\n</tr>\n</table>\n<table class=\"simpletable\">\n<tr>\n  <th>Omnibus:</th>       <td>65.961</td> <th>  Durbin-Watson:     </th> <td>   1.559</td> \n</tr>\n<tr>\n  <th>Prob(Omnibus):</th> <td> 0.000</td> <th>  Jarque-Bera (JB):  </th> <td> 859.400</td> \n</tr>\n<tr>\n  <th>Skew:</th>          <td> 1.392</td> <th>  Prob(JB):          </th> <td>2.42e-187</td>\n</tr>\n<tr>\n  <th>Kurtosis:</th>      <td>15.647</td> <th>  Cond. No.          </th> <td>1.81e+04</td> \n</tr>\n</table><br/><br/>Warnings:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.<br/>[2] The condition number is large, 1.81e+04. This might indicate that there are<br/>strong multicollinearity or other numerical problems."
     },
     "metadata": {},
     "execution_count": 8
    }
   ],
   "source": [
    "electric = pd.read_excel(r'datasets\\ELECTRIC.xls').drop([0, 0]).reset_index()\n",
    "formula_electric = sm.OLS(electric['Cost'].astype(float), sm.add_constant(electric['Output'].astype(float)))\n",
    "reg_electric = formula_electric.fit()\n",
    "reg_electric.summary()"
   ]
  },
  {
   "source": [
    "# Exercise 5.4\n",
    "Using Table 5.2 (or running a regression yourself using data set FOREST.XLS),\n",
    "test the hypothesis that α = 0."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "<class 'statsmodels.stats.contrast.ContrastResults'>\n",
       "                             Test for Constraints                             \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "c0             0.6000      0.112      5.342      0.000       0.376       0.824\n",
       "=============================================================================="
      ]
     },
     "metadata": {},
     "execution_count": 9
    }
   ],
   "source": [
    "forest = pd.read_excel(r'datasets\\FOREST.xls')\n",
    "formula_tab_5_2 = sm.OLS(forest['Forest loss'], sm.add_constant(forest['Pop dens']))\n",
    "reg_tab_5_2 = formula_tab_5_2.fit()\n",
    "#reg_tab_5_2.summary(alpha = 0.05)\n",
    "hypothesis = 'const = 0'\n",
    "t_test = reg_tab_5_2.t_test(hypothesis)\n",
    "t_test"
   ]
  },
  {
   "source": [
    "# Exercise 5.5\n",
    "In addition to deforestation rate (Y), data set FOREST.XLS also contains data on\n",
    "W, the percentage increase in cropland (labeled “Crop ch”), and Z, the percentage\n",
    "change in pasture land (labeled “Pasture Ch”).\n",
    "\n",
    "(a) Run a regression of Y on W and interpret your results. Can you reject the\n",
    "hypothesis that expansion of cropland has an effect on deforestation rates?\n"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:            Forest loss   R-squared:                       0.003\n",
       "Model:                            OLS   Adj. R-squared:                 -0.012\n",
       "Method:                 Least Squares   F-statistic:                    0.1875\n",
       "Date:                Thu, 11 Feb 2021   Prob (F-statistic):              0.666\n",
       "Time:                        14:41:50   Log-Likelihood:                -93.509\n",
       "No. Observations:                  70   AIC:                             191.0\n",
       "Df Residuals:                      68   BIC:                             195.5\n",
       "Df Model:                           1                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "const          1.1792      0.146      8.086      0.000       0.888       1.470\n",
       "Crop ch       -0.0059      0.014     -0.433      0.666      -0.033       0.021\n",
       "==============================================================================\n",
       "Omnibus:                       42.041   Durbin-Watson:                   1.619\n",
       "Prob(Omnibus):                  0.000   Jarque-Bera (JB):              116.832\n",
       "Skew:                           1.945   Prob(JB):                     4.27e-26\n",
       "Kurtosis:                       7.993   Cond. No.                         14.1\n",
       "==============================================================================\n",
       "\n",
       "Warnings:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "\"\"\""
      ],
      "text/html": "<table class=\"simpletable\">\n<caption>OLS Regression Results</caption>\n<tr>\n  <th>Dep. Variable:</th>       <td>Forest loss</td>   <th>  R-squared:         </th> <td>   0.003</td>\n</tr>\n<tr>\n  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>  -0.012</td>\n</tr>\n<tr>\n  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>  0.1875</td>\n</tr>\n<tr>\n  <th>Date:</th>             <td>Thu, 11 Feb 2021</td> <th>  Prob (F-statistic):</th>  <td> 0.666</td> \n</tr>\n<tr>\n  <th>Time:</th>                 <td>14:41:50</td>     <th>  Log-Likelihood:    </th> <td> -93.509</td>\n</tr>\n<tr>\n  <th>No. Observations:</th>      <td>    70</td>      <th>  AIC:               </th> <td>   191.0</td>\n</tr>\n<tr>\n  <th>Df Residuals:</th>          <td>    68</td>      <th>  BIC:               </th> <td>   195.5</td>\n</tr>\n<tr>\n  <th>Df Model:</th>              <td>     1</td>      <th>                     </th>     <td> </td>   \n</tr>\n<tr>\n  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n</tr>\n</table>\n<table class=\"simpletable\">\n<tr>\n     <td></td>        <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n</tr>\n<tr>\n  <th>const</th>   <td>    1.1792</td> <td>    0.146</td> <td>    8.086</td> <td> 0.000</td> <td>    0.888</td> <td>    1.470</td>\n</tr>\n<tr>\n  <th>Crop ch</th> <td>   -0.0059</td> <td>    0.014</td> <td>   -0.433</td> <td> 0.666</td> <td>   -0.033</td> <td>    0.021</td>\n</tr>\n</table>\n<table class=\"simpletable\">\n<tr>\n  <th>Omnibus:</th>       <td>42.041</td> <th>  Durbin-Watson:     </th> <td>   1.619</td>\n</tr>\n<tr>\n  <th>Prob(Omnibus):</th> <td> 0.000</td> <th>  Jarque-Bera (JB):  </th> <td> 116.832</td>\n</tr>\n<tr>\n  <th>Skew:</th>          <td> 1.945</td> <th>  Prob(JB):          </th> <td>4.27e-26</td>\n</tr>\n<tr>\n  <th>Kurtosis:</th>      <td> 7.993</td> <th>  Cond. No.          </th> <td>    14.1</td>\n</tr>\n</table><br/><br/>Warnings:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified."
     },
     "metadata": {},
     "execution_count": 10
    }
   ],
   "source": [
    "Y = forest['Forest loss']\n",
    "W = forest['Crop ch']\n",
    "Z = forest['Pasture ch']\n",
    "# Regression\n",
    "formula_Y_W = sm.OLS(Y, sm.add_constant(W))\n",
    "reg_Y_W = formula_Y_W.fit()\n",
    "reg_Y_W.summary()"
   ]
  },
  {
   "source": [
    "\n",
    "(b) Run a regression of Y on Z and interpret your results. Can you reject the\n",
    "hypothesis that expansion of pastureland has an effect on deforestation rates?"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "<class 'statsmodels.iolib.summary.Summary'>\n",
       "\"\"\"\n",
       "                            OLS Regression Results                            \n",
       "==============================================================================\n",
       "Dep. Variable:            Forest loss   R-squared:                       0.091\n",
       "Model:                            OLS   Adj. R-squared:                  0.078\n",
       "Method:                 Least Squares   F-statistic:                     6.805\n",
       "Date:                Thu, 11 Feb 2021   Prob (F-statistic):             0.0112\n",
       "Time:                        14:41:50   Log-Likelihood:                -90.268\n",
       "No. Observations:                  70   AIC:                             184.5\n",
       "Df Residuals:                      68   BIC:                             189.0\n",
       "Df Model:                           1                                         \n",
       "Covariance Type:            nonrobust                                         \n",
       "==============================================================================\n",
       "                 coef    std err          t      P>|t|      [0.025      0.975]\n",
       "------------------------------------------------------------------------------\n",
       "const          1.0390      0.113      9.181      0.000       0.813       1.265\n",
       "Pasture ch     0.0332      0.013      2.609      0.011       0.008       0.059\n",
       "==============================================================================\n",
       "Omnibus:                       61.573   Durbin-Watson:                   1.575\n",
       "Prob(Omnibus):                  0.000   Jarque-Bera (JB):              348.013\n",
       "Skew:                           2.614   Prob(JB):                     2.69e-76\n",
       "Kurtosis:                      12.591   Cond. No.                         9.47\n",
       "==============================================================================\n",
       "\n",
       "Warnings:\n",
       "[1] Standard Errors assume that the covariance matrix of the errors is correctly specified.\n",
       "\"\"\""
      ],
      "text/html": "<table class=\"simpletable\">\n<caption>OLS Regression Results</caption>\n<tr>\n  <th>Dep. Variable:</th>       <td>Forest loss</td>   <th>  R-squared:         </th> <td>   0.091</td>\n</tr>\n<tr>\n  <th>Model:</th>                   <td>OLS</td>       <th>  Adj. R-squared:    </th> <td>   0.078</td>\n</tr>\n<tr>\n  <th>Method:</th>             <td>Least Squares</td>  <th>  F-statistic:       </th> <td>   6.805</td>\n</tr>\n<tr>\n  <th>Date:</th>             <td>Thu, 11 Feb 2021</td> <th>  Prob (F-statistic):</th>  <td>0.0112</td> \n</tr>\n<tr>\n  <th>Time:</th>                 <td>14:41:50</td>     <th>  Log-Likelihood:    </th> <td> -90.268</td>\n</tr>\n<tr>\n  <th>No. Observations:</th>      <td>    70</td>      <th>  AIC:               </th> <td>   184.5</td>\n</tr>\n<tr>\n  <th>Df Residuals:</th>          <td>    68</td>      <th>  BIC:               </th> <td>   189.0</td>\n</tr>\n<tr>\n  <th>Df Model:</th>              <td>     1</td>      <th>                     </th>     <td> </td>   \n</tr>\n<tr>\n  <th>Covariance Type:</th>      <td>nonrobust</td>    <th>                     </th>     <td> </td>   \n</tr>\n</table>\n<table class=\"simpletable\">\n<tr>\n       <td></td>         <th>coef</th>     <th>std err</th>      <th>t</th>      <th>P>|t|</th>  <th>[0.025</th>    <th>0.975]</th>  \n</tr>\n<tr>\n  <th>const</th>      <td>    1.0390</td> <td>    0.113</td> <td>    9.181</td> <td> 0.000</td> <td>    0.813</td> <td>    1.265</td>\n</tr>\n<tr>\n  <th>Pasture ch</th> <td>    0.0332</td> <td>    0.013</td> <td>    2.609</td> <td> 0.011</td> <td>    0.008</td> <td>    0.059</td>\n</tr>\n</table>\n<table class=\"simpletable\">\n<tr>\n  <th>Omnibus:</th>       <td>61.573</td> <th>  Durbin-Watson:     </th> <td>   1.575</td>\n</tr>\n<tr>\n  <th>Prob(Omnibus):</th> <td> 0.000</td> <th>  Jarque-Bera (JB):  </th> <td> 348.013</td>\n</tr>\n<tr>\n  <th>Skew:</th>          <td> 2.614</td> <th>  Prob(JB):          </th> <td>2.69e-76</td>\n</tr>\n<tr>\n  <th>Kurtosis:</th>      <td>12.591</td> <th>  Cond. No.          </th> <td>    9.47</td>\n</tr>\n</table><br/><br/>Warnings:<br/>[1] Standard Errors assume that the covariance matrix of the errors is correctly specified."
     },
     "metadata": {},
     "execution_count": 11
    }
   ],
   "source": [
    "# Regression\n",
    "formula_Y_Z = sm.OLS(Y, sm.add_constant(Z))\n",
    "reg_Y_Z = formula_Y_Z.fit()\n",
    "reg_Y_Z.summary()"
   ]
  },
  {
   "source": [
    "# Exercise 5.6\n",
    "Use data sets FIG51.XLS, FIG52.XLS, FIG53.XLS and FIG54.XLS.\n",
    "\n",
    "(a) Test whether β = 0 using the confidence interval approach for each of the four\n",
    "data sets."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "       down        up\n",
       "β -1.569407  3.391359"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>down</th>\n      <th>up</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>β</th>\n      <td>-1.569407</td>\n      <td>3.391359</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 12
    }
   ],
   "source": [
    "ci_beta_fig51_95 = reg_fig51.conf_int(alpha = 0.05).iloc[1:2]\n",
    "ci_beta_fig51_95 = ci_beta_fig51_95.rename(index = {\"X\": \"β\"})\n",
    "ci_beta_fig51_95.rename(columns = {0: \"down\", 1: \"up\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "      down        up\n",
       "β  0.69805  1.380621"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>down</th>\n      <th>up</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>β</th>\n      <td>0.69805</td>\n      <td>1.380621</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 13
    }
   ],
   "source": [
    "ci_beta_fig52_95 = reg_fig52.conf_int(alpha = 0.05).iloc[1:2]\n",
    "ci_beta_fig52_95 = ci_beta_fig52_95.rename(index = {\"X\": \"β\"})\n",
    "ci_beta_fig52_95.rename(columns = {0: \"down\", 1: \"up\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "       down        up\n",
       "β  0.985592  1.019562"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>down</th>\n      <th>up</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>β</th>\n      <td>0.985592</td>\n      <td>1.019562</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 14
    }
   ],
   "source": [
    "ci_beta_fig53_95 = reg_fig53.conf_int(alpha = 0.05).iloc[1:2]\n",
    "ci_beta_fig53_95 = ci_beta_fig53_95.rename(index = {\"X\": \"β\"})\n",
    "ci_beta_fig53_95.rename(columns = {0: \"down\", 1: \"up\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "       down        up\n",
       "β -1.881627  4.912338"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>down</th>\n      <th>up</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>β</th>\n      <td>-1.881627</td>\n      <td>4.912338</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 15
    }
   ],
   "source": [
    "ci_beta_fig54_95 = reg_fig54.conf_int(alpha = 0.05).iloc[1:2]\n",
    "ci_beta_fig54_95 = ci_beta_fig54_95.rename(index = {\"X\": \"β\"})\n",
    "ci_beta_fig54_95.rename(columns = {0: \"down\", 1: \"up\"})"
   ]
  },
  {
   "source": [
    "(b) Test whether β = 0 using the P-value approach for each of the four data sets.\n",
    "Use the 5% level of significance."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "β p-value    0.326902\n"
     ]
    }
   ],
   "source": [
    "p_value_beta_fig51 = reg_fig51.pvalues.iloc[1:2]\n",
    "p_value_beta_fig51 = p_value_beta_fig51.rename(index = {\"X\": \"β p-value\"})\n",
    "print(p_value_beta_fig51.to_string(index = True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "β p-value    2.731821e-08\n"
     ]
    }
   ],
   "source": [
    "p_value_beta_fig52 = reg_fig52.pvalues.iloc[1:2]\n",
    "p_value_beta_fig52 = p_value_beta_fig52.rename(index = {\"X\": \"β p-value\"})\n",
    "print(p_value_beta_fig52.to_string(index = True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "β p-value    3.916818e-107\n"
     ]
    }
   ],
   "source": [
    "p_value_beta_fig53 = reg_fig53.pvalues.iloc[1:2]\n",
    "p_value_beta_fig53 = p_value_beta_fig53.rename(index = {\"X\": \"β p-value\"})\n",
    "print(p_value_beta_fig53.to_string(index = True))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "β p-value    0.37819\n"
     ]
    }
   ],
   "source": [
    "p_value_beta_fig54 = reg_fig54.pvalues.iloc[1:2]\n",
    "p_value_beta_fig54 = p_value_beta_fig54.rename(index = {\"X\": \"β p-value\"})\n",
    "print(p_value_beta_fig54.to_string(index = True))"
   ]
  },
  {
   "source": [
    "(c) Redo parts (a) and (b) for α."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Only examples for fig51 dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "      down        up\n",
       "α -7.07851  5.195277"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>down</th>\n      <th>up</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>α</th>\n      <td>-7.07851</td>\n      <td>5.195277</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 21
    }
   ],
   "source": [
    "ci_alpha_fig51_95 = reg_fig51.conf_int(alpha = 0.05).iloc[0:1]\n",
    "ci_alpha_fig51_95 = ci_alpha_fig51_95.rename(index = {\"const\": \"α\"})\n",
    "ci_alpha_fig51_95.rename(columns = {0: \"down\", 1: \"up\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "output_type": "stream",
     "name": "stdout",
     "text": [
      "β p-value    0.326902\n"
     ]
    }
   ],
   "source": [
    "p_value_alpha_fig51 = reg_fig51.pvalues.iloc[0:1]\n",
    "p_value_alpha_fig51 = p_value_alpha_fig51.rename(index = {\"const\": \"α p-value\"})\n",
    "print(p_value_beta_fig51.to_string(index = True))"
   ]
  },
  {
   "source": [
    "(d) Redo parts (a), (b) and (c) using the 1% level of significance."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Only examples for confidence intervals and fig51 dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "       down        up\n",
       "β -3.641399  5.463352"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>down</th>\n      <th>up</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>β</th>\n      <td>-3.641399</td>\n      <td>5.463352</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 24
    }
   ],
   "source": [
    "ci_beta_fig51_99 = reg_fig51.conf_int(alpha = 0.01).iloc[1:2]\n",
    "ci_beta_fig51_99 = ci_beta_fig51_99.rename(index = {\"X\": \"β\"})\n",
    "ci_beta_fig51_99.rename(columns = {0: \"down\", 1: \"up\"})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "        down         up\n",
       "α -12.204977  10.321744"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>down</th>\n      <th>up</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>α</th>\n      <td>-12.204977</td>\n      <td>10.321744</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 25
    }
   ],
   "source": [
    "ci_alpha_fig51_99 = reg_fig51.conf_int(alpha = 0.01).iloc[0:1]\n",
    "ci_alpha_fig51_99 = ci_alpha_fig51_99.rename(index = {\"const\": \"α\"})\n",
    "ci_alpha_fig51_99.rename(columns = {0: \"down\", 1: \"up\"})"
   ]
  },
  {
   "source": [
    "(e) Are your results sensible in light of the discussion in this chapter of the factors\n",
    "affecting the accuracy of OLS estimates?"
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Yes, e.g., 99% confidence intervals are wider than 95%"
   ]
  },
  {
   "source": [
    "# Exercise 5.7\n",
    "We have used the file ADVERT.XLS before. Remember that it contains data on the sales and advertising expenditure of 84 companies. Set up and run a regression using this data and discuss your results verbally as you would in a report. Include a discussion of the marginal effect of advertising on sales and a discussion of whether this marginal effect is statistically significant."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ],
      "text/html": "<table style=\"text-align:center\"><tr><td colspan=\"2\" style=\"border-bottom: 1px solid black\"></td></tr><tr><td style=\"text-align:left\"></td><td colspan=\"1\"><em>Dependent variable:</em></td></tr><tr><td style=\"text-align:left\"></td><tr><td style=\"text-align:left\"></td><td>(1)</td></tr><tr><td colspan=\"2\" style=\"border-bottom: 1px solid black\"></td></tr><tr><td style=\"text-align:left\">Ads$000</td><td>0.218<sup>***</sup></td></tr><tr><td style=\"text-align:left\"></td><td>(0.077)</td></tr><tr><td style=\"text-align:left\">const</td><td>502.917<sup>***</sup></td></tr><tr><td style=\"text-align:left\"></td><td>(4.132)</td></tr><td colspan=\"2\" style=\"border-bottom: 1px solid black\"></td></tr><tr><td style=\"text-align: left\">Observations</td><td>84.0</td></tr><tr><td style=\"text-align: left\">R<sup>2</sup></td><td>0.089</td></tr><tr><td style=\"text-align: left\">Adjusted R<sup>2</sup></td><td>0.078</td></tr><tr><td style=\"text-align: left\">Residual Std. Error</td><td>12.193(df = 82.0)</td></tr><tr><td style=\"text-align: left\">F Statistic</td><td>8.012<sup>***</sup>(df = 1.0; 82.0)</td></tr><tr><td colspan=\"2\" style=\"border-bottom: 1px solid black\"></td></tr><tr><td style=\"text-align: left\">Note:</td>\n <td colspan=\"1\" style=\"text-align: right\">\n  <sup>*</sup>p&lt;0.1;\n  <sup>**</sup>p&lt;0.05;\n  <sup>***</sup>p&lt;0.01\n </td></tr></table>"
     },
     "metadata": {},
     "execution_count": 27
    }
   ],
   "source": [
    "formula_advert = sm.OLS(advert['Sales$000'], sm.add_constant(advert['Ads$000']))\n",
    "reg_advert = formula_advert.fit()\n",
    "HTML(Stargazer([reg_advert]).render_html())"
   ]
  },
  {
   "source": [
    "# Exercise 5.8\n",
    "\n",
    "Use data sets FIG51.XLS, FIG52.XLS, FIG53.XLS and FIG54.XLS. Test whether $R^2 = 0$ for each of the four data sets. Compare your results with those of Exercise 5.6."
   ],
   "cell_type": "markdown",
   "metadata": {}
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "output_type": "execute_result",
     "data": {
      "text/plain": [
       "                 F stat      F p-value\n",
       "Model                                 \n",
       "reg_fig51      1.366152   3.269023e-01\n",
       "reg_fig52     36.522733   2.731821e-08\n",
       "reg_fig53  13721.340053  3.916818e-107\n",
       "reg_fig54      0.783665   3.781896e-01"
      ],
      "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>F stat</th>\n      <th>F p-value</th>\n    </tr>\n    <tr>\n      <th>Model</th>\n      <th></th>\n      <th></th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>reg_fig51</th>\n      <td>1.366152</td>\n      <td>3.269023e-01</td>\n    </tr>\n    <tr>\n      <th>reg_fig52</th>\n      <td>36.522733</td>\n      <td>2.731821e-08</td>\n    </tr>\n    <tr>\n      <th>reg_fig53</th>\n      <td>13721.340053</td>\n      <td>3.916818e-107</td>\n    </tr>\n    <tr>\n      <th>reg_fig54</th>\n      <td>0.783665</td>\n      <td>3.781896e-01</td>\n    </tr>\n  </tbody>\n</table>\n</div>"
     },
     "metadata": {},
     "execution_count": 28
    }
   ],
   "source": [
    "F_list = []\n",
    "F_list.append(['reg_fig51', reg_fig51.fvalue, reg_fig51.f_pvalue])\n",
    "F_list.append(['reg_fig52', reg_fig52.fvalue, reg_fig52.f_pvalue])\n",
    "F_list.append(['reg_fig53', reg_fig53.fvalue, reg_fig53.f_pvalue])\n",
    "F_list.append(['reg_fig54', reg_fig54.fvalue, reg_fig54.f_pvalue])\n",
    "pd.DataFrame(F_list, columns=['Model', 'F stat', 'F p-value']).set_index('Model')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "name": "python3",
   "display_name": "Python 3.7.4 64-bit (conda)",
   "metadata": {
    "interpreter": {
     "hash": "b3ba2566441a7c06988d0923437866b63cedc61552a5af99d1f4fb67d367b25f"
    }
   }
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4-final"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}