sliderule_dsi_inferential_statistics_exercise_2_CZ.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Examining Racial Discrimination in the US Job Market\n",
    "\n",
    "### Background\n",
    "Racial discrimination continues to be pervasive in cultures throughout the world. Researchers examined the level of racial discrimination in the United States labor market by randomly assigning identical résumés to black-sounding or white-sounding names and observing the impact on requests for interviews from employers.\n",
    "\n",
    "### Data\n",
    "In the dataset provided, each row represents a resume. The 'race' column has two values, 'b' and 'w', indicating black-sounding and white-sounding. The column 'call' has two values, 1 and 0, indicating whether the resume received a call from employers or not.\n",
    "\n",
    "Note that the 'b' and 'w' values in race are assigned randomly to the resumes when presented to the employer."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercises\n",
    "You will perform a statistical analysis to establish whether race has a significant impact on the rate of callbacks for resumes.\n",
    "\n",
    "Answer the following questions **in this notebook below and submit to your Github account**. \n",
    "\n",
    "   1. What test is appropriate for this problem? Does CLT apply?\n",
    "   2. What are the null and alternate hypotheses?\n",
    "   3. Compute margin of error, confidence interval, and p-value. Try using both the bootstrapping and the frequentist statistical approaches.\n",
    "   4. Write a story describing the statistical significance in the context or the original problem.\n",
    "   5. Does your analysis mean that race/name is the most important factor in callback success? Why or why not? If not, how would you amend your analysis?\n",
    "\n",
    "You can include written notes in notebook cells using Markdown: \n",
    "   - In the control panel at the top, choose Cell > Cell Type > Markdown\n",
    "   - Markdown syntax: http://nestacms.com/docs/creating-content/markdown-cheat-sheet\n",
    "\n",
    "#### Resources\n",
    "+ Experiment information and data source: http://www.povertyactionlab.org/evaluation/discrimination-job-market-united-states\n",
    "+ Scipy statistical methods: http://docs.scipy.org/doc/scipy/reference/stats.html \n",
    "+ Markdown syntax: http://nestacms.com/docs/creating-content/markdown-cheat-sheet\n",
    "+ Formulas for the Bernoulli distribution: https://en.wikipedia.org/wiki/Bernoulli_distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from scipy import stats"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/Users/colinechabloz/Desktop'"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pwd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "data = pd.io.stata.read_stata('us_job_market_discrimination.dta')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The number of callbacks for white-sounding names is: 235.0\n",
      "The number of callbacks for black-sounding names is: 157.0\n"
     ]
    }
   ],
   "source": [
    "# number of callbacks for both-sounding names\n",
    "print('The number of callbacks for white-sounding names is:', sum(data[data.race=='w'].call))\n",
    "print('The number of callbacks for black-sounding names is:', sum(data[data.race=='b'].call))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>ad</th>\n",
       "      <th>education</th>\n",
       "      <th>ofjobs</th>\n",
       "      <th>yearsexp</th>\n",
       "      <th>honors</th>\n",
       "      <th>volunteer</th>\n",
       "      <th>military</th>\n",
       "      <th>empholes</th>\n",
       "      <th>occupspecific</th>\n",
       "      <th>...</th>\n",
       "      <th>compreq</th>\n",
       "      <th>orgreq</th>\n",
       "      <th>manuf</th>\n",
       "      <th>transcom</th>\n",
       "      <th>bankreal</th>\n",
       "      <th>trade</th>\n",
       "      <th>busservice</th>\n",
       "      <th>othservice</th>\n",
       "      <th>missind</th>\n",
       "      <th>ownership</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>b</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>17</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>b</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>316</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>b</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>19</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>b</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>6</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>313</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td></td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>b</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>22</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>313</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>Nonprofit</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 65 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "  id ad  education  ofjobs  yearsexp  honors  volunteer  military  empholes  \\\n",
       "0  b  1          4       2         6       0          0         0         1   \n",
       "1  b  1          3       3         6       0          1         1         0   \n",
       "2  b  1          4       1         6       0          0         0         0   \n",
       "3  b  1          3       4         6       0          1         0         1   \n",
       "4  b  1          3       3        22       0          0         0         0   \n",
       "\n",
       "   occupspecific    ...      compreq  orgreq  manuf  transcom  bankreal trade  \\\n",
       "0             17    ...          1.0     0.0    1.0       0.0       0.0   0.0   \n",
       "1            316    ...          1.0     0.0    1.0       0.0       0.0   0.0   \n",
       "2             19    ...          1.0     0.0    1.0       0.0       0.0   0.0   \n",
       "3            313    ...          1.0     0.0    1.0       0.0       0.0   0.0   \n",
       "4            313    ...          1.0     1.0    0.0       0.0       0.0   0.0   \n",
       "\n",
       "  busservice othservice  missind  ownership  \n",
       "0        0.0        0.0      0.0             \n",
       "1        0.0        0.0      0.0             \n",
       "2        0.0        0.0      0.0             \n",
       "3        0.0        0.0      0.0             \n",
       "4        0.0        1.0      0.0  Nonprofit  \n",
       "\n",
       "[5 rows x 65 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    w\n",
       "1    w\n",
       "2    b\n",
       "3    b\n",
       "4    w\n",
       "Name: race, dtype: object"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['race'].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Question 1: What test is appropriate for this problem? Does CLT apply?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The goal here is to determine whether the sounding of names influences the rate of callbacks. One way to assess this would be to split the population into two samples: white-sounding and black-sounding, and compare the mean callback rates. This would correspond to a two-sample test. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Conditions for Central Limit Theorem to apply: \n",
    "<p> 1. independent samples </p>\n",
    "<p> 2. the sample size must not be bigger than 10% of the entire population </p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> 1. In this case, we know that \"the 'b' and 'w' values in race are assigned randomly to the resumes when presented to the employer\". This confirms that the samples are independent. </p>\n",
    "\n",
    "<p> 2. The population is the US labor market, which size is in millions. The sample here 235 and 137. Therefore, 10% rule is respected. </p>\n",
    "\n",
    "<p> Therefore, we can say that the Central Limit Theorem applies here. </p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Question 2: What are the null and alternate hypotheses? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> Since we said we would use a two sample test, we would like to test if the proportion of callbacks for white-sounding names is equal to the one for black-sounding names. Therefore: </p>\n",
    "<p> H0: prob(white) = prob(black) </p>\n",
    "<p> H1: prob(white) =! prob(black) </p>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Question 3: Compute margin of error, confidence, and p-value"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Margin of error and confidence interval with significance level of 0.05:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.032032854209445585\n",
      "0.007796894036170457\n",
      "(0.01675122270727635, 0.04731448571161482)\n"
     ]
    }
   ],
   "source": [
    "w = data[data.race=='w']\n",
    "b = data[data.race=='b']\n",
    "tot_w = len(w)\n",
    "tot_b = len(b)\n",
    "p_w = sum(w.call)/tot_w\n",
    "p_b = sum(b.call)/tot_b\n",
    "diff = p_w - p_b\n",
    "print(diff)\n",
    "\n",
    "p_all = (sum(w.call) + sum(b.call))/(tot_w+tot_b)\n",
    "SdErr = ((p_all*(1-p_all)/tot_w) + (p_all*(1-p_all)/tot_b))**0.5\n",
    "\n",
    "print(SdErr)\n",
    "\n",
    "z = 1 - (1-0.95)/2\n",
    "CI = (diff - SdErr * stats.norm.ppf(z), diff + SdErr * stats.norm.ppf(z))\n",
    "\n",
    "print(CI)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> H0 can be written as prob(white) - prob(black) = 0 </p>\n",
    "<p> z-score: </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4.108412152434346\n"
     ]
    }
   ],
   "source": [
    "z = (p_w - p_b - 0) / SdErr\n",
    "\n",
    "print(z)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "p-value:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "3.983886837585077e-05\n"
     ]
    }
   ],
   "source": [
    "p = stats.norm.sf(z)*2\n",
    "\n",
    "print(p)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Question 4: Write a story describing the statistical significance in the context of the original problem"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Conclusion: the p-value is < 0.05 (significance level), therefore we reject H0 in order to conclude that the proportion of call-back is different whether the name of the candidate is white- or black-sounding. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Question 5: Does your analysis mean that race/name is the most important factor in callback success? Why or why not? If not, how would you amend your analysis?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<p> The only answer our hypothesis test gives us is whether there is a significant difference between two proportions. The answer was yes, in this case. However, the test doesn't give us an indication of importance of a factor over another. </p>\n",
    "<p> In order to understand if a candidate's race/name is the most important factor in callback success, we would need to analyze other factors, such as age or education level, for example. And compare the result of different hypothesis testing. </p>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}