ML-Python-Project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "<a href=\"https://www.bigdatauniversity.com\"><img src = \"https://ibm.box.com/shared/static/cw2c7r3o20w9zn8gkecaeyjhgw3xdgbj.png\" width = 400, align = \"center\"></a>\n",
    "\n",
    "<h1 align=center><font size = 5> Classification with Python</font></h1>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "In this notebook we try to practice all the classification algorithms that we learned in this course.\n",
    "\n",
    "We load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n",
    "\n",
    "Lets first load required libraries:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [],
   "source": [
    "import itertools\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from matplotlib.ticker import NullFormatter\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.ticker as ticker\n",
    "from sklearn import preprocessing\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### About dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n",
    "\n",
    "| Field          | Description                                                                           |\n",
    "|----------------|---------------------------------------------------------------------------------------|\n",
    "| Loan_status    | Whether a loan is paid off on in collection                                           |\n",
    "| Principal      | Basic principal loan amount at the                                                    |\n",
    "| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n",
    "| Effective_date | When the loan got originated and took effects                                         |\n",
    "| Due_date       | Since it’s one-time payoff schedule, each loan has one single due date                |\n",
    "| Age            | Age of applicant                                                                      |\n",
    "| Education      | Education of applicant                                                                |\n",
    "| Gender         | The gender of applicant                                                               |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets download the dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Data From CSV File  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>9/22/2016</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>10/8/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30       9/8/2016   \n",
       "1           2             2     PAIDOFF       1000     30       9/8/2016   \n",
       "2           3             3     PAIDOFF       1000     15       9/8/2016   \n",
       "3           4             4     PAIDOFF       1000     30       9/9/2016   \n",
       "4           6             6     PAIDOFF       1000     30       9/9/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   45  High School or Below    male  \n",
       "1  10/7/2016   33              Bechalor  female  \n",
       "2  9/22/2016   27               college    male  \n",
       "3  10/8/2016   28               college  female  \n",
       "4  10/8/2016   29               college    male  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = pd.read_csv('loan_train.csv')\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(346, 10)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Convert to date time object "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0 2016-10-07   45  High School or Below    male  \n",
       "1 2016-10-07   33              Bechalor  female  \n",
       "2 2016-09-22   27               college    male  \n",
       "3 2016-10-08   28               college  female  \n",
       "4 2016-10-08   29               college    male  "
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['due_date'] = pd.to_datetime(df['due_date'])\n",
    "df['effective_date'] = pd.to_datetime(df['effective_date'])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Data visualization and pre-processing\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Let’s see how many of each class is in our data set "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "PAIDOFF       260\n",
       "COLLECTION     86\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['loan_status'].value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "260 people have paid off the loan on time while 86 have gone into collection \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets plot some columns to underestand data better:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAG4xJREFUeJzt3XucFOWd7/HPV5wVFaIioyKIMyKKqGTAWY3XJbCyqPF2jAbjUdx4DtFoXDbxeMt5aTa+1nghMclRibhyyCaKGrKgSxINUTmKiRfAEcELITrqKCAQN8YgBPB3/qiaSYM9zKV7pmu6v+/Xq15T9VTVU7+umWd+XU9XP6WIwMzMLGt2KHUAZmZm+ThBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBdRFJe0u6T9LrkhZJ+q2kM4tU92hJc4tRV3eQNF9SfanjsNIop7YgqVrSs5JekHR8Fx7nw66quydxguoCkgTMAZ6MiAMi4ghgAjCoRPHsWIrjmpVhWxgLvBoRIyPiqWLEZK1zguoaY4C/RMQPmwsi4s2I+D8AknpJulXS85KWSPpyWj46vdqYJelVSfemDRxJ49OyBcB/a65X0q6Spqd1vSDp9LT8Qkk/lfSfwK8KeTGSZkiaKumJ9F3w36XHfEXSjJztpkpaKGmZpH9ppa5x6TvoxWl8fQqJzTKvbNqCpDrgFuBkSQ2Sdm7t71lSo6Qb03ULJY2S9Kik30u6ON2mj6TH0n1fao43z3H/V875yduuylZEeCryBFwO3Lad9ZOA/53O7wQsBGqB0cAfSd5d7gD8FjgO6A28DQwFBDwIzE33vxH47+n87sByYFfgQqAJ6NdKDE8BDXmmv8+z7Qzg/vTYpwMfAIenMS4C6tLt+qU/ewHzgRHp8nygHugPPAnsmpZfBVxX6t+Xp66byrAtXAjcns63+vcMNAKXpPO3AUuAvkA18F5aviPwqZy6VgBKlz9Mf44DpqWvdQdgLnBCqX+v3TW566cbSLqDpHH9JSL+luSPboSkz6eb7EbS4P4CPBcRTel+DUAN8CHwRkT8Li3/CUnDJq3rNElXpMu9gcHp/LyI+EO+mCKio/3n/xkRIeklYHVEvJTGsiyNsQE4R9IkkoY3ABhO0jCbfSYtezp9M/w3JP94rEKUSVto1tbf88Ppz5eAPhHxJ+BPkjZI2h34M3CjpBOAj4GBwN7Aqpw6xqXTC+lyH5Lz82QnY+5RnKC6xjLgrOaFiLhUUn+Sd4eQvBv6akQ8mruTpNHAxpyiLfz1d9TaoIkCzoqI17ap6yiSBpB/J+kpknd027oiIn6dp7w5ro+3ifFjYEdJtcAVwN9GxPtp11/vPLHOi4hzW4vLyk45toXc423v73m7bQY4j+SK6oiI2CSpkfxt5tsRcdd24ihb/gyqazwO9JZ0SU7ZLjnzjwKXSKoCkHSQpF23U9+rQK2kIelyboN4FPhqTv/8yPYEGBHHR0Rdnml7DXJ7PkXyT+CPkvYGTsqzzTPAsZIOTGPdRdJBnTye9Qzl3BYK/XvejaS7b5OkzwL759nmUeBLOZ9tDZS0VweO0aM5QXWBSDqPzwD+TtIbkp4DfkTSRw3wb8DLwGJJS4G72M7VbERsIOnG+Hn6wfCbOatvAKqAJWldNxT79bRHRLxI0g2xDJgOPJ1nmzUkffgzJS0haeDDujFM62bl3BaK8Pd8L1AvaSHJ1dSreY7xK+A+4Ldp9/os8l/tlaXmD+TMzMwyxVdQZmaWSU5QZmaWSU5QZmaWSU5QZmaWSZlIUOPHjw+S7zZ48lQuU9G4fXgqs6ndMpGg1q5dW+oQzDLL7cMqVSYSlJmZ2bacoMzMLJOcoMzMLJM8WKyZlZVNmzbR1NTEhg0bSh1KRevduzeDBg2iqqqq03U4QZlZWWlqaqJv377U1NSQjhtr3SwiWLduHU1NTdTW1na6HnfxmVlZ2bBhA3vuuaeTUwlJYs899yz4KtYJyirG/gMGIKko0/4DBpT65dh2ODmVXjF+B+7is4rx1qpVNO07qCh1DXq3qSj1mFnrfAVlZmWtmFfO7b167tWrF3V1dRx22GGcffbZrF+/vmXd7NmzkcSrr/718U+NjY0cdthhAMyfP5/ddtuNkSNHcvDBB3PCCScwd+7creqfNm0aw4YNY9iwYRx55JEsWLCgZd3o0aM5+OCDqauro66ujlmzZm0VU/PU2NhYyGntFr6CMrOyVswrZ2jf1fPOO+9MQ0MDAOeddx4//OEP+drXvgbAzJkzOe6447j//vv55je/mXf/448/viUpNTQ0cMYZZ7DzzjszduxY5s6dy1133cWCBQvo378/ixcv5owzzuC5555jn332AeDee++lvr6+1Zh6ijavoCRNl/Re+oTK5rJvSnpHUkM6nZyz7hpJKyS9JukfuipwM7Oe4Pjjj2fFihUAfPjhhzz99NPcc8893H///e3av66ujuuuu47bb78dgJtvvplbb72V/v37AzBq1CgmTpzIHXfc0TUvoITa08U3Axifp/y2iKhLp18ASBoOTAAOTfe5U1KvYgVrZtaTbN68mV/+8pccfvjhAMyZM4fx48dz0EEH0a9fPxYvXtyuekaNGtXSJbhs2TKOOOKIrdbX19ezbNmyluXzzjuvpStv3bp1AHz00UctZWeeeWYxXl6Xa7OLLyKelFTTzvpOB+6PiI3AG5JWAEcCv+10hGZmPUxzMoDkCuqiiy4Cku69yZMnAzBhwgRmzpzJqFGj2qwvYvuDgEfEVnfNlUsXXyGfQV0m6QJgIfD1iHgfGAg8k7NNU1r2CZImAZMABg8eXEAYZuXH7aNny5cM1q1bx+OPP87SpUuRxJYtW5DELbfc0mZ9L7zwAocccggAw4cPZ9GiRYwZM6Zl/eLFixk+fHhxX0QGdPYuvqnAEKAOWAl8Jy3Pd+N73tQfEdMioj4i6qurqzsZhll5cvsoP7NmzeKCCy7gzTffpLGxkbfffpva2tqt7sDLZ8mSJdxwww1ceumlAFx55ZVcddVVLV13DQ0NzJgxg6985Std/hq6W6euoCJidfO8pLuB5nsgm4D9cjYdBLzb6ejMzAo0eJ99ivq9tcHpnXIdNXPmTK6++uqtys466yzuu+8+rrrqqq3Kn3rqKUaOHMn69evZa6+9+MEPfsDYsWMBOO2003jnnXc45phjkETfvn35yU9+woAy/PK42urbBEg/g5obEYelywMiYmU6/8/AURExQdKhwH0knzvtCzwGDI2ILdurv76+PhYuXFjI6zBrk6SiflG3jbZTtKEM3D465pVXXmnpDrPSauV30e620eYVlKSZwGigv6Qm4HpgtKQ6ku67RuDLABGxTNKDwMvAZuDStpKTmZlZPu25i+/cPMX3bGf7fwX+tZCgzMzMPNSRmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZW1fQcNLurjNvYd1L6RPVatWsWECRMYMmQIw4cP5+STT2b58uUsW7aMMWPGcNBBBzF06FBuuOGGlq8szJgxg8suu+wTddXU1LB27dqtymbMmEF1dfVWj9B4+eWXAVi+fDknn3wyBx54IIcccgjnnHMODzzwQMt2ffr0aXkkxwUXXMD8+fP53Oc+11L3nDlzGDFiBMOGDePwww9nzpw5LesuvPBCBg4cyMaNGwFYu3YtNTU1HfqdtJcft2FmZW3lO29z1HWPFK2+Z7+Vb+zsrUUEZ555JhMnTmwZtbyhoYHVq1dz4YUXMnXqVMaNG8f69es566yzuPPOO1tGiuiIL3zhCy2jnDfbsGEDp5xyCt/97nc59dRTAXjiiSeorq5uGX5p9OjRTJkypWW8vvnz57fs/+KLL3LFFVcwb948amtreeONNzjxxBM54IADGDFiBJA8W2r69OlccsklHY65I3wFZWZWZE888QRVVVVcfPHFLWV1dXUsX76cY489lnHjxgGwyy67cPvtt3PTTTcV7dj33XcfRx99dEtyAvjsZz/b8kDEtkyZMoVrr72W2tpaAGpra7nmmmu49dZbW7aZPHkyt912G5s3by5a3Pk4QZmZFdnSpUs/8UgMyP+ojCFDhvDhhx/ywQcfdPg4ud12dXV1fPTRR60eu73a8ziPwYMHc9xxx/HjH/+408dpD3fxmZl1k20fi5GrtfLtydfFV6h8MeYru/baaznttNM45ZRTinr8XL6CMjMrskMPPZRFixblLd92XMXXX3+dPn360Ldv3y49dkf23zbGfI/zOPDAA6mrq+PBBx/s9LHa4gRlZlZkY8aMYePGjdx9990tZc8//zxDhw5lwYIF/PrXvwaSBxtefvnlXHnllUU79he/+EV+85vf8POf/7yl7JFHHuGll15q1/5XXHEF3/72t2lsbASgsbGRG2+8ka9//euf2PYb3/gGU6ZMKUrc+biLz8zK2oCB+7XrzruO1NcWScyePZvJkydz00030bt3b2pqavje977HQw89xFe/+lUuvfRStmzZwvnnn7/VreUzZszY6rbuZ55JngE7YsQIdtghuaY455xzGDFiBA888MBWz5O68847OeaYY5g7dy6TJ09m8uTJVFVVMWLECL7//e+36/XV1dVx8803c+qpp7Jp0yaqqqq45ZZbWp4QnOvQQw9l1KhR7X50fUe163EbXc2PE7Du4MdtVAY/biM7Cn3cRptdfJKmS3pP0tKcslslvSppiaTZknZPy2skfSSpIZ1+2N5AzMzMcrXnM6gZwLbXx/OAwyJiBLAcuCZn3e8joi6dLsbMzKwT2kxQEfEk8Idtyn4VEc3f0HqG5NHuZmaZkIWPLipdMX4HxbiL70vAL3OWayW9IOn/STq+tZ0kTZK0UNLCNWvWFCEMs/Lh9tF5vXv3Zt26dU5SJRQRrFu3jt69exdUT0F38Un6Bsmj3e9Ni1YCgyNinaQjgDmSDo2IT3xFOiKmAdMg+RC4kDjMyo3bR+cNGjSIpqYmnNhLq3fv3gwaVFjnWqcTlKSJwOeAsZG+VYmIjcDGdH6RpN8DBwG+BcnMukVVVVXLOHLWs3Wqi0/SeOAq4LSIWJ9TXi2pVzp/ADAUeL0YgZqZWWVp8wpK0kxgNNBfUhNwPcldezsB89LxmZ5J79g7AfiWpM3AFuDiiPhD3orNzMy2o80EFRHn5im+p5Vtfwb8rNCgzMzMPBafmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUrsSlKTpkt6TtDSnrJ+keZJ+l/7cIy2XpB9IWiFpiaRRXRW8mZmVr/ZeQc0Axm9TdjXwWEQMBR5LlwFOInmS7lBgEjC18DDNzKzStCtBRcSTwLZPxj0d+FE6/yPgjJzyf4/EM8DukgYUI1gzM6schXwGtXdErARIf+6Vlg8E3s7Zrikt24qkSZIWSlq4Zs2aAsIwKz9uH2Zdc5OE8pTFJwoipkVEfUTUV1dXd0EYZj2X24dZYQlqdXPXXfrzvbS8CdgvZ7tBwLsFHMfMzCpQIQnqYWBiOj8ReCin/IL0br7PAH9s7go0MzNrrx3bs5GkmcBooL+kJuB64CbgQUkXAW8BZ6eb/wI4GVgBrAf+scgxm5lZBWhXgoqIc1tZNTbPtgFcWkhQZmZmHknCzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqV2jmecj6WDggZyiA4DrgN2B/wk0P6f62oj4RacjNDOzitTpBBURrwF1AJJ6Ae8As0me/3RbREwpSoRmZlaRitXFNxb4fUS8WaT6zMyswhUrQU0AZuYsXyZpiaTpkvbIt4OkSZIWSlq4Zs2afJuYVSy3D7MiJChJfwOcBvw0LZoKDCHp/lsJfCfffhExLSLqI6K+urq60DDMyorbh1lxrqBOAhZHxGqAiFgdEVsi4mPgbuDIIhzDzMwqTDES1LnkdO9JGpCz7kxgaRGOYWZmFabTd/EBSNoFOBH4ck7xLZLqgAAat1lnZmbWLgUlqIhYD+y5Tdn5BUVkZmaGR5IwM7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMKug2c7OeRL2qGPRuU9HqMrOu5QRlFSO2bOKo6x4pSl3Pfmt8Ueoxs9a5i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDKp4NvMJTUCfwK2AJsjol5SP+ABoIbkmVDnRMT7hR7LzMwqR7GuoD4bEXURUZ8uXw08FhFDgcfSZasw+w8YgKSCp/0HDGj7YGZWdrrqi7qnA6PT+R8B84GruuhYllFvrVpF076DCq6nWKM/mFnPUowrqAB+JWmRpElp2d4RsRIg/bnXtjtJmiRpoaSFa9asKUIYZuXD7cOsOAnq2IgYBZwEXCrphPbsFBHTIqI+Iuqrq6uLEIZZ+XD7MCtCgoqId9Of7wGzgSOB1ZIGAKQ/3yv0OGZmVlkKSlCSdpXUt3keGAcsBR4GJqabTQQeKuQ4ZmZWeQq9SWJvYLak5rrui4hHJD0PPCjpIuAt4OwCj2NmZhWmoAQVEa8Dn85Tvg4YW0jdZmZW2TyShJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZfIJ2F31RF0zM+tBsvgEbF9BmZlZJnU6QUnaT9ITkl6RtEzSP6Xl35T0jqSGdDq5eOGamVmlKKSLbzPw9YhYnD60cJGkeem62yJiSuHhmZlZpep0goqIlcDKdP5Pkl4BBhYrMDMzq2xF+QxKUg0wEng2LbpM0hJJ0yXt0co+kyQtlLRwzZo1xQjDrGy4fZgVIUFJ6gP8DJgcER8AU4EhQB3JFdZ38u0XEdMioj4i6qurqwsNw6ysuH2YFZigJFWRJKd7I+I/ACJidURsiYiPgbuBIwsP08zMKk0hd/EJuAd4JSK+m1Oe+y2tM4GlnQ/PzMwqVSF38R0LnA+8JKkhLbsWOFdSHRBAI/DlgiI0M7OKVMhdfAsA5Vn1i86HY2ZmlvBIEmZmlkkei8+6jHpVFWVcLvWqKkI0ZtbTOEFZl4ktmzjqukcKrufZb40vQjRm1tO4i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMrJtl8fHqWeS7+MzMulkWH6+eRb6CMjOzTHKCMjOzTHIXn5mZZXLkFycoMzPL5Mgv7uIzM7NM6rIEJWm8pNckrZB0daH1+bZMM7PK0iVdfJJ6AXcAJwJNwPOSHo6Ilztbp2/LNDOrLF31GdSRwIqIeB1A0v3A6UCnE1TW7D9gAG+tWlVwPYP32Yc3V64sQkTlTcr3bEzLIreNthXrhoQdelWVddtQRBS/UunzwPiI+B/p8vnAURFxWc42k4BJ6eLBwGtFD6T9+gNrS3j8Qjj20mgr9rUR0elPizPUPsr5d5Rl5Rx7u9tGV11B5UvpW2XCiJgGTOui43eIpIURUV/qODrDsZdGV8eelfbh31FpOPZEV90k0QTsl7M8CHi3i45lZmZlqKsS1PPAUEm1kv4GmAA83EXHMjOzMtQlXXwRsVnSZcCjQC9gekQs64pjFUnJu1IK4NhLoyfH3hE9+XU69tIoWuxdcpOEmZlZoTyShJmZZZITlJmZZVLFJChJvSS9IGluulwr6VlJv5P0QHozB5J2SpdXpOtrShz37pJmSXpV0iuSjpbUT9K8NPZ5kvZIt5WkH6SxL5E0qsSx/7OkZZKWSpopqXdWz7uk6ZLek7Q0p6zD51nSxHT730ma2J2vobPcNkoSu9tGO1RMggL+CXglZ/lm4LaIGAq8D1yUll8EvB8RBwK3pduV0veBRyJiGPBpktdwNfBYGvtj6TLAScDQdJoETO3+cBOSBgKXA/URcRjJzTITyO55nwFs++XBDp1nSf2A64GjSEZTub654Wac20Y3ctvoQNuIiLKfSL6H9RgwBphL8kXitcCO6fqjgUfT+UeBo9P5HdPtVKK4PwW8se3xSUYVGJDODwBeS+fvAs7Nt10JYh8IvA30S8/jXOAfsnzegRpgaWfPM3AucFdO+VbbZXFy23DbaGfMJWkblXIF9T3gSuDjdHlP4L8iYnO63ETyRwN//eMhXf/HdPtSOABYA/zftAvm3yTtCuwdESvTGFcCe6Xbt8Seyn1d3Soi3gGmAG8BK0nO4yJ6xnlv1tHznJnz3wFuG93MbWOr8u0q+wQl6XPAexGxKLc4z6bRjnXdbUdgFDA1IkYCf+avl9L5ZCb29PL9dKAW2BfYleTyf1tZPO9taS3WnvQa3DbcNrpCUdtG2Sco4FjgNEmNwP0kXRnfA3aX1PxF5dyhmFqGaUrX7wb8oTsDztEENEXEs+nyLJJGuVrSAID053s522dliKm/B96IiDURsQn4D+AYesZ5b9bR85yl898ebhul4bbRzvNf9gkqIq6JiEERUUPyQeTjEXEe8ATw+XSzicBD6fzD6TLp+scj7TTtbhGxCnhb0sFp0ViSR5bkxrht7Bekd9J8Bvhj82V4CbwFfEbSLpLEX2PP/HnP0dHz/CgwTtIe6bvkcWlZJrltuG0UoHvaRik+JCzVBIwG5qbzBwDPASuAnwI7peW90+UV6foDShxzHbAQWALMAfYg6X9+DPhd+rNfuq1IHhT5e+AlkruEShn7vwCvAkuBHwM7ZfW8AzNJPg/YRPJu76LOnGfgS+lrWAH8Y6n/5jvw+t02ujd2t412HNtDHZmZWSaVfRefmZn1TE5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QGSZpi6SGdMTjn0rapZXtfiFp907Uv6+kWQXE1yipf2f3N+sst43K4NvMM0zShxHRJ52/F1gUEd/NWS+S3+HHrdXRxfE1knzPYW0pjm+Vy22jMvgKqud4CjhQUo2SZ9/cCSwG9mt+t5az7m4lz5r5laSdASQdKOnXkl6UtFjSkHT7pen6CyU9JOkRSa9Jur75wJLmSFqU1jmpJK/erHVuG2XKCaoHSMffOonkm9kABwP/HhEjI+LNbTYfCtwREYcC/wWclZbfm5Z/mmTcr3zDvBwJnEfyDf2zJdWn5V+KiCOAeuBySaUeSdkMcNsod05Q2bazpAaS4VzeAu5Jy9+MiGda2eeNiGhI5xcBNZL6AgMjYjZARGyIiPV59p0XEesi4iOSASyPS8svl/Qi8AzJgI9DC35lZoVx26gAO7a9iZXQRxFRl1uQdK3z5+3sszFnfguwM/mHus9n2w8kQ9JoktGXj46I9ZLmk4wNZlZKbhsVwFdQFSAiPgCaJJ0BIGmnVu56OlFSv7Rv/gzgaZKh/d9PG+Aw4DPdFrhZF3PbyDYnqMpxPkl3xBLgN8A+ebZZQDKycgPws4hYCDwC7JjudwNJV4ZZOXHbyCjfZm5AcqcSyW2xl5U6FrMscdsoHV9BmZlZJvkKyszMMslXUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlkn/H+LDZoiBEQ8dAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import seaborn as sns\n",
    "\n",
    "bins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGfZJREFUeJzt3XuQVOW57/HvTxgdFbygo4yMwKgoopIBZ3tDDYJy2N49XuKOR7GOJx4Naqjo8ZZTVrLdZbyVmhwvkUQLK1HUmA26SUWDCidi4gVwRBBv0UFHQS7RKAchgs/5o9fMHqBhembWTK/u+X2qVnWvt1e/61lMvzy93vX2uxQRmJmZZc02xQ7AzMwsHycoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCeolEjaU9Ijkt6XNE/SXySdkVLdoyXNSKOu7iBptqT6YsdhxVdO7UJSlaSXJb0m6Zgu3M/qrqq71DhBpUCSgOnAnyJin4g4FDgXqClSPL2LsV+z1sqwXYwF3oqIERHxQhox2dY5QaVjDPCPiPhFc0FELImI/wMgqZek2yS9KmmBpP+ZlI9OzjaekPSWpIeTRo2k8UnZHOC/NtcraUdJDyZ1vSbptKT8Qkm/lfQfwB87czCSpki6T9Ks5Jvvt5N9LpY0pdV290maK2mRpJ9soa5xybfm+Ul8fToTm5WUsmkXkuqAW4ETJTVI2n5Ln21JjZJuSl6bK2mkpGck/VXSJck2fSQ9l7z3jeZ48+z3f7X698nbxspaRHjp5AJcAdy5ldcvBv538nw7YC5QC4wG/k7uG+U2wF+Ao4FK4CNgCCDgcWBG8v6bgP+WPN8FeAfYEbgQaAL6bSGGF4CGPMvxebadAjya7Ps04AvgkCTGeUBdsl2/5LEXMBsYnqzPBuqB3YE/ATsm5dcANxT77+Wle5YybBcXAncnz7f42QYagUuT53cCC4C+QBWwPCnvDezUqq73ACXrq5PHccDk5Fi3AWYAxxb779qdi7uCuoCke8g1qH9ExD+R+6ANl3RWssnO5BrZP4BXIqIpeV8DMBhYDXwQEe8m5b8h15hJ6jpV0lXJeiUwMHk+MyL+li+miGhvn/l/RERIegP4NCLeSGJZlMTYAJwj6WJyja0aGEauMTY7Iil7MfkCvC25/2ysByqTdtGsrc/2U8njG0CfiPgS+FLSWkm7AP8PuEnSscA3wABgT2BZqzrGJctryXofcv8+f+pgzCXHCSodi4Azm1ciYqKk3cl9I4TcN6DLI+KZ1m+SNBpY16poA//5N9nSJIkCzoyItzep63ByH/r8b5JeIPctblNXRcSzecqb4/pmkxi/AXpLqgWuAv4pIj5Luv4q88Q6MyL+ZUtxWVkrx3bRen9b+2xvtf0A55E7ozo0Ir6W1Ej+9vPTiLh/K3GUNV+DSsfzQKWkS1uV7dDq+TPApZIqACTtL2nHrdT3FlArad9kvXUjeAa4vFWf/IhCAoyIYyKiLs+ytUa4NTuRa/h/l7Qn8M95tnkJGCVpvyTWHSTt38H9Wekp53bR2c/2zuS6+76WdBwwKM82zwD/vdW1rQGS9mjHPkqeE1QKItdhfDrwbUkfSHoFeIhcvzTAr4A3gfmSFgL3s5Wz14hYS67r4vfJxeAlrV6+EagAFiR13Zj28RQiIl4n1/WwCHgQeDHPNivI9dtPlbSAXKMe2o1hWhGVc7tI4bP9MFAvaS65s6m38uzjj8AjwF+SrvYnyH+2V7aaL8qZmZllis+gzMwsk5ygzMwsk5ygzMwsk5ygzMwsk7o1QY0fPz7I/Y7Bi5dyXTrN7cRLD1gK0q0JauXKld25O7OS5HZiluMuPjMzyyQnKDMzyyQnKDMzyyRPFmtmZefrr7+mqamJtWvXFjuUHq2yspKamhoqKio69H4nKDMrO01NTfTt25fBgweTzB9r3SwiWLVqFU1NTdTW1naoDnfxmVnZWbt2LbvttpuTUxFJYrfdduvUWawTVDcaVF2NpFSWQdXVxT4cs0xzciq+zv4N3MXXjT5ctoymvWpSqavmk6ZU6jEzyyqfQZlZ2Uuz96LQHoxevXpRV1fHwQcfzNlnn82aNWtaXps2bRqSeOut/7wNVGNjIwcffDAAs2fPZuedd2bEiBEccMABHHvsscyYMWOj+idPnszQoUMZOnQohx12GHPmzGl5bfTo0RxwwAHU1dVRV1fHE088sVFMzUtjY2Nn/lm7nM+gzKzspdl7AYX1YGy//fY0NDQAcN555/GLX/yCH/7whwBMnTqVo48+mkcffZQf//jHed9/zDHHtCSlhoYGTj/9dLbffnvGjh3LjBkzuP/++5kzZw6777478+fP5/TTT+eVV16hf//+ADz88MPU19dvMaZS4DMoM7Mudswxx/Dee+8BsHr1al588UUeeOABHn300YLeX1dXxw033MDdd98NwC233MJtt93G7rvvDsDIkSOZMGEC99xzT9ccQJE4QZmZdaH169fzhz/8gUMOOQSA6dOnM378ePbff3/69evH/PnzC6pn5MiRLV2CixYt4tBDD93o9fr6ehYtWtSyft5557V05a1atQqAr776qqXsjDPOSOPwupS7+MzMukBzMoDcGdRFF10E5Lr3Jk2aBMC5557L1KlTGTlyZJv1RWx9EvCI2GjUXDl08RWUoCQ1Al8CG4D1EVEvqR/wGDAYaATOiYjPuiZMM7PSki8ZrFq1iueff56FCxciiQ0bNiCJW2+9tc36XnvtNQ488EAAhg0bxrx58xgzZkzL6/Pnz2fYsGHpHkSRtaeL77iIqIuI5pR8LfBcRAwBnkvWzcxsC5544gkuuOAClixZQmNjIx999BG1tbUbjcDLZ8GCBdx4441MnDgRgKuvvpprrrmmpeuuoaGBKVOm8P3vf7/Lj6E7daaL7zRgdPL8IWA2cE0n4zEzS93A/v1T/e3gwGSkXHtNnTqVa6/d+Lv8mWeeySOPPMI112z83+cLL7zAiBEjWLNmDXvssQc///nPGTt2LACnnnoqH3/8MUcddRSS6Nu3L7/5zW+oLrMf8Kutfk0ASR8An5G7E+L9ETFZ0ucRsUurbT6LiF3zvPdi4GKAgQMHHrpkyZLUgi81klL9oW4hfzvrdh366bzbSboWL17c0h1mxbWFv0VB7aTQLr5RETES+GdgoqRjCw0uIiZHRH1E1FdVVRX6NrMexe3EbHMFJaiI+CR5XA5MAw4DPpVUDZA8Lu+qIM3MrOdpM0FJ2lFS3+bnwDhgIfAUMCHZbALwZFcFaWZmPU8hgyT2BKYl4+t7A49ExNOSXgUel3QR8CFwdteFaWZmPU2bCSoi3ge+lad8FTC2K4IyMzPzVEdmZpZJTlBmVvb2qhmY6u029qoZWNB+ly1bxrnnnsu+++7LsGHDOPHEE3nnnXdYtGgRY8aMYf/992fIkCHceOONLT8bmTJlCpdddtlmdQ0ePJiVK1duVDZlyhSqqqo2uoXGm2++CcA777zDiSeeyH777ceBBx7IOeecw2OPPdayXZ8+fVpuyXHBBRcwe/ZsTj755Ja6p0+fzvDhwxk6dCiHHHII06dPb3ntwgsvZMCAAaxbtw6AlStXMnjw4Hb9TQrhufgKMKi6mg+XLSt2GGbWQUs//ojDb3g6tfpe/tfxbW4TEZxxxhlMmDChZdbyhoYGPv30Uy688ELuu+8+xo0bx5o1azjzzDO59957W2aKaI/vfOc7LbOcN1u7di0nnXQSd9xxB6eccgoAs2bNoqqqqmX6pdGjR3P77be3zNc3e/bslve//vrrXHXVVcycOZPa2lo++OADTjjhBPbZZx+GDx8O5O4t9eCDD3LppZe2O+ZCOUEVIK17yfguuGY9x6xZs6ioqOCSSy5pKaurq+OBBx5g1KhRjBs3DoAddtiBu+++m9GjR3coQeXzyCOPcOSRR7YkJ4Djjjuu4PfffvvtXH/99dTW1gJQW1vLddddx2233cavf/1rACZNmsSdd97J9773vVRizsddfGZmXWDhwoWb3RID8t8qY99992X16tV88cUX7d5P6267uro6vvrqqy3uu1CF3M5j4MCBHH300S0Jqyv4DMrMrBtteluM1rZUvjX5uvg6K1+M+cquv/56Tj31VE466aRU99/MZ1BmZl3goIMOYt68eXnL586du1HZ+++/T58+fejbt2+X7rs97980xny389hvv/2oq6vj8ccf7/C+tsYJysysC4wZM4Z169bxy1/+sqXs1VdfZciQIcyZM4dnn30WyN3Y8IorruDqq69Obd/f/e53+fOf/8zvf//7lrKnn36aN954o6D3X3XVVfz0pz+lsbERgMbGRm666SauvPLKzbb90Y9+xO23355K3JtyF5+Zlb3qAXsXNPKuPfW1RRLTpk1j0qRJ3HzzzVRWVjJ48GDuuusunnzySS6//HImTpzIhg0bOP/88zcaWj5lypSNhnW/9NJLAAwfPpxttsmdV5xzzjkMHz6cxx57bKP7Sd17770cddRRzJgxg0mTJjFp0iQqKioYPnw4P/vZzwo6vrq6Om655RZOOeUUvv76ayoqKrj11ltb7hDc2kEHHcTIkSMLvnV9exR0u4201NfXx6anjaUgrdtk1HzS5NttlL8O3W6jtVJtJ1ni221kR3fcbsPMzKxbOUGZmVkmOUGZWVlyF3jxdfZv4ARlZmWnsrKSVatWOUkVUUSwatUqKisrO1yHR/GZWdmpqamhqamJFStWFDuUHq2yspKamo4PDHOCKlHb0bFfneczsH9/lixdmkpdZllQUVHRMo+clS4nqBK1DlIdsm5mljUFX4OS1EvSa5JmJOu1kl6W9K6kxyRt23VhmplZT9OeQRI/ABa3Wr8FuDMihgCfARelGZiZmfVsBSUoSTXAScCvknUBY4Ankk0eAk7vigDNzKxnKvQM6i7gauCbZH034POIWJ+sNwED8r1R0sWS5kqa6xE1Zvm5nZhtrs0EJelkYHlEtJ67Pd/wsbw/OIiIyRFRHxH1VVVVHQzTrLy5nZhtrpBRfKOAUyWdCFQCO5E7o9pFUu/kLKoG+KTrwjQzs56mzTOoiLguImoiYjBwLvB8RJwHzALOSjabADzZZVGamVmP05mpjq4BfijpPXLXpB5IJyQzM7N2/lA3ImYDs5Pn7wOHpR+SmZmZJ4s1M7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMajNBSaqU9Iqk1yUtkvSTpLxW0suS3pX0mKRtuz5cMzPrKQo5g1oHjImIbwF1wHhJRwC3AHdGxBDgM+CirgvTzMx6mjYTVOSsTlYrkiWAMcATSflDwOldEqGZmfVIBV2DktRLUgOwHJgJ/BX4PCLWJ5s0AQO28N6LJc2VNHfFihVpxGxWdtxOzDZXUIKKiA0RUQfUAIcBB+bbbAvvnRwR9RFRX1VV1fFIzcqY24nZ5to1ii8iPgdmA0cAu0jqnbxUA3ySbmhmZtaTFTKKr0rSLsnz7YHjgcXALOCsZLMJwJNdFaSZmfU8vdvehGrgIUm9yCW0xyNihqQ3gUcl/RvwGvBAF8ZpZmY9TJsJKiIWACPylL9P7nqUmZlZ6jyThJmZZZITlJmZZZITlJmZZZITlJmZZVLZJqhB1dVISmUxM7PuV8gw85L04bJlNO1Vk0pdNZ80pVKPmZkVrmzPoMzMrLQ5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSa1maAk7S1plqTFkhZJ+kFS3k/STEnvJo+7dn24ZmbWUxRyBrUeuDIiDgSOACZKGgZcCzwXEUOA55J1MzOzVLSZoCJiaUTMT55/CSwGBgCnAQ8lmz0EnN5VQZqZWc/TrmtQkgYDI4CXgT0jYinkkhiwxxbec7GkuZLmrlixonPRmpUptxOzzRWcoCT1AX4HTIqILwp9X0RMjoj6iKivqqrqSIxmZc/txGxzBSUoSRXkktPDEfHvSfGnkqqT16uB5V0TopmZ9USFjOIT8ACwOCLuaPXSU8CE5PkE4Mn0w7PusB20edv7QpZB1dXFPhQzKyOF3PJ9FHA+8IakhqTseuBm4HFJFwEfAmd3TYjW1dYBTXvVdLqemk+aOh+MmVmizQQVEXMAbeHlsemGk03qVZHKf77qvW1q/4mrV0Uq9ZiZZVUhZ1A9Xmz4msNveLrT9bz8r+NTqae5LjOzcuapjszMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJPKdiaJtKYnMjOz4ijbBJXW9ETgaYXMzIrBXXxmZpZJTlBmZpZJTlBmZpZJZXsNqtylOQjE95ayrBlUXc2Hy5Z1up7tt+nFV99sSCEiGNi/P0uWLk2lLiuME1SJ8iAQK2cfLluW2l2e06inuS7rXm128Ul6UNJySQtblfWTNFPSu8njrl0bppmZ9TSFXIOaAmz6Ffta4LmIGAI8l6xbD7cdICmVZVB1dbEPx8yKrM0uvoj4k6TBmxSfBoxOnj8EzAauSTEuK0HrwN0pZpaajo7i2zMilgIkj3tsaUNJF0uaK2nuihUrOrg7s/JWDu1kUHV1amfQZtANgyQiYjIwGaC+vj66en9mpagc2klaAxvAZ9CW09EzqE8lVQMkj8vTC8nMzKzjCeopYELyfALwZDrhmJmZ5RQyzHwq8BfgAElNki4CbgZOkPQucEKybmZmlppCRvH9yxZeGptyLGZmZi0yNRefRwGZmVmzTE115FFAZmbWLFMJyoojrYlnPemsmaXJCcpSm3jWk86aWZoydQ3KzMysmROUmZllkhOUmZllkhOUmZllkhOUZZLvLdU9/NtDyzKP4rNM8r2luod/e2hZ5gRlqUnr91TNdZlZz+YEZalJ6/dU4N9UmZmvQZmZWUb5DMoyKc3uwm16VaRyEX9g//4sWbo0hYjKU6pdvL239fRbBRhUXc2Hy5alUlcWP99OUJZJaXcXpjEQwIMAti7tv5mn32pbuQ9ycRefmZllUqbOoNLsIjAzs9KWqQTlUWBmZtasUwlK0njgZ0Av4FcRcXMqUZmlqBzvd5XmxXErTFqDbQC26V3BN+u/TqWuctbhBCWpF3APcALQBLwq6amIeDOt4MzSUI73u0rr4ri71Av3jQfudLvODJI4DHgvIt6PiH8AjwKnpROWmZn1dIqIjr1ROgsYHxH/I1k/Hzg8Ii7bZLuLgYuT1QOAtzsebovdgZUp1JMFPpZs6uixrIyIdp9qdVE7Af9NsqqnH0tB7aQz16DydcZulu0iYjIwuRP72XzH0tyIqE+zzmLxsWRTdx9LV7QT8N8kq3wshelMF18TsHer9Rrgk86FY2ZmltOZBPUqMERSraRtgXOBp9IJy8zMeroOd/FFxHpJlwHPkBtm/mBELEotsq1LvSukiHws2VQux1IuxwE+lqzqsmPp8CAJMzOzruS5+MzMLJOcoMzMLJMyn6Ak7S1plqTFkhZJ+kFS3k/STEnvJo+7FjvWtkiqlPSKpNeTY/lJUl4r6eXkWB5LBp1knqRekl6TNCNZL8njAJDUKOkNSQ2S5iZlJfMZczvJtnJpK93dTjKfoID1wJURcSBwBDBR0jDgWuC5iBgCPJesZ906YExEfAuoA8ZLOgK4BbgzOZbPgIuKGGN7/ABY3Gq9VI+j2XERUdfqNx2l9BlzO8m2cmor3ddOIqKkFuBJcvP/vQ1UJ2XVwNvFjq2dx7EDMB84nNyvsHsn5UcCzxQ7vgLir0k+jGOAGeR+uF1yx9HqeBqB3TcpK9nPmNtJdpZyaivd3U5K4QyqhaTBwAjgZWDPiFgKkDzuUbzICpec6jcAy4GZwF+BzyNifbJJEzCgWPG1w13A1cA3yfpulOZxNAvgj5LmJdMOQel+xgbjdpIl5dRWurWdZOp+UFsjqQ/wO2BSRHyR1rT33S0iNgB1knYBpgEH5tuse6NqH0knA8sjYp6k0c3FeTbN9HFsYlREfCJpD2CmpLeKHVBHuJ1kSxm2lW5tJyWRoCRVkGt0D0fEvyfFn0qqjoilkqrJfdMqGRHxuaTZ5K4X7CKpd/KNqhSmjBoFnCrpRKAS2Inct8RSO44WEfFJ8rhc0jRys/WX1GfM7SSTyqqtdHc7yXwXn3JfAR8AFkfEHa1eegqYkDyfQK7PPdMkVSXfCJG0PXA8uQuns4Czks0yfywRcV1E1ETEYHJTXD0fEedRYsfRTNKOkvo2PwfGAQspoc+Y20k2lVNbKUo7KfZFtwIuyh1N7vR3AdCQLCeS68d9Dng3eexX7FgLOJbhwGvJsSwEbkjK9wFeAd4DfgtsV+xY23FMo4EZpXwcSdyvJ8si4EdJecl8xtxOsr+UelspRjvxVEdmZpZJme/iMzOznskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJqsRJmp5M3LioefJGSRdJekfSbEm/lHR3Ul4l6XeSXk2WUcWN3qx7uJ2UJv9Qt8RJ6hcRf0umhHkV+C/Ai8BI4EvgeeD1iLhM0iPAvRExR9JAclP855uE06ysuJ2UppKYLNa26gpJZyTP9wbOB/5vRPwNQNJvgf2T148HhrWa4XonSX0j4svuDNisCNxOSpATVAlLpu8/HjgyItYksz6/Tf5bE0CuS/fIiPiqeyI0Kz63k9Lla1ClbWfgs6TRDSV3S4IdgG9L2lVSb+DMVtv/EbiseUVSXbdGa1YcbiclygmqtD0N9Ja0ALgReAn4GLiJ3N1UnwXeBP6ebH8FUC9pgaQ3gUu6P2Szbud2UqI8SKIMSeoTEauTb4bTgAcjYlqx4zLLEreT7PMZVHn6saQGcvfS+QCYXuR4zLLI7STjfAZlZmaZ5DMoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpP8PlTlGZbaTvVAAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "\n",
    "bins = np.linspace(df.age.min(), df.age.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'age', bins=bins, ec=\"k\")\n",
    "\n",
    "g.axes[-1].legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Pre-processing:  Feature selection/extraction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Lets look at the day of the week people get the loan "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAGepJREFUeJzt3XmcVPW55/HPV2gvIriC2tIBWkQQldtgR+OCQUh4EdzwuoTEKGTMdTQuYQyDSzImN84YF8YlcSVq8EbEhUTMJTcaVIjgztKCiCFebbEVFJgYYxQFfeaPOt1poKGr6VPU6erv+/WqV1edOud3ntNdTz91fnXq91NEYGZmljU7FDsAMzOzprhAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlApUTS3pLuk/S6pAWSnpV0ckptD5U0M422tgdJcyRVFzsOK75SygtJ3SU9L2mRpCEF3M+HhWq7rXGBSoEkATOApyJiv4g4FBgDVBQpno7F2K9ZYyWYF8OBVyNiUETMTSMm2zoXqHQMAz6NiNvrF0TEmxHxcwBJHSRdJ+lFSYsl/fdk+dDkbGO6pFclTU2SGkkjk2XzgH+pb1fSzpLuTtpaJOmkZPk4SQ9J+g/gD605GElTJN0maXbyzvfLyT6XSZrSaL3bJM2XtFTSv22hrRHJu+aFSXxdWhObtSklkxeSqoBrgVGSaiTttKXXtqRaSVclz82XNFjSY5L+S9K5yTpdJD2RbLukPt4m9vs/G/1+msyxkhYRvrXyBlwE3LCV588Bfpjc/ydgPlAJDAX+Su4d5Q7As8DRQCfgLaAvIOBBYGay/VXAt5L7uwHLgZ2BcUAdsMcWYpgL1DRx+0oT604B7k/2fRLwAXBIEuMCoCpZb4/kZwdgDjAweTwHqAa6AU8BOyfLLwGuKPbfy7ftcyvBvBgH3Jzc3+JrG6gFzkvu3wAsBroC3YH3kuUdgV0atfUaoOTxh8nPEcDk5Fh3AGYCxxT777o9b+4KKgBJt5BLqE8j4ovkXmgDJZ2arLIruST7FHghIuqS7WqA3sCHwBsR8edk+b3kkpmkrRMlTUgedwJ6JvdnRcT/ayqmiGhpn/l/RERIWgK8GxFLkliWJjHWAKdLOodcspUDA8glY70vJcueTt4A70jun421QyWSF/Wae23/Nvm5BOgSEX8D/iZpnaTdgL8DV0k6Bvgc6AHsDaxq1MaI5LYoedyF3O/nqW2Muc1xgUrHUuCU+gcRcb6kbuTeEULuHdCFEfFY440kDQU+abToM/7xN9nSIIkCTomIP23S1uHkXvRNbyTNJfcublMTIuLxJpbXx/X5JjF+DnSUVAlMAL4YEX9Juv46NRHrrIj4xpbispJWinnReH9be21vNX+AM8idUR0aEesl1dJ0/vw0Iu7YShwlzZ9BpeNJoJOk8xot69zo/mPAeZLKACQdIGnnrbT3KlApqU/yuHESPAZc2KhPflA+AUbEkIioauK2tSTcml3IJf5fJe0NfK2JdZ4DjpK0fxJrZ0kHbOP+rO0p5bxo7Wt7V3LdfeslHQv0amKdx4D/1uizrR6S9mrBPto8F6gURK7DeDTwZUlvSHoBuIdcvzTAncArwEJJLwN3sJWz14hYR67r4nfJh8FvNnr6SqAMWJy0dWXax5OPiHiJXNfDUuBu4Okm1llNrt9+mqTF5JK6/3YM04qolPMihdf2VKBa0nxyZ1OvNrGPPwD3Ac8mXe3Tafpsr2TVfyhnZmaWKT6DMjOzTHKBMjOzTHKBMjOzTHKBMjOzTNquBWrkyJFB7nsMvvlWqrdWc5741g5uedmuBWrNmjXbc3dmbZLzxCzHXXxmZpZJLlBmZpZJLlBmZpZJHizWzErO+vXrqaurY926dcUOpV3r1KkTFRUVlJWVbdP2LlBmVnLq6uro2rUrvXv3Jhk/1raziGDt2rXU1dVRWVm5TW24i8/MSs66devYc889XZyKSBJ77rlnq85iXaCs5PUqL0dSq2+9ysuLfSjWAi5Oxdfav4G7+KzkrVi1irp9K1rdTsU7dSlEY2b58hmUmZW8tM6iW3I23aFDB6qqqjj44IM57bTT+Oijjxqee/jhh5HEq6/+Yxqo2tpaDj74YADmzJnDrrvuyqBBg+jXrx/HHHMMM2fO3Kj9yZMn079/f/r3789hhx3GvHnzGp4bOnQo/fr1o6qqiqqqKqZPn75RTPW32tra1vxaCy6vMyhJ/wP4DrkhKpYA3wbKgfuBPYCFwJkR8WmB4jQz22ZpnUXXy+dseqeddqKmpgaAM844g9tvv52LL74YgGnTpnH00Udz//338+Mf/7jJ7YcMGdJQlGpqahg9ejQ77bQTw4cPZ+bMmdxxxx3MmzePbt26sXDhQkaPHs0LL7zAPvvsA8DUqVOprq7eYkxtQbNnUJJ6ABcB1RFxMNABGANcA9wQEX2BvwBnFzJQM7O2asiQIbz22msAfPjhhzz99NPcdddd3H///XltX1VVxRVXXMHNN98MwDXXXMN1111Ht27dABg8eDBjx47llltuKcwBFEm+XXwdgZ0kdQQ6AyuBYeSmIIbcNM6j0w/PzKxt27BhA7///e855JBDAJgxYwYjR47kgAMOYI899mDhwoV5tTN48OCGLsGlS5dy6KGHbvR8dXU1S5cubXh8xhlnNHTlrV27FoCPP/64YdnJJ5+cxuEVVLNdfBHxtqRJwArgY+APwALg/YjYkKxWB/RoantJ5wDnAPTs2TONmM1KjvOk9NQXA8idQZ19dq6Tadq0aYwfPx6AMWPGMG3aNAYPHtxsexFbHwQ8Ija6aq4UuviaLVCSdgdOAiqB94GHgK81sWqTv72ImAxMBqiurs57mHWz9sR5UnqaKgZr167lySef5OWXX0YSn332GZK49tprm21v0aJFHHjggQAMGDCABQsWMGzYsIbnFy5cyIABA9I9iCLLp4vvK8AbEbE6ItYDvwGOBHZLuvwAKoB3ChSjmVlJmD59OmeddRZvvvkmtbW1vPXWW1RWVm50BV5TFi9ezJVXXsn5558PwMSJE7nkkksauu5qamqYMmUK3/3udwt+DNtTPlfxrQC+JKkzuS6+4cB8YDZwKrkr+cYCjxQqSDOz1ui5zz6pfo+tZ3KlXEtNmzaNSy+9dKNlp5xyCvfddx+XXHLJRsvnzp3LoEGD+Oijj9hrr7342c9+xvDhwwE48cQTefvttznyyCORRNeuXbn33nspL7Evk6u5fk0ASf8GfB3YACwid8l5D/5xmfki4FsR8cnW2qmuro758+e3NmazFpGU2hd188iXVg9f4DxpvWXLljV0h1lxbeFvkVee5PU9qIj4EfCjTRa/DhyWz/ZmZmYt5ZEkzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzKzk7VvRM9XpNvatyG84qlWrVjFmzBj69OnDgAEDGDVqFMuXL2fp0qUMGzaMAw44gL59+3LllVc2fIVhypQpXHDBBZu11bt3b9asWbPRsilTptC9e/eNptB45ZVXAFi+fDmjRo1i//3358ADD+T000/ngQceaFivS5cuDVNynHXWWcyZM4fjjz++oe0ZM2YwcOBA+vfvzyGHHMKMGTManhs3bhw9evTgk09y3yxas2YNvXv3btHfJB+esNDMSt7Kt9/i8CseTa29538ystl1IoKTTz6ZsWPHNoxaXlNTw7vvvsu4ceO47bbbGDFiBB999BGnnHIKt956a8NIES3x9a9/vWGU83rr1q3juOOO4/rrr+eEE04AYPbs2XTv3r1h+KWhQ4cyadKkhvH65syZ07D9Sy+9xIQJE5g1axaVlZW88cYbfPWrX2W//fZj4MCBQG5uqbvvvpvzzjuvxTHny2dQZmYFMHv2bMrKyjj33HMbllVVVbF8+XKOOuooRowYAUDnzp25+eabufrqq1Pb93333ccRRxzRUJwAjj322IYJEZszadIkLr/8ciorKwGorKzksssu47rrrmtYZ/z48dxwww1s2LBhS820mguUmVkBvPzyy5tNiQFNT5XRp08fPvzwQz744IMW76dxt11VVRUff/zxFvedr3ym8+jZsydHH300v/rVr7Z5P81xF5+Z2Xa06bQYjW1p+dY01cXXWk3F2NSyyy+/nBNPPJHjjjsu1f3X8xmUmVkBHHTQQSxYsKDJ5ZuOtfj666/TpUsXunbtWtB9t2T7TWNsajqP/fffn6qqKh588MFt3tfWuECZmRXAsGHD+OSTT/jFL37RsOzFF1+kb9++zJs3j8cffxzITWx40UUXMXHixNT2/c1vfpNnnnmG3/3udw3LHn30UZYsWZLX9hMmTOCnP/0ptbW1ANTW1nLVVVfx/e9/f7N1f/CDHzBp0qRU4t6Uu/jMrOSV9/hCXlfetaS95kji4YcfZvz48Vx99dV06tSJ3r17c+ONN/LII49w4YUXcv755/PZZ59x5plnbnRp+ZQpUza6rPu5554DYODAgeywQ+684vTTT2fgwIE88MADG80ndeutt3LkkUcyc+ZMxo8fz/jx4ykrK2PgwIHcdNNNeR1fVVUV11xzDSeccALr16+nrKyMa6+9tmGG4MYOOuggBg8enPfU9S2R13QbafE0AlYMnm6j/fF0G9nRmuk23MVnZmaZlKkC1au8PLVvevcqsZklzczam0x9BrVi1apUumKAVKd3NrO2Z2uXc9v20dqPkDJ1BmVmloZOnTqxdu3aVv+DtG0XEaxdu5ZOnTptcxuZOoMyM0tDRUUFdXV1rF69utihtGudOnWiomLbe8VcoMys5JSVlTWMI2dtl7v4zMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk/IqUJJ2kzRd0quSlkk6QtIekmZJ+nPyc/dCB2tmZu1HvmdQNwGPRkR/4J+BZcClwBMR0Rd4InlsZmaWimYLlKRdgGOAuwAi4tOIeB84CbgnWe0eYHShgjQzs/YnnzOo/YDVwC8lLZJ0p6Sdgb0jYiVA8nOvpjaWdI6k+ZLm+1vdZk1znphtLp8C1REYDNwWEYOAv9OC7ryImBwR1RFR3b17920M06y0OU/MNpdPgaoD6iLi+eTxdHIF611J5QDJz/cKE6KZmbVHzRaoiFgFvCWpX7JoOPAK8FtgbLJsLPBIQSI0M7N2Kd/BYi8EpkraEXgd+Da54vagpLOBFcBphQnRrHXUoSyV+cHUoSyFaMwsX3kVqIioAaqbeGp4uuGYpS8+W8/hVzza6nae/8nIFKIxs3x5JAkzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8ukvAuUpA6SFkmamTyulPS8pD9LekDSjoUL08zM2puWnEF9D1jW6PE1wA0R0Rf4C3B2moGZmVn7lleBklQBHAfcmTwWMAyYnqxyDzC6EAGamVn7lO8Z1I3ARODz5PGewPsRsSF5XAf0aGpDSedImi9p/urVq1sVrFmpcp6Yba7ZAiXpeOC9iFjQeHETq0ZT20fE5Iiojojq7t27b2OYZqXNeWK2uY55rHMUcKKkUUAnYBdyZ1S7SeqYnEVVAO8ULkwzM2tvmj2DiojLIqIiInoDY4AnI+IMYDZwarLaWOCRgkVpZmbtTmu+B3UJcLGk18h9JnVXOiGZmZnl18XXICLmAHOS+68Dh6UfkpmZmUeSMDOzjHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKB2o56lZcjKZVbr/LyYh+OmVlBtWg+KGudFatWUbdvRSptVbxTl0o7ZmZZ5TMoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLpGYLlKQvSJotaZmkpZK+lyzfQ9IsSX9Ofu5e+HDNzKy9yOcMagPw/Yg4EPgScL6kAcClwBMR0Rd4InlsZmaWimYLVESsjIiFyf2/AcuAHsBJwD3JavcAowsVpJmZtT8t+gxKUm9gEPA8sHdErIRcEQP22sI250iaL2n+6tWrWxetWYlynphtLu8CJakL8GtgfER8kO92ETE5Iqojorp79+7bEqNZyXOemG0urwIlqYxccZoaEb9JFr8rqTx5vhx4rzAhmplZe5TPVXwC7gKWRcT1jZ76LTA2uT8WeCT98MzMrL3KZ8LCo4AzgSWSapJllwNXAw9KOhtYAZxWmBDNzKw9arZARcQ8QFt4eni64ZiZWTH0Ki9nxapVqbTVc599eHPlyla34ynfzcyMFatWUbdvRSptVbxTl0o7HurIMqlXeTmSUrmVorR+P73Ky4t9KGZb5DMoy6QsvpvLkrR+P6X4u7HS4TMoMzPLpJI9g/onSK17J60P/Cx/6lDmd/dm7VzJFqhPwF1EbVh8tp7Dr3g0lbae/8nIVNoxs+3LXXxmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJJTvUkZmZ5S/N8S/VoSyVdlygzMwsk+NfuovPrB2rH/Xfkx9aFvkMyqwd86j/lmU+gzIzs0xygbLU7FvRM7XuIjMzd/FZala+/VbmPmQ1s7YrUwUqi5c5mtn216u8nBWrVrW6nZ777MObK1emEJEVQ6YKVBYvc8yq+quv0uAktqxZsWpVKhdv+MKNtq1VBUrSSOAmoANwZ0RcnUpU1ixffWVmpW6bL5KQ1AG4BfgaMAD4hqQBaQVmZtZaWf2eV6/y8lRi6tyhY0lfmNSaM6jDgNci4nUASfcDJwGvpBGYmVlrZbWnIc0uzCweX1oUEdu2oXQqMDIivpM8PhM4PCIu2GS9c4Bzkof9gD9tpdluwJptCqht8PG1bfkc35qIaPEHoC3Mk3xjact8fG1bc8eXV5605gyqqXPCzapdREwGJufVoDQ/IqpbEVOm+fjatkIeX0vypNCxZIGPr21L6/ha80XdOuALjR5XAO+0LhwzM7Oc1hSoF4G+kiol7QiMAX6bTlhmZtbebXMXX0RskHQB8Bi5y8zvjoilrYwn7y6ONsrH17Zl6fiyFEsh+PjatlSOb5svkjAzMyskDxZrZmaZ5AJlZmaZlJkCJWmkpD9Jek3SpcWOJ02SviBptqRlkpZK+l6xY0qbpA6SFkmaWexYCkHSbpKmS3o1+TseUaQ4nCdtXCnnStp5konPoJJhk5YDXyV3+fqLwDcioiRGpZBUDpRHxEJJXYEFwOhSOT4ASRcD1cAuEXF8seNJm6R7gLkRcWdy1WrniHh/O8fgPCkBpZwraedJVs6gGoZNiohPgfphk0pCRKyMiIXJ/b8By4AexY0qPZIqgOOAO4sdSyFI2gU4BrgLICI+3d7FKeE8aeNKOVcKkSdZKVA9gLcaPa6jxF6Y9ST1BgYBzxc3klTdCEwEPi92IAWyH7Aa+GXSNXOnpJ2LEIfzpO0r5VxJPU+yUqDyGjaprZPUBfg1MD4iPih2PGmQdDzwXkQsKHYsBdQRGAzcFhGDgL8Dxfj8x3nShrWDXEk9T7JSoEp+2CRJZeSSbmpE/KbY8aToKOBESbXkupyGSbq3uCGlrg6oi4j6d/PTySViMeJwnrRdpZ4rqedJVgpUSQ+bpNxkK3cByyLi+mLHk6aIuCwiKiKiN7m/25MR8a0ih5WqiFgFvCWpX7JoOMWZVsZ50oaVeq4UIk8yMeV7gYZNypKjgDOBJZJqkmWXR8R/FjEma5kLgalJYXgd+Pb2DsB5Ym1AqnmSicvMzczMNpWVLj4zM7ONuECZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUBlgKQfS5qQYnv9JdUkw430SavdRu3PkVSddrtmW+M8aX9coErTaOCRiBgUEf9V7GDMMsp5knEuUEUi6QfJvD6PA/2SZf8q6UVJL0n6taTOkrpKeiMZAgZJu0iqlVQmqUrSc5IWS3pY0u6SRgHjge8kc+tMlHRRsu0Nkp5M7g+vH2ZF0ghJz0paKOmhZCw0JB0q6Y+SFkh6LJkOofEx7CDpHkn/e7v94qxdcZ60by5QRSDpUHJDnQwC/gX4YvLUbyLiixHxz+SmGjg7mXZgDrkh+km2+3VErAf+HbgkIgYCS4AfJd+6vx24ISKOBZ4ChiTbVgNdkiQ+GpgrqRvwQ+ArETEYmA9cnKzzc+DUiDgUuBv4P40OoyMwFVgeET9M8ddjBjhPLCNDHbVDQ4CHI+IjAEn146kdnLzL2g3oQm5IG8jNHTMRmEFu6JB/lbQrsFtE/DFZ5x7goSb2tQA4VLkJ4D4BFpJLwCHARcCXgAHA07mh0NgReJbcu9WDgVnJ8g7Aykbt3gE8GBGNk9EsTc6Tds4FqniaGmNqCrkZRF+SNA4YChART0vqLenLQIeIeDlJvOZ3ErFeudGTvw08AywGjgX6kHv32QeYFRHfaLydpEOApRGxpSmbnwGOlfR/I2JdPrGYbQPnSTvmLr7ieAo4WdJOyTu2E5LlXYGVSbfBGZts8+/ANOCXABHxV+Avkuq7Jc4E/kjTngImJD/nAucCNZEbiPE54ChJ+wMk/fkHAH8Cuks6IlleJumgRm3eBfwn8JAkv9GxQnCetHMuUEWQTGv9AFBDbu6buclT/4vcDKKzgFc32WwqsDu55Ks3FrhO0mKgCvjJFnY5FygHno2Id4F19fuMiNXAOGBa0s5zQP9kSvFTgWskvZTEeuQmx3E9ua6QX0nya8lS5Twxj2beRkg6FTgpIs4sdixmWeU8KS0+5WwDJP0c+BowqtixmGWV86T0+AzKzMwyyf2hZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSf8feZ3K8s9z83MAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x216 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "df['dayofweek'] = df['effective_date'].dt.dayofweek\n",
    "bins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\n",
    "g = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\n",
    "g.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\n",
    "g.axes[-1].legend()\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>female</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below    male          3        0  \n",
       "1 2016-10-07   33              Bechalor  female          3        0  \n",
       "2 2016-09-22   27               college    male          3        0  \n",
       "3 2016-10-08   28               college  female          4        1  \n",
       "4 2016-10-08   29               college    male          4        1  "
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Convert Categorical features to numerical values"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets look at gender:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Gender  loan_status\n",
       "female  PAIDOFF        0.865385\n",
       "        COLLECTION     0.134615\n",
       "male    PAIDOFF        0.731293\n",
       "        COLLECTION     0.268707\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "86 % of female pay there loans while only 73 % of males pay there loan\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets convert male to 0 and female to 1:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "      <th>weekend</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>45</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>2</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>33</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>3</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-09-22</td>\n",
       "      <td>27</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>4</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>28</td>\n",
       "      <td>college</td>\n",
       "      <td>1</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>6</td>\n",
       "      <td>6</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-10-08</td>\n",
       "      <td>29</td>\n",
       "      <td>college</td>\n",
       "      <td>0</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           0             0     PAIDOFF       1000     30     2016-09-08   \n",
       "1           2             2     PAIDOFF       1000     30     2016-09-08   \n",
       "2           3             3     PAIDOFF       1000     15     2016-09-08   \n",
       "3           4             4     PAIDOFF       1000     30     2016-09-09   \n",
       "4           6             6     PAIDOFF       1000     30     2016-09-09   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  weekend  \n",
       "0 2016-10-07   45  High School or Below       0          3        0  \n",
       "1 2016-10-07   33              Bechalor       1          3        0  \n",
       "2 2016-09-22   27               college       0          3        0  \n",
       "3 2016-10-08   28               college       1          4        1  \n",
       "4 2016-10-08   29               college       0          4        1  "
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## One Hot Encoding  \n",
    "#### How about education?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "education             loan_status\n",
       "Bechalor              PAIDOFF        0.750000\n",
       "                      COLLECTION     0.250000\n",
       "High School or Below  PAIDOFF        0.741722\n",
       "                      COLLECTION     0.258278\n",
       "Master or Above       COLLECTION     0.500000\n",
       "                      PAIDOFF        0.500000\n",
       "college               PAIDOFF        0.765101\n",
       "                      COLLECTION     0.234899\n",
       "Name: loan_status, dtype: float64"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "\n",
    "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Feature befor One Hot Encoding"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>education</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>High School or Below</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>Bechalor</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>college</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender             education\n",
       "0       1000     30   45       0  High School or Below\n",
       "1       1000     30   33       1              Bechalor\n",
       "2       1000     15   27       0               college\n",
       "3       1000     30   28       1               college\n",
       "4       1000     30   29       0               college"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[['Principal','terms','age','Gender','education']].head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Feature = df[['Principal','terms','age','Gender','weekend']]\n",
    "Feature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\n",
    "Feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "Feature.head()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Feature selection"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Lets defind feature sets, X:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>45</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>33</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>15</td>\n",
       "      <td>27</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>28</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   45       0        0         0                     1   \n",
       "1       1000     30   33       1        0         1                     0   \n",
       "2       1000     15   27       0        0         0                     0   \n",
       "3       1000     30   28       1        1         0                     0   \n",
       "4       1000     30   29       0        1         0                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        1  \n",
       "3        1  \n",
       "4        1  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = Feature\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "What are our lables?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y = df['loan_status'].values\n",
    "y[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "## Normalize Data "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Data Standardization give data zero mean and unit variance (technically should be done after train test split )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n",
       "        -0.38170062,  1.13639374, -0.86968108],\n",
       "       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n",
       "         2.61985426, -0.87997669, -0.86968108],\n",
       "       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679],\n",
       "       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n",
       "        -0.38170062, -0.87997669,  1.14984679]])"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X= preprocessing.StandardScaler().fit(X).transform(X)\n",
    "X[0:5]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "# Classification "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\n",
    "You should use the following algorithm:\n",
    "- K Nearest Neighbor(KNN)\n",
    "- Decision Tree\n",
    "- Support Vector Machine\n",
    "- Logistic Regression\n",
    "\n",
    "\n",
    "\n",
    "__ Notice:__ \n",
    "- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n",
    "- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n",
    "- You should include the code of the algorithm in the following cells."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# K Nearest Neighbor(KNN)\n",
    "Notice: You should find the best k to build the model with the best accuracy.  \n",
    "**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "from sklearn.cross_validation import train_test_split\n",
    "import sklearn.metrics as metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "seed=50\n",
    "X_train, X_test, y_train, y_test = train_test_split(X,y, test_size=0.40, random_state=seed)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 :  0.6906474820143885\n",
      "2 :  0.5827338129496403\n",
      "3 :  0.7266187050359713\n",
      "4 :  0.6762589928057554\n",
      "5 :  0.7482014388489209\n",
      "6 :  0.6762589928057554\n",
      "7 :  0.7482014388489209\n",
      "8 :  0.7050359712230215\n",
      "9 :  0.7769784172661871\n",
      "10 :  0.7338129496402878\n",
      "11 :  0.7482014388489209\n",
      "12 :  0.7194244604316546\n",
      "13 :  0.7482014388489209\n",
      "14 :  0.7266187050359713\n",
      "15 :  0.7410071942446043\n",
      "16 :  0.7338129496402878\n",
      "17 :  0.7482014388489209\n",
      "18 :  0.7266187050359713\n",
      "19 :  0.7553956834532374\n",
      "20 :  0.7266187050359713\n",
      "21 :  0.762589928057554\n",
      "22 :  0.7194244604316546\n",
      "23 :  0.762589928057554\n",
      "24 :  0.7482014388489209\n",
      "25 :  0.762589928057554\n",
      "26 :  0.762589928057554\n",
      "27 :  0.762589928057554\n",
      "28 :  0.762589928057554\n",
      "29 :  0.762589928057554\n",
      "30 :  0.762589928057554\n",
      "31 :  0.7913669064748201\n",
      "32 :  0.7769784172661871\n",
      "33 :  0.7913669064748201\n",
      "34 :  0.7913669064748201\n",
      "35 :  0.7913669064748201\n",
      "36 :  0.7985611510791367\n",
      "37 :  0.7841726618705036\n",
      "38 :  0.7913669064748201\n",
      "39 :  0.7913669064748201\n",
      "40 :  0.7841726618705036\n",
      "41 :  0.8345323741007195\n",
      "42 :  0.8345323741007195\n",
      "43 :  0.8273381294964028\n",
      "44 :  0.8345323741007195\n",
      "45 :  0.8201438848920863\n",
      "46 :  0.8201438848920863\n",
      "47 :  0.8201438848920863\n",
      "48 :  0.8273381294964028\n",
      "49 :  0.8201438848920863\n",
      "50 :  0.8201438848920863\n",
      "51 :  0.8201438848920863\n",
      "52 :  0.8201438848920863\n",
      "53 :  0.8201438848920863\n",
      "54 :  0.8201438848920863\n",
      "55 :  0.8201438848920863\n",
      "56 :  0.8201438848920863\n",
      "57 :  0.8201438848920863\n",
      "58 :  0.8201438848920863\n",
      "59 :  0.8201438848920863\n",
      "60 :  0.8201438848920863\n",
      "61 :  0.8201438848920863\n",
      "62 :  0.8201438848920863\n",
      "63 :  0.8201438848920863\n",
      "64 :  0.8201438848920863\n",
      "65 :  0.8201438848920863\n",
      "66 :  0.8201438848920863\n",
      "67 :  0.8201438848920863\n",
      "68 :  0.8201438848920863\n",
      "69 :  0.8201438848920863\n",
      "70 :  0.8201438848920863\n",
      "71 :  0.8201438848920863\n",
      "72 :  0.8201438848920863\n",
      "73 :  0.8201438848920863\n",
      "74 :  0.8201438848920863\n",
      "75 :  0.8201438848920863\n",
      "76 :  0.8201438848920863\n",
      "77 :  0.8201438848920863\n",
      "78 :  0.8201438848920863\n",
      "79 :  0.8201438848920863\n",
      "80 :  0.8201438848920863\n",
      "81 :  0.8201438848920863\n",
      "82 :  0.8201438848920863\n",
      "83 :  0.8201438848920863\n",
      "84 :  0.8201438848920863\n",
      "85 :  0.8201438848920863\n",
      "86 :  0.8201438848920863\n",
      "87 :  0.8201438848920863\n",
      "88 :  0.8201438848920863\n",
      "89 :  0.8201438848920863\n",
      "90 :  0.8201438848920863\n",
      "91 :  0.8201438848920863\n",
      "92 :  0.8201438848920863\n",
      "93 :  0.8201438848920863\n",
      "94 :  0.8201438848920863\n",
      "95 :  0.8201438848920863\n",
      "96 :  0.8201438848920863\n",
      "97 :  0.8201438848920863\n",
      "98 :  0.8201438848920863\n",
      "99 :  0.8201438848920863\n"
     ]
    }
   ],
   "source": [
    "score=[]\n",
    "for k in range(1,100):\n",
    "    knn=KNeighborsClassifier(n_neighbors=k,weights='uniform')\n",
    "    knn.fit(X_train,y_train)\n",
    "    predKNN=knn.predict(X_test)\n",
    "    accuracy=metrics.accuracy_score(predKNN,y_test)\n",
    "    score.append(accuracy*100)\n",
    "    print (k,': ',accuracy)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "41  :  83.45 %\n"
     ]
    }
   ],
   "source": [
    "print(score.index(max(score))+1,' : ',round(max(score),2),'%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "WE can see that the highest accuracy achieved by K=41 as K starts from 1 not zero so the index of k is equal to (score.index(max(score))+1) with accuracy score= 83.45%"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0,0.5,'Train Accuracy')"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEKCAYAAAAfGVI8AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XucXGWd5/HPr6u6q++dvpGEhCQkhACC3CJXAygKI7qiLjjDugy6rM7o6Dio6zDOrMO4r3W9jjOujg5eUGcdB0QUdlZBB8FEB5CACEggN5IQCElfculrdVfVb/8451RXVbrT1elUX6q+79erX13n1Kmqp1Kd51fP83su5u6IiEjlqprtAoiIyOxSIBARqXAKBCIiFU6BQESkwikQiIhUOAUCEZEKp0AgIlLhFAhERCqcAoGISIWLz3YBitHR0eErVqyY7WKIiMwrjz32WLe7d0523bwIBCtWrGDjxo2zXQwRkXnFzHYWc526hkREKpwCgYhIhVMgEBGpcAoEIiIVToFARKTCKRCIiFQ4BQIRkQo3L+YRiBTj4NAo//TQDkZSGQBa6mt4x/nLqK2OZa9xd370xItctKqDhc21s1RSkblFgUDKxr89s5fP/XQzAGbgDoeGRrnp9Sdnr3lwcxc33f5b3nXxCv76P7xitooqMqeoa0jKRs9AEoDf/c2VPP+/3sgbz1jMreu3s+/QMADpjPOpHz8LwIYt3bNWTpG5RoFAykbvwCg18Srqa4KuoP925RpSmQxf+LctAPzgsd08t7eP805sY+u+fl46MDSbxRWZMxQIpGzsHxihrb4GMwNgRUcD7zh/Obc/uosndx/g8z97jrOXLeATVwddQhu2dM1mcUXmDAUCKRu9gyO0NtTknfvTy1fTUBPnHV97hL2HkvzlVaeyZmETC5sTrN+s7iERUCCQMtI7MEJbQ3XeubaGGt73mpPoS6a48hULWbuiDTNj3epOfrm1m3TGJ33e2x/dxZ/f+STuk18rMh9p1JCUjf0DI5x2fPNh59918QpG0xmuXbs0e27d6g7ufGw3T+4+wNnLWid8zr2HhrnlnmcYGk1z6ZpOrjpjcUnKLjKb1CKQstE7OEJbQdcQQG11jD+9fDWLW+qy59at7sRs8tFDX/jZZlKZDMvb6/n0vc9m5yiIlBMFAikLqXSGg0OjtNYfHgjG09ZQw+nHtxwxYbx5bx93bHyB6y9YwS3/4RXs7Bnku48Utc+HyLyiQCBl4eDQKO7Q3lhcIAC45OQOHt91gEPDo+Pe/6mfPEtDIs4HXnsSl63p5KJV7Xzx/i0TXi8yXylHIGVh/+AIQNEtAgi6h778wDa+v3E35y7PzxNs29fPz5/dx81vOCU7EuljV53Km/73L/nsvc/xH89dOt5TihxzJ3Y00FJXPfmF06BAIGWhdyD4lj5ejmAi5yxrpak2zv/412fGvX/JgjreedGK7PHpS1p429lL+KeHd/JPD6uLSGbGt971Ki5bc1xJX0OBQMpC78DUWwQ18Sp++L6LeaF3cNz7X7m0JW/BOoBPX/NK3nL2kqKGnYocC2csaSn5aygQSFmIuoam0iIAOOm4Rk46rrHo66tjVVxycueUXkNkrlOyWMpC1CJYUF/avlSRcqRAIGWhd2CEhprYYV05IjI5BQIpC/sHDl9nSESKo0AgZWGiWcUiMjkFAikL+wdGpjRiSETGKBBIWegdHKFdLQKRo6JAIGVh/8CocgQiR0mBQOa9ZCpNfzKlHIHIUVIgkHnvwGCwvIRyBCJHR4FA5r2e/mhWsSaTiRyNkgYCM7vJzH5nZk+b2ffMrNbMTjSzR8xsi5ndbmb6GifTcjQrj4rImJIFAjNbAvwpsNbdTwdiwB8Anwa+4O6rgf3AjaUqg1SGaHkJ5QhEjk6pu4biQJ2ZxYF6YA/wWuDO8P5vA28pcRmkzGVbBAoEIkelZIHA3V8EPgfsIggAB4HHgAPungov2w0sGe/xZvYeM9toZhu7uibeTlDKy0sHhqa8A1h2wbkSb94hUq5K2TXUClwNnAgcDzQAbxjn0nEXdnf3W919rbuv7ezUsr+V4MDgCFd9cQO33P27KT1u/8AIC+qricc09kHkaJTyf87rgOfdvcvdR4G7gIuABWFXEcBS4KUSlkHmkS/9fCsHBkd54Ll9ZKaw8Uvv4ChtShSLHLVSBoJdwAVmVm9mBlwOPAM8AFwTXnMDcHcJyyDzxAu9g3znoZ0c31LL/sFRfvfSoaIfq5VHRaanlDmCRwiSwo8DT4WvdSvw58CHzGwr0A58o1RlkPnjs/c9R1UV3PqHawFYv6X4vFCPFpwTmZaSdqq6+1+7+ynufrq7X+/uSXff7u7nuftJ7n6tuydLWQaZ+57cfYB7fvsS//XVKzl9SQuvOL6Z9ZuLDwT7B0Y0mUxkGrRnsRxTLx0Y4oe/eZH3XrqKqirLnh8eTfOpnzxLfzJ12GN+s2s/7Q01/NGlKwFYt7qTb/xyO/3JFI2J4E/0oW09/ODx3dnHnLm0hesvXIG70zuoriGR6VAgkGPqR0+8yGfve44LV7VzzrLW7Pn7N+3jW/++g4XNCeJV+Q3RWJXxP95yOk21wbf6S07u4Ku/2MbD23p43WkLOTg0ynu/+xjptNNcV00yleHOx3azemETZyxpYSSVUbJYZBoUCOSY2tk9CMCGzd15gWDDli6aEnF+9eevnXSY57nLW6mrjrFhSxevO20h//DgVg4OjfJ/3/9qTl/SwvBomtd87kE++eNNfOm6cwBNJhOZDg28lmNqZ+8AkJ/sdXfWb+7iopPaixrrn4jHuGBlGxu2dPPigSFu+9UO3nrWEk5f0gJAbXWMD1+xhid3H+Q7D+0AUItAZBoUCOSY2tUTtAieeOEAB4eCGcLbugZ46eAwl5xc/MTAS07uZHv3AP/t+78F4MNXrsm7/61nL+HUxc1881fPA2oRiEyHAoEcM8lUmj2Hhrn4pHbSGeehbT1A0C0EcMnq4gPBuvDaf9/Ww7suXsGSBXV598eqjI9ddQrRvDNtUyly9BQI5Jh5oXcId7j6rCU0JuLZ7qH1m7s4saOBE9rqi36uVZ0NLFlQR2t9Ne+77KRxr1m3ujPbylCLQOToKVksx8zOniA/sKqzkQtXtbN+cxfJVJqHt/dy7dqlU3ouM+Oz176SeFUVLUdYTO7T//EMfrW154jXiMiRKRDIMbMzzA+saK/nktUd/OyZvdz1+IsMjaan1C0UuWhVx6TXLG6p45pzpxZkRCSfuobkmNnVO0hjIk5bQ022j/9vf7aZeJVxwar2WS6diExEgUCOmZ09Ayxrq8fMWNHRwLK2err6kpy7vDU7Q1hE5h4Fgmnae2iYlw8Oz3Yx5oSdvYMsbx9LCK9bHXTtTGXYqIjMPAWCafrYXU/xoTuemO1izLp0xtndO8SynEBw5SsWUWVw+anHzWLJRGQyaq9P076+JAeGRma7GLNuz8EhRtIZVrQ3ZM9dcnInG//q9dpUXmSOU4tgmvqTKbr7FAiiGcXLC+YKKAiIzH0KBNPUN5xiaDTNwDjLK1eSnb1BIMjtGhKR+UGBYJr6k8F6Ol19lb2/zs6eQapjxuKWuskvFpE5RYFgGkbTGYZHMwB091d2INjVO8AJrfXEcjajEZH5QYFgGvqHx7qDKr1FsKM7f+ioiMwfCgTTkLvtYiW3CNydXb2DLM8ZMSQi84cCwTT0qUUAQO/ACP3JFMumsLqoiMwdCgTTkNsi6Oqv3CGk0YghdQ2JzE+aUDYN0YghmP8tgnTGSWUyR/XY7V3B8tMKBCLzkwLBNERdQ8e31M6pHMGPn9rD5376HPf92SVU5+wRvL2rn2u/+hD/8p4LWL2wKXt+cCTFZZ99kH3TCGZVBktbFQhE5iMFgmmIuoZO7GxgR/fgLJdmzMYd+9neNUB3fzJvXP8zew7RMzDCvU+/nBcIHnm+l319Sa6/YDmLWmqP6jVXdTZQWx2bdtlFZOYpEExDNHx0RXsDG3fsx90xm/1x9C8fGgKgu28kLxB0h9/412/p4gOXr86eX7+5i0S8ir9846mqzEUqkJLF09CfTGEW9I0nUxn65sgyE3vCZbG7+vOXx+4Ku68e33WAvuGx/MaGLd2cv7JdQUCkQikQTEPfcIrGRJzOpgQw9o17tkX7IxQuhhcdpzPOv2/rAeClA0Ns3dfPJasn3xZSRMqTAsE09CdTNCXidDYG/eozPXIoqsRzpdIZ9h6KWgT55enuT7L6uEbqa2Js2NIFkP2tzWNEKpcCwTT0D6doqq2moylYarl7hucS/PU9v+N9330s71xXf5KMh7cLAlNXf5LjF9Rx4cp2NmzpBmD9lm4WNidYfVzjjJRZROaeSQOBmS04mic2szVm9kTOzyEz+zMzu8XMXsw5f9XRPP9c0J9M0Vgbp7Mx6Brq6pvZLSu3dfXzfPcA6ajmZyw/AIcve9Hdl6SjMcElJ3eys2eQ7V39/HJLN+tWd86JJLeIzI5iWgSPmdn3zOyKqTyxuz/n7me5+1nAucAg8MPw7i9E97n7j6dY5jmjb3iUxkSc1voaYlU2oy2CdMZ5oXeQ0bSz5+BQ9nyUH2iqjee1CNyd7v4ROpsS2b2E/+HBbRwcGlW3kEiFKyYQrAa+A7zbzLaY2SfMbNUUX+dyYJu775xyCeewvrBFUFVltDfUzGiOYM/BIUbTQUtgZ89gzvkgEJyxpCWvRXBoKMVIOkNHYw0ndjSwtLWOHzy+GzN49UlKFItUskkDgbtn3P0n7n4t8G7gRuAJM7vfzM4r8nX+APhezvH7zexJM/ummbVOvdhzQ/9wkCwG6GhMzOjs4l05lX9uIHj54BC11VWcdFxjXmCKEsedTQnMjHWrO3EPAoa2kxSpbEXlCMzsT8zsEeBm4CagDfhL4PYiHl8DvBn4fnjqK8Aq4CxgD/D5CR73HjPbaGYbu7q6inkv05ZMpfmLu57MjrrJ9cX7t/Dr53vzzvUng+GjAB1N+YHg0R29fPreZw97nn2Hhvnonb9lcKT4OQdffmArDzy7L+9ctNBbcHsge/ulg8MsbqmjszHBoeEUyVQaGEscR/mMS08OWgHrNGxUpOIV0zX0KHAc8HZ3/z13v8PdR939YeBrRTz+DcDj7r4XwN33unva3TPh48dtVbj7re6+1t3XdnbOTB/21n39fO/XL/DLcERNTln43z/fwg9/szt7Lp1xBkfSNNYGgaCzMZH3DfzW9dv5yoPb8lYoBfj5s/u4Y+NuHtmeH1QmcnBwlM//9Dm+/dCOvPPR1pAr2uvzWgcvHxxmcUstHdHchjBvEQWp6PwlJ3fy5jOP59pzTyiqHCJSvooJBGvc/a/H6993908W8fjryOkWMrPFOfe9FXi6iOeYEdGSEYWVdzKVYTTtdOVM0IquGWsR1NDdP4K7M5rO8FA4YWtnz0Dec0Xf5J/Zc6ioMv1qWzcZh00F1+/sCbaGXNHRUNA1NMyiltrsN/9oklthi6C+Js4XrzubFR3aTEak0hUTCH6cO4TUzFrN7P8V8+RmVg+8Hrgr5/RnzOwpM3sSeA1BV9OcEFXuhYEgOs6doBWda8ppEYykMxwaSvHECwey9+d+W889LqzYJ7J+c9AttvdQkt6BsUC0s2eQZe31rGhvYFfvIO5OOuPsPZTfIogCQHd/kniV0VJXXdTrikjlKGbRuUXufiA6cPf9ZnZ8MU/u7oNAe8G566dWxJkTVd65O4/BWEshdwmJ6FxjIqhYo2UmuvqTrN/cRZVBxvP78mGsP7+YQODubNjSTXtDDT0DI2zac4iLT+rIbg153oltLGurpz+ZomdghEzGSWWcRS11dDRGk9zGWgQdjQmqtLm8iBQopkWQNrOl0YGZLStheWZVX7ZraDTvfG6LwN3zrsnNEUBQ4a7f0s3Zy1ppa6jJ67Zxd3b2DGIGz3cPMDyaznudVDp/Y5jt3QO8eGCIGy5aAYwFj9ytIaPNYHb2DPJSOHR0cXMtHY2HtwiiGdAiIrmKCQQfB35lZreZ2W3AeuBjpS3W7Mh2DRW0CKIAMZKzwmh0Luoairpitu7r48ndB1i3uoNlbfV5OYL9g6P0DadYu7yVjMPmvX3Z+7r7k5z1iZ9xx6MvZM9F3UJvPXsJnU2JbF4hd2vIKBDs6h3g5XBi2aKWWmqrYzTVxrMtgu7+kWywEhHJVcw8gv9HMLLnbuAe4Dx3/0mpCzYboqWZJ8oRwFj3UDZHkMhvEfzoiZdwD0blLG+vz2sRREHhylcsAvK7h9Zv7qI/meIz9z2bfe4NW7pZ0V7PCW31nLq4mWf3BIEjyjMsb69naWs9ZkGLIJpMdvyCYA+CzqZENq8RdQ2JiBQqdtG5YWAXsBc4ycwuKl2RZk/UEijMEeSu3R91tWRzBGGLoKWumniV8djO/TTXxnnlkhaWtzew5+AQI6mgy2dX+E1+3epOGmpibNoz1iLYsKWb+poY3f0j/OMvtpFMpXloW092+YdTFzexdV8/o+kMO3oGsHBryNrqGIuba9nVM8jLB4epiVfRWh/kLToaE3T3BbmDnoFkttUiIpJr0mSxmf0X4MPAEuAp4FXAw8BlJS3ZLOibZNQQjI3LLxw+WlVldDQmePnQMBef1EE8VsXytnoyDrv3D7KyszHbOljeXs+aRU3Zrp5MxtmwpYvXn7aQjMPXNmxnZWcDQ6Np1q0OAsFpi5sZSWfY1tXPrp5BFjfXZjeSWdZez87eQVIZZ3FLbXYBuc6mBM+8dIiDQ6OMpl1dQyIyrmJaBDcBa4Ed7r6OYAG5PSUt1SyZuEUwdhytMBqda6gZi6VRMjb6Fp9N5IYtgZ09gywKK/BTFzezac8h3J1NLx+iu3+Edas7+eiVa8hk4GN3PU28yrhwVTDo6tTFzUDQnbSzNxg6GlneFswlePngMIuax/Yc7mxM0N2XPGwymYhIrmICwbC7D0GwXIS7/w44pbTFmh1HmkcQr7K8FUaj3clyh2NG37ijRdyiynpnd5Ab2NkzkD136uJm+oZTvHhgKLs3wLrVHZzQVs8fXricodE05yxvzbY4VnY0UBOvYtOePnb2DLK8bWwi2LL2err7k2zr6mdxzubznU0J+pIpdu8fyiufiEiuYgLBnnBC2f8F7jOzHxDkCsrORKOG+odTNNdV05azwmh/cjRbSUfOWNLCuctbOaEtqOw7GxPU18TGWgS9g6zIBoImADbt6WP95i5OWdTEwvDb/PtfexILmxO86ZVjk7DjsSpOXtjIxh29dPcn81sE4e2egREW5WxWH80l2PRy0AXVqeGjIjKOSXME7v7m8OZ/N7PLgRagqJnF800UAEbSGZKpNIl40AcfLS7XkBgbjhltSpPrQ1es4SYf2yTGzFjWFqwFNDiSoqsvyfL24Jv8mkVBV8/ju/azccd+brhoefZxC+preOjmyw+b/HXqombufDxY72h5TiBY0T7WOihsEQDZpHS0paaISK4jBgIzixEsGHcmgLvfPyOlmiV9OV1C/cMpEo1BIIi6gTpyhmNG5woV7vS1vL2ebV0D2RFDy8LWQmMizvL2em5/9AVG0pnDNocZbwbwKYubieJMbuWf2zrIDQTRcNFNew5RE6uiua6YieQiUmmO2DXk7mngGTNbMkPlmVX9w6lsd0punqA/OUpjbZyOxpq8eQRNtZNXrMvDtYB2hHmC3G/ypy5qpndghES8iletaJv0uaLuJMiv/Jtrq7NDRhfndQ0FgWB7Vz8djTXajlJExlVMjqAD2GRm95nZXdFPqQs201LpDEOjaRaF36hzRwr1J4MNaDqbEtkVRvsnaBEUWtZWz0gqwyPhXga5Sd5oJNB5J7Zlh4IeyWnh9a311TTX5i8etyxsISzKaRG0h0Et4xoxJCITK6av4FMlL8UM29E9wBf+bTOfueaV2TzAQDJY92dRcx1Pv3gov0UwnGJVZzxvhdHiWwTBN/cNW7pZUF9NS/1YBR59w7+0yD2DF9TXsLilluOaD+/rX95WzzMvHaQ9Z7exRDxGS101B4dGNatYRCZUTLK47PICD2/v4e4nXuK9l63ilDBpeyicPRz1sfcXtAgawxYBQFf/cNgimHxJ56gvf+u+fs5c2pJ338UndXDdect4y9nF97zd9LqTaRinJfKO85dx6uLmw3ILnU0JDg6NauioiEyomJnFfUA0FCYOxICkuzeXsmClNBKu8pm7vn/UAoi6VnJbBIeGU2GOIKhM9x1K0j9y+Kih8SxuqSVeZaQynu2+iTQk4vyvt50xpbK//VXj7yh2/sp2zl/Zftj5jsYatu5DK4+KyISKaRFkM5RmVgW8DTizlIUqtWjtn/0DY2sIRRX/8QvCHEF4nEylGUllsjkCCOYDuI8tOHck8VgVS1vr2NEzyPK2+kmvP9Y6m4L3oxaBiEyk2EXnAHD3jLvfSbDr2LyVDANB72BOiyDsClrUXJd3HOUOGhNjLYJoBFAxLQIYS+TmjvSZKdEoKCWLRWQixXQNvTnnsIpg3aF5PQ5xrEUwFgiiFkBnUw3xKstuPDO2ymg1C8IVRp+PAkERLQIg2xKYnRZBEADUIhCRiRRTk12bczsF7ACuLklpZsi4OYLsRjPVNNbGxxagCwNCU22wrlB7Y81YICiyRbB6YSNVBid2zvxG8UvCvQmiPQpERAoVkyOYs3sMH61siyC3ayjaejIRpzERH1uSOgoQ0QY0TQk27+3POzeZt689gTOWtHBc08wv8fDGMxaztLUuu/6RiEihSXMEZvaNcNG56LjVzL5W2mKVVhQIClsEZlBfEwsCwXD+SqTRt/+OxkT28cW2CGqrY5y9rPWYlX8q4rEqzl0++axlEalcxSSLz3H3A9GBu+8n2JNg3hqvRdAXzhUwM5pyuoYKN6DJnZhVbI5ARGQuKyYQVJlZdiaUmbUCk8+kmsOiHEHe8NHhVLarpzERzwaAvoItKTtzRt801c7rfwYREaC4ZPHfAQ+Z2e0EE8v+APhMSUtVYuN1DfUNj00Qa6ytZke4reTYJvVj+wBH1CIQkXJQTLL4NjN7DHgtwbDR33f3p0peshKK5hEMjaYZGklTVxPLLiMB5OUI+oZHiVUZtdVB4ylqEdTXxIiNs1S0iMh8U8w8glcBm9z9yfC4yczWuvvGkpeuRKKuIQgmlS2pqaMvmaKlLvjW31Qbz5tHEOUOYGyClloDIlIuiskR3AoM5hwPAP9YmuLMjJFUOns7mlTWPzyazRE0JeIMj2YYTWeySeTIcWGLoNgRQyIic11RyWJ3z36FDm/P6yzpSCqT7eqJ8gR5XUNhJT+QTAVJ5JxKP8oRFDuHQERkrismEDxvZu81s5iZVZnZnxDMLp63RtIZFoVr+kdDSPtzk8VhJd83nMoLEAAtddVUx0wtAhEpG8UEgj8CLgf2hj+XAv+1lIUqtZFUhoVhIOgdGCGdcQZG0tkKP2oB9CdTh21Sb2Z0NCaUIxCRslHMqKG9wDW558zsbGDfkR5nZmuA23NOrQQ+DnwnPL+CoGXx9nCS2owZSWXobEpQZUGOYGAkWmcoahEEPV/9YdfQsoLlGT76e2uygUREZL4rehlqMzvZzD5uZs8Ct012vbs/5+5nuftZBDORB4EfAjcD97v7auD+8HhGBTmCGAvqa+gdHMlZcC4/R9A/nKJvnC0p33r2Ui5a1TGzhRYRKZEjtgjMbCnBBLLrCHYmOwE43923TvF1Lge2uftOM7sauCw8/23gQeDPp/h805JMZaiJV9FaX83+gdGcZSSqw99hjiCbLJ7XuXERkSOasEVgZuuBfwOagP8cfrM/dBRBAIJg8r3w9kJ33wMQ/j7uKJ5vWkZSGWpiVbQ11NA7MHLYMhJRC+Dg4AhDo2nlA0SkrB2pa6gPqANaCIIBjO1dXDQzqwHeDHx/io97j5ltNLONXV1dU33ZI0qmMyTiQSDYPzhC3/DYEtS5v/ccHM47FhEpRxMGAnd/I3A28AzwaTPbBrSa2TlTfI03AI+HSWeAvWa2GCD8PW7S2d1vdfe17r62s7Nzii85MXcPWgTxsRZBdj2hsCVQXxPDLCcQaKioiJSxIyaL3b03rJBfA6wDPgF81cx2TuE1rmOsWwjgHuCG8PYNwN1TeK5pG00HjZqaWBWt9VGLIH+paTOjMRFnz8EhQJPHRKS8FT1qyN1fcvcvuPt5wGuKeYyZ1RNsdH9XzulPAa83sy3hfZ+aQnmnLVpnKGoRjKadl8f55t9cWz3ueRGRcnNUNZy7by/yukGgveBcD8EoolkRLUFdE6+iORwNtKs3WEqpoWbsn6MxEWdHz9Q2qRcRmY+KbhGUi9xA0NYQrCS6q3eQhoJlpRtr49nlqgvnEYiIlJOyDgQ/eWoPf/vT5/LOZQNBrIrWnEBQ2P2T2wqI5heIiJSjYvYj6AD+C8GSENnr3f09pSvWsfHw9h5+9MRLfOiKNdlzI+lgCeqaeBVt9UEg6OpLsqqzIe+xuYFBOQIRKWfF1HB3Aw8DvwTSk1w7p9RWxxgezS9y1N2TiFfR2jD2Tb+xYPZwU3YEEdRXx0pcUhGR2VNMIGhw9w+XvCQlUFsdI5nKkMk4VWH/f26OoDERpzpmjKb9sCGi2cllNfHsY0VEylExOYKfmNkVJS9JCdSG3+SjVgDk5ghimFk2YVw4MqixYAE6EZFyVUwg+GPgXjPrN7NeM9tvZr2lLtixEO1Clts9lDuPAKA1zBNMlCzW0FERKXfF1HLzdr3lurBFMDSapjU8l9s1BGRbBIVDRJsKFqATESlXE9ZyZrba3bcAr5jgkidLU6RjJ+oaymsR5CSLgewQ0sNzBOGS1FqCWkTK3JG+7t4M3Ah8eZz7HLikJCU6hqKuoaEjdA21TdA1lG0RqGtIRMrchLWcu98Y/l43c8U5tsZaBGPJ4mTOhDIYaxEUThor3MheRKRcFVXLmdkpwGlAdqNed//nUhXqWMmOGjpC11BbfdQFVNAiSGjUkIhUhmJmFv8VcAVwCnAfcCXB5LI5Hwhyk8WRwmTxhDkCtQhEpEIUM3z09wmWnd7j7tcDZ3KUq5bOtPG6hgpzBMvbg6UlFrXU5j12QV0NddUxjl+Qf15EpNwUU6EPuXvazFJm1gS8DKwscbmOiXED1P2WAAAPUUlEQVSTxQU5grNOWMCGj76GE9rq8x5bVxPj5x+5lI7GxAyVVkRkdhQTCH5jZguAbwIbgUPA4yUt1TFSN8Hw0SqDeGysMVQYBCKLW+pKW0ARkTngiIHAzAy4xd0PAF82s/uAZnefF4EgMV4gSGey3UIiIjL5nsUO/GvO8db5EgRg4hZBTUyBQEQkUkyN+GszO6fkJSmB6phRZYfPI6iJa1lpEZHIkZaYiLt7Cng18G4z2wYMAEbQWJjzwcHMqK2OHZYsTqhrSEQk60g5gl8D5wBvmaGylERdweY0yhGIiOQ7UiAwAHffNkNlKYlgl7Lc/QjSyhGIiOQ4UiDoNLMPTXSnu/9tCcpzzCWqqw5PFqtFICKSdaRAEAMaCVsG81Vh11BSgUBEJM+RAsEed//EjJWkRGqrYwynNHxURGQiR6oR53VLIFJXHWNoRMliEZGJHKlGvHzGSlFCtdVVBcliBQIRkVwT1ojuPi82qJ9MonD4qAKBiEiesq8Rx0sWJ5QjEBHJKvsasba6iuFU/n4EahGIiIwpaY1oZgvM7E4ze9bMNpnZhWZ2i5m9aGZPhD9XlbIMhyWL1TUkIpKn1DuN/T1wr7tfY2Y1QD3BVpdfcPfPlfi1gbHho+6OmWmtIRGRAiWrEc2sGbgE+AaAu4+E+xrMqNrqGO5BbgDUNSQiUqiUNeJKoAu4zcx+Y2ZfN7OG8L73m9mTZvZNM2stYRmy+xYnRzOkM04649TEtAy1iEiklIEgTrB66Vfc/WyCJaxvBr4CrALOAvYAnx/vwWb2HjPbaGYbu7q6jroQ0b7Fw6n02H7FahGIiGSVskbcDex290fC4zuBc9x9r7un3T0DfA04b7wHu/ut7r7W3dd2dnYedSFqw01ohkYUCERExlOyGtHdXwZeMLM14anLgWfMbHHOZW8Fni5VGQDqasLtKlNpkulg9JACgYjImFKPGvoA8N1wxNB24F3AF83sLMCBHcAflbIAUdfQ0EiakUTQItCEMhGRMSUNBO7+BLC24PT1pXzNQrXZDewz6hoSERlH2deI2UCQSjOSViAQESlU9jVilCwezk0Wq2tIRCSr7GvE3GSxuoZERA5X9jVidh6BcgQiIuMq+xqxrnpsHkFSOQIRkcOUfY2YlyxWjkBE5DBlXyNGK43mJou1+qiIyJiyrxHNLLs5jXIEIiKHq4gasTbcrlLzCEREDlcRNWJtPJa/6JxyBCIiWRVRI9bVxNQ1JCIygYqoERPxqqBFoK4hEZHDVESNWFcTI5lKZ7erVNeQiMiYiqgRa+NhsjiVoSZWhZnNdpFEROaMyggE1VUMRYFA3UIiInkqolasq4kxPJohmUprMpmISIGKqBXzuoYUCERE8lRErVhbMzahTIFARCRfRdSKQYsgk00Wi4jImIqoFZUsFhGZWEXUinXVMdIZZ3AkrUAgIlKgImrFaE+Cg0Oj6hoSESlQEbVitF3loeFRtQhERApURK2Y2yLQPAIRkXwVUStGgaA/mVKLQESkQEXUitEG9u5acE5EpFBF1IpRiwC0BLWISKGKqBWjZDEoEIiIFKqIWjGvRRCLHeFKEZHKU3mBQC0CEZE8FVErqmtIRGRiJa0VzWyBmd1pZs+a2SYzu9DM2szsZ2a2JfzdWsoywNioIUDzCERECpS6Vvx74F53PwU4E9gE3Azc7+6rgfvD45LKzxEoEIiI5CpZrWhmzcAlwDcA3H3E3Q8AVwPfDi/7NvCWUpUhohyBiMjESlkrrgS6gNvM7Ddm9nUzawAWuvsegPD3ceM92MzeY2YbzWxjV1fXtAoSq7JsS0CBQEQkXylrxThwDvAVdz8bGGAK3UDufqu7r3X3tZ2dndMuTCJMGKtrSEQkXylrxd3Abnd/JDy+kyAw7DWzxQDh730lLENWlDBWi0BEJF/JakV3fxl4wczWhKcuB54B7gFuCM/dANxdqjLkivIEGjUkIpIvXuLn/wDwXTOrAbYD7yIIPneY2Y3ALuDaEpcBUItARGQiJQ0E7v4EsHacuy4v5euOJ5pUpkAgIpKvYmrFhLqGRETGVTG1YrZrSIvOiYjkqZhAoK4hEZHxVUytWKtksYjIuCqmVtSoIRGR8VVMrZhtEWhmsYhInoqpFdU1JCIyvoqpFaNksYaPiojkK/XM4jnjTa88nkQ8lrcktYiIVFAgOOm4Rk46rnG2iyEiMueon0REpMIpEIiIVDgFAhGRCqdAICJS4RQIREQqnAKBiEiFUyAQEalwCgQiIhXO3H22yzApM+sCdk7hIR1Ad4mKM5fpfVeWSn3fULnvfarve7m7d0520bwIBFNlZhvdfby9ksua3ndlqdT3DZX73kv1vtU1JCJS4RQIREQqXLkGgltnuwCzRO+7slTq+4bKfe8led9lmSMQEZHilWuLQEREilR2gcDMfs/MnjOzrWZ282yXp1TM7AQze8DMNpnZ78zsg+H5NjP7mZltCX+3znZZjzUzi5nZb8zsX8PjE83skfA9325mNbNdxlIwswVmdqeZPRt+7hdWyOd9U/g3/rSZfc/MasvxMzezb5rZPjN7OufcuJ+vBb4Y1nNPmtk503ntsgoEZhYDvgy8ATgNuM7MTpvdUpVMCviwu58KXAD8Sfhebwbud/fVwP3hcbn5ILAp5/jTwBfC97wfuHFWSlV6fw/c6+6nAGcS/BuU9edtZkuAPwXWuvvpQAz4A8rzM/8W8HsF5yb6fN8ArA5/3gN8ZTovXFaBADgP2Oru2919BPgX4OpZLlNJuPsed388vN1HUCksIXi/3w4v+zbwltkpYWmY2VLgjcDXw2MDXgvcGV5Sdu8ZwMyagUuAbwC4+4i7H6DMP+9QHKgzszhQD+yhDD9zd18P9BacnujzvRr4jgceBhaY2eKjfe1yCwRLgBdyjneH58qama0AzgYeARa6+x4IggVw3OyVrCT+DvgokAmP24ED7p4Kj8v1M18JdAG3hd1iXzezBsr883b3F4HPAbsIAsBB4DEq4zOHiT/fY1rXlVsgsHHOlfWwKDNrBH4A/Jm7H5rt8pSSmb0J2Ofuj+WeHufScvzM48A5wFfc/WxggDLrBhpP2Cd+NXAicDzQQNAtUqgcP/MjOaZ/9+UWCHYDJ+QcLwVemqWylJyZVRMEge+6+13h6b1REzH8vW+2ylcCFwNvNrMdBN1+ryVoISwIuw2gfD/z3cBud38kPL6TIDCU8+cN8DrgeXfvcvdR4C7gIirjM4eJP99jWteVWyB4FFgdjiioIUgq3TPLZSqJsG/8G8Amd//bnLvuAW4Ib98A3D3TZSsVd/8Ld1/q7isIPtufu/s7gAeAa8LLyuo9R9z9ZeAFM1sTnroceIYy/rxDu4ALzKw+/JuP3nfZf+ahiT7fe4A/DEcPXQAcjLqQjoq7l9UPcBWwGdgG/OVsl6eE7/PVBE3BJ4Enwp+rCPrM7we2hL/bZrusJXr/lwH/Gt5eCfwa2Ap8H0jMdvlK9J7PAjaGn/mPgNZK+LyBvwGeBZ4G/glIlONnDnyPIA8ySvCN/8aJPl+CrqEvh/XcUwSjqo76tTWzWESkwpVb15CIiEyRAoGISIVTIBARqXAKBCIiFU6BQESkwikQSMmYmZvZ53OOP2Jmtxyj5/6WmV0z+ZXTfp1rw5U+Hyg4vyJ8fx/IOfclM3vnJM/3x2b2h5Nc804z+9IE9/VPofhTFr6v3NUv321mj5fjqqYyRoFASikJvM3MOma7ILnCVWqLdSPwPnd/zTj37QM+OJUlkN39q+7+nSm8/jGTMxO32OuvBz4AXOHu+0tTKpkLFAiklFIEW+vdVHhH4Tf66JuumV1mZr8wszvMbLOZfcrM3mFmvzazp8xsVc7TvM7MNoTXvSl8fMzMPmtmj4brtP9RzvM+YGb/TDABp7A814XP/7SZfTo893GCiXtfNbPPjvP+uggm+dxQeIeZrTKze83ssbCMp4TnbzGzj4S3XxWW8aGwzE/nPMXx4eO3mNlnCp778+G39PvNrDM8d5aZPRw+3w9tbN36B83sk2b2C4KgdW34Hn9rZuvHeU/Ra7ydYC2jK9y9e6LrpEzM9mw6/ZTvD9APNAM7gBbgI8At4X3fAq7JvTb8fRlwAFhMMIP0ReBvwvs+CPxdzuPvJfgys5pgJmYtwdrsfxVekyCYiXti+LwDwInjlPN4gqUMOgkWd/s58JbwvgcZZ9YmsIJgpuuJBLNeY8CXgHeG998PrA5vn0+wHAbALcBHwttPAxeFtz8FPB3efiewPfw3qwV2AieE9znwjvD2x4EvhbefBC4Nb38i59/pQeAfcsr9FLAkvL1ggvfVR9DaWTLbf0P6mZkftQikpDxYEfU7BJuLFOtRD/ZbSBJMof9peP4pgooqcoe7Z9x9C0HFeQpwBcEaLE8QLMvdThAoAH7t7s+P83qvAh70YGGzFPBdgrX/i3l/zxMsdfCfonPhirAXAd8Py/GPBIGNnGsWAE3u/u/hqX8ueOr73f2guw8TrK2zPDyfAW4Pb/8f4NVm1kJQqf8iPP/tgvLfnnP7V8C3zOzdBMFrPF0EgfHtE75xKStT6jMUOUp/BzwO3JZzLkXYNRkuJpbbz57MuZ3JOc6Q/zdbuD6KE6zB8gF3vy/3DjO7jKBFMJ7xlvSdik8SrAYadbVUEayXf9YRHjPZa+b+G6SZ+P9qMWvEZN+3u/+xmZ1PsLnPE2Z2lrv3FFw/SLDU8y/NbJ+7f7eI15B5TC0CKTl37wXuIH87wR3AueHtq4Hqo3jqa82sKswbrASeA+4D3mvBEt2Y2ckWbOByJI8Al5pZR5hIvg74xSSPyXL3Zwm+tb8pPD4EPG9m14ZlMDM7s+Ax+4G+cOVICFZTLUYVY6tu/ifgl+5+ENhvZuvC89dPVH4zW+Xuj7j7x4Fu8pcyzi1fF8G2iZ80syuLLJvMU2oRyEz5PPD+nOOvAXeb2a8J+tMn+rZ+JM8RVHgLgT9292Ez+zpB99HjYUuji0m2MXT3PWb2FwRLGxvwY3ef6rLG/xP4Tc7xO4CvmNlfEQS5fwF+W/CYG4GvmdkAQV/+wSJeZwB4hZk9Fl7/++H5GwiS2vUE3WTvmuDxnzWz1QTv8/5xypTl7s+b2ZuBH5vZ23xsLwQpM1p9VGSWmFmju0ejpW4GFrv7B2e5WFKB1CIQmT1vDFsicYKRQe+c3eJIpVKLQESkwilZLCJS4RQIREQqnAKBiEiFUyAQEalwCgQiIhVOgUBEpML9f09okUuRdqPgAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(range(1,100),score)\n",
    "plt.xlabel('Number of Neighbors K')\n",
    "plt.ylabel('Train Accuracy')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy :  83.5 %\n"
     ]
    }
   ],
   "source": [
    "knn=KNeighborsClassifier(n_neighbors=41,weights='uniform')\n",
    "knn.fit(X_train,y_train)\n",
    "predKNN=knn.predict(X_test)\n",
    "accuracy=metrics.accuracy_score(predKNN,y_test)\n",
    "print(\"accuracy : \",round(accuracy,3)*100,'%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      " COLLECTION       0.67      0.16      0.26        25\n",
      "    PAIDOFF       0.84      0.98      0.91       114\n",
      "\n",
      "avg / total       0.81      0.83      0.79       139\n",
      "\n",
      "\n",
      "\n",
      "Jaccard Similarity Score :  83.45 %\n",
      "\n",
      "\n",
      "F1-SCORE :  [0.25806452 0.90688259]\n",
      "\n",
      "\n",
      "Train Accuracy:  72.46376811594203 %\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import classification_report,jaccard_similarity_score,log_loss,f1_score\n",
    "print(classification_report(y_test,predKNN))\n",
    "print('\\n')\n",
    "print('Jaccard Similarity Score : ',round(jaccard_similarity_score(y_test,predKNN)*100,2),'%')\n",
    "print('\\n')\n",
    "print('F1-SCORE : ',f1_score(y_test,predKNN,average=None))\n",
    "print('\\n')\n",
    "print('Train Accuracy: ',metrics.accuracy_score(y_train, knn.predict(X_train))*100,'%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Decision Tree"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\magic\\Anaconda3\\lib\\site-packages\\sklearn\\grid_search.py:42: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.\n",
      "  DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.grid_search import GridSearchCV\n",
    "dtree=DecisionTreeClassifier()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best Score: 0.7439613526570048\n",
      "Best params: {'criterion': 'entropy', 'max_depth': 5, 'max_features': 2, 'random_state': 0}\n"
     ]
    }
   ],
   "source": [
    "parameter_grid = {'max_depth': [1, 2, 3, 4, 5,6,5,9,15,20],\n",
    "                  'max_features': [1, 2, 3, 4,5,6,7,8],\n",
    "                 'random_state':[0,15,20,35,50,80,100,150,180,200],\n",
    "                 'criterion':['gini','entropy'],\n",
    "                 }\n",
    "\n",
    "grid_search = GridSearchCV(dtree, param_grid = parameter_grid,\n",
    "                          cv =10)\n",
    "\n",
    "grid_search.fit(X_train, y_train)\n",
    "\n",
    "print (\"Best Score: {}\".format(grid_search.best_score_))\n",
    "print (\"Best params: {}\".format(grid_search.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "dtree=DecisionTreeClassifier(max_depth=5,criterion='entropy',max_features=2,random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [],
   "source": [
    "dtree.fit(X_train,y_train)\n",
    "pred_Dtree=dtree.predict(X_test)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      " COLLECTION       0.32      0.40      0.36        25\n",
      "    PAIDOFF       0.86      0.82      0.84       114\n",
      "\n",
      "avg / total       0.76      0.74      0.75       139\n",
      "\n",
      "\n",
      "\n",
      "Jaccard Similarity Score :  74.1 %\n",
      "\n",
      "\n",
      "F1-SCORE :  [0.35714286 0.83783784]\n",
      "\n",
      "\n",
      "Train Accuracy:  77.29468599033817 %\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_test,pred_Dtree))\n",
    "print('\\n')\n",
    "print('Jaccard Similarity Score : ',round(jaccard_similarity_score(y_test,pred_Dtree)*100,2),'%')\n",
    "print('\\n')\n",
    "print('F1-SCORE : ',f1_score(y_test,pred_Dtree,average=None))\n",
    "print('\\n')\n",
    "print('Train Accuracy: ',metrics.accuracy_score(y_train, dtree.predict(X_train))*100,'%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Support Vector Machine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [],
   "source": [
    "svm=SVC().fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [],
   "source": [
    "pred_svm=svm.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      " COLLECTION       0.33      0.24      0.28        25\n",
      "    PAIDOFF       0.84      0.89      0.87       114\n",
      "\n",
      "avg / total       0.75      0.78      0.76       139\n",
      "\n",
      "\n",
      "\n",
      "Jaccard Similarity Score :  77.7 %\n",
      "\n",
      "\n",
      "F1-SCORE :  [0.27906977 0.86808511]\n",
      "\n",
      "\n",
      "Train Accuracy:  75.84541062801932 %\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_test,pred_svm))\n",
    "print('\\n')\n",
    "print('Jaccard Similarity Score : ',round(jaccard_similarity_score(y_test,pred_svm)*100,2),'%')\n",
    "print('\\n')\n",
    "print('F1-SCORE : ',f1_score(y_test,pred_svm,average=None))\n",
    "print('\\n')\n",
    "print('Train Accuracy: ',metrics.accuracy_score(y_train, svm.predict(X_train))*100,'%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.linear_model import LogisticRegression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "lgm=LogisticRegression()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,\n",
       "          intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,\n",
       "          penalty='l2', random_state=None, solver='liblinear', tol=0.0001,\n",
       "          verbose=0, warm_start=False)"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "lgm.fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "pred_lgm=lgm.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      " COLLECTION       0.33      0.24      0.28        25\n",
      "    PAIDOFF       0.84      0.89      0.87       114\n",
      "\n",
      "avg / total       0.75      0.78      0.76       139\n",
      "\n",
      "\n",
      "\n",
      "Jaccard Similarity Score :  77.7 %\n",
      "\n",
      "\n",
      "F1-SCORE :  [0.27906977 0.86808511]\n",
      "\n",
      "\n",
      "Train Accuracy:  73.91304347826086 %\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_test,pred_lgm))\n",
    "print('\\n')\n",
    "print('Jaccard Similarity Score : ',round(jaccard_similarity_score(y_test,pred_lgm)*100,2),'%')\n",
    "print('\\n')\n",
    "print('F1-SCORE : ',f1_score(y_test,pred_lgm,average=None))\n",
    "print('\\n')\n",
    "print('Train Accuracy: ',metrics.accuracy_score(y_train, lgm.predict(X_train))*100,'%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Model Evaluation using Test set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import jaccard_similarity_score\n",
    "from sklearn.metrics import f1_score\n",
    "from sklearn.metrics import log_loss"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, download and load the test set:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "source": [
    "### Load Test set for evaluation "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "button": false,
    "new_sheet": false,
    "run_control": {
     "read_only": false
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/8/2016</td>\n",
       "      <td>10/7/2016</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>9/9/2016</td>\n",
       "      <td>9/15/2016</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>9/10/2016</td>\n",
       "      <td>10/9/2016</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>9/11/2016</td>\n",
       "      <td>9/25/2016</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30       9/8/2016   \n",
       "1           5             5     PAIDOFF        300      7       9/9/2016   \n",
       "2          21            21     PAIDOFF       1000     30      9/10/2016   \n",
       "3          24            24     PAIDOFF       1000     30      9/10/2016   \n",
       "4          35            35     PAIDOFF        800     15      9/11/2016   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0  10/7/2016   50              Bechalor  female  \n",
       "1  9/15/2016   35       Master or Above    male  \n",
       "2  10/9/2016   43  High School or Below  female  \n",
       "3  10/9/2016   26               college    male  \n",
       "4  9/25/2016   29              Bechalor    male  "
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df = pd.read_csv('loan_test.csv')\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-09-15</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30     2016-09-08   \n",
       "1           5             5     PAIDOFF        300      7     2016-09-09   \n",
       "2          21            21     PAIDOFF       1000     30     2016-09-10   \n",
       "3          24            24     PAIDOFF       1000     30     2016-09-10   \n",
       "4          35            35     PAIDOFF        800     15     2016-09-11   \n",
       "\n",
       "    due_date  age             education  Gender  \n",
       "0 2016-10-07   50              Bechalor  female  \n",
       "1 2016-09-15   35       Master or Above    male  \n",
       "2 2016-10-09   43  High School or Below  female  \n",
       "3 2016-10-09   26               college    male  \n",
       "4 2016-09-25   29              Bechalor    male  "
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df['due_date'] = pd.to_datetime(test_df['due_date'])\n",
    "test_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>Unnamed: 0.1</th>\n",
       "      <th>loan_status</th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>effective_date</th>\n",
       "      <th>due_date</th>\n",
       "      <th>age</th>\n",
       "      <th>education</th>\n",
       "      <th>Gender</th>\n",
       "      <th>dayofweek</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-08</td>\n",
       "      <td>2016-10-07</td>\n",
       "      <td>50</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>female</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5</td>\n",
       "      <td>5</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>2016-09-09</td>\n",
       "      <td>2016-09-15</td>\n",
       "      <td>35</td>\n",
       "      <td>Master or Above</td>\n",
       "      <td>male</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>21</td>\n",
       "      <td>21</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>43</td>\n",
       "      <td>High School or Below</td>\n",
       "      <td>female</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>24</td>\n",
       "      <td>24</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>2016-09-10</td>\n",
       "      <td>2016-10-09</td>\n",
       "      <td>26</td>\n",
       "      <td>college</td>\n",
       "      <td>male</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>35</td>\n",
       "      <td>35</td>\n",
       "      <td>PAIDOFF</td>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>2016-09-11</td>\n",
       "      <td>2016-09-25</td>\n",
       "      <td>29</td>\n",
       "      <td>Bechalor</td>\n",
       "      <td>male</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n",
       "0           1             1     PAIDOFF       1000     30     2016-09-08   \n",
       "1           5             5     PAIDOFF        300      7     2016-09-09   \n",
       "2          21            21     PAIDOFF       1000     30     2016-09-10   \n",
       "3          24            24     PAIDOFF       1000     30     2016-09-10   \n",
       "4          35            35     PAIDOFF        800     15     2016-09-11   \n",
       "\n",
       "    due_date  age             education  Gender  dayofweek  \n",
       "0 2016-10-07   50              Bechalor  female          3  \n",
       "1 2016-09-15   35       Master or Above    male          4  \n",
       "2 2016-10-09   43  High School or Below  female          5  \n",
       "3 2016-10-09   26               college    male          5  \n",
       "4 2016-09-25   29              Bechalor    male          6  "
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df['dayofweek'] = test_df['effective_date'].dt.dayofweek\n",
    "test_df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_df['weekend']=test_df['dayofweek'].apply(lambda x: 1 if (x>3) else 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x1928af68e80>"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAE5CAYAAACwKcrAAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XvcpXO9//HX2zhGTkUOQ4RINkMTSiVE2BWJ0ElSOuxK7dLOru1UdufUbpdy9pNCpCY5TYSU02AwDJGmTJRdznKaud+/P77f5V73Peu+57rXdV33On2eHtdj7uta1/qs79zWfNd3fQ+fr2wTQgihdyzR6QKEEEKYmKi4Qwihx0TFHUIIPSYq7hBC6DFRcYcQQo+JijuEEHpMVNwhhNBjouIOIYQeU6rilrSrpDsl3S3ps1UVKoQQwtjU7spJSVOA3wM7A/OB64H9bd8+1nNmTd0zlmmGEAqZPv9nKhvj2b/fU6jOWeqFLyn9WpNpyRLP3Rq42/Y9AJLOBPYAxqy4QwhhUg0t7HQJalGmq2Rt4N6m8/n52giSDpY0S9Ksnz4xr8TLhRDCBHmo2NFjyrS4W321WORrie3jgeMhukpCCJNsqPcq5SLKVNzzgXWazqcC95UrTgghVMc92JouokzFfT2wkaT1gb8A+wHvqKRUIYRQhWhxj2R7gaSPAhcDU4CTbd9WWclCCKGsaHEvyvYFwAUVlSWEEKq18NlOl6AWpSruEELoatFVEkIIvSUGJ0MIoddEi3skScsCVwLL5Djn2D6iqoKFEEJp0eJexNPAjrYfl7QUcJWkC21fU1HZQgihnBicHMkpO9Xj+XSpfMTKyBBC9+jTrpKyaV2nSJoNPADMtH1ti3siV0kIoTP6NFdJqYrb9kLb00jL3beWtFmLe463Pd329L2WX6/My4UQwsQMDRU7ekwlO+DYfhi4HNi1inghhFAFe2Gho9e0XXFLWk3Syvnn5YA3AHdUVbAQQigtukoWsSbwa0m3kBJOzbR9fjXFCiGECixcUOwoYHFbNUpaRtJZ+fFrJa2Xry8l6TRJt0qaK+mwsn+tMrNKbgG2LFuAEEKoTUU74OStGr9L01aNkmaM2qrxIOAh2xtK2g/4CrAvsA+wjO1/kfQ84HZJP7Y9r93yxC7vIYT+VV1XyXNbNdp+Bmhs1dhsD+C0/PM5wE6SRJomvbykJYHlgGeAR8v8taLiDiH0r+pmlRTZqvG5e2wvAB4BXkCqxJ8A7gf+DHzd9oNl/lpRcYcQ+lfBFnfzepN8HDwqUpGtGse6Z2tgIbAWsD7wKUkvKfPXKpVkKs8qORHYLBfwfbavLhMzhBAqU3COdvPeuGMoslVj4575uVtkJeBB0s5gF9l+FnhA0m+B6cA9hQrXQtkW97dzgTYBtgDmlowXQgiV8cJnCx0FPLdVo6SlSVs1zhh1zwzggPzz3sBlOTXIn4EdlSwPbEvJqdNlsgOuCLwOeC9A7rB/pkxhQgihUhWtihxrq0ZJRwOzbM8ATgJOl3Q3qaW9X376d4FTgDmk7pRT8qy8tpXpKnkJ8H/AKZK2AG4ADrH9RPNNua/oYIDDVt6CWPYeQpg0FS6uabVVo+3Dm35+ijT1b/TzHm91vYwyXSVLAlsBx9nekjRqusik9MhVEkLomMhVsoj5wPymjIDnkCryEELoDn265L3Mysm/SrpX0sa27wR2Am5f3PNCCGHSFFzO3mvK7jn5MeCMPMp6D3Bg+SKFEEJFerAbpIhSFbft2aT5iCGE0H2i4g4hhB7Tg/3XRUTFHULoX9HiDiGEHtOnLe6ymwUfImmOpNskfaKqQoUQQiUq3Eihm5RZ8r4Z8AFS5qtngIsk/dL2XVUVLoQQSunTrpIyLe6XAdfY/mfOPXsF8NZqihVCCBWIlZOLmAO8TtIL8nY8uzMy7SHAiDy3P31iXomXCyGECbKLHT2mzMrJuZK+AswEHgduBhbpLGrOcztr6p699xsKIfSuHmxNF1FqcNL2Sba3sv06UhrD6N8OIXSPPu0qKbsDzuq2H5C0LrAX8KpqihVCCBXowRkjRZSdx32upBcAzwL/ZvuhCsoUQgjV6MH+6yLK5ip5bVUFCSGEyvVgN0gRsXIyhNC/ouIOIYQe06dL3qPiDiH0LS9Y2Oki1GKx0wElnSzpAUlzmq59TdIdkm6RdJ6klestZgghtKFPty4rMo/7VGDXUddmApvZ3hz4PXBYxeUKIYTyhlzs6DGLrbhtX0laXNN87ZKcnwTgGmBqDWULIYRy+nQBTqmVk9n7gAvHejBylYQQOiYq7kVJ+hwpP8kZY91j+3jb021P32v59cq8XAghTEwkmRpJ0gHAm4Cd7B78m4cQ+l+fzippq+KWtCvwH8D2tv9ZbZFCCKEiPThjpIgi0wF/DFwNbCxpvqSDgP8Fng/MlDRb0vdrLmcIIUxcn84qWWyL2/b+LS6fVENZQgihUu7BgcciYuVkCKF/9WBruoiouEMI/atP+7ij4g4h9K8+nVXSbq6SIyX9JQ9Mzpa0e73FDCGENvTp4GS7uUoAjrU9LR8XVFusEEKoQJ8mmSoyq+RKSevVX5QQQqhYD7amiyiz5P2jOa3ryZJWqaxEIYRQEQ8NFTqKkLSrpDsl3S3psy0eX0bSWfnxa0c3eCWtK+lxSZ8u+/dqt+I+DtgAmAbcD3xjrBsjyVQIoWMWDBU7FkPSFOC7wG7ApsD+kjYdddtBwEO2NwSOBb4y6vFjGSch30S0VXHb/pvthbaHgBOArce5N5JMhRA6o7o+7q2Bu23fY/sZ4Exgj1H37AGcln8+B9hJkgAk7QncA9xWxV+rrYpb0ppNp28F5ox1bwghdEzBWSXNPQP5OHhUpLWBe5vO5+drLe/J+xU8ArxA0vKk3E5HVfXXWuzgZM5V8nrghZLmA0cAr5c0DTAwD/hgVQUKIYSquODgpO3jgePHuUWtnlbwnqNIs/Aezw3w0iJXSQihf1U3q2Q+sE7T+VTgvjHumS9pSWAl0u5h2wB7S/oqsDIwJOkp2//bbmFi5WQIoX9Vl2TqemAjSesDfwH2A94x6p4ZwAGkbKp7A5flvQpe27hB0pHA42UqbYiKO4TQzwrMGCnC9gJJHwUuBqYAJ9u+TdLRwCzbM0g9EadLupvU0t6vkhdvISruEELfqnJzrrxC/IJR1w5v+vkpYJ/FxDiyirK0m6tkmqRrcp6SWZLGnA4YQggdE7lKRvgqcJTtacDh+TyEELpLn1bc7eYqMbBi/nklFh1dDSGEjis6HbDXtNvH/QngYklfJ7XaX11dkUIIoSJ9WnG3m6vkw8Anba8DfJJx5nVHrpIQQqd4gQsdvabdivsA4Kf5558QuUpCCN2oT/u426247wO2zz/vCNxVTXFCCKFCQwWPHtNurpIPAN/OyzqfAkYnZAkhhI4b2MHJMXKVALyi4rKEEEK1erA1XUSsnAwh9K1eHHgsIiruEELf6sF9gAuJijuE0L/6tOIukqtkHUm/ljRX0m2SDsnX98nnQ5Km11/UEEKYmOp2LusuRVrcC4BP2b5R0vOBGyTNJG1XthfwgzoLGEIIbevBSrmIIrNK7ift5I7txyTNBda2PROgqq14Qgihar3Ymi5iQn3cOdnUlsC1dRQmhBCqNLSg0yWoR+GVk5JWAM4FPmH70Qk8L3KVhBA6wyp29JhCFbekpUiV9hm2f7q4+5tFrpIQQqcM7OCkUif2ScBc29+sv0ghhFAND/Vea7qIIn3c2wHvBm6VNDtf+09gGeA7wGrALyXNtv3GeooZQggT14ut6SKKzCq5ChjrY+u8aosTQgjVGVo4uC3uEELoSYPcVRJCCD3J/ZljKiruEEL/6tcWd9u5Spoe/7QkS3phfcUMIYSJ85AKHb2m7Vwltm+XtA6wM/DnWksZQght6NeuksW2uG3fb/vG/PNjwFxg7fzwscBngD799YQQetnQwiUKHb2m7Vwlkt4C/MX2zZFoKoTQjfp1HndbuUpI3SefAw4v8LzIVRJC6Ighq9DRa9rNVbIBsD5ws6R5wFTgRklrjH5u5CoJIXSKrUJHr2krV4ntW4HVm+6ZB0y3/feayhlCCBPWizNGiijS4m7kKtlR0ux87F5zuUIIoTS72NFryuYqadyzXlUFCiGEqizswRkjRcTKyRBC3+rF/usi+vPjKIQQqLarRNKuku6UdLekz7Z4fBlJZ+XHr83TpxuPHZav3ympdPrrqLhDCH2rqumAkqYA3wV2AzYF9pe06ajbDgIesr0haXHiV/JzNwX2A14O7Ap8L8drW9u5SvInS2Owcl7TJgshhNAVKpwOuDVwt+17bD8DnAnsMeqePYDT8s/nADvlWXl7AGfaftr2H4G7c7y2lclVsm/jBknfAB4pU5AQQqjawuqmA64N3Nt0Ph/YZqx7bC+Q9Ajwgnz9mlHPXZsSyuYqaczzfjvw4zIFCSGEqhVtcTev8M7HwaNCtfoEGN07PtY9RZ47IW3nKmm6/Frgb7bvKlOQEEKoWtHl7LaPB44f55b5wDpN51OB+8a4Z76kJYGVgAcLPndC2spVYvvRpof2Z5zWduQqCSF0igseBVwPbCRpfUlLkwYbZ4y6ZwZwQP55b+Ay287X98uzTtYHNgKua/9vVbDF3SJXSeP6ksBewCvGem7zJ9msqXv24BqlEEKvqiqBVO6z/ihwMTAFONn2bZKOBmbZnkFKDXK6pLtJLe398nNvk3Q2cDtpzPDfbC8sU562cpU0eQNwh+35ZQoRQgh1qHIBju0LgAtGXTu86eengH3GeO4xwDFVlaVsrpL9iEHJEEKXWogKHb2mVK4S2++tukAhhFCVoT7tnI1cJSGEvjXUg63pIqLiDiH0LUfFHUIIvaVPt5yMijuE0L/6tcVdJMnUspKuk3RzTjJ1VL6+fk5deFdOOLV0/cUNIYTiFhQ8ek2R6YBPAzva3gKYBuwqaVtSysJjbW8EPERKaRhCCF3DqNDRa4okmbLtx/PpUvkwsCMpdSGkVIZ71lLCEEJo05CKHb2mUK4SSVNyvu0HgJnAH4CHbTe+ZYyZpjBylYQQOmUIFTp6TaGK2/ZC29NIWa22Bl7W6rYxnnu87em2p++1/HptFzSEECaqwiRTXWVCs0psPyzpcmBbYGVJS+ZWd+k0hSGEULUF6r3WdBFFZpWsJmnl/PNypMRSc4Ffk1IXQkpl+PO6ChlCCO0Y5Bb3msBpeXPLJYCzbZ8v6XbgTElfBG4iZRAMIYSuMbALcGzfQtr1ZvT1eyi54WUIIdSpF2eMFBErJ0MIfasXZ4wUERV3CKFv9WL/dRFRcYcQ+taC/mxwF9q6bFngSmCZfP85to+QdBIwnbTJwu+B9zatsAwhhI7r1xZ3mVwln7S9he3NgT8DH62xnCGEMGH9uuS9yKwSA4vkKrH9KDy3mfBy9O+HWwihR/XrdMC2cpXYvjZfPwX4K7AJ8J0xnhu5SkIIHTFU8Og1beUqkbRZvn4gsBZpJeW+Yzw3cpWEEDrCKnb0mkIVd4Pth4HLgV2bri0EzgLeVmnJQgihpIHdSGGMXCV3StowXxPwZuCOOgsaQggTFblKmnKVAL8EfiNpRdJ0wJuBD9dWyhBCaEMvzhgpou1cJcB21RcnhBCq04sDj0XEyskQQt+KijuEEHrMwkHtKgkhhF7Vry3uIrNKlpV0naSbJd0m6ah8XZKOkfR7SXMlfbz+4oYQQnGDPKukkavkcUlLAVdJupC0YfA6wCa2hyStXmdBQwhhooZ6slpevLZzlZCm/73D9lC+74G6ChlCCO0Y2K4SGDNXyQbAvjkPyYWSNhrjuZGrJITQEf3aVVImV8kywFO2pwMnACeP8dzIVRJC6IgFKnb0mjK5SuYD5+aHzgM2r7RkIYRQ0hAudJQlaVVJMyXdlf9cZYz7Dsj33CXpgBaPz5A0Z3Gv126ukjuAnwE75tu2J+2CE0IIXWMSu0o+C1xqeyPg0nw+gqRVgSOAbYCtgSOaK3hJezE8njiuIi3uNYFfS7oFuJ7Ux30+8GXgbZJuBb4EvL/IC4YQwmSZxHzcewCn5Z9PA/Zscc8bSfXng7YfAmaSM61KWgH4d+CLRV6s7VwludvkX4u8SAghdELRbhBJBwMHN1063vbxE3ipF9m+H8D2/WNMj14buLfpfH6+BvAF4BvAP4u8WKycDCH0rYUF78uV9LgVtaRfAWu0eOhzBV+m1TCoJU0DNrT9SUnrFQkUFXcIoW9VuQDH9hvGekzS3yStmVvba5KmTo82H3h90/lU0mSPVwGvkDSPVCevLuly269nDBOaVRJCCL1kEgcnZwCNWSIHAD9vcc/FwC6SVsmDkrsAF9s+zvZattcDXgP8frxKG8rlKtlR0o2S5kg6TVK03kMIXWUSBye/DOws6S5g53yOpOmSTgSw/SCpL/v6fBydr01Yu7lKLiaNnO5k+/eSjiZ9ypzUTiFCCKEOnqR1kbb/AezU4vosmmbc2T6ZMRYr5sfnAZst7vUW2+J2MjpXyULgaduNudszic2CQwhdZhJb3JOqrVwlwHXAUpKm51v2JmUKbPXcyFUSQuiIhbjQ0WvaylUCvBzYDzhW0nXAY4yxy33kKgkhdMpkLXmfbBMaULT9sKTLgV1tfx14LYCkXYCXVl+8EEJoXy92gxTRdq6SxsogScsA/wF8v86ChhDCRLngf72mSIt7TeA0SVNIFf3Zts+X9DVJb8rXjrN9WZ0FDSGEierXFneZXCWHAofWUagQQqhCL7ami4hFMyGEvrXAUXGHEEJP6c9qOyruEEIf68WpfkUUTjKVF+HcJOn8fH6GpDtzrpKT83L4EELoGv06q2Qi2QEPAeY2nZ8BbAL8C7AcsQNOCKHLDPqS96mk3W5ObFyzfUHOY2LSEvip9RQxhBDas5ChQkevKdri/hbwGVp8OOUukncDF7V6YuQqCSF0ysC2uPMimwds3zDGLd8DrrT9m1YPRq6SEEKn2C509Jois0q2A94iaXdgWWBFST+0/S5JRwCrAR+ss5AhhNCOgZ1VYvsw21Pztjr7AZflSvv9pO3m97fdi982Qgh9bmC7SsbxfeBFwNWSZks6vKIyhRBCJfp1OuBE07peTtqVGNuxeCeE0NUW9mlnQFS+IYS+1Z/VdlTcIYQ+1ovdIEVExR1C6FsDO6ukoUWuklMl/TEPTM6WNK2+YoYQwsQN8jzuhkaukhWbrh1q+5xqixRCCNUY6BZ3q1wlIYTQ7RZ6qNDRa8rmKjlG0i2Sjs2bBi8icpWEEDrFBY9eUyZXyWGktK6vBFYl7fS+iMhVEkLolCFc6Og1RVrcjVwl84AzgR1zrpL7c1bXp4FTgK1rLGcIIUzYwFbc4+QqWRNAkoA9gTm1ljSEECYoZpUs6gxJqwECZgMfqqZIIYRQjV7cJKGIMrlKdqyhPCGEUJlebE0XESsnQwh9qxf7r4uIijuE0LeixR1CCD2mX1vcZXKV7CTpxpyn5CpJG9ZXzBBCmLjJ2khB0qqSZkq6K/+5yhj3HZDvuUvSAU3X95d0a17QeJGkF473ehPZAaeRq6ThOOCdtqcBPwI+P4FYIYRQu0lc8v5Z4FLbGwGX5vMRJK0KHAFsQ1r3coSkVSQtCXwb2MH25sAtwEfHe7EyuUrMcMKplYD7isQKIYTJMmQXOiqwB3Ba/vk00tqW0d4IzLT9oO2HgJnArqQp1QKWz+tiVmQx9WmZXCXvBy6QNB94N/DlVk+MXCUhhE6ZxD0nX2T7foD85+ot7lkbuLfpfD6wtu1ngQ8Dt5Iq7E2Bk8Z7sTK5Sj4J7G57KmnJ+zdbPT9ylYQQOqVoi7u5gZmPg0fHkvQrSXNaHHsULI5aXLOkpUgV95bAWqSuksPGC1RkVkkjV8nuwLLAipJ+CWxi+9p8z1nARQULH0IIk6Joa9r28cDxi7nnDWM9Julvkta0fX9OB/JAi9vmA69vOp9KWtA4Lcf/Q451Ni36yJu1lauE1J+zkqSX5tt2ZuTAZQghdNwk9nHPABqzRA4Aft7inouBXfKA5CrALvnaX4BNcwoRKFCftjWP2/YCSR8AzpU0BDwEvK+dWCGEUJchL5ysl/oycLakg4A/A/sASJoOfMj2+20/KOkLwPX5OUfbfjDfdxRwpaRngT8B7x3vxTSZK4tmTd2zP2fDhxAqN33+z1r1CU/Ii1+weaE650//uKX0a02mWDkZQuhbseQ9hBB6TL8ueY+KO4TQt/q1xV105eS8vI5+tqRZ+do+km6TNJQ74EMIoav06y7vE2lx72D7703nc4C9gB9UW6QQQqhGv7a42+4qsT0XIC2tDyGE7tOvfdxFc5UYuETSDa2Wgo4ncpWEEDpl0DcL3s72fZJWB2ZKusP2lUWe2LyUNOZxhxAmU0WrIrtOoRa37fvynw8A55FyyYYQQlfr1xZ3keyAy0t6fuNn0vr6OXUXLIQQyhrkWSUvAs7Lg5BLAj+yfZGktwLfAVYDfilptu031lfUEEKYmH7tKllsxW37HmCLFtfPI3WbhBBCV6pok4SuEysnQwh9a2Bb3CGE0Kt6ceCxiKi4Qwh9a6gHBx6LiIo7hNC3osUdQgg9pj+rbYpPUJ/sAzi4l+L2YpnjdxG/i375XQzaUTRXSSdMKCdKF8StM3avxa0zdq/FrTN2r8WtO/bA6OaKO4QQQgtRcYcQQo/p5or7+B6LW2fsXotbZ+xei1tn7F6LW3fsgaE8YBBCCKFHdHOLO4QQQgtRcYcQQo/pigU4SjljtwbWJs2Zvw+4ztGPE0IIi+h4i1vSLsBdwJHA7sC/AkcBd+XH6njNygZIJG2XN5hA0rskfVPSi0vEW3W8o4LybiBpmfzz6yV9XNLKZePmeM+T9F+STsjnG0l6UxWx69T4/9fNJE2R9Ksa4x/d4vXOKBGv1vfxoOv44KSkucButueNur4+cIHtl7UZd6w3h4CbbU9tJ26L17mFlK98c+B04CRgL9vbtxnvj6RvHWrxsG2/pN2y5vizgenAesDFwAxgY9u7l4mbY58F3AC8x/ZmkpYDrrY9rWTcFwH/DaxlezdJmwKvsn1SybivBk4EVrC9rqQtgA/a/kjJuC8FDgVeTNO3Wts7low7A3i37UfKxBkj9qnAnba/lD/YfwLcaPvINuM1v4/XBR7KP68M/Nn2+lWUe1B1Q1fJksD8Ftf/AixVIu7/AX9iZAXYeCOtXiLuaAtsW9IewLdtnyTpgHaDTcIbesj2gryD0bdsf0fSTRXF3sD2vpL2B7D9ZO4GK+tU4BTgc/n898BZpA/JMo4F3kj68ML2zZJeVzImpErv+8AJwMIK4jU8BdwqaSbwROOi7Y9XEPtA4AxJhwE7ABfaPrbdYI33saTvAzNsX5DPdwPeUEF5B1o3VNwnA9dLOhO4N19bB9iPcv8w7wF2sv3n0Q9IurfF/e16LL/Z3wW8TtIUyn3gPEfSKsBGwLKNa7avLBn22VyxHgC8OV+rpLzAM7mVbUjdMsDTFcR9oe2z8++Z/MFTSYVo+95Rny1VxF1g+7gK4oz2y3xURtJWTaffBn4A/Ba4QtJWtm8s+RKvtP2hxontCyV9oWTMgdfxijt/NfsZsAfwKlKLeD7wTtu3lwj9LWAVYJGKG/hqibij7Qu8AzjI9l8lrQt8rWxQSe8HDgGmArOBbYGrgVJft0ktqw8Bx9j+Y+6S+mHJmA1HAhcB6+T+0e3y65X1hKQXMPyBsC1QRXfBvbm7xJKWBj4OzK0g7i8kfYS0td9zH1y2HywT1PZp+YNxXdt3lixjwzdGnT8EbJqvm/Lvt79L+jzpPWZSA+cfJWMOvI73cYfWJN0KvBK4xvY0SZsAR9net8NFG1euYLclfQBfY/vvFcTcirQx9WbAHNIG1fvYvrlk3BeSWplvIJX3EuAQ26Uqlty/O1oV4xNvBr4OLG17fUnTgKNtv6VM3DrlsaYjgEYX1JWk93GpD7FBN5AVt6Q1bP+1oliPMZz2d2lSt8PjtlcqGfd626/Mg4nb2H5a0uwKBvreBHyB4YEzkSqVFcvEzbEvtb3T4q61EXcZUhfGxqTy3gksYbuKbpieIekGUgv4cttb5mu32v6XCmLXMgAc6tHxrpIOOYk07bA0289vPpe0J2lOelnz8zS9nwEzJT1Emt9e1reAvYBbq5onL2lZ4HnAC3O/fKPTeEVgrQpe4mrbWwG3Nb3mjcBWYz9l8ST9T4vLjwCzbP+8RNylgA8z3Mq8HPiB7WfbjZktsP3IqD75qlpep1LDAHCeYfNp0iymymbYDLqBrLhtV1JpjxH7Z5I+W0Gct+Yfj5T0a2AlUv9xWfcCcype3PRB4BOkSvoGhivuR4HvthtU0hqkRVnLSdqSkR8Iz2u7tMOWBTYhzQIBeBvpw+EgSTvY/kSbcY8jffP6Xj5/d772/hJlBZgj6R3AFEkbkfrkf1cyZkNdA8CNGTYnUu0Mm4HWVRW3pINtHz/WeRvxal+RKWmvptMlSHOkq2rJbgW8Jsf7re1nKgj7GeACSVcwcuDsm+0GtP1t4NuSPmb7OxWUseGNwHtJA7TN5XsM+M8K4m8I7Gh7AYCk40j93DsDt5aI+0rbWzSdXyapVH989jFSi/hp4MekefhVzdCoawC4rhk2A62rKm4WXXTS9hxgpVWX3yOtyvxLvjwV2FDSR2xf0m7sUd7c9PMCYB5phkwpkg4H9gF+mi+dIukntr9YMvQxwOOk1ubSJWONkOeEb0aaldA8hfH/tRnvNOA0SW+zfW5FxWy2NrA8wxXU8qQ+3oWSyvSfL5S0ge0/AEh6CRW0Nm3/k1Rxf25x97bh30nz2TeQ9FvSAPDeFcStZYbNoOvbwcm6VmS2eJ3tbP92cdfaiDsX2NL2U/l8OdJKtlLlljTL9vQyMcaJfQTwelLFfQGwG3CV7dIVgKR/BV7OyA+Eo8d+RqGYBwGfJ/VBi9Qn/d+k1uyRtg9tM+5OpP7ie3LcFwMH2v51m/F+wTjf4qqaVSJpSZoGgCvok69ths2g63iLW9K/j/d4ia/wda3IHO07LDpI1uraRM0jVVJP5fNlgD+UjAnwK0m7VPiNo9nepOX/N9k+MM9UOLFs0Lz67nmkFX0n5te5rmzcvMr1QlIf9B1RhP2QAAATPklEQVSkbpL5tp8gLVlvN+6luQ+6UQneUXIGzNfzn3sBazA8735/0vukNEnPI7W6X2z7A0p5Zja2fX6ZuLG0vR4dr7iBxqyMjUnzlmfk8zeT5ny2q64VmQBIehXwamC1UR8+KwJTysYnfa28LS9vNqnf9arGTIgSy5z/DfhM7gp4lgqnAwJP2h6StEDSisADQBUtq1fb3lzSLbaPkvQNhruQ2lb1IidJO9q+bNS4B6TuB2y3VWbbV+T4X7DdvCT/F5LKrqRtOIU0sPyqfD6fNLBYquJu+kBY1/bBjQ+0sh8Ig67jFbftowAkXQJsZfuxfH4kw6P97cSta0Vmw9LACqTfYfOUwEeppm/wvHw0XF42YB6sfXmrNAAVmZWnMJ5AqgQep4KWMfBk/vOfktYirbyroiV3CMOLnHZoLHIqEW974DJGjns0mPIfNqtJeonte+C5br/VSsZsqCvPTOMD4dX5vJIPhEHX8Yq7ybpA86yJZ0hzP9tmey7VLGFuFfsKUj6HU23/qYb4p9UQ05LOA15Rdewcv5FV7/uSLgJWtH1LBaHPzx8IXwNuJFWCpbtggKdsPyUJScvYvkPSxu0Gs31E/vFo2yP6dnMlW9Yngcsl3ZPP1yNNxaxCXXlm6vpAGGjdVHGfDlyXKxYDbwXamo0wyf4p6WssOnDW7tfts22/XWnJ+yIDUrY3b7ukyTWSXmn7+pJxnqORiYoWecwlExXZbkx5O1fS+cCyria1aV2LnM5l0TGOcyj5gWn7otzVsEm+VLbvvNmRLJpn5r0VxK3rA2GgddWsklwBvDafXmm7qnSjtcldPGeRVod9iJR17/9s/0eb8da0fb/G2IyhbOte0u2k8YR5pNSgjT7utj8Q8gIhSB9c04Gbc9zNgWttv6bNuKP7ikdot894jNfanrzIqd358rmr5eWkJGbNg5srAofafnkF5Xw1i65CrKSBo3ryzOxMmrmzKWnwdzvgvbYvLxt7kHVTixvSzIFHbZ8iaTVJ64/+ytmFXpBnJxzS1H1yRbvBbN+ff1wCuH/UdMAXlS8uu1UQYwTbOwDkgeCDbd+azzcjfaC1q9FXvDqpj/SyfL4Dqc+/soq7MQBY0sbAm0ibBTT3cz8GfKBscEmnAxuQBlIb88JNBd9Mc+wrgd/YvqNsvAbbM5XSEzQ+EA6p4gNh4NnuioOUQewXwO/z+Vqk1YJVxD54vPOSsa/Jf15Myn+yJfCHCuLOImWBa5wvDVxfUZlfQ5pXDGlwa/2K4s4ucq2NuOcDazadrwn8tKr/h1UfpORMdcSdS/6WXEPsHYHDgZmkaafnkirZsnGPHnW+BHBGp/8f9frRTS3ut5IqvRsBbN8n6fnjP6WwylZktvBFSSsBnyLN316RlLejrCXd9JXd9jNKOaNLyYtkppNah6eQ5rT/kPQVtqy5kk5kZO7lKgaH1/PwNxGAvwEvrSBuXW6S9G8sOu7xvpJx55Dmcd+/uBsnymka4xWkWTY7kLr9Xk5Ke1vGupIO86gt0UrGHHjdVHE/Y9uSGoMYlW3gavsH452X9JDTQNkjpDc8kqqoBP9P0ltsz8gx9wCq+IpZ5wfkgaSseIfk8ytJyZXKulzSxaQVjSbNxW9rFeIkOZ20oOeNwNHAO6nmA+yFwO2SrmPk8vHSKyclXUpa8n818BtSvpUHysal4i3RQtI1g5OSPk3apmtn4EvA+4Afuc2kRTWuyBz9Ojc6pRwd91obcTcAziB1GYm0iOg9tu8uGfc621s3ypg/IK92+dkqRV77XNtva/O5b6UpGb/t88a7v5Mk3WR7y7xgaHOlNK8Xu/xmwS03oHYF/fOSjiXNenmatHXZlaT3xZPjPnHseM3v/6UY3hLtJACX3xJtoHVNi9v21/MI9KOkr/GH255ZImRdKzKB+ldOOiUo2lbSCqQP2MfKxszOlvQDYGVJHyB9QJ5QUezFaXsVZa6oW1bWkq62/apWj3VII8fHw3mA9q+UXJMAqYLOs402sv2rvCqxilW62P4kQH6/HUjqRluDlGqhHXVviTbQuqbihudGoK8ll0vSqm4zi5hrWpHZpNaVk7k/8G3kqV+NNQsumViJNBh5Dk0fkEzertt1fb1bdvG3TKrjlTaU+DypwbAC6fdcSv6gPRhYlTS7ZG1SrutSOwzl2B8jDVq/AvgTKWXEb9qN5zzTKNSjm7pKPkjqD3wSGGJ4fnHZffruALZwXqiQK8SbbW8y/jMLx3+x89xqSUsAK9h+tIK4F5H6zW+gKSWo7dEtmYnGbdW1c8skdZWU7kKazLjdRmkbu61Jc+Mr2bqsMeVW0qGkb6I3OOcnr4JiS7RadFOL+9OkPBpVz/Gse0XmlyR9iFS53gCsJOmbtsvu9D7V9q7li5dI+jDwEeAlkpqXoT+f1Pc4GQZiqbOk/wa+avvhfL4K8Cnbny8Z+uk8u6jxOktS/ltMY0XnrhW8Z1s5lRq2RBt0S3S6AE3+APyz6qC2jyH12T0EPEyav/zfFb7EprmFvScpB/W6pDShZf1OUulNYJv8iNS/PyP/2TheYftdVb2IpOXGyffR1mrSIi9bU9x27daotAFsPwTsXkHcKyT9J2krt51JXX6/KBlziTxF9KWS/n30UbrEeUs00rdocms+tjArqZta3IeRKqtrGTnVqd30pc3qXJG5VJ41sCfwv7afbUxpLOk1wHuVEtE/Tcml6U1TFvevoGwtSXozKXf00sD6kqaRFmC8JZeh7RzgowblliPNc28M2FbxQVmlKUpJqxrdc8vR/iBfs88CB5G2VTsY+KXtssm29iO9d0eP1VSlri3RBlo3Vdw/IC1pvpX86VyFmhecQCr3PFJ+jitzBVO6j5salqZPgiNJfbCXA9ieLWm9skFbDMpNpWlQzvacsq9RsR8Cl0o6hVRhvQ9oO9tjnsM/1fZ3gRPy72M14BWSHrZ9Truxbd8JfCWPc1zYbpxx1LUl2kDrpsHJ39l+9eLvnHDc2eQFJ00DOrUNxil1QE7x8Aa0B3gCKVolrWj7UUmrtnq83Vk2k0HStba3acxjztdK/67rGJSrm6TdSB8sAi6xfXGJWL8F9rN9bz6fTZpOtwJwiu0qZpWsREo70ZgrfwXp21Lp1rFq2BJt0HVTi/vXkg4m9dlVualobSsyW3H6JGwelT+EibW2fkRKVHQDqbXW3H9rqtlRpi5zJL2D1FWwEfBx4HcVxK1jUK5WufVaVQt26UalnV2V/108WOH7+WTSkvq35/N3k76hjpuhcXFU05Zog66bWty1bCqqildktvH6z7U+J/AcAeu4vp1qapH/kX4O2CVfuhj4onOGwxJxv0oaWH4P8DHS7Jjbbdex23lpkh5j+INlaVL33BNuc3s4SXfb3nCMx/5ge4P2Sjoizmzb0xZ3rY24Z5EaIe+xvVnu77+6bNxB1zUtbte0qWgNKzInXIQJP8H17lRTo41zZVp1hdo8KPdB0uydKnbAqYXtEYN8kvYkdfW061pJH7A9YoVrXvtQxdZwAE9Keo3tq3Ls7RjeMq6M2AGnBl3T4obn8jdvysiMalUliV+RkcnnJ6WvuJ0Wd37ed4FTXeFONXVT2lBhTdI0tTNt39bhInUNSdfY3rbN565O2qXnaYYz672CNFNlT9t/q6B800hdeivlSw8BB7jk1nOSfkfq6/+tU26cDYAf2y7zQTbwuqbizrM/Xk+quC8gzaq4ynapEei6VmRO4PX/1/ZH23he5TvVTAZJa5D6Sfcl5W05y/YXS8b8I623cevK/n6N3LlnCdKspu1dMp+KpB1JqVYBbrN92Xj3TzD2FNsLcwMHV7D6N8eNHXBq0E0V963AFsBNtrfIS2VPtN1qx+yJxL2LtMS2ll03NCqnSOO6S+YUUU1bl02WvHjoM8C+tkvlEc/zgBuWBfYBVrVdOv9HHfI0wIYFpA/fE1xNmtRa5A/Hc4CTnTbZriru6aQurieBe0gzg2IHnJK6po8beNL2kKQF+VP/AaqZQVHLiswmP2c4p0jpTVAlLUtKYr8h6Q1/kivMHVEnSS8jtbT3Bv4BnEnaYKIU2/8Ydelbkq6igsRNdbB9YKfL0IbNSYtxTlLKuXMyqburbMv7FNJisp1J/55nS7rSdtkNGgZaN7W4vwf8J+nN8yngcdK2V6X+EUjakvTmqWNFJpLm2N6silg53lmktKC/IXUX/cn2IeM/qztIuoa02cFPbFexW3ojbnMCqUbXw4dtb1HVa1RB0ncYZzC6qvdc3SS9jvT/cWVSK/wLLpEHXtIURu6s86QrSvI2qLqmxW37I/nH7ytlxlux7MBIVsuKzCa/k/QvzhvkVmDTxsISSSdR3ayB2rU7+FZAI4czDHc97FPTa5UxK/+5HalP96x8vg/pG1nXypXrv5Ly+qxH+p2fAbyWNObU1lZxqm9nnYHWNS1uAElrAy9mZF9xqU0P6lqR2RT/dlK3RiU5RTQqReno824k6Wzbb8/jFM1vqLK/i0aSIzFyMZKhul2MqpZn1+zSWCGolMvmEndxjmpJ95C2gzvJ9u9GPfY/7X5bUMU764Ska1rckr5C6h+9neHsYab8bjV1rchsqDqnyBaSGv2KImWCe5ThSrCtRRw1a3TlvKniuKN3Mfo56fdQyS5GNVqLVPbGe2yFfK2bbW778VYPlOnicfU76wS6qMUt6U7Sm6f0AN+ouLWsyGzxOqszcv55T616rIKkr9j+j8VdayPuJcDbPLyL0fNJ/eiV5SuvkqQDSXk/Ls+XtgeO9ARy1ky2PCh+EBXvTC/po6TulsbOOlcCv6lyKuMg6qZ83PeQlgZXyvb6LY7KKm1Jb8lTDv9ISswzj+pyVPSanVtcq+IbybrAM03nz1DBHo41OpU042Vz4KekiruyKXY1OZ3UEn4j6X08Fahin9PlgG8Cm9jeyfZRUWmX1zVdJaQpe7PzYEalsz/qXJEJfAHYFviV087eO1BjzutupOHddTbQorvrVJFkqtUuRl3begW+RxoIX872DKUdcM4ldfd0qw1t7yNpD9unSfoRKddMKa5nV52B100V9wyGd2KvzFgrMqlu+7Jnbf9D0hKSlrD969xfP0h+RPqW8SVSXpGGx6oYS7B9jKQLSV+5Ie1idFPZuDXaJi/vvgnSDjiSSi1CmgS17Ewf6tE1FXeN/X97M7wi88DGiswK4z+cB15+A5wh6QFGpnXte86760j6NvBgc1+0pG1sX1vBa9zIcJ6Obvdsnl7XSCW8GvVMRa1Sq53p/6uzRQpj6abBye1IO6g0pgNWtcv7dba3lnQDaQHAY8Ac2y9fzFOLxl8eeIpU3neSkvSc0WK1X9/LLcytnN9UeQXerG6fzlg1Se8kzZDaitSlszfweds/6WjBWlDrfSWfm3bZrVMuB13XtLhJuz5/krRQocrNRGdJWhk4Icd+nAoXtdh+IrfiX0la5n3hIFbamdzUEsgpDLrpPTYpbJ+RGwqNHXD2rDL/R8VGT7lsdFd2+5TLgdZNLe5rbW9T82usR3UrMhsx3w58jTT1S6R+2ENdYh/AXiXpp6Tfw3H50keAHWzv2bFChUJ6bcrloOumivvLwBTS9KnmWSWl+zXrWJHZFPtmYOfGMt7cn/mrbsujMRnyXPb/Ie2HaOBS4BOxxLn7SboD2MLDO9MvA9wcOUW6Uzd9jW20tqc3XTOpEmhbjSsyG5YYVTH9g+6aHz9p8u9hv06XI7Sl16ZcDrSuaXHXpa4VmU3xv0ZaaPHjfGlf4JayqwV7kaSXkrpJXuS0v+DmwFtcciOFMDlyFsbGlMsru3zK5UDreMXdYlTbwN9Ju9+0Wq4+0fgXAvuMlYehCpLeRsoIJ9Ib/ry6XqubSboCOBT4gfN2bVWnvQ0hdEdXyfNbXFsP+JykI22fWTJ+bSsym2KdS1oZN+ieZ/s6jdwLdqDmtIcwGTpecds+qtV1SasCvyLtolJGXSsyH6N10vxuzuJXt78rbQbbmMe9N3B/Z4sUQv/peFfJeNTmDumhMyS9BDgeeDVpl/A/Au90j+yTGUKv6HiLeyxKO1o/VEGcWlZkhmGSDnHaQ3BN22/Iq0mXaMwJDiFUq+Mt7ha7pgCsCtwHvMf2HSXj30GLFZkDvLqxcpJm257WC7v1hNAPuqHFPXrXFAP/sP1ERfEfsT2o+bEny1xJ84DVRqV1LbV1WQihtY63uOtW54rMMEzSGqT8zW8Z/Vj0cYdQrW5ocdetlhWZYZikS23vJOniqKRDqF/fV9zdvLN2H1lT0vbAmyUtMn0zvt2EUK2+7Sqpe0VmGJbnax8EvAa4vnE5/2nb8e0mhAr1c8V9RIvLq5I2Q61iRWYYRdLhpLnb69s+WtK6wBq2K8t/HkLo44p7LI0VmTFtrXqSvk+acrmj7ZflrbAusd3Nm+SG0HP6vo97NNsPalQyjVCZrXtwk9wQes7A5Y2uakVmaKkXN8kNoef0bYt7cSsyJ79EA+F/gPOA1SUdQ94kt7NFCqH/9G0ft6QXj7pU9YrM0IKkTRjeJPfSLt4kN4Se1bcVdwgh9KuB6+MOIYReFxV3CCH0mKi4Qwihx0TFHUIIPSYq7hBC6DH/HxgFQCbykBajAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.heatmap(test_df.isnull())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [],
   "source": [
    "dumm=pd.get_dummies(test_df['education'])\n",
    "#dumm=dumm.drop('Master or Above',axis=1,inplace=True)\n",
    "dumm=dumm[['Bechalor','High School or Below','college']]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Principal</th>\n",
       "      <th>terms</th>\n",
       "      <th>age</th>\n",
       "      <th>Gender</th>\n",
       "      <th>weekend</th>\n",
       "      <th>Bechalor</th>\n",
       "      <th>High School or Below</th>\n",
       "      <th>college</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>50</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>300</td>\n",
       "      <td>7</td>\n",
       "      <td>35</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>43</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1000</td>\n",
       "      <td>30</td>\n",
       "      <td>26</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>800</td>\n",
       "      <td>15</td>\n",
       "      <td>29</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n",
       "0       1000     30   50       1        0         1                     0   \n",
       "1        300      7   35       0        1         0                     0   \n",
       "2       1000     30   43       1        1         0                     1   \n",
       "3       1000     30   26       0        1         0                     0   \n",
       "4        800     15   29       0        1         1                     0   \n",
       "\n",
       "   college  \n",
       "0        0  \n",
       "1        0  \n",
       "2        0  \n",
       "3        1  \n",
       "4        0  "
      ]
     },
     "execution_count": 50,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_feature = test_df[['Principal','terms','age','Gender','weekend']]\n",
    "test_feature = pd.concat([test_feature,dumm], axis=1)\n",
    "#test_feature.drop(['Master or Above'], axis = 1,inplace=True)\n",
    "test_feature.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_data= test_feature\n",
    "test_data= preprocessing.StandardScaler().fit(test_data).transform(test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_t=test_df['loan_status']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# i will use train.csv for train model, and test.csv for testing model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# KNN"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy :  74.1 %\n"
     ]
    }
   ],
   "source": [
    "knn=KNeighborsClassifier()\n",
    "knn.fit(X,y)\n",
    "predKNN_test=knn.predict(test_data)\n",
    "accuracy=metrics.accuracy_score(predKNN_test,y_t)\n",
    "print(\"accuracy : \",round(accuracy,3)*100,'%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# BEST K FOR TEST DATA"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 :  0.7037037037037037\n",
      "2 :  0.5740740740740741\n",
      "3 :  0.6481481481481481\n",
      "4 :  0.6296296296296297\n",
      "5 :  0.7407407407407407\n",
      "6 :  0.6851851851851852\n",
      "7 :  0.7222222222222222\n",
      "8 :  0.7037037037037037\n",
      "9 :  0.7037037037037037\n",
      "10 :  0.6851851851851852\n",
      "11 :  0.6851851851851852\n",
      "12 :  0.6666666666666666\n",
      "13 :  0.7037037037037037\n",
      "14 :  0.7037037037037037\n",
      "15 :  0.7222222222222222\n",
      "16 :  0.7037037037037037\n",
      "17 :  0.7222222222222222\n",
      "18 :  0.7037037037037037\n",
      "19 :  0.7222222222222222\n",
      "20 :  0.7407407407407407\n",
      "21 :  0.7592592592592593\n",
      "22 :  0.7592592592592593\n",
      "23 :  0.7592592592592593\n",
      "24 :  0.7222222222222222\n",
      "25 :  0.7407407407407407\n",
      "26 :  0.7777777777777778\n",
      "27 :  0.7592592592592593\n",
      "28 :  0.7777777777777778\n",
      "29 :  0.7592592592592593\n",
      "30 :  0.7777777777777778\n",
      "31 :  0.7407407407407407\n",
      "32 :  0.7962962962962963\n",
      "33 :  0.7777777777777778\n",
      "34 :  0.7962962962962963\n",
      "35 :  0.7962962962962963\n",
      "36 :  0.7777777777777778\n",
      "37 :  0.7962962962962963\n",
      "38 :  0.7962962962962963\n",
      "39 :  0.7962962962962963\n",
      "40 :  0.7962962962962963\n",
      "41 :  0.7962962962962963\n",
      "42 :  0.7962962962962963\n",
      "43 :  0.7777777777777778\n",
      "44 :  0.7962962962962963\n",
      "45 :  0.7962962962962963\n",
      "46 :  0.7962962962962963\n",
      "47 :  0.7777777777777778\n",
      "48 :  0.7777777777777778\n",
      "49 :  0.7592592592592593\n",
      "50 :  0.7777777777777778\n",
      "51 :  0.7777777777777778\n",
      "52 :  0.7777777777777778\n",
      "53 :  0.7407407407407407\n",
      "54 :  0.7407407407407407\n",
      "55 :  0.7407407407407407\n",
      "56 :  0.7407407407407407\n",
      "57 :  0.7407407407407407\n",
      "58 :  0.7407407407407407\n",
      "59 :  0.7407407407407407\n",
      "60 :  0.7407407407407407\n",
      "61 :  0.7407407407407407\n",
      "62 :  0.7407407407407407\n",
      "63 :  0.7407407407407407\n",
      "64 :  0.7407407407407407\n",
      "65 :  0.7407407407407407\n",
      "66 :  0.7407407407407407\n",
      "67 :  0.7407407407407407\n",
      "68 :  0.7407407407407407\n",
      "69 :  0.7407407407407407\n",
      "70 :  0.7407407407407407\n",
      "71 :  0.7407407407407407\n",
      "72 :  0.7407407407407407\n",
      "73 :  0.7407407407407407\n",
      "74 :  0.7407407407407407\n",
      "75 :  0.7407407407407407\n",
      "76 :  0.7407407407407407\n",
      "77 :  0.7407407407407407\n",
      "78 :  0.7407407407407407\n",
      "79 :  0.7407407407407407\n",
      "80 :  0.7407407407407407\n",
      "81 :  0.7407407407407407\n",
      "82 :  0.7407407407407407\n",
      "83 :  0.7407407407407407\n",
      "84 :  0.7407407407407407\n",
      "85 :  0.7407407407407407\n",
      "86 :  0.7407407407407407\n",
      "87 :  0.7407407407407407\n",
      "88 :  0.7407407407407407\n",
      "89 :  0.7407407407407407\n",
      "90 :  0.7407407407407407\n",
      "91 :  0.7407407407407407\n",
      "92 :  0.7407407407407407\n",
      "93 :  0.7407407407407407\n",
      "94 :  0.7407407407407407\n",
      "95 :  0.7407407407407407\n",
      "96 :  0.7407407407407407\n",
      "97 :  0.7407407407407407\n",
      "98 :  0.7407407407407407\n",
      "99 :  0.7407407407407407\n"
     ]
    }
   ],
   "source": [
    "score=[]\n",
    "for k in range(1,100):\n",
    "    knn=KNeighborsClassifier(n_neighbors=k,weights='uniform')\n",
    "    knn.fit(X,y)\n",
    "    predKNN=knn.predict(test_data)\n",
    "    accuracy=metrics.accuracy_score(predKNN,y_t)\n",
    "    score.append(accuracy*100)\n",
    "    print (k,': ',accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "32  :  79.63 %\n"
     ]
    }
   ],
   "source": [
    "print(score.index(max(score))+1,' : ',round(max(score),2),'%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "accuracy :  79.60000000000001 %\n"
     ]
    }
   ],
   "source": [
    "knn=KNeighborsClassifier(n_neighbors=32)\n",
    "knn.fit(X,y)\n",
    "predKNN_test=knn.predict(test_data)\n",
    "accuracy=metrics.accuracy_score(predKNN_test,y_t)\n",
    "print(\"accuracy : \",round(accuracy,3)*100,'%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      " COLLECTION       0.71      0.36      0.48        14\n",
      "    PAIDOFF       0.81      0.95      0.87        40\n",
      "\n",
      "avg / total       0.78      0.80      0.77        54\n",
      "\n",
      "\n",
      "\n",
      "Jaccard Similarity Score :  79.63 %\n",
      "\n",
      "\n",
      "F1-SCORE :  [0.47619048 0.87356322]\n",
      "\n",
      "\n",
      "Train Accuracy:  74.85549132947978 %\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_t,predKNN_test))\n",
    "print('\\n')\n",
    "print('Jaccard Similarity Score : ',round(jaccard_similarity_score(y_t,predKNN_test)*100,2),'%')\n",
    "print('\\n')\n",
    "print('F1-SCORE : ',f1_score(y_t,predKNN_test,average=None))\n",
    "print('\\n')\n",
    "print('Train Accuracy: ',metrics.accuracy_score(y,knn.predict(X))*100,'%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Decision tree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### BEST PARAMETERS FOR DECISION TREE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Best Score: 0.7687861271676301\n",
      "Best params: {'criterion': 'entropy', 'max_depth': 6, 'max_features': 4, 'random_state': 20}\n"
     ]
    }
   ],
   "source": [
    "parameter_grid = {'max_depth': [1, 2, 3, 4, 5,6,5,9,15,20],\n",
    "                  'max_features': [1, 2, 3, 4,5,6,7,8],\n",
    "                 'random_state':[0,15,20,35,50,80,100,150,180,200],\n",
    "                 'criterion':['gini','entropy'],\n",
    "                 }\n",
    "\n",
    "grid_search = GridSearchCV(dtree, param_grid = parameter_grid,\n",
    "                          cv =10)\n",
    "\n",
    "grid_search.fit(X, y)\n",
    "\n",
    "print (\"Best Score: {}\".format(grid_search.best_score_))\n",
    "print (\"Best params: {}\".format(grid_search.best_params_))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [],
   "source": [
    "dtree=DecisionTreeClassifier(max_depth=6,criterion='entropy',max_features=4,random_state=20).fit(X,y)\n",
    "pred_Dtree=dtree.predict(test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      " COLLECTION       0.44      0.29      0.35        14\n",
      "    PAIDOFF       0.78      0.88      0.82        40\n",
      "\n",
      "avg / total       0.69      0.72      0.70        54\n",
      "\n",
      "\n",
      "\n",
      "Jaccard Similarity Score :  72.22 %\n",
      "\n",
      "\n",
      "F1-SCORE :  [0.34782609 0.82352941]\n",
      "\n",
      "\n",
      "Train Accuracy:  79.47976878612717 %\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_t,pred_Dtree))\n",
    "print('\\n')\n",
    "print('Jaccard Similarity Score : ',round(jaccard_similarity_score(y_t,pred_Dtree)*100,2),'%')\n",
    "print('\\n')\n",
    "print('F1-SCORE : ',f1_score(y_t,pred_Dtree,average=None))\n",
    "print('\\n')\n",
    "print('Train Accuracy: ',metrics.accuracy_score(y, dtree.predict(X))*100,'%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# SVM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [],
   "source": [
    "svm=SVC().fit(X,y)\n",
    "pred_svm=svm.predict(test_data)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      " COLLECTION       0.00      0.00      0.00        14\n",
      "    PAIDOFF       0.74      0.97      0.84        40\n",
      "\n",
      "avg / total       0.55      0.72      0.62        54\n",
      "\n",
      "\n",
      "\n",
      "Jaccard Similarity Score :  72.22 %\n",
      "\n",
      "\n",
      "F1-SCORE :  [0.         0.83870968]\n",
      "\n",
      "\n",
      "Train Accuracy:  76.01156069364163 %\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_t,pred_svm))\n",
    "print('\\n')\n",
    "print('Jaccard Similarity Score : ',round(jaccard_similarity_score(y_t,pred_svm)*100,2),'%')\n",
    "print('\\n')\n",
    "print('F1-SCORE : ',f1_score(y_t,pred_svm,average=None))\n",
    "print('\\n')\n",
    "print('Train Accuracy: ',metrics.accuracy_score(y, svm.predict(X))*100,'%')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# LOGISTIC REGRESSION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "lgm=LogisticRegression().fit(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [],
   "source": [
    "pred_lgm=lgm.predict(test_data)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "             precision    recall  f1-score   support\n",
      "\n",
      " COLLECTION       1.00      0.07      0.13        14\n",
      "    PAIDOFF       0.75      1.00      0.86        40\n",
      "\n",
      "avg / total       0.82      0.76      0.67        54\n",
      "\n",
      "\n",
      "\n",
      "Jaccard Similarity Score :  75.93 %\n",
      "\n",
      "\n",
      "F1-SCORE :  [0.13333333 0.86021505]\n",
      "\n",
      "\n",
      "Train Accuracy:  75.43352601156069 %\n"
     ]
    }
   ],
   "source": [
    "print(classification_report(y_t,pred_lgm))\n",
    "print('\\n')\n",
    "print('Jaccard Similarity Score : ',round(jaccard_similarity_score(y_t,pred_lgm)*100,2),'%')\n",
    "print('\\n')\n",
    "print('F1-SCORE : ',f1_score(y_t,pred_lgm,average=None))\n",
    "print('\\n')\n",
    "print('Train Accuracy: ',metrics.accuracy_score(y, lgm.predict(X))*100,'%')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "Algoritm=['KNN','Decision Tree','SVM','LogisticRegression']\n",
    "j_knn=round(jaccard_similarity_score(y_t,predKNN_test)*100,2)\n",
    "j_dtree=round(jaccard_similarity_score(y_t,pred_Dtree)*100,2)\n",
    "j_svm=round(jaccard_similarity_score(y_t,pred_svm)*100,2)\n",
    "j_lgm=round(jaccard_similarity_score(y_t,pred_lgm)*100,2)\n",
    "Jaccard=[j_knn,j_dtree,j_svm,j_lgm]\n",
    "\n",
    "f1_knn=f1_score(y_t,predKNN_test,average=None)\n",
    "f1_dtree=f1_score(y_t,pred_Dtree,average=None)\n",
    "f1_svm=f1_score(y_t,pred_svm,average=None)\n",
    "f1_lgm=f1_score(y_t,pred_lgm,average=None)\n",
    "F1_score=[f1_knn,f1_dtree,f1_svm,f1_lgm]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [],
   "source": [
    "table = pd.DataFrame({\n",
    "    \"Algorithm\": Algoritm,\n",
    "    \"Jaccatd\": Jaccard,\n",
    "    \"F1-Score\": F1_score,\n",
    "    \"LogLoss\":[np.NAN,np.NAN,np.NAN,np.NAN]})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Algorithm</th>\n",
       "      <th>Jaccatd</th>\n",
       "      <th>F1-Score</th>\n",
       "      <th>LogLoss</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>KNN</td>\n",
       "      <td>79.63</td>\n",
       "      <td>[0.4761904761904762, 0.8735632183908046]</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Decision Tree</td>\n",
       "      <td>72.22</td>\n",
       "      <td>[0.34782608695652173, 0.823529411764706]</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>SVM</td>\n",
       "      <td>72.22</td>\n",
       "      <td>[0.0, 0.8387096774193549]</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>LogisticRegression</td>\n",
       "      <td>75.93</td>\n",
       "      <td>[0.13333333333333333, 0.8602150537634409]</td>\n",
       "      <td>NaN</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            Algorithm  Jaccatd                                   F1-Score  \\\n",
       "0                 KNN    79.63   [0.4761904761904762, 0.8735632183908046]   \n",
       "1       Decision Tree    72.22   [0.34782608695652173, 0.823529411764706]   \n",
       "2                 SVM    72.22                  [0.0, 0.8387096774193549]   \n",
       "3  LogisticRegression    75.93  [0.13333333333333333, 0.8602150537634409]   \n",
       "\n",
       "   LogLoss  \n",
       "0      NaN  \n",
       "1      NaN  \n",
       "2      NaN  \n",
       "3      NaN  "
      ]
     },
     "execution_count": 68,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}