credit_risk_resampling.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Credit Risk Classification\n",
    "\n",
    "Credit risk poses a classification problem that’s inherently imbalanced. This is because healthy loans easily outnumber risky loans. In this project, we’ll use various techniques to train and evaluate models with imbalanced classes. We’ll use a dataset of historical lending activity from a peer-to-peer lending services company to build a model that can identify the creditworthiness of borrowers.\n",
    "\n",
    "## Steps:\n",
    "\n",
    "* Split of the Data into Training and Testing Sets\n",
    "\n",
    "* Creation of a Logistic Regression Model with the Original Data\n",
    "\n",
    "* Prediction of a Logistic Regression Model with Resampled Training Data.\n",
    "\n",
    "* Comparison between the two models.\n",
    "\n",
    "* Summary Report: Results are included in the README file, which in this case act as the Final Report (Overview, Results, and Summary)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the modules\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "from pathlib import Path\n",
    "from sklearn.metrics import balanced_accuracy_score\n",
    "from sklearn.metrics import confusion_matrix\n",
    "from imblearn.metrics import classification_report_imbalanced\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split the Data into Training and Testing Sets"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Read the `lending_data.csv` data from the `Resources` folder into a Pandas DataFrame."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_size</th>\n",
       "      <th>interest_rate</th>\n",
       "      <th>borrower_income</th>\n",
       "      <th>debt_to_income</th>\n",
       "      <th>num_of_accounts</th>\n",
       "      <th>derogatory_marks</th>\n",
       "      <th>total_debt</th>\n",
       "      <th>loan_status</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10700.0</td>\n",
       "      <td>7.672</td>\n",
       "      <td>52800</td>\n",
       "      <td>0.431818</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>22800</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8400.0</td>\n",
       "      <td>6.692</td>\n",
       "      <td>43600</td>\n",
       "      <td>0.311927</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>13600</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9000.0</td>\n",
       "      <td>6.963</td>\n",
       "      <td>46100</td>\n",
       "      <td>0.349241</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>16100</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10700.0</td>\n",
       "      <td>7.664</td>\n",
       "      <td>52700</td>\n",
       "      <td>0.430740</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>22700</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10800.0</td>\n",
       "      <td>7.698</td>\n",
       "      <td>53000</td>\n",
       "      <td>0.433962</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>23000</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77531</th>\n",
       "      <td>19100.0</td>\n",
       "      <td>11.261</td>\n",
       "      <td>86600</td>\n",
       "      <td>0.653580</td>\n",
       "      <td>12</td>\n",
       "      <td>2</td>\n",
       "      <td>56600</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77532</th>\n",
       "      <td>17700.0</td>\n",
       "      <td>10.662</td>\n",
       "      <td>80900</td>\n",
       "      <td>0.629172</td>\n",
       "      <td>11</td>\n",
       "      <td>2</td>\n",
       "      <td>50900</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77533</th>\n",
       "      <td>17600.0</td>\n",
       "      <td>10.595</td>\n",
       "      <td>80300</td>\n",
       "      <td>0.626401</td>\n",
       "      <td>11</td>\n",
       "      <td>2</td>\n",
       "      <td>50300</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77534</th>\n",
       "      <td>16300.0</td>\n",
       "      <td>10.068</td>\n",
       "      <td>75300</td>\n",
       "      <td>0.601594</td>\n",
       "      <td>10</td>\n",
       "      <td>2</td>\n",
       "      <td>45300</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>77535</th>\n",
       "      <td>15600.0</td>\n",
       "      <td>9.742</td>\n",
       "      <td>72300</td>\n",
       "      <td>0.585062</td>\n",
       "      <td>9</td>\n",
       "      <td>2</td>\n",
       "      <td>42300</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>77536 rows × 8 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       loan_size  interest_rate  borrower_income  debt_to_income  \\\n",
       "0        10700.0          7.672            52800        0.431818   \n",
       "1         8400.0          6.692            43600        0.311927   \n",
       "2         9000.0          6.963            46100        0.349241   \n",
       "3        10700.0          7.664            52700        0.430740   \n",
       "4        10800.0          7.698            53000        0.433962   \n",
       "...          ...            ...              ...             ...   \n",
       "77531    19100.0         11.261            86600        0.653580   \n",
       "77532    17700.0         10.662            80900        0.629172   \n",
       "77533    17600.0         10.595            80300        0.626401   \n",
       "77534    16300.0         10.068            75300        0.601594   \n",
       "77535    15600.0          9.742            72300        0.585062   \n",
       "\n",
       "       num_of_accounts  derogatory_marks  total_debt  loan_status  \n",
       "0                    5                 1       22800            0  \n",
       "1                    3                 0       13600            0  \n",
       "2                    3                 0       16100            0  \n",
       "3                    5                 1       22700            0  \n",
       "4                    5                 1       23000            0  \n",
       "...                ...               ...         ...          ...  \n",
       "77531               12                 2       56600            1  \n",
       "77532               11                 2       50900            1  \n",
       "77533               11                 2       50300            1  \n",
       "77534               10                 2       45300            1  \n",
       "77535                9                 2       42300            1  \n",
       "\n",
       "[77536 rows x 8 columns]"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Read the CSV file from the Resources folder into a Pandas DataFrame\n",
    "df=pd.read_csv(Path('Resources/lending_data.csv'))\n",
    "\n",
    "# Review the DataFrame\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Create the labels set (`y`)  from the “loan_status” column, and then create the features (`X`) DataFrame from the remaining columns."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Separate the data into labels and features\n",
    "\n",
    "# Separate the y variable, the labels\n",
    "y=df['loan_status']\n",
    "\n",
    "# Separate the X variable, the features\n",
    "X=df.drop(columns='loan_status')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    0\n",
       "1    0\n",
       "2    0\n",
       "3    0\n",
       "4    0\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Review the y variable Series\n",
    "y.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>loan_size</th>\n",
       "      <th>interest_rate</th>\n",
       "      <th>borrower_income</th>\n",
       "      <th>debt_to_income</th>\n",
       "      <th>num_of_accounts</th>\n",
       "      <th>derogatory_marks</th>\n",
       "      <th>total_debt</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>10700.0</td>\n",
       "      <td>7.672</td>\n",
       "      <td>52800</td>\n",
       "      <td>0.431818</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>22800</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>8400.0</td>\n",
       "      <td>6.692</td>\n",
       "      <td>43600</td>\n",
       "      <td>0.311927</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>13600</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>9000.0</td>\n",
       "      <td>6.963</td>\n",
       "      <td>46100</td>\n",
       "      <td>0.349241</td>\n",
       "      <td>3</td>\n",
       "      <td>0</td>\n",
       "      <td>16100</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>10700.0</td>\n",
       "      <td>7.664</td>\n",
       "      <td>52700</td>\n",
       "      <td>0.430740</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>22700</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>10800.0</td>\n",
       "      <td>7.698</td>\n",
       "      <td>53000</td>\n",
       "      <td>0.433962</td>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "      <td>23000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   loan_size  interest_rate  borrower_income  debt_to_income  num_of_accounts  \\\n",
       "0    10700.0          7.672            52800        0.431818                5   \n",
       "1     8400.0          6.692            43600        0.311927                3   \n",
       "2     9000.0          6.963            46100        0.349241                3   \n",
       "3    10700.0          7.664            52700        0.430740                5   \n",
       "4    10800.0          7.698            53000        0.433962                5   \n",
       "\n",
       "   derogatory_marks  total_debt  \n",
       "0                 1       22800  \n",
       "1                 0       13600  \n",
       "2                 0       16100  \n",
       "3                 1       22700  \n",
       "4                 1       23000  "
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Review the X variable DataFrame\n",
    "X.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Check the balance of the labels variable (`y`) by using the `value_counts` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    75036\n",
       "1     2500\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check the balance of our target values\n",
    "y.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(77536,)"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Split the data into training and testing datasets by using `train_test_split`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split the data using train_test_split\n",
    "# Assign a random_state of 1 to the function\n",
    "X_train, X_test, y_train, y_test=train_test_split(X,y, random_state=1, test_size=0.25)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a Logistic Regression Model with the Original Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "###  Step 1: Fit a logistic regression model by using the training data (`X_train` and `y_train`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LogisticRegression(random_state=1)"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Instantiate the Logistic Regression model with a random_state parameter of 1 to the model\n",
    "model=LogisticRegression(random_state=1)\n",
    "\n",
    "# Fit the model using training data\n",
    "model.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Save the predictions on the testing data labels by using the testing feature data (`X_test`) and the fitted model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make a prediction using the testing data\n",
    "y_pred_test=model.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Evaluate the model’s performance\n",
    "\n",
    "* Calculate the accuracy score of the model.\n",
    "\n",
    "* Generate the confusion matrix.\n",
    "\n",
    "* Print the classification report."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9520479254722232"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print the balanced_accuracy score of the model\n",
    "balanced_accuracy_score(y_test, y_pred_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[18663,   102],\n",
       "       [   56,   563]])"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Generate a confusion matrix for the model\n",
    "confusion_matrix(y_test, y_pred_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Confusion matrix considers all values 1s&0s\n",
    "confusion_matrix(y_test, y_pred_test).sum()==len(y_pred_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " \n",
      "             Classification Report with The Umbalaced Data Model\n",
      "                   pre       rec       spe        f1       geo       iba       sup\n",
      "\n",
      "          0       1.00      0.99      0.91      1.00      0.95      0.91     18765\n",
      "          1       0.85      0.91      0.99      0.88      0.95      0.90       619\n",
      "\n",
      "avg / total       0.99      0.99      0.91      0.99      0.95      0.91     19384\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Print the classification report for the model\n",
    "print(\" \\n             Classification Report with The Umbalaced Data Model\")\n",
    "print(classification_report_imbalanced(y_test, y_pred_test))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Logistic Regression Raw Model Assessment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, we can assess how well does the logistic regression model predict both the `0` (healthy loan) and `1` (high-risk loan) labels.\n",
    "\n",
    "Healthy loans are predicted with a precision of 100%, which is optimal, and a recall of 100%, which is also optimal. High-risk loans has a smaller precision of 85%, and a recall of 91%, which we hope we can improve by dealing with the imbalance data.The f1 score, which balances prediction and recall is equal to 88%, which is only OK.\n",
    "\n",
    "* The 91% recall means that out of the 635 defaulted loans, 578 (=0.91*635) this model correctly classified as hig-risk loans, and 57 were wrongly classified as healthy loans. This imply that 57 approved loans are high-risk, and have the cost associated to a default.\n",
    "* The 85% precision with 578 true positives (from the previous calculation), means that 680 (=578/0.85) loans where the total amount classified as high-risk loans. Then, 102 (0.15*680) healthy loans were wrongly classified as high-risk loans. This imply the loss of the opportunity cost of 102 loans that were not approved, but would have been profitable if we have done it.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predict a Logistic Regression Model with Resampled Training Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 1: Use the `RandomOverSampler` module from the imbalanced-learn library to resample the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import the RandomOverSampler module form imbalanced-learn\n",
    "from imblearn.over_sampling import RandomOverSampler\n",
    "\n",
    "# Instantiate the random oversampler model with a random_state parameter of 1 to the model\n",
    "random_oversampler=RandomOverSampler(random_state=1)\n",
    "\n",
    "# Fit the original training data to the random_oversampler model\n",
    "X_resampled, y_resampled=random_oversampler.fit_resample(X_train, y_train)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0    56271\n",
       "1    56271\n",
       "Name: loan_status, dtype: int64"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Count the distinct values of the resampled labels data. Of course they should match.\n",
    "y_resampled.value_counts()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 2: Use the `LogisticRegression` classifier and the resampled data to fit the model and make predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Instantiate the Logistic Regression model with a random_state parameter of 1\n",
    "model_oversampled=LogisticRegression(random_state=1)\n",
    "\n",
    "# Fit the model using the resampled training data\n",
    "model_oversampled.fit(X_resampled, y_resampled)\n",
    "\n",
    "# Make a prediction using the testing data\n",
    "y_pred_resampled=model_oversampled.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 3: Evaluate the model’s performance\n",
    "\n",
    "* Calculate the accuracy score of the model.\n",
    "\n",
    "* Generate a confusion matrix.\n",
    "\n",
    "* Print the classification report."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9936781215845847"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Print the balanced_accuracy score of the model \n",
    "balanced_accuracy_score(y_test, y_pred_resampled)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[18649,   116],\n",
       "       [    4,   615]])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Generate a confusion matrix for the model\n",
    "confusion_matrix(y_test, y_pred_resampled)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " \n",
      "             Classification Report with Re-balaced Data Model\n",
      "                   pre       rec       spe        f1       geo       iba       sup\n",
      "\n",
      "          0       1.00      0.99      0.99      1.00      0.99      0.99     18765\n",
      "          1       0.84      0.99      0.99      0.91      0.99      0.99       619\n",
      "\n",
      "avg / total       0.99      0.99      0.99      0.99      0.99      0.99     19384\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Print the classification report for the model\n",
    "print(\" \\n             Classification Report with Re-balaced Data Model\")\n",
    "print(classification_report_imbalanced(y_test, y_pred_resampled))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Step 4: Assessment of the Logistic Regression Model with Resampled Data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's comment on the classification report for the logistic regression model fit with oversampled data. How well the model predict both the `0` (healthy loan) and `1` (high-risk loan) labels.\n",
    "\n",
    "**High-risk loans**\n",
    "\n",
    "The model does predicted significantly better the high-risk loans when including resampling. Particularly, there is an amazing improvement in the recall from 0.91 to 1.00! That means that all of the negative falses (declare healthy loan when they were a highly risky) now are correctly labeled as 1 (high-risk loan). This is awesome in terms of controling the cost associated to the default of loans. Without resampling, the bank would have to absorve the default cost of those 57 high-risk loans badly classified by the previous model. Fortunately, when applying the resample to control the imbalance, we see that that cost of default is totally prevented by the improved model.\n",
    "\n",
    "The opportunity cost associated to the 102 loans wrongly classified as high-risk loans is still there, though. The resampling wasn't able to improve in recognizing that caracterization as we see the same 0.85 precision score in both models.\n",
    "\n",
    "**Healthy loans**\n",
    "\n",
    "Healthy loans are well classified in both models. Since it was a very good classification using original data, the addition of oversample did not improve significantly the classification.\n",
    "\n",
    "**Conclusions**\n",
    "\n",
    "There is a massive improvement in the overall accuracy of the logistic regression model classification on loan risks when using resampled data.\n",
    "\n",
    "A report with the full analysis is in the README file. Thanks for reading."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}