LGD_and_EAD_Models.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "LGD_and_EAD_Models.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true,
      "authorship_tag": "ABX9TyPjI8j+lIvINWFI+YpGd0vy",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/RegaipKURT/Credit-Risk-Modelling/blob/master/LGD_and_EAD_Models.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "d2rT8WLDG8_7",
        "colab_type": "text"
      },
      "source": [
        "# Credit Risk Modelling\n",
        "\n",
        "dataset = https://www.kaggle.com/wendykan/lending-club-loan-data\n",
        "\n",
        "The dataset contains all available data for more than 800,000 consumer loans issued from 2007 to 2015 by Lending Club: a large US peer-to-peer lending company. There are several different versions of this dataset. We have used a version available on kaggle.com. You can find it here: https://www.kaggle.com/wendykan/lending-club-loan-data/version/1\n",
        "We divided the data into two periods because we assume that some data are available at the moment when we need to build Expected Loss models, and some data comes from applications after. Later, we investigate whether the applications we have after we built the Probability of Default (PD) model have similar characteristics with the applications we used to build the PD model.\n",
        "\n",
        "You can download data from: https://www.dropbox.com/sh/1z45jcpzcxmg70z/AAABTpbtMzm1w7-2AoQhB7WJa?dl=0\n",
        "\n",
        "I will use this data via  my drive account. If you wanna learn more about how to use Colab with your own data: https://github.com/RegaipKURT/Artificial_Intelligence_PYTHON/blob/master/colab_ozel_verilerle_calisma.ipynb"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2mm74QmOlKl_",
        "colab_type": "text"
      },
      "source": [
        "# Import, Explore and Process Data "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "c5d-hesKUFPX",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        },
        "outputId": "d760c35c-c1f4-4410-dba0-a311f8beee77"
      },
      "source": [
        "# Import drive and extract data drom zip\n",
        "from google.colab import drive\n",
        "from zipfile import ZipFile\n",
        "\n",
        "drive.mount('GoogleDrive') # Import etmesini istediğimiz şekilde bir path belirliyoruz. (GoogleDrive isimli klasör olarak import edecek)\n",
        "#Çıkan linke tıklayıp aktivasyon kodumuzu giriyoruz\n",
        "\n",
        "#Copy data to our directory and exract from zip\n",
        "!cp \"/content/GoogleDrive/My Drive/Regaip/Kurslar/Credit Risk Modelling/Credit Risk Modeling in Python.zip\" .\n",
        "\n",
        "with ZipFile(\"Credit Risk Modeling in Python.zip\", \"r\") as myZip:\n",
        "    myZip.extractall()\n",
        "\n",
        "!rm -r \"Notebooks\"* \"Credit\"* #remove other directories except Dataset"
      ],
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Mounted at GoogleDrive\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ZDR8v1lfSpav",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 51
        },
        "outputId": "4d56b099-372e-4bf6-fba9-cc8b3e20faa7"
      },
      "source": [
        "import numpy as np\n",
        "import pandas as pd\n",
        "\n",
        "# Import data.\n",
        "loan_data_preprocessed = pd.read_csv('Dataset/loan_data_2007_2014_preprocessed.csv')"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/IPython/core/interactiveshell.py:2718: DtypeWarning: Columns (21,49) have mixed types.Specify dtype option on import or set low_memory=False.\n",
            "  interactivity=interactivity, compiler=compiler, result=result)\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zuwrelx-VqIU",
        "colab_type": "text"
      },
      "source": [
        "## Explore Data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "RR8MRVrPVqIV",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 221
        },
        "outputId": "b240960f-56ea-4292-960d-6a42327f92f1"
      },
      "source": [
        "# Displays all column names.\n",
        "loan_data_preprocessed.columns.values\n",
        "\n",
        "#Head and tail of the dataset\n",
        "loan_data_preprocessed.head()\n",
        "loan_data_preprocessed.tail()\n",
        "\n",
        "# Here we take only the accounts that were charged-off (written-off).\n",
        "loan_data_defaults = loan_data_preprocessed[loan_data_preprocessed['loan_status'].isin(['Charged Off','Does not meet the credit policy. Status:Charged Off'])]\n",
        "\n",
        "loan_data_defaults.shape\n",
        "\n",
        "pd.options.display.max_rows = 10 #change to see all rows\n",
        "# Sets the pandas dataframe options to display all columns/ rows.\n",
        "loan_data_defaults.isnull().sum() # Eksik veriler\n"
      ],
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Unnamed: 0               0\n",
              "Unnamed: 0.1             0\n",
              "id                       0\n",
              "member_id                0\n",
              "loan_amnt                0\n",
              "                        ..\n",
              "addr_state:WV            0\n",
              "addr_state:WY            0\n",
              "initial_list_status:f    0\n",
              "initial_list_status:w    0\n",
              "good_bad                 0\n",
              "Length: 209, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eodUrkMrVqI2",
        "colab_type": "text"
      },
      "source": [
        "## Independent Variables"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "b80alY07VqI3",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        },
        "outputId": "1d84d9d4-2014-4283-bbb8-09229ccc0b14"
      },
      "source": [
        "# We fill the missing values with zeroes.\n",
        "loan_data_defaults['mths_since_last_delinq'].fillna(0, inplace = True)\n",
        "loan_data_defaults['mths_since_last_record'].fillna(0, inplace=True)"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/pandas/core/generic.py:6245: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame\n",
            "\n",
            "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  self._update_inplace(new_data)\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zkNAwriJVqJD",
        "colab_type": "text"
      },
      "source": [
        "## Dependent Variables"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "mViadvUWVqJE",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 425
        },
        "outputId": "249d149a-04f1-4cd9-e7e3-8353c2ce2608"
      },
      "source": [
        "loan_data_defaults['recovery_rate'] = loan_data_defaults['recoveries'] / loan_data_defaults['funded_amnt']\n",
        "# We calculate the dependent variable for the LGD model: recovery rate.\n",
        "# It is the ratio of recoveries and funded amount.\n",
        "\n",
        "loan_data_defaults['recovery_rate'].describe()\n",
        "# Shows some descriptive statisics for the values of a column.\n",
        "\n",
        "loan_data_defaults['recovery_rate'] = np.where(loan_data_defaults['recovery_rate'] > 1, 1, loan_data_defaults['recovery_rate'])\n",
        "loan_data_defaults['recovery_rate'] = np.where(loan_data_defaults['recovery_rate'] < 0, 0, loan_data_defaults['recovery_rate'])\n",
        "# We set recovery rates that are greater than 1 to 1 and recovery rates that are less than 0 to 0.\n",
        "\n",
        "loan_data_defaults['recovery_rate'].describe()\n",
        "# Shows some descriptive statisics for the values of a column.\n",
        "\n",
        "loan_data_defaults['CCF'] = (loan_data_defaults['funded_amnt'] - loan_data_defaults['total_rec_prncp']) / loan_data_defaults['funded_amnt']\n",
        "# We calculate the dependent variable for the EAD model: credit conversion factor.\n",
        "# It is the ratio of the difference of the amount used at the moment of default to the total funded amount.\n",
        "\n",
        "loan_data_defaults['CCF'].describe()\n",
        "# Shows some descriptive statisics for the values of a column.\n",
        "\n",
        "loan_data_defaults.to_csv('loan_data_defaults.csv')\n",
        "# We save the data to a CSV file."
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:1: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame.\n",
            "Try using .loc[row_indexer,col_indexer] = value instead\n",
            "\n",
            "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  \"\"\"Entry point for launching an IPython kernel.\n",
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:8: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame.\n",
            "Try using .loc[row_indexer,col_indexer] = value instead\n",
            "\n",
            "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  \n",
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:9: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame.\n",
            "Try using .loc[row_indexer,col_indexer] = value instead\n",
            "\n",
            "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  if __name__ == '__main__':\n",
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:15: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame.\n",
            "Try using .loc[row_indexer,col_indexer] = value instead\n",
            "\n",
            "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  from ipykernel import kernelapp as app\n"
          ],
          "name": "stderr"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "APcSbn0bVqJg",
        "colab_type": "text"
      },
      "source": [
        "## Explore Dependent Variables"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "n23MJB6MVqJh",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 608
        },
        "outputId": "60a1378f-c771-46cc-ebd4-2eafcadae0d3"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "sns.set()\n",
        "\n",
        "plt.hist(loan_data_defaults['recovery_rate'], bins = 100)\n",
        "# We plot a histogram of a variable with 100 bins.\n",
        "\n",
        "plt.hist(loan_data_defaults['recovery_rate'], bins = 50)\n",
        "# We plot a histogram of a variable with 50 bins.\n",
        "\n",
        "plt.hist(loan_data_defaults['CCF'], bins = 100)\n",
        "# We plot a histogram of a variable with 100 bins.\n",
        "\n",
        "loan_data_defaults['recovery_rate_0_1'] = np.where(loan_data_defaults['recovery_rate'] == 0, 0, 1)\n",
        "# We create a new variable which is 0 if recovery rate is 0 and 1 otherwise.\n",
        "\n",
        "loan_data_defaults['recovery_rate_0_1']"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/statsmodels/tools/_testing.py:19: FutureWarning: pandas.util.testing is deprecated. Use the functions in the public API at pandas.testing instead.\n",
            "  import pandas.util.testing as tm\n",
            "/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:14: SettingWithCopyWarning: \n",
            "A value is trying to be set on a copy of a slice from a DataFrame.\n",
            "Try using .loc[row_indexer,col_indexer] = value instead\n",
            "\n",
            "See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy\n",
            "  \n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "1         1\n",
              "8         1\n",
              "9         1\n",
              "12        1\n",
              "14        1\n",
              "         ..\n",
              "466254    0\n",
              "466256    0\n",
              "466276    1\n",
              "466277    0\n",
              "466281    0\n",
              "Name: recovery_rate_0_1, Length: 43236, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAD7CAYAAACfQGjDAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAbW0lEQVR4nO3df2yV5f3/8dc5uNOJwA5tKjstbASi3QlNpuMkJMuq2ymuhbWQKY6mg00JapgSJ2k3k4ZWC9W1dH5ck/rFJcb9auh3hkBTwBYXYtBkU4kj2DDBAaKuZyCnVChYDj33/fmDcD4UuNpzek5PT895PpIm59zv+5xe76L3676v+z7ndti2bQsAgJtwTvQAAACpi5AAABgREgAAI0ICAGBESAAAjAgJAIARIQEAMLplogcwHs6evSDLiv3jHzk50xQMDozDiFIXPWeGTOs50/qV4uvZ6XRo5szbblpLy5CwLHtMIXH1tZmGnjNDpvWcaf1K49Mz000AACNCAgBgREgAAIwICQCAESEBADAiJAAARoQEAMAoLT8nMVbWUEi5udNvWD4UuqSzX4QmYEQAMLEIiWs4b3HpeMMDNyyfV7NdEiEBIPMw3QQAMCIkAABGhAQAwIiQAAAYERIAAKNRQ+Ls2bN65JFHVFJSovLycj3xxBPq6+uTJB08eFDLli1TSUmJ1qxZo2AwGHndeNQAAMk1akg4HA6tXbtW3d3d6uzs1Jw5c9Tc3CzLslRdXa3a2lp1d3fL5/OpublZksalBgBIvlFDwu12a9GiRZHnd911l3p7e9XT06OsrCz5fD5JUkVFhbq6uiRpXGoAgOSL6cN0lmVp27Zt8vv9CgQCysvLi9Sys7NlWZb6+/vHpeZ2u6MeZ07OtFjaisrNPomdLtK5NxN6Tn+Z1q80Pj3HFBKbNm3S1KlTtWrVKr3xxhsJH0yiBIMDY7qN30h/4M8/Px/PkFJWbu70tO3NhJ7TX6b1K8XXs9PpMO5cRx0SjY2NOnnypLZu3Sqn0ymPx6Pe3t5Iva+vT06nU263e1xqAIDki+oS2BdeeEE9PT1qbW2Vy+WSJBUWFmpwcFAHDhyQJLW3t6u0tHTcagCA5Bv1SOKjjz7Syy+/rLlz56qiokKSNHv2bLW2tqqpqUl1dXW6dOmS8vPztWXLFkmS0+lMeA0AkHwO27Zjn7xPcfGckzB9C2y6zm8yd5sZMq3nTOtXGr9zEnziGgBgREgAAIwICQCAESEBADAiJAAARoQEAMCIkAAAGBESAAAjQgIAYERIAACMCAkAgBEhAQAwIiQAAEaEBADAiJAAABhFdfvSxsZGdXd36z//+Y86Ozt155136rPPPtPjjz8eWef8+fMaGBjQu+++K0ny+/1yuVzKysqSJFVVVamoqEiSdPDgQdXW1g67sVBOTs6oNQBAckV1JFFcXKy2tjbl5+dHls2ePVsdHR2Rn+LiYpWVlQ17XUtLS6R+NSAsy1J1dbVqa2vV3d0tn8+n5ubmUWsAgOSLKiR8Pp88Ho+xHgqF1NnZqQceuPGubtfr6elRVlaWfD6fJKmiokJdXV2j1gAAyRfVdNNo9u3bp1mzZmnBggXDlldVVcm2bS1cuFAbNmzQjBkzFAgElJeXF1knOztblmWpv79/xJrb7U7EUAEAMUhISGzfvv2Go4i2tjZ5PB6FQiE1NDSovr4+aVNHpnu1xiM3d3rC3zNVpHNvJvSc/jKtX2l8eo47JE6dOqX33ntPTU1Nw5ZfnZ5yuVyqrKzUunXrIst7e3sj6/X19cnpdMrtdo9Yi0UwOCDLsmPuZaQ/cLreVJ0bxmeGTOs50/qV4uvZ6XQYd67jvgR2x44duvfeezVz5szIsosXL+r8+SuDtW1be/bskdfrlSQVFhZqcHBQBw4ckCS1t7ertLR01BoAIPmiOpLYvHmz9u7dqzNnzujhhx+W2+3W7t27JV0JiZqammHrB4NBrV+/XuFwWJZlaf78+aqrq5MkOZ1ONTU1qa6ubthlrqPVAADJ57BtO/Z5mRQXz3TT8YYbr9CaV7M9bQ9dOSzPDJnWc6b1K6XwdBMAIH0REgAAI0ICAGBESAAAjAgJAIARIQEAMCIkAABGhAQAwIiQAAAYERIAACNCAgBgREgAAIwICQCAESEBADAiJAAARoQEAMAoqpBobGyU3+9XQUGBjh49Glnu9/tVWlqq5cuXa/ny5XrrrbcitYMHD2rZsmUqKSnRmjVrFAwG464BAJIrqpAoLi5WW1ub8vPzb6i1tLSoo6NDHR0dKioqkiRZlqXq6mrV1taqu7tbPp9Pzc3NcdUAAMkXVUj4fD55PJ6o37Snp0dZWVny+XySpIqKCnV1dcVVAwAk3y3xvkFVVZVs29bChQu1YcMGzZgxQ4FAQHl5eZF1srOzZVmW+vv7x1xzu93xDhUAEKO4QqKtrU0ej0ehUEgNDQ2qr69Piekh0w2945GbOz3h75kq0rk3E3pOf5nWrzQ+PccVElenoFwulyorK7Vu3brI8t7e3sh6fX19cjqdcrvdY67FIhgckGXZMfcz0h/488/Px/x+k0Fu7vS07c2EntNfpvUrxdez0+kw7lyP+RLYixcv6vz5KwOybVt79uyR1+uVJBUWFmpwcFAHDhyQJLW3t6u0tDSuGgAg+aI6kti8ebP27t2rM2fO6OGHH5bb7dbWrVu1fv16hcNhWZal+fPnq66uTpLkdDrV1NSkuro6Xbp0Sfn5+dqyZUtcNQBA8jls2459XibFxTPddLzhgRuWz6vZnraHrhyWZ4ZM6znT+pVScLoJAJD+CAkAgBEhAQAwIiQAAEaEBADAiJAAABgREgAAI0ICAGBESAAAjAgJAIARIQEAMCIkAABGhAQAwIiQAAAYERIAACNCAgBgFFVINDY2yu/3q6CgQEePHpUknT17Vo888ohKSkpUXl6uJ554Qn19fZHXFBQUqLy8XMuXL9fy5ct15MiRSG3fvn0qLS3Vfffdp1/+8pf68ssvo6oBAJIrqpAoLi5WW1ub8vPzI8scDofWrl2r7u5udXZ2as6cOWpubh72uvb2dnV0dKijo0MFBQWSpAsXLmjjxo3aunWr3njjDd1222165ZVXRq0BAJIvqpDw+XzyeDzDlrndbi1atCjy/K677lJvb++o77V//34VFhZq7ty5kqSKigq9/vrro9YAAMl3SyLexLIsbdu2TX6/f9jy1atXKxwO65577tH69evlcrkUCASUl5cXWScvL0+BQECSRqzFwnSv1njk5k5P+HuminTuzYSe01+m9SuNT88JCYlNmzZp6tSpWrVqVWTZm2++KY/Ho4GBAVVXV6u1tVVPPfVUIn7dqILBAVmWHfPrRvoDp+tN1blhfGbItJ4zrV8pvp6dTodx5zruq5saGxt18uRJvfjii3I6/+/trk5PTZs2TQ8++KDef//9yPJrp6V6e3sj645UAwAkX1wh8cILL6inp0etra1yuVyR5V988YUGBwclSUNDQ+ru7pbX65UkFRUV6YMPPtDHH38s6crJ7SVLloxaAwAkX1TTTZs3b9bevXt15swZPfzww3K73XrxxRf18ssva+7cuaqoqJAkzZ49W62trTp+/Lhqa2vlcDg0NDSku+++W08++aSkK0cW9fX1euyxx2RZlrxer2pqakatAQCSz2HbduyT9ykunnMSxxseuGH5vJrtaTu/ydxtZsi0njOtXymFz0kAANIXIQEAMCIkAABGhAQAwIiQAAAYERIAACNCAgBgREgAAIwICQCAESEBADAiJAAARoQEAMCIkAAAGBESAAAjQgIAYDRqSDQ2Nsrv96ugoEBHjx6NLD9x4oRWrlypkpISrVy5MnI3ufGqAQCSb9SQKC4uVltbm/Lz84ctr6urU2Vlpbq7u1VZWana2tpxrQEAkm/UkPD5fPJ4PMOWBYNBHT58WGVlZZKksrIyHT58WH19feNSAwBMjKjucX29QCCgWbNmacqUKZKkKVOm6Pbbb1cgEJBt2wmvZWdnJ6JXAECMxhQSqc50r9Z45OZOT/h7pop07s2EntNfpvUrjU/PYwoJj8ejU6dOKRwOa8qUKQqHwzp9+rQ8Ho9s2054LVbB4IAsy475dSP9gdP1purcMD4zZFrPmdavFF/PTqfDuHM9pktgc3Jy5PV6tWvXLknSrl275PV6lZ2dPS41AMDEcNi2PeIu9+bNm7V3716dOXNGM2fOlNvt1u7du3Xs2DE9/fTTOnfunGbMmKHGxkbNmzdPksalFot4jiSONzxww/J5NdvTdq+EPa7MkGk9Z1q/0vgdSYwaEpMRIRE9/mfKDJnWc6b1K6XYdBMAIDMQEgAAI0ICAGBESAAAjAgJAIARIQEAMCIkAABGhAQAwIiQAAAYERIAACNCAgBgREgAAIwICQCAESEBADAiJAAARoQEAMBoTPe4vuqzzz7T448/Hnl+/vx5DQwM6N1335Xf75fL5VJWVpYkqaqqSkVFRZKkgwcPqra2VpcuXVJ+fr62bNminJycUWsAgOSKKyRmz56tjo6OyPOGhgaFw+HI85aWFt15553DXmNZlqqrq/X888/L5/PppZdeUnNzs55//vkRawCA5EvYdFMoFFJnZ6ceeODG239eq6enR1lZWfL5fJKkiooKdXV1jVoDACRfXEcS19q3b59mzZqlBQsWRJZVVVXJtm0tXLhQGzZs0IwZMxQIBJSXlxdZJzs7W5Zlqb+/f8Sa2+2Oeiyme7XGIzd3esLfM1Wkc28m9Jz+Mq1faXx6TlhIbN++fdhRRFtbmzwej0KhkBoaGlRfX6/m5uZE/boRBYMDsiw75teN9AdO15uqc8P4zJBpPWdav1J8PTudDuPOdUKmm06dOqX33ntP5eXlkWUej0eS5HK5VFlZqffffz+yvLe3N7JeX1+fnE6n3G73iDUAQPIlJCR27Nihe++9VzNnzpQkXbx4UefPX0k027a1Z88eeb1eSVJhYaEGBwd14MABSVJ7e7tKS0tHrQEAki8h0007duxQTU1N5HkwGNT69esVDodlWZbmz5+vuro6SZLT6VRTU5Pq6uqGXeY6Wg0AkHwO27Zjn7xPcfGckzjecOPVWfNqtqft/CZzt5kh03rOtH6lFD8nAQBIT4QEAMCIkAAAGBESAAAjQgIAYERIAACMCAkAgBEhAQAwIiQAAEaEBADAiJAAABgREgAAI0ICAGBESAAAjAgJAIBR3Dcd8vv9crlcysrKkiRVVVWpqKhIBw8eVG1t7bCbB+Xk5EjSmGsAgORKyJFES0uLOjo61NHRoaKiIlmWperqatXW1qq7u1s+n0/Nzc2SNOYaACD5xmW6qaenR1lZWfL5fJKkiooKdXV1xVUDACRfQu5xXVVVJdu2tXDhQm3YsEGBQEB5eXmRenZ2tizLUn9//5hrbrc7EUMFAMQg7pBoa2uTx+NRKBRSQ0OD6uvrdd999yVibGNmuldrPHJzpyf8PVNFOvdmQs/pL9P6lcan57hDwuPxSJJcLpcqKyu1bt06/exnP1Nvb29knb6+PjmdTrndbnk8njHVYhEMDsiy7Jh7GekPnK43VeeG8Zkh03rOtH6l+Hp2Oh3Gneu4zklcvHhR589fGZRt29qzZ4+8Xq8KCws1ODioAwcOSJLa29tVWloqSWOuZaKZX3MpN3f6TX9mfs010cMDkAHiOpIIBoNav369wuGwLMvS/PnzVVdXJ6fTqaamJtXV1Q27lFXSmGuZ6BZXlo43PHDT2rya7ZJCyR0QgIwTV0jMmTNHO3fuvGntO9/5jjo7OxNaAwAkF5+4BgAYERIAACNCAgBgREgAAIwICQCAESEBADBKyHc3ZaqZX3PpFlfWDcuHQpd09gs+wwBg8iMk4mD6sBsfdAOQLphuAgAYERIAACNCAgBgxDmJFGA6AQ4AE42QSAEjnwAHgInDdBMAwIiQAAAYERIAAKO4zkmcPXtWv/rVr/TJJ5/I5XLpm9/8purr65Wdna2CggLdeeedcjqv5FBTU5MKCgokSfv27VNTU5PC4bAWLFig559/XrfeeuuoNQBAcsV1JOFwOLR27Vp1d3ers7NTc+bMUXNzc6Te3t6ujo4OdXR0RALiwoUL2rhxo7Zu3ao33nhDt912m1555ZVRawCA5IsrJNxutxYtWhR5ftddd6m3t3fE1+zfv1+FhYWaO3euJKmiokKvv/76qDUAQPIl7BJYy7K0bds2+f3+yLLVq1crHA7rnnvu0fr16+VyuRQIBJSXlxdZJy8vT4FAQJJGrAEAki9hIbFp0yZNnTpVq1atkiS9+eab8ng8GhgYUHV1tVpbW/XUU08l6teNKCdnWsLfMzd3+riuPxaJ+h3JGGuqoef0l2n9SuPTc0JCorGxUSdPntTWrVsjJ6o9Ho8kadq0aXrwwQf16quvRpa/8847kdf29vZG1h2pFotgcECWZcf8upH+wJ9/fj7u9cfye0cSy+8Y6Xcn4n0mE3pOf5nWrxRfz06nw7hzHfclsC+88IJ6enrU2toql8slSfriiy80ODgoSRoaGlJ3d7e8Xq8kqaioSB988IE+/vhjSVdObi9ZsmTUGgAg+eI6kvjoo4/08ssva+7cuaqoqJAkzZ49W2vXrlVtba0cDoeGhoZ0991368knn5R05ciivr5ejz32mCzLktfrVU1Nzai1dMB3NAGYbOIKiTvuuENHjhy5aa2zs9P4usWLF2vx4sUx1yY7vqMJwGTDJ64BAEZ8C2wUrKFQRl4pAQCERBSct7hSbprIFFxDoUs6+wX31waQGITEJDVycBESABKDcxIAACOOJAAggaa7s/TVr1z5zNjg5ZDO918a0zqpgpAYB5zoBtKHaYNuWv7Vr7j0k/+/TpL015X/T+d1YwCY1knF8CAkxkEqnugGcKNoNsqmDXo0YRAKX47sMEaz0Y/mPZONkEgzXPUERG+8N8quKV9JuY1+rAiJNMNVT8Bw1x4tSOY9+mv3+i8NhZR1i2vEdaJZng4ICQCT1rUBcO2GPXTdEfXVvXlJ+suKlptu0K/f67/2sWmd0ZZfK54gmchzFYQEgAll2tCbThJfv5dv2rCbNtrRbNDHg+n3RhMeE3mugpAAMG6iCYDrN4BXH1+/xx9NAExGExVa0SIkAMQt1j1905TPtVJ945kpCAkAkmL/EFi00z43QwCMLpppqGv/PULhy+MyDkIiQ5gujbWGuOJpshlpYx7Nidxrl5s29Nfu6ccTBhi7aIL0+qk6aTDh40jJkDhx4oSefvpp9ff3y+12q7GxUXPnzp3oYU1qpktj5/56G5+rSLBoPqFr2lDHerL2ZtM20ZzIjfUqHsIgdST7ctuUDIm6ujpVVlZq+fLl6ujoUG1trf70pz9N9LDSUrqFRzQnSk3XwyfqsWTeiI+2oY71ZO21G/Pra0hPyZ6qS7mQCAaDOnz4sF599VVJUllZmTZt2qS+vj5lZ2dH9R5Op2PMv/+Wr+Wm7PLbf9agr87IkXRlQ3ez14TClyNf/zF4LqjTf/q/e4RfXX/Y+wyFbrr+kMOhq5u8S5cvKesrV+7NbTkdkY1YaCgk1zXTGaM9TuRGeKTa451Xemgtb4g8/p+lzwzb+N5snUQ+zp165b9V15Sv3HS5pJs+jnX96x9Hu14mP06VcYzH47Fu+0Z6ncO2bXtM7zpOenp69Otf/1q7d++OLFu6dKm2bNmiBQsWTODIACDzcD8JAIBRyoWEx+PRqVOnFA6HJUnhcFinT5+Wx+OZ4JEBQOZJuZDIycmR1+vVrl27JEm7du2S1+uN+nwEACBxUu6chCQdO3ZMTz/9tM6dO6cZM2aosbFR8+bNm+hhAUDGScmQAACkhpSbbgIApA5CAgBgREgAAIwICQCAUcaFxIkTJ7Ry5UqVlJRo5cqV+vjjj29YJxwO69lnn9XixYt133336bXXXkv+QBMomp5bW1v1ox/9SOXl5br//vv11ltvJX+gCRRNz1cdP35c3/72t9XY2Ji8AY6DaHves2ePysvLVVZWpvLycp05cya5A02QaPoNBoN69NFHVV5eriVLluiZZ57R0NBQ8gebII2NjfL7/SooKNDRo0dvuk7Ct192hlm9erW9c+dO27Zte+fOnfbq1atvWGfHjh32mjVr7HA4bAeDQbuoqMj+9NNPkz3UhImm5/3799sXL160bdu2//Wvf9kLFy60v/zyy6SOM5Gi6dm2bXtoaMhetWqVvWHDBvs3v/lNMoeYcNH0fOjQIXvJkiX26dOnbdu27XPnztmDg4NJHWeiRNPv5s2bI/+uoVDIXrFihb179+6kjjOR3nvvPbu3t9f+wQ9+YB85cuSm6yR6+5VRRxJXvzywrKxM0pUvDzx8+LD6+vqGrbdnzx49+OCDcjqdys7O1uLFi9XV1TURQ45btD0XFRXp1ltvlSQVFBTItm319/cnfbyJEG3PkvT73/9e3//+9yf9V9FH2/Mf/vAHrVmzRrm5V77scfr06crKykr6eOMVbb8Oh0MXLlyQZVkKhUK6fPmyZs2aNRFDTgifzzfqt08kevuVUSERCAQ0a9YsTZkyRZI0ZcoU3X777QoEAjesl5eXF3nu8Xj03//+N6ljTZRoe77Wzp079Y1vfENf//rXkzXMhIq25w8//FBvv/22HnrooQkYZWJF2/OxY8f06aef6qc//al+/OMf66WXXpI9CT8qFW2/v/jFL3TixAl973vfi/wsXLhwIoacNInefmVUSGB07777rn73u9/pt7/97UQPZVxdvnxZGzdu1LPPPhvZ0GSCcDisI0eO6NVXX9Wf//xn7d+/Xx0dHRM9rHHT1dWlgoICvf3229q/f78OHDgwaWcFJkpGhUS0Xx7o8XjU29sbeR4IBCbtXnUsX5j4z3/+U9XV1WptbZ3UX4MSTc+ff/65PvnkEz366KPy+/364x//qL/+9a/auHHjRA07LtH+O+fl5am0tFQul0vTpk1TcXGxDh06NBFDjku0/f7lL3/RsmXL5HQ6NX36dPn9fr3zzjsTMeSkSfT2K6NCItovDywtLdVrr70my7LU19env/3tbyopKZmIIcct2p4PHTqkp556Si0tLZP+vh3R9JyXl6d33nlH+/bt0759+/Tzn/9cP/nJT7Rp06aJGnZcov13Lisr09tvvy3btnX58mX94x//0Le+9a2JGHJcou139uzZ2r9/vyQpFArp73//u+64446kjzeZEr79GvMp70nq3//+t71ixQr7hz/8ob1ixQr72LFjtm3b9tq1a+1Dhw7Ztn3lipfa2lq7uLjYLi4uttvb2ydyyHGLpuf777/fXrRokb1s2bLIz4cffjiRw45LND1fq6WlZdJf3RRNz+Fw2H7uuefs0tJSe+nSpfZzzz1nh8PhiRz2mEXT78mTJ+2HHnrILisrs5csWWI/88wz9uXLlydy2HHZtGmTXVRUZHu9Xvu73/2uvXTpUtu2x3f7xRf8AQCMMmq6CQAQG0ICAGBESAAAjAgJAIARIQEAMCIkAABGhAQAwIiQAAAY/S/4m2PCd2wl/wAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "lZFjQAW2VqJ1",
        "colab_type": "text"
      },
      "source": [
        "# LGD Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iKF-ViNFVqJ2",
        "colab_type": "text"
      },
      "source": [
        "### Splitting Data"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "s8BPNO9KVqJ3",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "from sklearn.model_selection import train_test_split"
      ],
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p0sXD8kDVqJ5",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "# LGD model stage 1 datasets: recovery rate 0 or greater than 0.\n",
        "lgd_inputs_stage_1_train, lgd_inputs_stage_1_test, lgd_targets_stage_1_train, lgd_targets_stage_1_test = train_test_split(loan_data_defaults.drop(['good_bad', 'recovery_rate','recovery_rate_0_1', 'CCF'], axis = 1), loan_data_defaults['recovery_rate_0_1'], test_size = 0.2, random_state = 42)\n",
        "# Takes a set of inputs and a set of targets as arguments. Splits the inputs and the targets into four dataframes:\n",
        "# Inputs - Train, Inputs - Test, Targets - Train, Targets - Test."
      ],
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "RHAPMpWdVqJ8",
        "colab_type": "text"
      },
      "source": [
        "### Preparing the Inputs"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Pi5vJX8eVqJ9",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 221
        },
        "outputId": "7866e34f-ce91-4cdb-c0fc-d08b8798c2b4"
      },
      "source": [
        "# List of all independent variables for the models.\n",
        "features_all = ['grade:A',\n",
        "'grade:B',\n",
        "'grade:C',\n",
        "'grade:D',\n",
        "'grade:E',\n",
        "'grade:F',\n",
        "'grade:G',\n",
        "'home_ownership:MORTGAGE',\n",
        "'home_ownership:NONE',\n",
        "'home_ownership:OTHER',\n",
        "'home_ownership:OWN',\n",
        "'home_ownership:RENT',\n",
        "'verification_status:Not Verified',\n",
        "'verification_status:Source Verified',\n",
        "'verification_status:Verified',\n",
        "'purpose:car',\n",
        "'purpose:credit_card',\n",
        "'purpose:debt_consolidation',\n",
        "'purpose:educational',\n",
        "'purpose:home_improvement',\n",
        "'purpose:house',\n",
        "'purpose:major_purchase',\n",
        "'purpose:medical',\n",
        "'purpose:moving',\n",
        "'purpose:other',\n",
        "'purpose:renewable_energy',\n",
        "'purpose:small_business',\n",
        "'purpose:vacation',\n",
        "'purpose:wedding',\n",
        "'initial_list_status:f',\n",
        "'initial_list_status:w',\n",
        "'term_int',\n",
        "'emp_length_int',\n",
        "'mths_since_issue_d',\n",
        "'mths_since_earliest_cr_line',\n",
        "'funded_amnt',\n",
        "'int_rate',\n",
        "'installment',\n",
        "'annual_inc',\n",
        "'dti',\n",
        "'delinq_2yrs',\n",
        "'inq_last_6mths',\n",
        "'mths_since_last_delinq',\n",
        "'mths_since_last_record',\n",
        "'open_acc',\n",
        "'pub_rec',\n",
        "'total_acc',\n",
        "'acc_now_delinq',\n",
        "'total_rev_hi_lim']\n",
        "\n",
        "\n",
        "# List of the dummy variable reference categories. \n",
        "features_reference_cat = ['grade:G',\n",
        "'home_ownership:RENT',\n",
        "'verification_status:Verified',\n",
        "'purpose:credit_card',\n",
        "'initial_list_status:f']\n",
        "\n",
        "# Here we keep only the variables we need for the model.\n",
        "lgd_inputs_stage_1_train = lgd_inputs_stage_1_train[features_all]\n",
        "\n",
        "# Here we remove the dummy variable reference categories.\n",
        "lgd_inputs_stage_1_train = lgd_inputs_stage_1_train.drop(features_reference_cat, axis = 1)\n",
        "\n",
        "# Check for missing values. We check whether the value of each row for each column is missing or not,\n",
        "# then sum accross columns.\n",
        "lgd_inputs_stage_1_train.isnull().sum()"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "grade:A             0\n",
              "grade:B             0\n",
              "grade:C             0\n",
              "grade:D             0\n",
              "grade:E             0\n",
              "                   ..\n",
              "open_acc            0\n",
              "pub_rec             0\n",
              "total_acc           0\n",
              "acc_now_delinq      0\n",
              "total_rev_hi_lim    0\n",
              "Length: 44, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SfC-AGn9VqKN",
        "colab_type": "text"
      },
      "source": [
        "### Estimating the Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KvWrveWfVqKN",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 555
        },
        "outputId": "72ba05e1-c819-4add-c87a-bbc2bb1ba82b"
      },
      "source": [
        "# P values for sklearn logistic regression.\n",
        "from sklearn import linear_model\n",
        "import scipy.stats as stat\n",
        "\n",
        "# Class to display p-values for logistic regression in sklearn.\n",
        "class LogisticRegression_with_p_values:\n",
        "    \n",
        "    def __init__(self,*args,**kwargs):#,**kwargs):\n",
        "        self.model = linear_model.LogisticRegression(*args,**kwargs)#,**args)\n",
        "\n",
        "    def fit(self,X,y):\n",
        "        self.model.fit(X,y)\n",
        "        \n",
        "        #### Get p-values for the fitted model ####\n",
        "        denom = (2.0 * (1.0 + np.cosh(self.model.decision_function(X))))\n",
        "        denom = np.tile(denom,(X.shape[1],1)).T\n",
        "        F_ij = np.dot((X / denom).T,X) ## Fisher Information Matrix\n",
        "        Cramer_Rao = np.linalg.inv(F_ij) ## Inverse Information Matrix\n",
        "        sigma_estimates = np.sqrt(np.diagonal(Cramer_Rao))\n",
        "        z_scores = self.model.coef_[0] / sigma_estimates # z-score for eaach model coefficient\n",
        "        p_values = [stat.norm.sf(abs(x)) * 2 for x in z_scores] ### two tailed test for p-values\n",
        "        \n",
        "        self.coef_ = self.model.coef_\n",
        "        self.intercept_ = self.model.intercept_\n",
        "        #self.z_scores = z_scores\n",
        "        self.p_values = p_values\n",
        "        #self.sigma_estimates = sigma_estimates\n",
        "        #self.F_ij = F_ij\n",
        "\n",
        "reg_lgd_st_1 = LogisticRegression_with_p_values()\n",
        "# We create an instance of an object from the 'LogisticRegression' class.\n",
        "reg_lgd_st_1.fit(lgd_inputs_stage_1_train, lgd_targets_stage_1_train)\n",
        "# Estimates the coefficients of the object from the 'LogisticRegression' class\n",
        "# with inputs (independent variables) contained in the first dataframe\n",
        "# and targets (dependent variables) contained in the second dataframe.\n",
        "\n",
        "\n",
        "feature_name = lgd_inputs_stage_1_train.columns.values\n",
        "# Stores the names of the columns of a dataframe in a variable.\n",
        "\n",
        "\n",
        "summary_table = pd.DataFrame(columns = ['Feature name'], data = feature_name)\n",
        "# Creates a dataframe with a column titled 'Feature name' and row values contained in the 'feature_name' variable.\n",
        "summary_table['Coefficients'] = np.transpose(reg_lgd_st_1.coef_)\n",
        "# Creates a new column in the dataframe, called 'Coefficients',\n",
        "# with row values the transposed coefficients from the 'LogisticRegression' object.\n",
        "summary_table.index = summary_table.index + 1\n",
        "# Increases the index of every row of the dataframe with 1.\n",
        "summary_table.loc[0] = ['Intercept', reg_lgd_st_1.intercept_[0]]\n",
        "# Assigns values of the row with index 0 of the dataframe.\n",
        "summary_table = summary_table.sort_index()\n",
        "# Sorts the dataframe by index.\n",
        "p_values = reg_lgd_st_1.p_values\n",
        "# We take the result of the newly added method 'p_values' and store it in a variable 'p_values'.\n",
        "p_values = np.append(np.nan,np.array(p_values))\n",
        "# We add the value 'NaN' in the beginning of the variable with p-values.\n",
        "summary_table['p_values'] = p_values\n",
        "# In the 'summary_table' dataframe, we add a new column, called 'p_values', containing the values from the 'p_values' variable.\n",
        "summary_table\n",
        "\n",
        "\n",
        "summary_table = pd.DataFrame(columns = ['Feature name'], data = feature_name)\n",
        "summary_table['Coefficients'] = np.transpose(reg_lgd_st_1.coef_)\n",
        "summary_table.index = summary_table.index + 1\n",
        "summary_table.loc[0] = ['Intercept', reg_lgd_st_1.intercept_[0]]\n",
        "summary_table = summary_table.sort_index()\n",
        "p_values = reg_lgd_st_1.p_values\n",
        "p_values = np.append(np.nan,np.array(p_values))\n",
        "summary_table['p_values'] = p_values\n",
        "summary_table"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_logistic.py:940: ConvergenceWarning: lbfgs failed to converge (status=1):\n",
            "STOP: TOTAL NO. of ITERATIONS REACHED LIMIT.\n",
            "\n",
            "Increase the number of iterations (max_iter) or scale the data as shown in:\n",
            "    https://scikit-learn.org/stable/modules/preprocessing.html\n",
            "Please also refer to the documentation for alternative solver options:\n",
            "    https://scikit-learn.org/stable/modules/linear_model.html#logistic-regression\n",
            "  extra_warning_msg=_LOGISTIC_SOLVER_CONVERGENCE_MSG)\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Feature name</th>\n",
              "      <th>Coefficients</th>\n",
              "      <th>p_values</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Intercept</td>\n",
              "      <td>-0.000101</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>grade:A</td>\n",
              "      <td>-0.000012</td>\n",
              "      <td>9.998834e-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>grade:B</td>\n",
              "      <td>-0.000073</td>\n",
              "      <td>9.991412e-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>grade:C</td>\n",
              "      <td>-0.000140</td>\n",
              "      <td>9.982986e-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>grade:D</td>\n",
              "      <td>0.000019</td>\n",
              "      <td>9.997737e-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>40</th>\n",
              "      <td>open_acc</td>\n",
              "      <td>-0.002215</td>\n",
              "      <td>4.977482e-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41</th>\n",
              "      <td>pub_rec</td>\n",
              "      <td>-0.000059</td>\n",
              "      <td>9.988095e-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>42</th>\n",
              "      <td>total_acc</td>\n",
              "      <td>-0.005452</td>\n",
              "      <td>1.610537e-04</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>43</th>\n",
              "      <td>acc_now_delinq</td>\n",
              "      <td>0.000003</td>\n",
              "      <td>9.999843e-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>44</th>\n",
              "      <td>total_rev_hi_lim</td>\n",
              "      <td>-0.000007</td>\n",
              "      <td>6.108169e-22</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>45 rows × 3 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "        Feature name  Coefficients      p_values\n",
              "0          Intercept     -0.000101           NaN\n",
              "1            grade:A     -0.000012  9.998834e-01\n",
              "2            grade:B     -0.000073  9.991412e-01\n",
              "3            grade:C     -0.000140  9.982986e-01\n",
              "4            grade:D      0.000019  9.997737e-01\n",
              "..               ...           ...           ...\n",
              "40          open_acc     -0.002215  4.977482e-01\n",
              "41           pub_rec     -0.000059  9.988095e-01\n",
              "42         total_acc     -0.005452  1.610537e-04\n",
              "43    acc_now_delinq      0.000003  9.999843e-01\n",
              "44  total_rev_hi_lim     -0.000007  6.108169e-22\n",
              "\n",
              "[45 rows x 3 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "vf-UngaLVqKf",
        "colab_type": "text"
      },
      "source": [
        "### Testing the Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WimYMyNHVqKg",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        },
        "outputId": "3a1e7824-7038-44a3-c0fa-df505521f80d"
      },
      "source": [
        "lgd_inputs_stage_1_test = lgd_inputs_stage_1_test[features_all]\n",
        "# Here we keep only the variables we need for the model.\n",
        "\n",
        "lgd_inputs_stage_1_test = lgd_inputs_stage_1_test.drop(features_reference_cat, axis = 1)\n",
        "# Here we remove the dummy variable reference categories.\n",
        "\n",
        "y_hat_test_lgd_stage_1 = reg_lgd_st_1.model.predict(lgd_inputs_stage_1_test)\n",
        "# Calculates the predicted values for the dependent variable (targets)\n",
        "# based on the values of the independent variables (inputs) supplied as an argument.\n",
        "\n",
        "y_hat_test_lgd_stage_1\n",
        "\n",
        "y_hat_test_proba_lgd_stage_1 = reg_lgd_st_1.model.predict_proba(lgd_inputs_stage_1_test)\n",
        "# Calculates the predicted probability values for the dependent variable (targets)\n",
        "# based on the values of the independent variables (inputs) supplied as an argument.\n",
        "\n",
        "\n",
        "y_hat_test_proba_lgd_stage_1\n",
        "# This is an array of arrays of predicted class probabilities for all classes.\n",
        "# In this case, the first value of every sub-array is the probability for the observation to belong to the first class, i.e. 0,\n",
        "# and the second value is the probability for the observation to belong to the first class, i.e. 1.\n",
        "\n",
        "y_hat_test_proba_lgd_stage_1 = y_hat_test_proba_lgd_stage_1[: ][: , 1]\n",
        "# Here we take all the arrays in the array, and from each array, we take all rows, and only the element with index 1,\n",
        "# that is, the second element.\n",
        "# In other words, we take only the probabilities for being 1.\n",
        "\n",
        "\n",
        "y_hat_test_proba_lgd_stage_1\n",
        "\n",
        "lgd_targets_stage_1_test_temp = lgd_targets_stage_1_test\n",
        "\n",
        "lgd_targets_stage_1_test_temp.reset_index(drop = True, inplace = True)\n",
        "# We reset the index of a dataframe.\n",
        "\n",
        "df_actual_predicted_probs = pd.concat([lgd_targets_stage_1_test_temp, pd.DataFrame(y_hat_test_proba_lgd_stage_1)], axis = 1)\n",
        "# Concatenates two dataframes.\n",
        "\n",
        "df_actual_predicted_probs.columns = ['lgd_targets_stage_1_test', 'y_hat_test_proba_lgd_stage_1']\n",
        "\n",
        "df_actual_predicted_probs.index = lgd_inputs_stage_1_test.index\n",
        "# Makes the index of one dataframe equal to the index of another dataframe.\n",
        "\n",
        "df_actual_predicted_probs.head()"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>lgd_targets_stage_1_test</th>\n",
              "      <th>y_hat_test_proba_lgd_stage_1</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>178928</th>\n",
              "      <td>1</td>\n",
              "      <td>0.589279</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>69814</th>\n",
              "      <td>1</td>\n",
              "      <td>0.607614</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>101396</th>\n",
              "      <td>0</td>\n",
              "      <td>0.474360</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>463268</th>\n",
              "      <td>1</td>\n",
              "      <td>0.555265</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>253729</th>\n",
              "      <td>0</td>\n",
              "      <td>0.402595</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "        lgd_targets_stage_1_test  y_hat_test_proba_lgd_stage_1\n",
              "178928                         1                      0.589279\n",
              "69814                          1                      0.607614\n",
              "101396                         0                      0.474360\n",
              "463268                         1                      0.555265\n",
              "253729                         0                      0.402595"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "05Vz-ukGVqLQ",
        "colab_type": "text"
      },
      "source": [
        "### Estimating the Аccuracy of the Мodel"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "1qvbVpViVqLR",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 318
        },
        "outputId": "b6103f6e-51c7-4174-95a6-be560c26eec4"
      },
      "source": [
        "tr = 0.5\n",
        "# We create a new column with an indicator,\n",
        "# where every observation that has predicted probability greater than the threshold has a value of 1,\n",
        "# and every observation that has predicted probability lower than the threshold has a value of 0.\n",
        "df_actual_predicted_probs['y_hat_test_lgd_stage_1'] = np.where(df_actual_predicted_probs['y_hat_test_proba_lgd_stage_1'] > tr, 1, 0)\n",
        "\n",
        "pd.crosstab(df_actual_predicted_probs['lgd_targets_stage_1_test'], df_actual_predicted_probs['y_hat_test_lgd_stage_1'], \n",
        "            rownames = ['Actual'], colnames = ['Predicted'])\n",
        "# Creates a cross-table where the actual values are displayed by rows and the predicted values by columns.\n",
        "# This table is known as a Confusion Matrix.\n",
        "\n",
        "pd.crosstab(df_actual_predicted_probs['lgd_targets_stage_1_test'], df_actual_predicted_probs['y_hat_test_lgd_stage_1'], \n",
        "            rownames = ['Actual'], colnames = ['Predicted']) / df_actual_predicted_probs.shape[0]\n",
        "# Here we divide each value of the table by the total number of observations,\n",
        "# thus getting percentages, or, rates.\n",
        "\n",
        "\n",
        "\n",
        "from sklearn.metrics import roc_curve, roc_auc_score\n",
        "fpr, tpr, thresholds = roc_curve(df_actual_predicted_probs['lgd_targets_stage_1_test'], df_actual_predicted_probs['y_hat_test_proba_lgd_stage_1'])\n",
        "# Returns the Receiver Operating Characteristic (ROC) Curve from a set of actual values and their predicted probabilities.\n",
        "# As a result, we get three arrays: the false positive rates, the true positive rates, and the thresholds.\n",
        "# we store each of the three arrays in a separate variable.\n",
        "\n",
        "plt.plot(fpr, tpr)\n",
        "# We plot the false positive rate along the x-axis and the true positive rate along the y-axis,\n",
        "# thus plotting the ROC curve.\n",
        "plt.plot(fpr, fpr, linestyle = '--', color = 'k')\n",
        "# We plot a seconary diagonal line, with dashed line style and black color.\n",
        "plt.xlabel('False positive rate')\n",
        "# We name the x-axis \"False positive rate\".\n",
        "plt.ylabel('True positive rate')\n",
        "# We name the x-axis \"True positive rate\".\n",
        "plt.title('ROC curve')\n",
        "# We name the graph \"ROC curve\".\n",
        "\n",
        "AUROC = roc_auc_score(df_actual_predicted_probs['lgd_targets_stage_1_test'], df_actual_predicted_probs['y_hat_test_proba_lgd_stage_1'])\n",
        "# Calculates the Area Under the Receiver Operating Characteristic Curve (AUROC)\n",
        "# from a set of actual values and their predicted probabilities.\n",
        "AUROC"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0.6496840890974506"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYkAAAEcCAYAAAAydkhNAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdd1gU59rH8S8LKKCggqAoKnaxxm5UEsWuiCWWBCW2aKqeeEzeWKKAHUtMbDHGY+8lx4I1GntEY8RYsERFQUV6FRDYnfcP4p4QRRdld4G9P9fldbHD7M79COxv55mZe8wURVEQQgghnkNl7AKEEEIUXBISQgghciUhIYQQIlcSEkIIIXIlISGEECJXEhJCCCFyJSEhhBAiVxbGLkAIQ/Lw8CAmJgZzc3NsbGxwd3dn8uTJlChRQrvOhQsX+Pbbb7l8+TIqlYrmzZvzxRdfUKNGDe06KSkpfPfdd/z8888kJibi4OBA+/bt+fjjj7G3tzfG0ITQC9mTECZn2bJlBAcHs3PnTkJCQli+fLn2e8HBwYwYMYIOHTpw8uRJjhw5Qu3atXnvvfcIDw8HICMjgyFDhnDr1i1WrFjB77//zpYtWyhdujSXL1/WW91ZWVl6e20hciMhIUyWo6Mjbdu25dq1a9plc+fOpVevXgwZMoSSJUtSunRpxo4dS6NGjVi0aBEAu3btIiIigsWLF1OjRg1UKhUODg58+umnvP3228/d1p9//smwYcNo0aIFrVu3ZtmyZQCMHz+eBQsWaNc7e/Ysb731lvaxh4cHy5cvp2fPnrzxxhssX76cMWPG5Hjt6dOnM336dACSk5OZOHEibdu2xd3dnQULFqBWq/PnP0yYJAkJYbIePXrEyZMnqVy5MgBpaWkEBwfTtWvXZ9bt1q0bv/76KwC//vor7u7uOaaoXiQlJYVhw4bh7u7OyZMnOXToEG+++abOde7du5fly5dz/vx5evTowfHjx0lJSQFArVZz4MABPD09gezQsbCw4NChQ+zcuZPTp0+zbds2nbclxD9JSAiT8+mnn9K4cWPefvtt7O3ttZ/MExMT0Wg0ODo6PvMcR0dH4uPjAUhISHjuOrk5duwYZcuWZfjw4RQvXpySJUvSqFEjnZ/v4+ODs7MzVlZWVKxYkbp163L48GEAgoKCsLKy4o033iAmJobjx48zceJEbGxscHBwYOjQoezdu1fnbQnxT3LgWpicJUuW0Lp1a86dO8e4ceOIj4/Hzs4OOzs7VCoV0dHRVK9ePcdzoqOjKVOmDAClS5cmOjpa5+1FRERo91ZehbOzc47Hnp6eBAYG0rt3bwIDA7V7EQ8fPiQrK4u2bdtq19VoNM88X4i8kD0JYbJatGhB3759CQgIAMDGxoY33niDAwcOPLPu/v37adWqFQCtW7fm1KlTpKam6rQdZ2dn7UHvf7K2tiY9PV37OCYm5pl1zMzMcjzu1q0b586d49GjR/z888/07NkTgPLly1OsWDGCgoI4f/4858+f58KFC7InIV6LhIQwaUOGDOHXX3/l+vXrAIwbN46dO3eydu1aUlJSSExMZMGCBVy8eJHPPvsMgF69elG+fHlGjx7N7du30Wg0xMfHs2zZMo4fP/7MNtq1a0d0dDSrV68mIyODlJQU/vjjDwDc3Nw4fvw4CQkJREdHs2bNmpfWbG9vT4sWLZgwYQIuLi7avR4nJyfatGnD7NmzSUlJQaPREBYWxrlz5/Lrv0uYIAkJYdLs7e3p1asXS5YsAaBZs2asWLGCn3/+GXd3d9q3b8+1a9fYuHEjrq6uABQrVozVq1dTrVo1hg8fTtOmTenfvz/x8fE0bNjwmW2ULFmSlStXcvToUdq0aUOXLl04e/YskB04derUwcPDg+HDh9O9e3ed6vb09OTXX3/VTjU9NWfOHDIzM+nevTvNmzdnzJgxeZoaE+KfzOSmQ0IIIXIjexJCCCFyJSEhhBAiVxISQgghciUhIYQQIlcSEkIIIXIlISGEECJXRa4tR3z8YzSavJ/V6+BQktjYFD1UVHDJmE2DjNk0vOqYVSozypTJvVllkQsJjUZ5pZB4+lxTI2M2DTJm06CPMct0kxBCiFxJSAghhMiVhIQQQohcGSQkAgIC8PDwoHbt2ty8efO566jVavz9/enYsSOdOnWSu2kJIUQBYJCQ6NChAxs2bKBixYq5rrNnzx7CwsI4dOgQW7ZsYdGiRdy/f98Q5QkhhMiFQUKiWbNmL7071r59++jfvz8qlQp7e3s6duz43Ju/CCGEyEmfzbwLzDGJiIgIKlSooH3s7OzMo0ePjFiREEIUTIqi8CRTzb1HSQz4dAaeQydy+fazdzXMD0XuOgkHh5Kv/FxHR9t8rKRwkDGbBhlz4afRKPx+PZJlP10iKj6N9OQows5tICX6T0o518OxlBWOZV/9/S83BSYknJ2defjwofbOXv/cs9BVbGzKK11Q4uhoS3R0cp6fV5jJmE2DjLnw0WgUHsQ8JjQiid2nQ1EUSEzJQKMoKIqGxDvHCLu4GwsLS/41bjJDB3vjVLbkK41ZpTJ74YfrAhMSXbt2Zdu2bXTu3JmEhAQOHz7Mhg0bjF2WEEIY1L1Hyfiv/k37WGVmRhnb4rRvUpGypaxoWN2BaZM3U7mNOxMnTsHJqZxe6zFISEyfPp1Dhw4RExPDsGHDKF26NHv37mXkyJGMGTOGBg0a0KtXL/744w86d+4MwKeffkqlSpUMUZ4QQhjN4/RM7kYkcz0snr1n7mmXV3QswYD2Najnak9WViYrVy6nvVcfnB1KMH/+IqysrDAzM9N7fUXuHtcy3aQ7GbNpkDEXLKnpWcQlpxMWmczR4AfcfpCU4/tVnW15v0sdqpTPPqZy6dJF/Py+5s6dW4wbNx4fn6HPfd1XHXOhmW4SQoiiKiYhjat349h69BZpT9Q5vtegmgNv1CxLVWdbXBxLYmGefdJpWloqS5Z8x4YNaylXrjyLFy+nbdu3DF67hIQQQuSjLLWGuOQn3I1I4kZYAkeDH+T4fo2KpWjb0JkSVpbUq1oGq2LPfxv+8cfvWb9+DQMGeDNmzL8pWTL/z1zShYSEEEK8pswsDYd+CyPkbjzX7sXn+F5VZ1tsilvg3qgCb9QoSzFL81xfJykpifj4OKpUcWXYsFG0bfs2TZo003f5LyQhIYQQr2jnyTv8cuEBKWmZ2mXODjY0rO5A9QqlaFDdgeIvCIW/O3r0CDNn+lG2rCMbN+7A1tbW6AEBEhJCCJEnV+7EsuHnm0TGp2mX1apUmlb1ytGmfnksLXQLhadiY2MICJjBoUP7qVWrDpMnTzXIWUu6kpAQQoiX+ONWDAfPhXE9LEG7rKqzHU1qlaVd44qUsLJ8pde9efMGI0e+T2pqKp999jlDhozA0vLVXktfJCSEEOI5IuNTuXInjp0n7/A4PQuAktaWNKzugE+X2jpPIz2PWq3G3NycqlWr4eHRCR+fYVSrVj2/Ss9XEhJCCPE3Ry/cZ92h/933xs7Gkjb1y9P9zSo4O5R4rdfWaDRs27aZzZvXs2bNZuzs7PD1nf66JeuVhIQQwqTFJKRxIzyB/WfDeBjzWLu8WW1H2jdxoUbFUlhavH7D7Hv3QvH3n8yFC+dp1ao16elp2NnZvfbr6puEhBDCpGRmaTh/PYrjFx+QnJZJRGxqju+3beBMx2YuVC6XP11k1Wo1a9eu5PvvF1G8uBX+/jPx8upToA5Ov4iEhBCiyLv3KJmLt2I4fTmCmMR07XI7G0s8mlSkTuUyuLmWeeUD0C+iUqk4f/4c7u5vM378ZBwdnfJ9G/okISGEKLLuR6Xw3fY/iE16ol1mBvh0rY1b5TKUs7fRy3YzMjJYsWIZvXu/Q4UKFbUN+QojCQkhRJGTpdawat81Tl6KAKC8vQ1Du9WhhkspVHqe5rl48QL+/l8TGnoHO7tSDB48pNAGBEhICCGKiPjkJ9wMT+D3G1FcvBVDljq7G/SYdxryRs2yet9+aupjFi1awObNGyhf3pmlS3+kdWt3vW9X3yQkhBCFWmaWmg/nHc+xzP2NitSqaEcLt3L5cmaSLn78cRmbN2/g3XcH8dlnn1OihHEa8uU3CQkhRKGkKApnQyJZvicEAOvi5gzuXJt6Ve2pXsXBIPeTSEpK/KshX1WGDx/F22+35403muh9u4YkISGEKDSy1Nmnr+4LCuN+dAoA5iozurWqTB/3agY9rfTw4YPMmjUNR0cnNm3KbshX1AICJCSEEIVAVHwqh3+/z+Hz97XLrIub08KtHIM61dLeqMcQYmKimT17GocPH6JOnbr4+c0oNNc8vAoJCSFEgaRRFH67FsXBc2HcfZQ9dVTZqSRurmXo2LQSDqUMf8bQjRvXGTlyCOnpaYwZ8298fIYVuIZ8+U1CQghRoATfjOaX4AdcDY3TLithZcG/+jeiRsVSRqkpKysLCwsLqlWrTseOnXn//WG4ulYzSi2GJiEhhDCq5NQMjl18SFR8KmeuRKJRsk9dbeHmhFuVMjSo5oC9nXGuM9BoNGzZsoFNm9azfv027OzsmDJlmlFqMRYJCSGEwSmKwpXQOE5diuC361Ha5TVdSlHS2hKvNlWpUj5/eie9qtDQO/j7f83Fixdo3botT56kAwW/IV9+k5AQQhhMeFQKu0+F8vvNaO2yyuVK0qGpC2/WK2/QA9C5UavVrFr1Iz/8sARraxumTZuNp2evIn1w+kUkJIQQeqNRFK7ciePin9GcuxZF6pMs7fda1s0+M6mkdcE68KtSqQgO/p233/ZgwoTJODjo/2rtgkxCQgiRr9QaDb/8/oDfbkRx636idnmFsiWo4VIKjyYuNKhmX6A+maenp7NixTL69OlHxYouhbohX36TkBBC5Jv45CeMW3Ja+7h+NXtqVChFx2Yu2OihDXd+CA7+HT+/Sdy7dxd7e3u8vd+XgPgbCQkhxGuLT37C0eAHBP56FwAXxxJMGNwU6+IF9y3m8eMUFi78hi1bNlKhQkWWLVtJq1atjV1WgVNwf4JCiAIv6Oojbe+kp/q8VY2erV2NU1AerFjxA1u3bmLQoCF89tm/sLbWz70lCjsJCSFEniU9zmD1/utcvBUDQF3XMrSuX57GNR0L9N5DQkI8CQnxuLpWY/jwUbRv34GGDd8wdlkFWsH9aQohCpykxxnsOh3K0QsPAKjqbMvgzrWp6lywrx9QFEXbkM/JqZy2IZ8ExMtJSAghXiojU83Mdb8TFpXdedWpjDWDO9eiflUHI1f2ctHRUcyaNY1ffvmZunXr4etbtBvy5TeDhURoaCjjx48nISGB0qVLExAQgKura451YmNjmTBhAhEREWRlZdGyZUu+/vprLCwky4QwNLVGQ2hEMofOhXH+xv8ufvu8f0MaVi8c1w7cuHGdDz54n4yMJ/zrX1/g4zNU3k/yyGD/W76+vnh7e9OrVy927drFlClTWLt2bY51li1bRvXq1Vm+fDmZmZl4e3tz6NAhunfvbqgyhTB5MQlpBJ65y4k/IrTLbG0s8XzTlY7NXArFp/DMzEwAqlWrTpcu3fDxGUqVKlWNXFXhZJCQiI2NJSQkhFWrVgHg6enJtGnTiIuLw97eXruemZkZjx8/RqPRkJGRQWZmJuXKlTNEiUKYtD9uxXD49/s5Oq8CtG3oTNcWlalQtoSRKssbtVrN5s0b2LZtI2vXbsHOrhRff+1v7LIKNYOEREREBOXKlcPc3BwAc3NznJyciIiIyBESn3zyCaNHj6Zt27akpaUxaNAgmjZtmqdtOTi8+n1lHR2N21DMGGTMpiG3Mac/yWLG6nNc/KuXUpXytlR3KU27Ji40qumISlXw9xqeunnzJuPGjePChQt4eHhga1vM5H7W+hhvgZqcO3DgALVr12bNmjU8fvyYkSNHcuDAAbp27arza8TGpqDRKHnetqOjrUHuiVuQyJhNw/PGfOtBIjtP3iHkbrx22bQPWlLxb3sMsbEpBqvxdajVav7znx/48cfvKVGiBDNmzGXIkPeIiUkxqZ/1q/5uq1RmL/xwbZCQcHZ2JjIyErVajbm5OWq1mqioKJydnXOst379embOnIlKpcLW1hYPDw/Onj2bp5AQQuQuPSOLWesvEB71vwB4t0NNOjZ1KVR7DX+nUqm4dOkPOnToxP/93yTs7R0KxXGTwsIgIeHg4ICbmxuBgYH06tWLwMBA3Nzcckw1Abi4uHDixAkaNmxIRkYGZ86coVOnToYoUYgiS61R2HvmLsF/xnDnYZJ2+fhBTahVqbTxCnsN6enpLF++lHfeGfBXQ76FFC9e3NhlFUkGm27y8/Nj/PjxLF26FDs7OwICAgAYOXIkY8aMoUGDBkycOBFfX1969uyJWq2mZcuWDBgwwFAlClGkPIhOYeW+a4RG/G8Kwq1KGdyqlMGzELTNyM358+fw959MePg9HB0dee89HwkIPTJTFCXvE/gFmByT0J2MuWh6kqFm56k7HDwXrl3m0aQiA9rXoJiluRErez0pKSl89908tm3bjItLJaZMmUaLFq2eu64p/Jz/qVAfkxBC6F9SagZr9l8n+M8Y7bL3OtTEu3vdIvGG+Z///MCOHVvx8RnKJ5+MkYZ8BiIhIUQhl6XWsOf0Xfb81abbqbQ1HZq60K5xRSwtjH870NcRHx9PfHwc1apVZ8SID/Hw6EiDBo2MXZZJkZAQopBKSs3g+/9e4UZ4gnZZ+yYV8elc24hV5Q9FUTh4cB8BAdMpV86ZTZt2ULJkSQkII9A5JG7fvs2BAweIiYnB19eX27dvk5mZSZ06dfRZnxDiHxJSnrD16C2CrkZql/V9qxpdW1bGwrxw7zkAREZGMmuWP8eO/UL9+g3x85sup7QakU6/Ufv372fw4MFERkaya9cuAFJTU5k9e7ZeixNC5HTmyiP+vfi0NiC82riycrwHnq1di0RAXL9+jXfe6UFQ0K+MG/cVa9ZsokaNWsYuy6TptCexcOFCVq1aRZ06ddi/fz8AderU4fr163otTgiR7VFcKhOXB2kfd25eiQEeNVAVkU/YmZmZWFpaUr16Dbp168n77w+jUqXKxi5LoGNIxMXFUbt29jzn090+MzMz2QUUQo+y1Bo2/HyT62EJRMalapdPHtKswN/kR1dqtZqNG9eyZctGNm7cjp1dKSZN8jV2WeJvdAqJevXqsWvXLnr37q1dtnfvXho2bKi3woQwZUcv3GfdoZsA2JUoRocmLjSq4UD9agX/Jj+6unXrJn5+X3PlyiXeequdtr23KFh0ColJkyYxYsQItm/fTmpqKiNGjCA0NJSVK1fquz4hTIaiKPx+I5p1h26QnJr9htm5eSXe7VDTyJXlL7VazY8/fs+KFT9ga1uS2bPn06VLd5mZKKB0Conq1auzf/9+jh49Srt27XB2dqZdu3aUKFE4eswLUZBpNApbj97i0G/hOZZ/81kbSpcseu0mVCoVV69eplOnLnz55cRneriJgkWnkJg+fTpff/31M3eImzFjBpMmTdJLYUIUdUmpGew8Gcqx4AfaZbVcSjG6X0NKWFkasbL8l5aWxrJli+nf/11cXCoxf/4iihUrZuyyhA50Omfup59+eu7y3bt352sxQhR1iqJw7lok4384w+cLT2kDolal0vz4f+0YP7hpkQuI334Lon9/L9as+Q+nTp0AkIAoRF64J7F9+3Ygew7x6ddPhYeHU7p04WwzLIQhpaZnEhQSyZ2HSdwMTyAmMR2AMrbF6d6qCh5NKhbJ+fjk5GQWLJjDTz9to1KlKqxYsZZmzVoYuyyRRy8MiacXzmVmZmq/huzTX8uWLatt9y2EeFbwzWjWHLhOUur/ztqxMDejd9uquDeqQBnbone84e9WrlzOzp07GDJkBB999BnW1tbGLkm8gheGxLp16wBYsGABY8eONUhBQhR2KWmZzFz3O4/+urbBzCz77m9t6pfHpohNJf1TXFwcCQnx2oZ8HTt2pl69BsYuS7wGnQ5c/z0gFEXh77egUKkKfysAIV5XllrDH7diuRmewM/ns89SsjBXMffjNylVBM9Q+idFUdi/P5A5c2ZQvnwFbUM+CYjCT6eQiIyMZOrUqZw/f56kpKQc37t27ZpeChOiMNAoCst2XeX89SjtssrlSlK3ij0DPGoYsTLDefQoghkz/Dh58jgNGjTC11ca8hUlOoWEr68vVlZWrF69msGDB7NhwwYWLVrE22+/re/6hCiQ0jOy2HzkFkFXH5GRpQGgW6vKtH+jImVLm87c+/XrIYwY4YNareGLLybw3nuDMTcvvHe/E8/SKSSCg4M5evQoNjY2mJmZUadOHWbMmMG7774r96AWJiUyLpXF/73Mo9hU1BqF4pbmdGzmgmdrV+xsTOe0zszMDCwti1G9ek08PXvh4zMMF5dKxi5L6IFOIaFSqbCwyF7Vzs6OuLg4SpYsSWRk5EueKUTRcCMsnoCNwTmWfdSrHi3cyhmpIuPIyspi/fo1bN26kU2bdlCqVGkmTJhi7LKEHukUEo0aNeL48eN06tSJtm3b8vnnn2NlZUX9+vX1XZ8QRvMkQ83xiw/Y8+tdHqdnAWBvV5xh3dyoV9X0WkncvHkDP79JhIRcoX37jqjVamOXJAxAp5CYM2cOGk32vOvEiRNZuXIljx8/ZsiQIXotTghjSM/IYtepUA6e+18vJevi5kwd3hKHUlZGrMw41Go1P/ywhJUrl2NnV4o5c76lU6cucnDaRLw0JNRqNTNmzGDatGkAWFlZ8cknn+i9MCEMTaMo7D4Vyv6zYWRmaajoWIK3G1WgVb3ylLQu2tc3vIhKpeL69RC6du3BF1+Mp3TpMsYuSRjQS0PC3Nyc06dPy6cGUWRlZqnZHxTGzlOhANjaWPKBZ12a1XY02d/7tLRUvv9+EQMGeP/VkG8hlpamc2Be/I9O001Dhgxh0aJFjB49GktL0/1EJYqWzCw1aw/e4PTlR9plTmWsmTaiBZYWpnsaZ1DQr0ybNoUHD+5TsWIlBg70loAwYTqFxPr164mJiWHVqlXY29vn+HR17NgxfdUmhN5ExqUy4W/3jB7Vsy7N3ZwwN+EOAklJSXzzTQA7d+6gcuUqrFy5niZNmhm7LGFkOoXE3Llz9V2HEAaj1mi0AdG2gTODO9eimKXp7jk8tWrVj+zZs5Phw0cyatSnWFmZ3kF68SydQqJFC2nvKwq/K3di2b/5ItfuxgFQtpQVw3u4Gbkq44qNjSE+Pp4aNWoyYsSHdO7cFTe3esYuSxQgOoWEEIVZ2pMs5m+5yJ2H2X3HLMzNaFbbiQ886xq5MuNRFIXAwF3MnTuLChUqahvySUCIf5KQEEXayn3XOH0pAgWoUt6WQV3rUKO8rbHLMqqIiIdMn+7L6dMnadSoMX5+M0z2LC7xcgYLidDQUMaPH09CQgKlS5cmICAAV1fXZ9bbt28f33//PYqiYGZmxqpVqyhbtqyhyhRFhEaj8OG8Y6g12W3tfbrUpn3jijg62hIdnWzk6ozn2rWrjBjhg6LAV19NYuDAQdLuX7xQnkJCo9EQExODk5NTnjfk6+uLt7c3vXr1YteuXUyZMoW1a9fmWOfy5cssXryYNWvW4OjoSHJystwLV+RZekYWczcFo9YomAELP3cvcveNzquMjAyKFStGzZq16dXrHQYPHkLFii7GLksUAjp9hEhKSmLcuHE0bNiQzp07A3DkyBEWLFig00ZiY2MJCQnB09MTAE9PT0JCQoiLi8ux3urVqxk+fDiOjo4A2NraUrx40b9hi8g/F25G88k3JwiNSMaptDU/fNnOpAMiKyuLxYsX06tXVxITE7CwsOCrryZJQAid6RQSvr6+lCxZkl9++UV7MV3jxo3Zv3+/ThuJiIigXLly2j7z5ubmODk5ERERkWO927dvEx4ezqBBg+jTpw9Lly7NcRc8IZ4nM0vDzpN3GL/sDIt/ugzAex1qMvujN7EwN92plOvXr+HjM4BZs2ZRt2491GqNsUsShZBO001nzpzh5MmTWFpaag9w2dvbExsbm6/FqNVqbty4wapVq8jIyOCDDz6gQoUK9O7dW+fXcHAo+crbd3Q0vQOahX3Mh8/d47stFwEoXswcV2c7PuzTgPrVcz+OVdjH/DJqtZp58+axdOlSypQpw/Lly+nRo4exyzK4ov5zfh59jFmnkLC1tSU+Pj7HsYiHDx9qp4VextnZmcjISNRqNebm5qjVaqKionB2ds6xXoUKFejatSvFihWjWLFidOjQgUuXLuUpJGJjU9Bo8r73YYoHNAv7mC/djuG7bZcAaFyzLJ/1baD9EJPbuAr7mHWhKAp//HGZbt08+eKL8dSoUanIj/mfTOHn/E+vOmaVyuyFH6512hfv378/Y8aMISgoCI1GQ3BwMF999RXvvvuuTkU4ODjg5uZGYGAgAIGBgbi5uWFvn7Mnv6enJ6dOnUJRFDIzMwkKCqJOnTo6bUOYDkVR2Hb0Ft/+FRCf9K7P6HcamvRpnKmpj5k7dxbh4WGYmZkxb95Cpk2bTalSpY1dmijkdNqTGDlyJMWLF2fq1KlkZWUxceJEBg4cmKf7Sfj5+TF+/HiWLl2KnZ0dAQEB2tceM2YMDRo0oEePHly5coXu3bujUqlo27Yt/fr1e7WRiSIpLDKZORuDSX2SfROgr7wbU7uyabeu/vXXk0yb5sujRxG4ulalUqXK0ohT5BszpYgdGZbpJt0VpjE/yVDz/a4rXLqdfRyseR0nvDvVolSJvJ0iXZjG/DKJiQnMmzebPXt2UrVqNaZMmU7jxk2eWa8ojVlXMmbdvWy6Sac9CS8vL7y8vPD09KR8+fJ5LkKI17F891WCQrLvp25T3IL/825M5XKmd1Dyn1av/g/79u3hgw8+YuTIj+V0caEXOoXE6NGjCQwMZMmSJdSrVw9PT0+6du1K6dIy3yn0R63RMOGHIGIS0wHo2rIy/dtVN+ljDzEx0SQkxFOjRi0++OBDunTpTp06pt2kUOiXTiHRqVMnOnXqREpKCj///DOBgYHMnj2bVq1asWzZMn3XKEzMkww1/z15h0O//e8e0wtGt83z1FJRoigKu3f/l/nzA7QN+UqUKCkBIfQuT205SpYsiaenJ7a2tmRmZnLixAl91SVM1M3wBC8Ut9EAACAASURBVOZtDiZLnX1cya1KGT7v3whLC9O9KO7Bg/tMn+7LmTOnady4Kb6+00x6b0oYlk4hoSgKQUFB7Nmzh8OHD1OhQgU8PT21ZygJ8brUGg37g8L46cQdAN5qVIEhXWub/JvhtWtXGT7cBzMzmDBhCv37vysN+YRB6RQS7u7u2NjY0L17dzZt2kT16tX1XZcwEbGJ6Rw4G8aRC/e1y0Z61uXN+qZ9gsSTJ08oXrw4NWvWpm/f/gwa9D4VKlQ0dlnCBOkUEkuXLqVhw4b6rkWYmPWHbvDLhQdAdksNj8YV6dnGFatipnubk8zMTFav/g87dmxh8+afKF26DF9+OcHYZQkTlutf4/3793Fxye4UWaZMGcLDw5+7XqVKlfRTmSiyFEUhYMMFbt5PBGBEDzda1y8vU0vXruLrO4mbN6/TuXM3itYVTKKwyjUkevbsSXBwMJB9dpOZmdkzHVnNzMy4du2afisURUZ6RhZTV5/nUVyqdtncj1vjUMrKiFUZn1qtZvHib1m7diX29vZ8881iPDw6GrssIYAXhMTTgAC4fv26QYoRRdeBs2FsPXpL+7iPe1U6t6hMcUtzI1ZVMKhUKkJDb+Pl1YexY/8POzs7Y5ckhJZOp0lMnz79uctnzJiRr8WIokVRFK7ejePTBce1AeHVxpUVX7WnZ5uqJh0Qjx+nEBAwg7Cwe5iZmTF37nf4+k6XgBAFjk4h8dNPPz13+e7du/O1GFF0KIrC2oM3mL/5ImlP1BS3NGf2h63o7V4NlYkfezh16gTvvNOTzZvXc+7cGQBpyCcKrBeeRrJ9+3Yge8706ddPhYeHS1sO8VwZmWq+XnGWmMR0ilmqmOTTDBfHEiZ/YDohIZ5582YTGLiLatVqsHr1Rho1amzssoR4oReGxK5du4Ds0/Kefg3ZB6zLli0rF9OJHNQaDesO3uDM1UgyszTUr2rP6HcamvTV0n+3Zs1KDhzYy6hRn/DBBx9RrJjpthkRhccLQ2LdunUALFiwgLFjxxqkIFE4XQ2N49ttf6DWKJSwsmBQp1q81aiCscsyuqioSBITE6hZszYffPAh3bp5UqtWbWOXJYTOcg0JRVG00wP/+te/0GiefxN1aRFg2mIT0/ny+1+1j7u1rEz/9jWMWFHBoCgKO3fuYP78AFxcKmkb8klAiMIm15Bo2rQpFy5cAKBu3brPzCc/DRG5TsJ07Tkdyn9Phmofz/7oTZxKWxuxooLh/v1wpk2bwtmzZ2jatDm+vtNN/niMKLxyDYm9e/dqvz5y5IhBihGFx7JdVzh3LQqAod3qyNTSX65du8qwYYMxN1fx9dd+9O07QPa2RaGWa0g4Oztrv65YMWdjsfT0dFQqlRx4M1HfbL3IlTtxACz8lzslreX0zb835Bsw4F28vd+nfHnnlz9RiAJOp484AQEBXLp0CYBjx47RokULmjdvzi+//KLX4kTBM3Pd79qA+HZ0W5MPiMzMDJYvX4qXVxcSEuKxsLDg3//+SgJCFBk6hcSePXuoWbMmAEuWLGHu3Ll8//33LFiwQK/FiYIjS61h3aEb3HqQiLnKjG9Ht8XOhO8UB3DlymW8vfuxdOlCGjduYuxyhNALnXoyp6WlYW1tTXx8POHh4XTp0gWABw8e6LU4UTCkpGUy5ruTANjZWOI3vIVJB4RarWbhwm9Yt24VDg5l+fbbpbRr52HssoTQC51CwtXVld27dxMWFkabNm0AiIuLw8rKtLt3moLV+69z4o+HANjaWLJgdFuTP1NHpVIRHh5G797vMHbs/2Fra2vskoTQG51CwtfXl5kzZ2JhYcHMmTMBOHXqlDYwRNG0ct81Tl2KAKBz80q826GmkSsynuTkZBYvXoC39/tUqeLKnDkLsLAw3ZsjCdOh0295w4YN2bx5c45lXl5eeHl56aUoYVyKojBz/e/cfpBEMQsV8z9rQwkr0z1AfeLEMaZP9yUmJpqaNWtTpYqrBIQwGTr/pp89e5adO3cSFRWFk5MTvXr1olWrVvqsTRhBWGQyfqt+0z6e9eGbJhsQcXFxzJ07k/37A6lRoybz5y+iQQO5ja8wLTqd3bRt2zY+//xzHB0d6dSpE05OTowbN46tW7fquz5hQPHJT7QB0bVlZf7zVXvK2BY3clXGs27dKn7++SAfffQZmzbtkIAQJkmnPYkVK1awatUq6tSpo13WrVs3xowZw4ABA/RWnDCc8KgUfFeeA6BDExcGmGj/pcjI7IZ8tWrVZuTIj+jRoyc1atQydllCGI1OexIJCQlUr149x7Jq1aqRmJiol6KEYcUmpmsDonPzSgzqbHpvioqisGPHVt55pwdTpkxAURRsbEpIQAiTp1NINGnShNmzZ5OWlgZAamoqc+bMoXFjuWFKYRcakcSE5UEAvNuhpkmewRQeHsaoUUOZNm0Kbm71mDv3W5M/zVeIp3SabvL392fs2LE0a9aMUqVKkZiYSOPGjZk/f76+6xN6dPqPh8xeex6A9zrUpFPzSkauyPBCQq4wfPhgLCwsmDx5Kn379peAEOJvdAoJJycnNmzYwKNHj7RnN5UvXz5PGwoNDWX8+PEkJCRQunRpAgICcHV1fe66d+7coU+fPnh7e/PVV1/laTtCN5Fxqcxem32QevygJtSqZFq3ok1PT8fKyopateowcOAgvL3fp1y5csYuS4gCR+cexklJSZw7d077LykpKU8b8vX1xdvbm4MHD+Lt7c2UKVOeu55arcbX15eOHTvm6fWF7n67HqWdYurjXtWkAuLJkyd8//0ivLy6EB+f3ZBv7NgvJSCEyIVOIXHmzBk8PDxYt24dly9fZv369Xh4eHDmzBmdNhIbG0tISAienp4AeHp6EhISQlxc3DPrLl++nHbt2uW6lyFeT1xSOt/vvALAyN716dmmqpErMpzLl/+gW7du/PDDEpo3b4lKJdNKQryMTtNN06ZNY+rUqXTv3l27bP/+/fj7+3PgwIGXPj8iIoJy5cphbm4OgLm5OU5OTkRERGBvb69d7/r165w6dYq1a9eydOnSvI4FAAeHkq/0PABHx6Ldgyc8MpmJP54FoPfb1fFyr/6SZxQNWVlZTJ8+nRUrVlC+fHnWrFljcnuqRf13+3lkzPlDp5CIiorSdn59qlOnTkyePDnfCsnMzGTy5MnMmjVLGyavIjY2BY1GyfPzHB1tiY5OfuXtFnQX/4xh4Y7se4K0aVAerzerABTpMT+lKAp37tylX793mTbNl/R00xj3U0X9d/t5ZMy6U6nMXvjhWqeQ6NWrFxs2bOD999/XLtu0aRO9e/fWqQhnZ2ciIyNRq9WYm5ujVquJiorKcfe76OhowsLCGDVqFJB9DERRFFJSUpg2bZpO2xHPd+i3cDYf+ROAHm9W4Z23i/4eRFJSEosWLWDw4CF/NeT7FnNzc2xtbUlPN603DyFeh04hERISwubNm1mxYgXlypUjMjKSuLg4GjZsyKBBg7Trbdiw4bnPd3BwwM3NjcDAQHr16kVgYCBubm45ppoqVKjA2bNntY8XLVpEamqqnN30ms6GRGoDwm9YcyqXK/q74MeO/cKMGX7Exsbg5laXKlVcX2vvVAhTplNIDBgw4LXbb/j5+TF+/HiWLl2KnZ0dAQEBAIwcOZIxY8bQoEGD13p98azExxn8sPsqAHM+fpOypayNXJF+xcXFEhAwg4MH91GzZi2+/XYJ9erJ75UQr8NMUZS8T+AXYHJMIltiyhPGLj4NQP921enWqsoz6xS1MX/33XzWr1/NqFGfMHToCCwtn717XlEbsy5kzKbBqMckROHy+41olu/J3oNwb+j83IAoKh49iiAhIYE6ddwYNepjPD17Ub26aTYnFEIfJCSKmN2nQ9l5MhSAj3rVo4Vb0bxITKPRsH37Fr77bh6VK1dh48YdWFvbSEAIkc8kJIqQoJBH2oD4yrsxtSuXMXJF+nHv3l2mTv2a338/T8uWrZkyZar0WxJCTyQkiojtx26zL+geAP8e2KjIBsTVq5cZPnwwlpbF8PObQa9efSUghNAjndpyZGRksGDBAjp06EDTpk0BOHXqFOvXr9drcUI3mw7/qQ2I8YOaUL+qg5Eryn9P29TXqVOXQYPe56efAund+x0JCCH0TKeQmDlzJjdv3mTevHnaP8qaNWuyadMmvRYnXu7inzH8fD4cgBkjWxa5Zn0ZGRksWfKdtiGfubk5Y8aMw8mpaB5rEaKg0Wm66fDhwxw6dAgbGxtUquxceXpRnTAejUbRttqYPKQZzg4ljFxR/vrjj2D8/b/mzp3beHr2koZ8QhiBTiFhaWmJWq3OsSwuLo7SpYvWp9bC5oul2ddBtG3oTFVnOyNXk3+ysrL45ps5bNq0jvLlnVmy5EfatHE3dllCmCSdppu6du3KV199RXh49rRGVFQUU6dOpUePHnotTuRu7YHrJKRkADCsWx0jV5O/LCwsiIp6xIAB77F9+24JCCGMSKeQGDt2LC4uLnh5eZGUlESXLl1wcnLi008/1Xd94jnGLjrFsYsPAVgwum2ROHiblJTItGlTuHv3DgABAQuYMGEKJUq8eut3IcTr02m6qVixYkycOJGJEycSFxdHmTJlisQbU2GTmaXms29PkpmlAWD+p20oVeLZ1hOFzS+//MzMmVOJj4+jXr0GuLpWk4Z8QhQQOoXE02mmpx4/fqz9ulKlSvlbkcjVop8uk5mloaS1JbM/bIWNlaWxS3otMTHRzJ49ncOHD1K7thuLFi3Dza2escsSQvyNTiHRqVMnzMzM+HsvwKd7EteuXdNPZUJLoygs3nGZK3fiKGNbnPmftjF2Sfliw4a1nDhxlNGjx/L++8OxtCzcoSdEUaRTSFy/fj3H4+joaBYvXkyzZs30UpTI6eP5x7VTTFOGNjdyNa8nIuIhiYkJ1KlTl1GjPsbLqw9Vq1YzdllCiFzodOD6nxwdHZk0aRLffPNNftcj/mHvmbvagFj+ZbtCewxCo9GwefMG3nnHE3//r1EUBWtrGwkIIQq4V+7ddOfOHW2rBKEf569HseN49tk+cz5+EwvzV8p0o7t79w7+/pMJDv6dN99sw9df+8uJD0IUEjqFhLe3d44/6rS0NG7duiWnwOpRTEIaS3deAWDqiBaF9q5yV65cZvjwQVhZWTN16ix69uwtASFEIaJTSPTv3z/HY2tra+rUqYOrq6s+ajJ5WWoNvqt+A6C3e1VcHAvftQJpaalYW9vg5lYXH59hvPfeYMqWdTR2WUKIPHppSKjVaoKCgpg2bRrFihXO+fDCJCNTzUfzjwNQq1JpvNpUNXJFefPkyROWL1/K7t0/sWXLLuzt7Rk9eqyxyxJCvKKXhoS5uTmnT5+WKQIDSE7NwO+vPYiypaz4yruxkSvKm+DgC/j7T+Lu3VC8vPpgYSEXxAlR2Ok03TRkyBAWLVrE6NGj5Vx2Pbl4K4aF27M7uvZ2r1qo9iCysrKYN28WW7ZspHx5Z5YuXUHr1m2NXZYQIh+8MCQCAwPx9PRk/fr1xMTEsGrVKuzt7XPsVRw7dkzfNRZ5wX9Gs2jHZQAGtK9B15aVjVxR3lhYWBAbG8u77w5m9OjPsbEpWi3LhTBlLwyJKVOm4Onpydy5cw1Vj0laf+gmAKP7NqBxrcJxcDcxMYFvv53PkCHDcHWtRkDAN9p7jQghio4XhsTTNhwtWrQwSDGmRqNRmLHud+KTn2CuMis0AXH48EFmzZpGYmICjRq9gatrNQkIIYqoF4aERqMhKCgoR8+mf3rzzTfzvShTkJyawdxNF7kfnYK5yoyv3y/4LU6io6OYPXsaR478jJtbXZYs+ZE6ddyMXZYQQo9eGBIZGRlMmjQp15AwMzPjyJEjeimsqDt64QH3o1Po1rIyfd6qViiupt64cR0nTx7nX/8ah4/PMCwsXvmCfSFEIfHCv3Jra2sJAT1IfJzBkQv3AejfvoaRq3mxBw/uk5ycpG3I17t3X6pUKTxnXgkhXk/B//haxETGpTJ20SmSUzN5q5GzscvJlVqtZuPGtfTr58XUqZO1DfkkIIQwLToduBb5Q6MoTFgeBED9qvYM7VYw5/Pv3LmNv//X/PFHMG3auEtDPiFM2AtDIjg42FB1mIQLN6IBKFfGmn8PfMPI1TzflSuXGTbMGxsbG2bMmEP37j0lIIQwYQY78hgaGsr48eNJSEigdOnSBAQEPNMgcMmSJezbtw+VSoWlpSVjx47F3d3dUCXq3dajtwAYO6CRkSt51uPHKZQoURI3t7oMGzaSd98dhL29g7HLEkIYmcGOSfj6+uLt7c3Bgwfx9vZmypQpz6zTsGFDtm/fzp49e5g5cyZjx44lPT3dUCXq1a9XIohJTKeqsx1OZWyMXY5Weno63303Hy+vrsTFxWJubs4nn4yRgBBCAAYKidjYWEJCQvD09ATA09OTkJAQ4uLicqzn7u6OtXX2fRNq166NoigkJCQYokS92/Bz9lXVn/VtYORK/icoKIgBA3qxatWPvPVWOywspC+XECIng0w3RUREUK5cOczNs7uCmpub4+TkREREBPb29s99zs6dO6lcuTLly5c3RIl6NW3Nb6Q9UeNY2ooytsWNXQ5ZWVnMmTOTrVs3UrGiCz/8sIqWLeWiSCHEswrk1VDnzp3ju+++Y+XKlXl+roPDq9+gx9HR9pWfm5u0J1mERiQDMGf0W5QtXTDuMJeWlswHH3zAV199hY1NwZn+MgR9/JwLOhmzadDHmA0SEs7OzkRGRqJWqzE3N0etVhMVFYWz87PXCQQHB/Pll1+ydOlSqlWrludtxcamoNHk/dRdR0dboqOT8/y8l/nP3hAgu7urkpmll23oIiEhngUL5jJ06AdUrVoNf/8AypUrRXR0Mo8fG6cmY9DXz7kgkzGbhlcds0pl9sIP1wY5JuHg4ICbmxuBgYFAdgtyNze3Z6aaLl26xNixY1m4cCH16tUzRGl699v1KAC6tKhklO0risLBg/vp06cHe/fu5tKliwDSkE8IoRODTTf5+fkxfvx4li5dip2dHQEBAQCMHDmSMWPG0KBBA/z9/UlPT89x5tOcOXOoXbu2ocrMV3vP3CUjU0P1CnZGudYgKiqSmTOncuzYEerWrc8PP6yiVq3C+X8phDAOg4VE9erV2bZt2zPLf/zxR+3XO3bsMFQ5epeZpWF/UBhgvOsiNm9ez5kzpxg79ksGDRoiDfmEEHkm7xp6MmfjBVKfZNG+cUVsrAx3aun9++EkJSVSt259Ro78hN69+1G5chWDbV8IUbTIxLQe/HLhPrcfJmFd3ILBnWsZZJtqtZr169fQr58X06f7/tWQz1oCQgjxWmRPIp+lPcnS3o505siWBjkWcevWn/j7f83ly3/g7v62NOQTQuQbCYl8pNEoTF1zHsg+5bVUSf1fOHf58iWGDRuErW1JZs2aR9euPSQghBD5RkIiH+04fpvIuFQ6NnPR+ymvTxvy1a1bjxEjRjFw4KBcr14XQohXJcck8sn561HsPxtGqRLF6N+uut4+zaelpfHNN3Po2bOLtiHfxx+PloAQQuiF7EnkkzUHrgPZDfwsLcz1so3ffjvL1KmTCQ8Po2/f/lhaFtPLdoQQ4ikJidekURTmbQrmcXoWreqWo3rFUvm+jaysLGbNmsqOHVtxcanE8uWradGiVb5vRwgh/klC4jVFxqVyPSy7nfkHnnX1sg0LCwseP07Bx2con3zyL207dSGE0DcJide06cifAIx5pyEqVf4dh4iLi+ObbwIYMeJDqlatxsyZ86TfkhDC4ORd5zVExqdy5U72jZMa1sifO7kpisL+/YG8804PDhzYx5UrlwBpyCeEMA7Zk3gNu06FAjCsWx1U+XA2U2TkI2bM8OPEiWPUr98QP7/p1KhhmCu2hRDieSQkXsPDmMcAuDeqkC+vt2XLRs6dC2LcuPF4e/to7+QnhBDGIiHxitIzsgiLTMHBzuq1Xics7B7JyUnUq9eAkSM/pk+fflSqVDmfqhRCiNcjE92vaM/puwB4NK34Ss9Xq9WsXbuSAQN6MWOGn7YhnwSEEKIgkT2JVxCTmMb+s9n3iujaIu9v6rdu3cTXdxJXr17m7bfbM3Gin/RbEkIUSBISr2DVvuyrq73auOb5zf1/DflsCQj4hs6du0lACCEKLAmJPFJrNFy7Fw9Ab/dqOj8vOTkZW1tb6tWrz8iRHzFggDdlypTRV5lCCJEv5JhEHv10/A4AbeqX12n9tLRU5s2bhZdXF2JjY1CpVHz44acSEEKIQkH2JPIgIeWJ9ljEex1ffv3C2bNnmDp1Mg8e3Kd///coXvz1zoQSQghDk5DQkUajMOGHICD7hkI2Vrn/12VlZTFjhj///e82KleuwooVa2nWrIWhShVCiHwjIaGjfUH3eJKppkktR7q2fPEZTRYWFqSlpTJ06Ad89NFnWFnJHoQQonCSkNDR02mmT/vUf+734+JimT8/uyFftWrVmTVrnpy1JIQo9OTAtQ6O/H6ftCdZdGzq8swbv6Io7N27h759e3Do0H5CQq4ASEAIIYoE2ZN4icSUJ2z4+SYAXf5x4dyjRxFMn+7HqVPHadiwEb6+M6hevYbhixRCCD2RkHiJL5b+CsDQbnVwKJXz2MLWrRs5f/4cX345kXffHSQN+YQQRY6ExAuE3I1DrVEAeOuvTq/37oWSlJRMgwYNGTnyE955ZyAVK7oYs0whhNAbOSaRC41GYd7miwBM8mlKVlYWq1atYMCA3sya5a9tyCcBIYQoymRPIhf/2XsNgE7NKpGVEoHPFxO5di0ED49OTJgwWQ5MCyFMgoTEc1y8FcOZq48AqFkmmUGD3sfOrhRz535Lx45dJCCEECZDQuIfgkIesXx3CFkZqcz6uB3ODjZ8+OFnDBjwLqVKlTZ2eUIIYVAGOyYRGhrKwIED6dKlCwMHDuTu3bvPrKNWq/H396djx4506tSJbdu2Gao8AKLiU/l+xwXCL2zj9s/TsFKlo1KpGDnyIwkIIYRJMlhI+Pr64u3tzcGDB/H29mbKlCnPrLNnzx7CwsI4dOgQW7ZsYdGiRdy/f98g9YU+TOQT/1VcOzCD6JtH8ezeQ9ppCCFMnkFCIjY2lpCQEDw9PQHw9PQkJCSEuLi4HOvt27eP/v37o1KpsLe3p2PHjhw4cEDv9YVHJtJrwAhuHVuEjXVxVq5cz4QJUyhRoqTety2EEAWZQY5JREREUK5cOe3FZubm5jg5OREREYG9vX2O9SpUqKB97OzszKNHj/K0LQeHvL+xh4QnotFk0aOvDwvn+pnUHoSjo62xSzA4GbNpkDHnjyJ34Do2NgXNXxfA6crNxY5zv2zjcXI6ycmZJCdn6qm6gsXR0Zbo6GRjl2FQMmbTIGPWnUpl9sIP1waZbnJ2diYyMhK1Wg1kH6COiorC2dn5mfUePnyofRwREUH58rrdAe51mJmZYWNlqfftCCFEYWOQkHBwcMDNzY3AwEAAAgMDcXNzyzHVBNC1a1e2bduGRqMhLi6Ow4cP06VLF0OUKIQQ4jkMdnaTn58f69evp0uXLqxfvx5/f38ARo4cyeXLlwHo1asXLi4udO7cmQEDBvDpp59SqVIlQ5UohBDiH8wURcnbBH4B9yrHJEDmME2FjNk0yJh1VyCOSQghhCicJCSEEELkSkJCCCFErorcdRIq1at3aH2d5xZWMmbTIGM2Da8y5pc9p8gduBZCCJF/ZLpJCCFEriQkhBBC5EpCQgghRK4kJIQQQuRKQkIIIUSuJCSEEELkSkJCCCFEriQkhBBC5EpCQgghRK5MKiRCQ0MZOHAgXbp0YeDAgdy9e/eZddRqNf7+/nTs2JFOnTqxbds2wxeaj3QZ85IlS+jRowc9e/akb9++nDx50vCF5iNdxvzUnTt3aNSoEQEBAYYrUA90HfO+ffvo2bMnnp6e9OzZk5iYGMMWmo90GXNsbCyjRo2iZ8+edOvWDT8/P7KysgxfbD4ICAjAw8OD2rVrc/Pmzeeuo5f3L8WE+Pj4KDt37lQURVF27typ+Pj4PLPOf//7X2X48OGKWq1WYmNjFXd3dyU8PNzQpeYbXcZ84sQJJTU1VVEURbl27ZrStGlTJS0tzaB15iddxqwoipKVlaUMHjxY+fe//63Mnj3bkCXmO13GfOnSJaVbt25KVFSUoiiKkpSUpKSnpxu0zvyky5inT5+u/dlmZGQo/fr1U/bu3WvQOvPLb7/9pjx8+FBp3769cuPGjeeuo4/3L5PZk4iNjSUkJARPT08APD09CQkJIS4uLsd6+/bto3///qhUKuzt7enYsSMHDhwwRsmvTdcxu7u7Y21tDUDt2rVRFIWEhASD15sfdB0zwPLly2nXrh2urq4GrjJ/6Trm1atXM3z4cBwdHQGwtbWlePHiBq83P+g6ZjMzMx4/foxGoyEjI4PMzEzKlStnjJJfW7NmzXB2dn7hOvp4/zKZkIiIiKBcuXKYm5sDYG5ujpOTExEREc+sV6FCBe1jZ2dnHj16ZNBa84uuY/67nTt3UrlyZcqXL2+oMvOVrmO+fv06p06dYujQoUaoMn/pOubbt28THh7OoEGD6NOnD0uXLkUppP09dR3zJ598QmhoKG3bttX+a9q0qTFKNgh9vH+ZTEiIlzt37hzfffcd8+fPN3YpepWZmcnkyZPx9/fXvsmYArVazY0bN1i1ahXr1q3jxIkT7Nq1y9hl6dWBAweoXbs2p06d4sSJE5w/f77QzgwYi8mEhLOzM5GRkajVaiD7DyYqKuqZ3TdnZ2cePnyofRwREVFoP1XrOmaA4OBgvvzyS5YsWUK1atUMXWq+0WXM0dHRhIWFMWrUKDw8PFizZg1bt25l8uTJxir7tej6c65QoQJdu3alWLFilCxZkg4dOnDp0iVjlPzadB3z+vXr8fLyQqVSYWtri4eHB2fPnjVGyQahj/cvkwkJBwcH3NzcCAwMBCAwMBA3NzfsfHthBQAACPVJREFU7e1zrNe1a1e2bduGRqMhLi6Ow4cP06VLF2OU/Np0HfOlS5cYO3YsCxcupF69esYoNd/oMuYKFSpw9uxZfvnlF3755ReGDBnCgAEDmDZtmrHKfi26/pw9PT05deoUiqKQmZlJUFAQderUMUbJr03XMbu4uHDixAkAMjIyOHPmDDVr1jR4vYail/ev1zrsXcjcunVL6devn9K5c2elX79+yu3btxVFUZQPPvhAuXTpkqIo2We8TJkyRenQoYPSoUMHZfPmzcYs+bXpMua+ffsqLVu2VLy8vLT/rl+/bsyyX4suY/67hQsXFvqzm3QZs1qtVmbOnKl07dpV6d69uzJz5kxFrVYbs+zXosuY7927pwwdOlTx9PRUunXrpvj5+SmZmZnGLPuVTZs2TXF3d1fc3NyU1q1bK927d1cURf/vX3JnOiGEELkymekmIYQQeSchIYQQIlcSEkIIIXIlISGEECJXEhJCCCFyJSEhigQfH59C0bF32bJlTJo0Kdfv7969m+HDhxuwIiFeTE6BFQWOh4cHMTExOVpmHDhw4IWN2Xx8fPDy8qJ///6GKDFf3L9/nw4dOnD16lUsLCyMXU4OBbk2YVjy0xcF0rJly2jdurWxyyiy1Gq1SfWtEq9OpptEoZCYmMiHH35Iq1ataN68OR9++GGu3S3v3bvH4MGDadq0KS1btuTzzz/Xfu/27dsMGzaMFi1a0KVLF/bt25frNn18fJg/fz79+vWjSZMmfPzxxzlaqB85coQePXrQrFkzfHx8uH37tvZ7y5cvx93dncaNG9OlSxfOnDkDwKJFi/jiiy8AGDx4MADNmzencePGBAcH89NPP/Hee+8B4Ovr+8zNkD7++GNWrVoFQGRkJKNHj6ZVq1Z4eHiwdu3aXMcyfvx4fH19GTlyJG+88QZnz57l2LFj9O7dmyZNmvD222+zaNEi7frPqw1g+/btdOvWjebNmzNixAgePHiQ6zZFEfHa12wLkc/at2+vnD59OseyuLg45cCBA0pqaqqSnJysjB49Wvn444+13x88eLCydetWRVEUZezYscrSpUsVtVqtpKenK7/99puiKIry+PFj5a233lK2b9+uZGZmKlevXlVatGih/Pnnn8+tY/DgwUrbtm2VGzduKI8fP1Y+++wzZdy4cYqiKMqdO3eURo0aKadOnVIyMjKU5cuXKx07dlSePHmi3L59W3nrrbeUR48eKYqi/H97dxMSVRcGcPw/H87FEsXAwtQWBRZC0djITDhWZGHWmEWWBkFYIWMFgRFJSS36WjgtzAZEynUEQVRo5aIvyLCBalyM5PQ5aOVCjRzUuUznXcQ7ZHmt4F28xvPbzXDvuc+5l5mHcw73PCoSiah3794ppb5tAfJvG5FIROXm5k7aJuLatWuqqqpKKaVUd3e3WrVqlfr69atSSqmRkRG1dOlS9fHjRxWPx9XWrVtVc3OzmpiYUO/fv1dr165VDx8+nLIvR48eVfn5+SoQCCTuy5MnT1Rvb6+Kx+MqFAqplStXqs7OTsPYOjs71bp161Q4HFa6riu/368qKyt/+TzFzCYjCfG/dODAARwOBw6Hg/3795Oenk5JSQnJycmkpKRQW1vL06dPpzzXarUyMDDA4OAgmqbhcDgAuH//PllZWWzbtg2r1UpeXh4lJSXTbh1dXl5Obm4us2bN4tChQ9y+fZt4PE57ezurV6+msLCQpKQk9u7dy/j4OM+ePcNisRCLxXj16hW6rpOdnc2CBQv++B44HA5MJhOBQACAO3fusHz5cubNm0dPTw9DQ0McPHgQm81GTk4OO3bsmHZkVFxczIoVKzCbzWiahtPpZPHixZjNZpYsWcKmTZvo7u42PP/KlSvU1NSwaNEirFYrXq+XUCgko4m/nKxJiP8lv98/aU1ibGyMc+fO8ejRIz5//gxANBqdcm79yJEjNDU1UVFRQVpaGtXV1VRUVNDf308wGEwkDfg2N79582bDOL7fenr+/Pnous7w8DCDg4OTiruYzebE9tVOp5Njx47R3NxMOBzG7XZTX1//xxXRTCYTGzdu5NatWxQUFHDz5s1ErP39/QwODv7Ul+8/T9cXgBcvXuDz+ejr60PXdWKxGBs2bDA8f2BggLNnz06aAlNK8enTJ7Kysv6ob2LmkCQhZoS2tjbevHnD1atXycjIIBQKsWXLlikrq2VkZHD69GkAAoEA1dXVFBQUkJmZSUFBQWJO/3d8X+nsw4cPJCUlkZ6ezty5cycVo1dKJaqlAZSVlVFWVsbo6CgnTpzA5/PR2Ng4qW2TyfTL63s8Hvbs2UNNTQ3BYBC/3w98+8PPzs7m7t27v92XHx0+fJhdu3Zx6dIlNE3jzJkzDA8PG8aWmZmJ1+udNqmKv49MN4kZIRqNomkaqampjIyMcPHiRcNjOzo6EovaaWlpmEwmzGYza9as4e3bt1y/fh1d19F1nWAwOGnB+Uc3btwgHA4zNjZGU1MTJSUlWCwWSktLefDgAV1dXei6TltbGzabDbvdzuvXr+nq6iIWi2Gz2dA0DbP555/anDlzMJvNRCIRw+vn5eWRnp5OQ0MDbreb1NRUAJYtW8bs2bNpbW1lfHyceDzOy5cv/6iIUDQaJS0tDU3TCAaDidoMRrFVVVXR2tpKX18fAF++fKGjo+O3rydmJkkSYkbYvXs3ExMTuFwuKisrKSoqMjy2p6eH7du3Y7fbqa2t5fjx4+Tk5JCSksLly5dpb2+nqKgIt9uNz+cjFosZtlVeXk59fT2FhYXEYrHEi3ALFy6ksbGRU6dO4XK5uHfvHi0tLdhsNmKxGOfPn8fpdOJ2uxkaGqKuru6ntpOTk/F6vezcuROHw8Hz58+njMHj8fD48WM8Hk/iO4vFQktLC729vRQXF+NyuWhoaGB0dPR3byknT57kwoUL2O12/H4/paWl08a2fv169u3bR11dHfn5+Xg8nkRBH/H3kpfphDAwE1/QE+K/JiMJIYQQhiRJCCGEMCTTTUIIIQzJSEIIIYQhSRJCCCEMSZIQQghhSJKEEEIIQ5IkhBBCGJIkIYQQwtA/f/CO/WAhckAAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nli9W6_pVqLx",
        "colab_type": "text"
      },
      "source": [
        "### Saving the Model"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "R5wBQjZSVqLx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import pickle\n",
        "\n",
        "pickle.dump(reg_lgd_st_1, open('lgd_model_stage_1.sav', 'wb'))\n",
        "# Here we export our model to a 'SAV' file with file name 'lgd_model_stage_1.sav'."
      ],
      "execution_count": 13,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wksVi3wlVqL2",
        "colab_type": "text"
      },
      "source": [
        "### Stage 2 – Linear Regression"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lQJ_GaYZVqL3",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 419
        },
        "outputId": "a25574a8-2687-41dc-9651-02b2fd03bf33"
      },
      "source": [
        "lgd_stage_2_data = loan_data_defaults[loan_data_defaults['recovery_rate_0_1'] == 1]\n",
        "# Here we take only rows where the original recovery rate variable is greater than one,\n",
        "# i.e. where the indicator variable we created is equal to 1.\n",
        "\n",
        "# LGD model stage 2 datasets: how much more than 0 is the recovery rate\n",
        "lgd_inputs_stage_2_train, lgd_inputs_stage_2_test, lgd_targets_stage_2_train, lgd_targets_stage_2_test = train_test_split(lgd_stage_2_data.drop(['good_bad', 'recovery_rate','recovery_rate_0_1', 'CCF'], axis = 1), lgd_stage_2_data['recovery_rate'], test_size = 0.2, random_state = 42)\n",
        "# Takes a set of inputs and a set of targets as arguments. Splits the inputs and the targets into four dataframes:\n",
        "# Inputs - Train, Inputs - Test, Targets - Train, Targets - Test.\n",
        "\n",
        "from sklearn import linear_model\n",
        "from sklearn.metrics import mean_squared_error, r2_score\n",
        "\n",
        "# Since the p-values are obtained through certain statistics, we need the 'stat' module from scipy.stats\n",
        "import scipy.stats as stat\n",
        "\n",
        "# Since we are using an object oriented language such as Python, we can simply define our own \n",
        "# LinearRegression class (the same one from sklearn)\n",
        "# By typing the code below we will ovewrite a part of the class with one that includes p-values\n",
        "# Here's the full source code of the ORIGINAL class: https://github.com/scikit-learn/scikit-learn/blob/7b136e9/sklearn/linear_model/base.py#L362\n",
        "\n",
        "\n",
        "class LinearRegression(linear_model.LinearRegression):\n",
        "    \"\"\"\n",
        "    LinearRegression class after sklearn's, but calculate t-statistics\n",
        "    and p-values for model coefficients (betas).\n",
        "    Additional attributes available after .fit()\n",
        "    are `t` and `p` which are of the shape (y.shape[1], X.shape[1])\n",
        "    which is (n_features, n_coefs)\n",
        "    This class sets the intercept to 0 by default, since usually we include it\n",
        "    in X.\n",
        "    \"\"\"\n",
        "    \n",
        "    # nothing changes in __init__\n",
        "    def __init__(self, fit_intercept=True, normalize=False, copy_X=True,\n",
        "                 n_jobs=1):\n",
        "        self.fit_intercept = fit_intercept\n",
        "        self.normalize = normalize\n",
        "        self.copy_X = copy_X\n",
        "        self.n_jobs = n_jobs\n",
        "\n",
        "    \n",
        "    def fit(self, X, y, n_jobs=1):\n",
        "        self = super(LinearRegression, self).fit(X, y, n_jobs)\n",
        "        \n",
        "        # Calculate SSE (sum of squared errors)\n",
        "        # and SE (standard error)\n",
        "        sse = np.sum((self.predict(X) - y) ** 2, axis=0) / float(X.shape[0] - X.shape[1])\n",
        "        se = np.array([np.sqrt(np.diagonal(sse * np.linalg.inv(np.dot(X.T, X))))])\n",
        "\n",
        "        # compute the t-statistic for each feature\n",
        "        self.t = self.coef_ / se\n",
        "        # find the p-value for each feature\n",
        "        self.p = np.squeeze(2 * (1 - stat.t.cdf(np.abs(self.t), y.shape[0] - X.shape[1])))\n",
        "        return self\n",
        "\n",
        "\n",
        "import scipy.stats as stat\n",
        "\n",
        "class LinearRegression(linear_model.LinearRegression):\n",
        "    def __init__(self, fit_intercept=True, normalize=False, copy_X=True,\n",
        "                 n_jobs=1):\n",
        "        self.fit_intercept = fit_intercept\n",
        "        self.normalize = normalize\n",
        "        self.copy_X = copy_X\n",
        "        self.n_jobs = n_jobs\n",
        "    def fit(self, X, y, n_jobs=1):\n",
        "        self = super(LinearRegression, self).fit(X, y, n_jobs)\n",
        "        sse = np.sum((self.predict(X) - y) ** 2, axis=0) / float(X.shape[0] - X.shape[1])\n",
        "        se = np.array([np.sqrt(np.diagonal(sse * np.linalg.inv(np.dot(X.T, X))))])\n",
        "        self.t = self.coef_ / se\n",
        "        self.p = np.squeeze(2 * (1 - stat.t.cdf(np.abs(self.t), y.shape[0] - X.shape[1])))\n",
        "        return self\n",
        "\n",
        "lgd_inputs_stage_2_train = lgd_inputs_stage_2_train[features_all]\n",
        "# Here we keep only the variables we need for the model.\n",
        "\n",
        "lgd_inputs_stage_2_train = lgd_inputs_stage_2_train.drop(features_reference_cat, axis = 1)\n",
        "# Here we remove the dummy variable reference categories.\n",
        "\n",
        "reg_lgd_st_2 = LinearRegression()\n",
        "# We create an instance of an object from the 'LogisticRegression' class.\n",
        "reg_lgd_st_2.fit(lgd_inputs_stage_2_train, lgd_targets_stage_2_train)\n",
        "# Estimates the coefficients of the object from the 'LogisticRegression' class\n",
        "# with inputs (independent variables) contained in the first dataframe\n",
        "# and targets (dependent variables) contained in the second dataframe.\n",
        "\n",
        "feature_name = lgd_inputs_stage_2_train.columns.values\n",
        "# Stores the names of the columns of a dataframe in a variable.\n",
        "\n",
        "summary_table = pd.DataFrame(columns = ['Feature name'], data = feature_name)\n",
        "# Creates a dataframe with a column titled 'Feature name' and row values contained in the 'feature_name' variable.\n",
        "summary_table['Coefficients'] = np.transpose(reg_lgd_st_2.coef_)\n",
        "# Creates a new column in the dataframe, called 'Coefficients',\n",
        "# with row values the transposed coefficients from the 'LogisticRegression' object.\n",
        "summary_table.index = summary_table.index + 1\n",
        "# Increases the index of every row of the dataframe with 1.\n",
        "summary_table.loc[0] = ['Intercept', reg_lgd_st_2.intercept_]\n",
        "# Assigns values of the row with index 0 of the dataframe.\n",
        "summary_table = summary_table.sort_index()\n",
        "# Sorts the dataframe by index.\n",
        "p_values = reg_lgd_st_2.p\n",
        "# We take the result of the newly added method 'p_values' and store it in a variable 'p_values'.\n",
        "p_values = np.append(np.nan,np.array(p_values))\n",
        "# We add the value 'NaN' in the beginning of the variable with p-values.\n",
        "summary_table['p_values'] = p_values.round(3)\n",
        "# In the 'summary_table' dataframe, we add a new column, called 'p_values', containing the values from the 'p_values' variable.\n",
        "summary_table\n",
        "\n",
        "summary_table = pd.DataFrame(columns = ['Feature name'], data = feature_name)\n",
        "summary_table['Coefficients'] = np.transpose(reg_lgd_st_2.coef_)\n",
        "summary_table.index = summary_table.index + 1\n",
        "summary_table.loc[0] = ['Intercept', reg_lgd_st_2.intercept_]\n",
        "summary_table = summary_table.sort_index()\n",
        "p_values = reg_lgd_st_2.p\n",
        "p_values = np.append(np.nan,np.array(p_values))\n",
        "summary_table['p_values'] = p_values.round(3)\n",
        "summary_table"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Feature name</th>\n",
              "      <th>Coefficients</th>\n",
              "      <th>p_values</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Intercept</td>\n",
              "      <td>2.406858e-01</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>grade:A</td>\n",
              "      <td>-6.826892e-02</td>\n",
              "      <td>0.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>grade:B</td>\n",
              "      <td>-5.083556e-02</td>\n",
              "      <td>0.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>grade:C</td>\n",
              "      <td>-3.748066e-02</td>\n",
              "      <td>0.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>grade:D</td>\n",
              "      <td>-2.717310e-02</td>\n",
              "      <td>0.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>40</th>\n",
              "      <td>open_acc</td>\n",
              "      <td>-1.196505e-03</td>\n",
              "      <td>0.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41</th>\n",
              "      <td>pub_rec</td>\n",
              "      <td>3.447322e-03</td>\n",
              "      <td>0.208</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>42</th>\n",
              "      <td>total_acc</td>\n",
              "      <td>4.766629e-04</td>\n",
              "      <td>0.000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>43</th>\n",
              "      <td>acc_now_delinq</td>\n",
              "      <td>4.278394e-03</td>\n",
              "      <td>0.658</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>44</th>\n",
              "      <td>total_rev_hi_lim</td>\n",
              "      <td>2.263456e-07</td>\n",
              "      <td>0.000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>45 rows × 3 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "        Feature name  Coefficients  p_values\n",
              "0          Intercept  2.406858e-01       NaN\n",
              "1            grade:A -6.826892e-02     0.000\n",
              "2            grade:B -5.083556e-02     0.000\n",
              "3            grade:C -3.748066e-02     0.000\n",
              "4            grade:D -2.717310e-02     0.000\n",
              "..               ...           ...       ...\n",
              "40          open_acc -1.196505e-03     0.000\n",
              "41           pub_rec  3.447322e-03     0.208\n",
              "42         total_acc  4.766629e-04     0.000\n",
              "43    acc_now_delinq  4.278394e-03     0.658\n",
              "44  total_rev_hi_lim  2.263456e-07     0.000\n",
              "\n",
              "[45 rows x 3 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "iAaXxlsHVqMS",
        "colab_type": "text"
      },
      "source": [
        "### Stage 2 – Linear Regression Evaluation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ug_5MzyPVqMT",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 285
        },
        "outputId": "161204b0-245c-4060-f3e6-d3082ec36bba"
      },
      "source": [
        "lgd_inputs_stage_2_test = lgd_inputs_stage_2_test[features_all]\n",
        "# Here we keep only the variables we need for the model.\n",
        "\n",
        "lgd_inputs_stage_2_test = lgd_inputs_stage_2_test.drop(features_reference_cat, axis = 1)\n",
        "# Here we remove the dummy variable reference categories.\n",
        "\n",
        "lgd_inputs_stage_2_test.columns.values\n",
        "# Calculates the predicted values for the dependent variable (targets)\n",
        "# based on the values of the independent variables (inputs) supplied as an argument.\n",
        "\n",
        "y_hat_test_lgd_stage_2 = reg_lgd_st_2.predict(lgd_inputs_stage_2_test)\n",
        "# Calculates the predicted values for the dependent variable (targets)\n",
        "# based on the values of the independent variables (inputs) supplied as an argument.\n",
        "\n",
        "lgd_targets_stage_2_test_temp = lgd_targets_stage_2_test\n",
        "\n",
        "lgd_targets_stage_2_test_temp = lgd_targets_stage_2_test_temp.reset_index(drop = True)\n",
        "# We reset the index of a dataframe.\n",
        "\n",
        "pd.concat([lgd_targets_stage_2_test_temp, pd.DataFrame(y_hat_test_lgd_stage_2)], axis = 1).corr()\n",
        "# We calculate the correlation between actual and predicted values.\n",
        "\n",
        "sns.distplot(lgd_targets_stage_2_test - y_hat_test_lgd_stage_2)\n",
        "# We plot the distribution of the residuals.\n",
        "\n",
        "pickle.dump(reg_lgd_st_2, open('lgd_model_stage_2.sav', 'wb'))\n",
        "# Here we export our model to a 'SAV' file with file name 'lgd_model_stage_1.sav'."
      ],
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAEMCAYAAADUEk3/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3hU5Z0H8O85Z+7JJCEhgXARlAKicrFEoIhYAypogNXa4lJ1xSKP6+ra7bbV3bKCQNnG+ri1Vhbpuq5uqdu6q1bAirK4UrwgWhG5IwQI5D65zv3MOe/+MZkh18mEuWby/TyPT8vkzJnfO/PMN2/e8573lYQQAkRENODJqS6AiIjig4FORJQhGOhERBmCgU5ElCEY6EREGYKBTkSUIRjoREQZwpDqApqaXND1+E6FLyjIhsPhjOs5UyFT2gFkTlsypR1A5rRlsLVDliUMGZLV489SHui6LuIe6KHzZoJMaQeQOW3JlHYAmdMWtiOIQy5ERBmCgU5ElCEY6EREGYKBTkSUIRjoREQZgoFORJQhGOhpyqdq2P5BBXQuV09EUYoq0MvLy1FaWoqJEyfi+PHjAICmpibcf//9uPnmm7Fo0SI89NBDaGxsTGixg8mnR+uw6bUDqKhqTXUpRDRARBXo8+bNw5YtWzBy5MjwY5IkYcWKFdixYwe2bt2K0aNH46mnnkpYoYNNTaMbAOBo9aa4EiIaKKIK9JKSEhQXF3d6LC8vDzNnzgz/e9q0aaiqqopvdYNYtYOBTkT9E5db/3VdxyuvvILS0tJ+P7egIDseJXRTWGhPyHmTpb4lGOQevz7g2xLCdqSfTGkL2xEUl0Bft24dbDYb7rrrrn4/1+Fwxn0dhsJCO+rr2+J6zmTSdB1V9cFFes7Vtg3otoQM9M8kJFPaAWROWwZbO2RZ6rUjHHOgl5eX48yZM9i0aRNkmZNm4qGh2Qut/ZdcI4dciChKMQX6008/jYMHD2Lz5s0wmUzxqmnQq26/IHrZyFzUOlwproaIBoqoutTr16/H3LlzUVNTg+XLl+PWW2/FiRMn8Pzzz6Ourg533nknlixZgr/5m79JdL2DQk37BdGrJxTC5Q3A6w+kuCIiGgii6qGvWrUKq1at6vb4sWPH4l4QAdUOF3JsRowdkQsAcLT6MHJoypeuJ6I0x0HvNFTT6MbwfBuKhlgBcBydiKLDQE9D1Q43hhdkoTDPBoBz0YkoOgz0NOP0qHB6VAzPtyE/xwxZkthDJ6KocGA2TQR0wKcGUFETXLtlSI4ZjhYv8uwmOFp8Ka6OiAYCBnqa8KkB7DtSixPnWgAAtY1u/PlYHfKyzeyhE1FUOOSSZlpdPsiShGyrEQCQn2PmGDoRRYWBnmZaXSrsWUbIsgQAGGK3oKnNF/flEYgo8zDQ04zbF4DNfGEkLN9uhqYLtLj8KayKiAYCBnqa8fgCsHYI9CE5FgCci05EfWOgpxEhBLw+DRaTEn5siN0MgHPRiahvDPQ0ogZ06EJ07qEz0IkoSgz0NOLxaQAAq/lCD91qNsBqVtDYyrnoRBQZAz2NhFZVtJg63x5gMxvg9XHFRSKKjIGeRjz+7j10IBjwXlVLRUlENIAw0NNIqBfetYduNinw+RnoRBQZAz2NePwaJAQDvCOzUYGXgU5EfWCgpxGvLwCzSYEsSZ0et5gY6ETUNwZ6GvH4O89BDzGbFPhUXhQlosgY6GnE2+Uu0RCLycAxdCLqEwM9jXS97T/EYlQ4y4WI+sRATxNCCHgjDLn4VZ0rLhJRRAz0NOH1a9B0AUsPPXSzMRjyPvbSiSgCBnqaaHOrAABrDz10S/uNRpzpQkSRMNDTRJs7uN5515uKgOAYOsAeOhFFxkBPE6FA73rbP3DhRiPOdCGiSPoM9PLycpSWlmLixIk4fvx4+PGKigosXboUN998M5YuXYrTp08nss6M19o+5BKphx5avIuIqCd9Bvq8efOwZcsWjBw5stPjq1evxrJly7Bjxw4sW7YMjz/+eMKKHAzaXKEhl5566MGQ5xg6EUXSZ6CXlJSguLi402MOhwOHDx9GWVkZAKCsrAyHDx9GY2NjYqocBNrcfpiNSnhz6I5CIc8xdCKKpPvf91Gorq7GsGHDoCjBoFEUBUVFRaiurkZ+fn6/zlVQkH0xJfSpsNCekPMmisunIctqhD3b0ulxm82MrKzgY0azccC1q6OBXHtHmdIOIHPawnYEXVSgx5PD4Yz7DTOFhXbU17fF9ZyJ1tzmhckoo815Yas5e7YFbrcPEMH3p8HhGnDtChmIn0lPMqUdQOa0ZbC1Q5alXjvCFzXLpbi4GLW1tdC04BCApmmoq6vrNjRD0WtzqT3OQQcu3FjEi6JEFMlFBXpBQQEmTZqEbdu2AQC2bduGSZMm9Xu4hS5oc/t7nOECAAZFhkGRuJ4LEUXU55DL+vXr8c4776ChoQHLly9HXl4etm/fjjVr1uCxxx7Dxo0bkZOTg/Ly8mTUm5G8/gD8Ab3HOeghXHGRiPrSZ6CvWrUKq1at6vb4uHHj8OqrryakqMGmxdX7XaIhZiO3oSOiyFJ+UZSAFmcw0G2W7h+HJEtw+QIwGWW4vAG42vcdNRsNMPA+XyLqgIGeBpqdPgDocS10n6rhi+P1UAM6apvc2HekFgBwzaRhMPRwPBENXuzjpYHmtmCg2yIEtNEgI6DpySqJiAYgBnoaaHb6YVAkmIy9fxwGRUZA4wYXRNQ7BnoaaHb6kJtlhiR1v+0/xGiQoQbYQyei3jHQ00Cz04ecLFPEYwyKxCEXIoqIgZ4Gmp1+5Gb3FejsoRNRZAz0NBAccokc6EaDDE0X0AXH0YmoZwz0FPP4AvD6NeRmmyMeZ1SCHxWHXYioNwz0FAvdJdpXD93QfhdRIMAeOhH1jIGeYi3tNxX1fVGUPXQiioyBnmJN7YHe10VRY3sPnRdGiag3DPQUa24LDblEN4ausodORL1goKdYs9MHk0GOuHQuABgMwZuOAuyhE1EvGOgp1uLyIy878l2iwIUxdPbQiag3DPQUa27z9Tl+DnDaIhH1jYGeYs1OH/L6mIMOXJi2yIuiRNQbBnoKCSHQ7PRHFegXeuich05EPWOgp5DXr8Gnasiz9z3kIssSZFliD52IesVAT6HQTkXR9NCBYC+dY+hE1BsGego1t+8lmtfHXaIhBkXitEUi6hUDPYXCPXR7lD10g8xpi0TUKwZ6CvV3yIVrohNRJAz0FHK0eGE1K7CYIt8lGsKNookoEgZ6CtU1eVA0xNbnXaIh3CiaiCIxxHqC9957D8888wyEEBBC4KGHHsJNN90Uj9oyXl2TB2OL7VEfz42iiSiSmAJdCIEf//jH2LJlCyZMmICjR4/iL//yLzF//nzIMjv/kQQ0HQ0tXsy4oijq5xg4bZGIIog5dWVZRltbGwCgra0NRUVFDPMoOFq90IVAUZ4t6ucYDbyxiIh6F1MPXZIk/OIXv8CDDz4Im80Gl8uFzZs39+scBQXZsZTQq8LC6IcyUuGsww0AmHBpAQoL7RCNbtizLd2OMxoN4cdtVhM0XSA7ywybzYzC/Oh/GaSDdP9MopUp7QAypy1sR1BMgR4IBPD8889j48aNmD59Oj777DN8//vfx/bt25GVlRXVORwOJ3Q9vhf6CgvtqK9vi+s54+3E6UYAgFkSqK9vg9sXQJvT2+kYe7YFqnrhca19uKWpxQO324d6TUtu0TEYCJ9JNDKlHUDmtGWwtUOWpV47wjGNjRw5cgR1dXWYPn06AGD69OmwWq04efJkLKfNaAEdcPkCON/ggskoQzHIcPkCiOZ3mkFp3+SC4+hE1IOYAn348OGoqanBqVOnAAAnT56Ew+HAJZdcEpfiMpFPDWDfkVqcONeMLIsRnx6tw74jtQjofYe0of3ahMapi0TUg5iGXAoLC7FmzRo88sgj4bnUGzZsQF5eXlyKy2RtbhV5UWxs0VG4hx5F+BPR4BPzPPTFixdj8eLF8ahl0NCFgNPtx+ii/l0QNnDXIiKKgPMLU8DtCY6Z59iM/XqegZtcEFEEDPQUaHUHl821R7lsbojCi6JEFAEDPQXa3CqAi++h86IoEfWEgZ4CbW4/FFmC1dy/SxictkhEkTDQU6DNrcJuM0a9ymIIL4oSUSQM9BRoc/tht/Vv/BzgRVEiioyBngJOj4psa//Gz4ELF0U19tCJqAcM9CTz+TUENAGrObpdijqSJQmyLEFlD52IesBAT7LQlMX+XhANMSgSe+hE1CMGepK1OGMMdJnb0BFRzxjoSdbqir2HzlkuRNQTBnqSxRroiiIjEOf144koMzDQk6zF5YcsAWbjxb317KETUW8Y6EnW6vbDYjb0+6aiEIMi86IoEfWIgZ5krS4/bBc53AIEA50XRYmoJwz0JGt1+S96/BwI3lzEIRci6gkDPcmCgd7/m4pC2EMnot4w0JMooOlwetSYeui8KEpEvWGgJ1GsUxYBXhQlot4x0JOoJU6Brgsu0EVE3THQk6jZ6QMQY6DLwemO/gADnYg6Y6AnUWgdF1uMF0UBwK9qcamJiDIHAz2JWlx+SAAsptimLQLsoRNRdwz0JGpx+pBlNUKWL+4uUYA9dCLqHQM9iZqdfuRk9X/ruY7Cgc4eOhF1cfF/+7fz+XzYsGEDPvroI5jNZkybNg3r1q2LR20Zp8XlQ27Mgd4+5MIeOhF1EXOg//znP4fZbMaOHTsgSRIaGhriUVdGanb6MWG0LaZzXBhyYQ+diDqLKdBdLhfeeOMNvP/+++HVA4cOHRqXwjKNLgRaXbEPuVy4KMoeOhF1FlOgV1ZWIi8vD7/61a+wd+9eZGVl4ZFHHkFJSUnU5ygoyI6lhF4VFtoTct6L1eL0QdMFhg6xwZ5t6fZzo9EQ1eNCar/sIclp18a+DLR6e5Mp7QAypy1sR1BMga5pGiorK3HFFVfg0UcfxRdffIEHHngA7777LrKzowtqh8MJPc478BQW2lFf3xbXc8bqXJ0TAGA1ymhzerv9XFUD3R63Z1u6Pe7zBwAATrcv7doYSTp+JhcjU9oBZE5bBls7ZFnqtSMc0yyX4uJiGAwGlJWVAQCmTp2KIUOGoKKiIpbTZqTQbf9xm+XCMXQi6iKmQM/Pz8fMmTPxwQcfAAAqKirgcDgwZsyYuBSXSdo8wUDPshpjOo8ic5YLEfUs5lkuTzzxBP7xH/8R5eXlMBgMePLJJ5GTkxOP2jKK2xscKolltyIAkCQJBkWCynnoRNRFzIE+evRo/Od//mc8aslorlCgW2J+y6HIMnzsoRNRF7xTNElcHhVmoxIeA4+FQZF4pygRdcNATxK3NxCX3jkQvDDKMXQi6oqBniQur4qsuAU6e+hE1B0DPUlc3gCyLLHNcAkxKDJU9tCJqAsGepK4vWrchlwURYaP89CJqAsGepLEt4cuQeVaLkTUBQM9SVxeFVnW+F0U5bRFIuqKgZ4EAU2HX9Vhi2sPnUMuRNQZAz0JQjcVxWuWiyLLXMuFiLphoCeBy6MCQPzG0A3BeehCxHeVSiIa2BjoSeCOcw/dIEsQCA7lEBGFMNCTwOUN9tDjN4Ye/Ng4dZGIOmKgJ0Eo0ON5pyjAJXSJqDMGehKEL4rGuBZ6iBLuoTPQiegCBnoSxGst9JALPXQOuRDRBQz0JHB5VFjNCuT23YZiZWAPnYh6wEBPgnje9g906KHz9n8i6oCBngTxXJgL6NBD93PIhYguYKAnQfx76MGPjbNciKgjBnoSuBLUQ/cy0ImoAwZ6Erjj3EM3GtoD3ReI2zmJaOBjoCeYECKu288BwYuisgS4GehE1AEDPcH8AR0BTcR1yEWSJFhMBnh9HHIhogsY6AnmjvNdoiEWs8IeOhF1wkBPsHgvnRtiMRng9TPQieiCuAX6r371K0ycOBHHjx+P1ykzwoWVFuM35AIAFpMCD3voRNRBXAL90KFD2L9/P0aOHBmP02WUeK+FHmI1G+DhGDoRdRBzoPv9fqxduxZr1qyJQzmZx+lN1JALe+hE1FnM3cZnnnkGixcvxqhRoy7q+QUF2bGW0KPCQntCzttfskEBAFwyagiyrUaIRjfs2ZZuxxmNhn49bs8ywxdoSZt2RmMg1RpJprQDyJy2sB1BMQX6559/joMHD+KHP/zhRZ/D4XBC1+O7N2ZhoR319W1xPWd/BXTApwZwrrYNEoC6hjY0SBJ0AbQ5vd2OV9VAt8ft2ZYeHwcAgxy84JrqdkYrHT6TeMiUdgCZ05bB1g5ZlnrtCMc05LJv3z6cPHkS8+bNQ2lpKWpqavC9730Pe/bsieW0GcGnBrDvSC1OV7fCaJTx2dE67DtSi4AenwW1LGYDApqAGuACXUQUFFMPfeXKlVi5cmX436Wlpdi0aRMmTJgQc2GZwqdqMBuVuJ/XYgqe0+MLwGgwxf38RDTwcB56gvn8iQl0qyn4u9jDuehE1C6uc+l27doVz9NlBI8vALst/j1oi/lCD52ICGAPPeHcvkDcbyoCOvTQORediNox0BMooOnwqzqscdocuqOOY+hERAADPaFCYWtLRKCbQz10BjoRBTHQEyi0GiJ76ESUDAz0BPK0r+OSiDF0S3iWC8fQiSiIgZ5A7gQOuRgNMgyKzB46EYUx0BPI4wtAliWYjIl5m21mhfuKElEYAz2B3N4AbGYDJElKyPktZgN3LSKiMAZ6Anl8WkIuiIZYzQZ4OYZORO0Y6AmUqJuKQqwm7itKRBcw0BPI0z7kkihWs4Fj6EQUxkBPEK8/AFXTYTXHf2GukOA2dAx0IgpioCdIq8sPIDFz0EO4rygRdcRAT5AWZzDQE3tRVIHHH4AQ8d3xiYgGJgZ6grSEeugJHkMXIriJBhERAz1BWpw+AIA1obNcuIQuEV3AQE+QFpcfBkWCUUncW2zliotE1AEDPUFanH5YE3iXKIDwDBoGOhEBDPSEaXb6Ejp+DnTooXNfUSICAz1hWl3+hE5ZBDiGTkSdMdATQAgRHnJJJI6hE1FHDPQEcHpUqJqehCEXjqET0QUM9AQ4dLoRADA0z5rQ1wnvWsRAJyIw0BNi/4kG2G1GDM2zJPR1ZFmC2aRwDJ2IADDQ404N6Dhw0oHJlxVATuCUxRCb2cBZLkQEAIhpkLepqQk//vGPcfbsWZhMJowZMwZr165Ffn5+vOobcI6ebYLXr2HK14bC7VUT9jqSLMHlC8BsVOB0q3C1D7uYjQYY+GuaaFCK6asvSRJWrFiBHTt2YOvWrRg9ejSeeuqpeNU2IH1+vB5mo4IJo/MS+jo+VcO+I7UIaDpqm9zYd6QW+47Uwqeyt040WMUU6Hl5eZg5c2b439OmTUNVVVXMRQ1UuhD4/EQDJl+WD2OSuslmk8Jt6IgIQIxDLh3puo5XXnkFpaWl/XpeQUF2vEropLDQnpDz9qTG4UJVgwt1jW60uPy4fvpo2Gxm2LO7XxQ1Gg1xfXxIjgUNzc3hY2w2MwrzbXFoVfwl8zNJpExpB5A5bWE7guIW6OvWrYPNZsNdd93Vr+c5HE7oenzX8y4stKO+vi2u5+yN26fhsU0fwukJjpcbFRlFuWY4XT60Ob3djlfVQNSP27MtfR5vNsjwqRocTS6YjArcbh/qtfTrsSfzM0mkTGkHkDltGWztkGWp145wXAK9vLwcZ86cwaZNmyDLg+uK3IcHq+H0qPjGVcOQm2WC1WzAoYpGTJ1QmJTXz7YaAQBtHhUFxsRtd0dE6S/mQH/66adx8OBBbN68GSaTKR41DRhCCOzefx552SZ8bWRuQldW7E0o0J1uFQU5iZ33TkTpLaZAP3HiBJ5//nmMHTsWd955JwBg1KhReO655+JSXLo7eb4V5+pdmHXFsJSEOQBk29oD3ZO4KZJENDDEFOjjx4/HsWPH4lXLgLPzs0pYzQouHZGTshrMRgVGg8xAJyLeKXqxmp0+fHasHrOuHJ60KYq9ybYaGehExEC/WO/uq4QuBK6bOiLVpcBuM8LpZqATDXYM9Ivg9KjY9fl5zJg0DIUJXlExGqEeuhDxnf5JRAMLA/0i7Py0Ej6/hlu/MSbVpQAIBrqmC94xSjTIMdD7yeMLYOen53D1+KEYVZiYu1z7q+PURSIavBjo/fTe5+fh9gVQNntsqksJC01dbOOFUaJBjYHeD7ousOvP5zBpzBBcWpy6qYpdhXvoDHSiQY2B3g9fnGpEY6sP37hqOFy+AFy+AOK8DM1FMSgyLCaFgU40yCV2F+MM8/7n52AxKeG1yAEkbc2WvnAuOhGxhx6lZqcPB085MG5kLhQ5Nbf5R5LNuehEgx4DPUp7DlRDF8D4UbmpLqVHdqsRLq8a96WIiWjgYKBHQRcCu7+owvjRucjJSs8VJbOtRggB1Dd7Ul0KEaUIAz0Kh083oqHFi2snF6e6lF4VD82CLEv448dnUl0KEaUIAz0Ku/dXIdtqxJRxQ1NdSq+yrUZcdWk+PjtWj6NnmgAAX55y4PfvfYWDFQ4END3FFRJRonGWSx9aXH58fqIB80tGpXxVxb5cdVk+ztU7seXd45g8rgBv7z0LAHh771lYzQrmTB6Bm64ZjYJcboRBlIkY6H344MtqaLrA3DRYVbEvBkXGt775Nfz6zUM43+DCnCnFWHTtWJw834ovvmrArj+fw/9+dg43XD0SS+d9DQYlvX9BEVH/MNAjCF0MnTA6D8UFWXD5AqkuqU8Tx+Th6vFDkZNlwpjhdhyqaAQA3L3wctw6ewze+aQS//vnczhb34bvlV2BfLsVaf6HBxFFiV/lCI6daUJdkwfXT0v/3nmIJEmYPK4AY4bbOz3uUzV8da4Fl43IwbWTh+PkuRZseOlT1DW5UlQpEcUbAz2CP35yFtlWI0ompsfdoPEybmQubrxmNNy+ADa/eQg+LrtLlBEY6D0I6MChM404eKoRN3x9JPyaSJt1W+JlWL4Nc6eOQGWdE5u3HuINSUQZgIHeA58awO92noDJKCPbasS+I7XYd6QWAT2zpv6NKsrGt745Dp+faMDmrYfgGQDXCIiod7wo2oPKOifO1bswbfzQtJ+qGKtvfn0U/KqOrR9U4HRNG+69ZRK+NjKPF0qJBiB+bXvwx4/OwGiQcfkleakuJeF8qoa8bBNuvGY02twqntzyZzz1yp/x8aEauL3ssRMNJOyhd/Hx4Rp8ecqBq8cPhcmopLqcpBmeb8Oia8fgeGULzta2YfPWw5Ak4NLiHFxanIOhuRaYTQpOnm/BqapWDM+3Yc6UYkwZVwBFZr+AKB0w0DtobPXiNzuOY2yxHVdemp/qcpLOYjJgyrgC3LfoCpysbMbRs804frYZH3xZHd6AOstiwNjiHHx1vgWfn2hAjs2E2ZOHY/qEQhw+3Yg9X1ZD14FpXxuK8aNz0eZW0djmRY7dAgMAi0mBGtAhAEwdV4D8HN61ShQvMQd6RUUFHnvsMTQ3NyMvLw/l5eUYO3ZsHEpLLqdHxQvbj0DTBe5ZcDlOV7emuqSUUQM6mtp8GDbEimFDrBBCwK/quGxULs7VtkGSJFw9fijO1Tvx1bkWvPPJ2fAyAxNG58FsUrD7QBX+98/nAACKLEEXAqLLRJrfyhKuubwIpdNHYdyIHEhS93XmA5qOE+dacKqqBblZZgwvsGF0YTbMpsHz1xNRtGIO9NWrV2PZsmVYsmQJ/vCHP+Dxxx/Hyy+/HI/aEkbXBeqbPfjqfAtOnGvBV+dbUNUQvMHm3oWXozDPOqgDvStJkmA2KRiaZ8X5OicAQJYlXDLMjkuG2XHpyFzs/OQsioZYYbcFlxeefFk+Rg+zw241wp5lgtVqQr3DBZ+qwWYxwen24YMvq/HRwRp8fLgWhXkWXD2hCFkWA2QJaGz1oarBhdO1bd3mySuyhEuLc3D5mCEYXZSN4fk2GA0yNE2HQZGRm22C2ajA0erF+XoXWlx+eH0ByLKEKy/Nx/B8W/iXhxrQUO1ww9HqRWGeFcPzbX0uiSCEQE2jG8cqm3GisgWSFJwGmm83Q5YlSADGDLd3ep1IvP4AFFnO+AvwXQU0HbIsQY7iPUoENaChvtmLxjYvDLIMi1lBXrYZuVmmqD63dBRToDscDhw+fBgvvvgiAKCsrAzr1q1DY2Mj8vOjG7KQL2L3n7O1TpysaoGu61ADAh5fAO72L6zZICPHboEW0GAyyNAFoOk6PD4NrS4/Gtt8qGtyQw0EpyBaTAouLbZj7tUjcNnwHIweZocuAJvF2O11DYrc7fGeHovX41azIaHnj1eb8nMsmNzDSpQjCrNxpKIRqHchO8sMp8sHAJh0qRWnq72YeMkQjBuZi/P1TpytdeKLE/UIdeItJgWFeTbMLxmFy0bmweXxw69qaHOrcLR4UdfkwSdHavFJ+1aAXSmyBK2HufU7PzuHghwLjEYFbq8Kp1uF3uFPB0WRkJdlRrbNiGyrEVmW4P9CArSAQJPbj68qm+H2BneHyrIaocgSvjrf0u21CnIsGDPc3n4eAyBJgBDwBXS4vQG0OH2odrjR7Ay+LwaDDIvJAKtJgcmoIKDp8Ps1CACyDJiMCnKyTMi1mRDQBVxeFR5vAG5vAD5Vg8VsQJYl+Fo2iwFGRYbLG4Dbq0LVBYQuIMlS8PwGBYpBgsutwuPT4PT4oQsgL9uEvGwzzEYFRoMMXQh4/Rr8qgaDEvqlIxBavNNiUmAxGqDpOtSAjla3H45WL1pdKsxGGVazATazAVaTAQLBYc1mpw8+VYOmCyiKhCF2C4Zkm2AyKu2vIcGgBP+/ySBDUSRIEiAh+Muy/W2EqgVfUzHIaHP5oQb04HumanB6AnC2f0ZWkwKr2QCr2QCzUYHT7Yej1YcWpw893X1hMirIzzHDZjbAYjJAUWQosgRZCn4OshT8JSTLEgKaDp+qQ9cEzObg68iSBAggoOvwqQGoAQGzUYHNYgi2R5YwLD8LV4wd0u21o8nDSMdIQnT9Qzh6Bw8exKOPPort27eHH7vlllvw85//HFdeeeXFnpaIiC7C4Pobj4gog8UU6MXFxaitrYWmBcc4NU1DXV0diovTd2cfIqJMFVOgFxQUYNKkSdi2bRsAYNu2bZg0aVLU4+dERBQ/MY2hA8DJkyfx2GOPobW1FTk5OSgvL8dll10Wr/qIiChKMQc6ERGlB14UJXrZjbgAAAvUSURBVCLKEAx0IqIMwUAnIsoQDHQiogyREYHu8Xjw/e9/HzfeeCMWLFiA9957r8fjdu7cidtvvx1lZWW49dZb8e///u9JrrRnFRUVWLp0KW6++WYsXboUp0+f7naMpml44oknMH/+fNx444149dVXk19oH6Jpx3PPPYdbb70VixYtwu23344//elPyS+0D9G0I+TUqVOYOnUqysvLk1dgP0TblrfeeguLFi1CWVkZFi1ahIaGhuQW2odo2uFwOLBy5UosWrQICxcuxJo1axAIpNea/uXl5SgtLcXEiRNx/PjxHo+J6bsuMsCzzz4rfvKTnwghhKioqBCzZ88WTqez23H79+8XNTU1QgghWltbxfz588W+ffuSWmtP7r77bvHGG28IIYR44403xN13393tmNdff13cd999QtM04XA4xHXXXScqKyuTXWpE0bRj9+7dwu12CyGEOHLkiJg+fbrweDxJrbMv0bRDCCECgYC46667xA9+8APxs5/9LJklRi2athw4cEAsXLhQ1NXVCSGC3w2v15vUOvsSTTvWr18f/hz8fr+44447xPbt25NaZ1/27dsnqqqqxA033CCOHTvW4zGxfNczoof+xz/+EUuXLgUAjB07FldddRV2797d7bipU6di2LBhAAC73Y5x48bh/PnzSa21q9ACZ2VlZQCCC5wdPnwYjY2NnY5766238O1vfxuyLCM/Px/z58/H22+/nYqSexRtO6677jpYrVYAwMSJEyGEQHNzc9Lr7U207QCAzZs345vf/GbaLhcdbVv+4z/+A/fddx8KCwsBBL8bZrM56fX2Jtp2SJIEl8sFXdfh9/uhqmr4+54uSkpK+ryTPpbvekYEelVVFUaOHBn+d3FxMWpqaiI+5+TJk9i/fz9mzZqV6PIiqq6uxrBhw6AowfW9FUVBUVERqqurux03YsSI8L+jaWMyRduOjt544w1ccsklGD58eLLK7FO07Th69Cj27NmDe++9NwVVRifatpw8eRKVlZX47ne/i9tuuw0bN26ESKPbU6Jtx4MPPoiKigrMmTMn/N/06dNTUXJMYvmuD4gdi2677TZUVVX1+LMPP/yw3+erq6vDgw8+iNWrV6fdb/DB4pNPPsEzzzyTNtcx+kNVVfzTP/0T/vmf/zkcMgOZpmk4duwYXnzxRfj9fqxYsQIjRozAX/zFX6S6tH55++23MXHiRLz00ktwuVy4//778fbbb2PBggWpLi1pBkSgv/766xF/PmLECJw/fz68hkx1dTVmzpzZ47EOhwPLly/HihUrsHDhwrjX2l8dFzhTFKXXBc6Ki4tRVVWFKVOmAOj+WzzVom0HAHz++ef40Y9+hI0bN6bdMhHRtKO+vh5nz57FypUrAQCtra0QQsDpdGLdunWpKr2baD+TESNGYMGCBTCZTDCZTJg3bx4OHDiQNoEebTt+85vfYMOGDZBlGXa7HaWlpdi7d++AC/RYvusZMeSyYMEC/O53vwMAnD59Gl9++SWuu+66bsc1NTVh+fLl+O53v4tvf/vbyS6zR9EucLZgwQK8+uqr0HUdjY2N2LlzJ26++eZUlNyjaNtx4MAB/N3f/R1++ctfpuWa+dG0Y8SIEdi7dy927dqFXbt24a/+6q/wne98J63CHIj+MykrK8OePXsghICqqvj4449x+eWXp6LkHkXbjlGjRoWvnfn9fnz00UcYP3580uuNVUzf9XhdvU0ll8slHn74YTF//nxx0003iXfffTf8s1/84hfit7/9rRBCiJ/97Gdi8uTJYvHixeH//vu//ztVZYd99dVX4o477hA33XSTuOOOO8TJkyeFEEKsWLFCHDhwQAgRnFHx+OOPi3nz5ol58+aJ//qv/0plyT2Kph233367mDlzZqfP4OjRo6ksu5to2tHRL3/5y7Sd5RJNWzRNExs2bBALFiwQt9xyi9iwYYPQNC2VZXcTTTvOnDkj7r33XlFWViYWLlwo1qxZI1RVTWXZ3axbt05cd911YtKkSWL27NnilltuEULE77vOxbmIiDJERgy5EBERA52IKGMw0ImIMgQDnYgoQzDQiYgyBAOdiChDMNCJ0khpaelFLWdBBDDQKQ2k25rVXcWrvnRvJw18DHRKidLSUmzevBmLFi3CtGnT8Omnn+LOO+9ESUkJFi9ejL1794aPbW5uxj/8wz9gzpw5uOaaa/Dggw+Gf/b73/8eN954I2bMmIEHHngAtbW1AIDVq1d323Tir//6r/Hiiy8CAGpra/Hwww9j1qxZKC0txcsvvxw+7tlnn8Xf/u3f4oc//CG+/vWvY/PmzZg6dSqamprCxxw6dAizZs2Cqqq9tvG1117DnXfeiQ0bNmDmzJl49tlncfbsWdxzzz2YOXMmZs6cib//+79Ha2srAOBHP/oRqqqq8MADD+Dqq6/Gr3/9awDA/v37e31viDpJwN2tRH264YYbxOLFi0VVVZWoqakRM2bMEP/3f/8nNE0Te/bsETNmzBAOh0MIIcT9998vHnnkEdHc3Cz8fr/Yu3evEEKIDz/8UMyYMUMcPHhQ+Hw+sXbtWrFs2TIhhBCffPKJmDt3rtB1XQghRHNzs5g8ebKoqakRmqaJ2267TTz77LPC5/OJs2fPitLSUrF7924hRPBW/iuuuEK8++67QtM04fF4xIoVK8SWLVvC9f/0pz8Va9eujdjG//mf/xGTJk0SL7/8slBVVXg8HnH69GmxZ88e4fP5hMPhEMuWLRPr16/v9L588MEH4X/39d4QdcQeOqXM3XffjeLiYvzhD3/A3Llzcf3110OWZVx77bW46qqr8P7776Ourg67d+/GE088gdzcXBiNRsyYMQMAsHXrVnzrW9/ClVdeCZPJhB/84AfYv38/zp07h5KSEkiShE8//RQAsGPHDkybNg3Dhg3Dl19+icbGRjz00EMwmUwYPXo0vvOd7+Ctt94K1zZt2jTMnz8fsizDYrHgtttuw5tvvgkguNzs9u3bsWTJkj7bWFRUhLvvvhsGgwEWiwVjxozBtddeC5PJhPz8fCxfvhz79u3r9fmR3huirgbE8rmUmULLn1ZVVeHtt9/utBdsIBDAzJkzUVNTg9zcXOTm5nZ7fl1dXacVG7OyspCXl4fa2lqMGjUKt9xyC7Zt24ZrrrkGW7duxeLFiwEA58+fR11dHUpKSsLP1TSt07+7broxb948rF69GpWVlaioqEB2dnZ4edNIup6noaEBP/3pT/Hpp5/C5XJBCIGcnJxenx/pvSHqioFOKSNJEoBgsC9ZsgTr16/vdkxdXR1aWlrQ2traLfiKioo6bSHodrvR3Nwc3rSkrKwM9913H1auXIkDBw7gueeeC7/eqFGj8M477/RZW4jZbMbChQvx5ptv4tSpU1H1zns6z9NPPw1JkrB161bk5eVh586dWLt2ba/Pj/TeEHXFIRdKucWLF+O9997Dn/70J2iaBp/Ph71796KmpgZFRUWYO3cunnjiCbS0tEBV1fAQRVlZGV577TUcOXIEfr8fTz/9NKZMmYJRo0YBAK644goMGTIEq1atwpw5c8K/EKZMmYKsrCxs3rwZXq8Xmqbh+PHjOHDgQMQ6lyxZgtdffx27du2KOtC7crlcsNlssNvtqK2txb/92791+vnQoUNRWVkZ1XtD1BUDnVKuuLgYGzduxPPPP49vfOMbuP766/HCCy9A13UAwJNPPgmDwYCFCxdi9uzZeOmllwAAs2fPxiOPPIKHH34Yc+bMQWVlJf7lX/6l07nLysrw4YcfhjcYBoJ7Um7atAlHjx7FvHnzMGvWLKxatQpOpzNindOnT4csy7jyyis77WHbHw899BAOHz6MkpISrFy5EjfddFOnn69cuRL/+q//ipKSErzwwgt9vjdEHXE9dKJ+uOeee7Bo0aK02fGKqCP20ImidODAARw+fDgt9qIl6gkvihJF4dFHH8XOnTvxk5/8BNnZ2eHHH3/8cWzdurXb8YsWLYp4sZMoETjkQkSUITjkQkSUIRjoREQZgoFORJQhGOhERBmCgU5ElCEY6EREGeL/AZPoBxr/GPXbAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "HJkZL-AkVqMv",
        "colab_type": "text"
      },
      "source": [
        "### Combining Stage 1 and Stage 2"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8IJxU9ZXVqMw",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 297
        },
        "outputId": "0c4a595e-f79b-40dc-dda3-765986522f3e"
      },
      "source": [
        "y_hat_test_lgd_stage_2_all = reg_lgd_st_2.predict(lgd_inputs_stage_1_test)\n",
        "\n",
        "y_hat_test_lgd_stage_2_all\n",
        "\n",
        "y_hat_test_lgd = y_hat_test_lgd_stage_1 * y_hat_test_lgd_stage_2_all\n",
        "# Here we combine the predictions of the models from the two stages.\n",
        "\n",
        "pd.DataFrame(y_hat_test_lgd).describe()\n",
        "# Shows some descriptive statisics for the values of a column.\n",
        "\n",
        "y_hat_test_lgd = np.where(y_hat_test_lgd < 0, 0, y_hat_test_lgd)\n",
        "y_hat_test_lgd = np.where(y_hat_test_lgd > 1, 1, y_hat_test_lgd)\n",
        "# We set predicted values that are greater than 1 to 1 and predicted values that are less than 0 to 0.\n",
        "\n",
        "pd.DataFrame(y_hat_test_lgd).describe()\n",
        "# Shows some descriptive statisics for the values of a column."
      ],
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>8648.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.087556</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.049606</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.066174</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.101376</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>0.123556</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>0.236973</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                 0\n",
              "count  8648.000000\n",
              "mean      0.087556\n",
              "std       0.049606\n",
              "min       0.000000\n",
              "25%       0.066174\n",
              "50%       0.101376\n",
              "75%       0.123556\n",
              "max       0.236973"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "z2WwUZglVqM8",
        "colab_type": "text"
      },
      "source": [
        "# EAD Model (Exposure at Default)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "V4iAozx3VqM8",
        "colab_type": "text"
      },
      "source": [
        "### Estimation and Interpretation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "PleiEvpkVqM8",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 419
        },
        "outputId": "e58faa01-b7d9-4106-d84c-1b2972fc57d8"
      },
      "source": [
        "# EAD model datasets\n",
        "ead_inputs_train, ead_inputs_test, ead_targets_train, ead_targets_test = train_test_split(loan_data_defaults.drop(['good_bad', 'recovery_rate','recovery_rate_0_1', 'CCF'], axis = 1), loan_data_defaults['CCF'], test_size = 0.2, random_state = 42)\n",
        "# Takes a set of inputs and a set of targets as arguments. Splits the inputs and the targets into four dataframes:\n",
        "# Inputs - Train, Inputs - Test, Targets - Train, Targets - Test.\n",
        "\n",
        "ead_inputs_train.columns.values\n",
        "\n",
        "ead_inputs_train = ead_inputs_train[features_all]\n",
        "# Here we keep only the variables we need for the model.\n",
        "\n",
        "ead_inputs_train = ead_inputs_train.drop(features_reference_cat, axis = 1)\n",
        "# Here we remove the dummy variable reference categories.\n",
        "\n",
        "reg_ead = LinearRegression()\n",
        "# We create an instance of an object from the 'LogisticRegression' class.\n",
        "reg_ead.fit(ead_inputs_train, ead_targets_train)\n",
        "# Estimates the coefficients of the object from the 'LogisticRegression' class\n",
        "# with inputs (independent variables) contained in the first dataframe\n",
        "# and targets (dependent variables) contained in the second dataframe.\n",
        "\n",
        "feature_name = ead_inputs_train.columns.values\n",
        "\n",
        "summary_table = pd.DataFrame(columns = ['Feature name'], data = feature_name)\n",
        "# Creates a dataframe with a column titled 'Feature name' and row values contained in the 'feature_name' variable.\n",
        "summary_table['Coefficients'] = np.transpose(reg_ead.coef_)\n",
        "# Creates a new column in the dataframe, called 'Coefficients',\n",
        "# with row values the transposed coefficients from the 'LogisticRegression' object.\n",
        "summary_table.index = summary_table.index + 1\n",
        "# Increases the index of every row of the dataframe with 1.\n",
        "summary_table.loc[0] = ['Intercept', reg_ead.intercept_]\n",
        "# Assigns values of the row with index 0 of the dataframe.\n",
        "summary_table = summary_table.sort_index()\n",
        "# Sorts the dataframe by index.\n",
        "p_values = reg_lgd_st_2.p\n",
        "# We take the result of the newly added method 'p_values' and store it in a variable 'p_values'.\n",
        "p_values = np.append(np.nan,np.array(p_values))\n",
        "# We add the value 'NaN' in the beginning of the variable with p-values.\n",
        "summary_table['p_values'] = p_values\n",
        "# In the 'summary_table' dataframe, we add a new column, called 'p_values', containing the values from the 'p_values' variable.\n",
        "summary_table\n",
        "\n",
        "summary_table = pd.DataFrame(columns = ['Feature name'], data = feature_name)\n",
        "summary_table['Coefficients'] = np.transpose(reg_ead.coef_)\n",
        "summary_table.index = summary_table.index + 1\n",
        "summary_table.loc[0] = ['Intercept', reg_ead.intercept_]\n",
        "summary_table = summary_table.sort_index()\n",
        "p_values = reg_lgd_st_2.p\n",
        "p_values = np.append(np.nan,np.array(p_values))\n",
        "summary_table['p_values'] = p_values\n",
        "summary_table"
      ],
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Feature name</th>\n",
              "      <th>Coefficients</th>\n",
              "      <th>p_values</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Intercept</td>\n",
              "      <td>1.109746e+00</td>\n",
              "      <td>NaN</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>grade:A</td>\n",
              "      <td>-3.030033e-01</td>\n",
              "      <td>0.000000e+00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>grade:B</td>\n",
              "      <td>-2.364277e-01</td>\n",
              "      <td>0.000000e+00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>grade:C</td>\n",
              "      <td>-1.720232e-01</td>\n",
              "      <td>0.000000e+00</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>grade:D</td>\n",
              "      <td>-1.198470e-01</td>\n",
              "      <td>1.970202e-12</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>40</th>\n",
              "      <td>open_acc</td>\n",
              "      <td>-2.142130e-03</td>\n",
              "      <td>1.365854e-09</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>41</th>\n",
              "      <td>pub_rec</td>\n",
              "      <td>6.782062e-03</td>\n",
              "      <td>2.079574e-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>42</th>\n",
              "      <td>total_acc</td>\n",
              "      <td>4.518110e-04</td>\n",
              "      <td>5.133116e-08</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>43</th>\n",
              "      <td>acc_now_delinq</td>\n",
              "      <td>9.974801e-03</td>\n",
              "      <td>6.583494e-01</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>44</th>\n",
              "      <td>total_rev_hi_lim</td>\n",
              "      <td>2.166527e-07</td>\n",
              "      <td>2.592628e-08</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>45 rows × 3 columns</p>\n",
              "</div>"
            ],
            "text/plain": [
              "        Feature name  Coefficients      p_values\n",
              "0          Intercept  1.109746e+00           NaN\n",
              "1            grade:A -3.030033e-01  0.000000e+00\n",
              "2            grade:B -2.364277e-01  0.000000e+00\n",
              "3            grade:C -1.720232e-01  0.000000e+00\n",
              "4            grade:D -1.198470e-01  1.970202e-12\n",
              "..               ...           ...           ...\n",
              "40          open_acc -2.142130e-03  1.365854e-09\n",
              "41           pub_rec  6.782062e-03  2.079574e-01\n",
              "42         total_acc  4.518110e-04  5.133116e-08\n",
              "43    acc_now_delinq  9.974801e-03  6.583494e-01\n",
              "44  total_rev_hi_lim  2.166527e-07  2.592628e-08\n",
              "\n",
              "[45 rows x 3 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "F0tUVoc1VqNP",
        "colab_type": "text"
      },
      "source": [
        "### Model Validation"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "5_JxSjvIVqNP",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 565
        },
        "outputId": "46133689-c788-449a-8a0b-af1cf01c18f5"
      },
      "source": [
        "ead_inputs_test = ead_inputs_test[features_all]\n",
        "# Here we keep only the variables we need for the model.\n",
        "\n",
        "ead_inputs_test = ead_inputs_test.drop(features_reference_cat, axis = 1)\n",
        "# Here we remove the dummy variable reference categories.\n",
        "\n",
        "ead_inputs_test.columns.values\n",
        "\n",
        "y_hat_test_ead = reg_ead.predict(ead_inputs_test)\n",
        "# Calculates the predicted values for the dependent variable (targets)\n",
        "# based on the values of the independent variables (inputs) supplied as an argument.\n",
        "\n",
        "ead_targets_test_temp = ead_targets_test\n",
        "\n",
        "ead_targets_test_temp = ead_targets_test_temp.reset_index(drop = True)\n",
        "# We reset the index of a dataframe.\n",
        "\n",
        "pd.concat([ead_targets_test_temp, pd.DataFrame(y_hat_test_ead)], axis = 1).corr()\n",
        "# We calculate the correlation between actual and predicted values.\n",
        "\n",
        "sns.distplot(ead_targets_test - y_hat_test_ead)\n",
        "# We plot the distribution of the residuals.\n",
        "\n",
        "pd.DataFrame(y_hat_test_ead).describe()\n",
        "# Shows some descriptive statisics for the values of a column.\n",
        "\n",
        "y_hat_test_ead = np.where(y_hat_test_ead < 0, 0, y_hat_test_ead)\n",
        "y_hat_test_ead = np.where(y_hat_test_ead > 1, 1, y_hat_test_ead)\n",
        "# We set predicted values that are greater than 1 to 1 and predicted values that are less than 0 to 0.\n",
        "\n",
        "pd.DataFrame(y_hat_test_ead).describe()\n",
        "# Shows some descriptive statisics for the values of a column."
      ],
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>0</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>count</th>\n",
              "      <td>8648.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>mean</th>\n",
              "      <td>0.735992</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>std</th>\n",
              "      <td>0.105127</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>min</th>\n",
              "      <td>0.384774</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>25%</th>\n",
              "      <td>0.661553</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>50%</th>\n",
              "      <td>0.731750</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>75%</th>\n",
              "      <td>0.810625</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>max</th>\n",
              "      <td>1.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "                 0\n",
              "count  8648.000000\n",
              "mean      0.735992\n",
              "std       0.105127\n",
              "min       0.384774\n",
              "25%       0.661553\n",
              "50%       0.731750\n",
              "75%       0.810625\n",
              "max       1.000000"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEMCAYAAADOLq1xAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3de3wTdb438M9MLk3SpE2Tpm16494CykVEBF3QlWJZhVcrPiwedX0WFlwO7nL09awLz1mOHtBdt5x9cc7KC9fLruzxclyXx7UsUJWtV0QRRJBLS1tK703TNGmaJk3SXOb5ozS2tDRpm2Qm6ff9eiHpzC8z34Tk4/Q3M78fw3EcB0IIIXGH5bsAQgghkUEBTwghcYoCnhBC4hQFPCGExCkKeEIIiVMU8IQQEqco4AkhJE6J+S5goM5OB/z+6F+Wr9UqYTbbo77f8aK6oycWawZis+5YrBngp26WZZCSknjd9YIKeL+f4yXg+/cdi6ju6InFmoHYrDsWawaEVzd10RBCSJyigCeEkDhFAU8IIXGKAp4QQuIUBTwhhMQpCnhCCIlTFPCEEBKnBHUdPCEkPLx+wO3xgrP0oMftBQAkSMQQ0yHdhEIBT0gccnu8OFVphEopQ7fdBQC4ZVY6xAn0lZ9IQvrX3rJlC5qbm8GyLBQKBf7t3/4Ns2bNGtTG5/Ph2WefxbFjx8AwDB599FGsXbs2IkUTQggJLqSALykpgUqlAgCUl5fjX//1X/Huu+8OanPo0CE0Njbi6NGjsFqtKC4uxpIlS5CdnR3+qgkhhAQVUo9cf7gDgN1uB8MwQ9qUlZVh7dq1YFkWGo0GBQUFeP/998NXKSGEkFEJuUPuV7/6FY4fPw6O4/DHP/5xyHqDwYDMzMzAz3q9Hm1tbeGpkhBCyKiFHPC//vWvAQClpaXYvXs3XnnllbAXo9Uqw77NUOl0quCNBIjqjp5Yqpmz9ECllAFA4G+RWITWThc0STIkSEWBtnKZGCqFlJc6ryeW3uuBhFb3qE+pFxcX46mnnkJnZydSUlICy/V6PVpbWzF37lwAQ4/oQ2E223kZblOnU8Fk6o76fseL6o6eWKu5x+1Ft90FlVIGk8WOj79phcXmgs/PQSmX4J4luZBJ+77+t8xKh8vh5rni78Tae92Pj7pZlhnxwDhoH7zD4YDBYAj8/NFHHyE5ORlqtXpQu5UrV+LAgQPw+/2wWCwoLy9HYWHhOEonhITD2RozOrqcWHZTJm6ZmYYetxefnTUIbuxyEn5Bj+CdTif+5V/+BU6nEyzLIjk5GS+++CIYhsGmTZuwdetWzJkzB0VFRfj2229x9913AwAee+wx5OTkRPwFEEKuz9zlQk2TFfm5aqy6fQq+rTZBImbxxYU2nK4y4ZZZaXyXSCIoaMCnpqbir3/967DrBvbDi0Qi7Ny5M3yVEULG7YtzrZCIWcydnhpYNj07GRabC5UNnZiSmcRjdSTS6MZlQuJUi8mBRmM35k7TQjbgpCoAzJ+RCrGIQXWTlafqSDRQwBMSp2qarUiUiZE/ST1knVQiwuSMJNQbbHBeHauGxB8KeELiUK/Xh9YOB6ZkJUPEDv81z8tJhtfH4fSl9ihXR6KFAp6QOFTdaIXXx2GK/vp97NpkGVJUCTh+3gCOoytq4hEFPCFx6HytGRIRiyzd9a+RZhgGeTnJaDY5UGeIvevOSXAU8ITEGT/H4cIVCzJ1iRCJRv6KT8lMgkTM4suLNKxIPKKAJyTO1BlssPX0Iict+NAfUrEI07OSUVFviUJlJNoo4AmJM2drOsAyQJYuMaT2+blqGMw96OwWznAFJDwo4AmJM2drOjAtOxkJElHwxgDyJ/WNKfVtbQccbm/gj9cfySpJNND8XYTEkS5HL1o6HCj63pSQn5OaIkeCRITj5w2DltMUf7GPjuAJiSOXm7sAANOykkN+DsswyNDIYTD30OWScYYCnpA4UtNshUTMIjuEE6wDZWgT0ePyorvHE6HKCB8o4AmJIzXNXZii77v0cTT0WgUAwGDuiURZhCcU8ITECXevD43GbszIDr17pp9KIYFCJkab2RGByghfKOAJiRNXDDb4/NyYAp5hGOi1CrRZnNQPH0co4AmJEzXNVjAY3QnWgXTJcrg9PjicNLpkvKCAJyROXG7uQpYuEYkyyZier0lOAACYba5wlkV4RAFPSBzw+zlcbunCjOyhY7+HKkWZAIYBLBTwcYMCnpA40Gyyw9XrG1P/ez+RiIVamQCzjYYsiBcU8ITEgdqWvhucpo+x/72fJikBFpuLTrTGCQp4QmKY1w843F7UtHRBKZdAJhPD4fbCP8Z81ibJ4Or10TR+cYIGmiAkhrk9XpyqNOJSQyeSE6X4+ur0e/PydGPanjZJBgDUTRMn6AiekBjn8frRZe+FNlk27m2pVQlgAJi76ERrPKCAJyTGWbpd4ACkhiHgJWIWSUopXUkTJyjgCYlx/Ufb4TiCB/q6aaiLJj4E7YPv7OzEL3/5SzQ2NkIqlWLSpEnYtWsXNBrNoHbbt2/HF198gZSUvskDVq5ciX/+53+OTNWEkICOLhcUMjHkYRq7XZOUgCutNtgcvUik8eBjWtB/PYZhsHHjRtx6660AgJKSEvzud7/Db37zmyFtH330UTz88MPhr5IQcl3mLldYumf69Z9obWq3Q69RhG27JPqCdtGo1epAuAPA/Pnz0draGtGiCCGh6XF50N3jCYRyOKSo+oYsaO2gkSVj3aj64P1+P9566y3cddddw67fv38/Vq9ejS1btqC2tjYsBRJCrq/RaAcQvv53AJBKRFDIxBTwcWBUHWzPPPMMFArFsN0wTzzxBHQ6HViWRWlpKTZu3Ijy8nKIRKFN/AsAWu3oZqEJJ51Oxdu+x4Pqjh4h1mw40QgAmJSZDJn0u6+zRCKGStkX+v1/D1w20HDLU9VyGC09vL1mIb7XoRBa3SEHfElJCRoaGvDiiy+CZYce+KenpwceFxcX47nnnkNbWxuysrJCLsZstsM/1lvwxkGnU8Fk6o76fseL6o4eodZ8ubkTKoUEnl4vPL3f3X3q8XjRbXdBpZSh2+4atOxawy1XySWoauyEoa0LYlF0L7YT6nsdDB91sywz4oFxSP9ye/bswYULF7Bv3z5IpdJh2xiNxsDjY8eOgWXZQaFPCAm/5nY7NGHsf++nVkrh9XFo73SGfdskeoIewdfU1OCll17C5MmT8cADDwAAsrOzsW/fPhQVFeHll19Geno6tm3bBrPZDIZhoFQq8Yc//AFiMV1iRUik9Lg8sNjcmJQR/m6B/hOtzSY7MlMTw759Eh1BE3jGjBmoqqoadt3BgwcDj//85z+HrShCSHD9J1g1qvAfwScnSsEyQLPJgUWzwr55EiV0JyshMaqx/WrAJyWEfdsiEQtdihwtJnvYt02ihwKekBjVZOxGkkIatjtYr5WpTUSLiS6VjGUU8ITEqAajHVlpkesf16cmwmR1wt3ri9g+SGRRwBMSgzxePwxmB7J1kbt3JDM1ERyAVjMdxccqCnhCYlBrhwM+PxfRgNdr+347aG6nfvhYRQFPSAxqNPbdUJMdwS6a1GQZpGIWLTRkQcyigCckBjW225EgESFVLY/YPliWgT41EU10BB+zKOAJiUFNxm7kpCnBMkxE95ObpkRTux0cF/0hRMj4UcATEmP8HIfGdjty0iM/OF9uugp2pwdWe2/E90XCjwKekBjTYXXC1etDblrkAz7n6j76+/xJbKGAJyTG9N98lB3NgKd++JhEAU9IjGm+elVLpjbyg4DJE8RIU8vRREfwMYkCnpAY02KyIzVZFrEhCq6Vk66kI/gYRQFPSIzw+gGH24umdjsytAo43F5EY36c3DQlTJ1OON3e4I2JoFDAExIj3B4vTlxsQ5ulBwBwqtIIr98fsf0xLAOH24s0jQIcgMutXfBGbnckAijgCYkhNkcvOA5IUYZ/iOBruT0+nKo0oqOrb1an4+cNcHvoKD6WUMATEkOs3W4AgFoV+YDvp0gQI0EiQqfNHbV9kvCggCckhljtbjAMkJQ4/NzIkcAwDFKSEmDppoCPNRTwhMSQzm43khOlELGRHaLgWhpVAqzdbviicVaXhA0FPCExxGrvhToK/e/X0iTJ4PNzMNDY8DGFAp6QGOHu9cHu9ES1/71f2tVRK6+02qK+bzJ2FPCExIj+o2e1Mnr97/0S5WIoZGLUtnRFfd9k7CjgCYkR/QGfwsMRPMMwSFPLcaWli4YOjiEU8ITEiNaOHohFDJRyCS/7T0uRw2rvhbnLxcv+yegFDfjOzk5s2rQJhYWFWL16NX72s5/BYrEMaed0OvH4449jxYoVWLlyJT7++OOIFEzIRNXa4UCyMgFMhCf5uJ60lL5++Jpm6qaJFUEDnmEYbNy4ER988AEOHTqEnJwc/O53vxvS7k9/+hOUSiX+8Y9/4MUXX8SOHTvgcNAZd0LCxWB28NL/3k+tSoA8QYSaZitvNZDRCRrwarUat956a+Dn+fPno7W1dUi79957D+vWrQMATJ48GTfeeCM+++yzMJZKyMRl6+lFd48nKkMUXA/LMJiiT0I1HcHHjFH1wfv9frz11lu46667hqxrbW1FVlZW4Ge9Xo+2trbxV0gICUzywcclkgNNzUpGa4cDdqeH1zpIaEY1oPQzzzwDhUKBhx9+OCLFaLWRn6HmenQ6FW/7Hg+qO3r4rPnEJRMAIDs9CYkDTrJKJGKolLIh7Qcu7/87lLbBlmfoVDh8vB7tNjem5GrG/oKCiMXPByC8ukMO+JKSEjQ0NODFF18Eyw498M/MzERLSws0mr5/dIPBMKhrJxRmsx1+Hm6F1ulUMJlib8Yaqjt6+K65qt4MhUwMn9eLbrsvsNzj8aLbPvSqlv7lKqUssD5Y21CWT9OrIBYxOHnBgKkRmvSb7/d6rPiom2WZEQ+MQ+qi2bNnDy5cuIB9+/ZBKh3+JM/KlSvx9ttvAwDq6+tx/vx5LF26dAwlE0Ku1WyyIzM1kbcraPpJxCzyc1NwrtbMax0kNEEDvqamBi+99BLa29vxwAMPoKioCI899hgAoKioCEajEQDwk5/8BDabDStWrMBPf/pT7Nq1C0olf10uhMQLjuPQYnJAH4U5WEMxd5oWbZYeGDt7+C6FBBG0i2bGjBmoqqoadt3BgwcDjxUKBZ5//vnwVUYIAQBYbG64en3I1Cr4LgUAMG+aFm+V1+DcZTNW3CKMmsjw6E5WQgSu2dQ34XVmqjCO4NNSFNBrFfi2toPvUkgQFPCECFxLR98lkhkC6KLpn6d19mQNqhqtMHe74HB7aa5WgaKAJ0TgWkx2pKgSoJCN6qrmiOifp1XEMvD5ORw+Xo9TlUaaq1WgKOAJEbhmkwPZOmFdsJCWIodEzAZuwCLCRAFPiID5/H4YzA5k6fjvnhmIZRlkpiai2WSn4YMFjAKeEAFrszjh9XHIEdgRPABk6xLh6vXBbKPJuIWKAp4QAWtu77uCJjtNeAHf/1tFf41EeCjgCRGwZpMdIpaBXiDXwA8kk4qhU8uoH17AKOAJEbCmdjv0WgXEImF+VbN1SphtLnTZqZtGiIT5qSGEAOg7ghdi90y//m6aivpOnishw6GAJ0SgHC4PLDa3IE+w9uu/Pv9CHQ0+JkQU8IQIlJBPsPZjGAbZukRUNVjhodtZBYcCnhCBar568lJoNzldK1unhNvjQzXN1So4FPCECFRTux1KuYTXibZDka5RgGUZVNRb+C6FXIMCnhCBajbZka3jf5KPYCRiFpMzVKikE62CQwFPiAD5r07yIeT+94Hyc9VoaOuGw0WTcQsJBTwhAtRhdcLt8Qm+/71fXk4KOACXGqgfXkgo4AkRGK8fqGnpAgCkquVwuL1wuL3gYT76kE3WqyCVsKhsoH54IeF/gGlCyCBujxdfVRjBMIChw472q3OfzsvT8VzZ9YlFLPJy1KhsoH54IaEjeEIEyNzlQooqASKBDlFwLYZlMD0rGQZzD1rMDprlSSBi49NDyATCcRzMNhe0STK+SwmZ2+ODx9eX6O+faKBZngSCAp4QgenocqHX44c2OXYCHgA0qgRIJSzazD18l0KuooAnRGAajd0AEHMBzzAMMjQKGDudfJdCrqKAJ0Rgmox2sCyDFGUC36WMWlqKHHanBw4nXQ8vBBTwhAhMo7EbGlUCWFbYd7AOJz2lb2ISOooXhpACvqSkBHfddRfy8/NRXV09bJu9e/diyZIlKCoqQlFREXbu3BnWQgmZCPwch6Z2e8x1z/RLSUqARMQGLu0k/ArpOvjly5fjkUcewUMPPTRiu+LiYmzbti0shREyERktPXD1+mLqCpqBWIaBLkVOR/ACEdIR/MKFC6HX6yNdCyETXr0hNk+wDpSeIkeXvRd26ofnXVj74I8cOYLVq1djw4YNOHPmTDg3TciEUNdmg1TMIlngQwSPJE0jBwDUXh1ugfAnbEMVPPDAA9i8eTMkEgmOHz+OLVu2oKysDCkpKSFvQ6vlb2AlnU7F277Hg+qOnmjU3NTuwCR9EpJV8iHrJBIxVEpZ0GXXLu//O5S24difQi6FiG1GY7sdRXeO7T2Lxc8HILy6wxbwOt1342Tcfvvt0Ov1qKmpwaJFi0Lehtlsh5+HEZV0OhVMpu6o73e8qO7oiUbNrl4vLjdbsXxhNrrtriHrPR7vkOXDLRu4XKWUBdYHaxuO/fVLTZbhUr1lTO9ZLH4+AH7qZllmxAPjsHXRGI3GwOPKykq0tLRgypQp4do8IXGvttUGn5/D9KxkvksZtzSNAk3tdjjdNFwBn0I6gn/22Wdx9OhRdHR0YP369VCr1Thy5Ag2bdqErVu3Ys6cOdizZw8uXrwIlmUhkUiwe/fuQUf1hJCRVTdawTIMpmQm4Xytme9yxiU9RY7zHFDb2oUbp2j5LmfCCingd+zYgR07dgxZ/sorrwQel5SUhK8qQiagqiYrJmUoIZPG/ijeOrUcLANUN1kp4HlEd7ISIgAerw9XWm3Iy1HzXUpYSMQsstOUqG6iK2n4RAFPiADUGbrh9fnjJuABYFpWMq602uChgeF5QwFPiABUNVnBAJiRHT8BPz07GV6fH3UGG9+lTFgU8IQIQHWTFVk6JZRyCd+lhM20zL6rgaqbaCJuvlDAE8Izn9+Py81dyI+j7hkASJRLkJWaiOpmCni+UMATwrPaFhvcHh/yc+Mr4AFgRo4al5u7eLmBkVDAE8K781fMYBkGsydr+C4l7PJykuHq9aGp3c53KRMSBTwhPDtXa8aM7GQoZLF//fu18q6eNK6ifnheUMATwiNTlxtN7XbMnJQCh9sLh9uLeOnNYFgGCQliaJNkqGiwBF4fXTUZPfF3yEBIDPn2cjuAvpmcTlX2jec0Ly8+hvhwe3z4ttoEtVKKqgYrTla0gWEY3DIrHeIEip5ooCN4Qnh0sa4TCpkY6hge/z2YNI0cbo8PXY5evkuZcCjgCeGJ1+dHVWMnslITwTCxN8F2qPon4m6nafyijgKeEJ7UNHfB1etDli6R71IiSqWQQCYVwWihibijjQKeEJ58e7kDIpaBXhvfAc8wDNI1CjqC5wEFPCE84DgOp6tMmDkpBRJx/H8N01LkcLi8NBF3lMX/J4sQAapv64bZ5sL8Gal8lxIV6Sl9c8y2d1I3TTRRwBPCg6+r2iFiGcyZNjEmw1CrEiAVszBaqJsmmijgCYmygd0zibL4GT1yJCzDQJcih5H64aOKAp6QKGtqt6O904mF+fFxQ1OoMjQK2By96Ox2813KhEEBT0iUna4ygWGAm+LkjtVQZab2XQ9f1djJcyUTBwU8IVHEcRy+rmpHfo4aSYr4vXt1OGplAmRSES41UMBHCwU8IVHUaLTDYO7BLbPS+S4l6hiGQWZqIqoarfBzcTKimsBRwBMSBV4/4HB7cey8ASzL4IYpmrgaOTJUeq0CdqcHTUYaHz4aKOAJiQK3x4uvKtrw5YU2ZGoVqKi34FSlEV7/xBo7t/+u3Yp6C8+VTAwU8IREidHSA6fbiymZSXyXwhuFTAy9VoGLFPBRETTgS0pKcNdddyE/Px/V1dXDtvH5fNi5cycKCgqwYsUKHDhwIOyFEhLr6lq7IRYxyElT8l0Kr2ZOSkF1Uxd6PT6+S4l7QQN++fLlePPNN5GVlXXdNocOHUJjYyOOHj2Kt99+G3v37kVzc3NYCyUklnm8fjQYu5GbroJYNLF/cZ45KQVenx/VNI1fxAX9pC1cuBB6vX7ENmVlZVi7di1YloVGo0FBQQHef//9sBVJSKw7X2uGx+vH1AncPdNvelYyJGIW52rNfJcS98Iyb5bBYEBmZmbgZ71ej7a2tlFvR6vl71dXnU7F277Hg+qOnvHU/FXlWSgVEsyYpAE7YHIPiUQMlVI2qO1wy8bTtv/v0Ww3krWpkxWYN0OHi/WdSE1VDjvZSSx+PgDh1S2oiRHNZjv8PFw3ptOpYDJ1R32/40V1R894am7v7EFFnQXzp2vhcAy+Td/j8aLb7gq6bKxtVUpZYP1othvJ2np63JiZk4yvK404X2UcMh5+LH4+AH7qZllmxAPjsHQG6vV6tLa2Bn42GAzIyMgIx6YJiXmffWsAywDTs5P5LkUw5k7tG0XzPHXTRFRYAn7lypU4cOAA/H4/LBYLysvLUVhYGI5NExLTvD4/Pj/XihumaqGYICNHhiJVLUdmaiK+pYCPqKAB/+yzz2LZsmVoa2vD+vXrce+99wIANm3ahPPnzwMAioqKkJ2djbvvvhs//OEP8dhjjyEnJyeylRMSA87WdMDW48Htc0a+UGEiYVgGDrcXsyaloLrJCnO3C96Jdb9X1ATtg9+xYwd27NgxZPkrr7wSeCwSibBz587wVkZIHPjom2ZokhIwa1IKTle1812OILg9PnxbbYJIxMDn53D4eD3+1/enQ5wgqFOCcWFiX5BLSARVN1lxqdGKu2/JBcsOvVJkoktTyyERs2g20bg0kUIBT0iEHP6iHiqFBHfMzwzeeAJiWQZZqYloMTl4uXpuIqCAJyQCrrTacKHOgsJFuUiQiPguR7By0pVw9fpQb7DxXUpcooAnJAIOf1GPRJkY37/p+kN8ECBLlwiWAd3VGiEU8ISEWWW9BWcvd2DFwhzI6cThiKRiETK0Cpyr7QBHk4CEHQU8IWHkdHvxatklpGsUKLw1l+9yYkJOmgomqwut5h6+S4k7FPCEhNGBjy/D0u3CT+6dRX3vIeofPvlMtYnnSuIPBTwhYXKhzoxPzraicFEupmfRsAShUsjEmJShwpmaDr5LiTsU8ISEQY/Liz+/dwl6rQL3LZ3CdzkxZ+40LeoMNlhsQwcnI2NHAU/IGPVPpO1we/FmeTU6u9148O48cIwosLz/D13mPbL5M1IBAKerqJsmnOgUPyFj5PZ4carSiBaTHV9eaMONUzQwdTqRqevFt9f0J8/L0/FUZWxIS1EgJ02JU5fa8eA9s/kuJ27QETwh4+D2+PDFBSPUSinmzdDyXU5Mu2VmGi63dMHU6eS7lLhBAU/IOHxd2Q5Xrxe3zdFDxNLXaTxumZkGADh+rjVISxIq+kQSMkbna82obbXhxqlapCYPnZqOjE66RoHcNCU+/7aF71LiBgU8IWNgd3rwl/IapKgSMHcadc2MV/8Y8fNmpKKqoRNNJjscbi+NEz9OFPCEjMH/lFfD7vLgtjkZENFQwOPm9vhwqtIYeC///nkdTlUa4fZ4ea4stlHAEzJKp6tMOHHRiMJFudAmUddMOCUlSpGuUaC2pYvGpgkDCnhCRqG7pxevf3AJuWlKFC6iaSkjYeakFFjtvbDY3HyXEvMo4AkJEcdxeO2DKjhcXvxk1WyIRPT1iYQZOSkQsQwut3TxXUrMo08oISH65EwLTleZsGbZ1MAAWST8EqQi5KQrUWewwUNnWceFAp6QEFxp6cJbH17GnKlaGgY4CqZnJaPX48eFKzQRyHhQwBMSRI/Lg92vn4JSLsZPVs0Cy9BVM5GWoVVAIRPjxMU2vkuJaRTwhIzA6/Nj79/Oo83cg//9g5kQiVgaQCwKWIbB9KxkVNZ3wthJE4GMFQU8IdfBcRz2l11CVaMV3785G53dbpyqNAb+eP3UPxxJeTlqsCyD8q+b+S4lZoUU8HV1dVi3bh0KCwuxbt061NfXD2mzd+9eLFmyBEVFRSgqKsLOnTvDXSshUcNxHN7+6DK+vNiGe5dMQv4kDd8lTTgKmRg35+vw+TkDelwevsuJSSEF/NNPP40HH3wQH3zwAR588EE89dRTw7YrLi7GwYMHcfDgQTz99NNhLZSQaOE4Dm99WIOjp5qwfEE2nVTl0fcXZMPt8eGzbw18lxKTgga82WxGRUUFVq1aBQBYtWoVKioqYLFYIl4cIdHm93N442g1yr9uxoqFOXhwxQwwdFKVN9lpSszMVePD003wUZfYqAUNeIPBgPT0dIhEfRMIi0QipKWlwWAY+n/UI0eOYPXq1diwYQPOnDkT/moJiaBejw/73r2Aj8+0oGBhNlZ/bzJ6en10MpVnK27JgdnmxsnKdr5LiTlhm9HpgQcewObNmyGRSHD8+HFs2bIFZWVlSElJCXkbWi1/N4/odCre9j0eVHd4WLvd2P3WV6hq6MT35mUif7IWVc02AED+pL7PsEo5eNwZiUQ8ZNn1lvPZtv/v0WyX79fR/1ihSEDB4ik49EUDyk404N6l0wR9B7HQPtdBA16v18NoNMLn80EkEsHn86G9vR16vX5QO53uuynJbr/9duj1etTU1GDRokUhF2M22+Hn4XBJp1PBZOqO+n7Hi+oOj4a2buz92zl093iwYdVseLw+dNu/m/zZc3VEw4HL+pdfu+x6y/lqq1LKAutHs10+X8fAmnt63DD7fLh38STse/c8Dn16GbfP0Q/ZphDw8blmWWbEA+Og/yvUarWYNWsWDh8+DAA4fPgwZs2aBY1m8FUFRqMx8LiyshItLS2YMoVmlyfCduJiG5574zQ4Dvi/Dy8ITP5MhKF/nPj8SWpkpylx8PM6uGn4gpCF1EXz7//+79i+fTteeOEFJCUloaSkBACwadMmbN26FXPmzMGePXtw8eJFsCwLiUSC3bt3DzqqJ0RIHEeXXJIAABKoSURBVG4f/ucfVfjyQhumZibhJ6tmIylRSv3tAuP2+AITmM/ITsbH37Tg83OtWL4gm+fKYkNIAT9t2jQcOHBgyPJXXnkl8Lg/9AkRutrWLrx6pBIGcw9unKrB/OmpqGrsBADMy6ODEqHK1iVCmyRD2ZcNWDpHD6lExHdJgifcsxWEhFmPy4M3j1bjN6+dhtPtxfKbs7EgTweWZmSKCQzD4OZ8HTq73Sg/TXe3hiJsV9EQIlR2pwf/ONWE8tNNcLl9uOvmvpuXaKTC2JOhVeDGKRoc+bIeS+fqoVJI+S5J0CjgSdyqb7Pho29a8FWFER6vH/Onp6JwcS6ydUrqa49hRUun4rnXT+Pvx+vx0Io8vssRNAp4Encut3Th4Od1uFhngVTCYtGsNGiSZEhRJcDQ4YChw0F97TEsQ6vAsnl6fHKmBXfelIWs1ES+SxIsCngSN2w9Hvz3+5dwptoEpVyCoqVTcPscPRKkYpy+ZAy+ARIzipdNxalL7Xjjgyr88sGbaDiJ66CTrCQuXGm1YdefT+JsjQnzp2ux+vbJSE6U4sIVMw3rG4eSFFLcf+c0VDVZ8SVNCnJdFPAk5p293IHfvtl3s9LKRbmYOz0VEjF9tOPdsnmZmJqZhLc/ugwHDSc8LPoWkJh2tqYD+/52Hlk6JbY9tAC6FDnfJZEI67+71dnrw9rvT4fd6cFrH1SBbnAdivrgScw6U23CC6UXkJuuxP9ZNx8c9cNOCAPvbgWAOVO1OFXZjtmTW3HHvEweKxMeOoInMenUJRP2lV5ATpoSm4vngGMYuvRxgpo7TQudWoa3P6yByerkuxxBoYAnMefrS+146e8XoFElYPEN6bhYZ6Y5UicwlmWwdG7fkfuLBy+i1+PjuSLhoIAnMYPjOHxwshF/KL2ASekqFNySTeOREACAUiHBw4X5qDfY8PKhCl6GHRciCngSE3o9Prz2QRXe/ugybs7X4Wf3z4FUTOFOvjM/T4c1d0zDN9UmvH60CnaXBw63d0KffKWTrETwztZ04H/Kq9HR5cK9SybhvmVT4eylX8PJYG6PD4lyMWZNSsGnZ1th7nLhprxULJqdAXHCxIy6ifmqieA53V6cutSOY+cMqG3pQoZGgZ//r7nIy1HDSfOkkhEsnKmDz+/HhToLPD4/Fs5K57sk3lDAE8HwcxyqG634/LwBX1e1o9fjR7pGjltmpiE/V40uuxunKvuGHKCxZMj1MAyDW2enQyxiUVHfiT+XXcKjq2YjQTrxuvQo4AnvLDYXPj9vwOfnDeiwuiCTinDLzDTcekMGctNV+Kaqne8SSYzpHzteJhXhTLUJv379NH52/xykqSfWjXAU8IQXHMehtsWGo1834ZsqE/wch7wcNWbmqpGbroJYxKLD6kRW2vUnFCZkJAzD4MapWtx6Qwb+u+wSdu0/hR8V5uPW2ROny4YCnkRVr8eHf3zVgNJPL6PRaIciQYy7F+Xg+zdlQSGXBLpgCAmX2ZM1eOrHC/HyoQq89PeL+KbahIfvzpsQk4VQwJOoaO/swSdnWnHsXCscLi/0WgV+eNd0LJqVHugbpROnJBIYlkGiQoqta+fhw6+bUPZlA6qarPjxD2Zi/vRUvsuLKAp4EjGmLje+qjDgmyoT6tu6wTLA3OmpuPPmHJg7HWAYBudqOwLt6cQpiYSBY9ekqBLwg8W5OFPTgef/3zksviEd9y2bBoWsLwoTJGLE00CkFPAkrLrsbnxdZcLJSiNqmrsAAJqkBNyUl4qpmUlIlEkwNSsZFmsPz5WSiUqTJMPWtfPw+vuX8NVFI85dNmPxjenI1ilxy6z0uLpmPn5eCeGN38/hVJUJH3/TjJpmKzgO0GsVuGfJJEjFLJIS47+vk8QWsZjFgjwdctOV+OJ8Gz463YLJGSrk5aiRSAFPCNDj8uLYuVZ8eLoZHV0uKOUSzJmqxWS9CmplAubl6QYN60qI0KQmy3HvbZNw4YoF569Y8Myfv8Y9SyZh+YIsKGQSvssbNwp4MmqNxm4cO9d33bq714e8HDWKl02Fq9cLlsZkJzFGxLKYN72vC7GmuQvvfnYF751owLJ5mVg4Mw1T9Ulg2dj8XIcU8HV1ddi+fTusVivUajVKSkowefLkQW18Ph+effZZHDt2DAzD4NFHH8XatWsjUTOJIq/PD5ujF03tdtQ0d+HcFTOa2+0QixjcNEOH7y/IQk66Cn4ONLE1iWkqhRQ/LboRZqsT733ViA9PN+PoqSaoFBJM1SchS6dEli4R2TolMjSKmJgWMqSAf/rpp/Hggw+iqKgIBw8exFNPPYXXXnttUJtDhw6hsbERR48ehdVqRXFxMZYsWYLs7OyIFE7GhuM4eH1+2J1e2By96HK40eXovfq47++Bjx0ub+C5IpZBbroKi2anYUpGEhKkIrRZetBm6aErYEhcYFgGWrUcDxfmY80d01DZYEFlvRVN7d24UGeB7+q1vCzTd7I2PUUOXYoCaWo5pk/SQCYCdGo5EgQyjHXQgDebzaioqMD+/fsBAKtWrcIzzzwDi8UCjUYTaFdWVoa1a9eCZVloNBoUFBTg/fffx8aNG0MuZiy/BjndXpyuMsHr84MDAO67i6k5Dn3LBv736jJuQDuZXIqenl4MaDrg+VzgOQg8nwv83L8ZbmCjAcsG7rO/MRd4yA14/qAdDLP9gdvtq18qFcPt9g6ojRu4CfR6fXB7/HB7vHC7fXB7+v74rnPBeYJEhES5GEqFFBnaRKgUUihkYijlEmiTZcjSJUIsEuH8gEsb+4lF7JA+y+GW9S1nrrN8NNuIblt5ghg+rzBrG6ntwLpHs10+X0ewmiP5Onx+DpV1lsDPMqkYDxbmobLOAr+fg93Zd/DT3eOBRCyCxeZCncGGinoLPjnbEnieXCpGokIClVwCpVyCRJkYYrEIYpaBSMRALGLAsiwYAFKJCAvy+oZVGK1gmclwA5NuGBcuXMC2bdtw5MiRwLJ77rkH//Ef/4EbbrghsGz16tX49a9/jblz5wIAXnnlFRiNRuzYsWPURRNCCBk/4XciEUIIGZOgAa/X62E0GuHz9U2w4PP50N7eDr1eP6Rda2tr4GeDwYCMjIwwl0sIISRUQQNeq9Vi1qxZOHz4MADg8OHDmDVr1qD+dwBYuXIlDhw4AL/fD4vFgvLychQWFkamakIIIUEF7YMHgNraWmzfvh02mw1JSUkoKSnB1KlTsWnTJmzduhVz5syBz+fDrl27cPz4cQDApk2bsG7duoi/AEIIIcMLKeAJIYTEHjrJSgghcYoCnhBC4hQFPCGExCkKeEIIiVMTJuAPHjyI1atXY/bs2XjjjTdGbPvXv/4VK1asQEFBAXbt2gW/3x/SunBzOp14/PHHsWLFCqxcuRIff/zxsO1ee+01FBUVBf4sWLAAzz33HADgq6++wrx58wLrojEAXKh1B6tt3759KCgoQEFBAfbt2yeYusvLy7FmzRqsWrUK9957L1599dXAur/97W9YuHBh4DU99thjYa+zrq4O69atQ2FhIdatW4f6+vohbXw+H3bu3ImCggKsWLECBw4cCGldJIVS9759+3Dvvfdi9erVWLNmDY4dOxZYt337dixbtizw3v7hD38QTN179+7FkiVLArXt3LkzsC7Uz1VEcBNEVVUVV1NTwz355JPc66+/ft12jY2N3NKlSzmz2cz5fD5uw4YN3Lvvvht0XSTs3buX+9WvfsVxHMfV1dVxt912G2e320d8Tm9vL7d48WLu3LlzHMdx3IkTJ7j77rsvYjUOJ9S6R6rt5MmT3KpVqzin08k5nU5u1apV3MmTJwVR99mzZ7m2tjaO4zjOZrNxBQUF3KlTpziO47h33nmH+/nPfx7ROn/0ox9xpaWlHMdxXGlpKfejH/1oSJt3332X27BhA+fz+Tiz2cwtXbqUa2pqCrqO77o/++wzrqenh+M4jqusrORuvvlmzul0chzHcdu2bRvxuxspodT9/PPPc7/97W+Hff5YvsfhMmGO4PPy8jB9+nSw7Mgv+YMPPkBBQQE0Gg1YlsXatWtRVlYWdF0kvPfee4F7CSZPnowbb7wRn3322YjP+fjjj6HT6TBnzpyI1RXMWOq+VllZGYqLiyGTySCTyVBcXBzR9xoIve558+YhPT0dAKBSqTBt2jS0tLQMaRcJ/YP/rVq1CkDf4H8VFRWwWCyD2l1v8L9g6/iue+nSpZDL5QCA/Px8cBwHq9Ua0dpGEmrdIwnH92GsJkzAh8pgMCAzMzPwc2ZmJgwGQ9B1kdDa2oqsrKzAz3q9Hm1tbSM+55133sGaNWsGLauvr8d9992HtWvX4t13341IrQONpu7r1Xbte63X6yP6XgNje79ra2tx9uxZLF68OLDs5MmTKCoqwkMPPYRPPvkkrDUaDAakp6dDJOobeVAkEiEtLW3IezPc+9f/WkZaFymh1j1QaWkpcnNzBw15sn//fqxevRpbtmxBbW1tRGsebd1HjhzB6tWrsWHDBpw5cyawfCyfq3CJmxmd7rvvvkFj4Qz0xRdfBP6BhCRYzaPV3t6OEydOBPrfAeCGG27Ap59+CpVKhaamJqxfvx7p6em47bbbeK87ErWNJBLv95YtW/D0008HjujvvPNO3HPPPZDJZKioqMCmTZvw2muvYdq0aeOqfaI5efIkfv/73w86v/HEE09Ap9OBZVmUlpZi48aNKC8vF8R3+4EHHsDmzZshkUhw/PhxbNmyBWVlZUhJSeG1rrgJ+HAdmV47aFpra2tgYLWR1o1FsJozMzPR0tISGPfHYDDg1ltvvW770tJS3HHHHYPGCVIqlYHHOTk5KCgowDfffDOuEA1X3SPVNtzgdeN5r8NZN9D3q/v69euxceNG/OAHPwgsH/jez549GwsWLMC5c+fCFvADB/8TiURBB//rH7574FH7SOsiJdS6AeDMmTN48skn8cILL2Dq1KmB5f3/EwWA4uJiPPfcc2hraxt0dMxX3TrddxPe3H777dDr9aipqcGiRYtG/T0OJ+qiuUZhYSHKy8thsVjg9/tx4MCBwBd4pHWRsHLlSrz99tsA+royzp8/j6VLl163/TvvvIP7779/0LL29vbA5CZWqxXHjx/HzJkzI1YzEHrdI9W2cuVKlJaWwuVyweVyobS0NKLv9Wjq7uzsxPr16/HQQw8NufLHaPxu2sKWlhacPXsW+fn5YasxHIP/8TEwYKh1nzt3Dk888QSef/75QfNNAIPf22PHjoFl2UGhz2fdA2urrKxES0sLpkyZAmD03+NwmjBj0Rw+fBi7d++GzWaDRCKBXC7Hq6++iunTp+P3v/890tLS8E//9E8AgL/85S/44x//CKDv/8ZPPfVU4NfAkdaFW09PD7Zv347KykqwLIsnn3wSBQUFADCk5tOnT+Pxxx/HJ598MqieN954A2+99RbEYjF8Ph+Ki4tHNctWJOsOVtvevXtRWloKoO+I7ec//7kg6i4pKcGbb74Z+AIDwCOPPIL7778fe/bswYcffhj4N1i/fj3uu+++sNY53sH/+BoYMJS677//frS0tAwK7t27dyM/Px8//vGPYTabwTAMlEolfvnLX2L+/PmCqHvbtm24ePEiWJaFRCLB1q1bcccddwAY+XMVaRMm4AkhZKKhLhpCCIlTFPCEEBKnKOAJISROUcATQkicooAnhJA4RQFPCCFxKm7uZCVkLA4dOoT9+/ejrq4OiYmJmDlzJjZv3oyFCxeirq4O//mf/4mvvvoKXq8XmZmZWLNmDR555BEYDAYsX74cCoUisK2cnBz8/e9/5/HVEDIYBTyZsPbv34+XX34ZO3fuxPe+9z1IJBIcO3YMH374IdLS0vDDH/4Qa9aswaFDh5CWloYrV65g3759cDgcgW2cOnUKYjF9jYgw0Y1OZELq7u7GsmXL8Jvf/GbYIRB+8YtfwGaz4eWXXx72+c3NzVi+fDkuXrxIAU8Ei/rgyYR05swZuN1urFixYtj1X375ZcTHZyEk0ijgyYRktVqRkpJy3aNvq9U6aITA61m8eDEWLlyIhQsX4k9/+lO4yyRkXOh3SzIhqdVqdHZ2wuv1DhvyarUaJpMp6HZOnDhBXTREsOgInkxIN910E6RSKcrLy4ddv2TJEhw9ejTKVRESXhTwZEJSqVTYunUrdu3ahfLycjidTng8Hnz66afYvXs3tm7dijNnzqCkpCRwJN/Q0BA4+UpILKDfLcmEtWHDBqSmpuKFF17AL37xCyQmJuKGG27A5s2bkZubi7/85S/4r//6L6xatQperxdZWVlYs2YNEhMTKeRJTKDLJAkhJE5RFw0hhMQpCnhCCIlTFPCEEBKnKOAJISROUcATQkicooAnhJA4RQFPCCFxigKeEELiFAU8IYTEqf8PAMEPgaXr9nIAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "2NCV2J5OVqNn",
        "colab_type": "text"
      },
      "source": [
        "# Expected Loss"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pOaf27QqB4HN",
        "colab_type": "text"
      },
      "source": [
        "**BURADA PD MODEL DOSYASINI IMPORT ETMEZSEK HATA VERECEKTİR. PD MODEL KAYIT DOSYASINI KLASÖRE KOYMAYI UNUTMA.**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "GaAYqtOQVqNn",
        "colab_type": "code",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 545
        },
        "outputId": "90ac3a14-4a28-4608-b90f-072cb7726044"
      },
      "source": [
        "loan_data_preprocessed.head()\n",
        "\n",
        "loan_data_preprocessed['mths_since_last_delinq'].fillna(0, inplace = True)\n",
        "# We fill the missing values with zeroes.\n",
        "\n",
        "loan_data_preprocessed['mths_since_last_record'].fillna(0, inplace = True)\n",
        "# We fill the missing values with zeroes.\n",
        "\n",
        "loan_data_preprocessed_lgd_ead = loan_data_preprocessed[features_all]\n",
        "# Here we keep only the variables we need for the model.\n",
        "\n",
        "loan_data_preprocessed_lgd_ead = loan_data_preprocessed_lgd_ead.drop(features_reference_cat, axis = 1)\n",
        "# Here we remove the dummy variable reference categories.\n",
        "\n",
        "loan_data_preprocessed['recovery_rate_st_1'] = reg_lgd_st_1.model.predict(loan_data_preprocessed_lgd_ead)\n",
        "# We apply the stage 1 LGD model and calculate predicted values.\n",
        "\n",
        "loan_data_preprocessed['recovery_rate_st_2'] = reg_lgd_st_2.predict(loan_data_preprocessed_lgd_ead)\n",
        "# We apply the stage 2 LGD model and calculate predicted values.\n",
        "\n",
        "loan_data_preprocessed['recovery_rate'] = loan_data_preprocessed['recovery_rate_st_1'] * loan_data_preprocessed['recovery_rate_st_2']\n",
        "# We combine the predicted values from the stage 1 predicted model and the stage 2 predicted model\n",
        "# to calculate the final estimated recovery rate.\n",
        "\n",
        "loan_data_preprocessed['recovery_rate'] = np.where(loan_data_preprocessed['recovery_rate'] < 0, 0, loan_data_preprocessed['recovery_rate'])\n",
        "loan_data_preprocessed['recovery_rate'] = np.where(loan_data_preprocessed['recovery_rate'] > 1, 1, loan_data_preprocessed['recovery_rate'])\n",
        "# We set estimated recovery rates that are greater than 1 to 1 and  estimated recovery rates that are less than 0 to 0.\n",
        "\n",
        "loan_data_preprocessed['LGD'] = 1 - loan_data_preprocessed['recovery_rate']\n",
        "# We calculate estimated LGD. Estimated LGD equals 1 - estimated recovery rate.\n",
        "\n",
        "loan_data_preprocessed['LGD'].describe()\n",
        "# Shows some descriptive statisics for the values of a column.\n",
        "\n",
        "loan_data_preprocessed['CCF'] = reg_ead.predict(loan_data_preprocessed_lgd_ead)\n",
        "# We apply the EAD model to calculate estimated credit conversion factor.\n",
        "\n",
        "loan_data_preprocessed['CCF'] = np.where(loan_data_preprocessed['CCF'] < 0, 0, loan_data_preprocessed['CCF'])\n",
        "loan_data_preprocessed['CCF'] = np.where(loan_data_preprocessed['CCF'] > 1, 1, loan_data_preprocessed['CCF'])\n",
        "# We set estimated CCF that are greater than 1 to 1 and  estimated CCF that are less than 0 to 0.\n",
        "\n",
        "loan_data_preprocessed['EAD'] = loan_data_preprocessed['CCF'] * loan_data_preprocessed_lgd_ead['funded_amnt']\n",
        "# We calculate estimated EAD. Estimated EAD equals estimated CCF multiplied by funded amount.\n",
        "\n",
        "loan_data_preprocessed['EAD'].describe()\n",
        "# Shows some descriptive statisics for the values of a column.\n",
        "\n",
        "loan_data_preprocessed.head()\n",
        "\n",
        "loan_data_inputs_train = pd.read_csv('/content/GoogleDrive/My Drive/Regaip/Kurslar/Credit Risk Modelling/Dataset/loan_data_inputs_train.csv')\n",
        "# We import data to apply the PD model.\n",
        "\n",
        "loan_data_inputs_test = pd.read_csv('/content/GoogleDrive/My Drive/Regaip/Kurslar/Credit Risk Modelling/Dataset/loan_data_inputs_test.csv')\n",
        "# We import data to apply the PD model.\n",
        "\n",
        "loan_data_inputs_pd = pd.concat([loan_data_inputs_train, loan_data_inputs_test], axis = 0)\n",
        "# We concatenate the two dataframes along the rows.\n",
        "\n",
        "loan_data_inputs_pd.shape\n",
        "\n",
        "loan_data_inputs_pd.head()\n",
        "\n",
        "loan_data_inputs_pd = loan_data_inputs_pd.set_index('Unnamed: 0')\n",
        "# We set the index of the dataframe to the values of a specific column. \n",
        "\n",
        "loan_data_inputs_pd.head()\n",
        "\n",
        "features_all_pd = ['grade:A',\n",
        "'grade:B',\n",
        "'grade:C',\n",
        "'grade:D',\n",
        "'grade:E',\n",
        "'grade:F',\n",
        "'grade:G',\n",
        "'home_ownership:RENT_OTHER_NONE_ANY',\n",
        "'home_ownership:OWN',\n",
        "'home_ownership:MORTGAGE',\n",
        "'addr_state:ND_NE_IA_NV_FL_HI_AL',\n",
        "'addr_state:NM_VA',\n",
        "'addr_state:NY',\n",
        "'addr_state:OK_TN_MO_LA_MD_NC',\n",
        "'addr_state:CA',\n",
        "'addr_state:UT_KY_AZ_NJ',\n",
        "'addr_state:AR_MI_PA_OH_MN',\n",
        "'addr_state:RI_MA_DE_SD_IN',\n",
        "'addr_state:GA_WA_OR',\n",
        "'addr_state:WI_MT',\n",
        "'addr_state:TX',\n",
        "'addr_state:IL_CT',\n",
        "'addr_state:KS_SC_CO_VT_AK_MS',\n",
        "'addr_state:WV_NH_WY_DC_ME_ID',\n",
        "'verification_status:Not Verified',\n",
        "'verification_status:Source Verified',\n",
        "'verification_status:Verified',\n",
        "'purpose:educ__sm_b__wedd__ren_en__mov__house',\n",
        "'purpose:credit_card',\n",
        "'purpose:debt_consolidation',\n",
        "'purpose:oth__med__vacation',\n",
        "'purpose:major_purch__car__home_impr',\n",
        "'initial_list_status:f',\n",
        "'initial_list_status:w',\n",
        "'term:36',\n",
        "'term:60',\n",
        "'emp_length:0',\n",
        "'emp_length:1',\n",
        "'emp_length:2-4',\n",
        "'emp_length:5-6',\n",
        "'emp_length:7-9',\n",
        "'emp_length:10',\n",
        "'mths_since_issue_d:<38',\n",
        "'mths_since_issue_d:38-39',\n",
        "'mths_since_issue_d:40-41',\n",
        "'mths_since_issue_d:42-48',\n",
        "'mths_since_issue_d:49-52',\n",
        "'mths_since_issue_d:53-64',\n",
        "'mths_since_issue_d:65-84',\n",
        "'mths_since_issue_d:>84',\n",
        "'int_rate:<9.548',\n",
        "'int_rate:9.548-12.025',\n",
        "'int_rate:12.025-15.74',\n",
        "'int_rate:15.74-20.281',\n",
        "'int_rate:>20.281',\n",
        "'mths_since_earliest_cr_line:<140',\n",
        "'mths_since_earliest_cr_line:141-164',\n",
        "'mths_since_earliest_cr_line:165-247',\n",
        "'mths_since_earliest_cr_line:248-270',\n",
        "'mths_since_earliest_cr_line:271-352',\n",
        "'mths_since_earliest_cr_line:>352',\n",
        "'inq_last_6mths:0',\n",
        "'inq_last_6mths:1-2',\n",
        "'inq_last_6mths:3-6',\n",
        "'inq_last_6mths:>6',\n",
        "'acc_now_delinq:0',\n",
        "'acc_now_delinq:>=1',\n",
        "'annual_inc:<20K',\n",
        "'annual_inc:20K-30K',\n",
        "'annual_inc:30K-40K',\n",
        "'annual_inc:40K-50K',\n",
        "'annual_inc:50K-60K',\n",
        "'annual_inc:60K-70K',\n",
        "'annual_inc:70K-80K',\n",
        "'annual_inc:80K-90K',\n",
        "'annual_inc:90K-100K',\n",
        "'annual_inc:100K-120K',\n",
        "'annual_inc:120K-140K',\n",
        "'annual_inc:>140K',\n",
        "'dti:<=1.4',\n",
        "'dti:1.4-3.5',\n",
        "'dti:3.5-7.7',\n",
        "'dti:7.7-10.5',\n",
        "'dti:10.5-16.1',\n",
        "'dti:16.1-20.3',\n",
        "'dti:20.3-21.7',\n",
        "'dti:21.7-22.4',\n",
        "'dti:22.4-35',\n",
        "'dti:>35',\n",
        "'mths_since_last_delinq:Missing',\n",
        "'mths_since_last_delinq:0-3',\n",
        "'mths_since_last_delinq:4-30',\n",
        "'mths_since_last_delinq:31-56',\n",
        "'mths_since_last_delinq:>=57',\n",
        "'mths_since_last_record:Missing',\n",
        "'mths_since_last_record:0-2',\n",
        "'mths_since_last_record:3-20',\n",
        "'mths_since_last_record:21-31',\n",
        "'mths_since_last_record:32-80',\n",
        "'mths_since_last_record:81-86',\n",
        "'mths_since_last_record:>86']\n",
        "\n",
        "ref_categories_pd = ['grade:G',\n",
        "'home_ownership:RENT_OTHER_NONE_ANY',\n",
        "'addr_state:ND_NE_IA_NV_FL_HI_AL',\n",
        "'verification_status:Verified',\n",
        "'purpose:educ__sm_b__wedd__ren_en__mov__house',\n",
        "'initial_list_status:f',\n",
        "'term:60',\n",
        "'emp_length:0',\n",
        "'mths_since_issue_d:>84',\n",
        "'int_rate:>20.281',\n",
        "'mths_since_earliest_cr_line:<140',\n",
        "'inq_last_6mths:>6',\n",
        "'acc_now_delinq:0',\n",
        "'annual_inc:<20K',\n",
        "'dti:>35',\n",
        "'mths_since_last_delinq:0-3',\n",
        "'mths_since_last_record:0-2']\n",
        "\n",
        "print(loan_data_inputs_pd.columns)\n",
        "loan_data_inputs_pd_temp = loan_data_inputs_pd[features_all_pd]\n",
        "# Here we keep only the variables we need for the model.\n",
        "\n",
        "loan_data_inputs_pd_temp = loan_data_inputs_pd_temp.drop(ref_categories_pd, axis = 1)\n",
        "# Here we remove the dummy variable reference categories.\n",
        "\n",
        "loan_data_inputs_pd_temp.shape\n",
        "\n",
        "import pickle\n",
        "!cp /content/GoogleDrive/My\\ Drive/Regaip/Kurslar/Credit\\ Risk\\ Modelling/pd_model.sav ./\n",
        "!pwd\n",
        "reg_pd = pickle.load(open('pd_model.sav', 'rb'))\n",
        "# We import the PD model, stored in the 'pd_model.sav' file.\n",
        "\n",
        "reg_pd.model.predict_proba(loan_data_inputs_pd_temp)[: ][: , 0]\n",
        "# We apply the PD model to caclulate estimated default probabilities.\n",
        "\n",
        "loan_data_inputs_pd['PD'] = reg_pd.model.predict_proba(loan_data_inputs_pd_temp)[: ][: , 0]\n",
        "# We apply the PD model to caclulate estimated default probabilities.\n",
        "\n",
        "loan_data_inputs_pd['PD'].head()\n",
        "\n",
        "loan_data_inputs_pd['PD'].describe()\n",
        "# Shows some descriptive statisics for the values of a column.\n",
        "\n",
        "loan_data_preprocessed_new = pd.concat([loan_data_preprocessed, loan_data_inputs_pd], axis = 1)\n",
        "# We concatenate the dataframes where we calculated LGD and EAD and the dataframe where we calculated PD along the columns.\n",
        "\n",
        "loan_data_preprocessed_new.shape\n",
        "loan_data_preprocessed_new.head()\n",
        "\n",
        "loan_data_preprocessed_new['EL'] = loan_data_preprocessed_new['PD'] * loan_data_preprocessed_new['LGD'] * loan_data_preprocessed_new['EAD']\n",
        "# We calculate Expected Loss. EL = PD * LGD * EAD.\n",
        "\n",
        "loan_data_preprocessed_new['EL'].describe()\n",
        "# Shows some descriptive statisics for the values of a column.\n",
        "\n",
        "loan_data_preprocessed_new[['funded_amnt', 'PD', 'LGD', 'EAD', 'EL']].head()\n",
        "loan_data_preprocessed_new['funded_amnt'].describe()\n",
        "loan_data_preprocessed_new['EL'].sum()\n",
        "# Total Expected Loss for all loans.\n",
        "loan_data_preprocessed_new['funded_amnt'].sum()\n",
        "# Total funded amount for all loans.\n",
        "\n",
        "loan_data_preprocessed_new['EL'].sum() / loan_data_preprocessed_new['funded_amnt'].sum()\n",
        "# Total Expected Loss as a proportion of total funded amount for all loans.\n",
        "####\n",
        "####\n",
        "####\n",
        "# THE END."
      ],
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Index(['Unnamed: 0.1', 'id', 'member_id', 'loan_amnt', 'funded_amnt',\n",
            "       'funded_amnt_inv', 'term', 'int_rate', 'installment', 'grade',\n",
            "       ...\n",
            "       'dti:21.7-22.4', 'dti:22.4-35', 'dti:>35',\n",
            "       'mths_since_last_record:Missing', 'mths_since_last_record:0-2',\n",
            "       'mths_since_last_record:3-20', 'mths_since_last_record:21-31',\n",
            "       'mths_since_last_record:32-80', 'mths_since_last_record:81-86',\n",
            "       'mths_since_last_record:>86'],\n",
            "      dtype='object', length=324)\n",
            "/content\n"
          ],
          "name": "stdout"
        },
        {
          "output_type": "error",
          "ename": "ValueError",
          "evalue": "ignored",
          "traceback": [
            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
            "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
            "\u001b[0;32m<ipython-input-19-80514bbd1faf>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m    201\u001b[0m \u001b[0;31m# We import the PD model, stored in the 'pd_model.sav' file.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    202\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 203\u001b[0;31m \u001b[0mreg_pd\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mmodel\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mpredict_proba\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mloan_data_inputs_pd_temp\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m]\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m \u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    204\u001b[0m \u001b[0;31m# We apply the PD model to caclulate estimated default probabilities.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    205\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_logistic.py\u001b[0m in \u001b[0;36mpredict_proba\u001b[0;34m(self, X)\u001b[0m\n\u001b[1;32m   1649\u001b[0m                                                 self.solver == 'liblinear')))\n\u001b[1;32m   1650\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0movr\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1651\u001b[0;31m             \u001b[0;32mreturn\u001b[0m \u001b[0msuper\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_predict_proba_lr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1652\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1653\u001b[0m             \u001b[0mdecision\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdecision_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_base.py\u001b[0m in \u001b[0;36m_predict_proba_lr\u001b[0;34m(self, X)\u001b[0m\n\u001b[1;32m    305\u001b[0m         \u001b[0mmulticlass\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0mhandled\u001b[0m \u001b[0mby\u001b[0m \u001b[0mnormalizing\u001b[0m \u001b[0mthat\u001b[0m \u001b[0mover\u001b[0m \u001b[0mall\u001b[0m \u001b[0mclasses\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    306\u001b[0m         \"\"\"\n\u001b[0;32m--> 307\u001b[0;31m         \u001b[0mprob\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdecision_function\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    308\u001b[0m         \u001b[0mexpit\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mprob\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mout\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mprob\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    309\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mprob\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mndim\u001b[0m \u001b[0;34m==\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/sklearn/linear_model/_base.py\u001b[0m in \u001b[0;36mdecision_function\u001b[0;34m(self, X)\u001b[0m\n\u001b[1;32m    266\u001b[0m         \u001b[0mcheck_is_fitted\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    267\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 268\u001b[0;31m         \u001b[0mX\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcheck_array\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0maccept_sparse\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m'csr'\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    269\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    270\u001b[0m         \u001b[0mn_features\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mcoef_\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py\u001b[0m in \u001b[0;36mcheck_array\u001b[0;34m(array, accept_sparse, accept_large_sparse, dtype, order, copy, force_all_finite, ensure_2d, allow_nd, ensure_min_samples, ensure_min_features, warn_on_dtype, estimator)\u001b[0m\n\u001b[1;32m    576\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0mforce_all_finite\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    577\u001b[0m             _assert_all_finite(array,\n\u001b[0;32m--> 578\u001b[0;31m                                allow_nan=force_all_finite == 'allow-nan')\n\u001b[0m\u001b[1;32m    579\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    580\u001b[0m     \u001b[0;32mif\u001b[0m \u001b[0mensure_min_samples\u001b[0m \u001b[0;34m>\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;32m/usr/local/lib/python3.6/dist-packages/sklearn/utils/validation.py\u001b[0m in \u001b[0;36m_assert_all_finite\u001b[0;34m(X, allow_nan, msg_dtype)\u001b[0m\n\u001b[1;32m     58\u001b[0m                     \u001b[0mmsg_err\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mformat\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     59\u001b[0m                     (type_err,\n\u001b[0;32m---> 60\u001b[0;31m                      msg_dtype if msg_dtype is not None else X.dtype)\n\u001b[0m\u001b[1;32m     61\u001b[0m             )\n\u001b[1;32m     62\u001b[0m     \u001b[0;31m# for object dtype data, we only check for NaNs (GH-13254)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
            "\u001b[0;31mValueError\u001b[0m: Input contains NaN, infinity or a value too large for dtype('float64')."
          ]
        }
      ]
    }
  ]
}