Homework5.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Homework5.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/mhasanli7782/MH-Business-Analytics/blob/main/Homework5.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Case Intro\n",
        "Term deposits are a major source of income for a bank. A term deposit is a cash investment held at a financial institution. Your money is invested for an agreed rate of interest over a fixed amount of time, or term. The bank has various outreach plans to sell term deposits to their customers such as email marketing, advertisements, telephonic marketing, and digital marketing.\n",
        "\n",
        "Telephonic marketing campaigns still remain one of the most effective way to reach out to people. However, they require huge investment as large call centers are hired to actually execute these campaigns. Hence, it is crucial to identify the customers most likely to convert beforehand so that they can be specifically targeted via call.\n",
        "\n",
        "The data is related to direct marketing campaigns (phone calls) of a Portuguese banking institution. The classification goal is to predict if the client will subscribe to a term deposit (variable y).\n",
        "\n",
        "Content\n",
        "The data is related to the direct marketing campaigns of a Portuguese banking institution. The marketing campaigns were based on phone calls. Often, more than one contact to the same client was required, in order to access if the product (bank term deposit) would be ('yes') or not ('no') subscribed by the customer or not. The data folder contains two datasets:-\n",
        "\n",
        "Bank.csv: 45,211 rows and 18 columns ordered by date (from May 2008 to November 2010)\n",
        "\n",
        "Detailed Column Descriptions\n",
        "bank client data:\n",
        "\n",
        "1 - age (numeric)\n",
        "\n",
        "2 - job : type of job (categorical: \"admin.\",\"unknown\",\"unemployed\",\"management\",\"housemaid\",\"entrepreneur\",\"student\",\n",
        "\"blue-collar\",\"self-employed\",\"retired\",\"technician\",\"services\")\n",
        "\n",
        "3 - marital : marital status (categorical: \"married\",\"divorced\",\"single\"; note: \"divorced\" means divorced or widowed)\n",
        "\n",
        "4 - education (categorical: \"unknown\",\"secondary\",\"primary\",\"tertiary\")\n",
        "\n",
        "5 - default: has credit in default? (binary: \"yes\",\"no\")\n",
        "\n",
        "6 - balance: average yearly balance, in euros (numeric)\n",
        "\n",
        "7 - housing: has housing loan? (binary: \"yes\",\"no\")\n",
        "\n",
        "8 - loan: has personal loan? (binary: \"yes\",\"no\")\n",
        "# related with the last contact of the current campaign:\n",
        "9 - contact: contact communication type (categorical: \"unknown\",\"telephone\",\"cellular\")\n",
        "10 - day: last contact day of the month (numeric)\n",
        "\n",
        "11 - month: last contact month of year (categorical: \"jan\", \"feb\", \"mar\", …, \"nov\", \"dec\")\n",
        "\n",
        "12 - duration: last contact duration, in seconds (numeric)\n",
        "\n",
        "# other attributes:\n",
        "13 - campaign: number of contacts performed during this campaign and for this client (numeric, includes last contact)\n",
        "\n",
        "14 - pdays: number of days that passed by after the client was last contacted from a previous campaign (numeric, -1 means client was not previously contacted)\n",
        "\n",
        "15 - previous: number of contacts performed before this campaign and for this client (numeric)\n",
        "\n",
        "16 - poutcome: outcome of the previous marketing campaign (categorical: \"unknown\",\"other\",\"failure\",\"success\")\n",
        "\n",
        "Output variable (desired target):\n",
        "\n",
        "17 - y - has the client subscribed a term deposit? (binary: \"yes\",\"no\")\n",
        "\n",
        "Missing Attribute Values: None\n"
      ],
      "metadata": {
        "id": "Vk6nkafwIHMh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import pandas as pd\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt\n",
        "import seaborn as sns\n",
        "df=pd.read_csv('https://raw.githubusercontent.com/ogut77/DataScience/main/data/Bank.csv',sep = ';')\n",
        "df"
      ],
      "metadata": {
        "id": "EZdL1N4MZqZF",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 467
        },
        "outputId": "060c340d-110a-4e66-b587-0535794dde3e"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       age           job   marital  education default  balance housing loan  \\\n",
              "0       58    management   married   tertiary      no     2143     yes   no   \n",
              "1       44    technician    single  secondary      no       29     yes   no   \n",
              "2       33  entrepreneur   married  secondary      no        2     yes  yes   \n",
              "3       47   blue-collar   married    unknown      no     1506     yes   no   \n",
              "4       33       unknown    single    unknown      no        1      no   no   \n",
              "...    ...           ...       ...        ...     ...      ...     ...  ...   \n",
              "45206   51    technician   married   tertiary      no      825      no   no   \n",
              "45207   71       retired  divorced    primary      no     1729      no   no   \n",
              "45208   72       retired   married  secondary      no     5715      no   no   \n",
              "45209   57   blue-collar   married  secondary      no      668      no   no   \n",
              "45210   37  entrepreneur   married  secondary      no     2971      no   no   \n",
              "\n",
              "         contact  day month  duration  campaign  pdays  previous poutcome    y  \n",
              "0        unknown    5   may       261         1     -1         0  unknown   no  \n",
              "1        unknown    5   may       151         1     -1         0  unknown   no  \n",
              "2        unknown    5   may        76         1     -1         0  unknown   no  \n",
              "3        unknown    5   may        92         1     -1         0  unknown   no  \n",
              "4        unknown    5   may       198         1     -1         0  unknown   no  \n",
              "...          ...  ...   ...       ...       ...    ...       ...      ...  ...  \n",
              "45206   cellular   17   nov       977         3     -1         0  unknown  yes  \n",
              "45207   cellular   17   nov       456         2     -1         0  unknown  yes  \n",
              "45208   cellular   17   nov      1127         5    184         3  success  yes  \n",
              "45209  telephone   17   nov       508         4     -1         0  unknown   no  \n",
              "45210   cellular   17   nov       361         2    188        11    other   no  \n",
              "\n",
              "[45211 rows x 17 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-441f6a6a-52dd-41cc-82e1-e5f3c4d80ca8\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>job</th>\n",
              "      <th>marital</th>\n",
              "      <th>education</th>\n",
              "      <th>default</th>\n",
              "      <th>balance</th>\n",
              "      <th>housing</th>\n",
              "      <th>loan</th>\n",
              "      <th>contact</th>\n",
              "      <th>day</th>\n",
              "      <th>month</th>\n",
              "      <th>duration</th>\n",
              "      <th>campaign</th>\n",
              "      <th>pdays</th>\n",
              "      <th>previous</th>\n",
              "      <th>poutcome</th>\n",
              "      <th>y</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>58</td>\n",
              "      <td>management</td>\n",
              "      <td>married</td>\n",
              "      <td>tertiary</td>\n",
              "      <td>no</td>\n",
              "      <td>2143</td>\n",
              "      <td>yes</td>\n",
              "      <td>no</td>\n",
              "      <td>unknown</td>\n",
              "      <td>5</td>\n",
              "      <td>may</td>\n",
              "      <td>261</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>unknown</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>44</td>\n",
              "      <td>technician</td>\n",
              "      <td>single</td>\n",
              "      <td>secondary</td>\n",
              "      <td>no</td>\n",
              "      <td>29</td>\n",
              "      <td>yes</td>\n",
              "      <td>no</td>\n",
              "      <td>unknown</td>\n",
              "      <td>5</td>\n",
              "      <td>may</td>\n",
              "      <td>151</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>unknown</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>33</td>\n",
              "      <td>entrepreneur</td>\n",
              "      <td>married</td>\n",
              "      <td>secondary</td>\n",
              "      <td>no</td>\n",
              "      <td>2</td>\n",
              "      <td>yes</td>\n",
              "      <td>yes</td>\n",
              "      <td>unknown</td>\n",
              "      <td>5</td>\n",
              "      <td>may</td>\n",
              "      <td>76</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>unknown</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>47</td>\n",
              "      <td>blue-collar</td>\n",
              "      <td>married</td>\n",
              "      <td>unknown</td>\n",
              "      <td>no</td>\n",
              "      <td>1506</td>\n",
              "      <td>yes</td>\n",
              "      <td>no</td>\n",
              "      <td>unknown</td>\n",
              "      <td>5</td>\n",
              "      <td>may</td>\n",
              "      <td>92</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>unknown</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>33</td>\n",
              "      <td>unknown</td>\n",
              "      <td>single</td>\n",
              "      <td>unknown</td>\n",
              "      <td>no</td>\n",
              "      <td>1</td>\n",
              "      <td>no</td>\n",
              "      <td>no</td>\n",
              "      <td>unknown</td>\n",
              "      <td>5</td>\n",
              "      <td>may</td>\n",
              "      <td>198</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>unknown</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45206</th>\n",
              "      <td>51</td>\n",
              "      <td>technician</td>\n",
              "      <td>married</td>\n",
              "      <td>tertiary</td>\n",
              "      <td>no</td>\n",
              "      <td>825</td>\n",
              "      <td>no</td>\n",
              "      <td>no</td>\n",
              "      <td>cellular</td>\n",
              "      <td>17</td>\n",
              "      <td>nov</td>\n",
              "      <td>977</td>\n",
              "      <td>3</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>unknown</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45207</th>\n",
              "      <td>71</td>\n",
              "      <td>retired</td>\n",
              "      <td>divorced</td>\n",
              "      <td>primary</td>\n",
              "      <td>no</td>\n",
              "      <td>1729</td>\n",
              "      <td>no</td>\n",
              "      <td>no</td>\n",
              "      <td>cellular</td>\n",
              "      <td>17</td>\n",
              "      <td>nov</td>\n",
              "      <td>456</td>\n",
              "      <td>2</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>unknown</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45208</th>\n",
              "      <td>72</td>\n",
              "      <td>retired</td>\n",
              "      <td>married</td>\n",
              "      <td>secondary</td>\n",
              "      <td>no</td>\n",
              "      <td>5715</td>\n",
              "      <td>no</td>\n",
              "      <td>no</td>\n",
              "      <td>cellular</td>\n",
              "      <td>17</td>\n",
              "      <td>nov</td>\n",
              "      <td>1127</td>\n",
              "      <td>5</td>\n",
              "      <td>184</td>\n",
              "      <td>3</td>\n",
              "      <td>success</td>\n",
              "      <td>yes</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45209</th>\n",
              "      <td>57</td>\n",
              "      <td>blue-collar</td>\n",
              "      <td>married</td>\n",
              "      <td>secondary</td>\n",
              "      <td>no</td>\n",
              "      <td>668</td>\n",
              "      <td>no</td>\n",
              "      <td>no</td>\n",
              "      <td>telephone</td>\n",
              "      <td>17</td>\n",
              "      <td>nov</td>\n",
              "      <td>508</td>\n",
              "      <td>4</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>unknown</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45210</th>\n",
              "      <td>37</td>\n",
              "      <td>entrepreneur</td>\n",
              "      <td>married</td>\n",
              "      <td>secondary</td>\n",
              "      <td>no</td>\n",
              "      <td>2971</td>\n",
              "      <td>no</td>\n",
              "      <td>no</td>\n",
              "      <td>cellular</td>\n",
              "      <td>17</td>\n",
              "      <td>nov</td>\n",
              "      <td>361</td>\n",
              "      <td>2</td>\n",
              "      <td>188</td>\n",
              "      <td>11</td>\n",
              "      <td>other</td>\n",
              "      <td>no</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>45211 rows × 17 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-441f6a6a-52dd-41cc-82e1-e5f3c4d80ca8')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-441f6a6a-52dd-41cc-82e1-e5f3c4d80ca8 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-441f6a6a-52dd-41cc-82e1-e5f3c4d80ca8');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 1
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "print(df.shape)\n",
        "df.info()\n",
        "df.isnull().sum()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hm9ZMVW9aJNh",
        "outputId": "374e0e86-57d4-4bb5-b60b-19af58b8ba01"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "(45211, 17)\n",
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 45211 entries, 0 to 45210\n",
            "Data columns (total 17 columns):\n",
            " #   Column     Non-Null Count  Dtype \n",
            "---  ------     --------------  ----- \n",
            " 0   age        45211 non-null  int64 \n",
            " 1   job        45211 non-null  object\n",
            " 2   marital    45211 non-null  object\n",
            " 3   education  45211 non-null  object\n",
            " 4   default    45211 non-null  object\n",
            " 5   balance    45211 non-null  int64 \n",
            " 6   housing    45211 non-null  object\n",
            " 7   loan       45211 non-null  object\n",
            " 8   contact    45211 non-null  object\n",
            " 9   day        45211 non-null  int64 \n",
            " 10  month      45211 non-null  object\n",
            " 11  duration   45211 non-null  int64 \n",
            " 12  campaign   45211 non-null  int64 \n",
            " 13  pdays      45211 non-null  int64 \n",
            " 14  previous   45211 non-null  int64 \n",
            " 15  poutcome   45211 non-null  object\n",
            " 16  y          45211 non-null  object\n",
            "dtypes: int64(7), object(10)\n",
            "memory usage: 5.9+ MB\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "age          0\n",
              "job          0\n",
              "marital      0\n",
              "education    0\n",
              "default      0\n",
              "balance      0\n",
              "housing      0\n",
              "loan         0\n",
              "contact      0\n",
              "day          0\n",
              "month        0\n",
              "duration     0\n",
              "campaign     0\n",
              "pdays        0\n",
              "previous     0\n",
              "poutcome     0\n",
              "y            0\n",
              "dtype: int64"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "#For object check the data \n",
        "for cn in df.columns:\n",
        "  if(df[cn].dtype==object):\n",
        "    print(df[cn].value_counts())\n",
        "  "
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_LD1CE208fhi",
        "outputId": "4fdd87ae-8a8d-42d4-d84a-7aa47f7a660d"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "blue-collar      9732\n",
            "management       9458\n",
            "technician       7597\n",
            "admin.           5171\n",
            "services         4154\n",
            "retired          2264\n",
            "self-employed    1579\n",
            "entrepreneur     1487\n",
            "unemployed       1303\n",
            "housemaid        1240\n",
            "student           938\n",
            "unknown           288\n",
            "Name: job, dtype: int64\n",
            "married     27214\n",
            "single      12790\n",
            "divorced     5207\n",
            "Name: marital, dtype: int64\n",
            "secondary    23202\n",
            "tertiary     13301\n",
            "primary       6851\n",
            "unknown       1857\n",
            "Name: education, dtype: int64\n",
            "no     44396\n",
            "yes      815\n",
            "Name: default, dtype: int64\n",
            "yes    25130\n",
            "no     20081\n",
            "Name: housing, dtype: int64\n",
            "no     37967\n",
            "yes     7244\n",
            "Name: loan, dtype: int64\n",
            "cellular     29285\n",
            "unknown      13020\n",
            "telephone     2906\n",
            "Name: contact, dtype: int64\n",
            "may    13766\n",
            "jul     6895\n",
            "aug     6247\n",
            "jun     5341\n",
            "nov     3970\n",
            "apr     2932\n",
            "feb     2649\n",
            "jan     1403\n",
            "oct      738\n",
            "sep      579\n",
            "mar      477\n",
            "dec      214\n",
            "Name: month, dtype: int64\n",
            "unknown    36959\n",
            "failure     4901\n",
            "other       1840\n",
            "success     1511\n",
            "Name: poutcome, dtype: int64\n",
            "no     39922\n",
            "yes     5289\n",
            "Name: y, dtype: int64\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def Encoder(df):\n",
        "          from sklearn import preprocessing\n",
        "          columnsToEncode = list(df.select_dtypes(include=['category','object']))\n",
        "          le = preprocessing.LabelEncoder()\n",
        "          for feature in columnsToEncode:\n",
        "              try:\n",
        "                  df[feature] = le.fit_transform(df[feature])\n",
        "              except:\n",
        "                  print('Error encoding '+feature)\n",
        "          return df\n"
      ],
      "metadata": {
        "id": "JqEV5gKmD6_v"
      },
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "df=Encoder(df)\n",
        "df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "id": "U_XhnTUFBfMj",
        "outputId": "c35f7066-960b-4838-c757-4860e6c3547b"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       age  job  marital  education  default  balance  housing  loan  contact  \\\n",
              "0       58    4        1          2        0     2143        1     0        2   \n",
              "1       44    9        2          1        0       29        1     0        2   \n",
              "2       33    2        1          1        0        2        1     1        2   \n",
              "3       47    1        1          3        0     1506        1     0        2   \n",
              "4       33   11        2          3        0        1        0     0        2   \n",
              "...    ...  ...      ...        ...      ...      ...      ...   ...      ...   \n",
              "45206   51    9        1          2        0      825        0     0        0   \n",
              "45207   71    5        0          0        0     1729        0     0        0   \n",
              "45208   72    5        1          1        0     5715        0     0        0   \n",
              "45209   57    1        1          1        0      668        0     0        1   \n",
              "45210   37    2        1          1        0     2971        0     0        0   \n",
              "\n",
              "       day  month  duration  campaign  pdays  previous  poutcome  y  \n",
              "0        5      8       261         1     -1         0         3  0  \n",
              "1        5      8       151         1     -1         0         3  0  \n",
              "2        5      8        76         1     -1         0         3  0  \n",
              "3        5      8        92         1     -1         0         3  0  \n",
              "4        5      8       198         1     -1         0         3  0  \n",
              "...    ...    ...       ...       ...    ...       ...       ... ..  \n",
              "45206   17      9       977         3     -1         0         3  1  \n",
              "45207   17      9       456         2     -1         0         3  1  \n",
              "45208   17      9      1127         5    184         3         2  1  \n",
              "45209   17      9       508         4     -1         0         3  0  \n",
              "45210   17      9       361         2    188        11         1  0  \n",
              "\n",
              "[45211 rows x 17 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-55f6eff9-39fe-4f63-90db-d7fb6b7ff9d1\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>job</th>\n",
              "      <th>marital</th>\n",
              "      <th>education</th>\n",
              "      <th>default</th>\n",
              "      <th>balance</th>\n",
              "      <th>housing</th>\n",
              "      <th>loan</th>\n",
              "      <th>contact</th>\n",
              "      <th>day</th>\n",
              "      <th>month</th>\n",
              "      <th>duration</th>\n",
              "      <th>campaign</th>\n",
              "      <th>pdays</th>\n",
              "      <th>previous</th>\n",
              "      <th>poutcome</th>\n",
              "      <th>y</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>58</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>2143</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "      <td>8</td>\n",
              "      <td>261</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>44</td>\n",
              "      <td>9</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>29</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "      <td>8</td>\n",
              "      <td>151</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>33</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "      <td>8</td>\n",
              "      <td>76</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>47</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1506</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "      <td>8</td>\n",
              "      <td>92</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>33</td>\n",
              "      <td>11</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "      <td>8</td>\n",
              "      <td>198</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45206</th>\n",
              "      <td>51</td>\n",
              "      <td>9</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>825</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>17</td>\n",
              "      <td>9</td>\n",
              "      <td>977</td>\n",
              "      <td>3</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45207</th>\n",
              "      <td>71</td>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1729</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>17</td>\n",
              "      <td>9</td>\n",
              "      <td>456</td>\n",
              "      <td>2</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45208</th>\n",
              "      <td>72</td>\n",
              "      <td>5</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>5715</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>17</td>\n",
              "      <td>9</td>\n",
              "      <td>1127</td>\n",
              "      <td>5</td>\n",
              "      <td>184</td>\n",
              "      <td>3</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45209</th>\n",
              "      <td>57</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>668</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>17</td>\n",
              "      <td>9</td>\n",
              "      <td>508</td>\n",
              "      <td>4</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45210</th>\n",
              "      <td>37</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2971</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>17</td>\n",
              "      <td>9</td>\n",
              "      <td>361</td>\n",
              "      <td>2</td>\n",
              "      <td>188</td>\n",
              "      <td>11</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>45211 rows × 17 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-55f6eff9-39fe-4f63-90db-d7fb6b7ff9d1')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-55f6eff9-39fe-4f63-90db-d7fb6b7ff9d1 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-55f6eff9-39fe-4f63-90db-d7fb6b7ff9d1');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "y = df['y'] #Output\n",
        "X = df.drop('y',axis=1)\n",
        "X"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        },
        "id": "s8C7J0ObyNs5",
        "outputId": "3cd3449c-8b00-48e9-dfaf-837052e13b09"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "       age  job  marital  education  default  balance  housing  loan  contact  \\\n",
              "0       58    4        1          2        0     2143        1     0        2   \n",
              "1       44    9        2          1        0       29        1     0        2   \n",
              "2       33    2        1          1        0        2        1     1        2   \n",
              "3       47    1        1          3        0     1506        1     0        2   \n",
              "4       33   11        2          3        0        1        0     0        2   \n",
              "...    ...  ...      ...        ...      ...      ...      ...   ...      ...   \n",
              "45206   51    9        1          2        0      825        0     0        0   \n",
              "45207   71    5        0          0        0     1729        0     0        0   \n",
              "45208   72    5        1          1        0     5715        0     0        0   \n",
              "45209   57    1        1          1        0      668        0     0        1   \n",
              "45210   37    2        1          1        0     2971        0     0        0   \n",
              "\n",
              "       day  month  duration  campaign  pdays  previous  poutcome  \n",
              "0        5      8       261         1     -1         0         3  \n",
              "1        5      8       151         1     -1         0         3  \n",
              "2        5      8        76         1     -1         0         3  \n",
              "3        5      8        92         1     -1         0         3  \n",
              "4        5      8       198         1     -1         0         3  \n",
              "...    ...    ...       ...       ...    ...       ...       ...  \n",
              "45206   17      9       977         3     -1         0         3  \n",
              "45207   17      9       456         2     -1         0         3  \n",
              "45208   17      9      1127         5    184         3         2  \n",
              "45209   17      9       508         4     -1         0         3  \n",
              "45210   17      9       361         2    188        11         1  \n",
              "\n",
              "[45211 rows x 16 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-b0bd905b-ab4f-476b-9e9b-1252c126a063\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>age</th>\n",
              "      <th>job</th>\n",
              "      <th>marital</th>\n",
              "      <th>education</th>\n",
              "      <th>default</th>\n",
              "      <th>balance</th>\n",
              "      <th>housing</th>\n",
              "      <th>loan</th>\n",
              "      <th>contact</th>\n",
              "      <th>day</th>\n",
              "      <th>month</th>\n",
              "      <th>duration</th>\n",
              "      <th>campaign</th>\n",
              "      <th>pdays</th>\n",
              "      <th>previous</th>\n",
              "      <th>poutcome</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>58</td>\n",
              "      <td>4</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>2143</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "      <td>8</td>\n",
              "      <td>261</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>44</td>\n",
              "      <td>9</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>29</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "      <td>8</td>\n",
              "      <td>151</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>33</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "      <td>8</td>\n",
              "      <td>76</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>47</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1506</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "      <td>8</td>\n",
              "      <td>92</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>33</td>\n",
              "      <td>11</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>5</td>\n",
              "      <td>8</td>\n",
              "      <td>198</td>\n",
              "      <td>1</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45206</th>\n",
              "      <td>51</td>\n",
              "      <td>9</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>825</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>17</td>\n",
              "      <td>9</td>\n",
              "      <td>977</td>\n",
              "      <td>3</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45207</th>\n",
              "      <td>71</td>\n",
              "      <td>5</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1729</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>17</td>\n",
              "      <td>9</td>\n",
              "      <td>456</td>\n",
              "      <td>2</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45208</th>\n",
              "      <td>72</td>\n",
              "      <td>5</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>5715</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>17</td>\n",
              "      <td>9</td>\n",
              "      <td>1127</td>\n",
              "      <td>5</td>\n",
              "      <td>184</td>\n",
              "      <td>3</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45209</th>\n",
              "      <td>57</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>668</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>17</td>\n",
              "      <td>9</td>\n",
              "      <td>508</td>\n",
              "      <td>4</td>\n",
              "      <td>-1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>45210</th>\n",
              "      <td>37</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>2971</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>17</td>\n",
              "      <td>9</td>\n",
              "      <td>361</td>\n",
              "      <td>2</td>\n",
              "      <td>188</td>\n",
              "      <td>11</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>45211 rows × 16 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-b0bd905b-ab4f-476b-9e9b-1252c126a063')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-b0bd905b-ab4f-476b-9e9b-1252c126a063 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-b0bd905b-ab4f-476b-9e9b-1252c126a063');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.model_selection import  train_test_split\n",
        "\n",
        "\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=17)"
      ],
      "metadata": {
        "id": "aC4sGAvybIGe"
      },
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        " \n",
        "\n",
        "\n",
        "Q1)Using  Random Forest,XGBoost, Light GBM and Gradient Boosting Classifier with default parameters (no parameter specifications except random_state) calculate Accuracy on Test data. Which method gives the best accuracy on test data"
      ],
      "metadata": {
        "id": "VbZLvi_k2kjt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Random Forest:\n",
        "from sklearn.metrics import accuracy_score\n",
        "\n",
        "from sklearn.ensemble import RandomForestClassifier\n",
        "forest = RandomForestClassifier(random_state=15)\n",
        "forest.fit(X_train, y_train)\n",
        "\n",
        "forest_pred = forest.predict(X_test)\n",
        "print(accuracy_score(y_test, forest_pred))\n",
        "\n",
        "# XGBoost:\n",
        "from xgboost import XGBClassifier\n",
        "model = XGBClassifier(random_state=15)\n",
        "model.fit(X_train, y_train)\n",
        "\n",
        "xgb_pred = model.predict(X_test)\n",
        "print(accuracy_score(y_test, xgb_pred))\n",
        "\n",
        "# Light GBM:\n",
        "import lightgbm as lgb\n",
        "lgb_model = lgb.LGBMClassifier(random_state=15)\n",
        "lgb_model.fit(X_train, y_train)\n",
        "\n",
        "print(accuracy_score(y_test, lgb_model.predict(X_test)))\n",
        "\n",
        "# Gradient Boosting Classifier:\n",
        "from sklearn.ensemble import GradientBoostingClassifier\n",
        "gbm_model = GradientBoostingClassifier(random_state=15)\n",
        "gbm_model.fit(X_train, y_train)\n",
        "\n",
        "print(accuracy_score(y_test, gbm_model.predict(X_test)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2M7Ov0lrDeCl",
        "outputId": "52a670bd-f632-41a8-af22-258dbade3ea6"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "0.904007785543661\n",
            "0.9043616738918872\n",
            "0.9073697248518092\n",
            "0.9018844554543042\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Q2) Using optuna hyperparmeter optimization technique and 100 trial\n",
        "\n",
        " a)find best methods with  parameters  using Cross validation (CV=3) technique for the range of   parameters below. What are the best parameters for the method with highest cross validation accuracy?\n",
        "\n",
        "\"max_depth\": range(2, 16), \"max_features\": range(2, 16)\n",
        "\n",
        " b)Evaluate the performance of the  method with highest cross validation accuracy on test data.What is the accuracy value?\n"
      ],
      "metadata": {
        "id": "TK77k75qO7dW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install optuna"
      ],
      "metadata": {
        "id": "ZiYWyhdhiEps",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "c8d30c2d-28eb-4bd5-dc83-f03a6a602a6f"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Collecting optuna\n",
            "  Downloading optuna-3.1.1-py3-none-any.whl (365 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m365.7/365.7 KB\u001b[0m \u001b[31m15.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: sqlalchemy>=1.3.0 in /usr/local/lib/python3.9/dist-packages (from optuna) (1.4.47)\n",
            "Collecting cmaes>=0.9.1\n",
            "  Downloading cmaes-0.9.1-py3-none-any.whl (21 kB)\n",
            "Collecting colorlog\n",
            "  Downloading colorlog-6.7.0-py2.py3-none-any.whl (11 kB)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.9/dist-packages (from optuna) (4.65.0)\n",
            "Requirement already satisfied: PyYAML in /usr/local/lib/python3.9/dist-packages (from optuna) (6.0)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.9/dist-packages (from optuna) (1.22.4)\n",
            "Collecting alembic>=1.5.0\n",
            "  Downloading alembic-1.10.3-py3-none-any.whl (212 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m212.3/212.3 KB\u001b[0m \u001b[31m22.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.9/dist-packages (from optuna) (23.0)\n",
            "Collecting Mako\n",
            "  Downloading Mako-1.2.4-py3-none-any.whl (78 kB)\n",
            "\u001b[2K     \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m78.7/78.7 KB\u001b[0m \u001b[31m10.0 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25hRequirement already satisfied: typing-extensions>=4 in /usr/local/lib/python3.9/dist-packages (from alembic>=1.5.0->optuna) (4.5.0)\n",
            "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.9/dist-packages (from sqlalchemy>=1.3.0->optuna) (2.0.2)\n",
            "Requirement already satisfied: MarkupSafe>=0.9.2 in /usr/local/lib/python3.9/dist-packages (from Mako->alembic>=1.5.0->optuna) (2.1.2)\n",
            "Installing collected packages: Mako, colorlog, cmaes, alembic, optuna\n",
            "Successfully installed Mako-1.2.4 alembic-1.10.3 cmaes-0.9.1 colorlog-6.7.0 optuna-3.1.1\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import optuna\n",
        "\n",
        "import sklearn.datasets\n",
        "import sklearn.ensemble\n",
        "import sklearn.model_selection\n",
        "import sklearn.svm\n",
        "\n",
        "def objective(trial):\n",
        "    x, y = X_test,y_test\n",
        "\n",
        "    classifier_name = trial.suggest_categorical(\"classifier\", [\"Random Forest\",\"XGBoost\", \"LightGBM\",\"GradientBoostingClassifier\" ])\n",
        "    if classifier_name == \"Random Forest\":\n",
        "         from sklearn.ensemble import RandomForestClassifier\n",
        "         max_depth = trial.suggest_int(\"max_depth\",2,X_test.shape[1])\n",
        "         max_features = trial.suggest_int(\"max_features\",2,X_test.shape[1])\n",
        "         classifier_obj = sklearn.ensemble.RandomForestClassifier(random_state=15,  max_depth=max_depth, max_features=max_features )\n",
        "        \n",
        "         \n",
        "\n",
        "    elif classifier_name == \"XGBoost\":\n",
        "         from xgboost import XGBClassifier\n",
        "         max_depth = trial.suggest_int(\"max_depth\",2,X_test.shape[1])\n",
        "         max_features = trial.suggest_int(\"max_features\",2,X_test.shape[1])\n",
        "         classifier_obj = XGBClassifier(random_state=15,  max_depth=max_depth, max_features=max_features )\n",
        "        \n",
        "         \n",
        "\n",
        "    elif classifier_name == \"LightGBM\":\n",
        "         import lightgbm as lgb\n",
        "         max_depth = trial.suggest_int(\"max_depth\",2,X_test.shape[1])\n",
        "         max_features = trial.suggest_int(\"max_features\",2,X_test.shape[1])\n",
        "         classifier_obj = lgb.LGBMClassifier(random_state=15,  max_depth=max_depth, max_features=max_features )\n",
        "         \n",
        "    else:\n",
        "         max_depth = trial.suggest_int(\"max_depth\",2,X_test.shape[1])\n",
        "         max_features = trial.suggest_int(\"max_features\",2,X_test.shape[1])\n",
        "         classifier_obj = sklearn.ensemble.GradientBoostingClassifier(random_state=15,  max_depth=max_depth, max_features=max_features )\n",
        "        \n",
        "         \n",
        "    accuracy=sklearn.model_selection.cross_val_score(classifier_obj, x, y, n_jobs=-1, cv=3).mean()\n",
        "   \n",
        "    return accuracy\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    study = optuna.create_study(direction=\"maximize\")\n",
        "    study.optimize(objective, n_trials=100)\n",
        "    print(study.best_trial)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "1MNXcbEYEVct",
        "outputId": "26a45155-1f9c-47ca-a634-7b0829ad1095"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\u001b[32m[I 2023-04-08 18:12:26,072]\u001b[0m A new study created in memory with name: no-name-0ff3b688-b859-46ac-beaf-772f2acfa4fd\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:12:56,849]\u001b[0m Trial 0 finished with value: 0.8988765203312336 and parameters: {'classifier': 'GradientBoostingClassifier', 'max_depth': 14, 'max_features': 4}. Best is trial 0 with value: 0.8988765203312336.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:13:02,460]\u001b[0m Trial 1 finished with value: 0.8963110802625174 and parameters: {'classifier': 'XGBoost', 'max_depth': 13, 'max_features': 10}. Best is trial 0 with value: 0.8988765203312336.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:13:04,223]\u001b[0m Trial 2 finished with value: 0.9005578814117684 and parameters: {'classifier': 'LightGBM', 'max_depth': 9, 'max_features': 2}. Best is trial 2 with value: 0.9005578814117684.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:13:05,110]\u001b[0m Trial 3 finished with value: 0.9003807885028307 and parameters: {'classifier': 'LightGBM', 'max_depth': 10, 'max_features': 11}. Best is trial 2 with value: 0.9005578814117684.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:13:14,817]\u001b[0m Trial 4 finished with value: 0.899672604739148 and parameters: {'classifier': 'GradientBoostingClassifier', 'max_depth': 11, 'max_features': 6}. Best is trial 2 with value: 0.9005578814117684.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:13:16,378]\u001b[0m Trial 5 finished with value: 0.8879943125488584 and parameters: {'classifier': 'Random Forest', 'max_depth': 2, 'max_features': 13}. Best is trial 2 with value: 0.9005578814117684.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:13:45,162]\u001b[0m Trial 6 finished with value: 0.8990538011122403 and parameters: {'classifier': 'GradientBoostingClassifier', 'max_depth': 16, 'max_features': 7}. Best is trial 2 with value: 0.9005578814117684.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:13:48,716]\u001b[0m Trial 7 finished with value: 0.8981688532157405 and parameters: {'classifier': 'XGBoost', 'max_depth': 5, 'max_features': 15}. Best is trial 2 with value: 0.9005578814117684.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:13:54,287]\u001b[0m Trial 8 finished with value: 0.8963110802625174 and parameters: {'classifier': 'XGBoost', 'max_depth': 13, 'max_features': 10}. Best is trial 2 with value: 0.9005578814117684.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:00,195]\u001b[0m Trial 9 finished with value: 0.8954262732700693 and parameters: {'classifier': 'XGBoost', 'max_depth': 12, 'max_features': 14}. Best is trial 2 with value: 0.9005578814117684.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:01,086]\u001b[0m Trial 10 finished with value: 0.9015308708565519 and parameters: {'classifier': 'LightGBM', 'max_depth': 7, 'max_features': 2}. Best is trial 10 with value: 0.9015308708565519.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:01,883]\u001b[0m Trial 11 finished with value: 0.9015308708565519 and parameters: {'classifier': 'LightGBM', 'max_depth': 7, 'max_features': 2}. Best is trial 10 with value: 0.9015308708565519.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:02,499]\u001b[0m Trial 12 finished with value: 0.9015308708565519 and parameters: {'classifier': 'LightGBM', 'max_depth': 7, 'max_features': 2}. Best is trial 10 with value: 0.9015308708565519.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:03,070]\u001b[0m Trial 13 finished with value: 0.9033889491018869 and parameters: {'classifier': 'LightGBM', 'max_depth': 5, 'max_features': 5}. Best is trial 13 with value: 0.9033889491018869.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:03,490]\u001b[0m Trial 14 finished with value: 0.9010886669744012 and parameters: {'classifier': 'LightGBM', 'max_depth': 3, 'max_features': 5}. Best is trial 13 with value: 0.9033889491018869.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:05,558]\u001b[0m Trial 15 finished with value: 0.9011767085226849 and parameters: {'classifier': 'Random Forest', 'max_depth': 5, 'max_features': 8}. Best is trial 13 with value: 0.9033889491018869.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:06,168]\u001b[0m Trial 16 finished with value: 0.9006460168960867 and parameters: {'classifier': 'LightGBM', 'max_depth': 8, 'max_features': 4}. Best is trial 13 with value: 0.9033889491018869.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:06,706]\u001b[0m Trial 17 finished with value: 0.9033889491018869 and parameters: {'classifier': 'LightGBM', 'max_depth': 5, 'max_features': 4}. Best is trial 13 with value: 0.9033889491018869.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:07,179]\u001b[0m Trial 18 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:08,685]\u001b[0m Trial 19 finished with value: 0.8836591410752032 and parameters: {'classifier': 'Random Forest', 'max_depth': 3, 'max_features': 8}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:09,150]\u001b[0m Trial 20 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:09,621]\u001b[0m Trial 21 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:10,039]\u001b[0m Trial 22 finished with value: 0.9010886669744012 and parameters: {'classifier': 'LightGBM', 'max_depth': 3, 'max_features': 7}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:10,402]\u001b[0m Trial 23 finished with value: 0.9001151608814281 and parameters: {'classifier': 'LightGBM', 'max_depth': 2, 'max_features': 7}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:10,878]\u001b[0m Trial 24 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 9}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:11,455]\u001b[0m Trial 25 finished with value: 0.903388761229818 and parameters: {'classifier': 'LightGBM', 'max_depth': 6, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:11,955]\u001b[0m Trial 26 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 5}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:17,915]\u001b[0m Trial 27 finished with value: 0.9034774603303429 and parameters: {'classifier': 'GradientBoostingClassifier', 'max_depth': 6, 'max_features': 12}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:19,724]\u001b[0m Trial 28 finished with value: 0.8951602699045291 and parameters: {'classifier': 'Random Forest', 'max_depth': 4, 'max_features': 8}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:24,004]\u001b[0m Trial 29 finished with value: 0.8985228746455558 and parameters: {'classifier': 'GradientBoostingClassifier', 'max_depth': 9, 'max_features': 3}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:24,375]\u001b[0m Trial 30 finished with value: 0.9001151608814281 and parameters: {'classifier': 'LightGBM', 'max_depth': 2, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:24,840]\u001b[0m Trial 31 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 9}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:25,433]\u001b[0m Trial 32 finished with value: 0.903388761229818 and parameters: {'classifier': 'LightGBM', 'max_depth': 6, 'max_features': 9}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:26,094]\u001b[0m Trial 33 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 10}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:26,756]\u001b[0m Trial 34 finished with value: 0.9010886669744012 and parameters: {'classifier': 'LightGBM', 'max_depth': 3, 'max_features': 11}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:27,715]\u001b[0m Trial 35 finished with value: 0.9006460168960867 and parameters: {'classifier': 'LightGBM', 'max_depth': 8, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:29,091]\u001b[0m Trial 36 finished with value: 0.9017076819573865 and parameters: {'classifier': 'XGBoost', 'max_depth': 2, 'max_features': 7}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:29,587]\u001b[0m Trial 37 finished with value: 0.903388761229818 and parameters: {'classifier': 'LightGBM', 'max_depth': 6, 'max_features': 3}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:42,906]\u001b[0m Trial 38 finished with value: 0.8968413022089434 and parameters: {'classifier': 'GradientBoostingClassifier', 'max_depth': 10, 'max_features': 16}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:45,686]\u001b[0m Trial 39 finished with value: 0.8989650785277067 and parameters: {'classifier': 'Random Forest', 'max_depth': 16, 'max_features': 5}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:47,386]\u001b[0m Trial 40 finished with value: 0.902769417470712 and parameters: {'classifier': 'XGBoost', 'max_depth': 4, 'max_features': 11}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:47,759]\u001b[0m Trial 41 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 5}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:48,190]\u001b[0m Trial 42 finished with value: 0.9010886669744012 and parameters: {'classifier': 'LightGBM', 'max_depth': 3, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:48,757]\u001b[0m Trial 43 finished with value: 0.9033889491018869 and parameters: {'classifier': 'LightGBM', 'max_depth': 5, 'max_features': 9}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:49,381]\u001b[0m Trial 44 finished with value: 0.9008229689009729 and parameters: {'classifier': 'LightGBM', 'max_depth': 14, 'max_features': 3}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:49,924]\u001b[0m Trial 45 finished with value: 0.9033889491018869 and parameters: {'classifier': 'LightGBM', 'max_depth': 5, 'max_features': 4}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:50,392]\u001b[0m Trial 46 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 7}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:50,659]\u001b[0m Trial 47 finished with value: 0.9001151608814281 and parameters: {'classifier': 'LightGBM', 'max_depth': 2, 'max_features': 5}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:51,980]\u001b[0m Trial 48 finished with value: 0.9002925121144606 and parameters: {'classifier': 'XGBoost', 'max_depth': 3, 'max_features': 8}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:56,744]\u001b[0m Trial 49 finished with value: 0.9008232741930847 and parameters: {'classifier': 'GradientBoostingClassifier', 'max_depth': 7, 'max_features': 5}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:57,373]\u001b[0m Trial 50 finished with value: 0.9006460168960867 and parameters: {'classifier': 'LightGBM', 'max_depth': 8, 'max_features': 4}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:57,843]\u001b[0m Trial 51 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 10}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:58,378]\u001b[0m Trial 52 finished with value: 0.9033889491018869 and parameters: {'classifier': 'LightGBM', 'max_depth': 5, 'max_features': 9}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:58,844]\u001b[0m Trial 53 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 13}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:59,433]\u001b[0m Trial 54 finished with value: 0.903388761229818 and parameters: {'classifier': 'LightGBM', 'max_depth': 6, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:14:59,860]\u001b[0m Trial 55 finished with value: 0.9010886669744012 and parameters: {'classifier': 'LightGBM', 'max_depth': 3, 'max_features': 8}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:01,783]\u001b[0m Trial 56 finished with value: 0.9002035781738497 and parameters: {'classifier': 'Random Forest', 'max_depth': 5, 'max_features': 7}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:02,208]\u001b[0m Trial 57 finished with value: 0.9010886669744012 and parameters: {'classifier': 'LightGBM', 'max_depth': 3, 'max_features': 12}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:02,838]\u001b[0m Trial 58 finished with value: 0.9010884086503065 and parameters: {'classifier': 'LightGBM', 'max_depth': 11, 'max_features': 8}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:03,300]\u001b[0m Trial 59 finished with value: 0.9033889491018869 and parameters: {'classifier': 'LightGBM', 'max_depth': 5, 'max_features': 9}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:03,673]\u001b[0m Trial 60 finished with value: 0.9001151608814281 and parameters: {'classifier': 'LightGBM', 'max_depth': 2, 'max_features': 7}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:04,150]\u001b[0m Trial 61 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 10}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:04,621]\u001b[0m Trial 62 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 10}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:05,061]\u001b[0m Trial 63 finished with value: 0.9033889491018869 and parameters: {'classifier': 'LightGBM', 'max_depth': 5, 'max_features': 11}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:05,502]\u001b[0m Trial 64 finished with value: 0.9010886669744012 and parameters: {'classifier': 'LightGBM', 'max_depth': 3, 'max_features': 9}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:09,512]\u001b[0m Trial 65 finished with value: 0.8988767551713197 and parameters: {'classifier': 'XGBoost', 'max_depth': 6, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:12,539]\u001b[0m Trial 66 finished with value: 0.9015308004045263 and parameters: {'classifier': 'GradientBoostingClassifier', 'max_depth': 4, 'max_features': 12}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:13,136]\u001b[0m Trial 67 finished with value: 0.9015308708565519 and parameters: {'classifier': 'LightGBM', 'max_depth': 7, 'max_features': 10}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:14,135]\u001b[0m Trial 68 finished with value: 0.8806511683482157 and parameters: {'classifier': 'Random Forest', 'max_depth': 2, 'max_features': 4}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:14,560]\u001b[0m Trial 69 finished with value: 0.9010886669744012 and parameters: {'classifier': 'LightGBM', 'max_depth': 3, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:15,021]\u001b[0m Trial 70 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 8}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:15,491]\u001b[0m Trial 71 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 3}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:16,047]\u001b[0m Trial 72 finished with value: 0.9033889491018869 and parameters: {'classifier': 'LightGBM', 'max_depth': 5, 'max_features': 5}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:16,514]\u001b[0m Trial 73 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 5}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:16,930]\u001b[0m Trial 74 finished with value: 0.9010886669744012 and parameters: {'classifier': 'LightGBM', 'max_depth': 3, 'max_features': 4}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:17,522]\u001b[0m Trial 75 finished with value: 0.903388761229818 and parameters: {'classifier': 'LightGBM', 'max_depth': 6, 'max_features': 5}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:18,053]\u001b[0m Trial 76 finished with value: 0.9033889491018869 and parameters: {'classifier': 'LightGBM', 'max_depth': 5, 'max_features': 7}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:18,520]\u001b[0m Trial 77 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:20,086]\u001b[0m Trial 78 finished with value: 0.9002925121144606 and parameters: {'classifier': 'XGBoost', 'max_depth': 3, 'max_features': 4}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:21,891]\u001b[0m Trial 79 finished with value: 0.899672839579234 and parameters: {'classifier': 'GradientBoostingClassifier', 'max_depth': 2, 'max_features': 5}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:22,642]\u001b[0m Trial 80 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:23,166]\u001b[0m Trial 81 finished with value: 0.9033889491018869 and parameters: {'classifier': 'LightGBM', 'max_depth': 5, 'max_features': 7}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:23,633]\u001b[0m Trial 82 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 7}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:24,049]\u001b[0m Trial 83 finished with value: 0.9010886669744012 and parameters: {'classifier': 'LightGBM', 'max_depth': 3, 'max_features': 11}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:24,416]\u001b[0m Trial 84 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 9}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:25,006]\u001b[0m Trial 85 finished with value: 0.903388761229818 and parameters: {'classifier': 'LightGBM', 'max_depth': 6, 'max_features': 8}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:26,785]\u001b[0m Trial 86 finished with value: 0.8992302834369541 and parameters: {'classifier': 'Random Forest', 'max_depth': 5, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:27,258]\u001b[0m Trial 87 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 5}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:27,578]\u001b[0m Trial 88 finished with value: 0.9010886669744012 and parameters: {'classifier': 'LightGBM', 'max_depth': 3, 'max_features': 7}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:28,001]\u001b[0m Trial 89 finished with value: 0.9010886669744012 and parameters: {'classifier': 'LightGBM', 'max_depth': 3, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:28,548]\u001b[0m Trial 90 finished with value: 0.9033889491018869 and parameters: {'classifier': 'LightGBM', 'max_depth': 5, 'max_features': 10}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:29,035]\u001b[0m Trial 91 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 11}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:29,521]\u001b[0m Trial 92 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 10}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:30,006]\u001b[0m Trial 93 finished with value: 0.9037425478195473 and parameters: {'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 9}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:30,553]\u001b[0m Trial 94 finished with value: 0.9033889491018869 and parameters: {'classifier': 'LightGBM', 'max_depth': 5, 'max_features': 10}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:30,977]\u001b[0m Trial 95 finished with value: 0.9010886669744012 and parameters: {'classifier': 'LightGBM', 'max_depth': 3, 'max_features': 9}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:31,627]\u001b[0m Trial 96 finished with value: 0.9003807885028307 and parameters: {'classifier': 'LightGBM', 'max_depth': 10, 'max_features': 8}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:34,516]\u001b[0m Trial 97 finished with value: 0.8981688532157405 and parameters: {'classifier': 'XGBoost', 'max_depth': 5, 'max_features': 6}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:36,945]\u001b[0m Trial 98 finished with value: 0.9025926768219034 and parameters: {'classifier': 'GradientBoostingClassifier', 'max_depth': 3, 'max_features': 7}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:15:37,326]\u001b[0m Trial 99 finished with value: 0.9001151608814281 and parameters: {'classifier': 'LightGBM', 'max_depth': 2, 'max_features': 5}. Best is trial 18 with value: 0.9037425478195473.\u001b[0m\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "FrozenTrial(number=18, state=TrialState.COMPLETE, values=[0.9037425478195473], datetime_start=datetime.datetime(2023, 4, 8, 18, 14, 6, 708460), datetime_complete=datetime.datetime(2023, 4, 8, 18, 14, 7, 179543), params={'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 6}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'classifier': CategoricalDistribution(choices=('Random Forest', 'XGBoost', 'LightGBM', 'GradientBoostingClassifier')), 'max_depth': IntDistribution(high=16, log=False, low=2, step=1), 'max_features': IntDistribution(high=16, log=False, low=2, step=1)}, trial_id=18, value=None)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "study.best_params"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "7cycW29BFagM",
        "outputId": "e4c6c2a3-6c5a-4a6f-9e86-41d229ef6283"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "{'classifier': 'LightGBM', 'max_depth': 4, 'max_features': 6}"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import lightgbm as lgb\n",
        "lgb_model = lgb.LGBMClassifier(random_state=15,max_depth=11,max_features=9)\n",
        "lgb_model.fit(X_test, y_test)\n",
        "\n",
        "print(accuracy_score(y_test, lgb_model.predict(X_test)))"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "5otZP1ZdFiq5",
        "outputId": "367e64d7-2d72-47d2-fa32-a4d624683229"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[LightGBM] [Warning] Unknown parameter: max_features\n",
            "0.9577988144740335\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "For Q3 and Q4 ,use the following data."
      ],
      "metadata": {
        "id": "jR2k2_rfg5PD"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "dr=pd.read_csv('https://raw.githubusercontent.com/ogut77/DataScience/main/data/diamond.csv')\n",
        "dr"
      ],
      "metadata": {
        "id": "1XezD-d0fKRD",
        "outputId": "ad53c5ff-6774-4464-cac8-c23e8cb8ec59",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        }
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "      Carat Weight              Cut Color Clarity Polish Symmetry Report  \\\n",
              "0             1.10            Ideal     H     SI1     VG       EX    GIA   \n",
              "1             0.83            Ideal     H     VS1     ID       ID   AGSL   \n",
              "2             0.85            Ideal     H     SI1     EX       EX    GIA   \n",
              "3             0.91            Ideal     E     SI1     VG       VG    GIA   \n",
              "4             0.83            Ideal     G     SI1     EX       EX    GIA   \n",
              "...            ...              ...   ...     ...    ...      ...    ...   \n",
              "5995          1.03            Ideal     D     SI1     EX       EX    GIA   \n",
              "5996          1.00        Very Good     D     SI1     VG       VG    GIA   \n",
              "5997          1.02            Ideal     D     SI1     EX       EX    GIA   \n",
              "5998          1.27  Signature-Ideal     G     VS1     EX       EX    GIA   \n",
              "5999          2.19            Ideal     E     VS1     EX       EX    GIA   \n",
              "\n",
              "      Price  \n",
              "0      5169  \n",
              "1      3470  \n",
              "2      3183  \n",
              "3      4370  \n",
              "4      3171  \n",
              "...     ...  \n",
              "5995   6250  \n",
              "5996   5328  \n",
              "5997   6157  \n",
              "5998  11206  \n",
              "5999  30507  \n",
              "\n",
              "[6000 rows x 8 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-01ef02bf-6c3e-4d0d-a32c-12d9632a36a8\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Carat Weight</th>\n",
              "      <th>Cut</th>\n",
              "      <th>Color</th>\n",
              "      <th>Clarity</th>\n",
              "      <th>Polish</th>\n",
              "      <th>Symmetry</th>\n",
              "      <th>Report</th>\n",
              "      <th>Price</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1.10</td>\n",
              "      <td>Ideal</td>\n",
              "      <td>H</td>\n",
              "      <td>SI1</td>\n",
              "      <td>VG</td>\n",
              "      <td>EX</td>\n",
              "      <td>GIA</td>\n",
              "      <td>5169</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.83</td>\n",
              "      <td>Ideal</td>\n",
              "      <td>H</td>\n",
              "      <td>VS1</td>\n",
              "      <td>ID</td>\n",
              "      <td>ID</td>\n",
              "      <td>AGSL</td>\n",
              "      <td>3470</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.85</td>\n",
              "      <td>Ideal</td>\n",
              "      <td>H</td>\n",
              "      <td>SI1</td>\n",
              "      <td>EX</td>\n",
              "      <td>EX</td>\n",
              "      <td>GIA</td>\n",
              "      <td>3183</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.91</td>\n",
              "      <td>Ideal</td>\n",
              "      <td>E</td>\n",
              "      <td>SI1</td>\n",
              "      <td>VG</td>\n",
              "      <td>VG</td>\n",
              "      <td>GIA</td>\n",
              "      <td>4370</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.83</td>\n",
              "      <td>Ideal</td>\n",
              "      <td>G</td>\n",
              "      <td>SI1</td>\n",
              "      <td>EX</td>\n",
              "      <td>EX</td>\n",
              "      <td>GIA</td>\n",
              "      <td>3171</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5995</th>\n",
              "      <td>1.03</td>\n",
              "      <td>Ideal</td>\n",
              "      <td>D</td>\n",
              "      <td>SI1</td>\n",
              "      <td>EX</td>\n",
              "      <td>EX</td>\n",
              "      <td>GIA</td>\n",
              "      <td>6250</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5996</th>\n",
              "      <td>1.00</td>\n",
              "      <td>Very Good</td>\n",
              "      <td>D</td>\n",
              "      <td>SI1</td>\n",
              "      <td>VG</td>\n",
              "      <td>VG</td>\n",
              "      <td>GIA</td>\n",
              "      <td>5328</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5997</th>\n",
              "      <td>1.02</td>\n",
              "      <td>Ideal</td>\n",
              "      <td>D</td>\n",
              "      <td>SI1</td>\n",
              "      <td>EX</td>\n",
              "      <td>EX</td>\n",
              "      <td>GIA</td>\n",
              "      <td>6157</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5998</th>\n",
              "      <td>1.27</td>\n",
              "      <td>Signature-Ideal</td>\n",
              "      <td>G</td>\n",
              "      <td>VS1</td>\n",
              "      <td>EX</td>\n",
              "      <td>EX</td>\n",
              "      <td>GIA</td>\n",
              "      <td>11206</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5999</th>\n",
              "      <td>2.19</td>\n",
              "      <td>Ideal</td>\n",
              "      <td>E</td>\n",
              "      <td>VS1</td>\n",
              "      <td>EX</td>\n",
              "      <td>EX</td>\n",
              "      <td>GIA</td>\n",
              "      <td>30507</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>6000 rows × 8 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-01ef02bf-6c3e-4d0d-a32c-12d9632a36a8')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-01ef02bf-6c3e-4d0d-a32c-12d9632a36a8 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-01ef02bf-6c3e-4d0d-a32c-12d9632a36a8');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "def Encoder(df):\n",
        "          from sklearn import preprocessing\n",
        "          columnsToEncode = list(df.select_dtypes(include=['category','object']))\n",
        "          le = preprocessing.LabelEncoder()\n",
        "          for feature in columnsToEncode:\n",
        "              try:\n",
        "                  df[feature] = le.fit_transform(df[feature])\n",
        "              except:\n",
        "                  print('Error encoding '+feature)\n",
        "          return df\n"
      ],
      "metadata": {
        "id": "zTuFQHpVfoxH"
      },
      "execution_count": 15,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "dr=Encoder(dr)\n",
        "dr"
      ],
      "metadata": {
        "id": "nxX6G5tNgWOh",
        "outputId": "35fb6d8f-aa4a-4cb3-d158-4a57a8fb78c1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        }
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "      Carat Weight  Cut  Color  Clarity  Polish  Symmetry  Report  Price\n",
              "0             1.10    2      4        2       3         0       1   5169\n",
              "1             0.83    2      4        3       2         2       0   3470\n",
              "2             0.85    2      4        2       0         0       1   3183\n",
              "3             0.91    2      1        2       3         3       1   4370\n",
              "4             0.83    2      3        2       0         0       1   3171\n",
              "...            ...  ...    ...      ...     ...       ...     ...    ...\n",
              "5995          1.03    2      0        2       0         0       1   6250\n",
              "5996          1.00    4      0        2       3         3       1   5328\n",
              "5997          1.02    2      0        2       0         0       1   6157\n",
              "5998          1.27    3      3        3       0         0       1  11206\n",
              "5999          2.19    2      1        3       0         0       1  30507\n",
              "\n",
              "[6000 rows x 8 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-8b312b92-67fa-467c-aee3-36e4c872cd36\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Carat Weight</th>\n",
              "      <th>Cut</th>\n",
              "      <th>Color</th>\n",
              "      <th>Clarity</th>\n",
              "      <th>Polish</th>\n",
              "      <th>Symmetry</th>\n",
              "      <th>Report</th>\n",
              "      <th>Price</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1.10</td>\n",
              "      <td>2</td>\n",
              "      <td>4</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>5169</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.83</td>\n",
              "      <td>2</td>\n",
              "      <td>4</td>\n",
              "      <td>3</td>\n",
              "      <td>2</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>3470</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.85</td>\n",
              "      <td>2</td>\n",
              "      <td>4</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>3183</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.91</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>4370</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.83</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>3171</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5995</th>\n",
              "      <td>1.03</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>6250</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5996</th>\n",
              "      <td>1.00</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "      <td>5328</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5997</th>\n",
              "      <td>1.02</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>6157</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5998</th>\n",
              "      <td>1.27</td>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>11206</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5999</th>\n",
              "      <td>2.19</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>30507</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>6000 rows × 8 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8b312b92-67fa-467c-aee3-36e4c872cd36')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-8b312b92-67fa-467c-aee3-36e4c872cd36 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-8b312b92-67fa-467c-aee3-36e4c872cd36');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 16
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "y = dr['Price'] #Output\n",
        "X = dr.drop('Price',axis=1)\n",
        "X"
      ],
      "metadata": {
        "id": "Kv7SnSAahe1-",
        "outputId": "868772e2-9e45-4fb8-b231-0ce9267d6a5e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 423
        }
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "      Carat Weight  Cut  Color  Clarity  Polish  Symmetry  Report\n",
              "0             1.10    2      4        2       3         0       1\n",
              "1             0.83    2      4        3       2         2       0\n",
              "2             0.85    2      4        2       0         0       1\n",
              "3             0.91    2      1        2       3         3       1\n",
              "4             0.83    2      3        2       0         0       1\n",
              "...            ...  ...    ...      ...     ...       ...     ...\n",
              "5995          1.03    2      0        2       0         0       1\n",
              "5996          1.00    4      0        2       3         3       1\n",
              "5997          1.02    2      0        2       0         0       1\n",
              "5998          1.27    3      3        3       0         0       1\n",
              "5999          2.19    2      1        3       0         0       1\n",
              "\n",
              "[6000 rows x 7 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-3024a40f-4fa3-464d-bf0c-f261e531cb96\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Carat Weight</th>\n",
              "      <th>Cut</th>\n",
              "      <th>Color</th>\n",
              "      <th>Clarity</th>\n",
              "      <th>Polish</th>\n",
              "      <th>Symmetry</th>\n",
              "      <th>Report</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>1.10</td>\n",
              "      <td>2</td>\n",
              "      <td>4</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.83</td>\n",
              "      <td>2</td>\n",
              "      <td>4</td>\n",
              "      <td>3</td>\n",
              "      <td>2</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.85</td>\n",
              "      <td>2</td>\n",
              "      <td>4</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.91</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.83</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>...</th>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "      <td>...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5995</th>\n",
              "      <td>1.03</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5996</th>\n",
              "      <td>1.00</td>\n",
              "      <td>4</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5997</th>\n",
              "      <td>1.02</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5998</th>\n",
              "      <td>1.27</td>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5999</th>\n",
              "      <td>2.19</td>\n",
              "      <td>2</td>\n",
              "      <td>1</td>\n",
              "      <td>3</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>6000 rows × 7 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-3024a40f-4fa3-464d-bf0c-f261e531cb96')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-3024a40f-4fa3-464d-bf0c-f261e531cb96 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-3024a40f-4fa3-464d-bf0c-f261e531cb96');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "from sklearn.model_selection import  train_test_split\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25, random_state=17)"
      ],
      "metadata": {
        "id": "KOLT3hfbgz7F"
      },
      "execution_count": 18,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Q3)Using Linear Regression,Decison Tree Random Forest,XGBoost, Light GBM and Gradient Boosting Classifier with default parameters (no parameter specifications except random_state) calculate R2 statistics on test data. Which method gives the best accuracy on test data"
      ],
      "metadata": {
        "id": "GzkaMEVGZ0pS"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from pandas.core.common import random_state\n",
        "def evalmetric(y,ypred):\n",
        " import numpy as np\n",
        " e = y - ypred \n",
        " mse_f = np.mean(e**2)\n",
        " rmse_f = np.sqrt(mse_f)\n",
        " mae_f = np.mean(abs(e))\n",
        " mape_f = 100*np.mean(abs(e/y))\n",
        " r2_f = 1-(sum(e**2)/sum((y-np.mean(y))**2))\n",
        " print(\"MSE:\", mse_f)\n",
        " print(\"RMSE:\", rmse_f)\n",
        " print(\"MAE:\",mae_f)\n",
        " print(\"MAPE:\",mape_f)\n",
        " print(\"R-Squared:\", r2_f)\n",
        "\n",
        "#Linear Regression\n",
        "from sklearn.linear_model import LinearRegression\n",
        "reg = LinearRegression()\n",
        "reg.fit(X_train, y_train)\n",
        "\n",
        "print('Linear Regression')\n",
        "pred=reg.predict(X_test)\n",
        "evalmetric(y_test,pred)\n",
        "print( )\n",
        "\n",
        "#Decision Tree\n",
        "from sklearn.tree import DecisionTreeRegressor\n",
        "dtr = DecisionTreeRegressor(random_state=15)\n",
        "dtr.fit(X_train,y_train)\n",
        "\n",
        "print('Decision Tree')\n",
        "y_pred2 = dtr.predict(X_test)\n",
        "evalmetric(y_test,y_pred2)\n",
        "print( )\n",
        "\n",
        "#Random Forest\n",
        "from sklearn.ensemble import RandomForestRegressor\n",
        "rfr = RandomForestRegressor(random_state=15)\n",
        "rfr.fit(X_train,y_train)\n",
        "\n",
        "print('Random Forest')\n",
        "y_pred2 = rfr.predict(X_test)\n",
        "evalmetric(y_test,y_pred2)\n",
        "print( )\n",
        "\n",
        "#XGBoost\n",
        "from xgboost import XGBRegressor\n",
        "xgr = XGBRegressor(random_state=15)\n",
        "xgr.fit(X_train, y_train)\n",
        "\n",
        "print('TXGBoost')\n",
        "y_pred2 = xgr.predict(X_test)\n",
        "evalmetric(y_test,y_pred2)\n",
        "print( )\n",
        "\n",
        "#Light GBM\n",
        "from time import time\n",
        "import lightgbm as lgb\n",
        "t0 = time()\n",
        "lgr = lgb.LGBMRegressor(random_state=15)\n",
        "lgr.fit(X_train, y_train)\n",
        "\n",
        "print('Light GBM')\n",
        "y_pred2 = lgr.predict(X_test)\n",
        "evalmetric(y_test,y_pred2)\n",
        "print( )\n",
        "\n",
        "#Gradient Boosting\n",
        "from sklearn.ensemble import GradientBoostingRegressor\n",
        "from time import time\n",
        "\n",
        "gbr = GradientBoostingRegressor(random_state=15)\n",
        "gbr.fit(X_train, y_train)\n",
        "\n",
        "print('Gradient Boosting')\n",
        "y_pred2 = gbr.predict(X_test)\n",
        "evalmetric(y_test,y_pred2)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hhCtffU1GHTb",
        "outputId": "2d0ad330-10f2-44a5-a39b-0065fdc071a6"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Linear Regression\n",
            "MSE: 18292450.04932515\n",
            "RMSE: 4276.9673893221525\n",
            "MAE: 2701.9972766097503\n",
            "MAPE: 30.68442425409787\n",
            "R-Squared: 0.8233845829696964\n",
            "\n",
            "Decision Tree\n",
            "MSE: 3706593.620659981\n",
            "RMSE: 1925.2515733430737\n",
            "MAE: 989.8429285714286\n",
            "MAPE: 7.822563429503446\n",
            "R-Squared: 0.9642124714672173\n",
            "\n",
            "Random Forest\n",
            "MSE: 2053745.888353441\n",
            "RMSE: 1433.08963025815\n",
            "MAE: 766.9832157841048\n",
            "MAPE: 6.139558134085698\n",
            "R-Squared: 0.9801708800315027\n",
            "\n",
            "TXGBoost\n",
            "MSE: 2039193.623695792\n",
            "RMSE: 1428.003369637408\n",
            "MAE: 702.000625\n",
            "MAPE: 5.333095525431356\n",
            "R-Squared: 0.980311383588124\n",
            "\n",
            "Light GBM\n",
            "MSE: 1940840.8294753158\n",
            "RMSE: 1393.1406352107153\n",
            "MAE: 701.0864858904296\n",
            "MAPE: 5.433753614630331\n",
            "R-Squared: 0.9812609895578275\n",
            "\n",
            "Gradient Boosting\n",
            "MSE: 2702865.535065859\n",
            "RMSE: 1644.0393958375387\n",
            "MAE: 878.4993508475268\n",
            "MAPE: 7.394144503096012\n",
            "R-Squared: 0.9739035655494326\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Q4) Using optuna hyperparmeter optimization technique (100 trial)  with Random Forest,XGBoost, Light GBM and Gradient Boosting Regressor\n",
        "\n",
        "a)find best methods with  parameters  using Cross validation (CV=3) technique for the range of   parameters below. What are the best parameters for the method with highest cross validation R2?\n",
        "\n",
        "\"max_depth\": range(2, 7), \"max_features\": range(2, 7)\n",
        "\n",
        " b)Evaluate the performance of the  method with highest cross validation R2 on test data. What is the R2 value?\n"
      ],
      "metadata": {
        "id": "nksEpGl5bDv3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install optuna"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "l17wM78nGSYb",
        "outputId": "005092da-b4bb-4f75-98a7-1b638b69332d"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/\n",
            "Requirement already satisfied: optuna in /usr/local/lib/python3.9/dist-packages (3.1.1)\n",
            "Requirement already satisfied: PyYAML in /usr/local/lib/python3.9/dist-packages (from optuna) (6.0)\n",
            "Requirement already satisfied: cmaes>=0.9.1 in /usr/local/lib/python3.9/dist-packages (from optuna) (0.9.1)\n",
            "Requirement already satisfied: packaging>=20.0 in /usr/local/lib/python3.9/dist-packages (from optuna) (23.0)\n",
            "Requirement already satisfied: tqdm in /usr/local/lib/python3.9/dist-packages (from optuna) (4.65.0)\n",
            "Requirement already satisfied: numpy in /usr/local/lib/python3.9/dist-packages (from optuna) (1.22.4)\n",
            "Requirement already satisfied: colorlog in /usr/local/lib/python3.9/dist-packages (from optuna) (6.7.0)\n",
            "Requirement already satisfied: alembic>=1.5.0 in /usr/local/lib/python3.9/dist-packages (from optuna) (1.10.3)\n",
            "Requirement already satisfied: sqlalchemy>=1.3.0 in /usr/local/lib/python3.9/dist-packages (from optuna) (1.4.47)\n",
            "Requirement already satisfied: typing-extensions>=4 in /usr/local/lib/python3.9/dist-packages (from alembic>=1.5.0->optuna) (4.5.0)\n",
            "Requirement already satisfied: Mako in /usr/local/lib/python3.9/dist-packages (from alembic>=1.5.0->optuna) (1.2.4)\n",
            "Requirement already satisfied: greenlet!=0.4.17 in /usr/local/lib/python3.9/dist-packages (from sqlalchemy>=1.3.0->optuna) (2.0.2)\n",
            "Requirement already satisfied: MarkupSafe>=0.9.2 in /usr/local/lib/python3.9/dist-packages (from Mako->alembic>=1.5.0->optuna) (2.1.2)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import optuna\n",
        "\n",
        "import sklearn.datasets\n",
        "import sklearn.ensemble\n",
        "import sklearn.model_selection\n",
        "import sklearn.svm\n",
        "\n",
        "def objective(trial):\n",
        "    x, y = X_train,y_train\n",
        "\n",
        "    regressor_name = trial.suggest_categorical(\"regressor\", [\"GradientBoosting\", \"XGBoost\",\"LightGBM\",\"RandomForest\"])\n",
        "    if regressor_name == \"GradientBoosting\":\n",
        "         max_depth = trial.suggest_int(\"max_depth\", 2,7)\n",
        "         max_features = trial.suggest_int(\"max_features\", 2,7)\n",
        "         regressor_obj = sklearn.ensemble.GradientBoostingRegressor(random_state=15,  max_depth=max_depth, max_features=max_features )\n",
        "    \n",
        "    elif regressor_name == \"XGBoost\":\n",
        "         from xgboost import XGBRegressor\n",
        "         max_depth = trial.suggest_int(\"max_depth\", 2,7)\n",
        "         max_features = trial.suggest_int(\"max_features\", 2,7)\n",
        "         regressor_obj = XGBRegressor(random_state=15,  max_depth=max_depth, max_features=max_features )\n",
        "        \n",
        "         \n",
        "\n",
        "    elif regressor_name == \"LightGBM\":\n",
        "         import lightgbm as lgb\n",
        "         max_depth = trial.suggest_int(\"max_depth\", 2,7)\n",
        "         max_features = trial.suggest_int(\"max_features\", 2,7)\n",
        "         regressor_obj = lgb.LGBMRegressor(random_state=15,  max_depth=max_depth, max_features=max_features )\n",
        "        \n",
        "       \n",
        "   \n",
        "    else:\n",
        "         max_depth = trial.suggest_int(\"max_depth\", 2,7)\n",
        "         max_features = trial.suggest_int(\"max_features\", 2,7)\n",
        "         regressor_obj = sklearn.ensemble.RandomForestRegressor(random_state=15,  max_depth=max_depth, max_features=max_features )\n",
        "\n",
        "     \n",
        "\n",
        "    r2=sklearn.model_selection.cross_val_score(regressor_obj, x, y, n_jobs=-1, cv=3).mean()\n",
        "   \n",
        "    return r2\n",
        "\n",
        "\n",
        "if __name__ == \"__main__\":\n",
        "    study = optuna.create_study(direction=\"maximize\")\n",
        "    study.optimize(objective, n_trials=100)\n",
        "    print(study.best_trial)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_w1t4v17GXnL",
        "outputId": "01ef0241-46b4-4ab9-c50a-e6dca76e5a6c"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "\u001b[32m[I 2023-04-08 18:20:54,122]\u001b[0m A new study created in memory with name: no-name-ffd3b7d3-8337-4e23-b00d-df5bb7374b2f\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:00,651]\u001b[0m Trial 0 finished with value: 0.9765224083258769 and parameters: {'regressor': 'LightGBM', 'max_depth': 5, 'max_features': 7}. Best is trial 0 with value: 0.9765224083258769.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:02,143]\u001b[0m Trial 1 finished with value: 0.9783260831547227 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 3, 'max_features': 5}. Best is trial 1 with value: 0.9783260831547227.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:02,358]\u001b[0m Trial 2 finished with value: 0.9541001035136575 and parameters: {'regressor': 'LightGBM', 'max_depth': 2, 'max_features': 5}. Best is trial 1 with value: 0.9783260831547227.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:04,416]\u001b[0m Trial 3 finished with value: 0.9831543180213996 and parameters: {'regressor': 'XGBoost', 'max_depth': 5, 'max_features': 4}. Best is trial 3 with value: 0.9831543180213996.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:06,627]\u001b[0m Trial 4 finished with value: 0.984857971675666 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 5}. Best is trial 4 with value: 0.984857971675666.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:08,173]\u001b[0m Trial 5 finished with value: 0.9843782262600483 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 7}. Best is trial 4 with value: 0.984857971675666.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:08,429]\u001b[0m Trial 6 finished with value: 0.9761140473772666 and parameters: {'regressor': 'LightGBM', 'max_depth': 4, 'max_features': 2}. Best is trial 4 with value: 0.984857971675666.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:09,658]\u001b[0m Trial 7 finished with value: 0.9808302249981805 and parameters: {'regressor': 'XGBoost', 'max_depth': 6, 'max_features': 2}. Best is trial 4 with value: 0.984857971675666.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:10,217]\u001b[0m Trial 8 finished with value: 0.9663857447280503 and parameters: {'regressor': 'XGBoost', 'max_depth': 2, 'max_features': 7}. Best is trial 4 with value: 0.984857971675666.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:10,560]\u001b[0m Trial 9 finished with value: 0.9512701095198765 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 2, 'max_features': 4}. Best is trial 4 with value: 0.984857971675666.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:11,451]\u001b[0m Trial 10 finished with value: 0.972221867577607 and parameters: {'regressor': 'RandomForest', 'max_depth': 7, 'max_features': 6}. Best is trial 4 with value: 0.984857971675666.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:12,334]\u001b[0m Trial 11 finished with value: 0.984430117019658 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 7, 'max_features': 6}. Best is trial 4 with value: 0.984857971675666.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:13,224]\u001b[0m Trial 12 finished with value: 0.984430117019658 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 7, 'max_features': 6}. Best is trial 4 with value: 0.984857971675666.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:13,747]\u001b[0m Trial 13 finished with value: 0.9839030144679022 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 3}. Best is trial 4 with value: 0.984857971675666.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:14,415]\u001b[0m Trial 14 finished with value: 0.9185026936953719 and parameters: {'regressor': 'RandomForest', 'max_depth': 4, 'max_features': 6}. Best is trial 4 with value: 0.984857971675666.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:15,211]\u001b[0m Trial 15 finished with value: 0.9844984658243762 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 7, 'max_features': 5}. Best is trial 4 with value: 0.984857971675666.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:15,754]\u001b[0m Trial 16 finished with value: 0.9841571098443246 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 4}. Best is trial 4 with value: 0.984857971675666.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:16,480]\u001b[0m Trial 17 finished with value: 0.9851775266807685 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:17,330]\u001b[0m Trial 18 finished with value: 0.8691505687471929 and parameters: {'regressor': 'RandomForest', 'max_depth': 4, 'max_features': 3}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:18,533]\u001b[0m Trial 19 finished with value: 0.9851775266807685 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:19,367]\u001b[0m Trial 20 finished with value: 0.9839030144679022 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 3}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:20,277]\u001b[0m Trial 21 finished with value: 0.984857971675666 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:20,980]\u001b[0m Trial 22 finished with value: 0.9851775266807685 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:21,613]\u001b[0m Trial 23 finished with value: 0.9843093992594559 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 4}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:22,390]\u001b[0m Trial 24 finished with value: 0.9842632699866831 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 6}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:23,170]\u001b[0m Trial 25 finished with value: 0.9844984658243762 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 7, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:23,363]\u001b[0m Trial 26 finished with value: 0.9758941121368242 and parameters: {'regressor': 'LightGBM', 'max_depth': 6, 'max_features': 4}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:24,162]\u001b[0m Trial 27 finished with value: 0.9640929729158358 and parameters: {'regressor': 'RandomForest', 'max_depth': 6, 'max_features': 6}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:24,656]\u001b[0m Trial 28 finished with value: 0.9831543180213996 and parameters: {'regressor': 'XGBoost', 'max_depth': 5, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:24,821]\u001b[0m Trial 29 finished with value: 0.9765224083258769 and parameters: {'regressor': 'LightGBM', 'max_depth': 5, 'max_features': 7}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:25,219]\u001b[0m Trial 30 finished with value: 0.9756878022034616 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 3, 'max_features': 4}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:25,840]\u001b[0m Trial 31 finished with value: 0.984857971675666 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:26,546]\u001b[0m Trial 32 finished with value: 0.9851775266807685 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:27,341]\u001b[0m Trial 33 finished with value: 0.9844984658243762 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 7, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:28,039]\u001b[0m Trial 34 finished with value: 0.9851775266807685 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:28,827]\u001b[0m Trial 35 finished with value: 0.9842632699866831 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 6}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:29,529]\u001b[0m Trial 36 finished with value: 0.9836001110930862 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 7, 'max_features': 4}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:29,727]\u001b[0m Trial 37 finished with value: 0.9758941121368242 and parameters: {'regressor': 'LightGBM', 'max_depth': 6, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:30,283]\u001b[0m Trial 38 finished with value: 0.9831543180213996 and parameters: {'regressor': 'XGBoost', 'max_depth': 5, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:31,286]\u001b[0m Trial 39 finished with value: 0.9848066097721494 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 4, 'max_features': 6}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:31,919]\u001b[0m Trial 40 finished with value: 0.9756878022034616 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 3, 'max_features': 4}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:33,130]\u001b[0m Trial 41 finished with value: 0.9851775266807685 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:34,020]\u001b[0m Trial 42 finished with value: 0.9851775266807685 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:34,737]\u001b[0m Trial 43 finished with value: 0.9851775266807685 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:35,630]\u001b[0m Trial 44 finished with value: 0.984430117019658 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 7, 'max_features': 6}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:36,129]\u001b[0m Trial 45 finished with value: 0.9831543180213996 and parameters: {'regressor': 'XGBoost', 'max_depth': 5, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:36,744]\u001b[0m Trial 46 finished with value: 0.9843093992594559 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 6, 'max_features': 4}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:37,526]\u001b[0m Trial 47 finished with value: 0.9724812559261702 and parameters: {'regressor': 'RandomForest', 'max_depth': 7, 'max_features': 5}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:37,715]\u001b[0m Trial 48 finished with value: 0.9758941121368242 and parameters: {'regressor': 'LightGBM', 'max_depth': 6, 'max_features': 6}. Best is trial 17 with value: 0.9851775266807685.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:38,443]\u001b[0m Trial 49 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:39,202]\u001b[0m Trial 50 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:39,951]\u001b[0m Trial 51 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:40,578]\u001b[0m Trial 52 finished with value: 0.9854252425741864 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 4, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:41,217]\u001b[0m Trial 53 finished with value: 0.9854252425741864 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 4, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:41,849]\u001b[0m Trial 54 finished with value: 0.9854252425741864 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 4, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:42,492]\u001b[0m Trial 55 finished with value: 0.9854252425741864 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 4, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:43,114]\u001b[0m Trial 56 finished with value: 0.9854252425741864 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 4, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:43,912]\u001b[0m Trial 57 finished with value: 0.9070516695109928 and parameters: {'regressor': 'RandomForest', 'max_depth': 4, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:44,758]\u001b[0m Trial 58 finished with value: 0.9793776037779164 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 3, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:45,388]\u001b[0m Trial 59 finished with value: 0.9838296694494307 and parameters: {'regressor': 'XGBoost', 'max_depth': 4, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:46,391]\u001b[0m Trial 60 finished with value: 0.9854252425741864 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 4, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:47,378]\u001b[0m Trial 61 finished with value: 0.9854252425741864 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 4, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:47,997]\u001b[0m Trial 62 finished with value: 0.9854252425741864 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 4, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:48,735]\u001b[0m Trial 63 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:49,479]\u001b[0m Trial 64 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:50,214]\u001b[0m Trial 65 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:50,961]\u001b[0m Trial 66 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:51,703]\u001b[0m Trial 67 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:51,864]\u001b[0m Trial 68 finished with value: 0.9765224083258769 and parameters: {'regressor': 'LightGBM', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:52,603]\u001b[0m Trial 69 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:53,333]\u001b[0m Trial 70 finished with value: 0.9446194900183317 and parameters: {'regressor': 'RandomForest', 'max_depth': 5, 'max_features': 6}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:54,063]\u001b[0m Trial 71 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:54,813]\u001b[0m Trial 72 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:55,559]\u001b[0m Trial 73 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:56,304]\u001b[0m Trial 74 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:57,049]\u001b[0m Trial 75 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:57,921]\u001b[0m Trial 76 finished with value: 0.9852871555716267 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 6}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:58,546]\u001b[0m Trial 77 finished with value: 0.9784438476442162 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 2}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:21:59,829]\u001b[0m Trial 78 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:00,682]\u001b[0m Trial 79 finished with value: 0.9831543180213996 and parameters: {'regressor': 'XGBoost', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:01,472]\u001b[0m Trial 80 finished with value: 0.9852871555716267 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 6}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:02,213]\u001b[0m Trial 81 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:02,964]\u001b[0m Trial 82 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:03,736]\u001b[0m Trial 83 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:04,481]\u001b[0m Trial 84 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:05,221]\u001b[0m Trial 85 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:05,973]\u001b[0m Trial 86 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:06,151]\u001b[0m Trial 87 finished with value: 0.9765224083258769 and parameters: {'regressor': 'LightGBM', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:06,823]\u001b[0m Trial 88 finished with value: 0.9852871555716267 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 6}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:07,551]\u001b[0m Trial 89 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:08,273]\u001b[0m Trial 90 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:09,006]\u001b[0m Trial 91 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:09,747]\u001b[0m Trial 92 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:10,477]\u001b[0m Trial 93 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:11,332]\u001b[0m Trial 94 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:12,325]\u001b[0m Trial 95 finished with value: 0.9219702558676274 and parameters: {'regressor': 'RandomForest', 'max_depth': 5, 'max_features': 3}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:13,608]\u001b[0m Trial 96 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:14,615]\u001b[0m Trial 97 finished with value: 0.9852871555716267 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 6}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:15,114]\u001b[0m Trial 98 finished with value: 0.9831543180213996 and parameters: {'regressor': 'XGBoost', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n",
            "\u001b[32m[I 2023-04-08 18:22:15,894]\u001b[0m Trial 99 finished with value: 0.9854840921446306 and parameters: {'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}. Best is trial 49 with value: 0.9854840921446306.\u001b[0m\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "FrozenTrial(number=49, state=TrialState.COMPLETE, values=[0.9854840921446306], datetime_start=datetime.datetime(2023, 4, 8, 18, 21, 37, 717244), datetime_complete=datetime.datetime(2023, 4, 8, 18, 21, 38, 443489), params={'regressor': 'GradientBoosting', 'max_depth': 5, 'max_features': 7}, user_attrs={}, system_attrs={}, intermediate_values={}, distributions={'regressor': CategoricalDistribution(choices=('GradientBoosting', 'XGBoost', 'LightGBM', 'RandomForest')), 'max_depth': IntDistribution(high=7, log=False, low=2, step=1), 'max_features': IntDistribution(high=7, log=False, low=2, step=1)}, trial_id=49, value=None)\n"
          ]
        }
      ]
    }
  ]
}