3 - XGBoost.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "# Ames Housing Dataset - XGboost Regressor\n",
    "\n",
    "> Gianmaria Pizzo - 872966@stud.unive.it\n",
    "\n",
    "These notebooks represent the project submission for the course [Data and Web Mining](https://www.unive.it/data/course/337525) by Professor [Claudio Lucchese](https://www.unive.it/data/people/5590426) at [Ca' Foscari University of Venice](https://www.unive.it).\n",
    "\n",
    "---\n",
    "\n",
    "## Structure of this notebook\n",
    "\n",
    "This notebook covers the following points\n",
    "* The idea\n",
    "* Tuning:\n",
    "    * Automatic: GridSearchCV Hyperparameters tuning for XGBoost.\n",
    "    * Manual\n",
    "* Model validation\n",
    "* Results\n",
    "* Analysis of worst and best predictions.\n",
    "\n",
    "---\n",
    "\n",
    "### Before running this notebook\n",
    "\n",
    "To avoid issues, before running the following notebook it is best to\n",
    "* Clean previous cell outputs\n",
    "* Restart the kernel\n",
    "\n",
    "---\n",
    "\n",
    "## The idea\n",
    "\n",
    "As we know, different predictors have different flaws and strengths. This means we can train multiple models in order to exploit what they learnt and obtain a more accurate result.\n",
    "\n",
    "As we are using a boosting method, we expect to find some level of overfitting when testing it on the dataset where the outliers and most noise were removed. \n",
    "\n",
    "Plus, as the dataset shows very few instances, it migth be better to use this kind of model on a larger dataset.\n",
    "\n",
    "However, there should be some level of improvement given the boosting algorithm will try to lower the bias."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "### Environment, Globals and Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already satisfied: mlxtend in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (0.22.0)\n",
      "Requirement already satisfied: pandas>=0.24.2 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from mlxtend) (1.3.5)\n",
      "Requirement already satisfied: matplotlib>=3.0.0 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from mlxtend) (3.5.0)\n",
      "Requirement already satisfied: scipy>=1.2.1 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from mlxtend) (1.10.0)\n",
      "Requirement already satisfied: setuptools in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from mlxtend) (65.6.3)\n",
      "Requirement already satisfied: joblib>=0.13.2 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from mlxtend) (1.1.1)\n",
      "Requirement already satisfied: scikit-learn>=1.0.2 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from mlxtend) (1.2.1)\n",
      "Requirement already satisfied: numpy>=1.16.2 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from mlxtend) (1.20.3)\n",
      "Requirement already satisfied: kiwisolver>=1.0.1 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from matplotlib>=3.0.0->mlxtend) (1.4.4)\n",
      "Requirement already satisfied: pyparsing>=2.2.1 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from matplotlib>=3.0.0->mlxtend) (3.0.9)\n",
      "Requirement already satisfied: cycler>=0.10 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from matplotlib>=3.0.0->mlxtend) (0.11.0)\n",
      "Requirement already satisfied: python-dateutil>=2.7 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from matplotlib>=3.0.0->mlxtend) (2.8.2)\n",
      "Requirement already satisfied: fonttools>=4.22.0 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from matplotlib>=3.0.0->mlxtend) (4.25.0)\n",
      "Requirement already satisfied: packaging>=20.0 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from matplotlib>=3.0.0->mlxtend) (22.0)\n",
      "Requirement already satisfied: pillow>=6.2.0 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from matplotlib>=3.0.0->mlxtend) (8.4.0)\n",
      "Requirement already satisfied: pytz>=2017.3 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from pandas>=0.24.2->mlxtend) (2022.7)\n",
      "Requirement already satisfied: six>=1.5 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from python-dateutil>=2.7->matplotlib>=3.0.0->mlxtend) (1.16.0)\n",
      "Requirement already satisfied: threadpoolctl>=2.0.0 in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from scikit-learn>=1.0.2->mlxtend) (2.2.0)\n",
      "Requirement already satisfied: xgboost in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (1.6.0)\n",
      "Requirement already satisfied: scipy in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from xgboost) (1.10.0)\n",
      "Requirement already satisfied: numpy in c:\\users\\paythepizzo\\anaconda3\\lib\\site-packages (from xgboost) (1.20.3)\n"
     ]
    }
   ],
   "source": [
    "!pip install mlxtend\n",
    "!pip install xgboost"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Interactive\n",
    "%matplotlib notebook\n",
    "# Static\n",
    "# %matplotlib inline\n",
    "\n",
    "# Environment for this notebook\n",
    "import os\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "import scipy\n",
    "import warnings\n",
    "import sklearn \n",
    "import IPython\n",
    "import xgboost\n",
    "from scipy import stats\n",
    "from xgboost import XGBRegressor\n",
    "from sklearn.model_selection import RepeatedKFold\n",
    "\n",
    "# Set the style for the plots\n",
    "sns.set()\n",
    "plt.style.use('ggplot')\n",
    "sns.set_style(\"darkgrid\")\n",
    "# Ignore warnings\n",
    "warnings.filterwarnings('ignore') \n",
    "\n",
    "# Working folder\n",
    "WORKING_DIR = os.getcwd()\n",
    "# Resources folder\n",
    "RESOURCES_DIR = os.path.join(os.getcwd(), 'resources')\n",
    "# Name of file\n",
    "IN_LABEL = 'ames_housing_out_21.csv'\n",
    "IN_LABEL2 = 'ames_housing_out_22.csv'\n",
    "ORIG_LABEL = 'ames_housing_out_22_orig.csv'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Utils Module\n",
    "\n",
    "def sort_alphabetically(dataset, last_label = None):\n",
    "    \"\"\"\n",
    "    Sorts the dataset alphabetically \n",
    "\n",
    "    :param dataset: a pd.DataFrame\n",
    "    :param last_label: a str containing an existing column label in the dataset\n",
    "    :returns: pd.DataFrame\n",
    "    \"\"\"\n",
    "    # Sort\n",
    "    dataset = dataset.reindex(sorted(dataset.columns), axis=1)\n",
    "    # Move target column to last index\n",
    "    if last_label is not None:\n",
    "        col = dataset.pop(last_label)\n",
    "        dataset.insert(dataset.shape[1], last_label, col)\n",
    "    return dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# Module for train test split\n",
    "\n",
    "def get_X_y(dataset, label, ignore=None):\n",
    "    \"\"\"\n",
    "    Returns X and y and ignores labels in ignore\n",
    "    :param dataset: a pd.DataFrame\n",
    "    :param label: a str containing an existing target column label in the dataset\n",
    "    :param ignore: a list of str containing an existing column label in the dataset to ignore\n",
    "    :returns: tuple of pd.DataFrame\n",
    "    \"\"\"\n",
    "    if ignore is not None:\n",
    "        # Drop the labels \n",
    "        all_columns = list(dataset.columns)\n",
    "        # Include only columns that are existing \n",
    "        to_drop = [i for i in all_columns if i in ignore] +[label]\n",
    "        return dataset.drop(columns=to_drop), dataset[[label]]\n",
    "    return dataset.drop(columns=[label]), dataset[[label]]\n",
    "\n",
    "def get_train_test(X, y, size = 0.2, state = 33):\n",
    "    \"\"\"\n",
    "    Returns X_train_[size], X_test, y_train_[size], y_test\n",
    "    :param X: a pd.DataFrame without the target column\n",
    "    :param y: a pd.DataFrame with one column, the target\n",
    "    :param size: a float representing the fraction for the test size\n",
    "    :param state: an integer representing the random state for the test\n",
    "    :returns: 4 pd.DataFrame usually called \"X_train_[size], X_test, y_train_[size], y_test\"\n",
    "    \"\"\"\n",
    "    return train_test_split(X, y, test_size=size, random_state = state)\n",
    "\n",
    "def get_train_val_test(X, y, size_t=0.2, size_v=0.25, state_v = 42):\n",
    "    \"\"\"\n",
    "    Returns X_train, X_valid, X_test, y_train, y_valid, y_test\n",
    "    :param X: a pd.DataFrame without the target column\n",
    "    :param y: a pd.DataFrame with one column, the target\n",
    "    :param size_t: a float representing the fraction for the test size\n",
    "    :param size_v: a float representing the fraction for the validation\n",
    "    :param state_v: an integer representing the random state for the validation\n",
    "    :returns: 6 pd.DataFrame usually called X_train, X_valid, X_test, y_train, y_valid, y_test\n",
    "    \"\"\"\n",
    "    X_train_s, X_test, y_train_s, y_test = get_train_test(X, y, size = size_t)\n",
    "    X_train, X_valid, y_train, y_valid = get_train_test(X_train_s, y_train_s, size = size_v, state = state_v)\n",
    "    return X_train, X_valid, X_test, y_train, y_valid, y_test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import LeaveOneOut, GridSearchCV\n",
    "from sklearn.metrics import mean_squared_error, mean_squared_log_error, mean_absolute_error, r2_score, max_error \n",
    "from mlxtend.evaluate import bias_variance_decomp\n",
    "\n",
    "# Module for traininig and testing\n",
    "def get_regression_metrics(y_test, y_pred):\n",
    "    metrics = {\n",
    "            \"RMSE\": mean_squared_error(y_true=y_test, y_pred=y_pred, squared=False),\n",
    "            \"MSE\": mean_squared_error(y_true=y_test, y_pred=y_pred),\n",
    "            \"MSLE\": mean_squared_log_error(y_true=y_test, y_pred=y_pred),\n",
    "            \"MAE\": mean_absolute_error(y_true=y_test, y_pred=y_pred),\n",
    "            \"R2\": r2_score(y_true=y_test, y_pred=y_pred),\n",
    "            \"MAX_Err\": max_error(y_true=y_test, y_pred=y_pred)}\n",
    "    return metrics\n",
    "\n",
    "\n",
    "def get_bias_variance_decomp(dataset, model, label, split_size, ignore, \n",
    "                             num_rounds=50, random_state=230324945):\n",
    "    # Get split\n",
    "    X, y = get_X_y(dataset, label=label, ignore=None)\n",
    "    X_train, X_test, y_train, y_test = get_train_test(X, y, size = split_size, \n",
    "                                                      state = random_state)\n",
    "    # Only accepts np.arrays\n",
    "    mse, bias, var = bias_variance_decomp(estimator=model, \n",
    "                                          X_train=X_train.values, \n",
    "                                          y_train=y_train.values, \n",
    "                                          X_test=X_test.values, \n",
    "                                          y_test=y_test.values, \n",
    "                                          loss='mse', num_rounds=num_rounds, \n",
    "                                          random_seed=random_state)\n",
    "    print('Avg Expected RMSE: %.3f' % np.sqrt(mse))\n",
    "    print('Avg Expected MSE: %.3f' % mse)\n",
    "    print('Avg Bias: %.3f' % bias)\n",
    "    print('Avg Variance: %.3f' % var)\n",
    "    pass\n",
    "\n",
    "\n",
    "def LOO_estimator_eval(dataset, target, estimator, params, ignore=None):\n",
    "    \"\"\"\n",
    "    Function used to evaluate estimators, based on Leave One Out process. It adds a \n",
    "    column 'Predicted' to the given dataset, and returns the metrics used to evaluate the \n",
    "    performances\n",
    "    \n",
    "    :param dataset: a pd.DataFrame with the target column\n",
    "    :param target: a str representing the target\n",
    "    :param estimator: instance of some estimator (i.e. XGBoostRegressor())\n",
    "    :param params: a dictionary containing the parameters for the estimator\n",
    "    :param ignor: a list of strings representing the feature to ignore\n",
    "    :returns: the pd.DataFrame\n",
    "    \"\"\"\n",
    "    # Splitter\n",
    "    splitter = LeaveOneOut()\n",
    "    \n",
    "    # Add predicted\n",
    "    dataset['Predicted'] = 0.0\n",
    "    \n",
    "    # Ignore\n",
    "    if ignore is not None:\n",
    "        ignore = ignore + ['Predicted']\n",
    "    else:\n",
    "        ignore = ['Predicted']\n",
    "    \n",
    "    # Split X, y\n",
    "    X, y = get_X_y(dataset, label=target, ignore=ignore)\n",
    "    \n",
    "    # For each fold and tuple train, test indices\n",
    "    for i, (train_index, test_index) in enumerate(splitter.split(X)):\n",
    "        # Re-Assign\n",
    "        model = estimator\n",
    "        \n",
    "        # Base model initialized with some parameters\n",
    "        if params is not None: \n",
    "            print(params)\n",
    "            model.set_params(params)\n",
    "        \n",
    "        # Get train part\n",
    "        train = dataset.loc[train_index.tolist()]\n",
    "        X_train, y_train = get_X_y(train, label=target, ignore=ignore)\n",
    "       \n",
    "        # Train \n",
    "        model.fit(X_train, y_train)\n",
    "\n",
    "        # Get test part \n",
    "        test = dataset.loc[test_index.tolist()]\n",
    "        X_test, y_test = get_X_y(test, label=target, ignore=ignore)\n",
    "        \n",
    "        # Add predict to dataset\n",
    "        y_pred = model.predict(X_test)\n",
    "        dataset.loc[test_index.tolist()[0]]['Predicted'] = y_pred[0]\n",
    "    return get_regression_metrics(dataset[[target]], dataset[['Predicted']])\n",
    "\n",
    "\n",
    "def GridSearch_CV_Tuning(dataset, target, estimator, params, ignore=None, n_repeats=4, n_splits=4, \n",
    "                random_state=33412):\n",
    "    \"\"\"\n",
    "    Function used to evaluate estimators, based on GridSearchCV process. It evaluates the\n",
    "    performances through a Repeated K Fold, and returns the results\n",
    "    \n",
    "    :param dataset: a pd.DataFrame with the target column\n",
    "    :param target: a str representing the target\n",
    "    :param estimator: instance of some estimator (i.e. XGBoostRegressor())\n",
    "    :param params: a dictionary containing the parameters for the estimator\n",
    "    :param ignore: a list of strings representing the feature to ignore\n",
    "    :param n_repeats: a integer\n",
    "    :param n_splits: a integer\n",
    "    :returns: the pd.DataFrame containing the results\n",
    "    \"\"\"\n",
    "    # Ignore\n",
    "    if ignore is not None:\n",
    "        ignore = ignore + ['Predicted']\n",
    "    else:\n",
    "        ignore = ['Predicted']\n",
    "        \n",
    "    # RepeatedKFold splitter\n",
    "    splitter = RepeatedKFold(n_repeats=n_repeats, n_splits=n_splits, random_state=random_state)\n",
    "    \n",
    "    # GridSearchCV\n",
    "    clf = GridSearchCV(estimator=estimator, cv=splitter,\n",
    "                       param_grid=params, return_train_score = True,\n",
    "                       scoring =['neg_mean_squared_error', 'neg_root_mean_squared_error', 'r2'],\n",
    "                       refit=False, n_jobs=-1, verbose=3)\n",
    "    # X, y\n",
    "    X, y = get_X_y(dataset, label=target, ignore=(ignore + ['Predicted']))\n",
    "    # Train, Test split\n",
    "    X_train, X_test, y_train, y_test = get_train_test(X, y)\n",
    "    # Fit\n",
    "    clf.fit(X_train, y_train)\n",
    "    \n",
    "    return pd.DataFrame(clf.cv_results_)\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset Overview\n",
    "\n",
    "The dataset we are going to consider are the following ones:\n",
    "* The modified dataset, in two different subset versions\n",
    "* The original dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = pd.read_csv(os.path.join(RESOURCES_DIR, IN_LABEL))\n",
    "df2 = pd.read_csv(os.path.join(RESOURCES_DIR, IN_LABEL2))\n",
    "df_orig = pd.read_csv(os.path.join(RESOURCES_DIR, ORIG_LABEL))\n",
    "\n",
    "df.drop(columns=['Unnamed: 0', 'Latitude', 'Longitude'], inplace=True)\n",
    "df_orig.drop(columns=['Unnamed: 0', 'Latitude', 'Longitude'], inplace=True)\n",
    "\n",
    "df = sort_alphabetically(df, 'Sale_Price')\n",
    "df2 = sort_alphabetically(df2, 'Sale_Price')\n",
    "df_orig = sort_alphabetically(df_orig, 'Sale_Price')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## Hyperparameters Tuning\n",
    "\n",
    "First of all, let us try to use a Grid Search CV to find the best parameters.\n",
    "\n",
    "### Automatic Parameters Tuning: Randomized Grid Search\n",
    "\n",
    "By defining the repetitions, the splits and the parameters, we repeatedly train and test the models. From each one of the model, we obtain three scores which we can use to check the best a parameters.\n",
    "\n",
    "But first we can choose a parsimonious range of hyperparameters to test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "xgb_params = {\n",
    "    'n_estimators': [5, 7, 9, 11],\n",
    "    'max_depth': [5, 7, 9, 11],\n",
    "    'max_leaves': [6, 8, 10, 12],\n",
    "    'learning_rate': [0.5, 0.75, 1],\n",
    "    'booster' : ['gbtree'],\n",
    "    'importance_type': ['weight', 'gain'],\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 16 folds for each of 384 candidates, totalling 6144 fits\n"
     ]
    }
   ],
   "source": [
    "results = GridSearch_CV_Tuning(dataset=df, target='Sale_Price', estimator=XGBRegressor(), params=xgb_params)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From this dataframe we want to obtain the 10 best models for each metric we used. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_fit_time</th>\n",
       "      <th>std_fit_time</th>\n",
       "      <th>mean_score_time</th>\n",
       "      <th>std_score_time</th>\n",
       "      <th>param_booster</th>\n",
       "      <th>param_importance_type</th>\n",
       "      <th>param_learning_rate</th>\n",
       "      <th>param_max_depth</th>\n",
       "      <th>param_max_leaves</th>\n",
       "      <th>param_n_estimators</th>\n",
       "      <th>...</th>\n",
       "      <th>split8_train_r2</th>\n",
       "      <th>split9_train_r2</th>\n",
       "      <th>split10_train_r2</th>\n",
       "      <th>split11_train_r2</th>\n",
       "      <th>split12_train_r2</th>\n",
       "      <th>split13_train_r2</th>\n",
       "      <th>split14_train_r2</th>\n",
       "      <th>split15_train_r2</th>\n",
       "      <th>mean_train_r2</th>\n",
       "      <th>std_train_r2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.203375</td>\n",
       "      <td>0.040059</td>\n",
       "      <td>0.006625</td>\n",
       "      <td>0.001053</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>0.939692</td>\n",
       "      <td>0.939879</td>\n",
       "      <td>0.937185</td>\n",
       "      <td>0.938392</td>\n",
       "      <td>0.938539</td>\n",
       "      <td>0.939914</td>\n",
       "      <td>0.938959</td>\n",
       "      <td>0.941348</td>\n",
       "      <td>0.939427</td>\n",
       "      <td>0.001812</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.028000</td>\n",
       "      <td>0.004924</td>\n",
       "      <td>0.006000</td>\n",
       "      <td>0.001500</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "      <td>...</td>\n",
       "      <td>0.960308</td>\n",
       "      <td>0.958681</td>\n",
       "      <td>0.958085</td>\n",
       "      <td>0.959365</td>\n",
       "      <td>0.957841</td>\n",
       "      <td>0.959687</td>\n",
       "      <td>0.961159</td>\n",
       "      <td>0.959330</td>\n",
       "      <td>0.959426</td>\n",
       "      <td>0.001174</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.041125</td>\n",
       "      <td>0.003919</td>\n",
       "      <td>0.007062</td>\n",
       "      <td>0.000428</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>9</td>\n",
       "      <td>...</td>\n",
       "      <td>0.967721</td>\n",
       "      <td>0.965205</td>\n",
       "      <td>0.965395</td>\n",
       "      <td>0.965554</td>\n",
       "      <td>0.966298</td>\n",
       "      <td>0.965548</td>\n",
       "      <td>0.967224</td>\n",
       "      <td>0.965820</td>\n",
       "      <td>0.966173</td>\n",
       "      <td>0.001291</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.049375</td>\n",
       "      <td>0.002369</td>\n",
       "      <td>0.006813</td>\n",
       "      <td>0.000527</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0.971625</td>\n",
       "      <td>0.969861</td>\n",
       "      <td>0.968147</td>\n",
       "      <td>0.969142</td>\n",
       "      <td>0.970206</td>\n",
       "      <td>0.970058</td>\n",
       "      <td>0.970820</td>\n",
       "      <td>0.970608</td>\n",
       "      <td>0.970135</td>\n",
       "      <td>0.001262</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.026750</td>\n",
       "      <td>0.001090</td>\n",
       "      <td>0.007375</td>\n",
       "      <td>0.001053</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>0.939692</td>\n",
       "      <td>0.939879</td>\n",
       "      <td>0.937185</td>\n",
       "      <td>0.938392</td>\n",
       "      <td>0.938539</td>\n",
       "      <td>0.939914</td>\n",
       "      <td>0.938959</td>\n",
       "      <td>0.941348</td>\n",
       "      <td>0.939427</td>\n",
       "      <td>0.001812</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>379</th>\n",
       "      <td>0.112750</td>\n",
       "      <td>0.003648</td>\n",
       "      <td>0.006707</td>\n",
       "      <td>0.000459</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0.999969</td>\n",
       "      <td>0.999801</td>\n",
       "      <td>0.999885</td>\n",
       "      <td>0.999830</td>\n",
       "      <td>0.999920</td>\n",
       "      <td>0.999821</td>\n",
       "      <td>0.999897</td>\n",
       "      <td>0.999960</td>\n",
       "      <td>0.999909</td>\n",
       "      <td>0.000050</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>380</th>\n",
       "      <td>0.052499</td>\n",
       "      <td>0.001732</td>\n",
       "      <td>0.006938</td>\n",
       "      <td>0.000555</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>0.996973</td>\n",
       "      <td>0.993098</td>\n",
       "      <td>0.994866</td>\n",
       "      <td>0.993211</td>\n",
       "      <td>0.995452</td>\n",
       "      <td>0.994067</td>\n",
       "      <td>0.994033</td>\n",
       "      <td>0.995728</td>\n",
       "      <td>0.995468</td>\n",
       "      <td>0.001226</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>381</th>\n",
       "      <td>0.071563</td>\n",
       "      <td>0.002669</td>\n",
       "      <td>0.007125</td>\n",
       "      <td>0.001654</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>7</td>\n",
       "      <td>...</td>\n",
       "      <td>0.999463</td>\n",
       "      <td>0.997293</td>\n",
       "      <td>0.998219</td>\n",
       "      <td>0.997493</td>\n",
       "      <td>0.998828</td>\n",
       "      <td>0.998224</td>\n",
       "      <td>0.998711</td>\n",
       "      <td>0.999527</td>\n",
       "      <td>0.998584</td>\n",
       "      <td>0.000623</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>382</th>\n",
       "      <td>0.091937</td>\n",
       "      <td>0.001952</td>\n",
       "      <td>0.006445</td>\n",
       "      <td>0.000490</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>9</td>\n",
       "      <td>...</td>\n",
       "      <td>0.999869</td>\n",
       "      <td>0.999187</td>\n",
       "      <td>0.999434</td>\n",
       "      <td>0.999270</td>\n",
       "      <td>0.999681</td>\n",
       "      <td>0.999340</td>\n",
       "      <td>0.999657</td>\n",
       "      <td>0.999906</td>\n",
       "      <td>0.999633</td>\n",
       "      <td>0.000210</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>383</th>\n",
       "      <td>0.106180</td>\n",
       "      <td>0.005389</td>\n",
       "      <td>0.005688</td>\n",
       "      <td>0.001157</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0.999969</td>\n",
       "      <td>0.999801</td>\n",
       "      <td>0.999885</td>\n",
       "      <td>0.999830</td>\n",
       "      <td>0.999920</td>\n",
       "      <td>0.999821</td>\n",
       "      <td>0.999897</td>\n",
       "      <td>0.999960</td>\n",
       "      <td>0.999909</td>\n",
       "      <td>0.000050</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>384 rows × 122 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     mean_fit_time  std_fit_time  mean_score_time  std_score_time  \\\n",
       "0         0.203375      0.040059         0.006625        0.001053   \n",
       "1         0.028000      0.004924         0.006000        0.001500   \n",
       "2         0.041125      0.003919         0.007062        0.000428   \n",
       "3         0.049375      0.002369         0.006813        0.000527   \n",
       "4         0.026750      0.001090         0.007375        0.001053   \n",
       "..             ...           ...              ...             ...   \n",
       "379       0.112750      0.003648         0.006707        0.000459   \n",
       "380       0.052499      0.001732         0.006938        0.000555   \n",
       "381       0.071563      0.002669         0.007125        0.001654   \n",
       "382       0.091937      0.001952         0.006445        0.000490   \n",
       "383       0.106180      0.005389         0.005688        0.001157   \n",
       "\n",
       "    param_booster param_importance_type param_learning_rate param_max_depth  \\\n",
       "0          gbtree                weight                 0.5               5   \n",
       "1          gbtree                weight                 0.5               5   \n",
       "2          gbtree                weight                 0.5               5   \n",
       "3          gbtree                weight                 0.5               5   \n",
       "4          gbtree                weight                 0.5               5   \n",
       "..            ...                   ...                 ...             ...   \n",
       "379        gbtree                  gain                   1              11   \n",
       "380        gbtree                  gain                   1              11   \n",
       "381        gbtree                  gain                   1              11   \n",
       "382        gbtree                  gain                   1              11   \n",
       "383        gbtree                  gain                   1              11   \n",
       "\n",
       "    param_max_leaves param_n_estimators  ... split8_train_r2  split9_train_r2  \\\n",
       "0                  6                  5  ...        0.939692         0.939879   \n",
       "1                  6                  7  ...        0.960308         0.958681   \n",
       "2                  6                  9  ...        0.967721         0.965205   \n",
       "3                  6                 11  ...        0.971625         0.969861   \n",
       "4                  8                  5  ...        0.939692         0.939879   \n",
       "..               ...                ...  ...             ...              ...   \n",
       "379               10                 11  ...        0.999969         0.999801   \n",
       "380               12                  5  ...        0.996973         0.993098   \n",
       "381               12                  7  ...        0.999463         0.997293   \n",
       "382               12                  9  ...        0.999869         0.999187   \n",
       "383               12                 11  ...        0.999969         0.999801   \n",
       "\n",
       "     split10_train_r2  split11_train_r2  split12_train_r2  split13_train_r2  \\\n",
       "0            0.937185          0.938392          0.938539          0.939914   \n",
       "1            0.958085          0.959365          0.957841          0.959687   \n",
       "2            0.965395          0.965554          0.966298          0.965548   \n",
       "3            0.968147          0.969142          0.970206          0.970058   \n",
       "4            0.937185          0.938392          0.938539          0.939914   \n",
       "..                ...               ...               ...               ...   \n",
       "379          0.999885          0.999830          0.999920          0.999821   \n",
       "380          0.994866          0.993211          0.995452          0.994067   \n",
       "381          0.998219          0.997493          0.998828          0.998224   \n",
       "382          0.999434          0.999270          0.999681          0.999340   \n",
       "383          0.999885          0.999830          0.999920          0.999821   \n",
       "\n",
       "     split14_train_r2  split15_train_r2  mean_train_r2  std_train_r2  \n",
       "0            0.938959          0.941348       0.939427      0.001812  \n",
       "1            0.961159          0.959330       0.959426      0.001174  \n",
       "2            0.967224          0.965820       0.966173      0.001291  \n",
       "3            0.970820          0.970608       0.970135      0.001262  \n",
       "4            0.938959          0.941348       0.939427      0.001812  \n",
       "..                ...               ...            ...           ...  \n",
       "379          0.999897          0.999960       0.999909      0.000050  \n",
       "380          0.994033          0.995728       0.995468      0.001226  \n",
       "381          0.998711          0.999527       0.998584      0.000623  \n",
       "382          0.999657          0.999906       0.999633      0.000210  \n",
       "383          0.999897          0.999960       0.999909      0.000050  \n",
       "\n",
       "[384 rows x 122 columns]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "best_r2 = list(results[['rank_test_r2','mean_train_r2', 'mean_test_r2']][results['rank_test_r2']==1].index)\n",
    "best_mse = list(results[['rank_test_neg_mean_squared_error','mean_train_neg_mean_squared_error', 'mean_test_neg_mean_squared_error']][results['rank_test_neg_mean_squared_error']==1].index)\n",
    "best_rmse = list(results[['rank_test_neg_root_mean_squared_error','mean_train_neg_root_mean_squared_error', 'mean_test_neg_root_mean_squared_error']][results['rank_test_neg_root_mean_squared_error']==1].index)\n",
    "\n",
    "best = list(set(best_r2) | set(best_mse) | set(best_rmse))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "best_df = results[['mean_fit_time', 'mean_test_neg_mean_squared_error', 'mean_test_neg_root_mean_squared_error', 'mean_test_r2', 'params',]].loc[best].sort_values(by=['mean_fit_time'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_fit_time</th>\n",
       "      <th>mean_test_neg_mean_squared_error</th>\n",
       "      <th>mean_test_neg_root_mean_squared_error</th>\n",
       "      <th>mean_test_r2</th>\n",
       "      <th>params</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>199</th>\n",
       "      <td>0.047000</td>\n",
       "      <td>-5.608038e+08</td>\n",
       "      <td>-23650.231297</td>\n",
       "      <td>0.89926</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'gain...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>195</th>\n",
       "      <td>0.047687</td>\n",
       "      <td>-5.608038e+08</td>\n",
       "      <td>-23650.231297</td>\n",
       "      <td>0.89926</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'gain...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>203</th>\n",
       "      <td>0.047875</td>\n",
       "      <td>-5.608038e+08</td>\n",
       "      <td>-23650.231297</td>\n",
       "      <td>0.89926</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'gain...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.047999</td>\n",
       "      <td>-5.608038e+08</td>\n",
       "      <td>-23650.231297</td>\n",
       "      <td>0.89926</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'weig...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0.048125</td>\n",
       "      <td>-5.608038e+08</td>\n",
       "      <td>-23650.231297</td>\n",
       "      <td>0.89926</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'weig...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.048312</td>\n",
       "      <td>-5.608038e+08</td>\n",
       "      <td>-23650.231297</td>\n",
       "      <td>0.89926</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'weig...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>207</th>\n",
       "      <td>0.048375</td>\n",
       "      <td>-5.608038e+08</td>\n",
       "      <td>-23650.231297</td>\n",
       "      <td>0.89926</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'gain...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.049375</td>\n",
       "      <td>-5.608038e+08</td>\n",
       "      <td>-23650.231297</td>\n",
       "      <td>0.89926</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'weig...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     mean_fit_time  mean_test_neg_mean_squared_error  \\\n",
       "199       0.047000                     -5.608038e+08   \n",
       "195       0.047687                     -5.608038e+08   \n",
       "203       0.047875                     -5.608038e+08   \n",
       "7         0.047999                     -5.608038e+08   \n",
       "15        0.048125                     -5.608038e+08   \n",
       "11        0.048312                     -5.608038e+08   \n",
       "207       0.048375                     -5.608038e+08   \n",
       "3         0.049375                     -5.608038e+08   \n",
       "\n",
       "     mean_test_neg_root_mean_squared_error  mean_test_r2  \\\n",
       "199                          -23650.231297       0.89926   \n",
       "195                          -23650.231297       0.89926   \n",
       "203                          -23650.231297       0.89926   \n",
       "7                            -23650.231297       0.89926   \n",
       "15                           -23650.231297       0.89926   \n",
       "11                           -23650.231297       0.89926   \n",
       "207                          -23650.231297       0.89926   \n",
       "3                            -23650.231297       0.89926   \n",
       "\n",
       "                                                params  \n",
       "199  {'booster': 'gbtree', 'importance_type': 'gain...  \n",
       "195  {'booster': 'gbtree', 'importance_type': 'gain...  \n",
       "203  {'booster': 'gbtree', 'importance_type': 'gain...  \n",
       "7    {'booster': 'gbtree', 'importance_type': 'weig...  \n",
       "15   {'booster': 'gbtree', 'importance_type': 'weig...  \n",
       "11   {'booster': 'gbtree', 'importance_type': 'weig...  \n",
       "207  {'booster': 'gbtree', 'importance_type': 'gain...  \n",
       "3    {'booster': 'gbtree', 'importance_type': 'weig...  "
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "best_df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>booster</th>\n",
       "      <th>importance_type</th>\n",
       "      <th>learning_rate</th>\n",
       "      <th>max_depth</th>\n",
       "      <th>max_leaves</th>\n",
       "      <th>n_estimators</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>8</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>8</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>12</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>12</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  booster importance_type  learning_rate  max_depth  max_leaves  n_estimators\n",
       "0  gbtree            gain            0.5          5           8            11\n",
       "1  gbtree            gain            0.5          5           6            11\n",
       "2  gbtree            gain            0.5          5          10            11\n",
       "3  gbtree          weight            0.5          5           8            11\n",
       "4  gbtree          weight            0.5          5          12            11\n",
       "5  gbtree          weight            0.5          5          10            11\n",
       "6  gbtree            gain            0.5          5          12            11\n",
       "7  gbtree          weight            0.5          5           6            11"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(list(best_df.params))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Just to make sure this is the right way I want to re iter this on the original dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 16 folds for each of 384 candidates, totalling 6144 fits\n"
     ]
    }
   ],
   "source": [
    "results_orig = GridSearch_CV_Tuning(dataset=df_orig, target='Sale_Price', estimator=XGBRegressor(), params=xgb_params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_fit_time</th>\n",
       "      <th>std_fit_time</th>\n",
       "      <th>mean_score_time</th>\n",
       "      <th>std_score_time</th>\n",
       "      <th>param_booster</th>\n",
       "      <th>param_importance_type</th>\n",
       "      <th>param_learning_rate</th>\n",
       "      <th>param_max_depth</th>\n",
       "      <th>param_max_leaves</th>\n",
       "      <th>param_n_estimators</th>\n",
       "      <th>...</th>\n",
       "      <th>split8_train_r2</th>\n",
       "      <th>split9_train_r2</th>\n",
       "      <th>split10_train_r2</th>\n",
       "      <th>split11_train_r2</th>\n",
       "      <th>split12_train_r2</th>\n",
       "      <th>split13_train_r2</th>\n",
       "      <th>split14_train_r2</th>\n",
       "      <th>split15_train_r2</th>\n",
       "      <th>mean_train_r2</th>\n",
       "      <th>std_train_r2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.031613</td>\n",
       "      <td>0.000690</td>\n",
       "      <td>0.006688</td>\n",
       "      <td>0.000682</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>0.935909</td>\n",
       "      <td>0.938823</td>\n",
       "      <td>0.938464</td>\n",
       "      <td>0.938249</td>\n",
       "      <td>0.938388</td>\n",
       "      <td>0.938563</td>\n",
       "      <td>0.939364</td>\n",
       "      <td>0.935829</td>\n",
       "      <td>0.938349</td>\n",
       "      <td>0.002245</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.041187</td>\n",
       "      <td>0.003321</td>\n",
       "      <td>0.007125</td>\n",
       "      <td>0.000857</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "      <td>...</td>\n",
       "      <td>0.957856</td>\n",
       "      <td>0.958390</td>\n",
       "      <td>0.958449</td>\n",
       "      <td>0.958015</td>\n",
       "      <td>0.959361</td>\n",
       "      <td>0.957373</td>\n",
       "      <td>0.959001</td>\n",
       "      <td>0.956316</td>\n",
       "      <td>0.958192</td>\n",
       "      <td>0.001818</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.050437</td>\n",
       "      <td>0.003742</td>\n",
       "      <td>0.007125</td>\n",
       "      <td>0.002088</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>9</td>\n",
       "      <td>...</td>\n",
       "      <td>0.964789</td>\n",
       "      <td>0.964925</td>\n",
       "      <td>0.964157</td>\n",
       "      <td>0.964777</td>\n",
       "      <td>0.965492</td>\n",
       "      <td>0.962529</td>\n",
       "      <td>0.966543</td>\n",
       "      <td>0.962486</td>\n",
       "      <td>0.964714</td>\n",
       "      <td>0.001865</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.058125</td>\n",
       "      <td>0.002395</td>\n",
       "      <td>0.006563</td>\n",
       "      <td>0.000864</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0.969684</td>\n",
       "      <td>0.968406</td>\n",
       "      <td>0.969254</td>\n",
       "      <td>0.968750</td>\n",
       "      <td>0.969474</td>\n",
       "      <td>0.965576</td>\n",
       "      <td>0.971125</td>\n",
       "      <td>0.967228</td>\n",
       "      <td>0.968847</td>\n",
       "      <td>0.001954</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.031936</td>\n",
       "      <td>0.001088</td>\n",
       "      <td>0.006438</td>\n",
       "      <td>0.000788</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>0.935909</td>\n",
       "      <td>0.938823</td>\n",
       "      <td>0.938464</td>\n",
       "      <td>0.938249</td>\n",
       "      <td>0.938388</td>\n",
       "      <td>0.938563</td>\n",
       "      <td>0.939364</td>\n",
       "      <td>0.935829</td>\n",
       "      <td>0.938349</td>\n",
       "      <td>0.002245</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>379</th>\n",
       "      <td>0.135250</td>\n",
       "      <td>0.003579</td>\n",
       "      <td>0.006625</td>\n",
       "      <td>0.000484</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0.999856</td>\n",
       "      <td>0.999850</td>\n",
       "      <td>0.999924</td>\n",
       "      <td>0.999879</td>\n",
       "      <td>0.999720</td>\n",
       "      <td>0.999894</td>\n",
       "      <td>0.999860</td>\n",
       "      <td>0.999878</td>\n",
       "      <td>0.999886</td>\n",
       "      <td>0.000052</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>380</th>\n",
       "      <td>0.061438</td>\n",
       "      <td>0.001322</td>\n",
       "      <td>0.006500</td>\n",
       "      <td>0.000612</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>0.991559</td>\n",
       "      <td>0.991901</td>\n",
       "      <td>0.993149</td>\n",
       "      <td>0.993012</td>\n",
       "      <td>0.994604</td>\n",
       "      <td>0.994297</td>\n",
       "      <td>0.994597</td>\n",
       "      <td>0.993391</td>\n",
       "      <td>0.993604</td>\n",
       "      <td>0.001169</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>381</th>\n",
       "      <td>0.087124</td>\n",
       "      <td>0.002803</td>\n",
       "      <td>0.006815</td>\n",
       "      <td>0.000632</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>7</td>\n",
       "      <td>...</td>\n",
       "      <td>0.997510</td>\n",
       "      <td>0.997114</td>\n",
       "      <td>0.998506</td>\n",
       "      <td>0.998340</td>\n",
       "      <td>0.997651</td>\n",
       "      <td>0.998103</td>\n",
       "      <td>0.998580</td>\n",
       "      <td>0.997845</td>\n",
       "      <td>0.998100</td>\n",
       "      <td>0.000446</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>382</th>\n",
       "      <td>0.111750</td>\n",
       "      <td>0.003717</td>\n",
       "      <td>0.006750</td>\n",
       "      <td>0.000661</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>9</td>\n",
       "      <td>...</td>\n",
       "      <td>0.999495</td>\n",
       "      <td>0.999301</td>\n",
       "      <td>0.999580</td>\n",
       "      <td>0.999632</td>\n",
       "      <td>0.999307</td>\n",
       "      <td>0.999561</td>\n",
       "      <td>0.999461</td>\n",
       "      <td>0.999338</td>\n",
       "      <td>0.999560</td>\n",
       "      <td>0.000146</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>383</th>\n",
       "      <td>0.131062</td>\n",
       "      <td>0.009086</td>\n",
       "      <td>0.005812</td>\n",
       "      <td>0.000950</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0.999856</td>\n",
       "      <td>0.999850</td>\n",
       "      <td>0.999924</td>\n",
       "      <td>0.999879</td>\n",
       "      <td>0.999720</td>\n",
       "      <td>0.999894</td>\n",
       "      <td>0.999860</td>\n",
       "      <td>0.999878</td>\n",
       "      <td>0.999886</td>\n",
       "      <td>0.000052</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>384 rows × 122 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     mean_fit_time  std_fit_time  mean_score_time  std_score_time  \\\n",
       "0         0.031613      0.000690         0.006688        0.000682   \n",
       "1         0.041187      0.003321         0.007125        0.000857   \n",
       "2         0.050437      0.003742         0.007125        0.002088   \n",
       "3         0.058125      0.002395         0.006563        0.000864   \n",
       "4         0.031936      0.001088         0.006438        0.000788   \n",
       "..             ...           ...              ...             ...   \n",
       "379       0.135250      0.003579         0.006625        0.000484   \n",
       "380       0.061438      0.001322         0.006500        0.000612   \n",
       "381       0.087124      0.002803         0.006815        0.000632   \n",
       "382       0.111750      0.003717         0.006750        0.000661   \n",
       "383       0.131062      0.009086         0.005812        0.000950   \n",
       "\n",
       "    param_booster param_importance_type param_learning_rate param_max_depth  \\\n",
       "0          gbtree                weight                 0.5               5   \n",
       "1          gbtree                weight                 0.5               5   \n",
       "2          gbtree                weight                 0.5               5   \n",
       "3          gbtree                weight                 0.5               5   \n",
       "4          gbtree                weight                 0.5               5   \n",
       "..            ...                   ...                 ...             ...   \n",
       "379        gbtree                  gain                   1              11   \n",
       "380        gbtree                  gain                   1              11   \n",
       "381        gbtree                  gain                   1              11   \n",
       "382        gbtree                  gain                   1              11   \n",
       "383        gbtree                  gain                   1              11   \n",
       "\n",
       "    param_max_leaves param_n_estimators  ... split8_train_r2  split9_train_r2  \\\n",
       "0                  6                  5  ...        0.935909         0.938823   \n",
       "1                  6                  7  ...        0.957856         0.958390   \n",
       "2                  6                  9  ...        0.964789         0.964925   \n",
       "3                  6                 11  ...        0.969684         0.968406   \n",
       "4                  8                  5  ...        0.935909         0.938823   \n",
       "..               ...                ...  ...             ...              ...   \n",
       "379               10                 11  ...        0.999856         0.999850   \n",
       "380               12                  5  ...        0.991559         0.991901   \n",
       "381               12                  7  ...        0.997510         0.997114   \n",
       "382               12                  9  ...        0.999495         0.999301   \n",
       "383               12                 11  ...        0.999856         0.999850   \n",
       "\n",
       "     split10_train_r2  split11_train_r2  split12_train_r2  split13_train_r2  \\\n",
       "0            0.938464          0.938249          0.938388          0.938563   \n",
       "1            0.958449          0.958015          0.959361          0.957373   \n",
       "2            0.964157          0.964777          0.965492          0.962529   \n",
       "3            0.969254          0.968750          0.969474          0.965576   \n",
       "4            0.938464          0.938249          0.938388          0.938563   \n",
       "..                ...               ...               ...               ...   \n",
       "379          0.999924          0.999879          0.999720          0.999894   \n",
       "380          0.993149          0.993012          0.994604          0.994297   \n",
       "381          0.998506          0.998340          0.997651          0.998103   \n",
       "382          0.999580          0.999632          0.999307          0.999561   \n",
       "383          0.999924          0.999879          0.999720          0.999894   \n",
       "\n",
       "     split14_train_r2  split15_train_r2  mean_train_r2  std_train_r2  \n",
       "0            0.939364          0.935829       0.938349      0.002245  \n",
       "1            0.959001          0.956316       0.958192      0.001818  \n",
       "2            0.966543          0.962486       0.964714      0.001865  \n",
       "3            0.971125          0.967228       0.968847      0.001954  \n",
       "4            0.939364          0.935829       0.938349      0.002245  \n",
       "..                ...               ...            ...           ...  \n",
       "379          0.999860          0.999878       0.999886      0.000052  \n",
       "380          0.994597          0.993391       0.993604      0.001169  \n",
       "381          0.998580          0.997845       0.998100      0.000446  \n",
       "382          0.999461          0.999338       0.999560      0.000146  \n",
       "383          0.999860          0.999878       0.999886      0.000052  \n",
       "\n",
       "[384 rows x 122 columns]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results_orig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "best_r22 = list(results_orig[['rank_test_r2','mean_train_r2', 'mean_test_r2']][results_orig['rank_test_r2']==1].index)\n",
    "best_mse2 = list(results_orig[['rank_test_neg_mean_squared_error','mean_train_neg_mean_squared_error', 'mean_test_neg_mean_squared_error']][results_orig['rank_test_neg_mean_squared_error']==1].index)\n",
    "best_rmse2 = list(results_orig[['rank_test_neg_root_mean_squared_error','mean_train_neg_root_mean_squared_error', 'mean_test_neg_root_mean_squared_error']][results_orig['rank_test_neg_root_mean_squared_error']==1].index)\n",
    "\n",
    "best2 = list(set(best_r22) | set(best_mse2) | set(best_rmse2))\n",
    "\n",
    "best_df2 = results_orig[['mean_fit_time', 'mean_test_neg_mean_squared_error', 'mean_test_neg_root_mean_squared_error', 'mean_test_r2', 'params',]].loc[best2].sort_values(by=['mean_fit_time'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_fit_time</th>\n",
       "      <th>mean_test_neg_mean_squared_error</th>\n",
       "      <th>mean_test_neg_root_mean_squared_error</th>\n",
       "      <th>mean_test_r2</th>\n",
       "      <th>params</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.058125</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'weig...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>199</th>\n",
       "      <td>0.058188</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'gain...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>195</th>\n",
       "      <td>0.058438</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'gain...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>203</th>\n",
       "      <td>0.058937</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'gain...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.059875</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'weig...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.060063</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'weig...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>207</th>\n",
       "      <td>0.060687</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'gain...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0.061938</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'weig...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     mean_fit_time  mean_test_neg_mean_squared_error  \\\n",
       "3         0.058125                     -8.385927e+08   \n",
       "199       0.058188                     -8.385927e+08   \n",
       "195       0.058438                     -8.385927e+08   \n",
       "203       0.058937                     -8.385927e+08   \n",
       "7         0.059875                     -8.385927e+08   \n",
       "11        0.060063                     -8.385927e+08   \n",
       "207       0.060687                     -8.385927e+08   \n",
       "15        0.061938                     -8.385927e+08   \n",
       "\n",
       "     mean_test_neg_root_mean_squared_error  mean_test_r2  \\\n",
       "3                            -28775.492507      0.863008   \n",
       "199                          -28775.492507      0.863008   \n",
       "195                          -28775.492507      0.863008   \n",
       "203                          -28775.492507      0.863008   \n",
       "7                            -28775.492507      0.863008   \n",
       "11                           -28775.492507      0.863008   \n",
       "207                          -28775.492507      0.863008   \n",
       "15                           -28775.492507      0.863008   \n",
       "\n",
       "                                                params  \n",
       "3    {'booster': 'gbtree', 'importance_type': 'weig...  \n",
       "199  {'booster': 'gbtree', 'importance_type': 'gain...  \n",
       "195  {'booster': 'gbtree', 'importance_type': 'gain...  \n",
       "203  {'booster': 'gbtree', 'importance_type': 'gain...  \n",
       "7    {'booster': 'gbtree', 'importance_type': 'weig...  \n",
       "11   {'booster': 'gbtree', 'importance_type': 'weig...  \n",
       "207  {'booster': 'gbtree', 'importance_type': 'gain...  \n",
       "15   {'booster': 'gbtree', 'importance_type': 'weig...  "
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "best_df2 "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>booster</th>\n",
       "      <th>importance_type</th>\n",
       "      <th>learning_rate</th>\n",
       "      <th>max_depth</th>\n",
       "      <th>max_leaves</th>\n",
       "      <th>n_estimators</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>8</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>8</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>12</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>12</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  booster importance_type  learning_rate  max_depth  max_leaves  n_estimators\n",
       "0  gbtree          weight            0.5          5           6            11\n",
       "1  gbtree            gain            0.5          5           8            11\n",
       "2  gbtree            gain            0.5          5           6            11\n",
       "3  gbtree            gain            0.5          5          10            11\n",
       "4  gbtree          weight            0.5          5           8            11\n",
       "5  gbtree          weight            0.5          5          10            11\n",
       "6  gbtree            gain            0.5          5          12            11\n",
       "7  gbtree          weight            0.5          5          12            11"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(list(best_df2.params))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Fitting 16 folds for each of 384 candidates, totalling 6144 fits\n"
     ]
    }
   ],
   "source": [
    "results2 = GridSearch_CV_Tuning(dataset=df2, target='Sale_Price', estimator=XGBRegressor(), params=xgb_params)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_fit_time</th>\n",
       "      <th>std_fit_time</th>\n",
       "      <th>mean_score_time</th>\n",
       "      <th>std_score_time</th>\n",
       "      <th>param_booster</th>\n",
       "      <th>param_importance_type</th>\n",
       "      <th>param_learning_rate</th>\n",
       "      <th>param_max_depth</th>\n",
       "      <th>param_max_leaves</th>\n",
       "      <th>param_n_estimators</th>\n",
       "      <th>...</th>\n",
       "      <th>split8_train_r2</th>\n",
       "      <th>split9_train_r2</th>\n",
       "      <th>split10_train_r2</th>\n",
       "      <th>split11_train_r2</th>\n",
       "      <th>split12_train_r2</th>\n",
       "      <th>split13_train_r2</th>\n",
       "      <th>split14_train_r2</th>\n",
       "      <th>split15_train_r2</th>\n",
       "      <th>mean_train_r2</th>\n",
       "      <th>std_train_r2</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.032362</td>\n",
       "      <td>0.001469</td>\n",
       "      <td>0.006250</td>\n",
       "      <td>0.000433</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>0.944568</td>\n",
       "      <td>0.941767</td>\n",
       "      <td>0.943696</td>\n",
       "      <td>0.941051</td>\n",
       "      <td>0.941399</td>\n",
       "      <td>0.945594</td>\n",
       "      <td>0.944625</td>\n",
       "      <td>0.942266</td>\n",
       "      <td>0.943836</td>\n",
       "      <td>0.001773</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.041687</td>\n",
       "      <td>0.001488</td>\n",
       "      <td>0.006313</td>\n",
       "      <td>0.001211</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "      <td>...</td>\n",
       "      <td>0.964084</td>\n",
       "      <td>0.961504</td>\n",
       "      <td>0.964316</td>\n",
       "      <td>0.961951</td>\n",
       "      <td>0.962806</td>\n",
       "      <td>0.965201</td>\n",
       "      <td>0.963793</td>\n",
       "      <td>0.961944</td>\n",
       "      <td>0.963492</td>\n",
       "      <td>0.001656</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.052936</td>\n",
       "      <td>0.002512</td>\n",
       "      <td>0.006313</td>\n",
       "      <td>0.000463</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>9</td>\n",
       "      <td>...</td>\n",
       "      <td>0.970178</td>\n",
       "      <td>0.968268</td>\n",
       "      <td>0.973004</td>\n",
       "      <td>0.967771</td>\n",
       "      <td>0.969868</td>\n",
       "      <td>0.972482</td>\n",
       "      <td>0.969589</td>\n",
       "      <td>0.967652</td>\n",
       "      <td>0.970085</td>\n",
       "      <td>0.001639</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.063751</td>\n",
       "      <td>0.003113</td>\n",
       "      <td>0.007062</td>\n",
       "      <td>0.000899</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0.972959</td>\n",
       "      <td>0.972534</td>\n",
       "      <td>0.977260</td>\n",
       "      <td>0.971381</td>\n",
       "      <td>0.973782</td>\n",
       "      <td>0.976215</td>\n",
       "      <td>0.974092</td>\n",
       "      <td>0.972656</td>\n",
       "      <td>0.974491</td>\n",
       "      <td>0.001654</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.034748</td>\n",
       "      <td>0.002221</td>\n",
       "      <td>0.006626</td>\n",
       "      <td>0.000484</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>8</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>0.944568</td>\n",
       "      <td>0.941767</td>\n",
       "      <td>0.943696</td>\n",
       "      <td>0.941051</td>\n",
       "      <td>0.941399</td>\n",
       "      <td>0.945594</td>\n",
       "      <td>0.944625</td>\n",
       "      <td>0.942266</td>\n",
       "      <td>0.943836</td>\n",
       "      <td>0.001773</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>379</th>\n",
       "      <td>0.143123</td>\n",
       "      <td>0.003119</td>\n",
       "      <td>0.007068</td>\n",
       "      <td>0.000961</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0.999868</td>\n",
       "      <td>0.999923</td>\n",
       "      <td>0.999906</td>\n",
       "      <td>0.999869</td>\n",
       "      <td>0.999894</td>\n",
       "      <td>0.999907</td>\n",
       "      <td>0.999888</td>\n",
       "      <td>0.999849</td>\n",
       "      <td>0.999898</td>\n",
       "      <td>0.000037</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>380</th>\n",
       "      <td>0.066813</td>\n",
       "      <td>0.002215</td>\n",
       "      <td>0.006875</td>\n",
       "      <td>0.000484</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>5</td>\n",
       "      <td>...</td>\n",
       "      <td>0.994334</td>\n",
       "      <td>0.995166</td>\n",
       "      <td>0.996083</td>\n",
       "      <td>0.995620</td>\n",
       "      <td>0.996101</td>\n",
       "      <td>0.996302</td>\n",
       "      <td>0.994962</td>\n",
       "      <td>0.995141</td>\n",
       "      <td>0.995194</td>\n",
       "      <td>0.001346</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>381</th>\n",
       "      <td>0.090745</td>\n",
       "      <td>0.001554</td>\n",
       "      <td>0.006562</td>\n",
       "      <td>0.000609</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>7</td>\n",
       "      <td>...</td>\n",
       "      <td>0.998712</td>\n",
       "      <td>0.998670</td>\n",
       "      <td>0.998691</td>\n",
       "      <td>0.998546</td>\n",
       "      <td>0.999242</td>\n",
       "      <td>0.998843</td>\n",
       "      <td>0.998057</td>\n",
       "      <td>0.998055</td>\n",
       "      <td>0.998611</td>\n",
       "      <td>0.000460</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>382</th>\n",
       "      <td>0.116499</td>\n",
       "      <td>0.002500</td>\n",
       "      <td>0.007000</td>\n",
       "      <td>0.000354</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>9</td>\n",
       "      <td>...</td>\n",
       "      <td>0.999498</td>\n",
       "      <td>0.999467</td>\n",
       "      <td>0.999617</td>\n",
       "      <td>0.999442</td>\n",
       "      <td>0.999754</td>\n",
       "      <td>0.999670</td>\n",
       "      <td>0.999379</td>\n",
       "      <td>0.999605</td>\n",
       "      <td>0.999614</td>\n",
       "      <td>0.000124</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>383</th>\n",
       "      <td>0.133884</td>\n",
       "      <td>0.008094</td>\n",
       "      <td>0.005559</td>\n",
       "      <td>0.000929</td>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>1</td>\n",
       "      <td>11</td>\n",
       "      <td>12</td>\n",
       "      <td>11</td>\n",
       "      <td>...</td>\n",
       "      <td>0.999868</td>\n",
       "      <td>0.999923</td>\n",
       "      <td>0.999906</td>\n",
       "      <td>0.999869</td>\n",
       "      <td>0.999894</td>\n",
       "      <td>0.999907</td>\n",
       "      <td>0.999888</td>\n",
       "      <td>0.999849</td>\n",
       "      <td>0.999898</td>\n",
       "      <td>0.000037</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>384 rows × 122 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     mean_fit_time  std_fit_time  mean_score_time  std_score_time  \\\n",
       "0         0.032362      0.001469         0.006250        0.000433   \n",
       "1         0.041687      0.001488         0.006313        0.001211   \n",
       "2         0.052936      0.002512         0.006313        0.000463   \n",
       "3         0.063751      0.003113         0.007062        0.000899   \n",
       "4         0.034748      0.002221         0.006626        0.000484   \n",
       "..             ...           ...              ...             ...   \n",
       "379       0.143123      0.003119         0.007068        0.000961   \n",
       "380       0.066813      0.002215         0.006875        0.000484   \n",
       "381       0.090745      0.001554         0.006562        0.000609   \n",
       "382       0.116499      0.002500         0.007000        0.000354   \n",
       "383       0.133884      0.008094         0.005559        0.000929   \n",
       "\n",
       "    param_booster param_importance_type param_learning_rate param_max_depth  \\\n",
       "0          gbtree                weight                 0.5               5   \n",
       "1          gbtree                weight                 0.5               5   \n",
       "2          gbtree                weight                 0.5               5   \n",
       "3          gbtree                weight                 0.5               5   \n",
       "4          gbtree                weight                 0.5               5   \n",
       "..            ...                   ...                 ...             ...   \n",
       "379        gbtree                  gain                   1              11   \n",
       "380        gbtree                  gain                   1              11   \n",
       "381        gbtree                  gain                   1              11   \n",
       "382        gbtree                  gain                   1              11   \n",
       "383        gbtree                  gain                   1              11   \n",
       "\n",
       "    param_max_leaves param_n_estimators  ... split8_train_r2  split9_train_r2  \\\n",
       "0                  6                  5  ...        0.944568         0.941767   \n",
       "1                  6                  7  ...        0.964084         0.961504   \n",
       "2                  6                  9  ...        0.970178         0.968268   \n",
       "3                  6                 11  ...        0.972959         0.972534   \n",
       "4                  8                  5  ...        0.944568         0.941767   \n",
       "..               ...                ...  ...             ...              ...   \n",
       "379               10                 11  ...        0.999868         0.999923   \n",
       "380               12                  5  ...        0.994334         0.995166   \n",
       "381               12                  7  ...        0.998712         0.998670   \n",
       "382               12                  9  ...        0.999498         0.999467   \n",
       "383               12                 11  ...        0.999868         0.999923   \n",
       "\n",
       "     split10_train_r2  split11_train_r2  split12_train_r2  split13_train_r2  \\\n",
       "0            0.943696          0.941051          0.941399          0.945594   \n",
       "1            0.964316          0.961951          0.962806          0.965201   \n",
       "2            0.973004          0.967771          0.969868          0.972482   \n",
       "3            0.977260          0.971381          0.973782          0.976215   \n",
       "4            0.943696          0.941051          0.941399          0.945594   \n",
       "..                ...               ...               ...               ...   \n",
       "379          0.999906          0.999869          0.999894          0.999907   \n",
       "380          0.996083          0.995620          0.996101          0.996302   \n",
       "381          0.998691          0.998546          0.999242          0.998843   \n",
       "382          0.999617          0.999442          0.999754          0.999670   \n",
       "383          0.999906          0.999869          0.999894          0.999907   \n",
       "\n",
       "     split14_train_r2  split15_train_r2  mean_train_r2  std_train_r2  \n",
       "0            0.944625          0.942266       0.943836      0.001773  \n",
       "1            0.963793          0.961944       0.963492      0.001656  \n",
       "2            0.969589          0.967652       0.970085      0.001639  \n",
       "3            0.974092          0.972656       0.974491      0.001654  \n",
       "4            0.944625          0.942266       0.943836      0.001773  \n",
       "..                ...               ...            ...           ...  \n",
       "379          0.999888          0.999849       0.999898      0.000037  \n",
       "380          0.994962          0.995141       0.995194      0.001346  \n",
       "381          0.998057          0.998055       0.998611      0.000460  \n",
       "382          0.999379          0.999605       0.999614      0.000124  \n",
       "383          0.999888          0.999849       0.999898      0.000037  \n",
       "\n",
       "[384 rows x 122 columns]"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "best_r3 = list(results2[['rank_test_r2','mean_train_r2', 'mean_test_r2']][results2['rank_test_r2']==1].index)\n",
    "best_mse3 = list(results2[['rank_test_neg_mean_squared_error','mean_train_neg_mean_squared_error', 'mean_test_neg_mean_squared_error']][results2['rank_test_neg_mean_squared_error']==1].index)\n",
    "best_rmse3 = list(results2[['rank_test_neg_root_mean_squared_error','mean_train_neg_root_mean_squared_error', 'mean_test_neg_root_mean_squared_error']][results2['rank_test_neg_root_mean_squared_error']==1].index)\n",
    "\n",
    "best3 = list(set(best_r3) | set(best_mse3) | set(best_rmse3))\n",
    "\n",
    "best_df3 = results_orig[['mean_fit_time', 'mean_test_neg_mean_squared_error', 'mean_test_neg_root_mean_squared_error', 'mean_test_r2', 'params',]].loc[best3].sort_values(by=['mean_fit_time'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>mean_fit_time</th>\n",
       "      <th>mean_test_neg_mean_squared_error</th>\n",
       "      <th>mean_test_neg_root_mean_squared_error</th>\n",
       "      <th>mean_test_r2</th>\n",
       "      <th>params</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.058125</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'weig...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>199</th>\n",
       "      <td>0.058188</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'gain...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>195</th>\n",
       "      <td>0.058438</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'gain...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>203</th>\n",
       "      <td>0.058937</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'gain...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>0.059875</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'weig...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td>0.060063</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'weig...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>207</th>\n",
       "      <td>0.060687</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'gain...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>0.061938</td>\n",
       "      <td>-8.385927e+08</td>\n",
       "      <td>-28775.492507</td>\n",
       "      <td>0.863008</td>\n",
       "      <td>{'booster': 'gbtree', 'importance_type': 'weig...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     mean_fit_time  mean_test_neg_mean_squared_error  \\\n",
       "3         0.058125                     -8.385927e+08   \n",
       "199       0.058188                     -8.385927e+08   \n",
       "195       0.058438                     -8.385927e+08   \n",
       "203       0.058937                     -8.385927e+08   \n",
       "7         0.059875                     -8.385927e+08   \n",
       "11        0.060063                     -8.385927e+08   \n",
       "207       0.060687                     -8.385927e+08   \n",
       "15        0.061938                     -8.385927e+08   \n",
       "\n",
       "     mean_test_neg_root_mean_squared_error  mean_test_r2  \\\n",
       "3                            -28775.492507      0.863008   \n",
       "199                          -28775.492507      0.863008   \n",
       "195                          -28775.492507      0.863008   \n",
       "203                          -28775.492507      0.863008   \n",
       "7                            -28775.492507      0.863008   \n",
       "11                           -28775.492507      0.863008   \n",
       "207                          -28775.492507      0.863008   \n",
       "15                           -28775.492507      0.863008   \n",
       "\n",
       "                                                params  \n",
       "3    {'booster': 'gbtree', 'importance_type': 'weig...  \n",
       "199  {'booster': 'gbtree', 'importance_type': 'gain...  \n",
       "195  {'booster': 'gbtree', 'importance_type': 'gain...  \n",
       "203  {'booster': 'gbtree', 'importance_type': 'gain...  \n",
       "7    {'booster': 'gbtree', 'importance_type': 'weig...  \n",
       "11   {'booster': 'gbtree', 'importance_type': 'weig...  \n",
       "207  {'booster': 'gbtree', 'importance_type': 'gain...  \n",
       "15   {'booster': 'gbtree', 'importance_type': 'weig...  "
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "best_df3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>booster</th>\n",
       "      <th>importance_type</th>\n",
       "      <th>learning_rate</th>\n",
       "      <th>max_depth</th>\n",
       "      <th>max_leaves</th>\n",
       "      <th>n_estimators</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>8</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>6</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>8</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>10</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>gain</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>12</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td>gbtree</td>\n",
       "      <td>weight</td>\n",
       "      <td>0.5</td>\n",
       "      <td>5</td>\n",
       "      <td>12</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  booster importance_type  learning_rate  max_depth  max_leaves  n_estimators\n",
       "0  gbtree          weight            0.5          5           6            11\n",
       "1  gbtree            gain            0.5          5           8            11\n",
       "2  gbtree            gain            0.5          5           6            11\n",
       "3  gbtree            gain            0.5          5          10            11\n",
       "4  gbtree          weight            0.5          5           8            11\n",
       "5  gbtree          weight            0.5          5          10            11\n",
       "6  gbtree            gain            0.5          5          12            11\n",
       "7  gbtree          weight            0.5          5          12            11"
      ]
     },
     "execution_count": 30,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "pd.DataFrame(list(best_df3.params))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "## XGBoost Evaluation - Leave One Out Metrics Computing\n",
    "\n",
    "Now that we have some good knowledge about hyperparameters for the estimator, we can closely analyze how accurate the model is.\n",
    "\n",
    "To get the most accurate results on the test, we are going to use the Leave One Out Approach for the instances. The model is going to be evaluated multiple times and the dataset will be changed in place. \n",
    "This is going to allow us to find the best and worst predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'RMSE': 23420.254868662523,\n",
       " 'MSE': 548508338.1131107,\n",
       " 'MSLE': 0.015305551513536499,\n",
       " 'MAE': 16033.898817253621,\n",
       " 'R2': 0.9059901574555764,\n",
       " 'MAX_Err': 171266.25}"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df1 = df.copy()\n",
    "LOO_estimator_eval(dataset=test_df1, target='Sale_Price', \n",
    "                   estimator=XGBRegressor(\n",
    "                       booster='gbtree', importance_type='weight', learning_rate = 0.5,\n",
    "                       max_depth=5, max_leaves=6, n_estimators = 11), \n",
    "                   params=None, ignore=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'RMSE': 23420.254868662523,\n",
       " 'MSE': 548508338.1131107,\n",
       " 'MSLE': 0.015305551513536499,\n",
       " 'MAE': 16033.898817253621,\n",
       " 'R2': 0.9059901574555764,\n",
       " 'MAX_Err': 171266.25}"
      ]
     },
     "execution_count": 32,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df2 = df.copy()\n",
    "LOO_estimator_eval(dataset=test_df2, target='Sale_Price', \n",
    "                   estimator=XGBRegressor(\n",
    "                       booster='gbtree', importance_type='gain', learning_rate = 0.5,\n",
    "                       max_depth=5, max_leaves=8, n_estimators = 11), \n",
    "                   params=None, ignore=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that for the first dataset, the most parsimonious model is also the best one!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'RMSE': 198156.87459070655,\n",
       " 'MSE': 39266146947.557,\n",
       " 'MSLE': 145.12771146453164,\n",
       " 'MAE': 182843.00038124286,\n",
       " 'R2': -5.7298964033338144,\n",
       " 'MAX_Err': 615000.0}"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df3 = df2.copy()\n",
    "LOO_estimator_eval(dataset=test_df3, target='Sale_Price', \n",
    "                   estimator=XGBRegressor(\n",
    "                       booster='gbtree', importance_type='weight', learning_rate = 0.5,\n",
    "                       max_depth=5, max_leaves=6, n_estimators = 11), \n",
    "                   params=None, ignore=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'RMSE': 198156.87459070655,\n",
       " 'MSE': 39266146947.557,\n",
       " 'MSLE': 145.12771146453164,\n",
       " 'MAE': 182843.00038124286,\n",
       " 'R2': -5.7298964033338144,\n",
       " 'MAX_Err': 615000.0}"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df4 = df2.copy()\n",
    "LOO_estimator_eval(dataset=test_df4, target='Sale_Price', \n",
    "                   estimator=XGBRegressor(\n",
    "                       booster='gbtree', importance_type='gain', learning_rate = 0.5,\n",
    "                       max_depth=5, max_leaves=8, n_estimators = 11), \n",
    "                   params=None, ignore=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Curiosly enough, the RMSE is much smaller here, but the RSquared Value is looking very bad."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'RMSE': 28143.959008642858,\n",
       " 'MSE': 792082428.6801695,\n",
       " 'MSLE': 0.020786093949075216,\n",
       " 'MAE': 17273.812734641637,\n",
       " 'R2': 0.8758434180265611,\n",
       " 'MAX_Err': 371575.515625}"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df5 = df_orig.copy()\n",
    "LOO_estimator_eval(dataset=test_df5, target='Sale_Price', \n",
    "                   estimator=XGBRegressor(\n",
    "                       booster='gbtree', importance_type='weight', learning_rate = 0.5,\n",
    "                       max_depth=5, max_leaves=8, n_estimators = 11), \n",
    "                   params=None, ignore=None)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'RMSE': 28143.959008642858,\n",
       " 'MSE': 792082428.6801695,\n",
       " 'MSLE': 0.020786093949075216,\n",
       " 'MAE': 17273.812734641637,\n",
       " 'R2': 0.8758434180265611,\n",
       " 'MAX_Err': 371575.515625}"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df6 = df_orig.copy()\n",
    "LOO_estimator_eval(dataset=test_df6, target='Sale_Price', \n",
    "                   estimator=XGBRegressor(\n",
    "                       booster='gbtree', importance_type='gain', learning_rate = 0.5,\n",
    "                       max_depth=5, max_leaves=8, n_estimators = 11), \n",
    "                   params=None, ignore=None)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Worst Predictions and Best Predictions\n",
    "\n",
    "To consider the best and worst predictions, I decided to consider the models with the lowest error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Modified First Subset\n",
    "test_df1['Prediction_Error'] = np.abs(test_df1['Sale_Price']-test_df1['Predicted'])\n",
    "# Original Second Subset\n",
    "test_df6['Prediction_Error'] = np.abs(test_df6['Sale_Price']-test_df6['Predicted'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Most wrong on df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>BC_Bsmt_Unf_SF</th>\n",
       "      <th>Bedroom_AbvGr</th>\n",
       "      <th>Bsmt_Full_Bath</th>\n",
       "      <th>Bsmt_Unf_SF</th>\n",
       "      <th>Central_Air</th>\n",
       "      <th>External_Eval</th>\n",
       "      <th>Fireplace_Gr_Area_Ratio</th>\n",
       "      <th>Fireplace_Qu</th>\n",
       "      <th>Garage_Area</th>\n",
       "      <th>Garage_Cars</th>\n",
       "      <th>...</th>\n",
       "      <th>garage_type_1</th>\n",
       "      <th>hs_style_1</th>\n",
       "      <th>neighborhoods_1</th>\n",
       "      <th>neighborhoods_2</th>\n",
       "      <th>neighborhoods_3</th>\n",
       "      <th>neighborhoods_4</th>\n",
       "      <th>sale_cond_1</th>\n",
       "      <th>Sale_Price</th>\n",
       "      <th>Predicted</th>\n",
       "      <th>Prediction_Error</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1601</th>\n",
       "      <td>4.229794</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1795.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>98.000000</td>\n",
       "      <td>1795.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>895.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>147000.0</td>\n",
       "      <td>318266.250000</td>\n",
       "      <td>171266.250000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1191</th>\n",
       "      <td>3.545612</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>425.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>118.400000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>122000.0</td>\n",
       "      <td>285493.593750</td>\n",
       "      <td>163493.593750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2218</th>\n",
       "      <td>2.482185</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>48.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2681.800000</td>\n",
       "      <td>3500.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>959.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>584500.0</td>\n",
       "      <td>432576.000000</td>\n",
       "      <td>151924.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>957</th>\n",
       "      <td>3.341847</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>278.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1563.299900</td>\n",
       "      <td>1235.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>789.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>615000.0</td>\n",
       "      <td>482256.125000</td>\n",
       "      <td>132743.875000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1524</th>\n",
       "      <td>4.315548</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2153.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1344.599900</td>\n",
       "      <td>2643.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>694.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>380000.0</td>\n",
       "      <td>508107.656250</td>\n",
       "      <td>128107.656250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>389</th>\n",
       "      <td>4.213478</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1734.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1117.799900</td>\n",
       "      <td>2822.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1020.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>582933.0</td>\n",
       "      <td>456406.156250</td>\n",
       "      <td>126526.843750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1469</th>\n",
       "      <td>3.597804</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>474.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>280.000000</td>\n",
       "      <td>1419.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>567.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>392000.0</td>\n",
       "      <td>268796.093750</td>\n",
       "      <td>123203.906250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>388</th>\n",
       "      <td>3.441461</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>342.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1106.999900</td>\n",
       "      <td>1337.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>762.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>610000.0</td>\n",
       "      <td>487121.250000</td>\n",
       "      <td>122878.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2037</th>\n",
       "      <td>3.421390</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>328.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1502.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>286.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>330000.0</td>\n",
       "      <td>207637.312500</td>\n",
       "      <td>122362.687500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>40</th>\n",
       "      <td>3.016554</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>142.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>180.900000</td>\n",
       "      <td>1182.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>820.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>611657.0</td>\n",
       "      <td>494127.000000</td>\n",
       "      <td>117530.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2104</th>\n",
       "      <td>4.153759</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1528.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>642.600000</td>\n",
       "      <td>3390.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>758.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>545224.0</td>\n",
       "      <td>429872.468750</td>\n",
       "      <td>115351.531250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1398</th>\n",
       "      <td>2.934422</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>120.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>993.599900</td>\n",
       "      <td>1652.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>482.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>392500.0</td>\n",
       "      <td>286709.906250</td>\n",
       "      <td>105790.093750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1265</th>\n",
       "      <td>3.664462</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>545.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>50.699997</td>\n",
       "      <td>759.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>260000.0</td>\n",
       "      <td>154854.015625</td>\n",
       "      <td>105145.984375</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2148</th>\n",
       "      <td>3.653837</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>533.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>478.000000</td>\n",
       "      <td>1980.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>770.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>315000.0</td>\n",
       "      <td>419663.968750</td>\n",
       "      <td>104663.968750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>513</th>\n",
       "      <td>3.155462</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>189.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1649.100000</td>\n",
       "      <td>1007.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>864.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>275000.0</td>\n",
       "      <td>378369.687500</td>\n",
       "      <td>103369.687500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>479</th>\n",
       "      <td>4.240449</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1836.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>350.000000</td>\n",
       "      <td>1836.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>517.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>264500.0</td>\n",
       "      <td>163643.796875</td>\n",
       "      <td>100856.203125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>343</th>\n",
       "      <td>4.119120</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1420.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1188.000000</td>\n",
       "      <td>2524.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>542.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>278000.0</td>\n",
       "      <td>376732.625000</td>\n",
       "      <td>98732.625000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1383</th>\n",
       "      <td>3.580342</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>457.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>618.700000</td>\n",
       "      <td>977.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>431.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>311500.0</td>\n",
       "      <td>212800.390625</td>\n",
       "      <td>98699.609375</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>961</th>\n",
       "      <td>3.521349</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>404.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>120.000000</td>\n",
       "      <td>2649.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>746.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>421250.0</td>\n",
       "      <td>323316.468750</td>\n",
       "      <td>97933.531250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2593</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>312.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>230000.0</td>\n",
       "      <td>133930.328125</td>\n",
       "      <td>96069.671875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>269</th>\n",
       "      <td>3.619483</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>496.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>848.700000</td>\n",
       "      <td>951.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>567.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>335000.0</td>\n",
       "      <td>242187.640625</td>\n",
       "      <td>92812.359375</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>855</th>\n",
       "      <td>2.934422</td>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>120.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>993.599900</td>\n",
       "      <td>1648.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>482.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>230000.0</td>\n",
       "      <td>320112.843750</td>\n",
       "      <td>90112.843750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2608</th>\n",
       "      <td>2.869074</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>105.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>972.399960</td>\n",
       "      <td>2521.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>576.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>151400.0</td>\n",
       "      <td>239301.968750</td>\n",
       "      <td>87901.968750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1867</th>\n",
       "      <td>3.856621</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>816.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>179.200000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>576.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>112000.0</td>\n",
       "      <td>197711.171875</td>\n",
       "      <td>85711.171875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1896</th>\n",
       "      <td>3.403482</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>316.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>538.000000</td>\n",
       "      <td>2828.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1052.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>424870.0</td>\n",
       "      <td>339263.437500</td>\n",
       "      <td>85606.562500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>875</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1771.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>336.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>115000.0</td>\n",
       "      <td>199875.718750</td>\n",
       "      <td>84875.718750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>684</th>\n",
       "      <td>3.876568</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>851.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>290.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>97500.0</td>\n",
       "      <td>181545.125000</td>\n",
       "      <td>84045.125000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>749</th>\n",
       "      <td>4.231106</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1800.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>848.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>702.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>239000.0</td>\n",
       "      <td>322441.468750</td>\n",
       "      <td>83441.468750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1623</th>\n",
       "      <td>4.169567</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1580.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>746.000000</td>\n",
       "      <td>2466.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>610.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>340000.0</td>\n",
       "      <td>258578.953125</td>\n",
       "      <td>81421.046875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2150</th>\n",
       "      <td>4.058435</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1249.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>512.199950</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>840.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>342643.0</td>\n",
       "      <td>423082.875000</td>\n",
       "      <td>80439.875000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>30 rows × 39 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      BC_Bsmt_Unf_SF  Bedroom_AbvGr  Bsmt_Full_Bath  Bsmt_Unf_SF  Central_Air  \\\n",
       "1601        4.229794            2.0             0.0       1795.0          0.0   \n",
       "1191        3.545612            4.0             0.0        425.0          1.0   \n",
       "2218        2.482185            4.0             1.0         48.0          0.0   \n",
       "957         3.341847            1.0             2.0        278.0          0.0   \n",
       "1524        4.315548            3.0             0.0       2153.0          0.0   \n",
       "389         4.213478            4.0             0.0       1734.0          0.0   \n",
       "1469        3.597804            2.0             1.0        474.0          0.0   \n",
       "388         3.441461            2.0             2.0        342.0          0.0   \n",
       "2037        3.421390            1.0             1.0        328.0          0.0   \n",
       "40          3.016554            2.0             1.0        142.0          0.0   \n",
       "2104        4.153759            5.0             0.0       1528.0          0.0   \n",
       "1398        2.934422            2.0             1.0        120.0          0.0   \n",
       "1265        3.664462            2.0             1.0        545.0          0.0   \n",
       "2148        3.653837            3.0             1.0        533.0          0.0   \n",
       "513         3.155462            2.0             1.0        189.0          0.0   \n",
       "479         4.240449            3.0             0.0       1836.0          0.0   \n",
       "343         4.119120            4.0             1.0       1420.0          0.0   \n",
       "1383        3.580342            3.0             0.0        457.0          0.0   \n",
       "961         3.521349            4.0             1.0        404.0          0.0   \n",
       "2593        0.000000            2.0             0.0          0.0          0.0   \n",
       "269         3.619483            3.0             1.0        496.0          0.0   \n",
       "855         2.934422            2.0             1.0        120.0          0.0   \n",
       "2608        2.869074            5.0             1.0        105.0          0.0   \n",
       "1867        3.856621            3.0             0.0        816.0          0.0   \n",
       "1896        3.403482            4.0             1.0        316.0          0.0   \n",
       "875         0.000000            3.0             0.0          0.0          0.0   \n",
       "684         3.876568            6.0             0.0        851.0          1.0   \n",
       "749         4.231106            2.0             0.0       1800.0          0.0   \n",
       "1623        4.169567            4.0             0.0       1580.0          0.0   \n",
       "2150        4.058435            4.0             0.0       1249.0          0.0   \n",
       "\n",
       "      External_Eval  Fireplace_Gr_Area_Ratio  Fireplace_Qu  Garage_Area  \\\n",
       "1601      98.000000                   1795.0           4.0        895.0   \n",
       "1191     118.400000                      0.0           0.0          0.0   \n",
       "2218    2681.800000                   3500.0           3.0        959.0   \n",
       "957     1563.299900                   1235.0           4.0        789.0   \n",
       "1524    1344.599900                   2643.0           4.0        694.0   \n",
       "389     1117.799900                   2822.0           4.0       1020.0   \n",
       "1469     280.000000                   1419.0           4.0        567.0   \n",
       "388     1106.999900                   1337.0           4.0        762.0   \n",
       "2037       0.000000                   1502.0           3.0        286.0   \n",
       "40       180.900000                   1182.0           4.0        820.0   \n",
       "2104     642.600000                   3390.0           5.0        758.0   \n",
       "1398     993.599900                   1652.0           4.0        482.0   \n",
       "1265      50.699997                    759.0           4.0          0.0   \n",
       "2148     478.000000                   1980.0           4.0        770.0   \n",
       "513     1649.100000                   1007.0           4.0        864.0   \n",
       "479      350.000000                   1836.0           4.0        517.0   \n",
       "343     1188.000000                   2524.0           4.0        542.0   \n",
       "1383     618.700000                    977.0           3.0        431.0   \n",
       "961      120.000000                   2649.0           3.0        746.0   \n",
       "2593       0.000000                      0.0           0.0        312.0   \n",
       "269      848.700000                    951.0           3.0        567.0   \n",
       "855      993.599900                   1648.0           4.0        482.0   \n",
       "2608     972.399960                   2521.0           4.0        576.0   \n",
       "1867     179.200000                      0.0           0.0        576.0   \n",
       "1896     538.000000                   2828.0           3.0       1052.0   \n",
       "875        0.000000                   1771.0           3.0        336.0   \n",
       "684      290.000000                      0.0           0.0          0.0   \n",
       "749      848.000000                      0.0           0.0        702.0   \n",
       "1623     746.000000                   2466.0           4.0        610.0   \n",
       "2150     512.199950                      0.0           0.0        840.0   \n",
       "\n",
       "      Garage_Cars  ...  garage_type_1  hs_style_1  neighborhoods_1  \\\n",
       "1601          3.0  ...            1.0         1.0              0.0   \n",
       "1191          0.0  ...            0.0         1.0              1.0   \n",
       "2218          3.0  ...            1.0         1.0              0.0   \n",
       "957           3.0  ...            1.0         1.0              0.0   \n",
       "1524          3.0  ...            1.0         1.0              0.0   \n",
       "389           3.0  ...            1.0         1.0              0.0   \n",
       "1469          2.0  ...            1.0         1.0              0.0   \n",
       "388           3.0  ...            1.0         1.0              0.0   \n",
       "2037          1.0  ...            1.0         1.0              0.0   \n",
       "40            3.0  ...            1.0         1.0              0.0   \n",
       "2104          3.0  ...            1.0         1.0              0.0   \n",
       "1398          2.0  ...            1.0         1.0              0.0   \n",
       "1265          0.0  ...            0.0         1.0              1.0   \n",
       "2148          3.0  ...            1.0         1.0              0.0   \n",
       "513           3.0  ...            1.0         1.0              0.0   \n",
       "479           2.0  ...            1.0         1.0              0.0   \n",
       "343           2.0  ...            1.0         1.0              0.0   \n",
       "1383          2.0  ...            1.0         0.0              0.0   \n",
       "961           3.0  ...            1.0         1.0              0.0   \n",
       "2593          1.0  ...            1.0         1.0              0.0   \n",
       "269           2.0  ...            1.0         1.0              0.0   \n",
       "855           2.0  ...            1.0         1.0              0.0   \n",
       "2608          2.0  ...            1.0         1.0              0.0   \n",
       "1867          2.0  ...            1.0         1.0              1.0   \n",
       "1896          3.0  ...            1.0         1.0              0.0   \n",
       "875           2.0  ...            1.0         1.0              0.0   \n",
       "684           0.0  ...            0.0         0.0              1.0   \n",
       "749           2.0  ...            1.0         1.0              0.0   \n",
       "1623          2.0  ...            1.0         1.0              0.0   \n",
       "2150          3.0  ...            1.0         1.0              0.0   \n",
       "\n",
       "      neighborhoods_2  neighborhoods_3  neighborhoods_4  sale_cond_1  \\\n",
       "1601              0.0              1.0              0.0          0.0   \n",
       "1191              0.0              0.0              0.0          1.0   \n",
       "2218              0.0              0.0              0.0          1.0   \n",
       "957               0.0              0.0              0.0          1.0   \n",
       "1524              0.0              0.0              0.0          1.0   \n",
       "389               0.0              0.0              0.0          0.0   \n",
       "1469              0.0              0.0              0.0          0.0   \n",
       "388               0.0              0.0              0.0          1.0   \n",
       "2037              0.0              0.0              1.0          1.0   \n",
       "40                0.0              0.0              0.0          0.0   \n",
       "2104              0.0              0.0              0.0          0.0   \n",
       "1398              0.0              0.0              1.0          0.0   \n",
       "1265              0.0              0.0              0.0          1.0   \n",
       "2148              0.0              0.0              0.0          1.0   \n",
       "513               0.0              0.0              1.0          1.0   \n",
       "479               0.0              1.0              0.0          0.0   \n",
       "343               0.0              1.0              0.0          1.0   \n",
       "1383              0.0              0.0              1.0          1.0   \n",
       "961               0.0              0.0              0.0          1.0   \n",
       "2593              0.0              0.0              1.0          1.0   \n",
       "269               0.0              0.0              1.0          1.0   \n",
       "855               0.0              0.0              1.0          0.0   \n",
       "2608              1.0              0.0              0.0          1.0   \n",
       "1867              0.0              0.0              0.0          1.0   \n",
       "1896              0.0              0.0              1.0          0.0   \n",
       "875               0.0              0.0              1.0          1.0   \n",
       "684               0.0              0.0              0.0          1.0   \n",
       "749               0.0              0.0              1.0          1.0   \n",
       "1623              0.0              1.0              0.0          1.0   \n",
       "2150              0.0              0.0              0.0          0.0   \n",
       "\n",
       "      Sale_Price      Predicted  Prediction_Error  \n",
       "1601    147000.0  318266.250000     171266.250000  \n",
       "1191    122000.0  285493.593750     163493.593750  \n",
       "2218    584500.0  432576.000000     151924.000000  \n",
       "957     615000.0  482256.125000     132743.875000  \n",
       "1524    380000.0  508107.656250     128107.656250  \n",
       "389     582933.0  456406.156250     126526.843750  \n",
       "1469    392000.0  268796.093750     123203.906250  \n",
       "388     610000.0  487121.250000     122878.750000  \n",
       "2037    330000.0  207637.312500     122362.687500  \n",
       "40      611657.0  494127.000000     117530.000000  \n",
       "2104    545224.0  429872.468750     115351.531250  \n",
       "1398    392500.0  286709.906250     105790.093750  \n",
       "1265    260000.0  154854.015625     105145.984375  \n",
       "2148    315000.0  419663.968750     104663.968750  \n",
       "513     275000.0  378369.687500     103369.687500  \n",
       "479     264500.0  163643.796875     100856.203125  \n",
       "343     278000.0  376732.625000      98732.625000  \n",
       "1383    311500.0  212800.390625      98699.609375  \n",
       "961     421250.0  323316.468750      97933.531250  \n",
       "2593    230000.0  133930.328125      96069.671875  \n",
       "269     335000.0  242187.640625      92812.359375  \n",
       "855     230000.0  320112.843750      90112.843750  \n",
       "2608    151400.0  239301.968750      87901.968750  \n",
       "1867    112000.0  197711.171875      85711.171875  \n",
       "1896    424870.0  339263.437500      85606.562500  \n",
       "875     115000.0  199875.718750      84875.718750  \n",
       "684      97500.0  181545.125000      84045.125000  \n",
       "749     239000.0  322441.468750      83441.468750  \n",
       "1623    340000.0  258578.953125      81421.046875  \n",
       "2150    342643.0  423082.875000      80439.875000  \n",
       "\n",
       "[30 rows x 39 columns]"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df1.sort_values(by=['Prediction_Error', 'Sale_Price'], ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>BC_Bsmt_Unf_SF</th>\n",
       "      <th>Bedroom_AbvGr</th>\n",
       "      <th>Bsmt_Full_Bath</th>\n",
       "      <th>Bsmt_Unf_SF</th>\n",
       "      <th>Central_Air</th>\n",
       "      <th>External_Eval</th>\n",
       "      <th>Fireplace_Gr_Area_Ratio</th>\n",
       "      <th>Fireplace_Qu</th>\n",
       "      <th>Garage_Area</th>\n",
       "      <th>Garage_Cars</th>\n",
       "      <th>...</th>\n",
       "      <th>garage_type_1</th>\n",
       "      <th>hs_style_1</th>\n",
       "      <th>neighborhoods_1</th>\n",
       "      <th>neighborhoods_2</th>\n",
       "      <th>neighborhoods_3</th>\n",
       "      <th>neighborhoods_4</th>\n",
       "      <th>sale_cond_1</th>\n",
       "      <th>Sale_Price</th>\n",
       "      <th>Predicted</th>\n",
       "      <th>Prediction_Error</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>3.368790</td>\n",
       "      <td>3.120000</td>\n",
       "      <td>0.540000</td>\n",
       "      <td>764.200000</td>\n",
       "      <td>0.040000</td>\n",
       "      <td>633.975980</td>\n",
       "      <td>1620.180000</td>\n",
       "      <td>3.080000</td>\n",
       "      <td>613.640000</td>\n",
       "      <td>2.280000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.880000</td>\n",
       "      <td>0.920000</td>\n",
       "      <td>0.160000</td>\n",
       "      <td>0.020000</td>\n",
       "      <td>0.14000</td>\n",
       "      <td>0.30000</td>\n",
       "      <td>0.620000</td>\n",
       "      <td>323964.260000</td>\n",
       "      <td>299337.077500</td>\n",
       "      <td>94251.421875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.109907</td>\n",
       "      <td>1.099907</td>\n",
       "      <td>0.613122</td>\n",
       "      <td>666.234591</td>\n",
       "      <td>0.197949</td>\n",
       "      <td>558.912872</td>\n",
       "      <td>991.684445</td>\n",
       "      <td>1.588736</td>\n",
       "      <td>238.886512</td>\n",
       "      <td>0.809132</td>\n",
       "      <td>...</td>\n",
       "      <td>0.328261</td>\n",
       "      <td>0.274048</td>\n",
       "      <td>0.370328</td>\n",
       "      <td>0.141421</td>\n",
       "      <td>0.35051</td>\n",
       "      <td>0.46291</td>\n",
       "      <td>0.490314</td>\n",
       "      <td>132042.402367</td>\n",
       "      <td>104149.206779</td>\n",
       "      <td>25378.589595</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>97500.000000</td>\n",
       "      <td>133930.328125</td>\n",
       "      <td>68075.468750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>3.101604</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>169.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>185.675000</td>\n",
       "      <td>1005.125000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>495.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>236000.000000</td>\n",
       "      <td>214416.976562</td>\n",
       "      <td>74806.179688</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>3.636660</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>514.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>545.000000</td>\n",
       "      <td>1711.500000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>576.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>321500.000000</td>\n",
       "      <td>286101.750000</td>\n",
       "      <td>85241.140625</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>4.096378</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1354.750000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>920.175000</td>\n",
       "      <td>2507.250000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>784.250000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>382977.500000</td>\n",
       "      <td>368434.148438</td>\n",
       "      <td>105025.480469</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>4.315548</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>2153.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>2681.800000</td>\n",
       "      <td>3500.000000</td>\n",
       "      <td>5.000000</td>\n",
       "      <td>1052.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>615000.000000</td>\n",
       "      <td>508107.656250</td>\n",
       "      <td>171266.250000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 39 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       BC_Bsmt_Unf_SF  Bedroom_AbvGr  Bsmt_Full_Bath  Bsmt_Unf_SF  \\\n",
       "count       50.000000      50.000000       50.000000    50.000000   \n",
       "mean         3.368790       3.120000        0.540000   764.200000   \n",
       "std          1.109907       1.099907        0.613122   666.234591   \n",
       "min          0.000000       1.000000        0.000000     0.000000   \n",
       "25%          3.101604       2.000000        0.000000   169.500000   \n",
       "50%          3.636660       3.000000        0.000000   514.500000   \n",
       "75%          4.096378       4.000000        1.000000  1354.750000   \n",
       "max          4.315548       6.000000        2.000000  2153.000000   \n",
       "\n",
       "       Central_Air  External_Eval  Fireplace_Gr_Area_Ratio  Fireplace_Qu  \\\n",
       "count    50.000000      50.000000                50.000000     50.000000   \n",
       "mean      0.040000     633.975980              1620.180000      3.080000   \n",
       "std       0.197949     558.912872               991.684445      1.588736   \n",
       "min       0.000000       0.000000                 0.000000      0.000000   \n",
       "25%       0.000000     185.675000              1005.125000      3.000000   \n",
       "50%       0.000000     545.000000              1711.500000      4.000000   \n",
       "75%       0.000000     920.175000              2507.250000      4.000000   \n",
       "max       1.000000    2681.800000              3500.000000      5.000000   \n",
       "\n",
       "       Garage_Area  Garage_Cars  ...  garage_type_1  hs_style_1  \\\n",
       "count    50.000000    50.000000  ...      50.000000   50.000000   \n",
       "mean    613.640000     2.280000  ...       0.880000    0.920000   \n",
       "std     238.886512     0.809132  ...       0.328261    0.274048   \n",
       "min       0.000000     0.000000  ...       0.000000    0.000000   \n",
       "25%     495.000000     2.000000  ...       1.000000    1.000000   \n",
       "50%     576.000000     2.000000  ...       1.000000    1.000000   \n",
       "75%     784.250000     3.000000  ...       1.000000    1.000000   \n",
       "max    1052.000000     3.000000  ...       1.000000    1.000000   \n",
       "\n",
       "       neighborhoods_1  neighborhoods_2  neighborhoods_3  neighborhoods_4  \\\n",
       "count        50.000000        50.000000         50.00000         50.00000   \n",
       "mean          0.160000         0.020000          0.14000          0.30000   \n",
       "std           0.370328         0.141421          0.35051          0.46291   \n",
       "min           0.000000         0.000000          0.00000          0.00000   \n",
       "25%           0.000000         0.000000          0.00000          0.00000   \n",
       "50%           0.000000         0.000000          0.00000          0.00000   \n",
       "75%           0.000000         0.000000          0.00000          1.00000   \n",
       "max           1.000000         1.000000          1.00000          1.00000   \n",
       "\n",
       "       sale_cond_1     Sale_Price      Predicted  Prediction_Error  \n",
       "count    50.000000      50.000000      50.000000         50.000000  \n",
       "mean      0.620000  323964.260000  299337.077500      94251.421875  \n",
       "std       0.490314  132042.402367  104149.206779      25378.589595  \n",
       "min       0.000000   97500.000000  133930.328125      68075.468750  \n",
       "25%       0.000000  236000.000000  214416.976562      74806.179688  \n",
       "50%       1.000000  321500.000000  286101.750000      85241.140625  \n",
       "75%       1.000000  382977.500000  368434.148438     105025.480469  \n",
       "max       1.000000  615000.000000  508107.656250     171266.250000  \n",
       "\n",
       "[8 rows x 39 columns]"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df1.sort_values(by=['Prediction_Error', 'Sale_Price'], ascending=False).head(50).describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Best on df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>BC_Bsmt_Unf_SF</th>\n",
       "      <th>Bedroom_AbvGr</th>\n",
       "      <th>Bsmt_Full_Bath</th>\n",
       "      <th>Bsmt_Unf_SF</th>\n",
       "      <th>Central_Air</th>\n",
       "      <th>External_Eval</th>\n",
       "      <th>Fireplace_Gr_Area_Ratio</th>\n",
       "      <th>Fireplace_Qu</th>\n",
       "      <th>Garage_Area</th>\n",
       "      <th>Garage_Cars</th>\n",
       "      <th>...</th>\n",
       "      <th>garage_type_1</th>\n",
       "      <th>hs_style_1</th>\n",
       "      <th>neighborhoods_1</th>\n",
       "      <th>neighborhoods_2</th>\n",
       "      <th>neighborhoods_3</th>\n",
       "      <th>neighborhoods_4</th>\n",
       "      <th>sale_cond_1</th>\n",
       "      <th>Sale_Price</th>\n",
       "      <th>Predicted</th>\n",
       "      <th>Prediction_Error</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2310</th>\n",
       "      <td>3.562210</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>440.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>85500.0</td>\n",
       "      <td>85484.546875</td>\n",
       "      <td>15.453125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>463</th>\n",
       "      <td>3.973080</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1043.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>694.000000</td>\n",
       "      <td>1432.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>492.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>245000.0</td>\n",
       "      <td>245021.968750</td>\n",
       "      <td>21.968750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2462</th>\n",
       "      <td>3.979399</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1057.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>288.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>137000.0</td>\n",
       "      <td>137022.953125</td>\n",
       "      <td>22.953125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1270</th>\n",
       "      <td>3.373666</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>297.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>472.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>280.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>134500.0</td>\n",
       "      <td>134534.062500</td>\n",
       "      <td>34.062500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1626</th>\n",
       "      <td>3.472692</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>365.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>576.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>149000.0</td>\n",
       "      <td>148950.078125</td>\n",
       "      <td>49.921875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>654</th>\n",
       "      <td>3.783721</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>700.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>388.699980</td>\n",
       "      <td>1742.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>424.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>168000.0</td>\n",
       "      <td>168052.265625</td>\n",
       "      <td>52.265625</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1880</th>\n",
       "      <td>3.529576</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>411.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>399.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>118000.0</td>\n",
       "      <td>118053.171875</td>\n",
       "      <td>53.171875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>82</th>\n",
       "      <td>3.553431</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>432.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>280.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>122000.0</td>\n",
       "      <td>122053.960938</td>\n",
       "      <td>53.960938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2327</th>\n",
       "      <td>3.857784</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>818.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>58.499996</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>292.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>145250.0</td>\n",
       "      <td>145194.125000</td>\n",
       "      <td>55.875000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>965</th>\n",
       "      <td>4.094168</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1347.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>66.000000</td>\n",
       "      <td>1367.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>484.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>192000.0</td>\n",
       "      <td>191943.046875</td>\n",
       "      <td>56.953125</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 39 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "      BC_Bsmt_Unf_SF  Bedroom_AbvGr  Bsmt_Full_Bath  Bsmt_Unf_SF  Central_Air  \\\n",
       "2310        3.562210            2.0             0.0        440.0          0.0   \n",
       "463         3.973080            3.0             0.0       1043.0          0.0   \n",
       "2462        3.979399            3.0             0.0       1057.0          0.0   \n",
       "1270        3.373666            3.0             0.0        297.0          0.0   \n",
       "1626        3.472692            3.0             1.0        365.0          0.0   \n",
       "654         3.783721            4.0             0.0        700.0          0.0   \n",
       "1880        3.529576            3.0             0.0        411.0          0.0   \n",
       "82          3.553431            3.0             0.0        432.0          0.0   \n",
       "2327        3.857784            3.0             0.0        818.0          0.0   \n",
       "965         4.094168            2.0             0.0       1347.0          0.0   \n",
       "\n",
       "      External_Eval  Fireplace_Gr_Area_Ratio  Fireplace_Qu  Garage_Area  \\\n",
       "2310       0.000000                      0.0           0.0          0.0   \n",
       "463      694.000000                   1432.0           4.0        492.0   \n",
       "2462       0.000000                      0.0           0.0        288.0   \n",
       "1270     472.000000                      0.0           0.0        280.0   \n",
       "1626       0.000000                      0.0           0.0        576.0   \n",
       "654      388.699980                   1742.0           4.0        424.0   \n",
       "1880       0.000000                      0.0           0.0        399.0   \n",
       "82         0.000000                      0.0           0.0        280.0   \n",
       "2327      58.499996                      0.0           0.0        292.0   \n",
       "965       66.000000                   1367.0           4.0        484.0   \n",
       "\n",
       "      Garage_Cars  ...  garage_type_1  hs_style_1  neighborhoods_1  \\\n",
       "2310          0.0  ...            0.0         1.0              1.0   \n",
       "463           2.0  ...            1.0         1.0              0.0   \n",
       "2462          1.0  ...            1.0         1.0              1.0   \n",
       "1270          1.0  ...            0.0         1.0              1.0   \n",
       "1626          2.0  ...            0.0         1.0              1.0   \n",
       "654           2.0  ...            1.0         1.0              1.0   \n",
       "1880          1.0  ...            1.0         1.0              1.0   \n",
       "82            1.0  ...            0.0         1.0              1.0   \n",
       "2327          1.0  ...            1.0         1.0              1.0   \n",
       "965           2.0  ...            1.0         1.0              0.0   \n",
       "\n",
       "      neighborhoods_2  neighborhoods_3  neighborhoods_4  sale_cond_1  \\\n",
       "2310              0.0              0.0              0.0          1.0   \n",
       "463               0.0              1.0              0.0          1.0   \n",
       "2462              0.0              0.0              0.0          1.0   \n",
       "1270              0.0              0.0              0.0          1.0   \n",
       "1626              0.0              0.0              0.0          1.0   \n",
       "654               0.0              0.0              0.0          1.0   \n",
       "1880              0.0              0.0              0.0          1.0   \n",
       "82                0.0              0.0              0.0          1.0   \n",
       "2327              0.0              0.0              0.0          1.0   \n",
       "965               0.0              0.0              0.0          1.0   \n",
       "\n",
       "      Sale_Price      Predicted  Prediction_Error  \n",
       "2310     85500.0   85484.546875         15.453125  \n",
       "463     245000.0  245021.968750         21.968750  \n",
       "2462    137000.0  137022.953125         22.953125  \n",
       "1270    134500.0  134534.062500         34.062500  \n",
       "1626    149000.0  148950.078125         49.921875  \n",
       "654     168000.0  168052.265625         52.265625  \n",
       "1880    118000.0  118053.171875         53.171875  \n",
       "82      122000.0  122053.960938         53.960938  \n",
       "2327    145250.0  145194.125000         55.875000  \n",
       "965     192000.0  191943.046875         56.953125  \n",
       "\n",
       "[10 rows x 39 columns]"
      ]
     },
     "execution_count": 40,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df1.sort_values(by=['Prediction_Error', 'Sale_Price']).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>BC_Bsmt_Unf_SF</th>\n",
       "      <th>Bedroom_AbvGr</th>\n",
       "      <th>Bsmt_Full_Bath</th>\n",
       "      <th>Bsmt_Unf_SF</th>\n",
       "      <th>Central_Air</th>\n",
       "      <th>External_Eval</th>\n",
       "      <th>Fireplace_Gr_Area_Ratio</th>\n",
       "      <th>Fireplace_Qu</th>\n",
       "      <th>Garage_Area</th>\n",
       "      <th>Garage_Cars</th>\n",
       "      <th>...</th>\n",
       "      <th>garage_type_1</th>\n",
       "      <th>hs_style_1</th>\n",
       "      <th>neighborhoods_1</th>\n",
       "      <th>neighborhoods_2</th>\n",
       "      <th>neighborhoods_3</th>\n",
       "      <th>neighborhoods_4</th>\n",
       "      <th>sale_cond_1</th>\n",
       "      <th>Sale_Price</th>\n",
       "      <th>Predicted</th>\n",
       "      <th>Prediction_Error</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>3.326493</td>\n",
       "      <td>2.780000</td>\n",
       "      <td>0.340000</td>\n",
       "      <td>578.800000</td>\n",
       "      <td>0.060000</td>\n",
       "      <td>187.477998</td>\n",
       "      <td>603.080000</td>\n",
       "      <td>1.500000</td>\n",
       "      <td>399.160000</td>\n",
       "      <td>1.520000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.580000</td>\n",
       "      <td>0.760000</td>\n",
       "      <td>0.70000</td>\n",
       "      <td>0.020000</td>\n",
       "      <td>0.14000</td>\n",
       "      <td>0.040000</td>\n",
       "      <td>0.980000</td>\n",
       "      <td>153716.000000</td>\n",
       "      <td>153690.402344</td>\n",
       "      <td>168.007031</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>1.054502</td>\n",
       "      <td>0.887326</td>\n",
       "      <td>0.478518</td>\n",
       "      <td>409.131933</td>\n",
       "      <td>0.239898</td>\n",
       "      <td>210.001301</td>\n",
       "      <td>761.514632</td>\n",
       "      <td>1.821078</td>\n",
       "      <td>208.550817</td>\n",
       "      <td>0.735125</td>\n",
       "      <td>...</td>\n",
       "      <td>0.498569</td>\n",
       "      <td>0.431419</td>\n",
       "      <td>0.46291</td>\n",
       "      <td>0.141421</td>\n",
       "      <td>0.35051</td>\n",
       "      <td>0.197949</td>\n",
       "      <td>0.141421</td>\n",
       "      <td>52616.944039</td>\n",
       "      <td>52619.138127</td>\n",
       "      <td>113.449095</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>72000.000000</td>\n",
       "      <td>71654.640625</td>\n",
       "      <td>15.453125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>3.374874</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>297.750000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.300000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>280.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>118375.000000</td>\n",
       "      <td>118384.095703</td>\n",
       "      <td>66.750000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>3.608643</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>485.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>89.199997</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>399.500000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>138250.000000</td>\n",
       "      <td>138048.171875</td>\n",
       "      <td>124.121094</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>3.856617</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>816.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>352.274995</td>\n",
       "      <td>1373.750000</td>\n",
       "      <td>3.750000</td>\n",
       "      <td>490.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>181175.000000</td>\n",
       "      <td>181150.703125</td>\n",
       "      <td>277.234375</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>4.184280</td>\n",
       "      <td>6.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1630.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>732.000000</td>\n",
       "      <td>2287.000000</td>\n",
       "      <td>4.000000</td>\n",
       "      <td>907.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>332200.000000</td>\n",
       "      <td>332524.781250</td>\n",
       "      <td>395.593750</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 39 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       BC_Bsmt_Unf_SF  Bedroom_AbvGr  Bsmt_Full_Bath  Bsmt_Unf_SF  \\\n",
       "count       50.000000      50.000000       50.000000    50.000000   \n",
       "mean         3.326493       2.780000        0.340000   578.800000   \n",
       "std          1.054502       0.887326        0.478518   409.131933   \n",
       "min          0.000000       1.000000        0.000000     0.000000   \n",
       "25%          3.374874       2.000000        0.000000   297.750000   \n",
       "50%          3.608643       3.000000        0.000000   485.000000   \n",
       "75%          3.856617       3.000000        1.000000   816.000000   \n",
       "max          4.184280       6.000000        1.000000  1630.000000   \n",
       "\n",
       "       Central_Air  External_Eval  Fireplace_Gr_Area_Ratio  Fireplace_Qu  \\\n",
       "count    50.000000      50.000000                50.000000     50.000000   \n",
       "mean      0.060000     187.477998               603.080000      1.500000   \n",
       "std       0.239898     210.001301               761.514632      1.821078   \n",
       "min       0.000000       0.000000                 0.000000      0.000000   \n",
       "25%       0.000000       1.300000                 0.000000      0.000000   \n",
       "50%       0.000000      89.199997                 0.000000      0.000000   \n",
       "75%       0.000000     352.274995              1373.750000      3.750000   \n",
       "max       1.000000     732.000000              2287.000000      4.000000   \n",
       "\n",
       "       Garage_Area  Garage_Cars  ...  garage_type_1  hs_style_1  \\\n",
       "count    50.000000    50.000000  ...      50.000000   50.000000   \n",
       "mean    399.160000     1.520000  ...       0.580000    0.760000   \n",
       "std     208.550817     0.735125  ...       0.498569    0.431419   \n",
       "min       0.000000     0.000000  ...       0.000000    0.000000   \n",
       "25%     280.000000     1.000000  ...       0.000000    1.000000   \n",
       "50%     399.500000     2.000000  ...       1.000000    1.000000   \n",
       "75%     490.000000     2.000000  ...       1.000000    1.000000   \n",
       "max     907.000000     3.000000  ...       1.000000    1.000000   \n",
       "\n",
       "       neighborhoods_1  neighborhoods_2  neighborhoods_3  neighborhoods_4  \\\n",
       "count         50.00000        50.000000         50.00000        50.000000   \n",
       "mean           0.70000         0.020000          0.14000         0.040000   \n",
       "std            0.46291         0.141421          0.35051         0.197949   \n",
       "min            0.00000         0.000000          0.00000         0.000000   \n",
       "25%            0.00000         0.000000          0.00000         0.000000   \n",
       "50%            1.00000         0.000000          0.00000         0.000000   \n",
       "75%            1.00000         0.000000          0.00000         0.000000   \n",
       "max            1.00000         1.000000          1.00000         1.000000   \n",
       "\n",
       "       sale_cond_1     Sale_Price      Predicted  Prediction_Error  \n",
       "count    50.000000      50.000000      50.000000         50.000000  \n",
       "mean      0.980000  153716.000000  153690.402344        168.007031  \n",
       "std       0.141421   52616.944039   52619.138127        113.449095  \n",
       "min       0.000000   72000.000000   71654.640625         15.453125  \n",
       "25%       1.000000  118375.000000  118384.095703         66.750000  \n",
       "50%       1.000000  138250.000000  138048.171875        124.121094  \n",
       "75%       1.000000  181175.000000  181150.703125        277.234375  \n",
       "max       1.000000  332200.000000  332524.781250        395.593750  \n",
       "\n",
       "[8 rows x 39 columns]"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df1.sort_values(by=['Prediction_Error', 'Sale_Price']).head(50).describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Most wrong on df_orig"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>BC_Bsmt_Unf_SF</th>\n",
       "      <th>BC_External_SF</th>\n",
       "      <th>Baths</th>\n",
       "      <th>Bedroom_Liv_Area_Ratio</th>\n",
       "      <th>Bsmt_Eval</th>\n",
       "      <th>Bsmt_Unf_SF</th>\n",
       "      <th>Central_Air</th>\n",
       "      <th>Exter_Qual</th>\n",
       "      <th>Fireplace_Eval</th>\n",
       "      <th>...</th>\n",
       "      <th>bldg_type_3</th>\n",
       "      <th>garage_type_1</th>\n",
       "      <th>hs_style_1</th>\n",
       "      <th>neighborhoods_1</th>\n",
       "      <th>neighborhoods_3</th>\n",
       "      <th>neighborhoods_4</th>\n",
       "      <th>sale_cond_1</th>\n",
       "      <th>Sale_Price</th>\n",
       "      <th>Predicted</th>\n",
       "      <th>Prediction_Error</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>423</th>\n",
       "      <td>1.0</td>\n",
       "      <td>3.851456</td>\n",
       "      <td>20.762966</td>\n",
       "      <td>3.0</td>\n",
       "      <td>1201.0000</td>\n",
       "      <td>12840.1010</td>\n",
       "      <td>788.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2402.0000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>555000.0</td>\n",
       "      <td>183424.484375</td>\n",
       "      <td>371575.515625</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2181</th>\n",
       "      <td>0.0</td>\n",
       "      <td>3.903065</td>\n",
       "      <td>31.517199</td>\n",
       "      <td>4.5</td>\n",
       "      <td>1558.6666</td>\n",
       "      <td>13022.7000</td>\n",
       "      <td>878.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>9352.0000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>184750.0</td>\n",
       "      <td>544274.875000</td>\n",
       "      <td>359524.875000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2666</th>\n",
       "      <td>114.0</td>\n",
       "      <td>4.013484</td>\n",
       "      <td>32.752815</td>\n",
       "      <td>2.5</td>\n",
       "      <td>902.0000</td>\n",
       "      <td>3099.5999</td>\n",
       "      <td>1107.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3124.6196</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>475000.0</td>\n",
       "      <td>165385.421875</td>\n",
       "      <td>309614.578125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1182</th>\n",
       "      <td>31.0</td>\n",
       "      <td>3.708188</td>\n",
       "      <td>20.123798</td>\n",
       "      <td>3.0</td>\n",
       "      <td>981.3333</td>\n",
       "      <td>3056.5498</td>\n",
       "      <td>584.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>5888.0000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>150000.0</td>\n",
       "      <td>382997.406250</td>\n",
       "      <td>232997.406250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1637</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4.176208</td>\n",
       "      <td>27.650692</td>\n",
       "      <td>3.5</td>\n",
       "      <td>584.5000</td>\n",
       "      <td>10507.0000</td>\n",
       "      <td>1559.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2338.0000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>591587.0</td>\n",
       "      <td>403816.093750</td>\n",
       "      <td>187770.906250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2737</th>\n",
       "      <td>71.0</td>\n",
       "      <td>4.128849</td>\n",
       "      <td>35.326355</td>\n",
       "      <td>3.5</td>\n",
       "      <td>734.4000</td>\n",
       "      <td>4773.5996</td>\n",
       "      <td>1411.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3672.0000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>415000.0</td>\n",
       "      <td>238224.296875</td>\n",
       "      <td>176775.703125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2570</th>\n",
       "      <td>88.0</td>\n",
       "      <td>3.017334</td>\n",
       "      <td>40.354954</td>\n",
       "      <td>3.5</td>\n",
       "      <td>778.0000</td>\n",
       "      <td>3535.9998</td>\n",
       "      <td>140.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>6958.6436</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>235000.0</td>\n",
       "      <td>409790.281250</td>\n",
       "      <td>174790.281250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1063</th>\n",
       "      <td>5.0</td>\n",
       "      <td>3.350989</td>\n",
       "      <td>30.710903</td>\n",
       "      <td>3.5</td>\n",
       "      <td>2470.0000</td>\n",
       "      <td>10520.2500</td>\n",
       "      <td>278.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>2470.0000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>615000.0</td>\n",
       "      <td>441111.156250</td>\n",
       "      <td>173888.843750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1767</th>\n",
       "      <td>13.0</td>\n",
       "      <td>3.959812</td>\n",
       "      <td>26.965897</td>\n",
       "      <td>4.0</td>\n",
       "      <td>1079.0000</td>\n",
       "      <td>10142.6010</td>\n",
       "      <td>989.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>4825.4346</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>755000.0</td>\n",
       "      <td>586041.062500</td>\n",
       "      <td>168958.937500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1782</th>\n",
       "      <td>1.0</td>\n",
       "      <td>4.243074</td>\n",
       "      <td>9.692594</td>\n",
       "      <td>2.0</td>\n",
       "      <td>897.5000</td>\n",
       "      <td>7090.2500</td>\n",
       "      <td>1795.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3590.0000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>147000.0</td>\n",
       "      <td>310691.781250</td>\n",
       "      <td>163691.781250</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>10 rows × 37 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        Age  BC_Bsmt_Unf_SF  BC_External_SF  Baths  Bedroom_Liv_Area_Ratio  \\\n",
       "423     1.0        3.851456       20.762966    3.0               1201.0000   \n",
       "2181    0.0        3.903065       31.517199    4.5               1558.6666   \n",
       "2666  114.0        4.013484       32.752815    2.5                902.0000   \n",
       "1182   31.0        3.708188       20.123798    3.0                981.3333   \n",
       "1637    1.0        4.176208       27.650692    3.5                584.5000   \n",
       "2737   71.0        4.128849       35.326355    3.5                734.4000   \n",
       "2570   88.0        3.017334       40.354954    3.5                778.0000   \n",
       "1063    5.0        3.350989       30.710903    3.5               2470.0000   \n",
       "1767   13.0        3.959812       26.965897    4.0               1079.0000   \n",
       "1782    1.0        4.243074        9.692594    2.0                897.5000   \n",
       "\n",
       "       Bsmt_Eval  Bsmt_Unf_SF  Central_Air  Exter_Qual  Fireplace_Eval  ...  \\\n",
       "423   12840.1010        788.0          0.0         3.0       2402.0000  ...   \n",
       "2181  13022.7000        878.0          0.0         3.0       9352.0000  ...   \n",
       "2666   3099.5999       1107.0          0.0         2.0       3124.6196  ...   \n",
       "1182   3056.5498        584.0          0.0         2.0       5888.0000  ...   \n",
       "1637  10507.0000       1559.0          0.0         3.0       2338.0000  ...   \n",
       "2737   4773.5996       1411.0          0.0         1.0       3672.0000  ...   \n",
       "2570   3535.9998        140.0          0.0         2.0       6958.6436  ...   \n",
       "1063  10520.2500        278.0          0.0         3.0       2470.0000  ...   \n",
       "1767  10142.6010        989.0          0.0         3.0       4825.4346  ...   \n",
       "1782   7090.2500       1795.0          0.0         2.0       3590.0000  ...   \n",
       "\n",
       "      bldg_type_3  garage_type_1  hs_style_1  neighborhoods_1  \\\n",
       "423           0.0            1.0         1.0              0.0   \n",
       "2181          0.0            1.0         1.0              1.0   \n",
       "2666          0.0            0.0         0.0              1.0   \n",
       "1182          0.0            1.0         1.0              0.0   \n",
       "1637          0.0            1.0         1.0              0.0   \n",
       "2737          0.0            0.0         0.0              1.0   \n",
       "2570          0.0            0.0         1.0              1.0   \n",
       "1063          0.0            1.0         1.0              0.0   \n",
       "1767          0.0            1.0         1.0              0.0   \n",
       "1782          0.0            1.0         1.0              0.0   \n",
       "\n",
       "      neighborhoods_3  neighborhoods_4  sale_cond_1  Sale_Price  \\\n",
       "423               0.0              0.0          1.0    555000.0   \n",
       "2181              0.0              0.0          0.0    184750.0   \n",
       "2666              0.0              0.0          1.0    475000.0   \n",
       "1182              0.0              1.0          0.0    150000.0   \n",
       "1637              0.0              0.0          0.0    591587.0   \n",
       "2737              0.0              0.0          1.0    415000.0   \n",
       "2570              0.0              0.0          0.0    235000.0   \n",
       "1063              0.0              0.0          1.0    615000.0   \n",
       "1767              0.0              0.0          1.0    755000.0   \n",
       "1782              1.0              0.0          0.0    147000.0   \n",
       "\n",
       "          Predicted  Prediction_Error  \n",
       "423   183424.484375     371575.515625  \n",
       "2181  544274.875000     359524.875000  \n",
       "2666  165385.421875     309614.578125  \n",
       "1182  382997.406250     232997.406250  \n",
       "1637  403816.093750     187770.906250  \n",
       "2737  238224.296875     176775.703125  \n",
       "2570  409790.281250     174790.281250  \n",
       "1063  441111.156250     173888.843750  \n",
       "1767  586041.062500     168958.937500  \n",
       "1782  310691.781250     163691.781250  \n",
       "\n",
       "[10 rows x 37 columns]"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df6.sort_values(by=['Prediction_Error', 'Sale_Price'], ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>BC_Bsmt_Unf_SF</th>\n",
       "      <th>BC_External_SF</th>\n",
       "      <th>Baths</th>\n",
       "      <th>Bedroom_Liv_Area_Ratio</th>\n",
       "      <th>Bsmt_Eval</th>\n",
       "      <th>Bsmt_Unf_SF</th>\n",
       "      <th>Central_Air</th>\n",
       "      <th>Exter_Qual</th>\n",
       "      <th>Fireplace_Eval</th>\n",
       "      <th>...</th>\n",
       "      <th>bldg_type_3</th>\n",
       "      <th>garage_type_1</th>\n",
       "      <th>hs_style_1</th>\n",
       "      <th>neighborhoods_1</th>\n",
       "      <th>neighborhoods_3</th>\n",
       "      <th>neighborhoods_4</th>\n",
       "      <th>sale_cond_1</th>\n",
       "      <th>Sale_Price</th>\n",
       "      <th>Predicted</th>\n",
       "      <th>Prediction_Error</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>26.220000</td>\n",
       "      <td>3.588896</td>\n",
       "      <td>23.185593</td>\n",
       "      <td>3.120000</td>\n",
       "      <td>895.531331</td>\n",
       "      <td>6969.808950</td>\n",
       "      <td>841.620000</td>\n",
       "      <td>0.040000</td>\n",
       "      <td>2.060000</td>\n",
       "      <td>3555.082086</td>\n",
       "      <td>...</td>\n",
       "      <td>0.020000</td>\n",
       "      <td>0.820000</td>\n",
       "      <td>0.86000</td>\n",
       "      <td>0.260000</td>\n",
       "      <td>0.080000</td>\n",
       "      <td>0.260000</td>\n",
       "      <td>0.460000</td>\n",
       "      <td>347418.620000</td>\n",
       "      <td>334127.271875</td>\n",
       "      <td>133319.100000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>34.896518</td>\n",
       "      <td>0.859469</td>\n",
       "      <td>10.053021</td>\n",
       "      <td>0.759699</td>\n",
       "      <td>428.043946</td>\n",
       "      <td>3867.733648</td>\n",
       "      <td>627.092542</td>\n",
       "      <td>0.197949</td>\n",
       "      <td>0.766918</td>\n",
       "      <td>1917.580983</td>\n",
       "      <td>...</td>\n",
       "      <td>0.141421</td>\n",
       "      <td>0.388088</td>\n",
       "      <td>0.35051</td>\n",
       "      <td>0.443087</td>\n",
       "      <td>0.274048</td>\n",
       "      <td>0.443087</td>\n",
       "      <td>0.503457</td>\n",
       "      <td>173751.304594</td>\n",
       "      <td>124464.890199</td>\n",
       "      <td>64426.147216</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>288.000000</td>\n",
       "      <td>1556.099900</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>82500.000000</td>\n",
       "      <td>131277.468750</td>\n",
       "      <td>78640.968750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.362793</td>\n",
       "      <td>18.682363</td>\n",
       "      <td>2.500000</td>\n",
       "      <td>654.062475</td>\n",
       "      <td>4812.955950</td>\n",
       "      <td>285.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.250000</td>\n",
       "      <td>2385.500000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>221500.000000</td>\n",
       "      <td>241259.015625</td>\n",
       "      <td>93618.312500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>6.000000</td>\n",
       "      <td>3.780512</td>\n",
       "      <td>23.468677</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>768.500000</td>\n",
       "      <td>6365.600000</td>\n",
       "      <td>679.500000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>2.000000</td>\n",
       "      <td>3281.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>315750.000000</td>\n",
       "      <td>316755.671875</td>\n",
       "      <td>114122.218750</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>41.750000</td>\n",
       "      <td>4.128764</td>\n",
       "      <td>28.597953</td>\n",
       "      <td>3.500000</td>\n",
       "      <td>986.416675</td>\n",
       "      <td>8164.300000</td>\n",
       "      <td>1410.750000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>4586.651950</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.750000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.750000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>441446.750000</td>\n",
       "      <td>408296.734375</td>\n",
       "      <td>144971.804688</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>136.000000</td>\n",
       "      <td>4.329243</td>\n",
       "      <td>47.315372</td>\n",
       "      <td>4.500000</td>\n",
       "      <td>2470.000000</td>\n",
       "      <td>25356.500000</td>\n",
       "      <td>2153.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>3.000000</td>\n",
       "      <td>9352.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>755000.000000</td>\n",
       "      <td>611544.937500</td>\n",
       "      <td>371575.515625</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 37 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "              Age  BC_Bsmt_Unf_SF  BC_External_SF      Baths  \\\n",
       "count   50.000000       50.000000       50.000000  50.000000   \n",
       "mean    26.220000        3.588896       23.185593   3.120000   \n",
       "std     34.896518        0.859469       10.053021   0.759699   \n",
       "min      0.000000        0.000000        0.000000   2.000000   \n",
       "25%      1.000000        3.362793       18.682363   2.500000   \n",
       "50%      6.000000        3.780512       23.468677   3.000000   \n",
       "75%     41.750000        4.128764       28.597953   3.500000   \n",
       "max    136.000000        4.329243       47.315372   4.500000   \n",
       "\n",
       "       Bedroom_Liv_Area_Ratio     Bsmt_Eval  Bsmt_Unf_SF  Central_Air  \\\n",
       "count               50.000000     50.000000    50.000000    50.000000   \n",
       "mean               895.531331   6969.808950   841.620000     0.040000   \n",
       "std                428.043946   3867.733648   627.092542     0.197949   \n",
       "min                288.000000   1556.099900     0.000000     0.000000   \n",
       "25%                654.062475   4812.955950   285.000000     0.000000   \n",
       "50%                768.500000   6365.600000   679.500000     0.000000   \n",
       "75%                986.416675   8164.300000  1410.750000     0.000000   \n",
       "max               2470.000000  25356.500000  2153.000000     1.000000   \n",
       "\n",
       "       Exter_Qual  Fireplace_Eval  ...  bldg_type_3  garage_type_1  \\\n",
       "count   50.000000       50.000000  ...    50.000000      50.000000   \n",
       "mean     2.060000     3555.082086  ...     0.020000       0.820000   \n",
       "std      0.766918     1917.580983  ...     0.141421       0.388088   \n",
       "min      1.000000        0.000000  ...     0.000000       0.000000   \n",
       "25%      1.250000     2385.500000  ...     0.000000       1.000000   \n",
       "50%      2.000000     3281.000000  ...     0.000000       1.000000   \n",
       "75%      3.000000     4586.651950  ...     0.000000       1.000000   \n",
       "max      3.000000     9352.000000  ...     1.000000       1.000000   \n",
       "\n",
       "       hs_style_1  neighborhoods_1  neighborhoods_3  neighborhoods_4  \\\n",
       "count    50.00000        50.000000        50.000000        50.000000   \n",
       "mean      0.86000         0.260000         0.080000         0.260000   \n",
       "std       0.35051         0.443087         0.274048         0.443087   \n",
       "min       0.00000         0.000000         0.000000         0.000000   \n",
       "25%       1.00000         0.000000         0.000000         0.000000   \n",
       "50%       1.00000         0.000000         0.000000         0.000000   \n",
       "75%       1.00000         0.750000         0.000000         0.750000   \n",
       "max       1.00000         1.000000         1.000000         1.000000   \n",
       "\n",
       "       sale_cond_1     Sale_Price      Predicted  Prediction_Error  \n",
       "count    50.000000      50.000000      50.000000         50.000000  \n",
       "mean      0.460000  347418.620000  334127.271875     133319.100000  \n",
       "std       0.503457  173751.304594  124464.890199      64426.147216  \n",
       "min       0.000000   82500.000000  131277.468750      78640.968750  \n",
       "25%       0.000000  221500.000000  241259.015625      93618.312500  \n",
       "50%       0.000000  315750.000000  316755.671875     114122.218750  \n",
       "75%       1.000000  441446.750000  408296.734375     144971.804688  \n",
       "max       1.000000  755000.000000  611544.937500     371575.515625  \n",
       "\n",
       "[8 rows x 37 columns]"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df6.sort_values(by=['Prediction_Error', 'Sale_Price'], ascending=False).head(50).describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Best on df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>Alley_Access</th>\n",
       "      <th>Baths</th>\n",
       "      <th>Bedroom_AbvGr</th>\n",
       "      <th>Bedroom_Liv_Area_Ratio</th>\n",
       "      <th>Bsmt</th>\n",
       "      <th>Bsmt_Eval</th>\n",
       "      <th>Bsmt_Unf_SF</th>\n",
       "      <th>Central_Air</th>\n",
       "      <th>Electrical_SBrkr</th>\n",
       "      <th>...</th>\n",
       "      <th>TotBath_LivArea_Ratio</th>\n",
       "      <th>TotRms_AbvGrd</th>\n",
       "      <th>Total_Bsmt_SF</th>\n",
       "      <th>Total_SF</th>\n",
       "      <th>Year_Sold</th>\n",
       "      <th>neighborhoods_1</th>\n",
       "      <th>neighborhoods_5</th>\n",
       "      <th>Sale_Price</th>\n",
       "      <th>Predicted</th>\n",
       "      <th>Prediction_Error</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>2464</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>517.66670</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2324.7000</td>\n",
       "      <td>756.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>621.2000</td>\n",
       "      <td>6.0</td>\n",
       "      <td>756.0</td>\n",
       "      <td>2309.0</td>\n",
       "      <td>2006.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>186500.0</td>\n",
       "      <td>186499.171875</td>\n",
       "      <td>0.828125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>803</th>\n",
       "      <td>16.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>591.66670</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2275.4998</td>\n",
       "      <td>227.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>710.0000</td>\n",
       "      <td>7.0</td>\n",
       "      <td>740.0</td>\n",
       "      <td>2515.0</td>\n",
       "      <td>2009.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>213000.0</td>\n",
       "      <td>212992.625000</td>\n",
       "      <td>7.375000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2050</th>\n",
       "      <td>43.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>335.00000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2613.0000</td>\n",
       "      <td>348.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1005.0000</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1005.0</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>2007.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>115400.0</td>\n",
       "      <td>115421.437500</td>\n",
       "      <td>21.437500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2018</th>\n",
       "      <td>78.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>427.00000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2163.2000</td>\n",
       "      <td>832.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>854.0000</td>\n",
       "      <td>5.0</td>\n",
       "      <td>832.0</td>\n",
       "      <td>1686.0</td>\n",
       "      <td>2007.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>132000.0</td>\n",
       "      <td>132021.625000</td>\n",
       "      <td>21.625000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>327</th>\n",
       "      <td>37.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.5</td>\n",
       "      <td>2.0</td>\n",
       "      <td>472.50000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3472.8750</td>\n",
       "      <td>30.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>945.0000</td>\n",
       "      <td>5.0</td>\n",
       "      <td>945.0</td>\n",
       "      <td>1890.0</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>119500.0</td>\n",
       "      <td>119532.078125</td>\n",
       "      <td>32.078125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1497</th>\n",
       "      <td>49.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.5</td>\n",
       "      <td>5.0</td>\n",
       "      <td>764.00000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1091.4286</td>\n",
       "      <td>11.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3820.0</td>\n",
       "      <td>2008.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>284700.0</td>\n",
       "      <td>284664.875000</td>\n",
       "      <td>35.125000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2057</th>\n",
       "      <td>36.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>434.00000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2668.7998</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>868.0000</td>\n",
       "      <td>6.0</td>\n",
       "      <td>768.0</td>\n",
       "      <td>1636.0</td>\n",
       "      <td>2007.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>119900.0</td>\n",
       "      <td>119847.093750</td>\n",
       "      <td>52.906250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2072</th>\n",
       "      <td>23.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>507.33334</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3659.2498</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>761.0000</td>\n",
       "      <td>7.0</td>\n",
       "      <td>1190.0</td>\n",
       "      <td>2712.0</td>\n",
       "      <td>2007.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>182000.0</td>\n",
       "      <td>182053.062500</td>\n",
       "      <td>53.062500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2720</th>\n",
       "      <td>49.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>601.00000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2948.4000</td>\n",
       "      <td>284.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>901.5000</td>\n",
       "      <td>8.0</td>\n",
       "      <td>1134.0</td>\n",
       "      <td>2937.0</td>\n",
       "      <td>2006.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>155000.0</td>\n",
       "      <td>154946.109375</td>\n",
       "      <td>53.890625</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1623</th>\n",
       "      <td>11.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>517.66670</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2124.8250</td>\n",
       "      <td>277.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>621.2000</td>\n",
       "      <td>6.0</td>\n",
       "      <td>691.0</td>\n",
       "      <td>2244.0</td>\n",
       "      <td>2007.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>178750.0</td>\n",
       "      <td>178693.562500</td>\n",
       "      <td>56.437500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2758</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>578.00000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>6025.6500</td>\n",
       "      <td>434.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>867.0000</td>\n",
       "      <td>7.0</td>\n",
       "      <td>1734.0</td>\n",
       "      <td>3468.0</td>\n",
       "      <td>2006.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>260000.0</td>\n",
       "      <td>259940.953125</td>\n",
       "      <td>59.046875</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1097</th>\n",
       "      <td>9.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>550.33330</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2555.3250</td>\n",
       "      <td>831.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>660.4000</td>\n",
       "      <td>7.0</td>\n",
       "      <td>831.0</td>\n",
       "      <td>2482.0</td>\n",
       "      <td>2008.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>185900.0</td>\n",
       "      <td>185834.921875</td>\n",
       "      <td>65.078125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>721</th>\n",
       "      <td>89.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>494.00000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1689.9999</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1482.0000</td>\n",
       "      <td>7.0</td>\n",
       "      <td>650.0</td>\n",
       "      <td>2132.0</td>\n",
       "      <td>2009.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>143000.0</td>\n",
       "      <td>143081.703125</td>\n",
       "      <td>81.703125</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1133</th>\n",
       "      <td>5.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>554.33330</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2901.6250</td>\n",
       "      <td>835.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>665.2000</td>\n",
       "      <td>8.0</td>\n",
       "      <td>835.0</td>\n",
       "      <td>2498.0</td>\n",
       "      <td>2008.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>192500.0</td>\n",
       "      <td>192417.843750</td>\n",
       "      <td>82.156250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>775</th>\n",
       "      <td>27.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>430.00000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2520.0000</td>\n",
       "      <td>168.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>860.0000</td>\n",
       "      <td>4.0</td>\n",
       "      <td>840.0</td>\n",
       "      <td>1700.0</td>\n",
       "      <td>2009.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>137000.0</td>\n",
       "      <td>137086.734375</td>\n",
       "      <td>86.734375</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1037</th>\n",
       "      <td>36.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>288.00000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2246.4000</td>\n",
       "      <td>264.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>864.0000</td>\n",
       "      <td>5.0</td>\n",
       "      <td>864.0</td>\n",
       "      <td>1728.0</td>\n",
       "      <td>2008.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>111250.0</td>\n",
       "      <td>111346.359375</td>\n",
       "      <td>96.359375</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>283</th>\n",
       "      <td>69.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>462.00000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2066.4000</td>\n",
       "      <td>324.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>...</td>\n",
       "      <td>924.0000</td>\n",
       "      <td>5.0</td>\n",
       "      <td>672.0</td>\n",
       "      <td>1596.0</td>\n",
       "      <td>2010.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>80000.0</td>\n",
       "      <td>80097.570312</td>\n",
       "      <td>97.570312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2061</th>\n",
       "      <td>39.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>298.00000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2749.0498</td>\n",
       "      <td>402.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>894.0000</td>\n",
       "      <td>5.0</td>\n",
       "      <td>894.0</td>\n",
       "      <td>1788.0</td>\n",
       "      <td>2007.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>129000.0</td>\n",
       "      <td>128890.210938</td>\n",
       "      <td>109.789062</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>362</th>\n",
       "      <td>17.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>389.00000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>3560.8499</td>\n",
       "      <td>598.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>583.5000</td>\n",
       "      <td>6.0</td>\n",
       "      <td>1158.0</td>\n",
       "      <td>2325.0</td>\n",
       "      <td>2009.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>173000.0</td>\n",
       "      <td>172888.843750</td>\n",
       "      <td>111.156250</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>769</th>\n",
       "      <td>46.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>373.66666</td>\n",
       "      <td>1.0</td>\n",
       "      <td>2914.5999</td>\n",
       "      <td>470.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>...</td>\n",
       "      <td>1121.0000</td>\n",
       "      <td>5.0</td>\n",
       "      <td>1121.0</td>\n",
       "      <td>2242.0</td>\n",
       "      <td>2009.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>139000.0</td>\n",
       "      <td>139113.578125</td>\n",
       "      <td>113.578125</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>20 rows × 38 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       Age  Alley_Access  Baths  Bedroom_AbvGr  Bedroom_Liv_Area_Ratio  Bsmt  \\\n",
       "2464   1.0           0.0    2.5            3.0               517.66670   1.0   \n",
       "803   16.0           0.0    3.5            3.0               591.66670   1.0   \n",
       "2050  43.0           0.0    2.0            3.0               335.00000   1.0   \n",
       "2018  78.0           0.0    1.0            2.0               427.00000   1.0   \n",
       "327   37.0           0.0    2.5            2.0               472.50000   1.0   \n",
       "1497  49.0           0.0    3.5            5.0               764.00000   0.0   \n",
       "2057  36.0           0.0    2.0            2.0               434.00000   1.0   \n",
       "2072  23.0           0.0    2.0            3.0               507.33334   1.0   \n",
       "2720  49.0           0.0    3.0            3.0               601.00000   1.0   \n",
       "1623  11.0           0.0    2.5            3.0               517.66670   1.0   \n",
       "2758   1.0           0.0    3.0            3.0               578.00000   1.0   \n",
       "1097   9.0           0.0    2.5            3.0               550.33330   1.0   \n",
       "721   89.0           0.0    1.5            3.0               494.00000   1.0   \n",
       "1133   5.0           0.0    2.5            3.0               554.33330   1.0   \n",
       "775   27.0           0.0    2.0            2.0               430.00000   1.0   \n",
       "1037  36.0           0.0    1.0            3.0               288.00000   1.0   \n",
       "283   69.0           0.0    2.0            2.0               462.00000   1.0   \n",
       "2061  39.0           0.0    1.0            3.0               298.00000   1.0   \n",
       "362   17.0           0.0    3.0            3.0               389.00000   1.0   \n",
       "769   46.0           0.0    2.0            3.0               373.66666   1.0   \n",
       "\n",
       "      Bsmt_Eval  Bsmt_Unf_SF  Central_Air  Electrical_SBrkr  ...  \\\n",
       "2464  2324.7000        756.0          0.0               1.0  ...   \n",
       "803   2275.4998        227.0          0.0               1.0  ...   \n",
       "2050  2613.0000        348.0          0.0               1.0  ...   \n",
       "2018  2163.2000        832.0          0.0               0.0  ...   \n",
       "327   3472.8750         30.0          0.0               1.0  ...   \n",
       "1497     0.0000          0.0          0.0               1.0  ...   \n",
       "2057  2668.7998         20.0          0.0               1.0  ...   \n",
       "2072  3659.2498          0.0          0.0               1.0  ...   \n",
       "2720  2948.4000        284.0          0.0               1.0  ...   \n",
       "1623  2124.8250        277.0          0.0               1.0  ...   \n",
       "2758  6025.6500        434.0          0.0               1.0  ...   \n",
       "1097  2555.3250        831.0          0.0               1.0  ...   \n",
       "721   1689.9999         26.0          0.0               1.0  ...   \n",
       "1133  2901.6250        835.0          0.0               1.0  ...   \n",
       "775   2520.0000        168.0          0.0               1.0  ...   \n",
       "1037  2246.4000        264.0          0.0               1.0  ...   \n",
       "283   2066.4000        324.0          1.0               0.0  ...   \n",
       "2061  2749.0498        402.0          0.0               1.0  ...   \n",
       "362   3560.8499        598.0          0.0               1.0  ...   \n",
       "769   2914.5999        470.0          0.0               1.0  ...   \n",
       "\n",
       "      TotBath_LivArea_Ratio  TotRms_AbvGrd  Total_Bsmt_SF  Total_SF  \\\n",
       "2464               621.2000            6.0          756.0    2309.0   \n",
       "803                710.0000            7.0          740.0    2515.0   \n",
       "2050              1005.0000            5.0         1005.0    2010.0   \n",
       "2018               854.0000            5.0          832.0    1686.0   \n",
       "327                945.0000            5.0          945.0    1890.0   \n",
       "1497              1091.4286           11.0            0.0    3820.0   \n",
       "2057               868.0000            6.0          768.0    1636.0   \n",
       "2072               761.0000            7.0         1190.0    2712.0   \n",
       "2720               901.5000            8.0         1134.0    2937.0   \n",
       "1623               621.2000            6.0          691.0    2244.0   \n",
       "2758               867.0000            7.0         1734.0    3468.0   \n",
       "1097               660.4000            7.0          831.0    2482.0   \n",
       "721               1482.0000            7.0          650.0    2132.0   \n",
       "1133               665.2000            8.0          835.0    2498.0   \n",
       "775                860.0000            4.0          840.0    1700.0   \n",
       "1037               864.0000            5.0          864.0    1728.0   \n",
       "283                924.0000            5.0          672.0    1596.0   \n",
       "2061               894.0000            5.0          894.0    1788.0   \n",
       "362                583.5000            6.0         1158.0    2325.0   \n",
       "769               1121.0000            5.0         1121.0    2242.0   \n",
       "\n",
       "      Year_Sold  neighborhoods_1  neighborhoods_5  Sale_Price      Predicted  \\\n",
       "2464     2006.0              0.0              0.0    186500.0  186499.171875   \n",
       "803      2009.0              1.0              0.0    213000.0  212992.625000   \n",
       "2050     2007.0              1.0              0.0    115400.0  115421.437500   \n",
       "2018     2007.0              1.0              0.0    132000.0  132021.625000   \n",
       "327      2010.0              0.0              0.0    119500.0  119532.078125   \n",
       "1497     2008.0              1.0              0.0    284700.0  284664.875000   \n",
       "2057     2007.0              1.0              0.0    119900.0  119847.093750   \n",
       "2072     2007.0              0.0              0.0    182000.0  182053.062500   \n",
       "2720     2006.0              1.0              0.0    155000.0  154946.109375   \n",
       "1623     2007.0              0.0              0.0    178750.0  178693.562500   \n",
       "2758     2006.0              0.0              0.0    260000.0  259940.953125   \n",
       "1097     2008.0              0.0              0.0    185900.0  185834.921875   \n",
       "721      2009.0              1.0              0.0    143000.0  143081.703125   \n",
       "1133     2008.0              1.0              0.0    192500.0  192417.843750   \n",
       "775      2009.0              1.0              0.0    137000.0  137086.734375   \n",
       "1037     2008.0              1.0              0.0    111250.0  111346.359375   \n",
       "283      2010.0              1.0              0.0     80000.0   80097.570312   \n",
       "2061     2007.0              1.0              0.0    129000.0  128890.210938   \n",
       "362      2009.0              0.0              0.0    173000.0  172888.843750   \n",
       "769      2009.0              1.0              0.0    139000.0  139113.578125   \n",
       "\n",
       "      Prediction_Error  \n",
       "2464          0.828125  \n",
       "803           7.375000  \n",
       "2050         21.437500  \n",
       "2018         21.625000  \n",
       "327          32.078125  \n",
       "1497         35.125000  \n",
       "2057         52.906250  \n",
       "2072         53.062500  \n",
       "2720         53.890625  \n",
       "1623         56.437500  \n",
       "2758         59.046875  \n",
       "1097         65.078125  \n",
       "721          81.703125  \n",
       "1133         82.156250  \n",
       "775          86.734375  \n",
       "1037         96.359375  \n",
       "283          97.570312  \n",
       "2061        109.789062  \n",
       "362         111.156250  \n",
       "769         113.578125  \n",
       "\n",
       "[20 rows x 38 columns]"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df6.sort_values(by=['Prediction_Error', 'Sale_Price']).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Age</th>\n",
       "      <th>BC_Bsmt_Unf_SF</th>\n",
       "      <th>BC_External_SF</th>\n",
       "      <th>Baths</th>\n",
       "      <th>Bedroom_Liv_Area_Ratio</th>\n",
       "      <th>Bsmt_Eval</th>\n",
       "      <th>Bsmt_Unf_SF</th>\n",
       "      <th>Central_Air</th>\n",
       "      <th>Exter_Qual</th>\n",
       "      <th>Fireplace_Eval</th>\n",
       "      <th>...</th>\n",
       "      <th>bldg_type_3</th>\n",
       "      <th>garage_type_1</th>\n",
       "      <th>hs_style_1</th>\n",
       "      <th>neighborhoods_1</th>\n",
       "      <th>neighborhoods_3</th>\n",
       "      <th>neighborhoods_4</th>\n",
       "      <th>sale_cond_1</th>\n",
       "      <th>Sale_Price</th>\n",
       "      <th>Predicted</th>\n",
       "      <th>Prediction_Error</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>50.00000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.0</td>\n",
       "      <td>50.00000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "      <td>50.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>32.02000</td>\n",
       "      <td>3.257831</td>\n",
       "      <td>13.188114</td>\n",
       "      <td>2.21000</td>\n",
       "      <td>528.880000</td>\n",
       "      <td>3461.347952</td>\n",
       "      <td>575.980000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.48000</td>\n",
       "      <td>1341.020232</td>\n",
       "      <td>...</td>\n",
       "      <td>0.160000</td>\n",
       "      <td>0.720000</td>\n",
       "      <td>0.820000</td>\n",
       "      <td>0.520000</td>\n",
       "      <td>0.160000</td>\n",
       "      <td>0.200000</td>\n",
       "      <td>0.760000</td>\n",
       "      <td>176000.680000</td>\n",
       "      <td>175951.847969</td>\n",
       "      <td>162.118906</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>27.03021</td>\n",
       "      <td>1.072057</td>\n",
       "      <td>8.851206</td>\n",
       "      <td>0.70051</td>\n",
       "      <td>275.046371</td>\n",
       "      <td>1381.489824</td>\n",
       "      <td>542.450731</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.67733</td>\n",
       "      <td>1572.078932</td>\n",
       "      <td>...</td>\n",
       "      <td>0.370328</td>\n",
       "      <td>0.453557</td>\n",
       "      <td>0.388088</td>\n",
       "      <td>0.504672</td>\n",
       "      <td>0.370328</td>\n",
       "      <td>0.404061</td>\n",
       "      <td>0.431419</td>\n",
       "      <td>72205.008434</td>\n",
       "      <td>72187.339161</td>\n",
       "      <td>87.100631</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>245.000000</td>\n",
       "      <td>1478.400000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>86000.000000</td>\n",
       "      <td>85901.390625</td>\n",
       "      <td>4.562500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>4.00000</td>\n",
       "      <td>3.104215</td>\n",
       "      <td>7.308492</td>\n",
       "      <td>2.00000</td>\n",
       "      <td>385.500000</td>\n",
       "      <td>2484.950000</td>\n",
       "      <td>167.250000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>132625.000000</td>\n",
       "      <td>132509.878906</td>\n",
       "      <td>100.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>31.00000</td>\n",
       "      <td>3.511199</td>\n",
       "      <td>15.247634</td>\n",
       "      <td>2.00000</td>\n",
       "      <td>489.291670</td>\n",
       "      <td>3079.200000</td>\n",
       "      <td>387.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.00000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>145750.000000</td>\n",
       "      <td>145707.359375</td>\n",
       "      <td>157.265625</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>48.75000</td>\n",
       "      <td>3.883149</td>\n",
       "      <td>19.417033</td>\n",
       "      <td>2.50000</td>\n",
       "      <td>600.062500</td>\n",
       "      <td>4197.762200</td>\n",
       "      <td>844.500000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>2.00000</td>\n",
       "      <td>2685.000000</td>\n",
       "      <td>...</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>187375.000000</td>\n",
       "      <td>187250.054688</td>\n",
       "      <td>219.585938</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>93.00000</td>\n",
       "      <td>4.322152</td>\n",
       "      <td>28.365940</td>\n",
       "      <td>4.00000</td>\n",
       "      <td>2132.000000</td>\n",
       "      <td>8802.150000</td>\n",
       "      <td>2121.000000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>3.00000</td>\n",
       "      <td>4742.700000</td>\n",
       "      <td>...</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>415298.000000</td>\n",
       "      <td>415195.656250</td>\n",
       "      <td>334.218750</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>8 rows × 37 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "            Age  BC_Bsmt_Unf_SF  BC_External_SF     Baths  \\\n",
       "count  50.00000       50.000000       50.000000  50.00000   \n",
       "mean   32.02000        3.257831       13.188114   2.21000   \n",
       "std    27.03021        1.072057        8.851206   0.70051   \n",
       "min     0.00000        0.000000        0.000000   1.00000   \n",
       "25%     4.00000        3.104215        7.308492   2.00000   \n",
       "50%    31.00000        3.511199       15.247634   2.00000   \n",
       "75%    48.75000        3.883149       19.417033   2.50000   \n",
       "max    93.00000        4.322152       28.365940   4.00000   \n",
       "\n",
       "       Bedroom_Liv_Area_Ratio    Bsmt_Eval  Bsmt_Unf_SF  Central_Air  \\\n",
       "count               50.000000    50.000000    50.000000         50.0   \n",
       "mean               528.880000  3461.347952   575.980000          0.0   \n",
       "std                275.046371  1381.489824   542.450731          0.0   \n",
       "min                245.000000  1478.400000     0.000000          0.0   \n",
       "25%                385.500000  2484.950000   167.250000          0.0   \n",
       "50%                489.291670  3079.200000   387.500000          0.0   \n",
       "75%                600.062500  4197.762200   844.500000          0.0   \n",
       "max               2132.000000  8802.150000  2121.000000          0.0   \n",
       "\n",
       "       Exter_Qual  Fireplace_Eval  ...  bldg_type_3  garage_type_1  \\\n",
       "count    50.00000       50.000000  ...    50.000000      50.000000   \n",
       "mean      1.48000     1341.020232  ...     0.160000       0.720000   \n",
       "std       0.67733     1572.078932  ...     0.370328       0.453557   \n",
       "min       1.00000        0.000000  ...     0.000000       0.000000   \n",
       "25%       1.00000        0.000000  ...     0.000000       0.000000   \n",
       "50%       1.00000        0.000000  ...     0.000000       1.000000   \n",
       "75%       2.00000     2685.000000  ...     0.000000       1.000000   \n",
       "max       3.00000     4742.700000  ...     1.000000       1.000000   \n",
       "\n",
       "       hs_style_1  neighborhoods_1  neighborhoods_3  neighborhoods_4  \\\n",
       "count   50.000000        50.000000        50.000000        50.000000   \n",
       "mean     0.820000         0.520000         0.160000         0.200000   \n",
       "std      0.388088         0.504672         0.370328         0.404061   \n",
       "min      0.000000         0.000000         0.000000         0.000000   \n",
       "25%      1.000000         0.000000         0.000000         0.000000   \n",
       "50%      1.000000         1.000000         0.000000         0.000000   \n",
       "75%      1.000000         1.000000         0.000000         0.000000   \n",
       "max      1.000000         1.000000         1.000000         1.000000   \n",
       "\n",
       "       sale_cond_1     Sale_Price      Predicted  Prediction_Error  \n",
       "count    50.000000      50.000000      50.000000         50.000000  \n",
       "mean      0.760000  176000.680000  175951.847969        162.118906  \n",
       "std       0.431419   72205.008434   72187.339161         87.100631  \n",
       "min       0.000000   86000.000000   85901.390625          4.562500  \n",
       "25%       1.000000  132625.000000  132509.878906        100.000000  \n",
       "50%       1.000000  145750.000000  145707.359375        157.265625  \n",
       "75%       1.000000  187375.000000  187250.054688        219.585938  \n",
       "max       1.000000  415298.000000  415195.656250        334.218750  \n",
       "\n",
       "[8 rows x 37 columns]"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_df6.sort_values(by=['Prediction_Error', 'Sale_Price']).head(50).describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---\n",
    "\n",
    "### Final Comment"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The XGBoost Regressor tested present a lot of variance in their results, which is hard to account for.\n",
    "\n",
    "The most probable cause are the number of estimators."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}