06.03-Integer-Encoding.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Integer Encoding\n",
    "\n",
    "Integer encoding consist in replacing the categories by digits from 1 to n (or 0 to n-1, depending the implementation), where n is the number of distinct categories of the variable.\n",
    "\n",
    "The numbers are assigned arbitrarily. This encoding method allows for quick benchmarking of machine learning models. \n",
    "\n",
    "\n",
    "### Advantages\n",
    "\n",
    "- Straightforward to implement\n",
    "- Does not expand the feature space\n",
    "\n",
    "\n",
    "### Limitations\n",
    "\n",
    "- Does not capture any information about the categories labels\n",
    "- Not suitable for linear models.\n",
    "\n",
    "Integer encoding is better suited for non-linear methods which are able to navigate through the arbitrarily assigned digits to try and find patters that relate them to the target.\n",
    "\n",
    "\n",
    "## In this demo:\n",
    "\n",
    "We will see how to perform one hot encoding with:\n",
    "- pandas\n",
    "- Scikit-learn\n",
    "- Feature-Engine\n",
    "\n",
    "And the advantages and limitations of each implementation using the House Prices dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "# to split the datasets\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "# for integer encoding using sklearn\n",
    "from sklearn.preprocessing import LabelEncoder\n",
    "\n",
    "# for integer encoding using feature-engine\n",
    "from feature_engine.categorical_encoders import OrdinalCategoricalEncoder"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Neighborhood</th>\n",
       "      <th>Exterior1st</th>\n",
       "      <th>Exterior2nd</th>\n",
       "      <th>SalePrice</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>CollgCr</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>208500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Veenker</td>\n",
       "      <td>MetalSd</td>\n",
       "      <td>MetalSd</td>\n",
       "      <td>181500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>CollgCr</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>223500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Crawfor</td>\n",
       "      <td>Wd Sdng</td>\n",
       "      <td>Wd Shng</td>\n",
       "      <td>140000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>NoRidge</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>250000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  Neighborhood Exterior1st Exterior2nd  SalePrice\n",
       "0      CollgCr     VinylSd     VinylSd     208500\n",
       "1      Veenker     MetalSd     MetalSd     181500\n",
       "2      CollgCr     VinylSd     VinylSd     223500\n",
       "3      Crawfor     Wd Sdng     Wd Shng     140000\n",
       "4      NoRidge     VinylSd     VinylSd     250000"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# load dataset\n",
    "\n",
    "data = pd.read_csv(\n",
    "    '../houseprice.csv',\n",
    "    usecols=['Neighborhood', 'Exterior1st', 'Exterior2nd', 'SalePrice'])\n",
    "\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Neighborhood :  25  labels\n",
      "Exterior1st :  15  labels\n",
      "Exterior2nd :  16  labels\n",
      "SalePrice :  663  labels\n"
     ]
    }
   ],
   "source": [
    "# let's have a look at how many labels each variable has\n",
    "\n",
    "for col in data.columns:\n",
    "    print(col, ': ', len(data[col].unique()), ' labels')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['CollgCr', 'Veenker', 'Crawfor', 'NoRidge', 'Mitchel', 'Somerst',\n",
       "       'NWAmes', 'OldTown', 'BrkSide', 'Sawyer', 'NridgHt', 'NAmes',\n",
       "       'SawyerW', 'IDOTRR', 'MeadowV', 'Edwards', 'Timber', 'Gilbert',\n",
       "       'StoneBr', 'ClearCr', 'NPkVill', 'Blmngtn', 'BrDale', 'SWISU',\n",
       "       'Blueste'], dtype=object)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's explore the unique categories\n",
    "data['Neighborhood'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['VinylSd', 'MetalSd', 'Wd Sdng', 'HdBoard', 'BrkFace', 'WdShing',\n",
       "       'CemntBd', 'Plywood', 'AsbShng', 'Stucco', 'BrkComm', 'AsphShn',\n",
       "       'Stone', 'ImStucc', 'CBlock'], dtype=object)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['Exterior1st'].unique()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['VinylSd', 'MetalSd', 'Wd Shng', 'HdBoard', 'Plywood', 'Wd Sdng',\n",
       "       'CmentBd', 'BrkFace', 'Stucco', 'AsbShng', 'Brk Cmn', 'ImStucc',\n",
       "       'AsphShn', 'Stone', 'Other', 'CBlock'], dtype=object)"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data['Exterior2nd'].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Encoding important\n",
    "\n",
    "We select which digit to assign to each category using the train set, and then use those mappings in the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1022, 3), (438, 3))"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's separate into training and testing set\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    data[['Neighborhood', 'Exterior1st', 'Exterior2nd']], # predictors\n",
    "    data['SalePrice'],  # target\n",
    "    test_size=0.3,  # percentage of obs in test set\n",
    "    random_state=0)  # seed to ensure reproducibility\n",
    "\n",
    "X_train.shape, X_test.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Integer encoding with pandas\n",
    "\n",
    "\n",
    "### Advantages\n",
    "\n",
    "- quick\n",
    "- returns pandas dataframe\n",
    "\n",
    "### Limitations of pandas:\n",
    "\n",
    "- it does not preserve information from train data to propagate to test data\n",
    "\n",
    "We need to capture and save the mappings one by one, manually, if we are planing to use those in production."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'CollgCr': 0,\n",
       " 'ClearCr': 1,\n",
       " 'BrkSide': 2,\n",
       " 'Edwards': 3,\n",
       " 'SWISU': 4,\n",
       " 'Sawyer': 5,\n",
       " 'Crawfor': 6,\n",
       " 'NAmes': 7,\n",
       " 'Mitchel': 8,\n",
       " 'Timber': 9,\n",
       " 'Gilbert': 10,\n",
       " 'Somerst': 11,\n",
       " 'MeadowV': 12,\n",
       " 'OldTown': 13,\n",
       " 'BrDale': 14,\n",
       " 'NWAmes': 15,\n",
       " 'NridgHt': 16,\n",
       " 'SawyerW': 17,\n",
       " 'NoRidge': 18,\n",
       " 'IDOTRR': 19,\n",
       " 'NPkVill': 20,\n",
       " 'StoneBr': 21,\n",
       " 'Blmngtn': 22,\n",
       " 'Veenker': 23,\n",
       " 'Blueste': 24}"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# first let's create a dictionary with the mappings of categories to numbers\n",
    "\n",
    "ordinal_mapping = {\n",
    "    k: i\n",
    "    for i, k in enumerate(X_train['Neighborhood'].unique(), 0)\n",
    "}\n",
    "\n",
    "ordinal_mapping"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dictionary indicates which number will replace each category. Numbers were assigned arbitrarily from 0 to n - 1 where n is the number of distinct categories."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# replace the labels with the integers\n",
    "\n",
    "X_train['Neighborhood'] = X_train['Neighborhood'].map(ordinal_mapping)\n",
    "X_test['Neighborhood'] = X_test['Neighborhood'].map(ordinal_mapping)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "64      0\n",
       "682     1\n",
       "960     2\n",
       "1384    3\n",
       "1100    4\n",
       "416     5\n",
       "1034    6\n",
       "853     7\n",
       "472     3\n",
       "1011    3\n",
       "Name: Neighborhood, dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's explore the result\n",
    "\n",
    "X_train['Neighborhood'].head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# we can turn the previous commands into 2 functions\n",
    "\n",
    "\n",
    "def find_category_mappings(df, variable):\n",
    "    return {k: i for i, k in enumerate(df[variable].unique(), 0)}\n",
    "\n",
    "\n",
    "def integer_encode(train, test, variable, ordinal_mapping):\n",
    "\n",
    "    X_train[variable] = X_train[variable].map(ordinal_mapping)\n",
    "    X_test[variable] = X_test[variable].map(ordinal_mapping)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "# and now we run a loop over the remaining categorical variables\n",
    "\n",
    "for variable in ['Exterior1st', 'Exterior2nd']:\n",
    "    mappings = find_category_mappings(X_train, variable)\n",
    "    integer_encode(X_train, X_test, variable, mappings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Neighborhood</th>\n",
       "      <th>Exterior1st</th>\n",
       "      <th>Exterior2nd</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>682</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>960</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1384</th>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1100</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Neighborhood  Exterior1st  Exterior2nd\n",
       "64               0            0            0\n",
       "682              1            1            1\n",
       "960              2            1            2\n",
       "1384             3            2            3\n",
       "1100             4            1            1"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's see the result\n",
    "\n",
    "X_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Integer Encoding with Scikit-learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1022, 3), (438, 3))"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's separate into training and testing set\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    data[['Neighborhood', 'Exterior1st', 'Exterior2nd']], # predictors\n",
    "    data['SalePrice'],  # target\n",
    "    test_size=0.3,  # percentage of obs in test set\n",
    "    random_state=0)  # seed to ensure reproducibility\n",
    "\n",
    "X_train.shape, X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "LabelEncoder()"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's create an encoder\n",
    "\n",
    "le = LabelEncoder()\n",
    "le.fit(X_train['Neighborhood'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Blmngtn', 'Blueste', 'BrDale', 'BrkSide', 'ClearCr', 'CollgCr',\n",
       "       'Crawfor', 'Edwards', 'Gilbert', 'IDOTRR', 'MeadowV', 'Mitchel',\n",
       "       'NAmes', 'NPkVill', 'NWAmes', 'NoRidge', 'NridgHt', 'OldTown',\n",
       "       'SWISU', 'Sawyer', 'SawyerW', 'Somerst', 'StoneBr', 'Timber',\n",
       "       'Veenker'], dtype=object)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# we can see the unique classes\n",
    "\n",
    "le.classes_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Neighborhood</th>\n",
       "      <th>Exterior1st</th>\n",
       "      <th>Exterior2nd</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>5</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>VinylSd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>682</th>\n",
       "      <td>4</td>\n",
       "      <td>Wd Sdng</td>\n",
       "      <td>Wd Sdng</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>960</th>\n",
       "      <td>3</td>\n",
       "      <td>Wd Sdng</td>\n",
       "      <td>Plywood</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1384</th>\n",
       "      <td>7</td>\n",
       "      <td>WdShing</td>\n",
       "      <td>Wd Shng</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1100</th>\n",
       "      <td>18</td>\n",
       "      <td>Wd Sdng</td>\n",
       "      <td>Wd Sdng</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Neighborhood Exterior1st Exterior2nd\n",
       "64               5     VinylSd     VinylSd\n",
       "682              4     Wd Sdng     Wd Sdng\n",
       "960              3     Wd Sdng     Plywood\n",
       "1384             7     WdShing     Wd Shng\n",
       "1100            18     Wd Sdng     Wd Sdng"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train['Neighborhood'] = le.transform(X_train['Neighborhood'])\n",
    "X_test['Neighborhood'] = le.transform(X_test['Neighborhood'])\n",
    "\n",
    "X_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Unfortunately, the LabelEncoder works one variable at the time. However there is a way to automate this for all the categorical variables. I took the below from this [stackoverflow thread](https://stackoverflow.com/questions/24458645/label-encoding-across-multiple-columns-in-scikit-learn)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [],
   "source": [
    "# additional import required\n",
    "\n",
    "from collections import defaultdict"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1022, 3), (438, 3))"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's separate into training and testing set\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    data[['Neighborhood', 'Exterior1st', 'Exterior2nd']], # predictors\n",
    "    data['SalePrice'],  # target\n",
    "    test_size=0.3,  # percentage of obs in test set\n",
    "    random_state=0)  # seed to ensure reproducibility\n",
    "\n",
    "X_train.shape, X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "d = defaultdict(LabelEncoder)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Encoding the variable\n",
    "train_transformed = X_train.apply(lambda x: d[x.name].fit_transform(x))\n",
    "\n",
    "# # Using the dictionary to encode future data\n",
    "test_transformed = X_test.apply(lambda x: d[x.name].transform(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Neighborhood</th>\n",
       "      <th>Exterior1st</th>\n",
       "      <th>Exterior2nd</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>5</td>\n",
       "      <td>12</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>682</th>\n",
       "      <td>4</td>\n",
       "      <td>13</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>960</th>\n",
       "      <td>3</td>\n",
       "      <td>13</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1384</th>\n",
       "      <td>7</td>\n",
       "      <td>14</td>\n",
       "      <td>15</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1100</th>\n",
       "      <td>18</td>\n",
       "      <td>13</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Neighborhood  Exterior1st  Exterior2nd\n",
       "64               5           12           13\n",
       "682              4           13           14\n",
       "960              3           13           10\n",
       "1384             7           14           15\n",
       "1100            18           13           14"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_transformed.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Neighborhood</th>\n",
       "      <th>Exterior1st</th>\n",
       "      <th>Exterior2nd</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>529</th>\n",
       "      <td>6</td>\n",
       "      <td>13</td>\n",
       "      <td>11</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>491</th>\n",
       "      <td>12</td>\n",
       "      <td>13</td>\n",
       "      <td>14</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>459</th>\n",
       "      <td>3</td>\n",
       "      <td>8</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>279</th>\n",
       "      <td>4</td>\n",
       "      <td>9</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>655</th>\n",
       "      <td>2</td>\n",
       "      <td>6</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Neighborhood  Exterior1st  Exterior2nd\n",
       "529             6           13           11\n",
       "491            12           13           14\n",
       "459             3            8            8\n",
       "279             4            9           10\n",
       "655             2            6            7"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "test_transformed.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Neighborhood</th>\n",
       "      <th>Exterior1st</th>\n",
       "      <th>Exterior2nd</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>CollgCr</td>\n",
       "      <td>VinylSd</td>\n",
       "      <td>VinylSd</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>682</th>\n",
       "      <td>ClearCr</td>\n",
       "      <td>Wd Sdng</td>\n",
       "      <td>Wd Sdng</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>960</th>\n",
       "      <td>BrkSide</td>\n",
       "      <td>Wd Sdng</td>\n",
       "      <td>Plywood</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1384</th>\n",
       "      <td>Edwards</td>\n",
       "      <td>WdShing</td>\n",
       "      <td>Wd Shng</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1100</th>\n",
       "      <td>SWISU</td>\n",
       "      <td>Wd Sdng</td>\n",
       "      <td>Wd Sdng</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Neighborhood Exterior1st Exterior2nd\n",
       "64        CollgCr     VinylSd     VinylSd\n",
       "682       ClearCr     Wd Sdng     Wd Sdng\n",
       "960       BrkSide     Wd Sdng     Plywood\n",
       "1384      Edwards     WdShing     Wd Shng\n",
       "1100        SWISU     Wd Sdng     Wd Sdng"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# and to inverse transform to recover the original labels\n",
    "\n",
    "# # Inverse the encoded\n",
    "tmp = train_transformed.apply(lambda x: d[x.name].inverse_transform(x))\n",
    "tmp.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, there is another Scikit-learn transformer, the OrdinalEncoder, to encode multiple variables at the same time. However, this transformer returns a NumPy array without column names, so it is not my favourite implementation. More details here: https://scikit-learn.org/stable/modules/generated/sklearn.preprocessing.OrdinalEncoder.html "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Integer Encoding with Feature-Engine"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "((1022, 3), (438, 3))"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# let's separate into training and testing set\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(\n",
    "    data[['Neighborhood', 'Exterior1st', 'Exterior2nd']], # predictors\n",
    "    data['SalePrice'],  # target\n",
    "    test_size=0.3,  # percentage of obs in test set\n",
    "    random_state=0)  # seed to ensure reproducibility\n",
    "\n",
    "X_train.shape, X_test.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "OrdinalCategoricalEncoder(encoding_method='arbitrary',\n",
       "                          variables=['Neighborhood', 'Exterior1st',\n",
       "                                     'Exterior2nd'])"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ordinal_enc = OrdinalCategoricalEncoder(\n",
    "    encoding_method='arbitrary',\n",
    "    variables=['Neighborhood', 'Exterior1st', 'Exterior2nd'])\n",
    "\n",
    "ordinal_enc.fit(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'Neighborhood': {'CollgCr': 0,\n",
       "  'ClearCr': 1,\n",
       "  'BrkSide': 2,\n",
       "  'Edwards': 3,\n",
       "  'SWISU': 4,\n",
       "  'Sawyer': 5,\n",
       "  'Crawfor': 6,\n",
       "  'NAmes': 7,\n",
       "  'Mitchel': 8,\n",
       "  'Timber': 9,\n",
       "  'Gilbert': 10,\n",
       "  'Somerst': 11,\n",
       "  'MeadowV': 12,\n",
       "  'OldTown': 13,\n",
       "  'BrDale': 14,\n",
       "  'NWAmes': 15,\n",
       "  'NridgHt': 16,\n",
       "  'SawyerW': 17,\n",
       "  'NoRidge': 18,\n",
       "  'IDOTRR': 19,\n",
       "  'NPkVill': 20,\n",
       "  'StoneBr': 21,\n",
       "  'Blmngtn': 22,\n",
       "  'Veenker': 23,\n",
       "  'Blueste': 24},\n",
       " 'Exterior1st': {'VinylSd': 0,\n",
       "  'Wd Sdng': 1,\n",
       "  'WdShing': 2,\n",
       "  'HdBoard': 3,\n",
       "  'MetalSd': 4,\n",
       "  'AsphShn': 5,\n",
       "  'BrkFace': 6,\n",
       "  'Plywood': 7,\n",
       "  'CemntBd': 8,\n",
       "  'Stucco': 9,\n",
       "  'BrkComm': 10,\n",
       "  'AsbShng': 11,\n",
       "  'ImStucc': 12,\n",
       "  'CBlock': 13,\n",
       "  'Stone': 14},\n",
       " 'Exterior2nd': {'VinylSd': 0,\n",
       "  'Wd Sdng': 1,\n",
       "  'Plywood': 2,\n",
       "  'Wd Shng': 3,\n",
       "  'HdBoard': 4,\n",
       "  'MetalSd': 5,\n",
       "  'AsphShn': 6,\n",
       "  'CmentBd': 7,\n",
       "  'BrkFace': 8,\n",
       "  'Stucco': 9,\n",
       "  'ImStucc': 10,\n",
       "  'Stone': 11,\n",
       "  'AsbShng': 12,\n",
       "  'Brk Cmn': 13,\n",
       "  'CBlock': 14,\n",
       "  'Other': 15}}"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# in the encoder dict we can observe the numbers\n",
    "# assigned to each category for all the indicated variables\n",
    "\n",
    "ordinal_enc.encoder_dict_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['Neighborhood', 'Exterior1st', 'Exterior2nd']"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# this is the list of variables that the encoder will transform\n",
    "\n",
    "ordinal_enc.variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Neighborhood</th>\n",
       "      <th>Exterior1st</th>\n",
       "      <th>Exterior2nd</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>64</th>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>682</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>960</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1384</th>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1100</th>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "      Neighborhood  Exterior1st  Exterior2nd\n",
       "64               0            0            0\n",
       "682              1            1            1\n",
       "960              2            1            2\n",
       "1384             3            2            3\n",
       "1100             4            1            1"
      ]
     },
     "execution_count": 29,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train = ordinal_enc.transform(X_train)\n",
    "X_test = ordinal_enc.transform(X_test)\n",
    "\n",
    "# let's explore the result\n",
    "X_train.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note**\n",
    "\n",
    "If the argument variables is left to None, then the encoder will automatically identify all categorical variables. Is that not sweet?\n",
    "\n",
    "The encoder will not encode numerical variables. So if some of your numerical variables are in fact categories, you will need to re-cast them as object before using the encoder.\n",
    "\n",
    "Note, if there is a variable in the test set, for which the encoder doesn't have a number to assigned (the category was not seen in the train set), the encoder will return an error."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "feml",
   "language": "python",
   "name": "feml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": "block",
   "toc_window_display": true
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}