main_notebook.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 33,
   "id": "437350a7-1529-4f22-9eab-3fa3dd93603c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "from sklearn.neighbors import LocalOutlierFactor\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b50d7446-fe7f-4905-86e5-bb7601e8e74a",
   "metadata": {},
   "source": [
    "***"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "992d7d4f-4ec9-431a-871b-61d61ce90f38",
   "metadata": {},
   "source": [
    " ### Step 1: Loading and Preprocessing the Data\n",
    "---\n",
    "Read the CSV file and save it in a temporary dataframe.    \n",
    "To further safeguard patient privacy, the data has been subjected to extra deidentification prior to importing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "58546e64-ff80-4f2e-94d8-35376e9e7d0c",
   "metadata": {},
   "outputs": [],
   "source": [
    "temp_df = pd.read_csv(\"do_not_track/ckd_deidentified.csv\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "95e7cde1-70dc-4376-a5a5-71c8a8e53729",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Renaming Columns and Setting Index\n",
    "---\n",
    "* Rename the column \"Unnamed: 0\" to \"instance\" in the temporary dataframe.\n",
    "* Set the \"instance\" column as the index of the dataframe.\n",
    "* Remove the index name to improve clarity.\n",
    "* Create a new dataframe, \"df,\" as a copy of the temporary dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "id": "ac682257-eedc-4b61-8cc7-a08ab608d88e",
   "metadata": {},
   "outputs": [],
   "source": [
    "temp_df.rename(columns={\"Unnamed: 0\": \"instance\"}, inplace = True)\n",
    "temp_df.set_index('instance', inplace = True)\n",
    "temp_df.index.name = None\n",
    "df = temp_df.copy()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dcf722f2-1ab7-4f69-a4cc-851e7f59e173",
   "metadata": {},
   "source": [
    "\n",
    "### Displaying DataFrame Information\n",
    "---\n",
    "This next code snippet provides various insights and information about the DataFrame:    \n",
    "   \n",
    "* Displaying DataFrame Columns:    \n",
    "    The names of the columns in the dataframe are printed.\n",
    "   \n",
    "* Displaying Column Data Types:    \n",
    "    The data types of the columns in the dataframe are printed.\n",
    "   \n",
    "* Displaying DataFrame Shape:    \n",
    "    The number of rows and columns in the dataframe are printed.\n",
    "   \n",
    "* Displaying Summary Statistics:    \n",
    "    Summary statistics for the numerical columns in the dataframe are printed, rounded to two decimal places.\n",
    "   \n",
    "* Displaying Gender Distribution:    \n",
    "    The distribution of genders in the dataframe is printed as a percentage, rounded to five decimal places.    \n",
    "\n",
    "These lines of code help provide an overview of the dataframe's content, structure, and statistical information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "id": "989e8ac2-9d2c-4902-b0de-6d629c99f006",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1mThe DataFrame columns are:\u001b[0m\n",
      "\n",
      "Index(['hematocrit', 'hemoglobin', 'mch', 'mchc', 'mcv', 'platelet', 'rbc',\n",
      "       'rdw', 'wbc', 'albumin', 'globulin', 'total_protein', 'aniongap',\n",
      "       'bicarbonate', 'bun', 'calcium', 'chloride', 'creatinine', 'glucose',\n",
      "       'sodium', 'potassium', 'age', 'gender', 'ckd', 'hypertension',\n",
      "       'diabetes'],\n",
      "      dtype='object')\n",
      "\n",
      "\u001b[1mThe column data types are:\u001b[0m\n",
      "\n",
      "hematocrit       float64\n",
      "hemoglobin       float64\n",
      "mch              float64\n",
      "mchc             float64\n",
      "mcv              float64\n",
      "platelet         float64\n",
      "rbc              float64\n",
      "rdw              float64\n",
      "wbc              float64\n",
      "albumin          float64\n",
      "globulin         float64\n",
      "total_protein    float64\n",
      "aniongap         float64\n",
      "bicarbonate      float64\n",
      "bun              float64\n",
      "calcium          float64\n",
      "chloride         float64\n",
      "creatinine       float64\n",
      "glucose          float64\n",
      "sodium           float64\n",
      "potassium        float64\n",
      "age                int64\n",
      "gender            object\n",
      "ckd                int64\n",
      "hypertension       int64\n",
      "diabetes           int64\n",
      "dtype: object\n",
      "\n",
      "\u001b[1mThe dataframe has\u001b[0m 1636315 \u001b[1mlines and\u001b[0m 26 \u001b[1mcolumns\u001b[0m\n",
      "\n",
      "\u001b[1mThe summary statistics for the numerical columns are:\u001b[0m\n",
      "\n",
      "       hematocrit  hemoglobin         mch        mchc         mcv    platelet  \\\n",
      "count  1629960.00  1593606.00  1588370.00  1588542.00  1588470.00  1598386.00   \n",
      "mean        31.25       10.24       29.80       32.74       91.09      227.89   \n",
      "std          6.00        2.07        2.77        1.65        7.30      133.91   \n",
      "min          4.30        1.30       12.30       21.50        8.00        5.00   \n",
      "25%         26.60        8.60       28.40       31.70       87.00      143.00   \n",
      "50%         30.70       10.00       30.00       32.80       91.00      209.00   \n",
      "75%         35.40       11.70       31.40       33.80       95.00      288.00   \n",
      "max        248.00       23.30      374.60       47.60      156.00     2947.00   \n",
      "\n",
      "              rbc         rdw         wbc    albumin  ...     calcium  \\\n",
      "count  1588481.00  1587352.00  1590141.00  257170.00  ...  1372546.00   \n",
      "mean         3.46       15.56        9.48       3.24  ...        8.61   \n",
      "std          0.72        2.58        9.03       0.67  ...        0.75   \n",
      "min          0.45       10.40        0.10       0.20  ...        0.10   \n",
      "25%          2.91       13.70        5.80       2.80  ...        8.20   \n",
      "50%          3.40       15.00        8.10       3.20  ...        8.60   \n",
      "75%          3.95       16.80       11.40       3.70  ...        9.10   \n",
      "max          8.30      161.00      884.00       9.10  ...      132.00   \n",
      "\n",
      "         chloride  creatinine     glucose      sodium   potassium         age  \\\n",
      "count  1602819.00  1607316.00  1567033.00  1602243.00  1614938.00  1636315.00   \n",
      "mean       102.32        1.41      126.35      138.42        4.12       63.88   \n",
      "std          5.58        1.53       60.27        4.43        0.57       16.64   \n",
      "min         56.00        0.10        1.00       73.00        0.90       18.00   \n",
      "25%         99.00        0.70       94.00      136.00        3.80       54.00   \n",
      "50%        103.00        0.90      111.00      139.00        4.10       65.00   \n",
      "75%        106.00        1.40      140.00      141.00        4.40       76.00   \n",
      "max        155.00       74.00     5840.00      182.00       26.50      103.00   \n",
      "\n",
      "              ckd  hypertension    diabetes  \n",
      "count  1636315.00    1636315.00  1636315.00  \n",
      "mean         0.24          0.62        0.32  \n",
      "std          0.43          0.48        0.46  \n",
      "min          0.00          0.00        0.00  \n",
      "25%          0.00          0.00        0.00  \n",
      "50%          0.00          1.00        0.00  \n",
      "75%          0.00          1.00        1.00  \n",
      "max          1.00          1.00        1.00  \n",
      "\n",
      "[8 rows x 25 columns]\n",
      "\n",
      "\u001b[1mThe gender distribution is:\u001b[0m\n",
      "\n",
      "M    53.0%\n",
      "F    47.0%\n",
      "Name: gender, dtype: object\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(f\"\\033[1mThe DataFrame columns are:\\033[0m\\n\\n{df.columns}\\n\")\n",
    "print(f\"\\033[1mThe column data types are:\\033[0m\\n\\n{df.dtypes}\\n\")\n",
    "print(f\"\\033[1mThe dataframe has\\033[0m {df.shape[0]} \\033[1mlines and\\033[0m {df.shape[1]} \\033[1mcolumns\\033[0m\\n\")\n",
    "print(f\"\\033[1mThe summary statistics for the numerical columns are:\\033[0m\\n\\n{df.describe().round(2)}\\n\")\n",
    "print(f\"\\033[1mThe gender distribution is:\\033[0m\\n\\n{(df['gender'].value_counts(normalize=True).round(2) * 100).apply(lambda x: f'{x}%')}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "56d33062-ac5a-439a-9827-2d2135d4c128",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa106e85-cea7-4c60-8792-fcde6c393ea6",
   "metadata": {},
   "source": [
    "### Calculating Missing and Duplicate Values\n",
    "---\n",
    "This code snippet calculates and displays the following information about the dataframe:\n",
    "\n",
    "* Calculating Percentage of Missing Values:    \n",
    "    The percentage of missing values in each column is calculated and rounded to three decimal places.\n",
    "   \n",
    "* Calculating Percentage of Duplicate Values:    \n",
    "    The percentage of duplicate rows is calculated and rounded to two decimal places. \n",
    "\n",
    "These lines of code provide insights into the presence of missing values and duplicates within the dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "95d6a62c-4276-45e0-b9de-3e86e15e27b5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1mThe percentage of missing values is:\u001b[0m\n",
      "\n",
      "hematocrit        0.388%\n",
      "hemoglobin         2.61%\n",
      "mch                2.93%\n",
      "mchc               2.92%\n",
      "mcv               2.924%\n",
      "platelet          2.318%\n",
      "rbc               2.923%\n",
      "rdw               2.992%\n",
      "wbc               2.822%\n",
      "albumin          84.284%\n",
      "globulin         99.532%\n",
      "total_protein     99.19%\n",
      "aniongap          3.408%\n",
      "bicarbonate       3.223%\n",
      "bun               2.325%\n",
      "calcium           16.12%\n",
      "chloride          2.047%\n",
      "creatinine        1.772%\n",
      "glucose           4.234%\n",
      "sodium            2.082%\n",
      "potassium         1.306%\n",
      "age                 0.0%\n",
      "gender              0.0%\n",
      "ckd                 0.0%\n",
      "hypertension        0.0%\n",
      "diabetes            0.0%\n",
      "dtype: object\n",
      "\n",
      "\u001b[1mThe percentage of duplicate values is:\u001b[0m\n",
      "\n",
      "0.0%\n"
     ]
    }
   ],
   "source": [
    "missing_values_percentage = (df.isnull().mean()*100).round(3).apply(lambda x: f'{x}%')\n",
    "duplicates_count = (df.duplicated().mean()*100).round(2)\n",
    "\n",
    "print(f\"\\033[1mThe percentage of missing values is:\\033[0m\\n\\n{missing_values_percentage}\\n\")\n",
    "print(f\"\\033[1mThe percentage of duplicate values is:\\033[0m\\n\\n{duplicates_count}%\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9832893c-22fb-40c5-8c19-d4f13df83337",
   "metadata": {},
   "source": [
    "### Counting Rows with Serum Creatinine Equal to 0\n",
    "----------------------------------------------------\n",
    "This code snippet counts the number of rows in the dataframe where the 'creatinine' column has a value of 0.    \n",
    "The egfr equation contains this part $\\min\\left( \\frac{standardized\\ Scr}{a},\\ 1 \\right)^{- 0.241}$.    \n",
    "Thus in case of Scr equal to 0 we would need to raise 0 in a negative number (-0.241) which throws a ZeroDivisionError.\n",
    " \n",
    "* Counting Rows with Serum Creatinine Equal to 0:    \n",
    "    Rows in the dataframe where the 'creatinine' column is equal to 0 are counted.    \n",
    "    The count is stored in the variable 'creatinine_0' as a list and the calculated percentage is printed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "40f886af-a7cf-4952-8432-c42033f7a7ed",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1mNo indexes with creatinine equal to 0 found.\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "creatinine_0 = df[df['creatinine'] == 0]['creatinine'].value_counts().to_list()\n",
    "if len(creatinine_0) > 0:\n",
    "    percentage = round((100 * creatinine_0[0] / len(df)), 5)\n",
    "    print(f\"\\033[1mThe percentage of indexes with creatinine equal to 0 is\\033[0m {percentage}%\")\n",
    "else:\n",
    "    print(\"\\033[1mNo indexes with creatinine equal to 0 found.\\033[0m\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "30286968-07ba-4a5a-8e65-8621506f1c6f",
   "metadata": {},
   "source": [
    "### Data Cleaning Operations\n",
    "---\n",
    "This code snippet performs various data cleaning operations on the dataframe:\n",
    "\n",
    "* Removing Rows and columns with Missing Values:    \n",
    "    The columns albumin, globulin and total protein have an high percentage of missing values and will be droped.\n",
    "    All rows with missing values are droped as well.\n",
    "   \n",
    "* Removing Duplicate Rows:    \n",
    "    There are no duplicate rows that need to be removed.\n",
    "   \n",
    "* Removing Rows with Serum Creatinine Equal to 0:    \n",
    "    There are no rows where creatinine equals 0 that need to be removed.\n",
    "    \n",
    "These operations help ensure data quality and prepare the dataframe for further analysis."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "02be9bdc-84c3-409c-bde8-d4a61f525eaa",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.drop([\"albumin\", \"globulin\", \"total_protein\"], axis=1)\n",
    "df.dropna(axis=0, inplace=True)\n",
    "# df.drop_duplicates(inplace=True)\n",
    "# df = df[df['creatinine'] != 0].dropna()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "6d00bb22-e3ef-4769-a168-42823ee3dad7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[1mThe dataframe has\u001b[0m 1338774 \u001b[1mrows and\u001b[0m 23 \u001b[1mcolumns\u001b[0m\n",
      "\n",
      "\u001b[1mThe summary statistics for the numerical columns are:\u001b[0m\n",
      "\n",
      "       hematocrit  hemoglobin         mch        mchc         mcv    platelet  \\\n",
      "count  1338774.00  1338774.00  1338774.00  1338774.00  1338774.00  1338774.00   \n",
      "mean        31.04       10.15       29.81       32.69       91.24      226.79   \n",
      "std          5.98        2.05        2.76        1.66        7.35      135.67   \n",
      "min          4.30        1.30       12.30       22.00        8.00        5.00   \n",
      "25%         26.40        8.50       28.40       31.60       87.00      140.00   \n",
      "50%         30.40        9.90       30.00       32.70       91.00      208.00   \n",
      "75%         35.10       11.50       31.40       33.80       95.00      289.00   \n",
      "max         68.60       23.30       56.70       47.60      156.00     2669.00   \n",
      "\n",
      "              rbc         rdw         wbc    aniongap  ...     calcium  \\\n",
      "count  1338774.00  1338774.00  1338774.00  1338774.00  ...  1338774.00   \n",
      "mean         3.43       15.68        9.44       13.79  ...        8.62   \n",
      "std          0.72        2.62        9.06        3.52  ...        0.74   \n",
      "min          0.45       10.40        0.10      -22.00  ...        0.10   \n",
      "25%          2.88       13.80        5.70       11.00  ...        8.20   \n",
      "50%          3.36       15.10        8.10       13.00  ...        8.60   \n",
      "75%          3.91       17.00       11.30       16.00  ...        9.10   \n",
      "max          8.28       44.30      878.30       91.00  ...       61.00   \n",
      "\n",
      "         chloride  creatinine     glucose      sodium   potassium         age  \\\n",
      "count  1338774.00  1338774.00  1338774.00  1338774.00  1338774.00  1338774.00   \n",
      "mean       102.29        1.45      126.64      138.45        4.11       63.85   \n",
      "std          5.65        1.60       58.89        4.47        0.57       16.52   \n",
      "min         57.00        0.10        2.00       73.00        1.00       18.00   \n",
      "25%         99.00        0.70       95.00      136.00        3.70       54.00   \n",
      "50%        103.00        0.90      112.00      139.00        4.10       65.00   \n",
      "75%        106.00        1.40      141.00      141.00        4.40       76.00   \n",
      "max        155.00       74.00     3620.00      182.00       26.50      103.00   \n",
      "\n",
      "              ckd  hypertension    diabetes  \n",
      "count  1338774.00    1338774.00  1338774.00  \n",
      "mean         0.25          0.62        0.32  \n",
      "std          0.43          0.49        0.47  \n",
      "min          0.00          0.00        0.00  \n",
      "25%          0.00          0.00        0.00  \n",
      "50%          0.00          1.00        0.00  \n",
      "75%          0.00          1.00        1.00  \n",
      "max          1.00          1.00        1.00  \n",
      "\n",
      "[8 rows x 22 columns]\n",
      "\n",
      "\u001b[1mThe gender distribution is:\u001b[0m\n",
      "\n",
      "M    53.0%\n",
      "F    47.0%\n",
      "Name: gender, dtype: object\n",
      "\n"
     ]
    }
   ],
   "source": [
    "print(f\"\\033[1mThe dataframe has\\033[0m {df.shape[0]} \\033[1mrows and\\033[0m {df.shape[1]} \\033[1mcolumns\\033[0m\\n\")\n",
    "print(f\"\\033[1mThe summary statistics for the numerical columns are:\\033[0m\\n\\n{df.describe().round(2)}\\n\")\n",
    "print(f\"\\033[1mThe gender distribution is:\\033[0m\\n\\n{(df['gender'].value_counts(normalize=True).round(2) * 100).apply(lambda x: f'{x}%')}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ef63a380-2c81-4925-a851-7d02eced014a",
   "metadata": {},
   "source": [
    "### eGFR Calculation\n",
    "***\n",
    "\n",
    "The CKD-EPI(2021) eGFR equation differentiates depending on the patient gender. Based on the CKD-EPI(2021) equation from [kidney.org](https://www.kidney.org/professionals/kdoqi/gfr_calculator/formula) the gender specific equations are:    \n",
    "\n",
    "$eGFR\\_ female = 142*\\min\\left( \\frac{standardized\\ Scr}{0.7},\\ 1 \\right)^{- 0.241}*{\\max\\left( \\frac{standardized\\ Scr}{0.7},\\ 1 \\right)}^{- 1.2}*{0.9938}^{age}*1.012$    \n",
    "\n",
    "  \n",
    "\n",
    "$eGFR\\_ male = 142*\\min\\left( \\frac{standardized\\ Scr}{0.9},\\ 1 \\right)^{- 0.302}*{\\max\\left( \\frac{standardized\\ Scr}{0.9},\\ 1 \\right)}^{- 1.2}*{0.9938}^{age}$    \n",
    "\n",
    "---\n",
    "This code snippet defines two functions to calculate the estimated glomerular filtration rate (eGFR) based on age and serum creatinine levels. The calculated eGFR is rounded to two decimal places and returned as the result:\n",
    "\n",
    "* Equation for Females: egfr_f(age,s_cr).\n",
    "* Equation for Males: egfr_m(age,s_cr).    \n",
    "\n",
    "Where:    \n",
    "**age** represents the age of the individual    \n",
    "**s_cr** denotes the serum creatinine level"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "1855d220-5010-4658-a21b-ce31a159495f",
   "metadata": {},
   "outputs": [],
   "source": [
    "# For females\n",
    "def egfr_f(age,s_cr):\n",
    "    result = round(142 * pow(min((s_cr/0.7),1), -0.241) * pow(max((s_cr/0.7),1),-1.2) * pow(0.9938, age) * 1.012 , 2)\n",
    "    return result\n",
    "\n",
    "# For males\n",
    "def egfr_m(age,s_cr):\n",
    "    result = round(142 * pow(min((s_cr/0.9),1), -0.302) * pow(max((s_cr/0.9),1),-1.2) * pow(0.9938, age) , 2)\n",
    "    return result"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d301da38-ceab-4dbc-9ccd-ffe23373d29d",
   "metadata": {},
   "source": [
    "### Calculating eGFR for Each Row\n",
    "---    \n",
    "This code snippet calculates the estimated glomerular filtration rate (eGFR) for each row in the dataframe based on gender-specific equations and the values of 'age' and 'creatinine' columns.\n",
    "\n",
    "1. Applying eGFR Calculation:\n",
    "   - The `apply()` function is used to iterate over each row in the dataframe and calculate the eGFR.\n",
    "   - For females (gender == 'F'), the `egfr_f()` function is applied with the 'age' and 'creatinine' values from the current row.\n",
    "   - For males (gender != 'F'), the `egfr_m()` function is applied with the 'age' and 'creatinine' values from the current row.\n",
    "\n",
    "2. Creating 'egfr' Column:\n",
    "   - The calculated eGFR values are assigned to a new column named 'egfr' in the dataframe."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "27beecb0-3bd6-4ff2-96ac-dddd4115c0e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "df['egfr'] = df.apply(lambda row: egfr_f(row['age'], row['creatinine']) if row['gender'] == 'F' else egfr_m(row['age'], row['creatinine']), axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31a80f71-1b7c-4a28-9f0c-520c58a5780c",
   "metadata": {},
   "source": [
    "### DataSet\n",
    "\n",
    "After all these operations, we have a dataset with 1,338,774 entries in 24 columns that does not contain any missing data. Each row in the dataset represents a specific point in time when blood samples were taken. For example, multiple rows could correspond to blood samples taken during the same hospital admission from the same patient but at different times. Alternatively, they could represent blood samples from the same patient but during different hospital admissions.\n",
    "\n",
    "The dataset includes flags for diabetes, high blood pressure, and chronic kidney disease (CKD), as well as the calculated estimated glomerular filtration rate (eGFR). This dataset could serve as a valuable starting point for data science projects focused on CKD or other health-related issues."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "5d2451ed-6b09-4990-862d-48f8bc412e74",
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0000003c42d42886f0512786c8cef89336c3dbc4a25e5dd9531479e383b8844b    0\n",
       "00000727efa061858c1e6611657af5b6defd0ab071091a096c393afe938b1489    1\n",
       "00004622a4c6b7bb6b873558488dd2e828dc77a5d9587f6382aa48cb940f589f    0\n",
       "00004741d15bcf0f0453f94cdd9eeb85c663584f26c5fe0705e96c8aa01eab66    1\n",
       "0000498d704cb63e36fbf9d490e4feb2d028bdf6bc749567e99465f5acdb5d9e    0\n",
       "                                                                   ..\n",
       "ffffa4205384a823a1039146987a197f356678e79cf4aff59ab75d0ac575bdcf    1\n",
       "ffffbc4b10d69e134494f554c6d11aa5c7b3d62a3ed9f2779dd95f0a499c8dc9    1\n",
       "ffffcd111a56292182022764fb3047749d3f2aa3fd4801efbc5b24fbec3f580c    1\n",
       "ffffd3e90e8f8f17ca53a0fd689e5fa276d7b14f088b2b397b3ef36d118737ac    1\n",
       "fffff04f91d9bb69086b462eed519b73e03567d768e873746fe6c101e48f1fec    0\n",
       "Name: gender, Length: 1338774, dtype: int64"
      ]
     },
     "execution_count": 43,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mapping = {'F': 0, 'M': 1}\n",
    "df[\"gender\"] = df[\"gender\"].map(mapping)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "10dca538-9513-4293-a938-edef4a215a45",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "lof = LocalOutlierFactor()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "420a6f02-0853-4956-b319-cecb334bdd74",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "outlier_labels = lof.fit_predict(df)\n",
    "inlier_labels = lof.negative_outlier_factor_ > lof.threshold_\n",
    "outlier_scores = lof.negative_outlier_factor_\n",
    "negative_outlier_factor = lof.negative_outlier_factor_\n",
    "decision_function = lof.decision_function(df) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bc72a2b0-8fbf-42f5-a808-bdcf807b1bbb",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.11"
  },
  "toc-showcode": true
 },
 "nbformat": 4,
 "nbformat_minor": 5
}