BIOT7060_ml_workshop_p1.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_FsyuNEbzbMx"
   },
   "source": [
    "# Machine Learning Workshop - Predicting Patient Diabetes - Part 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PqyMvnL-zbM6"
   },
   "source": [
    "## Accessing this material\n",
    "\n",
    "All material used in this workshop can be downloaded at **[https://github.com/alexgcsa/ml_workshop_Oct2024](https://github.com/alexgcsa/ml_workshop_Oct2024)**.\n",
    "\n",
    "Instructions for local installation are detailed in the *README.md* file.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QpIRzrnkzbM7"
   },
   "source": [
    "## Outline\n",
    "- Task Overview;\n",
    "- Download Data and Requirement Files;\n",
    "- Patient Data Exploration;\n",
    "- Supervised Machine Learning:\n",
    "    - Classification -- Identifying diabetic and non-diabetic (healthy) patients.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task Overview"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MofvlAP1zbM8"
   },
   "source": [
    "The problem we will solve is **predicting patient diabetes with machine learning using medical and demographic data**.\n",
    "\n",
    "For this task, we will use a modified/compacted version of a dataset from **[Kaggle](https://www.kaggle.com/datasets/iammustafatz/diabetes-prediction-dataset/)**. This dataset includes variables (features) such as:\n",
    " 1. age;\n",
    " 2. gender -- Distinguished in this data between male (1) and female (0);\n",
    " 3. If the patient has or does not have hypertension (0: not having; 1: having);\n",
    " 4. If the patient has or does not have heart disease (0: not having; 1: having);\n",
    " 5. Body Mass Index (BMI);\n",
    " 6. Smoking history -- Never or ever smoked (0: never; ever: 1);\n",
    "\n",
    "\n",
    "Together with these features, patients are differentiated by their IDs and diabetes status (non-diabetic patient = **400**; diabetic patient = **150**).\n",
    "\n",
    "We can use this data and these features to build **machine learning models** to predict diabetes in patients based on their medical history and demographic information. \n",
    "\n",
    "This can be useful for healthcare professionals in identifying patients at risk of developing diabetes and in developing personalised treatment plans."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "iWJprSfJzbM-"
   },
   "source": [
    "## Download Data and Requirement Files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "EJQlfbS-zbM_"
   },
   "outputs": [],
   "source": [
    "!wget https://raw.githubusercontent.com/alexgcsa/ml_workshop_Oct2023/master/data/diabetes_compact.csv\n",
    "!wget https://raw.githubusercontent.com/alexgcsa/ml_workshop_Oct2023/master/requirements.txt\n",
    "!wget https://raw.githubusercontent.com/alexgcsa/ml_workshop_Oct2023/master/utils.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "t5QcOsL2zbNB"
   },
   "source": [
    "## Installing dependencies"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "MmpcOpqDZjWF"
   },
   "outputs": [],
   "source": [
    "!pip install -r requirements.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4lZKLe4IzbND"
   },
   "source": [
    "## Loading libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "BQ9flrTHaIEb"
   },
   "outputs": [],
   "source": [
    "%load_ext autoreload\n",
    "%autoreload 2\n",
    "\n",
    "from plotly.offline import iplot, init_notebook_mode\n",
    "from plotly.subplots import make_subplots\n",
    "import matplotlib.pyplot as plt\n",
    "import plotly.express as px\n",
    "import plotly.graph_objects as go\n",
    "\n",
    "from sklearn import tree\n",
    "from sklearn import ensemble\n",
    "from sklearn.model_selection import cross_val_predict, train_test_split\n",
    "\n",
    "\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import time\n",
    "\n",
    "import utils\n",
    "\n",
    "import warnings\n",
    "\n",
    "warnings.simplefilter(action='ignore', category=FutureWarning)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "N_15evfSzbNE"
   },
   "source": [
    "## Exploring the data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "6IMnadvDatyN"
   },
   "outputs": [],
   "source": [
    "# Load data from file to a DataFrame structure:\n",
    "df_data = pd.read_csv(\"diabetes_compact.csv\") \n",
    "\n",
    "print(df_data.shape) # .shape displays how the dataframe (matrix) looks like\n",
    "df_data.head(10) # .head(10) displays the first 10 items in the dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_data.tail(10) # .tail(10) displays the last 10 items in the dataframe"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotting the distribution of diabetic patients *versus* non-diabetic patients"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "P_k_WsFgcIg1"
   },
   "outputs": [],
   "source": [
    "#Counting the frequencies:\n",
    "diabetes_counts = dict(df_data[\"class\"].value_counts())\n",
    "\n",
    "#getting the classes:\n",
    "classes = list(diabetes_counts.keys())\n",
    "\n",
    "#Getting the values per class:\n",
    "values = list(diabetes_counts.values())\n",
    "\n",
    "#Making the plot:\n",
    "fig = plt.bar(classes, values, color ='purple',  width = 0.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### What can you conclude from the plot above?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plotting the distribution of patient features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "E3P_qZGdcoZs"
   },
   "outputs": [],
   "source": [
    "columns = [\"gender\", \"age\", \"hypertension\", \"heart_disease\", \"bmi\", \"smoking_history\"]\n",
    "\n",
    "fig = make_subplots(rows=3, cols=2, start_cell=\"bottom-left\")\n",
    "\n",
    "nrows = 3\n",
    "ncols = 2\n",
    "\n",
    "# Iterate through columns in reverse order\n",
    "for i in range(nrows):\n",
    "    for j in range(ncols):\n",
    "        col_index = (nrows - i - 1) * ncols + j  # Calculate the correct column index in reverse order\n",
    "        if col_index < len(columns):\n",
    "            column_name = columns[col_index]\n",
    "            fig.add_trace(go.Histogram(x=df_data[column_name], name=column_name), row=i + 1, col=j + 1)\n",
    "\n",
    "\n",
    "fig.show(renderer=\"colab\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### What can you conclude from the several plots above?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Supervised Machine Learning (Classification)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gdhIrs-CzbNL"
   },
   "source": [
    "Your task is to build a classifier to differentiate diabetic patients from non-diabetic patients.\n",
    "\n",
    "First, use the code below to select  specific columns for the basic features (**df_data**)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "YDw_V5O3zbNL"
   },
   "outputs": [],
   "source": [
    "# The target class and the main features from the dataset:\n",
    "target = \"class\"\n",
    "features =  [\"gender\", \"age\", \"hypertension\", \"heart_disease\", \"bmi\", \"smoking_history\"]\n",
    "\n",
    "# Select them from data:\n",
    "X = df_data[features]\n",
    "y = df_data[target]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Data Splitting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What is the best TEST_SIZE? \n",
    "TEST_SIZE = 0.25\n",
    "\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=TEST_SIZE, \\\n",
    "                                                    random_state=42)\n",
    "\n",
    "utils.plot_train_test_class(y_train, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Why is data splitting necessary?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "oQakcgszzbNQ"
   },
   "source": [
    "If you check in the code lines above, you used a function **train_test_split**. This function divides our data into two subsets: the **training set** and the **testing set**. The training set is used to train (i.e., build) the model. The testing set is used to evaluate the model, assessing its predictive performance.\n",
    "\n",
    "This step is very important because it helps ensure the creation of machine learning models is valid and accurate. Using the testing set (a subset completely unseen during the training of the model), you can estimate the model's performance when it is applied to new data points.  \n",
    "\n",
    "**What do you think of having the same proportions of Diabetic and Non-Diabetic patients in the training and testing sets? Is this a fair approach?**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### [K-Fold Cross-Validation](https://en.wikipedia.org/wiki/Cross-validation_(statistics))\n",
    "\n",
    "Cross-validation is a resampling method that uses different data portions to train and validate a model on different iterations. It is mainly used in settings where the goal is prediction, and one wants to estimate how accurately a predictive model will perform in practice."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### First model: Decision Tree"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4f2dqbUIzbNS"
   },
   "source": [
    "Decision trees are composed of if/else questions which are disposed of hierarchically. Following these questions, the model is capable of reaching a decision. In the case of our question, the actual output is 'diabetic' or 'diabetic' labels. The decision to reach a prediction is based on the features (patient demographics and medical information) we used as input for the ML algorithm."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pexx2g4ZzbNS"
   },
   "source": [
    "##### Training the decision tree on the set of features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What happens when you change the max depth of the decision tree?\n",
    "clf_dt = tree.DecisionTreeClassifier(max_depth=3, random_state=42)\n",
    "\n",
    "# What happens if you change the number of folds in the cross-validation from 5 to 3 (or 10)?\n",
    "y_pred_train_dt = cross_val_predict(clf_dt, X_train, y_train, cv=5)\n",
    "clf_dt.fit(X_train, y_train)\n",
    "y_pred_test_dt = clf_dt.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " ##### Confusion Matrix Cases:\n",
    " 1. Patient(s) is (are) Diabetic (1), and the model predicted them as Diabetic (1). **Correct Prediction (True Positive [TP])**.\n",
    " 2. Patient(s) is (are) Non-Diabetic (0), and the model predicted them as Diabetic (1). **Wrong Prediction (False Positive [FP], Type 1 Error)**.\n",
    " 3. Patient(s) is (are) Non-Diabetic (0), and the model predicted them as Non-Diabetic (0). **Correct Prediction (True Negative [TN])**.\n",
    " 4. Patient(s) is (are) Diabetic (1), and the model predicted them as Diabetic (0).  **Wrong Prediction (False Negative [FN], Type II Error)**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " ##### Performance Metrics:\n",
    "\n",
    " Given the Confusion Metrics, we can calculate a range of metrics, such as:\n",
    " 1. **Precision** = $TP/(TP + FP)$\n",
    " 2. **Recall** = $TP/(TP + FN)$\n",
    " 3. **F1-score** = $2* (TP/(2TP + FP + FN))$ = $2 * ((Precision * Recall)/(Precision + Recall))$\n",
    "\n",
    "    \n",
    "\n",
    "**Precision** determines how precise/accurate the model is out of those predicted positives (i.e., Diabetic Patients). It basically measures how many of them are actually positive (i.e., Diabetic Patients).\n",
    "\n",
    "**Recall** calculates how many of the Actual Positives (Truly Diabetic Patients) our model identifies by labelling them as Positive (True Positive, i.e., Diabetic).\n",
    "\n",
    "**F1-score** is a balance between Precision and Recall. It is useful when there is an uneven class distribution (i.e., when there is a large number of Actual Negatives/Non-Diabetic against Actual Positives/Diabetic; or vice-versa)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Estimate the predictive performance given the metrics Precision, Recall and F1-Score, which are calculated using the Confusion Matrix below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "7w6lPO30zbNS"
   },
   "outputs": [],
   "source": [
    "utils.gen_train_test_performances(y_train, y_pred_train_dt, y_test, y_pred_test_dt, ['Not-Diabetic', 'Diabetic'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### What metric do you consider as important for predicting diabetes? How to define a good metric for it clinically?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### [**ROC AUC**](https://en.wikipedia.org/wiki/Receiver_operating_characteristic)\n",
    "It stands for the \"Area under the receiver operating characteristic (ROC) curve\".\n",
    "\n",
    "ROC AUC is a graphical plot that determines the diagnostic ability of a classification model considering its discrimination (probability) threshold is varied. \n",
    "\n",
    "The ROC AUC is the plot of the true positive rate against the false positive rate, at various threshold settings.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### How do the ROC AUC plots look like?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "LWK4tBzAzbNS"
   },
   "outputs": [],
   "source": [
    "utils.gen_train_test_roc(y_train, y_pred_train_dt, y_test, y_pred_test_dt, ['Not-Diabetic', 'Diabetic'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PJunlx9gzbNT"
   },
   "source": [
    "##### What does the decision tree looks like?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "E-dBBsTRzbNT"
   },
   "outputs": [],
   "source": [
    "fig, axes = plt.subplots(nrows=1 ,ncols=1, figsize=(9,6), dpi=300)\n",
    "tree.plot_tree(clf_dt, feature_names=X_train.columns, class_names=['Not-Diabetic', 'Diabetic'], filled=True)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 6,
     "status": "ok",
     "timestamp": 1688358319189,
     "user": {
      "displayName": "Alex de Sá",
      "userId": "12505277789962402202"
     },
     "user_tz": -600
    },
    "id": "QpFtcF6d_hPq"
   },
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "0PfXUQk_zbNT"
   },
   "source": [
    "##### Which features are more important for the model?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "3lB0f0rTzbNT"
   },
   "outputs": [],
   "source": [
    "utils.feat_importance(clf_dt)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### What can you conclude from the predictive performance of the Decision Tree? What can you say about the important features above?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### TO-DO: Go back and modify a few things for the Decision Tree. What happens with your results and derived conclusions?\n",
    "\n",
    "1. Change the percentage value for the testing set in **Data Splitting**. What are the outcomes?\n",
    "2. Change the **max_depth** when setting the decision tree. Please increase and reduce the **max_depth**. What happens? Does anything change visually? What about the predictive performance?\n",
    "3. Change the number of folds in the cross-validation (**cv**) procedure from 5 to 3 or 10. Why is setting this number important?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Second model: Random Forest"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A Random Forest ( RF) is also a tree-based model. However, it is an ensemble of multiple random decision trees of different kinds (considering different data and different features). The final value of the model is the average of all the prediction/estimates created by each individual decision tree."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Training the random forest on the set of features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What happens when you change the number of trees (n_estimators) in the random forest?\n",
    "clf_rf = ensemble.RandomForestClassifier(n_estimators=300, random_state=42)\n",
    "\n",
    "# What happens if you change the number of folds in the cross-validation from 5 to 3 (or 10)?\n",
    "y_pred_train_rf = cross_val_predict(clf_rf, X_train, y_train, cv=5)\n",
    "clf_rf.fit(X_train, y_train)\n",
    "y_pred_test_rf = clf_rf.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Training the decision tree on the set of features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "utils.gen_train_test_performances(y_train, y_pred_train_rf, y_test, y_pred_test_rf, ['Not-Diabetic', 'Diabetic'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### How do the ROC AUC plot look like?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "utils.gen_train_test_roc(y_train, y_pred_train_rf, y_test, y_pred_test_rf, ['Not-Diabetic', 'Diabetic'])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### Which features are more important for the model?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "utils.feat_importance(clf_rf)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "##### What can you conclude from the predictive performance of the Random Forest? What about its most important features? How can we compare the performance of Random Forest to the Decision Tree?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### TO-DO: Go back and modify a few things for the Random Forest. What happens with your predictive results and derived conclusions if you:\n",
    "\n",
    "1. Change the percentage value (**TEST_SIZE**) for the testing set in **Data Splitting** for training Random Forest. What are the outcomes?\n",
    "2. Change the **n_estimators** (number of trees) when setting the Random Forest. What happens?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    " "
   ]
  }
 ],
 "metadata": {
  "colab": {
   "provenance": [
    {
     "file_id": "https://github.com/carlosmr12/mlwk2023/blob/master/mlapp.ipynb",
     "timestamp": 1688355066303
    }
   ]
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}