spam_detector.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Spam Detector"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Retrieve the Data\n",
    "\n",
    "The data is located at [https://static.bc-edx.com/ai/ail-v-1-0/m13/challenge/spam-data.csv](https://static.bc-edx.com/ai/ail-v-1-0/m13/challenge/spam-data.csv)\n",
    "\n",
    "Dataset Source: [UCI Machine Learning Library](https://archive.ics.uci.edu/dataset/94/spambase)\n",
    "\n",
    "Import the data using Pandas. Display the resulting DataFrame to confirm the import was successful."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>word_freq_make</th>\n",
       "      <th>word_freq_address</th>\n",
       "      <th>word_freq_all</th>\n",
       "      <th>word_freq_3d</th>\n",
       "      <th>word_freq_our</th>\n",
       "      <th>word_freq_over</th>\n",
       "      <th>word_freq_remove</th>\n",
       "      <th>word_freq_internet</th>\n",
       "      <th>word_freq_order</th>\n",
       "      <th>word_freq_mail</th>\n",
       "      <th>...</th>\n",
       "      <th>char_freq_;</th>\n",
       "      <th>char_freq_(</th>\n",
       "      <th>char_freq_[</th>\n",
       "      <th>char_freq_!</th>\n",
       "      <th>char_freq_$</th>\n",
       "      <th>char_freq_#</th>\n",
       "      <th>capital_run_length_average</th>\n",
       "      <th>capital_run_length_longest</th>\n",
       "      <th>capital_run_length_total</th>\n",
       "      <th>spam</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.00</td>\n",
       "      <td>0.64</td>\n",
       "      <td>0.64</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.32</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>...</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.778</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>3.756</td>\n",
       "      <td>61</td>\n",
       "      <td>278</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.21</td>\n",
       "      <td>0.28</td>\n",
       "      <td>0.50</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.28</td>\n",
       "      <td>0.21</td>\n",
       "      <td>0.07</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.94</td>\n",
       "      <td>...</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.132</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.372</td>\n",
       "      <td>0.180</td>\n",
       "      <td>0.048</td>\n",
       "      <td>5.114</td>\n",
       "      <td>101</td>\n",
       "      <td>1028</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.06</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.71</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.23</td>\n",
       "      <td>0.19</td>\n",
       "      <td>0.19</td>\n",
       "      <td>0.12</td>\n",
       "      <td>0.64</td>\n",
       "      <td>0.25</td>\n",
       "      <td>...</td>\n",
       "      <td>0.01</td>\n",
       "      <td>0.143</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.276</td>\n",
       "      <td>0.184</td>\n",
       "      <td>0.010</td>\n",
       "      <td>9.821</td>\n",
       "      <td>485</td>\n",
       "      <td>2259</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.63</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.31</td>\n",
       "      <td>0.63</td>\n",
       "      <td>0.31</td>\n",
       "      <td>0.63</td>\n",
       "      <td>...</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.137</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.137</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>3.537</td>\n",
       "      <td>40</td>\n",
       "      <td>191</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.63</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.31</td>\n",
       "      <td>0.63</td>\n",
       "      <td>0.31</td>\n",
       "      <td>0.63</td>\n",
       "      <td>...</td>\n",
       "      <td>0.00</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.135</td>\n",
       "      <td>0.000</td>\n",
       "      <td>0.000</td>\n",
       "      <td>3.537</td>\n",
       "      <td>40</td>\n",
       "      <td>191</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>5 rows × 58 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "   word_freq_make  word_freq_address  word_freq_all  word_freq_3d  \\\n",
       "0            0.00               0.64           0.64           0.0   \n",
       "1            0.21               0.28           0.50           0.0   \n",
       "2            0.06               0.00           0.71           0.0   \n",
       "3            0.00               0.00           0.00           0.0   \n",
       "4            0.00               0.00           0.00           0.0   \n",
       "\n",
       "   word_freq_our  word_freq_over  word_freq_remove  word_freq_internet  \\\n",
       "0           0.32            0.00              0.00                0.00   \n",
       "1           0.14            0.28              0.21                0.07   \n",
       "2           1.23            0.19              0.19                0.12   \n",
       "3           0.63            0.00              0.31                0.63   \n",
       "4           0.63            0.00              0.31                0.63   \n",
       "\n",
       "   word_freq_order  word_freq_mail  ...  char_freq_;  char_freq_(  \\\n",
       "0             0.00            0.00  ...         0.00        0.000   \n",
       "1             0.00            0.94  ...         0.00        0.132   \n",
       "2             0.64            0.25  ...         0.01        0.143   \n",
       "3             0.31            0.63  ...         0.00        0.137   \n",
       "4             0.31            0.63  ...         0.00        0.135   \n",
       "\n",
       "   char_freq_[  char_freq_!  char_freq_$  char_freq_#  \\\n",
       "0          0.0        0.778        0.000        0.000   \n",
       "1          0.0        0.372        0.180        0.048   \n",
       "2          0.0        0.276        0.184        0.010   \n",
       "3          0.0        0.137        0.000        0.000   \n",
       "4          0.0        0.135        0.000        0.000   \n",
       "\n",
       "   capital_run_length_average  capital_run_length_longest  \\\n",
       "0                       3.756                          61   \n",
       "1                       5.114                         101   \n",
       "2                       9.821                         485   \n",
       "3                       3.537                          40   \n",
       "4                       3.537                          40   \n",
       "\n",
       "   capital_run_length_total  spam  \n",
       "0                       278     1  \n",
       "1                      1028     1  \n",
       "2                      2259     1  \n",
       "3                       191     1  \n",
       "4                       191     1  \n",
       "\n",
       "[5 rows x 58 columns]"
      ]
     },
     "execution_count": 67,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Import the data\n",
    "data = pd.read_csv(\"https://static.bc-edx.com/ai/ail-v-1-0/m13/challenge/spam-data.csv\")\n",
    "data.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predict Model Performance\n",
    "\n",
    "You will be creating and comparing two models on this data: a Logistic Regression, and a Random Forests Classifier. Before you create, fit, and score the models, make a prediction as to which model you think will perform better. You do not need to be correct! \n",
    "\n",
    "Write down your prediction in the designated cells in your Jupyter Notebook, and provide justification for your educated guess."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based on accuracy of prior practice activities, I'd expect Random Forest Classifier to have a higher accuracy. Other reasons the Random Forest Classifier is expected to be more accurate than the Logistic Regression can be found in this [article](https://www.geeksforgeeks.org/logistic-regression-vs-random-forest-classifier/). With 58 columns (57 features) a model that handles high-dimensional data would be better. Also, with this data the decision boundary may not necessarily be linear. Random Forest is better able to handle nonlinear decision boundaries."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Split the Data into Training and Testing Sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create the labels set `y` and features DataFrame `X`\n",
    "y = data['spam']\n",
    "X = data.drop('spam', axis='columns')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "spam\n",
       "0    2788\n",
       "1    1813\n",
       "Name: count, dtype: int64"
      ]
     },
     "execution_count": 69,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check the balance of the labels variable (`y`) by using the `value_counts` function.\n",
    "y.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split the data into X_train, X_test, y_train, y_test\n",
    "# NOTE: accuracy scores differ slightly from the starter code. I could not find a random_state for the split that would match\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scale the Features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the `StandardScaler` to scale the features data. Remember that only `X_train` and `X_test` DataFrames should be scaled."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "# Create the StandardScaler instance\n",
    "scaler = StandardScaler()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Fit the Standard Scaler with the training data\n",
    "scaler = scaler.fit(X_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Scale the training data\n",
    "X_train_scaled = scaler.transform(X_train)\n",
    "X_test_scaled = scaler.transform(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create and Fit a Logistic Regression Model\n",
    "\n",
    "Create a Logistic Regression model, fit it to the training data, make predictions with the testing data, and print the model's accuracy score. You may choose any starting settings you like. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train a Logistic Regression model and print the model score\n",
    "from sklearn.linear_model import LogisticRegression\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 0, 0, ..., 0, 1, 0])"
      ]
     },
     "execution_count": 75,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Make and save testing predictions with the saved logistic regression model using the test data\n",
    "lr_model = LogisticRegression(random_state=1)\n",
    "lr_model.fit(X_train_scaled, y_train)\n",
    "\n",
    "# Review the predictions\n",
    "lr_testing_predictions = lr_model.predict(X_test_scaled)\n",
    "lr_testing_predictions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9226759339704604"
      ]
     },
     "execution_count": 76,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Calculate the accuracy score by evaluating `y_test` vs. `testing_predictions`.\n",
    "accuracy_score(y_test, lr_testing_predictions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create and Fit a Random Forest Classifier Model\n",
    "\n",
    "Create a Random Forest Classifier model, fit it to the training data, make predictions with the testing data, and print the model's accuracy score. You may choose any starting settings you like. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train a Random Forest Classifier model and print the model score\n",
    "from sklearn.ensemble import RandomForestClassifier\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Make and save testing predictions with the saved logistic regression model using the test data\n",
    "rfc_model = RandomForestClassifier(random_state=1)\n",
    "rfc_model.fit(X_train_scaled, y_train)\n",
    "\n",
    "# Review the predictions\n",
    "rfc_testing_predictions = rfc_model.predict(X_test_scaled)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.9600347523892268"
      ]
     },
     "execution_count": 79,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Calculate the accuracy score by evaluating `y_test` vs. `testing_predictions`.\n",
    "accuracy_score(y_test, rfc_testing_predictions)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluate the Models\n",
    "\n",
    "Which model performed better? How does that compare to your prediction? Write down your results and thoughts in the following markdown cell."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With Logistic Regression scoring around 0.922 accuracy and Random Forest Classifier scoring about 0.960, Random Forest was more accurate, which matched my prediction. Though the results were not surprising, for a more robust comparison of the two models we might consider (1) tuning the hyperparameters, for example, C and/or penalty for Logistic Regression, n_estimators for Random Forest, (2) check K-Fold Cross Validation to see if we get different scores with different samples, and (3) view the Confusion Matrix and ROC AUC to break down the accuracy score into true/false positives/negatives."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}