7_megaline_classification_prediction_model.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "596f9ec8",
   "metadata": {},
   "source": [
    "# Contenido<a id='contenido'></a>\n",
    "* [Introducción](#int)\n",
    "* [Iniciación de datos](#ini)\n",
    "    *  [Examina datos](#exa)\n",
    "    *  [Descripción de datos](#des)\n",
    "* [Machine Learning](#ml)\n",
    "    *  [Segmentación de datos](#seg)\n",
    "    *  [Entrenamiento](#ent)\n",
    "        * [Algoritmo DecisionTreeClassifier](#alg1)\n",
    "        * [Algoritmo RandomForestClassifier](#alg2)\n",
    "        * [Algoritmo LogisticRegression](#alg3)\n",
    "    *  [Calidad del modelo](#cal)\n",
    "    *  [Prueba de cordura](#pru)\n",
    "* [Conclusión general](#con)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6d368af",
   "metadata": {},
   "source": [
    "# Introducción<a id='intr'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "621dbceb",
   "metadata": {},
   "source": [
    "La compañía móvil Megaline no está satisfecha al ver que muchos de sus clientes utilizan planes heredados. Quieren desarrollar un modelo que pueda analizar el comportamiento de los clientes y recomendar uno de los nuevos planes de Megaline: **Smart o Ultra**. Tienes acceso a los datos de comportamiento de los suscriptores que ya se han cambiado a los planes nuevos. \n",
    "\n",
    "Para esta tarea de clasificación vamos a crear un modelo que escoja el plan correcto. \n",
    "\n",
    "Desarrollaremos un modelo con la mayor exactitud posible con al menos un umbral de exactitud del 75%. Usaremos el dataset para comprobar la exactitud.\n",
    "\n",
    "## Objetivo\n",
    "\n",
    "El objetivo es obtener un modelo capaz de predecir el tipo de plan Ultra o Smart que necesitan los usuarios."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "848bc149",
   "metadata": {},
   "source": [
    "# Iniciación de datos<a id='ini'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8caf2ad4",
   "metadata": {},
   "source": [
    "Importamos las librerias:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "10d4ab3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# librerias generales\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "\n",
    "# machine learning\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import accuracy_score"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6262b7bc",
   "metadata": {},
   "source": [
    "Cargamos el dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "7e37678e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# carga de datos\n",
    "try:\n",
    "    users = pd.read_csv('users_behavior.csv')\n",
    "except:\n",
    "    users = pd.read_csv('datasets/users_behavior.csv')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "159fddb4",
   "metadata": {},
   "source": [
    "## Examina datos<a id='exa'></a>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "7ea42ae9",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>calls</th>\n",
       "      <th>minutes</th>\n",
       "      <th>messages</th>\n",
       "      <th>mb_used</th>\n",
       "      <th>is_ultra</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>40.0</td>\n",
       "      <td>311.90</td>\n",
       "      <td>83.0</td>\n",
       "      <td>19915.42</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>85.0</td>\n",
       "      <td>516.75</td>\n",
       "      <td>56.0</td>\n",
       "      <td>22696.96</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>77.0</td>\n",
       "      <td>467.66</td>\n",
       "      <td>86.0</td>\n",
       "      <td>21060.45</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>106.0</td>\n",
       "      <td>745.53</td>\n",
       "      <td>81.0</td>\n",
       "      <td>8437.39</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>66.0</td>\n",
       "      <td>418.74</td>\n",
       "      <td>1.0</td>\n",
       "      <td>14502.75</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   calls  minutes  messages   mb_used  is_ultra\n",
       "0   40.0   311.90      83.0  19915.42         0\n",
       "1   85.0   516.75      56.0  22696.96         0\n",
       "2   77.0   467.66      86.0  21060.45         0\n",
       "3  106.0   745.53      81.0   8437.39         1\n",
       "4   66.0   418.74       1.0  14502.75         0"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "users.head()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "12701229",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 3214 entries, 0 to 3213\n",
      "Data columns (total 5 columns):\n",
      " #   Column    Non-Null Count  Dtype  \n",
      "---  ------    --------------  -----  \n",
      " 0   calls     3214 non-null   float64\n",
      " 1   minutes   3214 non-null   float64\n",
      " 2   messages  3214 non-null   float64\n",
      " 3   mb_used   3214 non-null   float64\n",
      " 4   is_ultra  3214 non-null   int64  \n",
      "dtypes: float64(4), int64(1)\n",
      "memory usage: 125.7 KB\n"
     ]
    }
   ],
   "source": [
    "users.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "10a24116",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "users.duplicated().sum()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac2232d8",
   "metadata": {},
   "source": [
    "El dataset contiene 3214 registros y 5 columnas, los tipos de datos son correctos, no tenemos valores ausentes ni registros duplicados."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f234abdd",
   "metadata": {},
   "source": [
    "## Descripción de los datos <a id='des'></a>\n",
    "\n",
    "\n",
    "Cada observación en el dataset contiene información del comportamiento mensual sobre un usuario. La información dada es la siguiente:\n",
    "\n",
    "- `сalls`: número de llamadas\n",
    "- `minutes`: duración total de la llamada en minutos\n",
    "- `messages`: número de mensajes de texto\n",
    "- `mb_used`: Tráfico de Internet utilizado en MB\n",
    "- `is_ultra`: plan para el mes actual (Ultra - 1, Smart - 0)\n",
    "\n",
    "La columna is_ultra será nuestra variable dependiente / objetivo y nuestras variables independientes serán las otras columnas calls, minutes, messages y mb_used."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32f6437e",
   "metadata": {},
   "source": [
    "# Machine Learning<a id='ml'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f5fc61d5",
   "metadata": {},
   "source": [
    "## Segmentación de datos\n",
    "\n",
    "Antes que todo necesitamos segmentar nuestro dataset `users` en 3 partes con proporciones 3:1:1 de la siguiente manera:\n",
    "\n",
    "- 60% de datos para entrenamiento\n",
    "- 20% de datos para validación\n",
    "- 20% de datos para prueba"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "e53e83ef",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "dataset (3214, 5)\n",
      "features_train: (1928, 4)\n",
      "target_train: (1928,)\n",
      "features_valid: (643, 4)\n",
      "target_valid: (643,)\n",
      "features_test: (643, 4)\n",
      "target_test: (643,)\n"
     ]
    }
   ],
   "source": [
    "# features variable independiente/caracteristicas\n",
    "features = users.drop('is_ultra', axis=1)\n",
    "\n",
    "# target variable dependiente/objetivo\n",
    "target = users['is_ultra']\n",
    "\n",
    "# Segmentación de datos entrenamiento 60%, validación 20% y prueba 20%\n",
    "features_train, features_valid, target_train, target_valid = train_test_split(features, target, test_size=0.40, random_state=12345)\n",
    "features_valid, features_test, target_valid, target_test = train_test_split(features_valid, target_valid, test_size=0.50, random_state=12345)\n",
    "\n",
    "# Tamaño de los segmentos de datos\n",
    "print('dataset', users.shape)\n",
    "print('features_train:', features_train.shape)\n",
    "print('target_train:', target_train.shape)\n",
    "\n",
    "print('features_valid:', features_valid.shape)\n",
    "print('target_valid:', target_valid.shape)\n",
    "\n",
    "print('features_test:', features_test.shape)\n",
    "print('target_test:', target_test.shape)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8aaf083",
   "metadata": {},
   "source": [
    "## Entrenamiento<a id='ent'></a>\n",
    "\n",
    "Realizaremos la creación de modelos usando 3 algoritmos de entrenamiento para clasificación e hiperparámetros que nos permitiran aprender de los datos y predecir nuevas observaciones.\n",
    "\n",
    "Usaremos:\n",
    " \n",
    "- Arbol de decision `DecisionTreeClassifier`, usaremos `max_depth` para determinar varias profundidades.\n",
    "- Bosque aleatorio `RamdomForestClassifier`, usaremos `n_stimators` para determinar varios arboles y 'max_depth' varias profundidades.\n",
    "- Regresion logística `LogisticRegression`, usaremos 'solver-liblinear', random_state=12345."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "991ca849",
   "metadata": {},
   "source": [
    "### Algoritmo DecisionTreeClassifier<a id='alg1'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e77e134",
   "metadata": {},
   "source": [
    "Vamos a entrenar un modelo con el algoritmo de Arbol de decision. Los hiperparámetros a usar son: \n",
    "- ramdom_state=12345\n",
    "- max_depth= 1 a 5"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "64c5480a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "depth 1\n",
      "Train data accuracy  0.7577800829875518\n",
      "Valid data accuracy  0.7542768273716952\n",
      "\n",
      "depth 2\n",
      "Train data accuracy  0.7878630705394191\n",
      "Valid data accuracy  0.7822706065318819\n",
      "\n",
      "depth 3\n",
      "Train data accuracy  0.8075726141078838\n",
      "Valid data accuracy  0.7853810264385692\n",
      "\n",
      "depth 4\n",
      "Train data accuracy  0.8106846473029046\n",
      "Valid data accuracy  0.7791601866251944\n",
      "\n",
      "depth 5\n",
      "Train data accuracy  0.8200207468879668\n",
      "Valid data accuracy  0.7791601866251944\n",
      "\n",
      "RESULT: Best model with DecisionTreeClassifier on the validation dataset with depth = 3 and accuracy 0.7853810264385692\n"
     ]
    }
   ],
   "source": [
    "# Arbol de decision con profundidad 10\n",
    "best_est = 0\n",
    "best_score = 0\n",
    "\n",
    "for depth in range(1, 6):\n",
    "    print('depth', depth)\n",
    "\n",
    "    # creación de modelo de entrenamiento mediante arbol de decisión\n",
    "    model = DecisionTreeClassifier(random_state=12345, max_depth=depth)\n",
    "\n",
    "    # entrena el modelo con el conjunto de entrenamiento\n",
    "    model.fit(features_train, target_train)\n",
    "\n",
    "    # predicción para datos de entrenamiento\n",
    "    prediction_train = model.predict(features_train)\n",
    "    score_train = accuracy_score(target_train, prediction_train)\n",
    "    print('Train data accuracy ', score_train)\n",
    "\n",
    "    # prediccion para datos de valiación\n",
    "    prediction_valid = model.predict(features_valid)\n",
    "    score = accuracy_score(target_valid, prediction_valid)\n",
    "    print('Valid data accuracy ', score)\n",
    "    print()\n",
    "\n",
    "    if score > best_score:\n",
    "        best_score = score\n",
    "        best_est = depth\n",
    "\n",
    "\n",
    "print('RESULT: Best model with DecisionTreeClassifier on the validation dataset with depth = {} and accuracy {}'.format(best_est, best_score))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e4de1fef",
   "metadata": {},
   "source": [
    "Hemos realizado nuestro entrenamiento mediante clasificación por árbol de decision `DecisionTreeClassifier` variando la profundidad del árbol de 1 a 10. Obtuvimos que el mejor modelo tiene profundidad de 3 y su exactitud es del 78%. Además pudimos observar el comportamiento de las predicciones con los datos de entrenamiento y con los datos de validación, podemos notar que entre más profundidad le demos al árbol empezará a mostrarse un sobreajuste en los datos de entrenamiento, ya que los datos de entrenamientos tienen un mayor valor de precisión respecto a las predicciones realizadas en el conjunto de datos de validación. "
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "6fd2761b",
   "metadata": {},
   "source": [
    "### Algoritmo RandomForestClassifier<a id='alg2'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "64cef8ce",
   "metadata": {},
   "source": [
    "Usaremos el algoritmo de bosque aleatorio, con los hiperparámetros: \n",
    "- ramdom_state=12345\n",
    "- n_stimators= 10 a 20\n",
    "- max_depth= 1 a 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "e4aefb3c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---------- 10 tree - depth 1 ----------\n",
      "Train data accuracy 0.7442946058091287\n",
      "Valid data accuracy 0.7558320373250389\n",
      "\n",
      "---------- 10 tree - depth 2 ----------\n",
      "Train data accuracy 0.7785269709543569\n",
      "Valid data accuracy 0.7776049766718507\n",
      "\n",
      "---------- 10 tree - depth 3 ----------\n",
      "Train data accuracy 0.8101659751037344\n",
      "Valid data accuracy 0.7853810264385692\n",
      "\n",
      "---------- 11 tree - depth 1 ----------\n",
      "Train data accuracy 0.7448132780082988\n",
      "Valid data accuracy 0.7542768273716952\n",
      "\n",
      "---------- 11 tree - depth 2 ----------\n",
      "Train data accuracy 0.7883817427385892\n",
      "Valid data accuracy 0.7853810264385692\n",
      "\n",
      "---------- 11 tree - depth 3 ----------\n",
      "Train data accuracy 0.8101659751037344\n",
      "Valid data accuracy 0.7838258164852255\n",
      "\n",
      "---------- 12 tree - depth 1 ----------\n",
      "Train data accuracy 0.7442946058091287\n",
      "Valid data accuracy 0.7527216174183515\n",
      "\n",
      "---------- 12 tree - depth 2 ----------\n",
      "Train data accuracy 0.7883817427385892\n",
      "Valid data accuracy 0.7838258164852255\n",
      "\n",
      "---------- 12 tree - depth 3 ----------\n",
      "Train data accuracy 0.8096473029045643\n",
      "Valid data accuracy 0.7869362363919129\n",
      "\n",
      "---------- 13 tree - depth 1 ----------\n",
      "Train data accuracy 0.7442946058091287\n",
      "Valid data accuracy 0.7527216174183515\n",
      "\n",
      "---------- 13 tree - depth 2 ----------\n",
      "Train data accuracy 0.7894190871369294\n",
      "Valid data accuracy 0.7884914463452566\n",
      "\n",
      "---------- 13 tree - depth 3 ----------\n",
      "Train data accuracy 0.8096473029045643\n",
      "Valid data accuracy 0.7884914463452566\n",
      "\n",
      "---------- 14 tree - depth 1 ----------\n",
      "Train data accuracy 0.7453319502074689\n",
      "Valid data accuracy 0.7527216174183515\n",
      "\n",
      "---------- 14 tree - depth 2 ----------\n",
      "Train data accuracy 0.7889004149377593\n",
      "Valid data accuracy 0.7838258164852255\n",
      "\n",
      "---------- 14 tree - depth 3 ----------\n",
      "Train data accuracy 0.8091286307053942\n",
      "Valid data accuracy 0.7884914463452566\n",
      "\n",
      "---------- 15 tree - depth 1 ----------\n",
      "Train data accuracy 0.7442946058091287\n",
      "Valid data accuracy 0.7527216174183515\n",
      "\n",
      "---------- 15 tree - depth 2 ----------\n",
      "Train data accuracy 0.7883817427385892\n",
      "Valid data accuracy 0.7838258164852255\n",
      "\n",
      "---------- 15 tree - depth 3 ----------\n",
      "Train data accuracy 0.8101659751037344\n",
      "Valid data accuracy 0.7869362363919129\n",
      "\n",
      "---------- 16 tree - depth 1 ----------\n",
      "Train data accuracy 0.745850622406639\n",
      "Valid data accuracy 0.7527216174183515\n",
      "\n",
      "---------- 16 tree - depth 2 ----------\n",
      "Train data accuracy 0.7883817427385892\n",
      "Valid data accuracy 0.7838258164852255\n",
      "\n",
      "---------- 16 tree - depth 3 ----------\n",
      "Train data accuracy 0.8096473029045643\n",
      "Valid data accuracy 0.7869362363919129\n",
      "\n",
      "---------- 17 tree - depth 1 ----------\n",
      "Train data accuracy 0.7536307053941909\n",
      "Valid data accuracy 0.7651632970451011\n",
      "\n",
      "---------- 17 tree - depth 2 ----------\n",
      "Train data accuracy 0.7878630705394191\n",
      "Valid data accuracy 0.7822706065318819\n",
      "\n",
      "---------- 17 tree - depth 3 ----------\n",
      "Train data accuracy 0.8091286307053942\n",
      "Valid data accuracy 0.7853810264385692\n",
      "\n",
      "---------- 18 tree - depth 1 ----------\n",
      "Train data accuracy 0.7520746887966805\n",
      "Valid data accuracy 0.7651632970451011\n",
      "\n",
      "---------- 18 tree - depth 2 ----------\n",
      "Train data accuracy 0.7883817427385892\n",
      "Valid data accuracy 0.7822706065318819\n",
      "\n",
      "---------- 18 tree - depth 3 ----------\n",
      "Train data accuracy 0.8096473029045643\n",
      "Valid data accuracy 0.7853810264385692\n",
      "\n",
      "---------- 19 tree - depth 1 ----------\n",
      "Train data accuracy 0.7525933609958506\n",
      "Valid data accuracy 0.7651632970451011\n",
      "\n",
      "---------- 19 tree - depth 2 ----------\n",
      "Train data accuracy 0.7878630705394191\n",
      "Valid data accuracy 0.7822706065318819\n",
      "\n",
      "---------- 19 tree - depth 3 ----------\n",
      "Train data accuracy 0.8101659751037344\n",
      "Valid data accuracy 0.7853810264385692\n",
      "\n",
      "---------- 20 tree - depth 1 ----------\n",
      "Train data accuracy 0.7588174273858921\n",
      "Valid data accuracy 0.7667185069984448\n",
      "\n",
      "---------- 20 tree - depth 2 ----------\n",
      "Train data accuracy 0.7883817427385892\n",
      "Valid data accuracy 0.7838258164852255\n",
      "\n",
      "---------- 20 tree - depth 3 ----------\n",
      "Train data accuracy 0.8096473029045643\n",
      "Valid data accuracy 0.7869362363919129\n",
      "\n",
      "RESULT: Best model with RandomForestClassifier on the validation dataset with trees:13-depth:2 and accuracy 0.7884914463452566\n"
     ]
    }
   ],
   "source": [
    "# Bosque aleatorio\n",
    "best_score = 0\n",
    "best_tree = 0\n",
    "best_depth = 0\n",
    "\n",
    "# Variar el número de arboles tree y la profundidad de cada arbol depth\n",
    "for tree in range(10, 21):\n",
    "    for depth in range(1, 4):\n",
    "        \n",
    "        print('----------', tree,'tree -', 'depth', depth,'----------')\n",
    "\n",
    "        # creación del modelo de entrenamiento mediante bosque aleatorio\n",
    "        model = RandomForestClassifier(\n",
    "            random_state=12345, n_estimators=tree, max_depth=depth)\n",
    "\n",
    "        # entrena el modelo con el conjunto de entrenamiento\n",
    "        model.fit(features_train, target_train)\n",
    "\n",
    "        # prediccion para datos de entrenamiento\n",
    "        score_train = model.score(features_train, target_train)\n",
    "        print('Train data accuracy', score_train)\n",
    "\n",
    "        # prediccion para datos de valiación\n",
    "        score = model.score(features_valid, target_valid)\n",
    "        print('Valid data accuracy', score)\n",
    "        print()\n",
    "\n",
    "        if score > best_score:\n",
    "            best_score = score\n",
    "            best_tree = tree\n",
    "            best_depth = depth\n",
    "\n",
    "print('RESULT: Best model with RandomForestClassifier on the validation dataset with trees:{}-depth:{} and accuracy {}'.format(best_tree, best_depth, best_score))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b957c746",
   "metadata": {},
   "source": [
    "Hemos probado varios modelos de entrenamiento con el algoritmo bosque aleatorio `RandomForestClassifier` y obtuvimos el mejor modelo obtuvo un 79% de precisión usando 13 árboles con profundidad de 2. Además notamos que también se obtuvo un 79% de presición con el conjunto de datos de entremiento, por lo que hemos controlado que no exista un sobreajuste en nuestro modelo y que en el modelo anterior si ocurría."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "38a78222",
   "metadata": {},
   "source": [
    "### Algoritmo LogisticRegression<a id='alg3'></a>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "606c4593",
   "metadata": {},
   "source": [
    "El último algoritmo de aprendizaje que probaremos es el de regresión logística. Los hiperparámetros a usar son: \n",
    "- ramdom_state=12345\n",
    "- solver= liblinear"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "d0bf4535",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the Logistic Regression Model in the training dataset 0.7510373443983402\n",
      "Accuracy of the Logistic Regression Model in the validation dataset: 0.7573872472783826\n"
     ]
    }
   ],
   "source": [
    "# Regresion Logística\n",
    "model = LogisticRegression(random_state=12345, solver='liblinear')\n",
    "\n",
    "# entrena el modelo con el conjunto de entrenamiento\n",
    "model.fit(features_train, target_train)\n",
    "\n",
    "# calcula la precisión de la predicción con los datos de entrenamiento y luego con los datos de validación\n",
    "score_train = model.score(features_train, target_train)\n",
    "score_valid = model.score(features_valid, target_valid)\n",
    "\n",
    "print(\"Accuracy of the Logistic Regression Model in the training dataset\", score_train)\n",
    "print(\"Accuracy of the Logistic Regression Model in the validation dataset:\", score_valid)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7bf4e27a",
   "metadata": {},
   "source": [
    "Con la regresión logística tenemos que nuestro modelo obtuvo un 76% de exactitud, siendo menor que los modelos anteriores. También podemos observar que la predicción con el conjunto de entrenamiento no se adelanta a la predición con el conjunto de validación, es decir no existe un sobreajuste por parte de los datos de entrenamiento."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d447d76b",
   "metadata": {},
   "source": [
    "De acuerdo a los modelos revisados anteriormente obtuvimos que el mejor modelo tiene las siguientes características:\n",
    "\n",
    "\n",
    "    - Algoritmo de aprendizaje: RandomForestClassifier\n",
    "    - hiperparametros a usar 13 arboles y profundidad 2."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1fc5917",
   "metadata": {},
   "source": [
    "## Calidad del modelo <a id='cal'></a>\n",
    "Comprobaremos la calidad de nuestro mejor modelo usando ahora usando nuestro conjunto de prueba."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "6a81b262",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the best model RandomForestClassifier on the test dataset 0.7744945567651633\n"
     ]
    }
   ],
   "source": [
    "# creación del modelo de entrenamiento\n",
    "best_model = RandomForestClassifier(random_state=12345, n_estimators=13, max_depth=2)\n",
    "\n",
    "# entrena el modelo\n",
    "best_model.fit(features_train, target_train)\n",
    "\n",
    "# Guardamos nuestras prediciones del modelo con los datos de prueba\n",
    "predictions_test = best_model.predict(features_test)\n",
    "\n",
    "# Métrica de evaluación del modelo\n",
    "score_test = accuracy_score(target_test, predictions_test)\n",
    "print('Accuracy of the best model RandomForestClassifier on the test dataset', score_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dc54dc3f",
   "metadata": {},
   "source": [
    "Hemos usado nuestro modelo (RandomForestClassifier con 13 arboles con profundidad de 2) y obtuvimos las predicciones del dataset de prueba obteniendo un 77% de exactitud, siendo este 2 puntos porcentuales menor que lo obtenido en nuestro dataset de validación."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "51122870",
   "metadata": {},
   "source": [
    "## Prueba de cordura <a id='pru'> </a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "9eed4fa8",
   "metadata": {},
   "source": [
    "Para demostrar que nuestro modelo funciona mejor que la casualidad, realizaremos una prueba de cordura. Esta prueba buscará problemas de clasificación en nuestro modelo por lo que vamos a comparar nuestras predicciones con prediciones basadas en la aleatoriedad y demostrar que nuestro modelo es mejor.\n",
    "\n",
    "Dicho esto, simularemos respuestas de manera aleatoria para evaluar su precisión, entonces crearemos una serie del mismo tamaño de observaciones que nuestro conjunto Objetivo de prueba y  rellenaremos con 0 y 1 de forma aleatoria.  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "01a0a562",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1    328\n",
      "0    315\n",
      "dtype: int64\n",
      "\n",
      "0      0\n",
      "1      1\n",
      "2      1\n",
      "3      1\n",
      "4      0\n",
      "      ..\n",
      "638    0\n",
      "639    1\n",
      "640    1\n",
      "641    0\n",
      "642    1\n",
      "Length: 643, dtype: int32\n"
     ]
    }
   ],
   "source": [
    "# random.seed nos permite establecer como parametro '12345' como generador de números pseudoaleatorios\n",
    "np.random.seed(12345)\n",
    "\n",
    "# Creamos una serie de valores 1 y 0 aleatorias del tamaño de nuestro target de prueba\n",
    "random_test =  pd.Series(np.random.choice([0, 1], size=len(target_test)))\n",
    "\n",
    "# Revisamos como quedaron distribuidos nuestros 0 y 1\n",
    "print(random_test.value_counts(dropna=False))\n",
    "print()\n",
    "print(random_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9ddaf9f8",
   "metadata": {},
   "source": [
    "Como vemos `random_test` contiene respuestas 0 y 1 de manera aleatoria. Para concretar nuestra prueba de cordura vamos a medir la exactitud de las predicciones aleatorias"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "933d497f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy of the random responses: 0.4821150855365474\n",
      "Accuracy of the model RandomForestClassifier 0.7744945567651633\n"
     ]
    }
   ],
   "source": [
    "print('Accuracy of the random responses:', accuracy_score(target_test, random_test))\n",
    "print('Accuracy of the model RandomForestClassifier', score_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b09bdf39",
   "metadata": {},
   "source": [
    "Hemos medido la exactitud de las respuestas aleatorias y obtuvimos un 48%, claramente nuestro modelo de predicción es mejor ya que tenemos un 77% de exactitud, es decir nuestro modelo ha pasado la prueba de cordura ya que predice mejor que predicciones al azar."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0113ff92",
   "metadata": {},
   "source": [
    "# Conclusión general<a id='con'></a>"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "1a7d75b6",
   "metadata": {},
   "source": [
    "Realizamos una investigación sobre los algoritmos de clasificación para crear modelos predictivos variando sus hiperparámetros. De lo cual obtuvimos:\n",
    "\n",
    "- Con `DecisionTreeClassifier` variamos la profundidad del árbol de 1 a 10. Obtuvimos que el mejor modelo tiene profundidad de 3 y su exactitud es del 78%, sin embargo las predicciones en el conjunto de entrenamiento estuvo sobreajustado.\n",
    "\n",
    "- Con `RandomForestClassifier` obtuvimos el mejor modelo con un 79% de precisión usando 13 árboles con profundidad de 2, las predicciones en el conjunto de entrenamiento no presentó sobreajuste por lo que es un buen indicativo.\n",
    "\n",
    "- Con `LogisticRegression` nuestro modelo obtuvo un 76% de exactitud, siendo menor que los modelos anteriores. También podemos observar que la predicción con el conjunto de entrenamiento no se adelanta a la predición con el conjunto de validación, es decir no existe un sobreajuste por parte de los datos de entrenamiento.\n",
    "\n",
    "Por lo tanto, podemos determinar que el modelo de predicción usa el algoritmo de aprendizaje de bosque aleatorio `RandomForestClassifier`. El bosque está compuesta de 13 árboles con profundidad de 2. La calidad del modelo fue medida mediante la exactitud del 77% superando el umbral de exactitud planteado inicialmente en el objetivo de este análisis y además superando la prueba de cordura.\n",
    "\n",
    "Hemos obtenido un modelo capaz de predecir el tipo de plan (Ultra o Smart) que necesitan los usuarios de la compañía Megaline. \n",
    "\n"
   ]
  }
 ],
 "metadata": {
  "hide_input": false,
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.15"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": true,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  },
  "vscode": {
   "interpreter": {
    "hash": "d8acedbd0dbe0ef603ed509b87f45f14acec3adee6d1fcd590fdbbdea64891c7"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}