04-Implementing-New-Test.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sentiment Analysis - Shopee Code League 2020\n",
    "\n",
    "## 4.0 Implementing New Trained Model\n",
    "\n",
    "### Introduction\n",
    "After reviewing the task from [Shopee Code League 2020 - Sentiment Analysis](https://www.kaggle.com/c/student-shopee-code-league-sentiment-analysis/overview), our team has built and trained the earlier model. \n",
    "\n",
    "We will now implement these models for the actual submission\n",
    "\n",
    "#### Team Introduction\n",
    "Team Name: **JNNY** <br/>\n",
    "Team Members: **Natalie, James, Yong Xian, Nicky** <br/>\n",
    "Script Prepared by: **Nicky** [@ahjimomo](https://github.com/ahjimomo)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.1 Library & Data Import"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np \n",
    "import pandas as pd\n",
    "import pickle\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "import re"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>review_id</th>\n",
       "      <th>review</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Great danger, cool, motif and cantik2 jg model...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>One of the shades don't fit well</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Very comfortable</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Fast delivery. Product expiry is on Dec 2022. ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>it's sooooo cute! i like playing with the glit...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60422</th>\n",
       "      <td>60423</td>\n",
       "      <td>Product has been succesfully ordered and shipp...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60423</th>\n",
       "      <td>60424</td>\n",
       "      <td>Opening time a little scared. Fear dalemnya de...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60424</th>\n",
       "      <td>60425</td>\n",
       "      <td>The product quality is excellent. The origina...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60425</th>\n",
       "      <td>60426</td>\n",
       "      <td>They 're holding up REALLY well also .</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>60426</th>\n",
       "      <td>60427</td>\n",
       "      <td>Rapid response and detail ...\\nThanks gan, the...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>60427 rows × 2 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       review_id                                             review\n",
       "0              1  Great danger, cool, motif and cantik2 jg model...\n",
       "1              2                   One of the shades don't fit well\n",
       "2              3                                   Very comfortable\n",
       "3              4  Fast delivery. Product expiry is on Dec 2022. ...\n",
       "4              5  it's sooooo cute! i like playing with the glit...\n",
       "...          ...                                                ...\n",
       "60422      60423  Product has been succesfully ordered and shipp...\n",
       "60423      60424  Opening time a little scared. Fear dalemnya de...\n",
       "60424      60425   The product quality is excellent. The origina...\n",
       "60425      60426             They 're holding up REALLY well also .\n",
       "60426      60427  Rapid response and detail ...\\nThanks gan, the...\n",
       "\n",
       "[60427 rows x 2 columns]"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Importing the dataset\n",
    "\n",
    "test_raw = pd.read_csv('input/test.csv')\n",
    "\n",
    "test_raw # 60427 reviews imported"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.2 Data Cleaning & Preparation\n",
    "\n",
    "Similarly to how we prepare the date for training of the model, we will prepare the data for the test model as well.\n",
    "\n",
    "**1. Basic text pre-processing techniques with [Python Regular Expressions](https://developers.google.com/edu/python/regular-expressions) to clean the data** <br/>\n",
    "    - Removing regular expressions\n",
    "    - Removing single characters\n",
    "    - Removing words containing numbers\n",
    "    - Removing multiple whitespaces\n",
    "    - Making text all lowercase\n",
    "    \n",
    " **2. NEW*: Advanced text pre-processing techniques that we can apply**\n",
    "    - Lemmatization with spaCy\n",
    "\n",
    "After the cleaning process, we will then move to tokenizing the data for the TF-IDF approach"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Great danger, cool, motif and cantik2 jg models. Delivery cepet. Tp packing less okay krn only wear clear plastic nerawang klihtan contents jd \n",
      "\n",
      "One of the shades don't fit well \n",
      "\n",
      "Very comfortable \n",
      "\n",
      "Fast delivery. Product expiry is on Dec 2022. Product wrap properly. No damage on the item. \n",
      "\n",
      "it's sooooo cute! i like playing with the glitters better than browsing on my phone now. item was also deliered earlier than i expected. thank you seller! may you have more buyers to come. 😊😊😊 \n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Extract the labels & sentiments from the training data\n",
    "\n",
    "test_features = test_raw.iloc[:, 1].values\n",
    "\n",
    "# Check features\n",
    "i = 0\n",
    "\n",
    "while i < 5:\n",
    "    print(test_features[i], \"\\n\")\n",
    "    i += 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "import spacy\n",
    "\n",
    "nlp_en = spacy.load(\"en_core_web_sm\", disable = ['parser', 'ner'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "processed_test_features = []\n",
    "\n",
    "for sentence in range(0, len(test_features)):\n",
    "    \n",
    "    # Remove all special characters\n",
    "    processed_feature = re.sub(r'\\W', ' ', str(test_features[sentence]))\n",
    "    \n",
    "    # Remove all words that include digits / numbers\n",
    "    processed_feature = re.sub(r'\\w*\\d\\w*', ' ', processed_feature)\n",
    "    \n",
    "    # Remove all single characters\n",
    "    processed_feature = re.sub(r'\\s+[a-zA-Z]\\s+', ' ', processed_feature)\n",
    "    \n",
    "    # Remove single characters from the start\n",
    "    processed_feature = re.sub(r'\\^[a-zA-Z]\\s+', ' ', processed_feature)\n",
    "    \n",
    "    # NEW: Applying English - spaCy Lemmatization\n",
    "    processed_feature = nlp_en(processed_feature)\n",
    "    processed_feature = \" \".join([token.lemma_ for token in processed_feature])\n",
    "    \n",
    "    # NEW: Applying Bahasa Indonesia - spaCy Lemmatization\n",
    "    # processed_feature = nlp_id(processed_feature)\n",
    "    # processed_feature = \" \".join([token.lemma_ for token in processed_feature])\n",
    "    \n",
    "    # Substituing multiple spaces with single space\n",
    "    processed_feature = re.sub(r'\\s+', ' ', processed_feature, flags = re.I)\n",
    "    \n",
    "    # Converting to lowercase\n",
    "    processed_featured = processed_feature.lower()\n",
    "    \n",
    "    # Append cleaned review to processed list\n",
    "    processed_test_features.append(processed_feature)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "great danger cool motif and jg model Delivery cepet Tp packing less okay krn only wear clear plastic nerawang klihtan content jd \n",
      "\n",
      "one of the shade don fit well \n",
      "\n",
      "very comfortable \n",
      "\n",
      "fast delivery product expiry be on Dec product wrap properly no damage on the item \n",
      "\n",
      "-PRON- sooooo cute like play with the glitter well than browse on -PRON- phone now item be also deliere early than expect thank -PRON- seller may -PRON- have more buyer to come  \n",
      "\n"
     ]
    }
   ],
   "source": [
    "# Checking if labels / reviews have been cleaned up\n",
    "i = 0\n",
    "\n",
    "while i < 5:\n",
    "    print(processed_test_features[i], \"\\n\")\n",
    "    i += 1"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Vectorization\n",
    "\n",
    "We will use the same `TF-IDF` vectorizer that would include the function of tokenization and filtering of stop words to help with our test dataset. <br/>\n",
    "\n",
    "All variables will follow the same as per the model building."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load Indonesian & English library from SpaCy\n",
    "from spacy.lang.id.stop_words import STOP_WORDS as id_stop\n",
    "from spacy.lang.en.stop_words import STOP_WORDS as en_stop\n",
    "\n",
    "# Compiling stopwords list\n",
    "final_stopwords_list = list(id_stop) + list(en_stop)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\kokwo\\Anaconda3\\envs\\shopeeleague\\lib\\site-packages\\sklearn\\feature_extraction\\text.py:383: UserWarning: Your stop_words may be inconsistent with your preprocessing. Tokenizing the stop words generated tokens ['baiknya', 'berkali', 'kali', 'kurangnya', 'll', 'mata', 'olah', 'sekurang', 'setidak', 'tama', 'tidaknya', 've'] not in stop_words.\n",
      "  warnings.warn('Your stop_words may be inconsistent with '\n"
     ]
    }
   ],
   "source": [
    "from nltk.corpus import stopwords\n",
    "from sklearn.feature_extraction.text import TfidfVectorizer\n",
    "\n",
    "vectorizer = TfidfVectorizer(max_features = 3500, min_df = 7, max_df = 0.75, stop_words = final_stopwords_list)\n",
    "processed_test_features1 = vectorizer.fit_transform(processed_test_features).toarray()\n",
    "\n",
    "vectorizer2 = TfidfVectorizer(max_features = 2500, min_df = 7, max_df = 0.75, stop_words = final_stopwords_list)\n",
    "processed_test_features2 = vectorizer2.fit_transform(processed_test_features).toarray()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.3 Time to predict the test data for submission! \n",
    "\n",
    "We will have to now the unPickle the trained model to make predictions to the test data & output in the format required by the challenge.\n",
    "\n",
    "Since we have 2 trained models, we will prepare 2 sets of submission in `csv` format. \n",
    "\n",
    "The task requires our submission to have only 2 columns:\n",
    "    - review_id (int) which are the indexes of the reviews, starting from 0\n",
    "    - rating (int) the predictions made by our model\n",
    "    - output in csv format"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "# UnPickle our earlier trained model\n",
    "\n",
    "# New Model 1 : Accuracy of 45.17% on y-test\n",
    "\n",
    "filename_01 = \"rfc_new_model_01.pkl\"\n",
    "with open(filename_01, 'rb') as file1:\n",
    "    pickle_rfc_01 = pickle.load(file1)\n",
    "    \n",
    "# New Model 2 : Accuracy of 44.19% on y-test\n",
    "filename_02 = \"rfc_new_model_02.pkl\"\n",
    "with open(filename_02, 'rb') as file2:\n",
    "    pickle_rfc_02 = pickle.load(file2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=1)]: Using backend SequentialBackend with 1 concurrent workers.\n",
      "[Parallel(n_jobs=1)]: Done   1 out of   1 | elapsed:    0.3s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done   2 out of   2 | elapsed:    0.9s remaining:    0.0s\n",
      "[Parallel(n_jobs=1)]: Done 300 out of 300 | elapsed:  1.2min finished\n"
     ]
    }
   ],
   "source": [
    "# Time to predict!\n",
    "\n",
    "predictions_01 = pickle_rfc_01.predict(processed_test_features1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[Parallel(n_jobs=8)]: Using backend ThreadingBackend with 8 concurrent workers.\n",
      "[Parallel(n_jobs=8)]: Done  16 tasks      | elapsed:    1.2s\n",
      "[Parallel(n_jobs=8)]: Done 112 tasks      | elapsed:    6.6s\n",
      "[Parallel(n_jobs=8)]: Done 272 tasks      | elapsed:   15.8s\n",
      "[Parallel(n_jobs=8)]: Done 300 out of 300 | elapsed:   17.3s finished\n"
     ]
    }
   ],
   "source": [
    "predictions_02 = pickle_rfc_02.predict(processed_test_features2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1 1\n",
      "5 5\n",
      "60427 60427 \n",
      "Original: 60427\n"
     ]
    }
   ],
   "source": [
    "# Checking the prediction output accurate & is between 1 to 5\n",
    "# Check length to ensure input features = output predicted labels\n",
    "\n",
    "print(min(predictions_01), min(predictions_02))\n",
    "print(max(predictions_01), max(predictions_02))\n",
    "print(len(predictions_01), len(predictions_02), \"\\nOriginal:\", len(test_raw))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Prearing the submission document\n",
    "\n",
    "We will need to attach the predictions to the original indexing before removing the features (reviews/sentiments) to have the document in the correct submission format. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>review_id</th>\n",
       "      <th>review</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Great danger, cool, motif and cantik2 jg model...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>One of the shades don't fit well</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Very comfortable</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Fast delivery. Product expiry is on Dec 2022. ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>it's sooooo cute! i like playing with the glit...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   review_id                                             review\n",
       "0          1  Great danger, cool, motif and cantik2 jg model...\n",
       "1          2                   One of the shades don't fit well\n",
       "2          3                                   Very comfortable\n",
       "3          4  Fast delivery. Product expiry is on Dec 2022. ...\n",
       "4          5  it's sooooo cute! i like playing with the glit..."
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Creating copies for ease of reusibility later\n",
    "\n",
    "submission_03 = test_raw.copy()\n",
    "\n",
    "submission_04 = test_raw.copy()\n",
    "\n",
    "# Checking if copy is successful\n",
    "submission_03.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>review_id</th>\n",
       "      <th>review</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>Great danger, cool, motif and cantik2 jg model...</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>One of the shades don't fit well</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>Very comfortable</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>Fast delivery. Product expiry is on Dec 2022. ...</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>it's sooooo cute! i like playing with the glit...</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   review_id                                             review  rating\n",
       "0          1  Great danger, cool, motif and cantik2 jg model...       5\n",
       "1          2                   One of the shades don't fit well       1\n",
       "2          3                                   Very comfortable       5\n",
       "3          4  Fast delivery. Product expiry is on Dec 2022. ...       5\n",
       "4          5  it's sooooo cute! i like playing with the glit...       1"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Attaching the predictions to it's review_id\n",
    "submission_03['rating'] = predictions_01\n",
    "submission_04['rating'] = predictions_02\n",
    "\n",
    "# Checking if columns are correctly attached\n",
    "submission_03.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>review_id</th>\n",
       "      <th>rating</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>3</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   review_id  rating\n",
       "0          1       5\n",
       "1          2       1\n",
       "2          3       5\n",
       "3          4       5\n",
       "4          5       1"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Removing the review column\n",
    "\n",
    "submission_03 = submission_03.drop(\"review\", axis = 1)\n",
    "submission_04 = submission_04.drop(\"review\", axis = 1)\n",
    "\n",
    "# Checks if column has been dropped correctly\n",
    "submission_03.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "submission doc 3: (60427, 2)\n",
      "submission doc 4: (60427, 2)\n"
     ]
    }
   ],
   "source": [
    "# Last check before saving to csv.\n",
    "\n",
    "print(\"submission doc 3:\", submission_03.shape)\n",
    "print(\"submission doc 4:\", submission_04.shape)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Output results in csv format\n",
    "\n",
    "submission_03.to_csv(\"shopeecodeleague_SentimentAnalysis_TeamJnny_03.csv\", index = False, encoding = 'utf8')\n",
    "submission_04.to_csv(\"shopeecodeleague_SentimentAnalysis_TeamJnny_04.csv\", index = False, encoding = 'utf8')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Congratulations!\n",
    "\n",
    "We have now completed & submitted our work. The test score as follows on the [Kraggle Leaderboard](https://www.kaggle.com/c/student-shopee-code-league-sentiment-analysis/leaderboard#score):\n",
    "\n",
    "`Model_01`: 0.18133 <br/>\n",
    "`Model_02`: 0.18122 <br/>\n",
    "`New_Model_01` : 0.23146 <br/>\n",
    "`New_Model_02` : 0.19644\n",
    "\n",
    "Not a significant improvement but it was a fun excercise nonetheless for our team doing a sentiment analysis as our first!\n",
    "\n",
    "#### Thank you ;)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}