Word_Embeddings_Technique-1_Bag of Words(BOW).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "be985945",
   "metadata": {},
   "source": [
    "1. A Quick Example\n",
    "\n",
    "- Let’s look at an easy example to understand the concepts previously explained. We could be interested in analyzing the reviews about Game of Thrones:\n",
    "\n",
    "- **Review 1:** Game of Thrones is an amazing tv series!\n",
    "\n",
    "- **Review 2:** Game of Thrones is the best tv series!\n",
    "\n",
    "- **Review 3:** Game of Thrones is so great\n",
    "\n",
    "In the table, I show all the calculations to obtain the Bag-Of-Words approach:"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5b2ad435",
   "metadata": {},
   "source": [
    "**Vocabulary**\n",
    "\n",
    "- Vocabulary means a kind of dictionary\n",
    "\n",
    "- Every data has its own vocabulary \n",
    "\n",
    "- Here also we have vocabulary for above 3 sentences"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3ba353f3",
   "metadata": {},
   "source": [
    "<img decoding=\"async\" loading=\"lazy\" class=\"alignnone\" src=\"https://cdn-images-1.medium.com/max/1000/1*cHKkqYIhaYuYwuuhBiSlHw.png\" alt=\"Bag-of-Words with Python&nbsp; example\" width=\"815\" height=\"190\" data-image-id=\"1*cHKkqYIhaYuYwuuhBiSlHw.png\" data-width=\"815\" data-height=\"190\">"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9545327f",
   "metadata": {},
   "source": [
    "Each row corresponds to a different review, while the rows are the unique words, contained in the three documents."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d28d44a8",
   "metadata": {},
   "source": [
    "**Sentence-1**\n",
    "\n",
    "- sky is nice\n",
    "\n",
    "**Sentence-2**\n",
    "\n",
    "- clouds are nice\n",
    "\n",
    "**Sentence-3**\n",
    "\n",
    "- Sky is nice and Clouds are nice\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a83d72ff",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "4028d1d1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['sky nice', 'cloud nice', 'sky nice cloud nice']\n"
     ]
    }
   ],
   "source": [
    "from nltk.corpus import stopwords\n",
    "stentences= ['sky is nice','cloud is nice','sky is nice and cloud are nice']\n",
    "cleaned_sentance=[]\n",
    "for sentence in stentences:\n",
    "    word= sentence.lower()\n",
    "    word= word.split()\n",
    "    word=[i for i in word if i not in set(stopwords.words('english'))]\n",
    "    word= \" \".join(word)\n",
    "    cleaned_sentance.append(word)\n",
    "    \n",
    "print(cleaned_sentance)    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60200a06",
   "metadata": {},
   "source": [
    "after data preprocess and apply Stop words\n",
    "\n",
    "**Sentence-1**\n",
    "\n",
    "- sky nice\n",
    "\n",
    "**Sentence-2**\n",
    "\n",
    "- clouds nice\n",
    "\n",
    "**Sentence-3**\n",
    "\n",
    "- Sky nice Clouds nice\n",
    "\n",
    "**Vocabulary**\n",
    "\n",
    "   - clouds, nice,sky\n",
    "   \n",
    "\n",
    "|vocabulary|Frequency|\n",
    "|----------------|-----|\n",
    "|clouds|2|\n",
    "|nice|4|\n",
    "|sky|2|\n",
    "\n",
    "\n",
    "and number of voacbulry becomes number of features\n",
    "\n",
    "\n",
    "|sentence|feature1(clouds)|featur2(nice)|feature3(sky)|\n",
    "|--------|-----|---|----|\n",
    "|sky nice|0|1|1|\n",
    "|clouds nice|1|1|0|\n",
    "|Sky nice Clouds nice|1|2|1|"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "8a8c6f8d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "  (0, 2)\t1\n",
      "  (0, 1)\t1\n",
      "  (1, 1)\t1\n",
      "  (1, 0)\t1\n",
      "  (2, 2)\t1\n",
      "  (2, 1)\t2\n",
      "  (2, 0)\t1\n"
     ]
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "cv= CountVectorizer(max_features= 3)\n",
    "Bagofwords= cv.fit_transform(cleaned_sentance)\n",
    "print(Bagofwords)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "5a6c5d6a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[0 1 1]\n",
      " [1 1 0]\n",
      " [1 2 1]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "cv= CountVectorizer(max_features= 3)\n",
    "Bagofwords= cv.fit_transform(cleaned_sentance).toarray()\n",
    "print(Bagofwords)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "18dc8e51",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cloud</th>\n",
       "      <th>nice</th>\n",
       "      <th>sky</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   cloud  nice  sky\n",
       "0      0     1    1\n",
       "1      1     1    0\n",
       "2      1     2    1"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "pd.DataFrame(Bagofwords,columns=['cloud','nice','sky'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "3cb2e52b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'sky': 2, 'nice': 1, 'cloud': 0}"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cv.vocabulary_\n",
    "\n",
    "# unique words with index\n",
    "# cloud is first word: first feature\n",
    "# nice is second word: second feature\n",
    "# sky is third word: third feature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "1aa8740a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0, 1, 1],\n",
       "       [1, 1, 0],\n",
       "       [1, 2, 1]], dtype=int64)"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "Bagofwords.view()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "0103afe9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['sky nice', 'clouds nice', 'sky nice clouds nice']\n",
      "[[0 0 1 0 1 1]\n",
      " [1 1 1 0 0 0]\n",
      " [1 1 2 1 1 1]]\n"
     ]
    }
   ],
   "source": [
    "### All together\n",
    "from nltk.corpus import stopwords\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "\n",
    "sentences = ['sky is nice', 'clouds are nice', 'Sky is nice and Clouds are nice']\n",
    "\n",
    "cleaned_sentence = []\n",
    "\n",
    "for sentence in sentences:\n",
    "    word = sentence.lower()  \n",
    "    ##lowering all the letters becaz we dont want it to treat uppercase and lower case words differently\n",
    "    \n",
    "    word = word.split()    ##splitting our sentence into words \n",
    "    \n",
    "    ##removing stop words\n",
    "    word = [i for i in word if i not in set(stopwords.words('english'))]          \n",
    "    word = \" \".join(word)               ##joining our words back to sentences\n",
    "    cleaned_sentence.append(word)       ##appending our preprocessed sentence into a new list\n",
    "    \n",
    "    \n",
    "## printing our new list\n",
    "print(cleaned_sentence) \n",
    "\n",
    "cv = CountVectorizer(ngram_range=(1,2))  ##give it a max features as 3\n",
    "Bagofwords = cv.fit_transform(cleaned_sentence).toarray()\n",
    "\n",
    "print(Bagofwords)\n",
    "\n",
    "# Task for you is:  Identify the output logic"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "8aada87d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'sky': 4,\n",
       " 'nice': 2,\n",
       " 'sky nice': 5,\n",
       " 'clouds': 0,\n",
       " 'clouds nice': 1,\n",
       " 'nice clouds': 3}"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cv.vocabulary_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "e242a4c7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['Game Thrones amazing tv series!', 'Game Thrones best tv series!', 'Game Thrones great']\n",
      "[[1 0 1 0 1 1 1]\n",
      " [0 1 1 0 1 1 1]\n",
      " [0 0 1 1 0 1 0]]\n"
     ]
    }
   ],
   "source": [
    "# All together\n",
    "\n",
    "from nltk.corpus import stopwords\n",
    "from sklearn.feature_extraction.text import CountVectorizer\n",
    "\n",
    "sentences=['Game of Thrones is an amazing tv series!', \n",
    "           'Game of Thrones is the best tv series!', \n",
    "           'Game of Thrones is so great']\n",
    "\n",
    "cleaned_sentance = []\n",
    "\n",
    "for sentence in sentences:\n",
    "    word= sentence.lower()\n",
    "#lowering all the letters becaz we dont want it to treat uppercase and lower case words differently\n",
    "    word= sentence.split()     ##splitting our sentence into words \n",
    "    \n",
    "    ## removing stop words\n",
    "    word= [i for i in word if i not in set(stopwords.words('english'))]\n",
    "    word= \" \".join(word)             # joining our words back to sentence\n",
    "    cleaned_sentance.append(word)    # appending our preprocessed sentence into a new list\n",
    "    \n",
    "    \n",
    "## printing our new list\n",
    "print(cleaned_sentance)\n",
    "\n",
    "cv=(cleaned_sentance)\n",
    "\n",
    "cv= CountVectorizer()   # give it a max feature as 3\n",
    "Bagofwords= cv.fit_transform(cleaned_sentance).toarray()\n",
    "\n",
    "print(Bagofwords)\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "6c38e46f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'game': 2,\n",
       " 'thrones': 5,\n",
       " 'amazing': 0,\n",
       " 'tv': 6,\n",
       " 'series': 4,\n",
       " 'best': 1,\n",
       " 'great': 3}"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "cv.vocabulary_   # Index mapping (by alphabets numberning)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9a1856c",
   "metadata": {},
   "source": [
    "Some disadvantages of BOWS:\n",
    "> It won’t provide any semantic information about the words. It only gives how many times has word occurred in a sentence and not its location or correlation with other words in the sentence.\n",
    "\n",
    ">It gives equal importance to all the words in the sentence. Hence it is most useful for simple processes.\n",
    "\n",
    ">There are other methods like Tfidf, word2Vec which are more complex and useful than BOWs."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "86e5f778",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "vocabulary: ['nice', 'sky', 'clouds']\n",
      "[{'clouds': 0, 'nice': 1, 'sky': 1}, {'clouds': 1, 'nice': 1, 'sky': 0}, {'clouds': 1, 'nice': 1, 'sky': 1}]\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[dict_values([0, 1, 1]), dict_values([1, 1, 0]), dict_values([1, 1, 1])]"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "################################ With logic##################################\n",
    "sen=' '.join(cleaned_sentence)\n",
    "l=list(set(sen.split()))\n",
    "print(\"vocabulary:\",l)\n",
    "d={}\n",
    "l1=[]\n",
    "for sentence in cleaned_sentence:\n",
    "    for i in l:\n",
    "        if i in sentence:\n",
    "            d[i]=1\n",
    "        else:\n",
    "            d[i]=0\n",
    "    myKeys = list(d.keys())\n",
    "    myKeys.sort()\n",
    "    sorted_dict = {i: d[i] for i in myKeys}\n",
    "    l1.append(sorted_dict)\n",
    "\n",
    "print(l1)\n",
    "l2=[i.values() for i in l1]\n",
    "l2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "b0e75e15",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'sky': 1, 'is': 2, 'nice': 4, 'clouds': 1, 'are': 2, 'Sky': 1, 'and': 1, 'Clouds': 1}\n"
     ]
    }
   ],
   "source": [
    "# Creating word histogram\n",
    "import nltk\n",
    "word2count = {}\n",
    "for data in sentences:\n",
    "    words = nltk.word_tokenize(data) # we are split into words\n",
    "    for word in words:               # we are calling each word\n",
    "        if word not in word2count.keys(): # if the word not in dictionary, we are \n",
    "            word2count[word] = 1\n",
    "        else:\n",
    "            word2count[word] += 1\n",
    "print(word2count)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}