NLP_1_2_Tf-IDF.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TF-IDF algorithm"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "texts = ['i hate you','i love you','i really hate you','i like you']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['i', 'love', 'hate', 'you', 'really', 'like']"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "vocabulary = list(set(' '.join(texts).split()))\n",
    "vocabulary"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TF of term v in the sentence t is equal to the number of times v occurred in t devided by the total number of terms in t. Hence:\n",
    "\n",
    "$ TF(v,t) = BOW(v,t)\\ /\\ len(t) $\n",
    "\n",
    "where BOW is the bag of word algorithm result which is basically same as counting the number of times v occurred in t.\n",
    "\n",
    "Example: \n",
    "\n",
    "$ TF(i, 0) = {\\frac{1}{3}} $, on the first sentence, 'i' occurred 1 time and there are 3 words in that sentence"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[1., 0., 1., 1., 0., 0.],\n",
       "       [1., 1., 0., 1., 0., 0.],\n",
       "       [1., 0., 1., 1., 1., 0.],\n",
       "       [1., 0., 0., 1., 0., 1.]])"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "bow = np.zeros((len(texts),len(vocabulary)))\n",
    "for i, t in enumerate(texts):\n",
    "    for v in t.split():\n",
    "        bow[i, vocabulary.index(v)] += 1\n",
    "\n",
    "bow"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0., 0., 0., 0., 0., 0.],\n",
       "       [0., 0., 0., 0., 0., 0.],\n",
       "       [0., 0., 0., 0., 0., 0.],\n",
       "       [0., 0., 0., 0., 0., 0.]])"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tf = np.zeros((len(texts),len(vocabulary)))\n",
    "tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['i', 'hate', 'you']\n",
      "['i', 'love', 'you']\n",
      "['i', 'really', 'hate', 'you']\n",
      "['i', 'like', 'you']\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "array([[0.33333333, 0.        , 0.33333333, 0.33333333, 0.        ,\n",
       "        0.        ],\n",
       "       [0.33333333, 0.33333333, 0.        , 0.33333333, 0.        ,\n",
       "        0.        ],\n",
       "       [0.25      , 0.        , 0.25      , 0.25      , 0.25      ,\n",
       "        0.        ],\n",
       "       [0.33333333, 0.        , 0.        , 0.33333333, 0.        ,\n",
       "        0.33333333]])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "tf = np.zeros((len(texts),len(vocabulary)))\n",
    "for i, t in enumerate(texts):\n",
    "    tf[i] = bow[i] / len(t.split())\n",
    "#     print (t.split())\n",
    "        \n",
    "tf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "IDF of term v is equal to:\n",
    "\n",
    "$\\log(\\frac{N}{n})$\n",
    "\n",
    "where N is the total number of documents and n is the number document that contain term v.\n",
    "\n",
    "Example: \n",
    "\n",
    "$ IDF(i) = {\\log(\\frac{4}{4})} $, total of 4 sentences which all 4 include 'i'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'hate': 0.6931471805599453,\n",
       " 'i': 0.0,\n",
       " 'like': 1.3862943611198906,\n",
       " 'love': 1.3862943611198906,\n",
       " 'really': 1.3862943611198906,\n",
       " 'you': 0.0}"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "idf = {}\n",
    "for v in vocabulary:\n",
    "    idf[v] = 0\n",
    "    for t in texts:\n",
    "        if v in t.split():\n",
    "            idf[v] += 1\n",
    "    idf[v] = np.log(len(texts)/idf[v])\n",
    "\n",
    "idf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "X = np.zeros((len(texts),len(vocabulary)))\n",
    "for i, t in enumerate(texts):\n",
    "    for v in t.split():\n",
    "        X[i, vocabulary.index(v)] = tf[i, vocabulary.index(v)] * idf[v]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[0.        , 0.        , 0.        , 0.23104906, 0.        ,\n",
       "        0.        ],\n",
       "       [0.46209812, 0.        , 0.        , 0.        , 0.        ,\n",
       "        0.        ],\n",
       "       [0.        , 0.        , 0.        , 0.1732868 , 0.        ,\n",
       "        0.34657359],\n",
       "       [0.        , 0.        , 0.46209812, 0.        , 0.        ,\n",
       "        0.        ]])"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Example:\n",
    "$ TF(i, 0) \\times IDF(i) = {\\frac{1}{3} \\times \\log(\\frac{4}{4})}  = {0} $"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}