02_1_chatbot_using_Llama2_&_lanceDB.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/gulabpatel/LLMs/blob/main/02_1_chatbot_using_Llama2_%26_lanceDB.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Install  the  packages\n",
        "\n",
        "\n",
        "\n"
      ],
      "metadata": {
        "id": "UGOkPYguQ2z-"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "eRjCooE9IBAX"
      },
      "outputs": [],
      "source": [
        "!pip install --quiet openai langchain\n",
        "!pip install --quiet -U lancedb\n",
        "!pip install pypdf\n",
        "!pip install sentence-transformers\n",
        "!pip install unstructured\n",
        "!pip install ctransformers"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Mount your drive for uploading data & model"
      ],
      "metadata": {
        "id": "Tbcx0cXSQ8PU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import drive\n",
        "drive.mount('/content/drive')"
      ],
      "metadata": {
        "id": "e-Usavvchqtm",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6c172052-1fe0-44e0-b6eb-e1874caa4cbc"
      },
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Mounted at /content/drive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "crete folder inside drive & nevigate to it\n",
        "i have llm folder inside drive so\n",
        "\n"
      ],
      "metadata": {
        "id": "TYXKyGqvdStz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%cd /content/drive/llm"
      ],
      "metadata": {
        "id": "6Vbev0MgdPkr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Import the packges"
      ],
      "metadata": {
        "id": "8p8ggl_XZ56g"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import lancedb\n",
        "import re\n",
        "import pickle\n",
        "import requests\n",
        "import zipfile\n",
        "from pathlib import Path\n",
        "from langchain.document_loaders import UnstructuredHTMLLoader\n",
        "from langchain.embeddings import OpenAIEmbeddings\n",
        "from langchain.embeddings import HuggingFaceEmbeddings\n",
        "from langchain.document_loaders import PyPDFLoader, DirectoryLoader\n",
        "from langchain.text_splitter import RecursiveCharacterTextSplitter\n",
        "from langchain.llms import CTransformers\n",
        "from langchain.chains import RetrievalQA\n",
        "from langchain.vectorstores import LanceDB\n",
        "from langchain.document_loaders import TextLoader\n",
        "from langchain.text_splitter import CharacterTextSplitter\n"
      ],
      "metadata": {
        "id": "CEOZFyNEinYw"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's load the embedding model\n",
        "The all-mpnet-base-v2 model provides the best quality, while all-MiniLM-L6-v2 is 5 times faster and still offers good quality. you can try different models\n",
        "we are using all -MiniLM-L6-V2 models for this demo, & we are using cpu for inference"
      ],
      "metadata": {
        "id": "BJO6pcBXRFRK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#load embedding model\n",
        "\n",
        "embeddings_mini = HuggingFaceEmbeddings(model_name='sentence-transformers/all-MiniLM-L6-v2',\n",
        "                                   model_kwargs={'device': 'cpu'})"
      ],
      "metadata": {
        "id": "IhUkKPE3jc71"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "select anyone below method"
      ],
      "metadata": {
        "id": "IB4nKLLCSscE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# #Load the  multiple pdfs\n",
        "# pdf_folder_path = '/content/drive/llm/pdf_paths/'\n",
        "\n",
        "# from langchain.document_loaders import PyPDFDirectoryLoader\n",
        "# loader = PyPDFDirectoryLoader(pdf_folder_path)\n",
        "# docs = loader.load()"
      ],
      "metadata": {
        "id": "PzQqTDX3RD-Q"
      },
      "execution_count": 5,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# load single pdf\n",
        "from langchain.document_loaders import PyPDFLoader\n",
        "loader = PyPDFLoader(\"/content/drive/MyDrive/Gulab Patel/Papers/Two queue model/A queueing Inventory System with two class of customers.pdf\")\n",
        "pages = loader.load_and_split()"
      ],
      "metadata": {
        "id": "D_gdAL1Wjf4c"
      },
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        " Now that we have our raw documents loaded, we need to pre-process them to generate embeddings:\n",
        "\n",
        "\n",
        "\n",
        " Initialize a RecursiveCharacterTextSplitter to split text documents into smaller chunks.\n",
        " This helps in processing large text data efficiently."
      ],
      "metadata": {
        "id": "YT2fJIyBSzSU"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "text_splitter = RecursiveCharacterTextSplitter(\n",
        "    chunk_size=200,\n",
        "    chunk_overlap=50,\n",
        ")\n",
        "documents = text_splitter.split_documents(pages)\n",
        "embeddings = embeddings_mini"
      ],
      "metadata": {
        "id": "hYwyV99Jji_8"
      },
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Lets use lanceDB as vectore store database & crete temp table"
      ],
      "metadata": {
        "id": "xbGpYTajS5Cf"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "db = lancedb.connect('/tmp/lancedb')\n",
        "table = db.create_table(\"pandas_docs\", data=[\n",
        "    {\"vector\": embeddings.embed_query(\"Hello World\"), \"text\": \"Hello World\", \"id\": \"1\"}\n",
        "], mode=\"overwrite\")\n",
        "docsearch = LanceDB.from_documents(documents, embeddings, connection=table)"
      ],
      "metadata": {
        "id": "-5hPRjS9jlUS"
      },
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, as we are using the Llama 2 quantized model, you need to download and upload it to your Google Drive. You can choose different versions based on your system specifications. For demonstration purposes, we are using a small model, but feel free to explore other options\n",
        "download the models from Thebloke repo.\n",
        "\n",
        "\n",
        "https://huggingface.co/TheBloke/Llama-2-7B-GGML/tree/main\n",
        "\n",
        "download the .bin file & kept in your current directory"
      ],
      "metadata": {
        "id": "XKwken-bb8nB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "#Loading the model\n",
        "def load_llm():\n",
        "    # Load the locally downloaded model here\n",
        "    llm = CTransformers(\n",
        "        model = \"llama-2-7b-chat.ggmlv3.q2_K.bin\",\n",
        "        model_type=\"llama\",\n",
        "        max_new_tokens = 512,\n",
        "        temperature = 0.6\n",
        "    )\n",
        "    return llm\n",
        "\n",
        "qa = RetrievalQA.from_chain_type(llm=load_llm(), chain_type=\"stuff\", retriever=docsearch.as_retriever())\n"
      ],
      "metadata": {
        "id": "3A8Ct9QqjoIq"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "You can ask the quetions to your data & i will give you response"
      ],
      "metadata": {
        "id": "H6zvmlYlTFIe"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "query = \"Give brief of the paper summary?\"\n",
        "qa.run(query)"
      ],
      "metadata": {
        "id": "WCRPud93jqz3"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "query = \"What is LayoutParser ?\"\n",
        "qa.run(query)"
      ],
      "metadata": {
        "id": "y4RY2KAqjs3N"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}