2. Querying the indexed data using LLMs.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "In this tutorial, we will build a Conversational AI System that can answer questions by retrieving the answers from a document.\n",
        "\n",
        "This whole tutorial is divided into 2 parts:\n",
        "- Part 1: Indexing and Storing the Data\n",
        "    - Step 1: Install all the dependencies (Python and Database)\n",
        "        - Python Dependencies\n",
        "        - Database Dependencies\n",
        "        - PGVector Extension\n",
        "    - Step 2: Load the LLMs and Embeddings Model\n",
        "        - LLMs\n",
        "        - Embeddings Model\n",
        "    - Step 3: Setup the database\n",
        "    - Step 4: Index the documents\n",
        "        - Download the data\n",
        "        - Load the data\n",
        "        - Index and store the embeddings\n",
        "- Part 2: Querying the indexed data using LLMs\n",
        "    - Step 1: Load the LLMs and Embeddings Model\n",
        "        - LLMs\n",
        "        - Embeddings Model\n",
        "    - Step 2: Load the index from the database\n",
        "    - Step 3: Setup the query engine\n",
        "    - Step 4: RAG pipeline in action\n",
        "    - Step 5: Using gradio to build a UI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "In this notebook, we will only cover the second part. The first part is covered in the notebook `1. Indexing and Storing the Embeddings in PGVector.ipynb`."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Part 2: Querying the indexed data using LLMs"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Step 1: Load the LLMs and Embeddings Model\n",
        "\n",
        "Similar to the first part, we will load the LLMs and Embeddings Model. For more details, refer to the first part."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### PHI-2 as an LLM (or SLM?)\n",
        "\n",
        "In the first part, we discussed all the components of `PHI-2`. Refer to the first part if you want to know more about `PHI-2` model, it's performance, how it works, how to load it, etc. In this part, we will only load the model with few additional parameters that we will discuss below."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Similar to first part, we will define our model name. We also have additional models to compare with `PHI-2` model."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {},
      "outputs": [],
      "source": [
        "# model_name = 'mistralai/Mistral-7B-Instruct-v0.2'\n",
        "# model_name = \"mistralai/Mistral-7B-v0.1\"\n",
        "model_name = \"microsoft/phi-2\""
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "There are 2 additional parameters that we will pass to the `PHI-2` model:\n",
        "- `system_prompt`: System prompt are the set of instructions or contextual information that is provided to the LLMs to guide its behavior and responses.\n",
        "- `query_wrapper_prompt`: Query wrapper prompts are also a type prompt that is used to frame or structure user queries before they are sent to a language model for processing."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {},
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "d72d2dcb04e243d48f0ba5a4b3777b14",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "Loading checkpoint shards:   0%|          | 0/3 [00:00<?, ?it/s]"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        }
      ],
      "source": [
        "import torch\n",
        "from llama_index.prompts.prompts import SimpleInputPrompt\n",
        "from llama_index.llms import HuggingFaceLLM\n",
        "\n",
        "# Context Window specifies how many tokens to use as context for the LLM\n",
        "context_window = 2048\n",
        "# Max New Tokens specifies how many new tokens to generate for the LLM\n",
        "max_new_tokens = 256\n",
        "# Device specifies which device to use for the LLM\n",
        "device = \"cuda\"\n",
        "\n",
        "# This is the prompt that will be used to instruct the model behavior\n",
        "system_prompt = \"You are a Q&A assistant. Your goal is to answer questions as accurately as possible based on the instructions and context provided.\"\n",
        "\n",
        "# This will wrap the default prompts that are internal to llama-index\n",
        "query_wrapper_prompt = SimpleInputPrompt(\"<|USER|>{query_str}<|ASSISTANT|>\")\n",
        "\n",
        "# Create the LLM using the HuggingFaceLLM class\n",
        "llm = HuggingFaceLLM(\n",
        "    context_window=context_window,\n",
        "    max_new_tokens=max_new_tokens,\n",
        "    system_prompt=system_prompt,\n",
        "    query_wrapper_prompt=query_wrapper_prompt,\n",
        "    tokenizer_name=model_name,\n",
        "    model_name=model_name,\n",
        "    device_map=device,\n",
        "    # uncomment this if using CUDA to reduce memory usage\n",
        "    model_kwargs={\"torch_dtype\": torch.bfloat16}\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### BGE as an Embeddings Model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "We load the `BGE` model in the exact same way as we did in the first part. Refer to the first part if you want to know more about `BGE` model.\n",
        "\n",
        "Here embedding model is used to embed the user query and using this embedding we will apply similarity search on the indexed documents which in turn will return the most relevant documents."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {},
      "outputs": [],
      "source": [
        "embedding_model_name = \"BAAI/bge-large-en-v1.5\""
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {},
      "outputs": [],
      "source": [
        "from langchain.embeddings.huggingface import HuggingFaceBgeEmbeddings\n",
        "from llama_index.embeddings import LangchainEmbedding\n",
        "\n",
        "# Create the embedding model using the HuggingFaceBgeEmbeddings class\n",
        "embed_model = LangchainEmbedding(\n",
        "  HuggingFaceBgeEmbeddings(model_name=embedding_model_name)\n",
        ")\n",
        "\n",
        "# Get the embedding dimension of the model by doing a forward pass with a dummy input\n",
        "embed_dim = len(embed_model.get_text_embedding(\"Hello world\")) # 1024"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Step 2: Load the index from the database"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Setting up the database configuration"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Similar to the first part, we will setup the database configuration. Refer to the first part if you want to know more about the database configuration."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {},
      "outputs": [],
      "source": [
        "connection_string = \"postgresql://postgres:test123@localhost:5432\"\n",
        "db_name = \"ragdb\"\n",
        "table_name = 'embeddings'"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Setting up the Service context"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Similar to the first part, we will setup the service context. Refer to the first part if you want to know more about the service context."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {},
      "outputs": [],
      "source": [
        "from llama_index import ServiceContext\n",
        "from llama_index import set_global_service_context\n",
        "\n",
        "# Set the chunk size and overlap that controls how the documents are chunked\n",
        "chunk_size = 1024\n",
        "chunk_overlap = 32\n",
        "\n",
        "# Create the service context\n",
        "service_context = ServiceContext.from_defaults(\n",
        "    embed_model=embed_model,\n",
        "    llm=llm,\n",
        "    chunk_size=chunk_size,\n",
        "    chunk_overlap=chunk_overlap,\n",
        ")\n",
        "\n",
        "# Set the global service context\n",
        "set_global_service_context(service_context)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Loading the Indexed data from the database"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Similar to the first part, we will create a `vector_store` using the already indexed data in the database from the first part. Refer to the first part if you want to know more about how to create a `vector_store` from the documents."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "id": "UlIx6-dIbUxW"
      },
      "outputs": [],
      "source": [
        "from sqlalchemy import make_url\n",
        "from llama_index.vector_stores import PGVectorStore\n",
        "\n",
        "# Creates a URL object from the connection string\n",
        "url = make_url(connection_string)\n",
        "\n",
        "# Create the vector store\n",
        "vector_store = PGVectorStore.from_params(\n",
        "    database=db_name,\n",
        "    host=url.host,\n",
        "    password=url.password,\n",
        "    port=url.port,\n",
        "    user=url.username,\n",
        "    table_name=table_name,\n",
        "    embed_dim=embed_dim,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now, we will load the indexed data from the database. We will use the `vector_store` to load the indexed data from the database.\n",
        "\n",
        "In the first part, we used `from_documents` method of `VectorStoreIndex` class to create the index and save it in the database. Now, we will use `from_vector_store` method of `VectorStoreIndex` class to load the already indexed and saved data from the database."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {},
      "outputs": [],
      "source": [
        "from llama_index import VectorStoreIndex\n",
        "\n",
        "# Load the index from the vector store of the database\n",
        "index = VectorStoreIndex.from_vector_store(vector_store=vector_store)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Step 3: Setup the query engine"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now that we have loaded the indexed data from the database, we also have the LLMs and Embeddings Model ready, we will setup the query engine.\n",
        "\n",
        "Query engine is the component that is responsible for querying the indexed data using the LLMs and Embeddings Model.\n",
        "\n",
        "In essence, the query engine first embeds the user query using the Embeddings Model. Then, it uses those embeddings to retrieve the most relevant documents from the indexed data. Once it has extracted the most relevant documents, it uses to extracted docs as a context. Then, this context along with the user query is passed to the LLMs to generate the answer."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now, we will setup the query engine. Creating a query engine is a 2 step process:\n",
        "\n",
        "- Step 1: Create a `retriever` using `VectorIndexRetriever`. This retriever is responsible for retrieving the most relevant documents from the indexed data. With the help of `index`, `Retriever` gets access to the indexed data. Then we set `similarity_top_k` as 2, which means that we want to retrieve 2 most relevant documents from the indexed data. You can play around with this parameter to see how it affects the performance of the query engine.\n",
        "\n",
        "- Step 2: create a `response_synthesizer` using `get_response_synthesizer`. Here we define the `response_mode` as `refine`. There are several options available for `response_mode` which you can explore. Follow this [link](https://docs.llamaindex.ai/en/stable/module_guides/querying/response_synthesizers/root.html#configuring-the-response-mode) to know more about the different options available for `response_mode`. `refine` option provides us with more refined answers. You can play around with this parameter to see how it affects the performance of the query engine.\n",
        "\n",
        "We finally combine the `retriever` and `response_synthesizer` to create a `query_engine` using `RetrieverQueryEngine`.\n",
        "\n",
        "There are other ways as well to create a query engine. The simplest way to create a query engine is to use `index.as_query_engine()` method. This method will create a query engine with default parameters. But, if you want to customize the query engine, you can use the method shown above."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {},
      "outputs": [],
      "source": [
        "from llama_index.query_engine import RetrieverQueryEngine\n",
        "from llama_index.retrievers import VectorIndexRetriever\n",
        "from llama_index.response_synthesizers import get_response_synthesizer\n",
        "\n",
        "# Create the retriever that manages the index and the number of results to return\n",
        "retriever = VectorIndexRetriever(\n",
        "      index=index,\n",
        "      similarity_top_k=2,\n",
        ")\n",
        "\n",
        "# Create the response synthesizer that will be used to synthesize the response\n",
        "response_synthesizer = get_response_synthesizer(\n",
        "      response_mode='refine',\n",
        ")\n",
        "\n",
        "# Create the query engine that will be used to query the retriever and synthesize the response\n",
        "query_engine = RetrieverQueryEngine(\n",
        "      retriever=retriever,\n",
        "      response_synthesizer=response_synthesizer,\n",
        ")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Step 4: RAG pipeline in action"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Wow! It was a long journey to reach here. But, we are finally here. Now, we will be able to see our implemented RAG pipeline in action.\n",
        "\n",
        "Before we start, let's first address few critical question: Will `PHI-2` give us the correct and relevant answers given that it is a foundational model? \n",
        "\n",
        "`PHI-2` model which we are using is not a instruction-fine-tuned model like Mistral-7B-Instruct. That's why earlier I mentioned that we will be comparing `PHI-2` with other models. More specifically, we will be comparing `PHI-2` with `Mistral-7B` and `Mistral-7B-Instruct` models. We will see that `Mistral-7B-Instruct` model will give us the most relevant answers. But, `PHI-2` will also give us some relevant answers along with some irrelevant responses. We will see that `Mistral-7B` model will also suffer from the same problem as `PHI-2` model. But, `Mistral-7B-Instruct` model will give us the most relevant answers."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {},
      "outputs": [],
      "source": [
        "import textwrap\n",
        "\n",
        "# Create the text wrapper that will be used to wrap the response\n",
        "# This is optional and can be removed if you don't want to wrap the response\n",
        "# This is done to make the response more readable\n",
        "wrapper = textwrap.TextWrapper(width=75) "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n",
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
          ]
        }
      ],
      "source": [
        "response = query_engine.query('what is this document about?')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 12,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Based on the context provided, this document is a part of the 'MEMENTO'\n",
            "project's 'Pink Revisions' from September 7, 1999. The document is labeled\n",
            "as page 116 and contains text and annotations. The provided context\n",
            "suggests that this document is related to a conversation between a\n",
            "character named Leonard and another person, with Leonard asking a question.\n",
            "However, without further information, it's still impossible to determine\n",
            "the exact content or subject matter of the document.\n"
          ]
        }
      ],
      "source": [
        "print(wrapper.fill(text=response.response))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 13,
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n",
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
          ]
        }
      ],
      "source": [
        "response = query_engine.query('what is the name of the movie?')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 14,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Memento is a movie that follows the character Leonard as he searches for\n",
            "his wife's killer. In one scene, he chases a man named Dodd and later\n",
            "discovers a file card that reads \"Tattoo: Access to Drugs.\" In another\n",
            "scene, Leonard is seen at a motel, exiting with a shopping bag and later\n",
            "burning a stuffed toy and an old paperback book by a reservoir. The movie\n",
            "is known for its nonlinear storytelling and use of color to distinguish\n",
            "different sequences.\n"
          ]
        }
      ],
      "source": [
        "print(wrapper.fill(text=response.response))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 15,
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n",
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
          ]
        }
      ],
      "source": [
        "response = query_engine.query('what is the name of Director and the year of release?')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 16,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The name of the director is Christopher Nolan and \"Memento\" was released in\n",
            "the year 2000.\n"
          ]
        }
      ],
      "source": [
        "print(wrapper.fill(text=response.response))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 17,
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n",
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
          ]
        }
      ],
      "source": [
        "response = query_engine.query('who is the writer of the movie?')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 18,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "The movie \"Memento\" is based on a short story by Jonathan Nolan. However,\n",
            "the screenplay was written by Christopher Nolan.\n"
          ]
        }
      ],
      "source": [
        "print(wrapper.fill(text=response.response))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 19,
      "metadata": {},
      "outputs": [
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n",
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
          ]
        }
      ],
      "source": [
        "response = query_engine.query('can you pointout the pecularities of this movie in terms of screenplay, like what confused the audience?')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 20,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Based on the provided context from the \"Memento\" screenplay, the movie is\n",
            "known for its unique narrative structure, which can be confusing for some\n",
            "audiences. The story is told in a reverse chronological order, with scenes\n",
            "presented in a non-linear fashion. This unconventional storytelling method\n",
            "can make it challenging for viewers to follow the sequence of events and\n",
            "understand the cause-and-effect relationships between different scenes.\n",
            "Additionally, the protagonist, Leonard Shelby, suffers from anterograde\n",
            "amnesia, which means he cannot remember new information. He relies on\n",
            "notes, photographs, and tattoos to navigate his world. This narrative style\n",
            "and the protagonist's condition create a sense of disorientation and\n",
            "uncertainty, which can be intriguing but also confusing for some viewers.\n",
            "The context from the screenplay further illustrates this, as it shows\n",
            "Leonard being suspected of being a bad actor and undergoing more tests, but\n",
            "the significance of these events and their relationship to the rest of the\n",
            "story is not clear until later in the film when the narrative unfolds in\n",
            "reverse.\n"
          ]
        }
      ],
      "source": [
        "print(wrapper.fill(text=response.response))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "As can be seen from the above results, `Mistral-7B-Instruct` model gives us the most relevant answers. But, `PHI-2` model, being a foundational model, lacks the ability to follow the instructions. That's why it gives us some relevant answers along with some irrelevant answers. `Mistral-7B` model also suffers from the same problem as `PHI-2` model. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Finally! It was a long journey. But, we finally made it. We implemented the RAG pipeline from scratch. We also saw how to index the documents and query the indexed data using LLMs. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Step 5: Using gradio to build a UI"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Here is an additional step that you can try out. You can use `gradio` to build a UI for the RAG pipeline. `gradio` is a python library that allows you to quickly create UIs for your machine learning models. You can read more about `gradio` [here](https://gradio.app/).\n",
        "\n",
        "For `gradio`, you first need to define a function that takes in a query along additional parameters named `history`. Ths function is then passed to `gradio` and it creates a UI for you. You can play around with the UI to see how the RAG pipeline works."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 21,
      "metadata": {
        "id": "UuixUCYtRpPE"
      },
      "outputs": [],
      "source": [
        "def predict(input, history):\n",
        "    response = query_engine.query(input)\n",
        "    return str(response)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 22,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Running on local URL:  http://127.0.0.1:7860\n",
            "Running on public URL: https://2b4a874fb3ce4df436.gradio.live\n",
            "\n",
            "This share link expires in 72 hours. For free permanent hosting and GPU upgrades, run `gradio deploy` from Terminal to deploy to Spaces (https://huggingface.co/spaces)\n"
          ]
        },
        {
          "data": {
            "text/html": [
              "<div><iframe src=\"https://2b4a874fb3ce4df436.gradio.live\" width=\"100%\" height=\"500\" allow=\"autoplay; camera; microphone; clipboard-read; clipboard-write;\" frameborder=\"0\" allowfullscreen></iframe></div>"
            ],
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": []
          },
          "execution_count": 22,
          "metadata": {},
          "output_type": "execute_result"
        },
        {
          "name": "stderr",
          "output_type": "stream",
          "text": [
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n",
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n",
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n",
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n",
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n",
            "Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.\n"
          ]
        }
      ],
      "source": [
        "import gradio as gr\n",
        "\n",
        "gr.ChatInterface(predict).launch(share=True)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {},
      "outputs": [],
      "source": []
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "gpuType": "T4",
      "provenance": []
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.11.5"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "025bdfb78a9b495981ceec1e747fc68b": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0f5c9c41e09b4468b04d3a6707170dfd": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "16dc15cedaec4d15ae8ea92711a625bb": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "180cbca02722485aa2a68c070c0abc6d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d212af7f4483484a94cb4a85369e8204",
            "placeholder": "​",
            "style": "IPY_MODEL_950d1458347f44fba73bc8549c052c0c",
            "value": " 133M/133M [00:05&lt;00:00, 12.6MB/s]"
          }
        },
        "1a06300215414266815835d5b2507708": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "26231fdb729b4803bb3a263ff3d32203": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8f1e9546b1b74a43ad73fc6912b11a7f",
            "placeholder": "​",
            "style": "IPY_MODEL_4574c26bf62d473abce016c7429a1ed2",
            "value": "model.safetensors: 100%"
          }
        },
        "26e3ad873f8c499585377b8ce677d7fe": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_26231fdb729b4803bb3a263ff3d32203",
              "IPY_MODEL_999f7b7a359040bfb01d7eaec8655296",
              "IPY_MODEL_180cbca02722485aa2a68c070c0abc6d"
            ],
            "layout": "IPY_MODEL_fc6c35e70bad4a398f567e26a77d2b03"
          }
        },
        "27507badb5f8493da1934cfe1da5ba41": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a01e34ba924b43ef940cd97a9d59d980",
              "IPY_MODEL_4c81cc7ad46f49e0a1214911db7b0216",
              "IPY_MODEL_8fd1d1a8e9ad441cb41ee6e20d9286a9"
            ],
            "layout": "IPY_MODEL_025bdfb78a9b495981ceec1e747fc68b"
          }
        },
        "2b12cb3355f04a4c8ea3dff9c09c983d": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2bd4026542b5422aa28484eb3ef3ef5c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "4574c26bf62d473abce016c7429a1ed2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4616eab641294af58a6c6f8789f3e247": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4c81cc7ad46f49e0a1214911db7b0216": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "danger",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_aed7aa134da94b438b5d66dea8ab1f01",
            "max": 2,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_79f9976653974f869855ed7b689eba47",
            "value": 0
          }
        },
        "5a0c125f8c674b4eaaad2ebcb63b4f4e": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_618f34070d7c42129962dc254e19c430",
            "placeholder": "​",
            "style": "IPY_MODEL_4616eab641294af58a6c6f8789f3e247",
            "value": "Loading checkpoint shards: 100%"
          }
        },
        "618f34070d7c42129962dc254e19c430": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "65c2d119edd54dd89276a1393890be1d": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_df0ba84b58354927ac669c52dd0869bd",
            "max": 743,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_947570370b7248b39872c826d476e85b",
            "value": 743
          }
        },
        "67f594bdc73448a39f5465425c61ae84": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ed6bc34454ff458fb9a741c8e267a9e3",
            "placeholder": "​",
            "style": "IPY_MODEL_0f5c9c41e09b4468b04d3a6707170dfd",
            "value": " 2/2 [00:22&lt;00:00,  9.85s/it]"
          }
        },
        "79f9976653974f869855ed7b689eba47": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "86537e03e7e746709a90490ea725725f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8c2c86cacd3543b984c42b6ce2dd10cc": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "8f1e9546b1b74a43ad73fc6912b11a7f": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8fd1d1a8e9ad441cb41ee6e20d9286a9": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b2202d25c3634a4780c883e8709aa9df",
            "placeholder": "​",
            "style": "IPY_MODEL_d17ecb20128d488d9b7a523475f116a6",
            "value": " 0/2 [00:10&lt;?, ?it/s]"
          }
        },
        "91aeb063adbb44348fb9693fa63d2005": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f0b0ce90167a4b3b87692c76de5395dc",
            "placeholder": "​",
            "style": "IPY_MODEL_b6adf6cf2bac4201a5635e1c54fa325a",
            "value": " 743/743 [00:00&lt;00:00, 42.2kB/s]"
          }
        },
        "947570370b7248b39872c826d476e85b": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "950d1458347f44fba73bc8549c052c0c": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "961af18da51542668c00d4d60c8fa3f2": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "999f7b7a359040bfb01d7eaec8655296": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a3b99b558ea04833ade298b9bc813908",
            "max": 133466304,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_2bd4026542b5422aa28484eb3ef3ef5c",
            "value": 133466304
          }
        },
        "9fb3c036ef67410fa4eb16c70bbd5b22": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_a156226c205c4d1c936a1b979729563f",
              "IPY_MODEL_65c2d119edd54dd89276a1393890be1d",
              "IPY_MODEL_91aeb063adbb44348fb9693fa63d2005"
            ],
            "layout": "IPY_MODEL_2b12cb3355f04a4c8ea3dff9c09c983d"
          }
        },
        "a01e34ba924b43ef940cd97a9d59d980": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a5971739fc3944af8f9c47c6fc61e1c8",
            "placeholder": "​",
            "style": "IPY_MODEL_8c2c86cacd3543b984c42b6ce2dd10cc",
            "value": "Loading checkpoint shards:   0%"
          }
        },
        "a156226c205c4d1c936a1b979729563f": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HTMLModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_961af18da51542668c00d4d60c8fa3f2",
            "placeholder": "​",
            "style": "IPY_MODEL_16dc15cedaec4d15ae8ea92711a625bb",
            "value": "config.json: 100%"
          }
        },
        "a3b99b558ea04833ade298b9bc813908": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "a5971739fc3944af8f9c47c6fc61e1c8": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ab9dde6023e94dc0a2ff1ed896b27f31": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "FloatProgressModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_86537e03e7e746709a90490ea725725f",
            "max": 2,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e5eae726ad884609a43d14e8caf131c2",
            "value": 2
          }
        },
        "aed7aa134da94b438b5d66dea8ab1f01": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b2202d25c3634a4780c883e8709aa9df": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b6adf6cf2bac4201a5635e1c54fa325a": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cee6d23af4cc47fc8742a4d37f954eec": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "HBoxModel",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_5a0c125f8c674b4eaaad2ebcb63b4f4e",
              "IPY_MODEL_ab9dde6023e94dc0a2ff1ed896b27f31",
              "IPY_MODEL_67f594bdc73448a39f5465425c61ae84"
            ],
            "layout": "IPY_MODEL_1a06300215414266815835d5b2507708"
          }
        },
        "d17ecb20128d488d9b7a523475f116a6": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "DescriptionStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d212af7f4483484a94cb4a85369e8204": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "df0ba84b58354927ac669c52dd0869bd": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e5eae726ad884609a43d14e8caf131c2": {
          "model_module": "@jupyter-widgets/controls",
          "model_module_version": "1.5.0",
          "model_name": "ProgressStyleModel",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ed6bc34454ff458fb9a741c8e267a9e3": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f0b0ce90167a4b3b87692c76de5395dc": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fc6c35e70bad4a398f567e26a77d2b03": {
          "model_module": "@jupyter-widgets/base",
          "model_module_version": "1.2.0",
          "model_name": "LayoutModel",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        }
      }
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}