03_QuestionAnswering/01_qa_w_rag_claude.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Retrieval Augmented Question & Answering with Amazon Bedrock using LangChain\n",
    "\n",
    "> *This notebook should work well with the **`Data Science 3.0`** kernel in SageMaker Studio*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Context\n",
    "Previously we saw that the model told us how to to change the tire, however we had to manually provide it with the relevant data and provide the contex ourselves. We explored the approach to leverage the model availabe under Bedrock and ask questions based on it's knowledge learned during training as well as providing manual context. While that approach works with short documents or single-ton applications, it fails to scale to enterprise level question answering where there could be large enterprise documents which cannot all be fit into the prompt sent to the model. \n",
    "\n",
    "### Pattern\n",
    "We can improve upon this process by implementing an architecure called Retreival Augmented Generation (RAG). RAG retrieves data from outside the language model (non-parametric) and augments the prompts by adding the relevant retrieved data in context. \n",
    "\n",
    "In this notebook we explain how to approach the pattern of Question Answering to find and leverage the documents to provide answers to the user questions.\n",
    "\n",
    "### Challenges\n",
    "- How to manage large document(s) that exceed the token limit\n",
    "- How to find the document(s) relevant to the question being asked\n",
    "\n",
    "### Proposal\n",
    "To the above challenges, this notebook proposes the following strategy\n",
    "#### Prepare documents\n",
    "![Embeddings](./images/Embeddings_lang.png)\n",
    "\n",
    "Before being able to answer the questions, the documents must be processed and a stored in a document store index\n",
    "- Load the documents\n",
    "- Process and split them into smaller chunks\n",
    "- Create a numerical vector representation of each chunk using Amazon Bedrock Titan Embeddings model\n",
    "- Create an index using the chunks and the corresponding embeddings\n",
    "#### Ask question\n",
    "![Question](./images/Chatbot_lang.png)\n",
    "\n",
    "When the documents index is prepared, you are ready to ask the questions and relevant documents will be fetched based on the question being asked. Following steps will be executed.\n",
    "- Create an embedding of the input question\n",
    "- Compare the question embedding with the embeddings in the index\n",
    "- Fetch the (top N) relevant document chunks\n",
    "- Add those chunks as part of the context in the prompt\n",
    "- Send the prompt to the model under Amazon Bedrock\n",
    "- Get the contextual answer based on the documents retrieved"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use Case\n",
    "#### Dataset\n",
    "To explain this architecture pattern we are using the documents from IRS. These documents explain topics such as:\n",
    "- Original Issue Discount (OID) Instruments\n",
    "- Reporting Cash Payments of Over $10,000 to IRS\n",
    "- Employer's Tax Guide\n",
    "\n",
    "#### Persona\n",
    "Let's assume a persona of a layman who doesn't have an understanding of how IRS works and if some actions have implications or not.\n",
    "\n",
    "The model will try to answer from the documents in easy language.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Implementation\n",
    "In order to follow the RAG approach this notebook is using the LangChain framework where it has integrations with different services and tools that allow efficient building of patterns such as RAG. We will be using the following tools:\n",
    "\n",
    "- **LLM (Large Language Model)**: Anthropic Claude V1 available through Amazon Bedrock\n",
    "\n",
    "  This model will be used to understand the document chunks and provide an answer in human friendly manner.\n",
    "- **Embeddings Model**: Amazon Titan Embeddings available through Amazon Bedrock\n",
    "\n",
    "  This model will be used to generate a numerical representation of the textual documents\n",
    "- **Document Loader**: PDF Loader available through LangChain\n",
    "\n",
    "  This is the loader that can load the documents from a source, for the sake of this notebook we are loading the sample files from a local path. This could easily be replaced with a loader to load documents from enterprise internal systems.\n",
    "\n",
    "- **Vector Store**: FAISS available through LangChain\n",
    "\n",
    "  In this notebook we are using this in-memory vector-store to store both the embeddings and the documents. In an enterprise context this could be replaced with a persistent store such as AWS OpenSearch, RDS Postgres with pgVector, ChromaDB, Pinecone or Weaviate.\n",
    "- **Index**: VectorIndex\n",
    "\n",
    "  The index helps to compare the input embedding and the document embeddings to find relevant document\n",
    "- **Wrapper**: wraps index, vector store, embeddings model and the LLM to abstract away the logic from the user."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Setup\n",
    "\n",
    "⚠️ ⚠️ ⚠️ Before running this notebook, ensure you've run the [Bedrock boto3 setup notebook](../00_Intro/bedrock_boto3_setup.ipynb#Prerequisites) notebook. ⚠️ ⚠️ ⚠️\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Create new client\n",
      "  Using region: us-west-2\n",
      "boto3 Bedrock client successfully created!\n",
      "bedrock-runtime(https://bedrock-runtime.us-west-2.amazonaws.com)\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "import os\n",
    "import sys\n",
    "\n",
    "import boto3\n",
    "\n",
    "module_path = \"..\"\n",
    "sys.path.append(os.path.abspath(module_path))\n",
    "from utils import bedrock, print_ww\n",
    "\n",
    "\n",
    "# ---- ⚠️ Un-comment and edit the below lines as needed for your AWS setup ⚠️ ----\n",
    "\n",
    "# os.environ[\"AWS_DEFAULT_REGION\"] = \"<REGION_NAME>\"  # E.g. \"us-east-1\"\n",
    "# os.environ[\"AWS_PROFILE\"] = \"<YOUR_PROFILE>\"\n",
    "# os.environ[\"BEDROCK_ASSUME_ROLE\"] = \"<YOUR_ROLE_ARN>\"  # E.g. \"arn:aws:...\"\n",
    "\n",
    "boto3_bedrock = bedrock.get_bedrock_client(\n",
    "    assumed_role=os.environ.get(\"BEDROCK_ASSUME_ROLE\", None),\n",
    "    region=os.environ.get(\"AWS_DEFAULT_REGION\", None)\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Configure langchain\n",
    "\n",
    "We begin with instantiating the LLM and the Embeddings model. Here we are using Anthropic Claude for text generation and Amazon Titan for text embedding.\n",
    "\n",
    "Note: It is possible to choose other models available with Bedrock. You can replace the `model_id` as follows to change the model.\n",
    "\n",
    "`llm = Bedrock(model_id=\"amazon.titan-text-express-v1\")`\n",
    "\n",
    "Check [documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html) for Available text generation and embedding models Ids under Amazon Bedrock."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# We will be using the Titan Embeddings Model to generate our Embeddings.\n",
    "from langchain.embeddings import BedrockEmbeddings\n",
    "from langchain.llms.bedrock import Bedrock\n",
    "\n",
    "# - create the Anthropic Model\n",
    "llm = Bedrock(model_id=\"anthropic.claude-v2\", client=boto3_bedrock, model_kwargs={'max_tokens_to_sample':200})\n",
    "bedrock_embeddings = BedrockEmbeddings(model_id=\"amazon.titan-embed-text-v1\", client=boto3_bedrock)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preparation\n",
    "Let's first download some of the files to build our document store. For this example we will be using public IRS documents from [here](https://www.irs.gov/publications)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from urllib.request import urlretrieve\n",
    "\n",
    "os.makedirs(\"data\", exist_ok=True)\n",
    "files = [\n",
    "    \"https://www.irs.gov/pub/irs-pdf/p1544.pdf\",\n",
    "    \"https://www.irs.gov/pub/irs-pdf/p15.pdf\",\n",
    "    \"https://www.irs.gov/pub/irs-pdf/p1212.pdf\",\n",
    "]\n",
    "for url in files:\n",
    "    file_path = os.path.join(\"data\", url.rpartition(\"/\")[2])\n",
    "    urlretrieve(url, file_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After downloading we can load the documents with the help of [DirectoryLoader from PyPDF available under LangChain](https://python.langchain.com/en/latest/reference/modules/document_loaders.html) and splitting them into smaller chunks.\n",
    "\n",
    "Note: The retrieved document/text should be large enough to contain enough information to answer a question; but small enough to fit into the LLM prompt. Also the embeddings model has a limit of the length of input tokens limited to 8192 tokens, which roughly translates to ~32,000 characters. For the sake of this use-case we are creating chunks of roughly 1000 characters with an overlap of 100 characters using [RecursiveCharacterTextSplitter](https://python.langchain.com/en/latest/modules/indexes/text_splitters/examples/recursive_text_splitter.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "from langchain.text_splitter import CharacterTextSplitter, RecursiveCharacterTextSplitter\n",
    "from langchain.document_loaders import PyPDFLoader, PyPDFDirectoryLoader\n",
    "\n",
    "loader = PyPDFDirectoryLoader(\"./data/\")\n",
    "\n",
    "documents = loader.load()\n",
    "# - in our testing Character split works better with this PDF data set\n",
    "text_splitter = RecursiveCharacterTextSplitter(\n",
    "    # Set a really small chunk size, just to show.\n",
    "    chunk_size = 1000,\n",
    "    chunk_overlap  = 100,\n",
    ")\n",
    "docs = text_splitter.split_documents(documents)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Average length among 73 documents loaded is 5850 characters.\n",
      "After the split we have 503 documents more than the original 73.\n",
      "Average length among 503 documents (after split) is 910 characters.\n"
     ]
    }
   ],
   "source": [
    "avg_doc_length = lambda documents: sum([len(doc.page_content) for doc in documents])//len(documents)\n",
    "avg_char_count_pre = avg_doc_length(documents)\n",
    "avg_char_count_post = avg_doc_length(docs)\n",
    "print(f'Average length among {len(documents)} documents loaded is {avg_char_count_pre} characters.')\n",
    "print(f'After the split we have {len(docs)} documents more than the original {len(documents)}.')\n",
    "print(f'Average length among {len(docs)} documents (after split) is {avg_char_count_post} characters.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We had 3 PDF documents which have been split into smaller ~500 chunks.\n",
    "\n",
    "Now we can see how a sample embedding would look like for one of those chunks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sample embedding of a document chunk:  [ 0.11621094  0.06494141 -0.23730469 ...  0.11962891 -0.29882812\n",
      " -0.27929688]\n",
      "Size of the embedding:  (1536,)\n"
     ]
    }
   ],
   "source": [
    "try:\n",
    "    \n",
    "    sample_embedding = np.array(bedrock_embeddings.embed_query(docs[0].page_content))\n",
    "    print(\"Sample embedding of a document chunk: \", sample_embedding)\n",
    "    print(\"Size of the embedding: \", sample_embedding.shape)\n",
    "\n",
    "except ValueError as error:\n",
    "    if  \"AccessDeniedException\" in str(error):\n",
    "        print(f\"\\x1b[41m{error}\\\n",
    "        \\nTo troubeshoot this issue please refer to the following resources.\\\n",
    "         \\nhttps://docs.aws.amazon.com/IAM/latest/UserGuide/troubleshoot_access-denied.html\\\n",
    "         \\nhttps://docs.aws.amazon.com/bedrock/latest/userguide/security-iam.html\\x1b[0m\\n\")      \n",
    "        class StopExecution(ValueError):\n",
    "            def _render_traceback_(self):\n",
    "                pass\n",
    "        raise StopExecution        \n",
    "    else:\n",
    "        raise error"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Following the similar pattern embeddings could be generated for the entire corpus and stored in a vector store.\n",
    "\n",
    "This can be easily done using [FAISS](https://github.com/facebookresearch/faiss) implementation inside [LangChain](https://python.langchain.com/en/latest/modules/indexes/vectorstores/examples/faiss.html) which takes  input the embeddings model and the documents to create the entire vector store. Using the Index Wrapper we can abstract away most of the heavy lifting such as creating the prompt, getting embeddings of the query, sampling the relevant documents and calling the LLM. [VectorStoreIndexWrapper](https://python.langchain.com/en/latest/modules/indexes/getting_started.html#one-line-index-creation) helps us with that.\n",
    "\n",
    "**⚠️⚠️⚠️ NOTE: it might take few minutes to run the following cell ⚠️⚠️⚠️**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.chains.question_answering import load_qa_chain\n",
    "from langchain.vectorstores import FAISS\n",
    "from langchain.indexes import VectorstoreIndexCreator\n",
    "from langchain.indexes.vectorstore import VectorStoreIndexWrapper\n",
    "\n",
    "vectorstore_faiss = FAISS.from_documents(\n",
    "    docs,\n",
    "    bedrock_embeddings,\n",
    ")\n",
    "\n",
    "wrapper_store_faiss = VectorStoreIndexWrapper(vectorstore=vectorstore_faiss)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Question Answering\n",
    "\n",
    "Now that we have our vector store in place, we can start asking questions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "query = \"\"\"Is it possible that I get sentenced to jail due to failure in filings?\"\"\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first step would be to create an embedding of the query such that it could be compared with the documents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([-0.18066406,  0.25976562, -0.09082031, ...,  0.3984375 ,\n",
       "       -0.58984375, -0.27148438])"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "query_embedding = vectorstore_faiss.embedding_function.embed_query(query)\n",
    "np.array(query_embedding)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can use this embedding of the query to then fetch relevant documents.\n",
    "Now our query is represented as embeddings we can do a similarity search of our query against our data store providing us with the most relevant information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "4 documents are fetched which are relevant to the query.\n",
      "----\n",
      "## Document 1: There are civil penalties for failure to:\n",
      "File a correct Form 8300 by the date it is\n",
      "due, and\n",
      "Provide the required statement to those\n",
      "named in the Form 8300.\n",
      "If you intentionally disregard the requirement\n",
      "to file a correct Form 8300 by the date it is due,\n",
      "the penalty is the greater of:\n",
      "1.$25,000, or\n",
      "2.The amount of cash you received and\n",
      "were required to report (up to $100,000).\n",
      "There are criminal penalties for:\n",
      "Willful failure to file Form 8300,\n",
      "Willfully filing a false or fraudulent Form\n",
      "8300,\n",
      "Stopping or trying to stop Form 8300 from\n",
      "being filed, and\n",
      "Setting up, helping to set up, or trying to\n",
      "set up a transaction in a way that would\n",
      "make it seem unnecessary to file Form\n",
      "8300.\n",
      "If you willfully fail to file Form 8300, you can\n",
      "be fined up to $250,000 for individuals\n",
      "RECORDS($500,000 for corporations) or sentenced to up\n",
      "to 5 years in prison, or both. These dollar\n",
      "amounts are based on Section 3571 of Title 18\n",
      "of the U.S. Code........\n",
      "---\n",
      "## Document 2: filed when required, there is a failure -to-file (FTF) penalty\n",
      "of 5% of the unpaid tax due with that return. The maximum\n",
      "penalty is generally 25% of the tax due. Also, for each\n",
      "whole or part month the tax is paid late, there is a fail-\n",
      "ure-to- pay (FTP) penalty of 0.5% per month of the amount\n",
      "of tax. For individual filers only, the FTP penalty is re-\n",
      "duced from 0.5% per month to 0.25% per month if an in-\n",
      "stallment agreement is in effect. You must have filed your\n",
      "return on or before the due date of the return to qualify for\n",
      "the reduced penalty. The maximum amount of the FTP\n",
      "penalty is also 25% of the tax due. If both penalties apply\n",
      "in any month, the FTF penalty is reduced by the amount of\n",
      "the FTP penalty. The penalties won't be charged if you\n",
      "have reasonable cause for failing to file or pay. If you re-\n",
      "ceive a penalty notice, you can provide an explanation of\n",
      "why you believe reasonable cause exists.\n",
      "Note.  In addition to any penalties, interest accrues.......\n",
      "---\n",
      "## Document 3: clerks of federal or state courts are discussed\n",
      "later under Bail received by court clerks.\n",
      "However, you do not have to file Form 8300\n",
      "if the transaction is not related to your trade or\n",
      "business. For example, if you own a jewelry\n",
      "store and sell your personal automobile for\n",
      "more than $10,000 in cash, you would not sub-\n",
      "mit a Form 8300 for that transaction.\n",
      "Transaction defined.  A “transaction” occurs\n",
      "when:\n",
      "Goods, services, or property are sold;\n",
      "Property is rented;\n",
      "Cash is exchanged for other cash;\n",
      "A contribution is made to a trust or escrow\n",
      "account;\n",
      "A loan is made or repaid; or\n",
      "Cash is converted to a negotiable instru-\n",
      "ment, such as a check or a bond.\n",
      "Person defined.  A “person” includes an indi-\n",
      "vidual, a company, a corporation, a partnership,\n",
      "an association, a trust, or an estate.\n",
      "Exempt organizations, including employee\n",
      "plans, are also “persons.” However, exempt or-\n",
      "ganizations do not have to file Form 8300 for a\n",
      "more-than-$10,000 charitable cash contribution.......\n",
      "---\n",
      "## Document 4: reasonable cause exists.\n",
      "If you timely filed your employment tax return, the IRS\n",
      "may also waive deposit penalties if you inadvertently\n",
      "failed to deposit and it was the first quarter that you were\n",
      "required to deposit any employment tax, or if you inadver-\n",
      "tently failed to deposit the first time after your deposit fre-\n",
      "quency changed. You must also meet the net worth and\n",
      "size limitations applicable to awards of administrative and\n",
      "litigation costs under section 7430; for individuals, this\n",
      "means that your net worth can't exceed $2 million, and for\n",
      "businesses, your net worth can't exceed $7 million and\n",
      "you also can't have more than 500 employees.\n",
      "The IRS may also waive the deposit penalty the first\n",
      "time you're required to make a deposit if you inadvertently\n",
      "send the payment to the IRS rather than deposit it by EFT.\n",
      "For amounts not properly or timely deposited, the pen-\n",
      "alty rates are as follows.\n",
      "TIP\n",
      "Page 30 Publication 15 (2023).......\n",
      "---\n"
     ]
    }
   ],
   "source": [
    "relevant_documents = vectorstore_faiss.similarity_search_by_vector(query_embedding)\n",
    "print(f'{len(relevant_documents)} documents are fetched which are relevant to the query.')\n",
    "print('----')\n",
    "for i, rel_doc in enumerate(relevant_documents):\n",
    "    print_ww(f'## Document {i+1}: {rel_doc.page_content}.......')\n",
    "    print('---')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have the relevant documents, it's time to use the LLM to generate an answer based on these documents. \n",
    "\n",
    "We will take our inital prompt, together with our relevant documents which were retreived based on the results of our similarity search. We then by combining these create a prompt that we feed back to the model to get our result. At this point our model should give us highly informed information on how we can change the tire of our specific car as it was outlined in our manual.\n",
    "\n",
    "LangChain provides an abstraction of how this can be done easily."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Quick way\n",
    "You have the possibility to use the wrapper provided by LangChain which wraps around the Vector Store and takes input the LLM.\n",
    "This wrapper performs the following steps behind the scences:\n",
    "- Take the question as input\n",
    "- Create question embedding\n",
    "- Fetch relevant documents\n",
    "- Stuff the documents and the question into a prompt\n",
    "- Invoke the model with the prompt and generate the answer in a human readable manner."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from langchain.prompts import PromptTemplate\n",
    "from langchain.chains import RetrievalQA\n",
    "prompt_template = \"\"\"\n",
    "\n",
    "Human: Use the following pieces of context to provide a concise answer to the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.\n",
    "<context>\n",
    "{context}\n",
    "</context\n",
    "\n",
    "Question: {question}\n",
    "\n",
    "Assistant:\"\"\"\n",
    "\n",
    "PROMPT = PromptTemplate(\n",
    "    template=prompt_template, input_variables=[\"context\", \"question\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'query': 'Is it possible that I get sentenced to jail due to failure in filings?', 'result': '\n",
      "Based on the information provided in the context, yes it is possible to be sentenced to jail for\n",
      "certain failures related to filing Form 8300. The context states:\\n\\n\"There are criminal penalties\n",
      "for:\\n- Willful failure to file Form 8300,  \\n- Willfully filing a false or fraudulent Form 8300,\\n-\n",
      "Stopping or trying to stop Form 8300 from being filed, and \\n- Setting up, helping to set up, or\n",
      "trying to set up a transaction in a way that would make it seem unnecessary to file Form 8300.\n",
      "\\n\\nIf you willfully fail to file Form 8300, you can be fined up to $250,000 for individuals\n",
      "($500,000 for corporations) or sentenced to up to 5 years in prison, or both.\"\\n\\nSo willful failure\n",
      "to file Form 8300, or doing things to avoid having to file it, can result in fines and/or up to 5\n",
      "years in prison.', 'source_documents': [Document(page_content='There are civil penalties for failure\n",
      "to:\\nFile a correct Form 8300 by the date it is \\ndue, and\\nProvide the required statement to those\n",
      "\\nnamed in the Form 8300.\\nIf you intentionally disregard the requirement \\nto file a correct Form\n",
      "8300 by the date it is due, \\nthe penalty is the greater of:\\n1.$25,000, or\\n2.The amount of cash\n",
      "you received and \\nwere required to report (up to $100,000).\\nThere are criminal penalties\n",
      "for:\\nWillful failure to file Form 8300,\\nWillfully filing a false or fraudulent Form\n",
      "\\n8300,\\nStopping or trying to stop Form 8300 from \\nbeing filed, and\\nSetting up, helping to set\n",
      "up, or trying to \\nset up a transaction in a way that would \\nmake it seem unnecessary to file Form\n",
      "\\n8300.\\nIf you willfully fail to file Form 8300, you can \\nbe fined up to $250,000 for individuals\n",
      "\\nRECORDS($500,000 for corporations) or sentenced to up \\nto 5 years in prison, or both. These\n",
      "dollar \\namounts are based on Section 3571 of Title 18 \\nof the U.S. Code.', metadata={'source':\n",
      "'data/p1544.pdf', 'page': 3}), Document(page_content=\"filed when required, there is a failure -to-\n",
      "file (FTF) penalty \\nof 5% of the unpaid tax due with that return. The maximum \\npenalty is\n",
      "generally 25% of the tax due. Also, for each \\nwhole or part month the tax is paid late, there is a\n",
      "fail-\\nure-to- pay (FTP) penalty of 0.5% per month of the amount \\nof tax. For individual filers\n",
      "only, the FTP penalty is re-\\nduced from 0.5% per month to 0.25% per month if an in-\\nstallment\n",
      "agreement is in effect. You must have filed your \\nreturn on or before the due date of the return to\n",
      "qualify for \\nthe reduced penalty. The maximum amount of the FTP \\npenalty is also 25% of the tax\n",
      "due. If both penalties apply \\nin any month, the FTF penalty is reduced by the amount of \\nthe FTP\n",
      "penalty. The penalties won't be charged if you \\nhave reasonable cause for failing to file or pay.\n",
      "If you re-\\nceive a penalty notice, you can provide an explanation of \\nwhy you believe reasonable\n",
      "cause exists.\\nNote.  In addition to any penalties, interest accrues\", metadata={'source':\n",
      "'data/p15.pdf', 'page': 32}), Document(page_content='clerks of federal or state courts are discussed\n",
      "\\nlater under Bail received by court clerks.\\nHowever, you do not have to file Form 8300 \\nif the\n",
      "transaction is not related to your trade or \\nbusiness. For example, if you own a jewelry \\nstore\n",
      "and sell your personal automobile for \\nmore than $10,000 in cash, you would not sub-\\nmit a Form\n",
      "8300 for that transaction.\\nTransaction defined.  A “transaction” occurs \\nwhen:\\nGoods, services,\n",
      "or property are sold;\\nProperty is rented;\\nCash is exchanged for other cash;\\nA contribution is\n",
      "made to a trust or escrow \\naccount;\\nA loan is made or repaid; or\\nCash is converted to a\n",
      "negotiable instru-\\nment, such as a check or a bond.\\nPerson defined.  A “person” includes an\n",
      "indi-\\nvidual, a company, a corporation, a partnership, \\nan association, a trust, or an\n",
      "estate.\\nExempt organizations, including employee \\nplans, are also “persons.” However, exempt\n",
      "or-\\nganizations do not have to file Form 8300 for a \\nmore-than-$10,000 charitable cash\n",
      "contribution', metadata={'source': 'data/p1544.pdf', 'page': 1})]}\n"
     ]
    }
   ],
   "source": [
    "qa = RetrievalQA.from_chain_type(\n",
    "    llm=llm,\n",
    "    chain_type=\"stuff\",\n",
    "    retriever=vectorstore_faiss.as_retriever(\n",
    "        search_type=\"similarity\", search_kwargs={\"k\": 3}\n",
    "    ),\n",
    "    return_source_documents=True,\n",
    "    chain_type_kwargs={\"prompt\": PROMPT}\n",
    ")\n",
    "answer = qa({\"query\": query})\n",
    "print_ww(answer)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's ask a different question:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "query_2 = \"What is the difference between market discount and qualified stated interest\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'query': 'What is the difference between market discount and qualified stated interest', 'result':\n",
      "' Based on the context provided:\\n\\nMarket discount is the difference between the issue price plus\n",
      "accrued OID and the adjusted basis of the debt instrument when it is purchased in the secondary\n",
      "market at a lower price. \\n\\nQualified stated interest is stated interest that is unconditionally\n",
      "payable in cash or property at least annually over the term of the debt instrument at a single fixed\n",
      "rate.\\n\\nSo the key difference is that market discount refers to the discount in price/basis when a\n",
      "debt instrument is purchased below its issue price in the secondary market, while qualified stated\n",
      "interest refers to a specific type of interest that is payable on the debt instrument. Market\n",
      "discount is a discount on the purchase price, while qualified stated interest is contractually\n",
      "payable interest.', 'source_documents': [Document(page_content=\"Market discount.  An OID debt\n",
      "instrument \\ngenerally has market discount if your adjusted \\nbasis in the debt instrument\n",
      "immediately after \\nyou acquired it (usually its purchase price) was \\nless than the debt\n",
      "instrument’s issue price plus \\nthe total OID that accrued before you acquired \\nit. In general, a\n",
      "debt instrument is purchased in \\nthe secondary market at a market discount \\nwhen the value of the\n",
      "debt instrument has de-\\ncreased since the instrument’s issue date (for \\nexample, because of an\n",
      "increase in interest \\nrates). An OID debt instrument has market dis-\\ncount if your adjusted basis\n",
      "in the debt instru-\\nment immediately after you acquired it (usually \\nits purchase price) was less\n",
      "than the debt in-\\nstrument's issue price plus the total OID that ac-\\ncrued before you acquired it.\n",
      "The market dis-\\ncount is the difference between the issue price \\nplus accrued OID and your\n",
      "adjusted basis.\\nPremium. A debt instrument is purchased at a \\npremium if its adjusted basis\n",
      "immediately after\", metadata={'source': 'data/p1212.pdf', 'page': 2}), Document(page_content='count,\n",
      "your broker will use a constant yield \\nmethod to determine accruals of market dis-\\ncount rather\n",
      "than a ratable method.\\nSee Market Discount Bonds  in chapter 1 of \\nPub. 550  for information on\n",
      "how to figure ac-\\ncrued market discount and include it in your in-\\ncome currently and for other\n",
      "information about \\nmarket discount bonds.\\nIf you choose to use the constant yield \\nmethod to\n",
      "figure accrued market discount, also \\nsee Figuring OID on Long -Term Debt Instru-\\nments , later.\n",
      "The constant yield method of figur-\\ning accrued OID, explained under Debt Instru-\\nments Issued\n",
      "After July 1, 1982, and Before \\n1985  or Debt Instruments Issued After 1984 , as \\nappropriate, is\n",
      "also used to figure accrued mar-\\nket discount.For more information concerning premium \\nor market\n",
      "discount on an inflation -indexed debt \\ninstrument, see Regulations section 1.1275-7.\\nSale,\n",
      "exchange, or redemption.  Generally, \\nyou treat your gain or loss from the sale, ex-\\nchange, or\n",
      "redemption of an OID debt instru-', metadata={'source': 'data/p1212.pdf', 'page': 5}),\n",
      "Document(page_content=\"Premium. A debt instrument is purchased at a \\npremium if its adjusted basis\n",
      "immediately after \\npurchase is greater than the total of all amounts \\npayable on the debt\n",
      "instrument after the pur-\\nchase date, other than qualified stated interest. \\nThe premium is the\n",
      "excess of the adjusted ba-\\nsis over the payable amounts.\\nPremium will generally eliminate the\n",
      "future \\nreporting of OID in income by the purchaser, as \\ndiscussed under Information for Owners of\n",
      "OID \\nDebt Instruments , later. See Pub. 550  for more \\ninformation on the tax treatment of bond\n",
      "pre-\\nmium.\\nQualified stated interest.  In general, qualified \\nstated interest is stated interest\n",
      "that is uncondi-\\ntionally payable in cash or property (other than \\ndebt instruments of the issuer)\n",
      "at least annually \\nover the term of the debt instrument at a single \\nfixed rate.\\nStated\n",
      "redemption price at maturity.  A debt \\ninstrument's stated redemption price at maturity \\nis the\n",
      "sum of all amounts (principal and interest)\", metadata={'source': 'data/p1212.pdf', 'page': 2})]}\n"
     ]
    }
   ],
   "source": [
    "answer_2  =answer = qa({\"query\": query_2})\n",
    "print_ww(answer_2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Customisable option\n",
    "In the above scenario you explored the quick and easy way to get a context-aware answer to your question. Now let's have a look at a more customizable option with the helpf of [RetrievalQA](https://python.langchain.com/en/latest/modules/chains/index_examples/vector_db_qa.html) where you can customize how the documents fetched should be added to prompt using `chain_type` parameter. Also, if you want to control how many relevant documents should be retrieved then change the `k` parameter in the cell below to see different outputs. In many scenarios you might want to know which were the source documents that the LLM used to generate the answer, you can get those documents in the output using `return_source_documents` which returns the documents that are added to the context of the LLM prompt. `RetrievalQA` also allows you to provide a custom [prompt template](https://python.langchain.com/en/latest/modules/prompts/prompt_templates/getting_started.html) which can be specific to the model.\n",
    "\n",
    "Note: In this example we are using Anthropic Claude as the LLM under Amazon Bedrock. This particular model [performs best](https://docs.anthropic.com/claude/docs/human-and-assistant-formatting) if the inputs are provided under `Human:` and the model is requested to generate an output after `Assistant:`. In the cell below you see an example of how to control the prompt such that the LLM stays grounded and doesn't answer outside the context."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      " Based on the context provided, yes it is possible to be sentenced to jail for willful failures\n",
      "related to filing Form 8300. Specifically, the context states:\n",
      "\n",
      "\"There are criminal penalties for:\n",
      "- Willful failure to file Form 8300,\n",
      "- Willfully filing a false or fraudulent Form 8300,\n",
      "- Stopping or trying to stop Form 8300 from being filed, and\n",
      "- Setting up, helping to set up, or trying to set up a transaction in a way that would make it seem\n",
      "unnecessary to file Form 8300.\n",
      "\n",
      "If you willfully fail to file Form 8300, you can be fined up to $250,000 for individuals ($500,000\n",
      "for corporations) or sentenced to up to 5 years in prison, or both.\"\n",
      "\n",
      "So in summary, yes it is possible to be sentenced to jail for up to 5 years for willful failure to\n",
      "properly file Form 8300.\n"
     ]
    }
   ],
   "source": [
    "from langchain.chains import RetrievalQA\n",
    "from langchain.prompts import PromptTemplate\n",
    "\n",
    "prompt_template = \"\"\"\n",
    "\n",
    "Human: Use the following pieces of context to provide a concise answer to the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer.\n",
    "<context>\n",
    "{context}\n",
    "</context\n",
    "\n",
    "Question: {question}\n",
    "\n",
    "Assistant:\"\"\"\n",
    "\n",
    "PROMPT = PromptTemplate(\n",
    "    template=prompt_template, input_variables=[\"context\", \"question\"]\n",
    ")\n",
    "\n",
    "qa = RetrievalQA.from_chain_type(\n",
    "    llm=llm,\n",
    "    chain_type=\"stuff\",\n",
    "    retriever=vectorstore_faiss.as_retriever(\n",
    "        search_type=\"similarity\", search_kwargs={\"k\": 3}\n",
    "    ),\n",
    "    return_source_documents=True,\n",
    "    chain_type_kwargs={\"prompt\": PROMPT}\n",
    ")\n",
    "query = \"Is it possible that I get sentenced to jail due to failure in filings?\"\n",
    "result = qa({\"query\": query})\n",
    "print_ww(result['result'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Document(page_content='There are civil penalties for failure to:\\nFile a correct Form 8300 by the date it is \\ndue, and\\nProvide the required statement to those \\nnamed in the Form 8300.\\nIf you intentionally disregard the requirement \\nto file a correct Form 8300 by the date it is due, \\nthe penalty is the greater of:\\n1.$25,000, or\\n2.The amount of cash you received and \\nwere required to report (up to $100,000).\\nThere are criminal penalties for:\\nWillful failure to file Form 8300,\\nWillfully filing a false or fraudulent Form \\n8300,\\nStopping or trying to stop Form 8300 from \\nbeing filed, and\\nSetting up, helping to set up, or trying to \\nset up a transaction in a way that would \\nmake it seem unnecessary to file Form \\n8300.\\nIf you willfully fail to file Form 8300, you can \\nbe fined up to $250,000 for individuals \\nRECORDS($500,000 for corporations) or sentenced to up \\nto 5 years in prison, or both. These dollar \\namounts are based on Section 3571 of Title 18 \\nof the U.S. Code.', metadata={'source': 'data/p1544.pdf', 'page': 3}),\n",
       " Document(page_content=\"filed when required, there is a failure -to-file (FTF) penalty \\nof 5% of the unpaid tax due with that return. The maximum \\npenalty is generally 25% of the tax due. Also, for each \\nwhole or part month the tax is paid late, there is a fail-\\nure-to- pay (FTP) penalty of 0.5% per month of the amount \\nof tax. For individual filers only, the FTP penalty is re-\\nduced from 0.5% per month to 0.25% per month if an in-\\nstallment agreement is in effect. You must have filed your \\nreturn on or before the due date of the return to qualify for \\nthe reduced penalty. The maximum amount of the FTP \\npenalty is also 25% of the tax due. If both penalties apply \\nin any month, the FTF penalty is reduced by the amount of \\nthe FTP penalty. The penalties won't be charged if you \\nhave reasonable cause for failing to file or pay. If you re-\\nceive a penalty notice, you can provide an explanation of \\nwhy you believe reasonable cause exists.\\nNote.  In addition to any penalties, interest accrues\", metadata={'source': 'data/p15.pdf', 'page': 32}),\n",
       " Document(page_content='clerks of federal or state courts are discussed \\nlater under Bail received by court clerks.\\nHowever, you do not have to file Form 8300 \\nif the transaction is not related to your trade or \\nbusiness. For example, if you own a jewelry \\nstore and sell your personal automobile for \\nmore than $10,000 in cash, you would not sub-\\nmit a Form 8300 for that transaction.\\nTransaction defined.  A “transaction” occurs \\nwhen:\\nGoods, services, or property are sold;\\nProperty is rented;\\nCash is exchanged for other cash;\\nA contribution is made to a trust or escrow \\naccount;\\nA loan is made or repaid; or\\nCash is converted to a negotiable instru-\\nment, such as a check or a bond.\\nPerson defined.  A “person” includes an indi-\\nvidual, a company, a corporation, a partnership, \\nan association, a trust, or an estate.\\nExempt organizations, including employee \\nplans, are also “persons.” However, exempt or-\\nganizations do not have to file Form 8300 for a \\nmore-than-$10,000 charitable cash contribution', metadata={'source': 'data/p1544.pdf', 'page': 1})]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "result['source_documents']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "Congratulations on completing this module on retrieval augmented generation! This is an important technique that combines the power of large language models with the precision of retrieval methods. By augmenting generation with relevant retrieved examples, the responses we recieved become more coherent, consistent and grounded. You should feel proud of learning this innovative approach. I'm sure the knowledge you've gained will be very useful for building creative and engaging language generation systems. Well done!\n",
    "\n",
    "In the above implementation of RAG based Question Answering we have explored the following concepts and how to implement them using Amazon Bedrock and it's LangChain integration.\n",
    "\n",
    "- Loading documents and generating embeddings to create a vector store\n",
    "- Retrieving documents to the question\n",
    "- Preparing a prompt which goes as input to the LLM\n",
    "- Present an answer in a human friendly manner\n",
    "\n",
    "### Take-aways\n",
    "- Experiment with different Vector Stores\n",
    "- Leverage various models available under Amazon Bedrock to see alternate outputs\n",
    "- Explore options such as persistent storage of embeddings and document chunks\n",
    "- Integration with enterprise data stores\n",
    "\n",
    "# Thank You"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "availableInstances": [
   {
    "_defaultOrder": 0,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.t3.medium",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 1,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.t3.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 2,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.t3.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 3,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.t3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 4,
    "_isFastLaunch": true,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 5,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 6,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 7,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 8,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 9,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 10,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 11,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 12,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.m5d.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 13,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.m5d.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 14,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.m5d.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 15,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.m5d.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 16,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.m5d.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 17,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.m5d.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 18,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.m5d.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 19,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.m5d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 20,
    "_isFastLaunch": false,
    "category": "General purpose",
    "gpuNum": 0,
    "hideHardwareSpecs": true,
    "memoryGiB": 0,
    "name": "ml.geospatial.interactive",
    "supportedImageNames": [
     "sagemaker-geospatial-v1-0"
    ],
    "vcpuNum": 0
   },
   {
    "_defaultOrder": 21,
    "_isFastLaunch": true,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 4,
    "name": "ml.c5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 22,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 8,
    "name": "ml.c5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 23,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.c5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 24,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.c5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 25,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 72,
    "name": "ml.c5.9xlarge",
    "vcpuNum": 36
   },
   {
    "_defaultOrder": 26,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 96,
    "name": "ml.c5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 27,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 144,
    "name": "ml.c5.18xlarge",
    "vcpuNum": 72
   },
   {
    "_defaultOrder": 28,
    "_isFastLaunch": false,
    "category": "Compute optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.c5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 29,
    "_isFastLaunch": true,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g4dn.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 30,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g4dn.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 31,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g4dn.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 32,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g4dn.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 33,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g4dn.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 34,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g4dn.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 35,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 61,
    "name": "ml.p3.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 36,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 244,
    "name": "ml.p3.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 37,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 488,
    "name": "ml.p3.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 38,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.p3dn.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 39,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.r5.large",
    "vcpuNum": 2
   },
   {
    "_defaultOrder": 40,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.r5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 41,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.r5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 42,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.r5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 43,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.r5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 44,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.r5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 45,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 512,
    "name": "ml.r5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 46,
    "_isFastLaunch": false,
    "category": "Memory Optimized",
    "gpuNum": 0,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.r5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 47,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 16,
    "name": "ml.g5.xlarge",
    "vcpuNum": 4
   },
   {
    "_defaultOrder": 48,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 32,
    "name": "ml.g5.2xlarge",
    "vcpuNum": 8
   },
   {
    "_defaultOrder": 49,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 64,
    "name": "ml.g5.4xlarge",
    "vcpuNum": 16
   },
   {
    "_defaultOrder": 50,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 128,
    "name": "ml.g5.8xlarge",
    "vcpuNum": 32
   },
   {
    "_defaultOrder": 51,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 1,
    "hideHardwareSpecs": false,
    "memoryGiB": 256,
    "name": "ml.g5.16xlarge",
    "vcpuNum": 64
   },
   {
    "_defaultOrder": 52,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 192,
    "name": "ml.g5.12xlarge",
    "vcpuNum": 48
   },
   {
    "_defaultOrder": 53,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 4,
    "hideHardwareSpecs": false,
    "memoryGiB": 384,
    "name": "ml.g5.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 54,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 768,
    "name": "ml.g5.48xlarge",
    "vcpuNum": 192
   },
   {
    "_defaultOrder": 55,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4d.24xlarge",
    "vcpuNum": 96
   },
   {
    "_defaultOrder": 56,
    "_isFastLaunch": false,
    "category": "Accelerated computing",
    "gpuNum": 8,
    "hideHardwareSpecs": false,
    "memoryGiB": 1152,
    "name": "ml.p4de.24xlarge",
    "vcpuNum": 96
   }
  ],
  "instance_type": "ml.t3.medium",
  "kernelspec": {
   "display_name": "Python 3 (Data Science 3.0)",
   "language": "python",
   "name": "python3__SAGEMAKER_INTERNAL__arn:aws:sagemaker:us-west-2:236514542706:image/sagemaker-data-science-310-v1"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}