diff --git a/site/en/docs/react_gemini_prompting.ipynb b/site/en/docs/react_gemini_prompting.ipynb new file mode 100644 index 000000000..8ef03fd5a --- /dev/null +++ b/site/en/docs/react_gemini_prompting.ipynb @@ -0,0 +1,1036 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": { + "id": "T85zXWw3Zs05" + }, + "source": [ + "##### Copyright 2024 Google LLC." + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "cellView": "form", + "id": "X4uPZ83DbUTq" + }, + "outputs": [], + "source": [ + "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n", + "# you may not use this file except in compliance with the License.\n", + "# You may obtain a copy of the License at\n", + "#\n", + "# https://www.apache.org/licenses/LICENSE-2.0\n", + "#\n", + "# Unless required by applicable law or agreed to in writing, software\n", + "# distributed under the License is distributed on an \"AS IS\" BASIS,\n", + "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", + "# See the License for the specific language governing permissions and\n", + "# limitations under the License." + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "cellView": "form", + "id": "FUqzNst0YN9P" + }, + "outputs": [], + "source": [ + "# The non-source code materials on this page are licensed under Creative Commons - Attribution-ShareAlike CC-BY-SA 4.0,\n", + "# https://creativecommons.org/licenses/by-sa/4.0/legalcode." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "vX-FA27MbYpQ" + }, + "source": [ + "# ReAct + Gemini: A prompting method for demonstrating reasoning and acting in LLMs" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Pk4Y-PKWc3MU" + }, + "source": [ + "\n", + " \n", + " \n", + " \n", + "
\n", + " View on ai.google.dev\n", + " \n", + " Run in Google Colab\n", + " \n", + " View source on GitHub\n", + "
" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "sdkuZY1IdRal" + }, + "source": [ + "This notebook is a minimal implementation of [ReAct: Synergizing Reasoning and Acting in Language Models](https://arxiv.org/abs/2210.03629) with the Google `gemini-pro` model.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PSr-BK-5meRo" + }, + "source": [ + "This notebook demonstrates the use of `gemini-pro` to generate reasoning traces and task-specific actions by leveraging a **Few-shot ReAct Prompt**. In this walkthrough, you will learn how to:\n", + "\n", + "\n", + "1. Set up your development environment and API access to use Gemini.\n", + "2. Prompt Gemini with ReAct.\n", + "3. Use the newly prompted model for multi-turn conversations (chat).\n", + "4. How ReAct overcomes issues of hallucination and error propagation by seeking external groundtruth via **Wikipedia API**.\n", + "5. Have conversations with deployed **ReAct prompted Gemini bot 🤖**\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "lSkx3VHr3WYb" + }, + "source": [ + "### Background\n", + "\n", + " \n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PqoT0ojAcV9P" + }, + "source": [ + "According to the [research paper](https://arxiv.org/abs/2210.03629), ReAct is a prompting method which allows language models to showcase the trace of thinking processes/steps involved in answering a user's query. This improves human interpretability and trustworthiness. ReAct prompted models generate **Thought-Action-Observation** triplets for every iteration.\n", + "\n", + "\n", + "- Why to use ReAct when you can simply instruct the model: \"Explain step-by-step\"\n", + "> Large Language models (LLMs) generate responses by extrapolating the prompts they are provided with. This is achieved by using their internal knowledge representation. In short, they are oblivious to the external/updated information, which makes them hallucinate(generate wrong/unreal answers) at times.\n", + "\n", + " - ReAct tries to prevent hallucination by mandating language models to seek factual information strictly from the external environment they are provided with.\n", + "\n", + "\n", + "- How does it work?\n", + "\n", + " - Model receives the query from the user.\n", + " - This query with previous chat history(for continuous context link) if any, will be analyzed by the model.\n", + " - The model will use its internal knowledge to decide which among the following action it needs to take.\n", + " - Search[entity]: By analyzing the query, the model will decide what `entity` to query to the Wikipedia API.\n", + " - Lookup[phrase]: Model will analyze the content of the Wikipedia page returned by `Search[entity]` action and then decide what specific information or `phrase` is needed to be retrieved from that page to answer the user's question.\n", + " - Finish[Answer]: Return the `Answer` to the user." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "cVvxnBG-thZG" + }, + "source": [ + "## Setup\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Xq6NOA99tiHK" + }, + "source": [ + "### Install the Python SDK\n", + "\n", + "The Python SDK for the Gemini API, is contained in the [`google-generativeai`](https://pypi.org/project/google-generativeai/) package. Install the dependency using pip:\n", + "\n", + "You will also need to install the **Wikipedia** API.\n" + ] + }, + { + "cell_type": "code", + "execution_count": 1, + "metadata": { + "id": "Twc_XZ7h7Bb4" + }, + "outputs": [], + "source": [ + "!pip install -q google.generativeai" + ] + }, + { + "cell_type": "code", + "execution_count": 2, + "metadata": { + "id": "7oZwkgQpfrLl" + }, + "outputs": [], + "source": [ + "!pip install -q wikipedia" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "DVWIqdtbffau" + }, + "source": [ + "Note: The [`wikipedia` package](https://pypi.org/project/wikipedia/) notes that it was \"designed for ease of use and simplicity, not for advanced use\", and that production or heavy use should instead \"use [Pywikipediabot](http://www.mediawiki.org/wiki/Manual:Pywikipediabot) or one of the other more advanced [Python MediaWiki API wrappers](http://en.wikipedia.org/wiki/Wikipedia:Creating_a_bot#Python)\"." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "vqv5MnQUuBZJ" + }, + "source": [ + "### Import packages" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "qS5HJk_VuCup" + }, + "source": [ + "Import the necessary packages." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": { + "id": "Jz5HOLy47VX0" + }, + "outputs": [], + "source": [ + "import re\n", + "import os\n", + "\n", + "import wikipedia\n", + "from wikipedia.exceptions import DisambiguationError, PageError\n", + "\n", + "import google.generativeai as genai" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "4xsPDHz_uSYH" + }, + "source": [ + "### Set up your API key\n", + "\n", + "Before you can use the Gemini API, you must first obtain an API key. If you don't already have one, create a key with one click in Google AI Studio.\n", + "\n", + "Get an API key\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "3QC1DUOxuWDS" + }, + "source": [ + "In Colab, add the key to the secrets manager under the \"🔑\" in the left panel. Give it the name `GOOGLE_API_KEY`." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "SAvjxTybuWw-" + }, + "source": [ + "Once you have the API key, pass it to the SDK. You can do this in two ways:\n", + "\n", + "* Put the key in the `GOOGLE_API_KEY` environment variable (the SDK will automatically pick it up from there).\n", + "* Pass the key to `genai.configure(api_key=...)`\n" + ] + }, + { + "cell_type": "code", + "execution_count": 7, + "metadata": { + "id": "JAzIedGr9PdN" + }, + "outputs": [], + "source": [ + "try:\n", + " from google.colab import userdata\n", + " GOOGLE_API_KEY=userdata.get('GOOGLE_API_KEY')\n", + "except ImportError as e:\n", + " GOOGLE_API_KEY = os.environ['GOOGLE_API_KEY']\n", + "genai.configure(api_key=GOOGLE_API_KEY)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Sqkwp87FumIp" + }, + "source": [ + "## The ReAct prompt" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "lLv9Kuuu5Ffs" + }, + "source": [ + "The original prompts used in the paper is available at [https://github.com/ysymyth/ReAct/tree/master/prompts](https://github.com/ysymyth/ReAct/tree/master/prompts)\n", + "\n", + "Here, you will be working with the following ReAct prompt with a few minor adjustments." + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "07ed55c29a1d" + }, + "source": [ + "> Note: The prompt and in-context examples used here are taken from [https://github.com/ysymyth/ReAct](https://github.com/ysymyth/ReAct) which is published under [MIT license](https://opensource.org/licenses/MIT)." + ] + }, + { + "cell_type": "code", + "execution_count": 8, + "metadata": { + "id": "g8klL8df4iXe" + }, + "outputs": [], + "source": [ + "model_instructions = \"\"\"Solve a question answering task with interleaving Thought, Action, Observation steps. Thought can reason about the current situation, Observation is understanding relevant information from an Action's output and Action can be of three types:\n", + "(1) entity, which searches the exact entity on Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search and you can try to search the information from those topics.\n", + "(2) keyword, which returns the next sentence containing keyword in the current context. This only does exact matches, so keep your searches short.\n", + "(3) answer, which returns the answer and finishes the task.\n", + "\"\"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Fw52CHAG0aRr" + }, + "source": [ + "### Few-shot prompting to enable in-context learning with Gemini\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "-jhaD4ChNv6M" + }, + "source": [ + "While large language models show good understanding of the instructions they are prompted with, they still may perform poorly on complex tasks in a zero-shot setting. Hence, you will now provide a few examples along with your prompt to steer the model's output according to your needs. This **in-context learning** improves the model's performance significantly." + ] + }, + { + "cell_type": "code", + "execution_count": 9, + "metadata": { + "id": "tZ7vezr02qv0" + }, + "outputs": [], + "source": [ + "examples = \"\"\"\n", + "Here are some examples.\n", + "\n", + "Question\n", + "What is the elevation range for the area that the eastern sector of the Colorado orogeny extends into?\n", + "\n", + "Thought 1\n", + "I need to search Colorado orogeny, find the area that the eastern sector of the Colorado orogeny extends into, then find the elevation range of the area.\n", + "\n", + "Action 1\n", + "Colorado orogeny\n", + "\n", + "Observation 1\n", + "The Colorado orogeny was an episode of mountain building (an orogeny) in Colorado and surrounding areas.\n", + "\n", + "Thought 2\n", + "It does not mention the eastern sector. So I need to look up eastern sector.\n", + "\n", + "Action 2\n", + "eastern sector\n", + "\n", + "Observation 2\n", + "The eastern sector extends into the High Plains and is called the Central Plains orogeny.\n", + "\n", + "Thought 3\n", + "The eastern sector of Colorado orogeny extends into the High Plains. So I need to search High Plains and find its elevation range.\n", + "\n", + "Action 3\n", + "High Plains\n", + "\n", + "Observation 3\n", + "High Plains refers to one of two distinct land regions\n", + "\n", + "Thought 4\n", + "I need to instead search High Plains (United States).\n", + "\n", + "Action 4\n", + "High Plains (United States)\n", + "\n", + "Observation 4\n", + "The High Plains are a subregion of the Great Plains. From east to west, the High Plains rise in elevation from around 1,800 to 7,000 ft (550 to 2,130m).\n", + "\n", + "Thought 5\n", + "High Plains rise in elevation from around 1,800 to 7,000 ft, so the answer is 1,800 to 7,000 ft.\n", + "\n", + "Action 5\n", + "1,800 to 7,000 ft\n", + "\n", + "Question\n", + "Musician and satirist Allie Goertz wrote a song about the \"The Simpsons\" character Milhouse, who Matt Groening named after who?\n", + "\n", + "Thought 1\n", + "The question simplifies to \"The Simpsons\" character Milhouse is named after who. I only need to search Milhouse and find who it is named after.\n", + "\n", + "Action 1\n", + "Milhouse\n", + "\n", + "Observation 1\n", + "Milhouse Mussolini Van Houten is a recurring character in the Fox animated television series The Simpsons voiced by Pamela Hayden and created by Matt Groening.\n", + "\n", + "Thought 2\n", + "The paragraph does not tell who Milhouse is named after, maybe I can look up \"named after\".\n", + "\n", + "Action 2\n", + "named after\n", + "\n", + "Observation 2\n", + "Milhouse was named after U.S. president Richard Nixon, whose middle name was Milhous.\n", + "\n", + "Thought 3\n", + "Milhouse was named after U.S. president Richard Nixon, so the answer is Richard Nixon.\n", + "\n", + "Action 3\n", + "Richard Nixon\n", + "\n", + "Question\n", + "Which documentary is about Finnish rock groups, Adam Clayton Powell or The Saimaa Gesture?\n", + "\n", + "Thought 1\n", + "I need to search Adam Clayton Powell and The Saimaa Gesture, and find which documentary is about Finnish rock groups.\n", + "\n", + "Action 1\n", + "Adam Clayton Powell\n", + "\n", + "Observation 1\n", + "Could not find [Adam Clayton Powell]. Similar: [’Adam Clayton Powell III’, ’Seventh Avenue (Manhattan)’, ’Adam Clayton Powell Jr. State Office Building’, ’Isabel Washington Powell’, ’Adam Powell’, ’Adam Clayton Powell (film)’, ’Giancarlo Esposito’].\n", + "\n", + "Thought 2\n", + "To find the documentary, I can search Adam Clayton Powell (film).\n", + "\n", + "Action 2\n", + "Adam Clayton Powell (film)\n", + "\n", + "Observation 2\n", + "Adam Clayton Powell is a 1989 American documentary film directed by Richard Kilberg. The film is about the rise and fall of influential African-American politician Adam Clayton Powell Jr.[3][4] It was later aired as part of the PBS series The American Experience.\n", + "\n", + "Thought 3\n", + "Adam Clayton Powell (film) is a documentary about an African-American politician, not Finnish rock groups. So the documentary about Finnish rock groups must instead be The Saimaa Gesture.\n", + "\n", + "Action 3\n", + "The Saimaa Gesture\n", + "\n", + "Question\n", + "What profession does Nicholas Ray and Elia Kazan have in common?\n", + "\n", + "Thought 1\n", + "I need to search Nicholas Ray and Elia Kazan, find their professions, then find the profession they have in common.\n", + "\n", + "Action 1\n", + "Nicholas Ray\n", + "\n", + "Observation 1\n", + "Nicholas Ray (born Raymond Nicholas Kienzle Jr., August 7, 1911 - June 16, 1979) was an American film director, screenwriter, and actor best known for the 1955 film Rebel Without a Cause.\n", + "\n", + "Thought 2\n", + "Professions of Nicholas Ray are director, screenwriter, and actor. I need to search Elia Kazan next and find his professions.\n", + "\n", + "Action 2\n", + "Elia Kazan\n", + "\n", + "Observation 2\n", + "Elia Kazan was an American film and theatre director, producer, screenwriter and actor.\n", + "\n", + "Thought 3\n", + "Professions of Elia Kazan are director, producer, screenwriter, and actor. So profession Nicholas Ray and Elia Kazan have in common is director, screenwriter, and actor.\n", + "\n", + "Action 3\n", + "director, screenwriter, actor\n", + "\n", + "Question\n", + "Which magazine was started first Arthur’s Magazine or First for Women?\n", + "\n", + "Thought 1\n", + "I need to search Arthur’s Magazine and First for Women, and find which was started first.\n", + "\n", + "Action 1\n", + "Arthur’s Magazine\n", + "\n", + "Observation 1\n", + "Arthur’s Magazine (1844-1846) was an American literary periodical published in Philadelphia in the 19th century.\n", + "\n", + "Thought 2\n", + "Arthur’s Magazine was started in 1844. I need to search First for Women next.\n", + "\n", + "Action 2\n", + "First for Women\n", + "\n", + "Observation 2\n", + "First for Women is a woman’s magazine published by Bauer Media Group in the USA.[1] The magazine was started in 1989.\n", + "\n", + "Thought 3\n", + "First for Women was started in 1989. 1844 (Arthur’s Magazine) < 1989 (First for Women), so Arthur’s Magazine was started first.\n", + "\n", + "Action 3\n", + "Arthur’s Magazine\n", + "\n", + "Question\n", + "Were Pavel Urysohn and Leonid Levin known for the same type of work?\n", + "\n", + "Thought 1\n", + "I need to search Pavel Urysohn and Leonid Levin, find their types of work, then find if they are the same.\n", + "\n", + "Action 1\n", + "Pavel Urysohn\n", + "\n", + "Observation 1\n", + "Pavel Samuilovich Urysohn (February 3, 1898 - August 17, 1924) was a Soviet mathematician who is best known for his contributions in dimension theory.\n", + "\n", + "Thought 2\n", + "Pavel Urysohn is a mathematician. I need to search Leonid Levin next and find its type of work.\n", + "\n", + "Action 2\n", + "Leonid Levin\n", + "\n", + "Observation 2\n", + "Leonid Anatolievich Levin is a Soviet-American mathematician and computer scientist.\n", + "\n", + "Thought 3\n", + "Leonid Levin is a mathematician and computer scientist. So Pavel Urysohn and Leonid Levin have the same type of work.\n", + "\n", + "Action 3\n", + "yes\n", + "\n", + "Question\n", + "{question}\"\"\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "xeCImqiN3WiQ" + }, + "source": [ + "Copy the instructions along with examples in a file called `model_instructions.txt`" + ] + }, + { + "cell_type": "code", + "execution_count": 10, + "metadata": { + "id": "ZyTfAdpk26oB" + }, + "outputs": [], + "source": [ + "ReAct_prompt = model_instructions + examples\n", + "with open('model_instructions.txt', 'w') as f:\n", + " f.write(ReAct_prompt)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Is8BIVQP3u95" + }, + "source": [ + "## The Gemini-ReAct pipeline" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "PqEwKVDgM1MF" + }, + "source": [ + "### Setup" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "T4M3lxEoM3k0" + }, + "source": [ + "You will now build an end-to-end pipeline to facilitate multi-turn chat with the ReAct-prompted Gemini model." + ] + }, + { + "cell_type": "code", + "execution_count": 11, + "metadata": { + "id": "vssDZcroN-Ob" + }, + "outputs": [], + "source": [ + "class ReAct:\n", + " def __init__(self, model: str, ReAct_prompt: str | os.PathLike):\n", + " \"\"\"Prepares Gemini to follow a `Few-shot ReAct prompt` by imitating\n", + " `function calling` technique to generate both reasoning traces and\n", + " task-specific actions in an interleaved manner.\n", + "\n", + " Args:\n", + " model: name to the model.\n", + " ReAct_prompt: ReAct prompt OR path to the ReAct prompt.\n", + " \"\"\"\n", + " self.model = genai.GenerativeModel(model)\n", + " self.chat = self.model.start_chat(history=[])\n", + " self.should_continue_prompting = True\n", + " self._search_history: list[str] = []\n", + " self._search_urls: list[str] = []\n", + "\n", + " try:\n", + " # try to read the file\n", + " with open(ReAct_prompt, 'r') as f:\n", + " self._prompt = f.read()\n", + " except FileNotFoundError:\n", + " # assume that the parameter represents prompt itself rather than path to the prompt file.\n", + " self._prompt = ReAct_prompt\n", + "\n", + " @property\n", + " def prompt(self):\n", + " return self._prompt\n", + "\n", + " @classmethod\n", + " def add_method(cls, func):\n", + " setattr(cls, func.__name__, func)\n", + "\n", + " @staticmethod\n", + " def clean(text: str):\n", + " \"\"\"Helper function for responses.\"\"\"\n", + " text = text.replace(\"\\n\", \" \")\n", + " return text" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "xKfThpmhMZYZ" + }, + "source": [ + "### Define tools\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "dnvZ2jqdRHE1" + }, + "source": [ + "As instructed by the prompt, the model will be generating **Thought-Action-Observation** traces, where every **Action** trace could be one of the following tokens:\n", + "\n", + "\n", + "1. : Perform a Wikipedia search via external API.\n", + "2. : Lookup for specific information on a page with the Wikipedia API.\n", + "3. : Stop the execution of the model and return the answer.\n", + "\n", + "If the model encounters any of these tokens, the model should make use of the `tools` made available to the model. This understanding of the model to leverage acquired toolsets to collect information from the external world is often referred to as **function calling**. Therefore, the next goal is to imitate this function calling technique in order to allow ReAct prompted Gemini model to access the external groundtruth.\n", + "\n", + "The Gemini API supports function calling and you could use this feature to set up your tools. However, for this tutorial, you will learn to simulate it using `stop_sequences` parameter.\n", + "\n", + "\n", + "Define the tools:" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ysHN4y4FPlJZ" + }, + "source": [ + "#### Search\n", + "Define a method to perform Wikipedia searches" + ] + }, + { + "cell_type": "code", + "execution_count": 12, + "metadata": { + "id": "yCRB4g4BNzak" + }, + "outputs": [], + "source": [ + "@ReAct.add_method\n", + "def search(self, query: str):\n", + " \"\"\"Perfoms search on `query` via Wikipedia api and returns its summary.\n", + "\n", + " Args:\n", + " query: Search parameter to query the Wikipedia API with.\n", + "\n", + " Returns:\n", + " observation: Summary of Wikipedia search for `query` if found else\n", + " similar search results.\n", + " \"\"\"\n", + " observation = None\n", + " query = query.strip()\n", + " try:\n", + " # try to get the summary for requested `query` from the Wikipedia\n", + " observation = wikipedia.summary(query, sentences=4, auto_suggest=False)\n", + " wiki_url = wikipedia.page(query, auto_suggest=False).url\n", + " observation = self.clean(observation)\n", + "\n", + " # if successful, return the first 2-3 sentences from the summary as model's context\n", + " observation = self.model.generate_content(f'Retun the first 2 or 3 \\\n", + " sentences from the following text: {observation}')\n", + " observation = observation.text\n", + "\n", + " # keep track of the model's search history\n", + " self._search_history.append(query)\n", + " self._search_urls.append(wiki_url)\n", + " print(f\"Information Source: {wiki_url}\")\n", + "\n", + " # if the page is ambiguous/does not exist, return similar search phrases for model's context\n", + " except (DisambiguationError, PageError) as e:\n", + " observation = f'Could not find [\"{query}\"].'\n", + " # get a list of similar search topics\n", + " search_results = wikipedia.search(query)\n", + " observation += f' Similar: {search_results}. You should search for one of those instead.'\n", + "\n", + " return observation" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "v3fUbHUsPyoF" + }, + "source": [ + "#### Look up\n", + "Look for a specific phrase on the Wikipedia page." + ] + }, + { + "cell_type": "code", + "execution_count": 13, + "metadata": { + "id": "_F4kAF77O0E_" + }, + "outputs": [], + "source": [ + "@ReAct.add_method\n", + "def lookup(self, phrase: str, context_length=200):\n", + " \"\"\"Searches for the `phrase` in the lastest Wikipedia search page\n", + " and returns number of sentences which is controlled by the\n", + " `context_length` parameter.\n", + "\n", + " Args:\n", + " phrase: Lookup phrase to search for within a page. Generally\n", + " attributes to some specification of any topic.\n", + "\n", + " context_length: Number of words to consider\n", + " while looking for the answer.\n", + "\n", + " Returns:\n", + " result: Context related to the `phrase` within the page.\n", + " \"\"\"\n", + " # get the last searched Wikipedia page and find `phrase` in it.\n", + " page = wikipedia.page(self._search_history[-1], auto_suggest=False)\n", + " page = page.content\n", + " page = self.clean(page)\n", + " start_index = page.find(phrase)\n", + "\n", + " # extract sentences considering the context length defined\n", + " result = page[max(0, start_index - context_length):start_index+len(phrase)+context_length]\n", + " print(f\"Information Source: {self._search_urls[-1]}\")\n", + " return result" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "Tc4mq2qlQCnE" + }, + "source": [ + "#### Finish\n", + "Instruct the pipline to terminate its execution." + ] + }, + { + "cell_type": "code", + "execution_count": 14, + "metadata": { + "id": "0Wxpx8COPak_" + }, + "outputs": [], + "source": [ + "@ReAct.add_method\n", + "def finish(self, _):\n", + " \"\"\"Finishes the conversation on encountering token by\n", + " setting the `self.should_continue_prompting` flag to `False`.\n", + " \"\"\"\n", + " self.should_continue_prompting = False\n", + " print(f\"Information Sources: {self._search_urls}\")" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "u9Tl6W98Zhut" + }, + "source": [ + "### Stop tokens and function calling imitation" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "0VnX9zpBcdA0" + }, + "source": [ + "Now that you are all set with function definitions, the next step is to instruct the model to interrupt its execution upon encountering any of the action tokens. You will make use of the `stop_sequences` parameter from [`genai.GenerativeModel.GenerationConfig`](https://ai.google.dev/api/python/google/generativeai/GenerationConfig) class to instruct the model when to stop. Upon encountering an action token, the pipeline will simply extract what specific token from the `stop_sequences` argument terminated the model's execution thereby calling appropriate **tool**(function).\n", + "\n", + "Function's response will be added to model's chat history for continuing the context link." + ] + }, + { + "cell_type": "code", + "execution_count": 15, + "metadata": { + "id": "vnQom1aQOsK8" + }, + "outputs": [], + "source": [ + "@ReAct.add_method\n", + "def __call__(self, user_question, max_calls: int=8, **generation_kwargs):\n", + " \"\"\"Starts multi-turn conversation with the chat models with function calling\n", + "\n", + " Args:\n", + " max_calls: max calls made to the model to get the final answer.\n", + "\n", + " generation_kwargs: Same as genai.GenerativeModel.GenerationConfig\n", + " candidate_count: (int | None) = None,\n", + " stop_sequences: (Iterable[str] | None) = None,\n", + " max_output_tokens: (int | None) = None,\n", + " temperature: (float | None) = None,\n", + " top_p: (float | None) = None,\n", + " top_k: (int | None) = None\n", + "\n", + " Raises:\n", + " AssertionError: if max_calls is not between 1 and 8\n", + " \"\"\"\n", + "\n", + " # hyperparameter fine-tuned according to the paper\n", + " assert 0 < max_calls <= 8, \"max_calls must be between 1 and 8\"\n", + "\n", + " if len(self.chat.history) == 0:\n", + " model_prompt = self.prompt.format(question=user_question)\n", + " else:\n", + " model_prompt = user_question\n", + "\n", + " # stop_sequences for the model to immitate function calling\n", + " callable_entities = ['', '', '']\n", + "\n", + " generation_kwargs.update({'stop_sequences': callable_entities})\n", + "\n", + " self.should_continue_prompting = True\n", + " for idx in range(max_calls):\n", + "\n", + " self.response = self.chat.send_message(content=[model_prompt],\n", + " generation_config=generation_kwargs, stream=True)\n", + "\n", + " for chunk in self.response:\n", + " print(chunk.text, end=' ')\n", + "\n", + " response_cmd = self.chat.history[-1].parts[-1].text\n", + "\n", + " try:\n", + " # regex to extract \n", + " cmd = re.findall(r'<(.*)>', response_cmd)[-1]\n", + " print(f'')\n", + " # regex to extract param\n", + " query = response_cmd.split(f'<{cmd}>')[-1].strip()\n", + " # call to appropriate function\n", + " observation = self.__getattribute__(cmd)(query)\n", + "\n", + " if not self.should_continue_prompting:\n", + " break\n", + "\n", + " stream_message = f\"\\nObservation {idx + 1}\\n{observation}\"\n", + " print(stream_message)\n", + " # send function's output as user's response\n", + " model_prompt = f\"<{cmd}>{query}'s Output: {stream_message}\"\n", + "\n", + " except (IndexError, AttributeError) as e:\n", + " model_prompt = \"Please try to generate thought-action-observation traces \\\n", + " as instructed by the prompt.\"" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "xtndhebkhW62" + }, + "source": [ + "### Test ReAct prompted Gemini model" + ] + }, + { + "cell_type": "code", + "execution_count": 20, + "metadata": { + "id": "h_KWkXWwfZ5h" + }, + "outputs": [ + { + "name": "stdout", + "output_type": "stream", + "text": [ + "Thought 1\n", + "I need to search the main trio from the new Percy Jackson and the Olympians TV series, find their ages in real life, then sum them up.\n", + "\n", + "Action 1\n", + "Percy Jackson and the Olymp ians TV series \n", + "\n", + "Observation 1\n", + "Could not find [\"Percy Jackson and the Olympians TV series\"]. Similar: ['Percy Jackson and the Olympians (TV series)', 'Percy Jackson & the Olympians', 'Percy Jackson (film series)', 'Percy Jackson & the Olympians: The Lightning Thief', 'Percy Jackson (disambiguation)', 'Percy Jackson', 'List of characters in mythology novels by Rick Riordan', 'The Lightning Thief', 'The Heroes of Olympus', 'Walker Scobell']. You should search for one of those instead.\n", + "Thought 2\n", + "I can search Percy Jackson and the Olympians (TV series ) instead.\n", + "\n", + "Action 2\n", + "Percy Jackson and the Olympians (TV series) \n", + "Information Source: https://en.wikipedia.org/wiki/Percy_Jackson_and_the_Olympians_(TV_series)\n", + "\n", + "Observation 2\n", + "Percy Jackson and the Olympians is an American fantasy television series created by Rick Riordan and Jonathan E. Steinberg for Disney+, based on the book series of the same name by Riordan. Walker Scobell stars as Percy Jackson, alongside Leah Sava Jeffries and Aryan Simhadri. Development on the series began in May 2020, following a pitch by Riordan to Disney Branded Television.\n", + "Thought 3\n", + "The main trio from the new Percy Jackson and the Olympians TV series are Walker Scobell, Leah Sava Jeffries, and Aryan Simhadri. I need to find their ages in real life.\n", + "\n", + "Action 3\n", + " Walker Scobell \n", + "Information Source: https://en.wikipedia.org/wiki/Walker_Scobell\n", + "\n", + "Observation 3\n", + "Walker Scobell, born on January 5, 2009, is an American actor who has starred in the 2022 action comedy films The Adam Project and Secret Headquarters.\n", + "\n", + "In 2023, Scobell began playing the title character of Percy Jackson in the Disney+ fantasy series Percy Jackson and the Olympians.\n", + "Thought 4\n", + "Walker Scobell was born on January 5, 2 009, so he is 14 years old. I need to find the ages of Leah Sava Jeffries and Aryan Simhadri.\n", + "\n", + "Action 4\n", + "Leah Sava Jeffries \n", + "Information Source: https://en.wikipedia.org/wiki/Leah_Jeffries\n", + "\n", + "Observation 4\n", + "Leah Sava Jeffries, born on September 25, 2009, is an American child actress. She made her acting debut in the American musical drama, Empire, in 2015, and later made her feature film debut in the action-thriller Beast, released in 2022.\n", + "Thought 5\n", + "Leah Sava Jeffries was born on September 25, 2009, so she is 13 years old. I need to find the age of Aryan Simhadri.\n", + "\n", + "Action 5\n", + "Aryan Simhadri \n", + "Information Source: https://en.wikipedia.org/wiki/Aryan_Simhadri\n", + "\n", + "Observation 5\n", + "Aryan Simhadri, born on May 6, 2006, is an American actor of Indian descent. He is best known for portraying Grover Underwood in the Disney+ series \"Percy Jackson and the Olympians.\" \n", + "\n", + "In 2021, Simhadri made his Broadway debut as Walter in the production of \"Trevor: The Musical.\"\n", + "Thought 6\n", + "Aryan Simhadri was born on May 6, 2006, so he is 17 years old. The sum of ages of the main trio from the new Percy Jackson and the Olympians TV series is 14 + 13 + 17 = 44.\n", + "\n", + "Action 6\n", + "44 \n", + "Information Sources: ['https://en.wikipedia.org/wiki/Percy_Jackson_and_the_Olympians_(TV_series)', 'https://en.wikipedia.org/wiki/Walker_Scobell', 'https://en.wikipedia.org/wiki/Leah_Jeffries', 'https://en.wikipedia.org/wiki/Aryan_Simhadri']\n" + ] + } + ], + "source": [ + "gemini_ReAct_chat = ReAct(model='gemini-pro', ReAct_prompt='model_instructions.txt')\n", + "# Note: try different combinations of generational_config parameters for variational results\n", + "gemini_ReAct_chat(\"What is the total of ages of the main trio from the new Percy Jackson and the Olympians TV series in real life?\", temperature=0.2)" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "ZIfeyyI6hoIE" + }, + "source": [ + "Now, try asking the same question to `gemini-pro` model without the ReAct prompt." + ] + }, + { + "cell_type": "code", + "execution_count": 22, + "metadata": { + "id": "_NUXNbTuakSC" + }, + "outputs": [ + { + "data": { + "text/plain": [ + "'The TV series has not yet been released, so the real-life ages of the main trio are not yet known.'" + ] + }, + "execution_count": 22, + "metadata": {}, + "output_type": "execute_result" + } + ], + "source": [ + "gemini_ReAct_chat.model.generate_content(\"What is the total of ages of the main trio from the new Percy Jackson and the Olympians TV series in real life?\").text" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "B-jsJSyBtrP8" + }, + "source": [ + "## Summary\n", + "\n", + "The ReAct prompted Gemini model is grounded by external information sources and hence is less prone to hallucination. Furthermore, **Thought-Action-Observation** traces generated by the model enhance human interpretability and trustworthiness by allowing users to witness the model's reasoning process for answering the user's query.\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "vmdNYTm5Lobz" + }, + "source": [ + "## Further reading\n" + ] + }, + { + "cell_type": "markdown", + "metadata": { + "id": "iTiDOoTkLvH6" + }, + "source": [ + "Head over to the [Streamlit app](https://mayochat.streamlit.app/) to interact with a ReAct prompted Gemini bot." + ] + } + ], + "metadata": { + "colab": { + "name": "react_gemini_prompting.ipynb", + "toc_visible": true + }, + "kernelspec": { + "display_name": "Python 3", + "name": "python3" + } + }, + "nbformat": 4, + "nbformat_minor": 0 +}