What is the ReAct Framework?

[AaronWard]

ReAct: Reasoning and Action with Large Language Models

Overview

ReAct is a method designed to facilitate interactive decision-making tasks using Large Language Models (LLMs). This approach combines the power of reasoning and action, enabling LLMs to both think about a task and act on it. ReAct has been tested on various tasks, showcasing its robustness and versatility.

Core Concepts

1. Reasoning + Acting

• ReAct intertwines reasoning (thinking) and action (doing) in a closed-loop system.
• This synergy allows the model to reason about its decisions, perform actions, and then re-evaluate based on the outcomes of those actions.

2. Applications and Experiments

• ALFWorld: A synthetic text-based game where agents interact with a virtual household environment. Agents must navigate and interact with objects to achieve specific goals, such as examining a paper under a desk lamp.
  • Prompt Example: An agent might be prompted to "go to coffeetable 1, take paper 2, use desklamp 1".
• WebShop: An online shopping environment with over a million real-world products. Agents must purchase products based on user instructions.
  • Prompt Example: A user might instruct, “I am looking for a nightstand with drawers. It should have a nickel finish and be priced lower than $140”. The agent then interacts with the website to find a suitable product.

3. Performance

• On ALFWorld, ReAct achieved an average success rate of 71%, outperforming other methods significantly.
• On WebShop, ReAct performed better than other one-shot prompting techniques, showcasing its capability in real-world, noisy environments.

4. Comparison with Other Approaches

• Inner Monologue (IM): A previous method where actions are motivated by an "inner monologue". However, IM's reasoning is more limited and tied to external feedback. In contrast, ReAct offers more flexible and diverse reasoning capabilities.
  • For example: While IM might simply react to a given situation, ReAct could reason that "desklamps are usually found on desks or shelves" and take action accordingly.

Advantages of ReAct

1. Exploiting Pre-trained Knowledge: ReAct capitalizes on the commonsense knowledge embedded in LLMs. This is beneficial in tasks like ALFWorld, where commonsense reasoning about object locations is essential.
2. Sparse Reasoning: Unlike methods that rely heavily on dense external feedback, ReAct uses sparse reasoning, which allows for more varied and flexible reasoning types.
3. Interpretable Decision Traces: ReAct not only makes decisions but also provides an interpretable trail of its reasoning process.

Future Directions and Challenges

1. Scaling Limitations: Complex tasks with large action spaces can quickly exceed the input length limit of in-context learning with LLMs.
2. Potential for Fine-tuning: Preliminary results on tasks like HotpotQA show promise for fine-tuning ReAct. However, more high-quality human annotations will be essential to further improve performance.
3. Integration with Other Paradigms: Combining ReAct with other methods, such as reinforcement learning, could open up new avenues for enhanced performance and capabilities.

paper

Example from LangChain:

Link

 > Entering new AgentExecutor chain...
     I need to find out who Leo DiCaprio's girlfriend is and then calculate her age raised to the 0.43 power.
    Action: Search
    Action Input: "Leo DiCaprio girlfriend"model Vittoria Ceretti I need to find out Vittoria Ceretti's age
    Action: Search
    Action Input: "Vittoria Ceretti age"25 years I need to calculate 25 raised to the 0.43 power
    Action: Calculator
    Action Input: 25^0.43Answer: 3.991298452658078 I now know the final answer
    Final Answer: Leo DiCaprio's girlfriend is Vittoria Ceretti and her current age raised to the 0.43 power is 3.991298452658078.
    
    > Finished chain.