Merge branch 'main' into release

.github/.codecov.yml

@@ -2,6 +2,11 @@ codecov:
   notify:
     wait_for_ci: true
 
+ignore:
+  - ".github" # ignore the .github directory
+  - "docs" # ignore the tests directory
+  - "figs" # ignore the figs directory
+
 coverage:
   status:
     patch:

.github/workflows/mypy.yml

@@ -34,7 +34,6 @@ jobs:
     - name: Install dependencies
       run: |
         curl -sSL https://install.python-poetry.org | python3
-        poetry lock
         poetry install --with test -E chat
     - name: Type-checking package with mypy
       run: |

.github/workflows/pre-commit.yml

@@ -1,4 +1,4 @@
-name: pre-commit
+name: autofix.ci
 
 concurrency:
   group: ${{ github.workflow }}-${{ github.ref }}
@@ -13,9 +13,11 @@ jobs:
   pre-commit:
     runs-on: ubuntu-latest
     steps:
-    - uses: actions/checkout@v3
+    - uses: actions/checkout@v4
     - name: Set up Python 3.11
       uses: actions/setup-python@v4
       with:
         python-version: 3.11.2
     - uses: pre-commit/action@v3.0.0
+    - uses: autofix-ci/action@dd55f44df8f7cdb7a6bf74c78677eb8acd40cd0a
+      if: always()

.github/workflows/tests.yml

@@ -38,7 +38,6 @@ jobs:
       uses: abatilo/actions-poetry@v2
     - name: Install dependencies
       run: |
-        poetry lock
         poetry install --with test -E chat
     - name: Test with pytest
       env: # Or as an environment variable

README.md

@@ -45,57 +45,24 @@ Sotopia is an open-ended social learning environment that allows agents to inter
 ## Help
 See [documentation](https://docs.sotopia.world) for more details.
 
-
 ## Get started
-### Use on Google Colab
-
-If you want to try it out on Google Colab first, please check out our Colab Tutorial Series:
-
-<ol>
-<li> Basic
-  <ol>
-    <li> <a href="https://colab.research.google.com/github/sotopia-lab/sotopia/blob/main/notebooks/tutorials/1.1-setup.ipynb">1.1 Playing with Sotopia in a Google Colab Notebook</a> </li>
-    <li> <a href="https://colab.research.google.com/github/sotopia-lab/sotopia/blob/main/notebooks/tutorials/1.2-browse-data.ipynb">1.2 Browsing sotopia data</a> </li>
-  </ol>
-</li>
-<li>
-Building your own social agent (coming soon!)
-</li>
-</ol>
 
 ### Install locally
-This package supports Python 3.11 and above. In one line,
-`pip install sotopia` or `pip install uv; uv pip install sotopia`.
-
-Or from scratch, use a virtual environment, e.g. with anaconda3: `conda create -n sotopia python=3.11; conda activate sotopia; curl -sSL https://install.python-poetry.org | python3`. Then, install the requirements and this package.
-```bash
-poetry install
-```
-
-OpenAI key is required to run the code. Please set the environment variable `OPENAI_API_KEY` to your key. The recommend way is to add the key to the conda environment:
-```bash
-conda env config vars set OPENAI_API_KEY=your_key
-```
+We recommend using a virtual environment, e.g. with anaconda3: `conda create -n sotopia python=3.11; conda activate sotopia; curl -sSL https://install.python-poetry.org | python3`.
 
-For some experiments, TogetherAI key is required to run the code. Please set the environment variable `TOGETHER_API_KEY` to your key. The recommend way is to add the key to the conda environment:
-```bash
-conda env config vars set TOGETHER_API_KEY=your_key
-```
+Then:
+`python -m pip install sotopia; sotopia install`
 
-A redis-stack server is required to run the code. Please follow the [instruction](https://redis.io/docs/stack/get-started/install/docker/) to start a redis-stack server or use an existing server. You can also check [Q&A](/docs/troubleshooting.md) to initiate the redis server with the Sotopia data.
+This will setup the necessary environment variables and download the necessary data.
 
-The `REDIS_OM_URL` need to be set before loading and saving agents:
-```bash
-conda env config vars set REDIS_OM_URL="redis://user:password@host:port"
-```
+> [!TIP]
+> Having trouble installing? Or don't want to install redis for now? We are working on a public redis server for you to use. Stay tuned!
 
-Make a folder to store the logs:
+OpenAI key is required to run the code. Please set the environment variable `OPENAI_API_KEY` to your key. The recommend way is to add the key to the conda environment:
 ```bash
-mkdir logs
+conda env config vars set OPENAI_API_KEY=your_key
 ```
 
-
-
 ## Easy Sample Server
 You can view an episode demo with default parameters with the following:
 ```python
@@ -118,82 +85,3 @@ or run
 ```bash
 python examples/minimalist_demo.py
 ```
-
-## Contribution
-#### Install dev options
-Follow the installation instruction above and then, instead of running `python -m pip install -e .`, run the following commands:
-
-```bash
-python -m pip install -e ".[dev]"
-mypy --install-types --non-interactive sotopia
-python -m pip install pre-commit
-pre-commit install
-```
-#### New branch for each feature
-`git checkout -b feature/feature-name` and PR to `main` branch.
-#### Before committing
-Run `pytest` to make sure all tests pass (this will ensure dynamic typing passed with beartype) and `mypy --strict .` to check static typing.
-(You can also run `pre-commit run --all-files` to run all checks)
-#### Check github action result
-Check the github action result to make sure all tests pass. If not, fix the errors and push again.
-
-## Running Experiments
-We use `gin-config` to configure the experiments. You don't need to be an expert to use it. The basic syntax is
-```bash
-python <code_file.py> --gin_file <gin_file1> --gin_file <gin_file2> '--gin.PARAM1=value1' '--gin.PARAM2=value2'
-```
-The `--gin_file` is used to load and compose the default configuration. The `--gin.PARAM1=value1` is used to overwrite the default configuration. The later configuration will always overwrite the previous one.
-
-Here is an example of running an experiment:
-
-```bash
-python examples/experiment_eval.py --gin_file sotopia_conf/generation_utils_conf/generate.gin --gin_file sotopia_conf/server_conf/server.gin --gin_file sotopia_conf/run_async_server_in_batch.gin '--gin.ENV_IDS=["01H7VFHPDZVVCDZR3AARA547CY"]' '--gin.AGENT1_MODEL="gpt-4"' '--gin.BATCH_SIZE=20' '--gin.PUSH_TO_DB=False' '--gin.TAG="test"'
-```
-For the complete set of parameters, please check the `sotopia_conf` folder.
-
-To run a large batch of environments, you can change the `ENV_IDS` parameter in `sotopia_conf/run_async_server_in_batch.gin` to a list of environment ids. When `gin.ENV_IDS==[]`, all environments on the DB will be used.
-
-## Getting access to your simulation
-After running experiments, you can go to the `examples/redis_stats.ipynb` notebook to check the existing episodes (Episode Log section), as well as calculate the performance.
-
-For the original Sotopia simulation in our paper's experiments, you can find how to get them in the [Q&A](/docs/troubleshooting.md) section in the `./docs` folder.
-
-## Adding new characters and environments
-You can use the following function with the `**kwargs` being the properties of the `AgentProfile` class. This is the same for the scenarios/environments.
-```python
-class AgentProfile(JsonModel):
-    first_name: str = Field(index=True)
-    last_name: str = Field(index=True)
-    age: int = Field(index=True, default_factory=lambda: 0)
-    occupation: str = Field(index=True, default_factory=lambda: "")
-    gender: str = Field(index=True, default_factory=lambda: "")
-    gender_pronoun: str = Field(index=True, default_factory=lambda: "")
-    public_info: str = Field(index=True, default_factory=lambda: "")
-    big_five: str = Field(index=True, default_factory=lambda: "")
-    moral_values: list[str] = Field(index=False, default_factory=lambda: [])
-    schwartz_personal_values: list[str] = Field(index=False, default_factory=lambda: [])
-    personality_and_values: str = Field(index=True, default_factory=lambda: "")
-    decision_making_style: str = Field(index=True, default_factory=lambda: "")
-    secret: str = Field(default_factory=lambda: "")
-    model_id: str = Field(default_factory=lambda: "")
-
-class EnvironmentProfile(JsonModel):
-    codename: str = Field(...)
-    source: str = Field(...)
-    scenario: str = Field(...)
-    agent_goals: list[str] = Field(...)
-    ...
-```
-
-```python
-
-from sotopia.database.persistent_profile import AgentProfile, EnvironmentProfile
-
-def add_agent_to_database(**kwargs: dict[str, Any]) -> None:
-    agent = AgentProfile(**kwargs)
-    agent.save()
-
-def add_env_profile(**kwargs: dict[str, Any]) -> None:
-    env_profile = EnvironmentProfile(**kwargs)
-    env_profile.save()
-```

docs/pages/_meta.json

@@ -1,37 +1,23 @@
 {
     "index": {
-        "type": "page",
-        "title": "Home"
-    },
-    "contribution_menu": {
-        "title": "Documentation",
-        "type": "menu",
-        "items": {
-            "agents": {
-                "title": "Agents",
-                "href": "/agents"
-            },
-            "environments": {
-                "title": "Environments",
-                "href": "/environments"
-            },
-            "examples": {
-                "title": "Examples",
-                "href": "/examples"
-            },
-            "hyperparameters": {
-                "title": "Hyperparameters",
-                "href": "/hyperparameters"
-            },
-            "scripts": {
-                "title": "Scripts",
-                "href": "/scripts"
-            },
-            "xml": {
-                "title": "XML Formatter",
-                "href": "/xml"
-
-            }
+        "title": "Introduction",
+        "theme": {
+            "breadcrumb": true
         }
+    },
+    "---": {
+        "type": "separator"
+    },
+    "concepts": {
+        "title": "Concepts",
+        "type": "page"
+    },
+    "examples": {
+        "title": "Examples",
+        "type": "page"
+    },
+    "contribution": {
+        "title": "Contribution",
+        "type": "page"
     }
 }

docs/pages/concepts/agents.md

@@ -0,0 +1,71 @@
+## Agents
+Sotopia supports a variety of agents that can support social simulations for a variety of purposes.
+
+### LLMAgent
+
+LLMAgent is the core agent that is powered by a large language model
+```python
+class LLMAgent(BaseAgent[Observation, AgentAction]):
+    def __init__(
+        self,
+        agent_name: str | None = None,
+        uuid_str: str | None = None,
+        agent_profile: AgentProfile | None = None,
+        model_name: str = "gpt-3.5-turbo",
+        script_like: bool = False,
+    ) -> None:
+```
+
+### ScriptWritingAgent
+Script Writing Agent is an agent that is specialized in writing scripts of social interactions. It is a subclass of LLMAgent and supports the investigation of [*Is this the real life? Is this just fantasy? The Misleading Success of Simulating Social Interactions With LLMs*](https://arxiv.org/abs/2403.05020)
+
+```python
+class ScriptWritingAgent(LLMAgent):
+    def __init__(
+        self,
+        agent_name: str | None = None,
+        uuid_str: str | None = None,
+        agent_profile: AgentProfile | None = None,
+        model_name: str = "gpt-3.5-turbo",
+        agent_names: list[str] = [],
+        background: ScriptBackground | None = None,
+    ) -> None:
+```
+
+### HumanAgent
+This agent allows for human input to be used in the simulation through the command line.
+```python
+class HumanAgent(BaseAgent[Observation, AgentAction]):
+    """
+    A human agent that takes input from the command line.
+    """
+
+    def __init__(
+        self,
+        agent_name: str | None = None,
+        uuid_str: str | None = None,
+        agent_profile: AgentProfile | None = None,
+    ) -> None:
+        super().__init__(
+            agent_name=agent_name,
+            uuid_str=uuid_str,
+            agent_profile=agent_profile,
+        )
+        self.model_name: LLM_Name = "human"
+```
+
+### RedisAgent
+
+RedisAgent is an agent that can be used to interact with the Redis database.
+```python
+class RedisAgent(BaseAgent[Observation, AgentAction]):
+    """An agent use redis as a message broker."""
+
+    def __init__(
+        self,
+        agent_name: str | None = None,
+        uuid_str: str | None = None,
+        session_id: str | None = None,
+        agent_profile: AgentProfile | None = None,
+    ) -> None:
+```