sync docs, stubs, tests and fastapi

.github/workflows/mypy.yml

@@ -1,11 +1,5 @@
 name: Mypy
-on:
-  push:
-    branches:
-    - main
-  pull_request:
-    branches:
-    - main
+on: [push]
 
 jobs:
   Static-Type-Checking:
@@ -21,15 +15,10 @@ jobs:
         python-version: 3.11.2
     - name: Install dependencies
       run: |
-        pip install --upgrade pip
-        pip install -r requirements.txt
-        pip install -e .[dev]
+        curl -sSL https://install.python-poetry.org | python3
+        poetry install --all-extras
     - name: Type-checking package with mypy
       run: |
-        # Manually install mypy in the standard way.
-        pip --quiet install -U mypy
-        # Log this mypy version for debuggability.
-        mypy --version
         # Run this mypy instance against our main package.
-        mypy --install-types --non-interactive sotopia
-        mypy --strict .
+        poetry run mypy --install-types --non-interactive sotopia
+        poetry run mypy --strict .

.github/workflows/tests.yml

@@ -1,11 +1,5 @@
 name: Pytest
-on:
-  push:
-    branches:
-    - main
-  pull_request:
-    branches:
-    - main
+on: [push]
 
 jobs:
   Pytest:
@@ -21,13 +15,12 @@ jobs:
         python-version: 3.11.2
     - name: Install dependencies
       run: |
-        pip install --upgrade pip
-        pip install -r requirements.txt
-        pip install -e .[dev]
+        curl -sSL https://install.python-poetry.org | python3
+        poetry install --all-extras
     - name: Test with pytest
       env: # Or as an environment variable
         OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
         REDIS_OM_URL: ${{ secrets.REDIS_OM_URL }}
         TOGETHER_API_KEY: ${{ secrets.TOGETHER_API_KEY }}
       run: |
-        pytest
+        poetry run pytest

README.md

@@ -10,11 +10,12 @@
 
 
 ## Installation
+This package supports Python 3.11 and above. In one line,
+`pip install sotopia`.
 
-This package supports Python 3.11 and above. We recommend using a virtual environment to install this package, e.g. with anaconda3: `conda create -n sotopia python=3.11; conda activate sotopia; conda install -c conda-forge pip`. Then, install the requirements and this package.
+Or from scratch, use a virtual environment, e.g. with anaconda3: `conda create -n sotopia python=3.11; conda activate sotopia; curl -sSL https://install.python-poetry.org | python3`. Then, install the requirements and this package.
 ```bash
-python -m pip install -r requirements.txt # make sure the packages are installed in the specific conda environment
-python -m pip install -e .
+poetry install
 ```
 
 OpenAI key is required to run the code. Please set the environment variable `OPENAI_API_KEY` to your key. The recommend way is to add the key to the conda environment:

docs/all_the_issues.md

@@ -0,0 +1,5 @@
+# Come here if you encounter any issues
+
+## Missing episodes
+
+Large batch size may cause some episodes to be skipped. This is due to the fact that the server may not be able to handle the load. Try reducing the batch size. But you can also use the script in `examples/fix_missing_episodes.py` to fix the missing episodes.

docs/examples.md

@@ -0,0 +1,12 @@
+# Example Scripts For Using The Library
+
+## Example 1: Evaluating existing episodes
+
+```python
+python examples/evaluate_existing_episodes.py --tag=<tag to upload to the database> --model=<the model used to re-evaluate the existing episodes> --batch_size=<batch size used for evaluation> --push-to-db
+```
+
+Run ```python examples/evaluate_existing_episodes.py --help``` for more information.
+
+## Example 2: Generate script-like episodes
+See `docs/simulation_modes.md` for more information.

docs/hyperparameters.md

@@ -0,0 +1,6 @@
+# Hyperparameters that are used in the simulation
+
+## Tags
+
+- `TAG`: The tag of the simulation. This tag is used to identify the simulation in the database.
+- `TAG_TO_CHECK_EXISTING_EPISODES`: Scripts like `examples/experiment_eval.py` checks if there are existing episodes with the same tag in the database. If there are, the simulation **will not** be run. This is to avoid running the same simulation twice. If you want to run the simulation again, you can change the tag or set `TAG_TO_CHECK_EXISTING_EPISODES` to `None`.

docs/simulation_modes.md

@@ -0,0 +1,45 @@
+# Different Modes of Simulation
+
+## Simulation Modes
+
+The simulation can be run in different modes. The mode is specified in the configuration file. The following modes are available:
+
+### Sotopia-lite
+
+- `lite`: The simulation runs without characters' detailed background information but just names. To use this mode, set `lite` to `True` in the gin configuration command.
+e.g.,
+```bash
+python examples/experiment_eval.py \
+ --gin_file sotopia_conf/generation_utils_conf/generate.gin \
+ --gin_file sotopia_conf/server_conf/server.gin \
+ --gin_file sotopia_conf/run_async_server_in_batch.gin \
+ '--gin.ENV_IDS=[]' \
+ '--gin.AGENT1_MODEL="gpt-3.5-turbo"' \
+ '--gin.AGENT2_MODEL="gpt-3.5-turbo"' \
+ '--gin.BATCH_SIZE=5' \
+ '--gin.TAG="lite_gpt3.5_gpt3.5"' \
+ '--gin.TAG_TO_CHECK_EXISTING_EPISODES="lite_gpt3.5_gpt3.5"' \
+ '--gin.PUSH_TO_DB=False' \
+ '--gin.OMNISCIENT=False' \
+ '--gin.VERBOSE=False' \
+ '--gin.LITE=True' \
+```
+
+### Sotopia-script
+
+- `script`: The simulation runs with enabling LLMs generating the interaction in one shot with a script writing setting. To use this mode, set `script` to `True` in the gin configuration command.
+
+e.g.,
+
+```bash
+python examples/generate_script.py \
+ --gin_file sotopia_conf/generation_utils_conf/generate.gin \
+ --gin_file sotopia_conf/run_async_server_in_batch_script.gin \
+ '--gin.ENV_IDS=[]' \
+ '--gin.SCRIPT_MODEL="gpt-3.5-turbo"' \
+ '--gin.BATCH_SIZE=5' \
+ '--gin.TAG="lite_script_gpt3.5_gpt3.5"' \
+ '--gin.TAG_TO_CHECK_EXISTING_EPISODES="lite_script_gpt3.5_gpt3.5"' \
+ '--gin.PUSH_TO_DB=True' \
+ '--gin.VERBOSE=False' \
+ ```