Skip to content

Commit

Permalink
Update README.md
Browse files Browse the repository at this point in the history
  • Loading branch information
anas-awadalla committed Aug 10, 2023
1 parent e39574e commit 51dff49
Showing 1 changed file with 2 additions and 2 deletions.
4 changes: 2 additions & 2 deletions open_flamingo/eval/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ print(checkpoint_path)

This should place the OpenFlamingo model at the expected location in the evaluation script.

For TextVQA and VizWiz we expect annotations to be formatted differently than the original datasets. We provide the custom annotations in `open_flamingo/open_flamingo/eval/data/`.
For TextVQA and VizWiz we expect annotations to be formatted differently than the original datasets. We provide the custom annotations in `open_flamingo/open_flamingo/eval/data/`. We have also uploaded all the annotation files in a [huggingface dataset](https://huggingface.co/datasets/openflamingo/eval_benchmark/tree/main) for easy access.

# Evaluating using RICES (Retrieval-based In-Context Example Selection)

Expand All @@ -65,4 +65,4 @@ python cache_rices_features.py \
--output_dir /path/to/coco/features
```

This will create a directory at `/path/to/coco/features` containing a file named `coco.pkl` with the extracted features. You can then use this directory to evaluate using RICES by passing the `--rices` flag to the evaluation script, specifying the path to the features directory using the `--cached_demonstration_features` flag, and specifying the vision encoder to use for RICES using the `--rices_vision_encoder_path` and `--rices_vision_encoder_pretrained` flags.
This will create a directory at `/path/to/coco/features` containing a file named `coco.pkl` with the extracted features. You can then use this directory to evaluate using RICES by passing the `--rices` flag to the evaluation script, specifying the path to the features directory using the `--cached_demonstration_features` flag, and specifying the vision encoder to use for RICES using the `--rices_vision_encoder_path` and `--rices_vision_encoder_pretrained` flags.

0 comments on commit 51dff49

Please sign in to comment.