Skip to content

Commit

Permalink
[Docs] Updated readme with VMAS plots
Browse files Browse the repository at this point in the history
Signed-off-by: Matteo Bettini <matbet@meta.com>
  • Loading branch information
matteobettini committed Oct 13, 2023
1 parent 4ddfae3 commit 7139ccd
Show file tree
Hide file tree
Showing 2 changed files with 15 additions and 6 deletions.
13 changes: 11 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -273,6 +273,13 @@ Currently available ones are:

- **VMAS**: [![Conf](https://img.shields.io/badge/Conf-purple.svg)](fine_tuned/vmas/conf/config.yaml) [![Static Badge](https://img.shields.io/badge/Benchmarks-Wandb-yellow)](https://api.wandb.ai/links/benchmarl/xf5h3p3m)

In the following, we report a table of the results:

| **<p align="center">Environment</p>** | **<p align="center">Sample efficiency curves (all tasks)</p>** | **<p align="center">Performance profile</p>** | **<p align="center">Aggregate scores</p>** |
|---------------------------------------|-------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------|
| VMAS | <img src="https://drive.google.com/uc?export=view&id=1fzfFn0q54gsALRAwmqD1hRTqQIadGPoE"/> | <img src="https://drive.google.com/uc?export=view&id=151pSR2sBluSpWiYxtq3jNX0tfE0vgAuR"/> | <img src="https://drive.google.com/uc?export=view&id=1q2So9V6sL8NHMtj6vL-S3KyzZi11Vfia"/> |


## Reporting and plotting

Reporting and plotting is compatible with [marl-eval](https://github.com/instadeepai/marl-eval).
Expand All @@ -283,8 +290,10 @@ your benchmarks. No more struggling with matplotlib and latex!

[![Example](https://img.shields.io/badge/Example-blue.svg)](examples/plotting)

![aggregate_scores](https://drive.google.com/uc?export=view&id=1-f3NolMSjsWppCSXv_DJcs_GUD_fv7vO)
![sample_efficiancy](https://drive.google.com/uc?export=view&id=1FK37EfiqD3AQXWlQj7HQCkQDRNe2TuLy)
![aggregate_scores](https://drive.google.com/uc?export=view&id=1q2So9V6sL8NHMtj6vL-S3KyzZi11Vfia)
![sample_efficiancy](https://drive.google.com/uc?export=view&id=1fzfFn0q54gsALRAwmqD1hRTqQIadGPoE)
![performace_profile](https://drive.google.com/uc?export=view&id=151pSR2sBluSpWiYxtq3jNX0tfE0vgAuR)


## Extending
One of the core tenets of BenchMARL is allowing users to leverage the existing algorithm
Expand Down
8 changes: 4 additions & 4 deletions examples/plotting/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,10 @@
Run the [`plot_benchmark.py`](plot_benchmark.py) to generate beautiful plots like this, using [marl-eval](https://github.com/instadeepai/marl-eval).

# Aggregate scores
![aggregate_scores](https://drive.google.com/uc?export=view&id=1-f3NolMSjsWppCSXv_DJcs_GUD_fv7vO)
![aggregate_scores](https://drive.google.com/uc?export=view&id=1q2So9V6sL8NHMtj6vL-S3KyzZi11Vfia)
# Sample efficiency curves
![sample_efficiancy](https://drive.google.com/uc?export=view&id=1FK37EfiqD3AQXWlQj7HQCkQDRNe2TuLy)
![sample_efficiancy](https://drive.google.com/uc?export=view&id=1fzfFn0q54gsALRAwmqD1hRTqQIadGPoE)
# Performance profile
![performace_profile](https://drive.google.com/uc?export=view&id=1EAVTNrW8-is9HP7-M19L4pj-pJdfikvc)
![performace_profile](https://drive.google.com/uc?export=view&id=151pSR2sBluSpWiYxtq3jNX0tfE0vgAuR)
# Probability of improvement
![performace_profile](https://drive.google.com/uc?export=view&id=1MZujCPhRkCulj1P5oOZoMWfJxAA2x2te)
![poi](https://drive.google.com/uc?export=view&id=1MZujCPhRkCulj1P5oOZoMWfJxAA2x2te)

0 comments on commit 7139ccd

Please sign in to comment.