MORL Baselines Release v1.1.0: Gymnasium 1.0, MORL/D, HPO, bug fixes & improvements

What's Changed

• Move util functions to make more sense by @ffelten in #62
• Implement fast pareto and convex hull pruning by @wilrop in #60
• Support Gymnasium 0.29 by @LucasAlegre in #64
• Add minecart-deterministic-v0 to envs with known pfs by @ffelten in #65
• Support for image-based observations in GPI-LS and Envelope by @LucasAlegre in #63
• Implement a fix to #69 by @wilrop in #70
• Feature/hpo by @ffelten in #74
• Use deterministic policies when evaluating PCN by @vaidas-sl in #75
• Add bibtex for NeurIPS paper by @ffelten in #77
• Update README.md by @eltociear in #80
• Log all training parameters given to the algorithm by @wilrop in #83
• Add support for continuous action spaces to PCN by @vaidas-sl in #82
• Fix bug where tolist was called on a float by @wilrop in #86
• Adding cardinality metric by @ffelten in #87
• Remove dropout at evaluation time on GPI-LS by @LucasAlegre in #90
• MORL/D by @ffelten in #89
• Chore/remove tensorboard stuff make gym logging optional by @ffelten in #94
• Make number of sampled weights used to compute utility metrics parameterizable by @LucasAlegre in #95
• Dev eupg by @omidsbhn in #99
• Fix PGMORL example by @ffelten in #104
• Bump actions/download-artifact from 2 to 4.1.7 in /.github/workflows by @dependabot in #116
• Fix EUPG's inconsistent expansion of observations by @timondesch in #119
• Migration to gymnasium 1.0 by @ffelten in #109
• Add save_freq option to CAPQL train method by @LucasAlegre in #122
• In GPI-LS for continuous action, use GPI only for selecting weights as default by @LucasAlegre in #126
• Add python 3.12 support by @ffelten in #125

New Contributors

• @vaidas-sl made their first contribution in #75
• @eltociear made their first contribution in #80
• @omidsbhn made their first contribution in #99
• @dependabot made their first contribution in #116
• @timondesch made their first contribution in #119

Full Changelog: 1.0.0...v1.1.0

MORL-Baselines 1.0.0

This release marks the first stable version of MORL-Baselines. After having thoroughly tested the algorithms on various environments fixing bugs for the past few weeks. We feel the library is stable enough to deserve a proper release.

Features

• Over 10 MORL algorithms supported under the MO-Gymnasium API (multi & single policy, under SER and ESR criteria);
• Automated reporting to Weights and Biases dashboards... of various metrics (see screenshot below);
• Clean, documented, and tested code, and this is enforced by our CI hooks;
• Utility functions to help researchers build new algorithms, e.g. ParetoArchive, NatureCNN, PrioritizedReplayBuffer;
• Performances have been tested and reported in a reproducible manner: see #43 and https://wandb.ai/openrlbenchmark/MORL-Baselines.

Example of our dashboards: Pareto front and multi-objective metrics are visible in real-time.

1.0.0-rc2 bugfixes and enhancements

What's Changed

• Change PQL to linearly decaying exploration by @ffelten in #48
• Refactor random seed by @LucasAlegre in #49
• Recover from solver error in OLS by @ffelten in #51

Full Changelog: 1.0.0-rc1...1.0.0-rc2

1.0.0-rc1 Stabilizing and performance assessment

First release candidate aiming at stabilizing and reporting the performances of the algorithms in the codebase. We aim to fix bugs as we encounter them when assessing performances and bumping RC numbers along the way. Once we have finished the performance assessments, we should be able to release 1.0.0.