Skip to content

Latest commit

 

History

History
168 lines (115 loc) · 10.2 KB

README.md

File metadata and controls

168 lines (115 loc) · 10.2 KB
Raw

MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learning

NeurIPS MetaBox has been published at NeurIPS 2023！

MetaBox is the first benchmark platform expressly tailored for developing and evaluating MetaBBO-RL methods. MetaBox offers a flexible algorithmic template that allows users to effortlessly implement their unique designs within the platform. Moreover, it provides a broad spectrum of over 300 problem instances, collected from synthetic to realistic scenarios, and an extensive library of 19 baseline methods, including both traditional black-box optimizers and recent MetaBBO-RL methods. Besides, MetaBox introduces three standardized performance metrics, enabling a more thorough assessment of the methods.

😁Contact Us

👨‍💻👩‍💻We are a research team mainly focus on Meta-Black-Box-Optimization (MetaBBO), which assists automated algorithm design for Evolutionary Computation.

Here is our homepage and github. 🥰🥰🥰Please feel free to contact us—any suggestions are welcome!

If you have any question or want to contact us：

  • 🌱Fork, Add, and Merge
  • ❓️Report an issue
  • 📧Contact WenJie Qiu (wukongqwj@gmail.com)
  • 🚨We warmly invite you to join our QQ group for further communication (Group Number: 952185139).

Installations

You can access all MetaBox files with the command:

git clone git@github.com:GMC-DRL/MetaBox.git
cd MetaBox

Citing MetaBox

The PDF version of the paper is available here. If you find our MetaBox useful, please cite it in your publications or projects.

@inproceedings{metabox,
author={Ma, Zeyuan and Guo, Hongshu and Chen, Jiacheng and Li, Zhenrui and Peng, Guojun and Gong, Yue-Jiao and Ma, Yining and Cao, Zhiguang},
title={MetaBox: A Benchmark Platform for Meta-Black-Box Optimization with Reinforcement Learning},
booktitle = {Advances in Neural Information Processing Systems},
year={2023},
volume = {36}
}

Requirements

Python >=3.7.1 with the following packages installed:

  • numpy==1.21.2
  • torch==1.9.0
  • matplotlib==3.4.3
  • pandas==1.3.3
  • scipy==1.7.1
  • scikit_optimize==0.9.0
  • deap==1.3.3
  • tqdm==4.62.3
  • openpyxl==3.1.2

Quick Start

  • To obtain the figures in our paper, run the following commands:

    cd for_review
python paper_experiment.py

    then corresponding figures will be output to for_revivew/pics.

    The quick usage of the four main running interfaces is listed as follows, in the following command, we specifically take RLEPSO as an example.

    Firstly, get into the main code folder, src:

    cd ../src

  • To trigger the entire workflow, including train, rollout and test, run the following command:

    python main.py --run_experiment --problem bbob --difficulty easy --train_agent RLEPSO_Agent --train_optimizer RLEPSO_Optimizer

  • To trigger the standalone process of training:

    python main.py --train --problem bbob --difficulty easy --train_agent RLEPSO_Agent --train_optimizer RLEPSO_Optimizer

  • To trigger the standalone process of testing:

    python main.py --test --problem bbob --difficulty easy --agent_load_dir agent_model/test/bbob_easy/ --agent_for_cp RLEPSO_Agent --l_optimizer_for_cp RLEPSO_Optimizer --t_optimizer_for_cp DEAP_CMAES Random_search

Documentation

For more details about the usage of MetaBox, please refer to MetaBox User's Guide.

Datasets

At present, three benchmark suites are integrated in MetaBox:

  • Synthetic contains 24 noiseless functions, borrowed from coco:bbob with original paper.
  • Noisy-Synthetic contains 30 noisy functions, borrowed from coco:bbob-noisy with original paper.
  • Protein-Docking contains 280 problem instances, which simulate the application of protein docking as a 12-dimensional optimization problem, borrowed from LOIS with original paper.

Baseline Library

7 MetaBBO-RL optimizers, 1 MetaBBO-SL optimizer and 11 classic optimizers have been integrated into MetaBox. They are listed below.

Supported MetaBBO-RL optimizers:

Name Year Related paper
DE-DDQN 2019 Deep reinforcement learning based parameter control in differential evolution
QLPSO 2019 A reinforcement learning-based communication topology in particle swarm optimization
DEDQN 2021 Differential evolution with mixed mutation strategy based on deep reinforcement learning
LDE 2021 Learning Adaptive Differential Evolution Algorithm From Optimization Experiences by Policy Gradient
RL-PSO 2021 Employing reinforcement learning to enhance particle swarm optimization methods
RLEPSO 2022 RLEPSO:Reinforcement learning based Ensemble particle swarm optimizer✱
RL-HPSDE 2022 Differential evolution with hybrid parameters and mutation strategies based on reinforcement learning
GLEET 2024 Auto-configuring Exploration-Exploitation Tradeoff in Evolutionary Computation via Deep Reinforcement Learning
SYMBOL 2024 Symbol: Generating Flexible Black-Box Optimizers through Symbolic Equation Learning
RL-DAS 2024 Deep Reinforcement Learning for Dynamic Algorithm Selection: A Proof-of-Principle Study on Differential Evolution

Supported MetaBBO-SL optimizer:

Name Year Related paper
RNN-OI 2017 Learning to learn without gradient descent by gradient descent

Supported MetaBBO-NE optimizer:

Name Year Related paper
LES 2023 Discovering evolution strategies via meta-black-box optimization

Supported classic optimizers:

Name Year Related paper
PSO 1995 Particle swarm optimization
DE 1997 Differential Evolution – A Simple and Efficient Heuristic for Global Optimization over Continuous Spaces
CMA-ES 2001 Completely Derandomized Self-Adaptation in Evolution Strategies
Bayesian Optimization 2014 Bayesian Optimization: Open source constrained global optimization tool for Python
GL-PSO 2015 Genetic Learning Particle Swarm Optimization
sDMS_PSO 2015 A Self-adaptive Dynamic Particle Swarm Optimizer
j21 2021 Self-adaptive Differential Evolution Algorithm with Population Size Reduction for Single Objective Bound-Constrained Optimization: Algorithm j21
MadDE 2021 Improving Differential Evolution through Bayesian Hyperparameter Optimization
SAHLPSO 2021 Self-Adaptive two roles hybrid learning strategies-based particle swarm optimization
NL_SHADE_LBC 2022 NL-SHADE-LBC algorithm with linear parameter adaptation bias change for CEC 2022 Numerical Optimization
Random Search - -

Note that Random Search is to randomly sample candidate solutions from the searching space.

Post-processing

In a bid to illustrate the utility of MetaBox for facilitating rigorous evaluation and in-depth analysis, as mentioned in our paper, we carry out a wide-ranging benchmarking study on existing MetaBBO-RL methods. The post-processed data is available in content.md.

Acknowledgements

The code and the framework are based on the repos DEAP, coco and Protein-protein docking V4.0.