or-gym

Environments for OR and RL Research

This library contains environments consisting of operations research problems which adhere to the OpenAI Gym API. The purpose is to bring reinforcement learning to the operations research community via accessible simulation environments featuring classic problems that are solved both with reinforcement learning as well as traditional OR techniques.

Installation

This library requires Python 3.5+ in order to function.

For the RL algorithms, Ray 1.0.0 is required.

For the MP algorithms, an LP solver (e.g., Gurobi or GLPK needs to be installed).

Installation is possible via pip:

You can install directly from GitHub with:

git clone https://github.com/grossmann-group/or-gym.git
cd or-gym
pip install -e .

Citation

@misc{HubbsOR-Gym,
    author={Christian D. Hubbs and Hector D. Perez and Owais Sarwar and Nikolaos V. Sahinidis and Ignacio E. Grossmann and John M. Wassick},
    title={OR-Gym: A Reinforcement Learning Library for Operations Research Problems},
    year={2020},
    Eprint={arXiv:2008.06319}
}

Environments

• Knapsack-v0: a small version of the classic unbounded knapsack problem with 200 items.
• Knapsack-v1: binary (0-1) knapsack problem with 200 items.
• Knapsack-v2: bounded knapsack problem with 200 items.
• Knapsack-v3: stochastic, online knapsack problem.
• BinPacking-v0 through BinPacking-v5: online bin packing problem taken from Balaji et al..
• Newsvendor-v0: multi-period newsvendor problem with lead times taken from Balaji et al..
• VMPacking-v0: permanent, multi-dimensional virtual machine packing problem.
• VMPacking-v1: temporary, multi-dimensional virtual machine packing problem.
• VehicleRouting-v0: pick-up and delivery problem with delivery windows taken from Balaji et al..
• InvManagement-v0: multi-echelon supply chain re-order problem with backlogs.
• InvManagement-v1: multi-echelon supply chain re-order problem without backlog.
• NetworkManagement-v0: multi-echelon supply chain network problem with backlogs from Perez et al..
• NetworkManagement-v1: multi-echelon supply chain network problem without backlogs from Perez et al..
• PortfolioOpt-v0: Multi-period asset allocation problem for managing investment decisions taken from Dantzig and Infanger.
• TSP-v0: traveling salesman problem with bi-directional connections and uniform cost.
• TSP-v1: traveling salesman problem with bi-directional connections.

Resources

Information on results and supporting models can be found here.

Examples

• Action Masking with RLlib using the Knapsack Environment
• How to Use Deep Reinforcement Learning to Improve your Supply Chain
• MDPI Article:

@article{perez2021algorithmic,
  title={Algorithmic Approaches to Inventory Management Optimization},
  author={Perez, Hector D and Hubbs, Christian D and Li, Can and Grossmann, Ignacio E},
  journal={Processes},
  volume={9},
  number={1},
  pages={102},
  year={2021},
  publisher={Multidisciplinary Digital Publishing Institute}
}