OR-Gym: A Reinforcement Learning Library for Operations Research Problems

TL;DR

This paper introduces OR-Gym, an open-source reinforcement learning library tailored for operations research problems, demonstrating its application and benchmarking against traditional optimization methods.

Contribution

The paper presents OR-Gym, a novel library that adapts classic OR problems into RL environments, enabling new approaches and benchmarking in operations research.

Findings

RL solutions outperform heuristics in certain problems

Benchmarking shows RL can be competitive with MILP methods

OR-Gym facilitates cross-disciplinary research in RL and OR

Abstract

Reinforcement learning (RL) has been widely applied to game-playing and surpassed the best human-level performance in many domains, yet there are few use-cases in industrial or commercial settings. We introduce OR-Gym, an open-source library for developing reinforcement learning algorithms to address operations research problems. In this paper, we apply reinforcement learning to the knapsack, multi-dimensional bin packing, multi-echelon supply chain, and multi-period asset allocation model problems, as well as benchmark the RL solutions against MILP and heuristic models. These problems are used in logistics, finance, engineering, and are common in many business operation settings. We develop environments based on prototypical models in the literature and implement various optimization and heuristic models in order to benchmark the RL results. By re-framing a series of classic optimization problems as RL tasks, we seek to provide a new tool for the operations research community, while also opening those in the RL community to many of the problems and challenges in the OR field.