QGym: Scalable Simulation and Benchmarking of Queuing Network Controllers

TL;DR

QGym is an open-source simulation framework designed to benchmark and compare various queueing control policies, including RL and classical methods, across diverse realistic network scenarios to advance research in resource allocation.

Contribution

The paper introduces QGym, a modular, open-source simulation platform for benchmarking queueing policies in complex, realistic environments, addressing a gap in empirical evaluation tools.

Findings

QGym supports diverse queueing network types including hospital systems.

It enables comparison of RL and classical queueing policies.

The framework broadens empirical benchmarking in queueing control research.

Abstract

Queuing network control determines the allocation of scarce resources to manage congestion, a fundamental problem in manufacturing, communications, and healthcare. Compared to standard RL problems, queueing problems are distinguished by unique challenges: i) a system operating in continuous time, ii) high stochasticity, and iii) long horizons over which the system can become unstable (exploding delays). To spur methodological progress tackling these challenges, we present an open-sourced queueing simulation framework, QGym, that benchmark queueing policies across realistic problem instances. Our modular framework allows the researchers to build on our initial instances, which provide a wide range of environments including parallel servers, criss-cross, tandem, and re-entrant networks, as well as a realistically calibrated hospital queuing system. QGym makes it easy to compare multiple policies, including both model-free RL methods and classical queuing policies. Our testbed complements the traditional focus on evaluating algorithms based on mathematical guarantees in idealized settings, and significantly expands the scope of empirical benchmarking in prior work. QGym code is open-sourced at https://github.com/namkoong-lab/QGym.