Admission Control of Quasi-Reversible Queueing Systems: Optimization and Reinforcement Learning

TL;DR

This paper develops a unified approach for optimizing admission control in quasi-reversible queueing systems, combining theoretical insights with reinforcement learning techniques to enhance system performance.

Contribution

It introduces a new definition of quasi-reversibility, extends balanced arrival policies, and applies these concepts to optimize admission control via reinforcement learning.

Findings

Balanced arrival control preserves quasi-reversibility.

The stationary measures are explicitly characterized.

Reinforcement learning effectively optimizes admission policies.

Abstract

In this paper, we introduce a versatile scheme for optimizing the arrival rates of quasi-reversible queueing systems. We first propose an alternative definition of quasi-reversibility that encompasses reversibility and highlights the importance of the definition of customer classes. Then we introduce balanced arrival control policies, which generalize the notion of balanced arrival rates introduced in the context of Whittle networks, to the much broader class of quasi-reversible queueing systems. We prove that supplementing a quasi-reversible queueing system with a balanced arrival-control policy preserves the quasi-reversibility, and we specify the form of the stationary measures. We revisit two canonical examples of quasi-reversible queueing systems, Whittle networks and order-independent queues. Lastly, we focus on the problem of admission control and leverage our results in the frameworks of optimization and reinforcement learning.