Optimal control of Markov-modulated multiclass many-server queues

TL;DR

This paper analyzes multiclass many-server queues modulated by a Markov process, establishing ergodicity, a functional limit theorem, and asymptotic optimality for control policies in a critically loaded regime.

Contribution

It introduces a framework for analyzing Markov-modulated queues in the Halfin-Whitt regime, proving ergodicity, deriving diffusion limits, and solving optimal control problems.

Findings

Markov-modulated queues are geometrically ergodic under static priority policies.

Diffusion limits are characterized as controlled diffusions with piecewise linear drift.

Optimal control policies are asymptotically optimal in the long run.

Abstract

We study multiclass many-server queues for which the arrival, service and abandonment rates are all modulated by a common finite-state Markov process. We assume that the system operates in the "averaged" Halfin-Whitt regime, which means that it is critically loaded in the average sense, although not necessarily in each state of the Markov process. We show that under any static priority policy, the Markov-modulated diffusion-scaled queueing process is geometrically ergodic. This is accomplished by employing a solution to an associated Poisson equation in order to construct a suitable Lyapunov function. We establish a functional central limit theorem for the diffusion-scaled queueing process and show that the limiting process is a controlled diffusion with piecewise linear drift and constant covariance matrix. We address the infinite-horizon discounted and long-run average (ergodic) optimal control problems and establish asymptotic optimality.