The Role of Information in System Stability with Partially Observable Servers

TL;DR

This paper investigates how different levels of information availability affect the stability of a controlled queueing system with Markov-modulated servers, using POMDP and QBD models to analyze stability bounds.

Contribution

It introduces a POMDP framework for stability analysis in partially observable queueing systems and derives matrix-analytic expressions for stability bounds.

Findings

Stability region expands with increased belief states.

Closed-form stability descriptions are lacking for the considered regimes.

Numerical methods reveal structural properties of the stability region.

Abstract

We consider a simple discrete-time controlled queueing system, where the controller has a choice of which server to use at each time slot and server performance varies according to a Markov modulated random environment. We explore the role of information in the system stability region. At the extreme cases of information availability, that is when there is either full information or no information, stability regions and maximally stabilizing policies are trivial. But in the more realistic cases where only the environment state of the selected server is observed, only the service successes are observed or only queue length is observed, finding throughput maximizing control laws is a challenge. To handle these situations, we devise a Partially Observable Markov Decision Process (POMDP) formulation of the problem and illustrate properties of its solution. We further model the system under given decision rules, using Quasi-Birth-and-Death (QBD) structure to find a matrix analytic expression for the stability bound. We use this formulation to illustrate how the stability region grows as the number of controller belief states increases. Our focus in this paper is on the simple case of two servers where the environment of each is modulated according to a two-state Markov chain. As simple as this case seems, there appear to be no closed form descriptions of the stability region under the various regimes considered. Our numerical approximations to the POMDP Bellman equations and the numerical solutions of the QBDs hint at a variety of structural results.