A New Method for Computing Stationary Distribution and Steady-State Performance Measures of a Continuous-State Markov Chain with a Queuing Application

TL;DR

This paper introduces a novel numerical method for approximating the stationary distribution and steady-state performance metrics of continuous-state Markov chains, with applications to queueing systems, providing accuracy guarantees and outperforming existing methods.

Contribution

The paper develops a general, finite approximation approach for continuous-state Markov chains, with proven consistency, error bounds, and superior numerical performance.

Findings

Method achieves high accuracy in queueing applications

Outperforms standard MCMC by several orders of magnitude

Provides deterministic error bounds for approximations

Abstract

Applications of stochastic models often involve the evaluation of steady-state performance, which requires solving a set of balance equations. In most cases of interest, the number of equations is infinite or even uncountable. As a result, numerical or analytical solutions are unavailable. This is true even when the system state is one-dimensional. This paper develops a general method for computing stationary distributions and steady-state performance measures of stochastic systems that can be described as continuous-state Markov chains supported on R. The balance equations are numerically solved by properly constructing a proxy Markov chain with finite states. We show the consistency of the approximate solution and provide deterministic non-asymptotic error bounds under the supremum norm. Our finite approximation method is near-optimal among all approximation methods using discrete distributions, including the empirical distributions generated by a simulation approach. We apply the developed method to compute the stationary distribution of virtual waiting time in a G/G/1+G queue and associated performance measures under certain mild but general differentiability and boundedness assumptions on the inter-arrival, service, and patience time distributions. Numerical experiments validate the accuracy and efficiency of our method, and show it outperforms a standard Markov chain Monte Carlo method by several orders of magnitude. The developed method is also significantly more accurate than the available fluid approximations for this queue.