Detecting Markov Chain Instability: A Monte Carlo Approach

TL;DR

This paper introduces a Monte Carlo method using simulated annealing to detect instability in non-negative Markov chains across parameter spaces, providing statistically rigorous tests with applications to queueing networks.

Contribution

It presents a novel Monte Carlo algorithm for stability detection in Markov chains, leveraging a theoretical link between parameter set stability and chain stability.

Findings

Algorithm successfully detects instability in various queueing network models.

Provides performance guarantees under mild assumptions.

Enables statistically rigorous instability testing.

Abstract

We devise a Monte Carlo based method for detecting whether a non-negative Markov chain is stable for a given set of parameter values. More precisely, for a given subset of the parameter space, we develop an algorithm that is capable of deciding whether the set has a subset of positive Lebesgue measure for which the Markov chain is unstable. The approach is based on a variant of simulated annealing, and consequently only mild assumptions are needed to obtain performance guarantees. The theoretical underpinnings of our algorithm are based on a result stating that the stability of a set of parameters can be phrased in terms of the stability of a single Markov chain that searches the set for unstable parameters. Our framework leads to a procedure that is capable of performing statistically rigorous tests for instability, which has been extensively tested using several examples of standard and non-standard queueing networks.