Optimization-based Lyapunov function construction for continuous-time Markov chains with affine transition rates

TL;DR

This paper presents an optimization-based method for constructing Lyapunov functions for continuous-time Markov chains with affine transition rates, enabling efficient analysis of their stationary behavior without stochastic simulation.

Contribution

The authors develop a novel optimization approach leveraging Foster-Lyapunov bounds to estimate stationary regions and moments for Markov chains with affine rates.

Findings

Efficient estimation of high stationary probability regions.

Upper bounds on moments of the Markov chain.

Method avoids stochastic simulation for stationary analysis.

Abstract

We address the problem of Lyapunov function construction for a class of continuous-time Markov chains with affine transition rates, typically encountered in stochastic chemical kinetics. Following an optimization approach, we take advantage of existing bounds from the Foster-Lyapunov stability theory to obtain functions that enable us to estimate the region of high stationary probability, as well as provide upper bounds on moments of the chain. Our method can be used to study the stationary behavior of a given chain without resorting to stochastic simulation, in a fast and efficient manner.