Random-Time, State-Dependent Stochastic Drift for Markov Chains and Application to Stochastic Stabilization Over Erasure Channels

TL;DR

This paper develops a randomized multi-step Lyapunov framework for analyzing Markov chain stability and applies it to remote control over erasure channels, establishing stability criteria related to channel capacity.

Contribution

It extends Lyapunov theory to randomized multi-step bounds and applies these results to stochastic stabilization over erasure channels, providing new stability conditions.

Findings

Stability is guaranteed if channel capacity exceeds the logarithm of the unstable eigenvalue plus a correction.

Finite second moments in steady-state are achievable under certain conditions.

The approach simplifies stability verification for complex stochastic network models.

Abstract

It is known that state-dependent, multi-step Lyapunov bounds lead to greatly simplified verification theorems for stability for large classes of Markov chain models. This is one component of the "fluid model" approach to stability of stochastic networks. In this paper we extend the general theory to randomized multi-step Lyapunov theory to obtain criteria for stability and steady-state performance bounds, such as finite moments. These results are applied to a remote stabilization problem, in which a controller receives measurements from an erasure channel with limited capacity. Based on the general results in the paper it is shown that stability of the closed loop system is assured provided that the channel capacity is greater than the logarithm of the unstable eigenvalue, plus an additional correction term. The existence of a finite second moment in steady-state is established under additional conditions.