Structural classification of continuous time Markov chains with applications

TL;DR

This paper provides a comprehensive classification of continuous time Markov chains based on their structure, extending beyond birth-death processes, with applications in biology, ecology, and network theory.

Contribution

It characterizes the structure of Markov chains via their Q-matrices and reaction graphs, introducing the concept of structural equivalence and broadening analysis beyond traditional birth-death models.

Findings

Characterization of state space decomposition into classes

Conditions for structural equivalence of Q-matrices

Applications to diverse models like ecology and systems biology

Abstract

This paper is motivated by examples from stochastic reaction network theory. The $Q$-matrix of a stochastic reaction network can be derived from the reaction graph, an edge-labelled directed graph encoding the jump vectors of an associated continuous time Markov chain on the invariant space $\mathbb{N}^d_0$. An open question is how to decompose the space $\mathbb{N}^d_0$ into neutral, trapping, and escaping states, and open and closed communicating classes, and whether this can be done from the reaction graph alone. Such general continuous time Markov chains can be understood as natural generalizations of birth-death processes, incorporating multiple different birth and death mechanisms. We characterize the structure of $\mathbb{N}^d_0$ imposed by a general $Q$-matrix generating continuous time Markov chains with values in $\mathbb{N}^d_0$, in terms of the set of jump vectors and their corresponding transition rate functions. Thus the setting is not limited to stochastic reaction networks. Furthermore, we define structural equivalence of two $Q$-matrices, and provide sufficient conditions for structural equivalence. Examples are abundant in applications. We apply the results to stochastic reaction networks, a Lotka-Volterra model in ecology, the EnvZ-OmpR system in systems biology, and a class of extended branching processes, none of which are birth-death processes.