A computational approach to extreme values and related hitting probabilities in level-dependent quasi-birth-death processes

TL;DR

This paper develops a computational method to analyze extreme values and hitting probabilities in level-dependent quasi-birth-death processes, with applications to epidemic models.

Contribution

It introduces a novel approach using Laplace-Stieltjes transforms to characterize maximum levels and hitting times in complex Markov processes with block-tridiagonal structure.

Findings

Derived explicit formulas for maximum level probabilities

Applied methods to epidemic models demonstrating practical utility

Provided algorithms for computing hitting probabilities in level-dependent processes

Abstract

This paper analyzes the dynamics of a level-dependent quasi-birth-death process ${\cal X}=\{(I(t),J(t)): t\geq 0\}$, i.e., a bi-variate Markov chain defined on the countable state space $\cup_{i=0}^{\infty} l(i)$ with $l(i)=\{(i,j) : j\in\{0,...,M_i\}\}$, for integers $M_i\in\mathbb{N}_0$ and $i\in\mathbb{N}_0$, which has the special property that its $q$-matrix has a block-tridiagonal form. Under the assumption that the first passage to the subset $l(0)$ occurs in a finite time with certainty, we characterize the probability law of $(\tau_{\max},I_{\max},J(\tau_{\max}))$, where $I_{\max}$ is the running maximum level attained by process ${\cal X}$ before its first visit to states in $l(0)$, $\tau_{\max}$ is the first time that the level process $\{I(t): t\geq 0\}$ reaches the running maximum $I_{\max}$, and $J(\tau_{\max})$ is the phase at time $\tau_{\max}$. Our methods rely on the use of restricted Laplace-Stieltjes transforms of $\tau_{\max}$ on the set of sample paths $\{I_{\max}=i,J(\tau_{\max})=j\}$, and related processes under taboo of certain subsets of states. The utility of the resulting computational algorithms is demonstrated in two epidemic models: the SIS model for horizontally and vertically transmitted diseases; and the SIR model with constant population size.