Hoeffding's inequality for continuous-time Markov chains

TL;DR

This paper extends Hoeffding's inequality to continuous-time Markov chains, providing probabilistic bounds for their time-averaged functions, and discusses its application to jump processes on general state spaces.

Contribution

The paper introduces Hoeffding's inequality for irreducible, positive recurrent CTMCs with spectral gap, using skeleton chains and truncation techniques, extending previous discrete-time results.

Findings

Derived Hoeffding's inequality for CTMCs with spectral gap

Established probabilistic bounds for time-averaged functions of CTMCs

Discussed extension to jump processes on general state spaces

Abstract

Hoeffding's inequality is a fundamental tool widely applied in probability theory, statistics, and machine learning. In this paper, we establish Hoeffding's inequalities specifically tailored for an irreducible and positive recurrent continuous-time Markov chain (CTMC) on a countable state space with the invariant probability distribution ${\pi}$ and an $\mathcal{L}^{2}(\pi)$-spectral gap ${\lambda}(Q)$. More precisely, for a function $g:E\to [a,b]$ with a mean $\pi(g)$, and given $t,\varepsilon>0$, we derive the inequality \[ \mathbb{P}_{\pi}\left(\frac{1}{t} \int_{0}^{t} g\left(X_{s}\right)\mathrm{d}s-\pi (g) \geq \varepsilon \right) \leq \exp\left\{-\frac{{\lambda}(Q)t\varepsilon^2}{(b-a)^2} \right\}, \] which can be viewed as a generalization of Hoeffding's inequality for discrete-time Markov chains (DTMCs) presented in [J. Fan et al., J. Mach. Learn. Res., 22(2022), pp. 6185-6219] to the realm of CTMCs. The key analysis enabling the attainment of this inequality lies in the utilization of the techniques of skeleton chains and augmented truncation approximations. Furthermore, we also discuss Hoeffding's inequality for a jump process on a general state space.