A Hoeffding inequality for Markov chains

TL;DR

This paper establishes improved deviation bounds for sums of functions of Markov chain states, enhancing previous Hoeffding inequalities by reducing dependence on the maximum individual bounds.

Contribution

It introduces a new Hoeffding inequality for Markov chains with dependence on the quadratic sum of bounds, not the maximum, and extends to vector-valued sums and matrix norms.

Findings

Bound improves dependence from max bound times sqrt(n) to sqrt of sum of squares.

Provides deviation bounds for vector-valued sums from Markov chains.

Applies to bounding expected Schatten norm of Markov chain-based random matrices.

Abstract

We prove deviation bounds for the random variable $\sum_{i=1}^{n} f_i(Y_i)$ in which $\{Y_i\}_{i=1}^{\infty}$ is a Markov chain with stationary distribution and state space $[N]$, and $f_i: [N] \rightarrow [-a_i, a_i]$. Our bound improves upon previously known bounds in that the dependence is on $\sqrt{a_1^2+\cdots+a_n^2}$ rather than $\max_{i}\{a_i\}\sqrt{n}.$ We also prove deviation bounds for certain types of sums of vector--valued random variables obtained from a Markov chain in a similar manner. One application includes bounding the expected value of the Schatten $\infty$-norm of a random matrix whose entries are obtained from a Markov chain.