Information divergences of Markov chains and their applications

TL;DR

This paper introduces new information divergences for finite Markov chains, extending classical divergences, and explores their properties and applications in hypothesis testing, mixing times, and ergodicity analysis.

Contribution

It defines novel divergences for Markov chains, derives their fundamental properties, and applies them to hypothesis testing and mixing time analysis.

Findings

New divergences generalize classical information measures to Markov chains.

Derived Markov chain versions of Pinsker's inequality and Chernoff information.

Applied divergences to hypothesis testing and ergodicity analysis.

Abstract

In this paper, we first introduce and define several new information divergences in the space of transition matrices of finite Markov chains which measure the discrepancy between two Markov chains. These divergences offer natural generalizations of classical information-theoretic divergences, such as the $f$-divergences and the R\'enyi divergence between probability measures, to the context of finite Markov chains. We begin by detailing and deriving fundamental properties of these divergences and notably gives a Markov chain version of the Pinsker's inequality and Chernoff information. We then utilize these notions in a few applications. First, we investigate the binary hypothesis testing problem of Markov chains, where the newly defined R\'enyi divergence between Markov chains and its geometric interpretation play an important role in the analysis. Second, we propose and analyze information-theoretic (Ces\`aro) mixing times and ergodicity coefficients, along with spectral bounds of these notions in the reversible setting. Examples of the random walk on the hypercube, as well as the connections between the critical height of the low-temperature Metropolis-Hastings chain and these proposed ergodicity coefficients, are highlighted.