Empirical and Instance-Dependent Estimation of Markov Chain and Mixing Time

TL;DR

This paper proposes a new data-driven method for estimating the mixing time of Markov chains using contraction coefficients, providing more practical and instance-specific insights than traditional spectral gap approaches.

Contribution

It introduces a contraction-based estimation approach for mixing time, with improved confidence intervals and instance-dependent analysis beyond worst-case scenarios.

Findings

Thinner, data-dependent confidence intervals for contraction coefficients.

Method applicable to non-reversible Markov chains.

Instance-dependent rates for transition matrix estimation.

Abstract

We address the problem of estimating the mixing time of a Markov chain from a single trajectory of observations. Unlike most previous works which employed Hilbert space methods to estimate spectral gaps, we opt for an approach based on contraction with respect to total variation. Specifically, we estimate the contraction coefficient introduced in Wolfer [2020], inspired from Dobrushin's. This quantity, unlike the spectral gap, controls the mixing time up to strong universal constants and remains applicable to non-reversible chains. We improve existing fully data-dependent confidence intervals around this contraction coefficient, which are both easier to compute and thinner than spectral counterparts. Furthermore, we introduce a novel analysis beyond the worst-case scenario by leveraging additional information about the transition matrix. This allows us to derive instance-dependent rates for estimating the matrix with respect to the induced uniform norm, and some of its mixing properties.