Statistical Estimation of Ergodic Markov Chain Kernel over Discrete State Space

TL;DR

This paper analyzes the statistical complexity of estimating Markov chain kernels over discrete state spaces, revealing how mixing properties influence sample complexity in finite and countably infinite cases.

Contribution

It provides a characterization of the minimax sample complexity for Markov kernel estimation based on mixing properties, extending to both finite and countably infinite state spaces.

Findings

Sample complexity depends on the chain's mixing properties.

Finite-sample estimators with empirical confidence intervals exist for finite states.

Analysis extends to countably infinite state spaces with natural norms.

Abstract

We investigate the statistical complexity of estimating the parameters of a discrete-state Markov chain kernel from a single long sequence of state observations. In the finite case, we characterize (modulo logarithmic factors) the minimax sample complexity of estimation with respect to the operator infinity norm, while in the countably infinite case, we analyze the problem with respect to a natural entry-wise norm derived from total variation. We show that in both cases, the sample complexity is governed by the mixing properties of the unknown chain, for which, in the finite-state case, there are known finite-sample estimators with fully empirical confidence intervals.