Stationary and Transition Probabilities in Slow Mixing, Long Memory Markov Processes

TL;DR

This paper investigates how to estimate stationary and transition probabilities in unknown, slow-mixing, long-memory Markov processes from finite samples, providing conditions for accurate estimation and deviation bounds.

Contribution

It introduces methods to identify when naive estimates are accurate in slow-mixing Markov models, extending understanding beyond existing pointwise convergence results.

Findings

Conditions for accurate probability estimates in slow-mixing models

Bounds on deviations of naive estimates from true probabilities

Extension of universal compression techniques to slow-mixing scenarios

Abstract

We observe a length-$n$ sample generated by an unknown,stationary ergodic Markov process (\emph{model}) over a finite alphabet $\mathcal{A}$. Given any string $\bf{w}$ of symbols from $\mathcal{A}$ we want estimates of the conditional probability distribution of symbols following $\bf{w}$, as well as the stationary probability of $\bf{w}$. Two distinct problems that complicate estimation in this setting are (i) long memory, and (ii) \emph{slow mixing} which could happen even with only one bit of memory. Any consistent estimator in this setting can only converge pointwise over the class of all ergodic Markov models. Namely, given any estimator and any sample size $n$, the underlying model could be such that the estimator performs poorly on a sample of size $n$ with high probability. But can we look at a length-$n$ sample and identify \emph{if} an estimate is likely to be accurate? Since the memory is unknown \emph{a-priori}, a natural approach is to estimate a potentially coarser model with memory $k_n=\mathcal{O}(\log n)$. As $n$ grows, pointwise consistent estimates that hold eventually almost surely (eas) are known so long as the scaling of $k_n$ is not superlogarithmic in $n$. Here, rather than eas convergence results, we want the best answers possible with a length-$n$ sample. Combining results in universal compression with Aldous' coupling arguments, we obtain sufficient conditions on the length-$n$ sample (even for slow mixing models) to identify when naive (i) estimates of the conditional probabilities and (ii) estimates related to the stationary probabilities are accurate; and also bound the deviations of the naive estimates from true values.