Large deviation principles for words drawn from correlated letter sequences

TL;DR

This paper extends large deviation principles for empirical word processes from i.i.d. letter sequences to correlated sequences satisfying a mixing condition, with rate functions based on relative entropy.

Contribution

It generalizes existing LDP results to non-i.i.d. letter sequences under summable variation, providing new rate functions and approximation techniques.

Findings

LDPs extend to correlated letter sequences with summable variation.

Rate functions are expressed via specific relative entropies.

Both annealed and quenched LDPs are established for non-i.i.d. sequences.

Abstract

When an i.i.d.\ sequence of letters is cut into words according to i.i.d.\ renewal times, an i.i.d.\ sequence of words is obtained. In the \emph{annealed} LDP (large deviation principle) for the empirical process of words, the rate function is the specific relative entropy of the observed law of words w.r.t.\ the reference law of words. In Birkner, Greven and den Hollander \cite{BGdH10} the \emph{quenched} LDP (= conditional on a typical letter sequence) was derived for the case where the renewal times have an \emph{algebraic} tail. The rate function turned out to be a sum of two terms, one being the annealed rate function, the other being proportional to the specific relative entropy of the observed law of letters w.r.t.\ the reference law of letters, obtained by concatenating the words and randomising the location of the origin. The proportionality constant equals the tail exponent of the renewal process. The purpose of the present paper is to extend both LDP's to letter sequences that are not i.i.d. It is shown that both LDP's carry over when the letter sequence satisfies a mixing condition called \emph{summable variation}. The rate functions are again given by specific relative entropies w.r.t.\ the reference law of words, respectively, letters. But since neither of these reference laws is i.i.d., several approximation arguments are needed to obtain the extension.