On the Ziv-Merhav theorem beyond Markovianity II: leveraging the thermodynamic formalism

TL;DR

This paper extends the Ziv-Merhav theorem to a broader class of measures using thermodynamic formalism, demonstrating the asymptotic consistency of a modified cross-entropy estimator beyond Markovian processes.

Contribution

It introduces a thermodynamic formalism approach to analyze a modified cross-entropy estimator for general shift space measures, extending previous Markovian results.

Findings

Proves asymptotic consistency of the estimator for stationary, irreducible, finite-state Markov chains.

Develops the concept of cross-entropic pressure within thermodynamic formalism.

Extends the Ziv-Merhav theorem beyond Markovian settings.

Abstract

We prove asymptotic results for a modification of the cross-entropy estimator originally introduced by Ziv and Merhav in the Markovian setting in 1993. Our results concern a more general class of decoupled measures on shift spaces over a finite alphabet and in particular imply strong asymptotic consistency of the modified estimator for all pairs of functions of stationary, irreducible, finite-state Markov chains satisfying a mild decay condition. Our approach is based on the study of a rescaled cumulant-generating function called the cross-entropic pressure, importing to information theory some techniques from the study of large deviations within the thermodynamic formalism.