On the Ziv-Merhav theorem beyond Markovianity

TL;DR

This paper extends the Ziv-Merhav theorem to a wider class of measures beyond Markovian processes, including g-measures and equilibrium measures, enhancing universal entropy estimation methods.

Contribution

It generalizes the Ziv-Merhav theorem to decoupled measures like g-measures and equilibrium measures, broadening its applicability.

Findings

Extends the theorem to non-Markovian measures.

Includes measures from statistical mechanics.

Provides a unified framework for entropy estimation.

Abstract

We generalize to a broader class of decoupled measures a result of Ziv and Merhav on universal estimation of the specific cross (or relative) entropy for a pair of multi-level Markov measures. The result covers pairs of suitably regular g-measures and pairs of equilibrium measures arising from the small space of interactions in mathematical statistical mechanics.