Problems of robustness for universal coding schemes

TL;DR

This paper investigates the robustness of universal coding schemes, like Lempel-Ziv, under slight deviations from ergodicity, showing they are generally non-robust except for sequences from ergodic Markov chains.

Contribution

It introduces a measure of randomness deficiency to analyze the robustness of universal coding schemes and demonstrates their limitations under small violations of ergodicity.

Findings

Universal schemes are non-robust if randomness deficiency grows slowly.

Lempel-Ziv algorithms are robust for sequences from ergodic Markov chains.

Robustness depends on the growth rate of initial fragment deficiency.

Abstract

The Lempel-Ziv universal coding scheme is asymptotically optimal for the class of all stationary ergodic sources. A problem of robustness of this property under small violations of ergodicity is studied. A notion of deficiency of algorithmic randomness is used as a measure of disagreement between data sequence and probability measure. We prove that universal compressing schemes from a large class are non-robust in the following sense: if the randomness deficiency grows arbitrarily slowly on initial fragments of an infinite sequence then the property of asymptotic optimality of any universal compressing algorithm can be violated. Lempel-Ziv compressing algorithms are robust on infinite sequences generated by ergodic Markov chains when the randomness deficiency of its initial fragments of length $n$ grows as $o(n)$.