On Match Lengths, Zero Entropy and Large Deviations - with Application to Sliding Window Lempel-Ziv Algorithm

TL;DR

This paper analyzes match lengths and recurrence times in the Sliding Window Lempel-Ziv algorithm for sources with zero entropy and weak memory, establishing optimality, compression ratios, and large deviation properties.

Contribution

It provides a detailed study of match lengths and recurrence times under zero entropy and mixing conditions, proving optimality and deriving compression ratios and large deviation principles.

Findings

Optimality of FSLZ and SWLZ algorithms for zero entropy sources.

Compression ratio of O(log n_w / n_w^a) for certain ergodic processes.

Large deviation property established for mixing processes.

Abstract

The Sliding Window Lempel-Ziv (SWLZ) algorithm that makes use of recurrence times and match lengths has been studied from various perspectives in information theory literature. In this paper, we undertake a finer study of these quantities under two different scenarios, i) \emph{zero entropy} sources that are characterized by strong long-term memory, and ii) the processes with weak memory as described through various mixing conditions. For zero entropy sources, a general statement on match length is obtained. It is used in the proof of almost sure optimality of Fixed Shift Variant of Lempel-Ziv (FSLZ) and SWLZ algorithms given in literature. Through an example of stationary and ergodic processes generated by an irrational rotation we establish that for a window of size $n_w$, a compression ratio given by $O(\frac{\log n_w}{{n_w}^a})$ where $a$ depends on $n_w$ and approaches 1 as $n_w \rightarrow \infty$, is obtained under the application of FSLZ and SWLZ algorithms. Also, we give a general expression for the compression ratio for a class of stationary and ergodic processes with zero entropy. Next, we extend the study of Ornstein and Weiss on the asymptotic behavior of the \emph{normalized} version of recurrence times and establish the \emph{large deviation property} (LDP) for a class of mixing processes. Also, an estimator of entropy based on recurrence times is proposed for which large deviation principle is proved for sources satisfying similar mixing conditions.