Lempel-Ziv Complexity, Empirical Entropies, and Chain Rules

TL;DR

This paper establishes bounds on the compression ratio of the Lempel-Ziv algorithm using empirical entropies and introduces a chain rule for LZ complexity that decomposes joint complexity into individual and conditional complexities.

Contribution

It derives bounds on LZ compression ratios based on empirical entropies and formulates a chain rule for LZ complexity, enhancing understanding of sequence decomposition.

Findings

Bounds relate LZ compression to empirical entropies.

A chain rule for LZ complexity decomposes joint complexity.

Insights into the trade-offs of block length adjustments.

Abstract

We derive upper and lower bounds on the overall compression ratio of the 1978 Lempel-Ziv (LZ78) algorithm, applied independently to $k$-blocks of a finite individual sequence. Both bounds are given in terms of normalized empirical entropies of the given sequence. For the bounds to be tight and meaningful, the order of the empirical entropy should be small relative to $k$ in the upper bound, but large relative to $k$ in the lower bound. Several non-trivial conclusions arise from these bounds. One of them is a certain form of a chain rule of the Lempel-Ziv (LZ) complexity, which decomposes the joint LZ complexity of two sequences, say, $\bx$ and $\by$, into the sum of the LZ complexity of $\bx$ and the conditional LZ complexity of $\by$ given $\bx$ (up to small terms). The price of this decomposition, however, is in changing the length of the block. Additional conclusions are discussed as well.