Scaling behaviour of entropy estimates

TL;DR

This paper investigates the scaling behavior of entropy estimates, particularly Lempel-Ziv estimates, for complex dynamical systems with long-range correlations, revealing their asymptotic properties at critical points.

Contribution

It provides the first analysis of the scaling law of Lempel-Ziv entropy estimates for the critical circle map and Feigenbaum point of the logistic map.

Findings

Lempel-Ziv entropy estimates scale logarithmically with sequence length.

The scaling law differs from probabilistic entropy, highlighting non-probabilistic complexity.

Results enhance understanding of entropy estimation in systems with long memory.

Abstract

Entropy estimation of information sources is highly non trivial for symbol sequences with strong long-range correlations. The rabbit sequence, related to the symbolic dynamics of the nonlinear circle map at the critical point as well as the logistic map at the Feigenbaum point have been argued to exhibit long memory tails. For both dynamical systems the scaling behavior of the block entropy of order n has been shown to increase like as log(n). In contrast to probabilistic concepts, we investigate the scaling behavior of certain non-probabilistic entropy estimation schemes suggested by Lempel and Ziv in the context of algorithmic complexity and data compression. These are applied in a sequential manner with the scaling variable being the length N of the sequence. We determine the scaling law for the Lempel-Ziv entropy estimate applied to the case of the critical circle map and the logistic map at the Feigenbaum point in a binary partition.