Entropy of finite random binary sequences with weak long-range correlations

TL;DR

This paper develops a method to calculate the entropy of finite binary sequences with weak long-range correlations using pair correlation functions, enabling analysis of longer sequences and revealing self-similar entropy structures, with applications to DNA.

Contribution

It introduces a novel approach expressing entropy through pair correlators, allowing entropy calculation at longer distances than traditional methods.

Findings

Entropy can be expressed as a functional of pair correlations.

Finiteness introduces a significant fluctuation contribution to entropy.

Self-similar entropy structures are observed under decimation transformations.

Abstract

We study the N-step binary stationary ergodic Markov chain and analyze its differential entropy. Supposing that the correlations are weak we express the conditional probability function of the chain through the pair correlation function and represent the entropy as a functional of the pair correlator. Since the model uses the two-point correlators instead of the block probability, it makes it possible to calculate the entropy of strings at much longer distances than using standard methods. A fluctuation contribution to the entropy due to finiteness of random chains is examined. This contribution can be of the same order as its regular part even at the relatively short lengths of subsequences. A self-similar structure of entropy with respect to the decimation transformations is revealed for some specific forms of the pair correlation function. Application of the theory to the DNA sequence of the R3 chromosome of Drosophila melanogaster is presented.