Entropy-Variance curves of binary sequences generated by random substitutions of constant length

TL;DR

This paper analyzes the entropy and variance of binary sequences generated by a specific random substitution process, deriving their statistical properties and revealing two distinct regimes of entropy-variance dependence.

Contribution

It introduces a novel analysis of entropy-variance curves for sequences generated by constant-length random substitutions, including explicit formulas and characterization of regimes.

Findings

Derived the probability distribution of the number of ones after substitutions.

Calculated the mean, variance, and entropy as functions of substitution length and iteration.

Identified two regimes of entropy-variance dependence in the generated sequences.

Abstract

We study some properties of binary sequences generated by random substitutions of constant length. Specifically, assuming the alphabet $\{0,1\}$, we consider the following asymmetric substitution rule of length $k$: $0 \to \langle 0, 0, \ldots,0\rangle$ and $1 \to \langle Y_1, Y_2, \ldots, Y_k \rangle$, where $Y_i$ is a Bernoulli random variable with parameter $p \in [0,1]$. We obtain by recurrence the discrete probability distribution of the stochastic variable that counts the number of ones in the sequence formed after a number $i$ of substitutions (iterations). We derive its first two statistical moments, mean and variance, and the entropy of the generated sequences as a function of the substitution length $k$ for any successive iteration $i$, and characterize the values of $p$ where the maxima of these measures occur. Finally, we obtain the parametric curves entropy-variance for each iteration and substitution length. We find two regimes of dependence between these two variables that, to our knowledge, have not been previously described. Besides, it allows to compare sequences with the same entropy but different variance and vice versa.