Mixing, Ergodic, and Nonergodic Processes with Rapidly Growing Information between Blocks

TL;DR

This paper constructs specific mixing and ergodic processes with mutual information between blocks growing as a power law, advancing understanding of information growth in stochastic processes.

Contribution

It introduces new mixing and ergodic processes with mutual information growth rates of n^β, extending nonergodic Santa Fe processes and analyzing their properties.

Findings

Mutual information between blocks grows as n^β for 0<β<1.

Infinite direct products of mixing processes remain mixing.

Rates of mutual information for modified Santa Fe processes are characterized.

Abstract

We construct mixing processes over an infinite alphabet and ergodic processes over a finite alphabet for which Shannon mutual information between adjacent blocks of length $n$ grows as $n^\beta$, where $\beta\in(0,1)$. The processes are a modification of nonergodic Santa Fe processes, which were introduced in the context of natural language modeling. The rates of mutual information for the latter processes are alike and also established in this paper. As an auxiliary result, it is shown that infinite direct products of mixing processes are also mixing.