On the Entropy Rate of Pattern Processes

TL;DR

This paper investigates the entropy rate of pattern sequences derived from various stochastic processes, providing a comprehensive characterization of their relationship to the original process's entropy rate across different types of processes.

Contribution

It offers a complete characterization of the entropy rate relationship for i.i.d., stationary ergodic, and certain uncountable alphabet processes, including growth rates for infinite entropy cases.

Findings

Characterization for i.i.d. processes over arbitrary alphabets

Results for stationary ergodic processes over discrete alphabets

Analysis of entropy growth rates for infinite entropy pattern processes

Abstract

We study the entropy rate of pattern sequences of stochastic processes, and its relationship to the entropy rate of the original process. We give a complete characterization of this relationship for i.i.d. processes over arbitrary alphabets, stationary ergodic processes over discrete alphabets, and a broad family of stationary ergodic processes over uncountable alphabets. For cases where the entropy rate of the pattern process is infinite, we characterize the possible growth rate of the block entropy.