On generators with infinite entropy

TL;DR

This paper explores a class of stationary processes with infinite entropy where the metric entropy equals the conditional entropy, extending Pitskel's observation beyond Markov chains.

Contribution

It identifies non-Markov processes for which Pitskel's entropy equality holds, broadening understanding of entropy in stationary processes.

Findings

Identifies processes with infinite entropy where metric entropy equals conditional entropy

Extends Pitskel's observation beyond Markov chains

Provides examples of non-Markov processes satisfying the entropy relation

Abstract

Many years ago B.S. Pitskel observed that the metric entropy of the shift transformation in the sample space of a stationary random process $X=\{X_n,\,n\in \mathbb Z\}$ with a countable number of states is equal to the conditional entropy $H(X_0|X_{-1},X_{-2},\dots)$ if $X$ is a stationary Markov chain (in which case the above conditional entropy is $H(X_0|X_{-1}))$, whether the entropy $H(X_0)$ is finite or not, while in general the statement is not true. In this note we present a class of processes for which Pitskel's observation holds, despite the fact that no of these processes is a Markov chain of some order.