Renyi Entropy Rate of Stationary Ergodic Processes

TL;DR

This paper investigates the properties of Renyi entropy rate in stationary ergodic processes, proving existence and approximation methods for special cases, and disproving a conjecture about convergence to Shannon entropy rate.

Contribution

It establishes the existence and approximation techniques for Renyi entropy rate in certain processes and refutes a conjecture about its convergence to Shannon entropy rate.

Findings

Renyi entropy rate always exists for a special class of processes.

Polynomial and exponential approximation methods are developed.

Counterexample disproves the convergence conjecture as alpha approaches 1.

Abstract

In this paper, we examine the Renyi entropy rate of stationary ergodic processes. For a special class of stationary ergodic processes, we prove that the Renyi entropy rate always exists and can be polynomially approximated by its defining sequence; moreover, using the Markov approximation method, we show that the Renyi entropy rate can be exponentially approximated by that of the Markov approximating sequence, as the Markov order goes to infinity. For the general case, by constructing a counterexample, we disprove the conjecture that the Renyi entropy rate of a general stationary ergodic process always converges to its Shannon entropy rate as {\alpha} goes to 1.