An entropic characterization of long memory stationary process

TL;DR

This paper introduces an entropic approach to characterize long memory in stationary processes, defining long memory via infinite excess entropy, which applies even to processes with weak moment conditions.

Contribution

It proposes a novel entropic definition of long memory based on excess entropy, extending applicability to processes with unbounded moments and establishing invariance properties.

Findings

Excess entropy equals mutual information between past and future for finite entropy processes.

Fractional Gaussian noise has infinite excess entropy when Hurst parameter H > 1/2.

The new definition applies to processes without bounded second moments.

Abstract

Long memory or long range dependency is an important phenomenon that may arise in the analysis of time series or spatial data. Most of the definitions of long memory of a stationary process $X=\{X_1, X_2,\cdots,\}$ are based on the second-order properties of the process. The excess entropy of a stationary process is the summation of redundancies which relates to the rate of convergence of the conditional entropy $H(X_n|X_{n-1},\cdots, X_1)$ to the entropy rate. It is proved that the excess entropy is identical to the mutual information between the past and the future when the entropy $H(X_1)$ is finite. We suggest the definition that a stationary process is long memory if the excess entropy is infinite. Since the definition of excess entropy of a stationary process requires very weak moment condition on the distribution of the process, it can be applied to processes whose distributions without bounded second moment. A significant property of excess entropy is that it is invariant under invertible transformation, which enables us to know the excess entropy of a stationary process from the excess entropy of other process. For stationary Guassian process, the excess entropy characterization of long memory relates to popular characterization well. It is proved that the excess entropy of fractional Gaussian noise is infinite if the Hurst parameter $H \in (1/2, 1)$.