Predictive Information Rate in Discrete-time Gaussian Processes

TL;DR

This paper derives formulas for the predictive information rate in Gaussian processes, revealing a duality with the multi-information rate and exploring its behavior in AR and MA processes.

Contribution

It provides new expressions for PIR in AR(N) processes and uncovers a duality with the multi-information rate, including properties of MA processes.

Findings

PIR is maximized in smooth AR processes with multiple poles at zero frequency.

PIR and multi-information rate are dual for processes with inverse power spectra.

PIR is unbounded for MA(N) processes.

Abstract

We derive expressions for the predicitive information rate (PIR) for the class of autoregressive Gaussian processes AR(N), both in terms of the prediction coefficients and in terms of the power spectral density. The latter result suggests a duality between the PIR and the multi-information rate for processes with mutually inverse power spectra (i.e. with poles and zeros of the transfer function exchanged). We investigate the behaviour of the PIR in relation to the multi-information rate for some simple examples, which suggest, somewhat counter-intuitively, that the PIR is maximised for very `smooth' AR processes whose power spectra have multiple poles at zero frequency. We also obtain results for moving average Gaussian processes which are consistent with the duality conjectured earlier. One consequence of this is that the PIR is unbounded for MA(N) processes.