A note on rate-distortion functions for nonstationary Gaussian autoregressive processes

TL;DR

This paper clarifies the consistency of rate-distortion formulas for nonstationary Gaussian autoregressive processes, resolving apparent contradictions and highlighting differences from stationary cases with concrete examples.

Contribution

It demonstrates the equivalence of existing formulas for nonstationary Gaussian AR processes and clarifies misconceptions about their validity.

Findings

Gray and Hashimoto-Arimoto formulas are consistent and equivalent.

Hashimoto-Arimoto example does not contradict Gray’s results.

Classic Kolmogorov formula differs from autoregressive formula in nonstationary cases.

Abstract

Source coding theorems and Shannon rate-distortion functions were studied for the discrete-time Wiener process by Berger and generalized to nonstationary Gaussian autoregressive processes by Gray and by Hashimoto and Arimoto. Hashimoto and Arimoto provided an example apparently contradicting the methods used in Gray, implied that Gray's rate-distortion evaluation was not correct in the nonstationary case, and derived a new formula that agreed with previous results for the stationary case and held in the nonstationary case. In this correspondence it is shown that the rate-distortion formulas of Gray and Hashimoto and Arimoto are in fact consistent and that the example of of Hashimoto and Arimoto does not form a counter example to the methods or results of the earlier paper. Their results do provide an alternative, but equivalent, formula for the rate-distortion function in the nonstationary case and they provide a concrete example that the classic Kolmogorov formula differs from the autoregressive formula when the autoregressive source is not stationary. Some observations are offered on the different versions of the Toeplitz asymptotic eigenvalue distribution theorem used in the two papers to emphasize how a slight modification of the classic theorem avoids the problems with certain singularities.