Stationarity in the Realizations of the Causal Rate-Distortion Function for One-Sided Stationary Sources

TL;DR

This paper investigates the causal rate-distortion function for one-sided stationary sources, revealing limitations of stationarity in realizations and providing new bounds and characterizations for practical encoding schemes.

Contribution

It extends the understanding of the causal IRDF for one-sided stationary sources, showing when stationarity can be assumed and improving the definition for two-sided sources.

Findings

Causal IRDF cannot generally be realized by a stationary distribution.

For large classes of distortion criteria, the search can be restricted to jointly stationary output sequences.

Practical zero-delay encoders approach the IRDF within approximately 1.254 bits per sample.

Abstract

This paper derives novel results on the characterization of the the causal information rate-distortion function (IRDF) $R_{c}^{it}(D)$ for arbitrarily-distributed one-sided stationary $\kappa$-th order Markov source x(1),x(2),.... It is first shown that Gorbunov and Pinsker's results on the stationarity of the realizations to the causal IRDF (stated for two-sided stationary sources) do not apply to the commonly used family of asymptotic average single-letter (AASL) distortion criteria. Moreover, we show that, in general, a reconstruction sequence cannot be both jointly stationary with a one-sided stationary source sequence and causally related to it. This implies that, in general, the causal IRDF for one-sided stationary sources cannot be realized by a stationary distribution. However, we prove that for an arbitrarily distributed one-sided stationary source and a large class of distortion criteria (including AASL), the search for $R_{c}^{it}(D)$ can be restricted to distributions which yield the output sequence y(1), y(2),... jointly stationary with the source after $\kappa$ samples. Finally, we improve the definition of the stationary causal IRDF $\overline{R}_{c}^{it}(D)$ previously introduced by Derpich and {\O}stergaard for two-sided Markovian stationary sources and show that $\overline{R}_{c}^{it}(D)$ for a two-sided source ...,x(-1),x(0),x(1),... equals $R_{c}^{it}(D)$ for the associated one-sided source x(1), x(2),.... This implies that, for the Gaussian quadratic case, the practical zero-delay encoder-decoder pairs proposed by Derpich and {\O}stergaard for approaching $R_{c}^{it}(D)$ achieve an operational data rate which exceeds $R_{c}^{it}(D)$ by less than $1+0.5 \log_2(2 \pi e /12) \simeq 1.254$ bits per sample.