Incremental Refinement using a Gaussian Test Channel

TL;DR

This paper investigates the behavior of the additive rate-distortion function (ARDF) at low resolutions, establishing its relation to the Gaussian RDF, analyzing rate loss, and demonstrating the optimality of incremental refinement in the low-rate limit.

Contribution

It rigorously analyzes ARDF behavior at low resolutions, links it to Gaussian RDF, and proves the optimality of unconditional incremental refinement in this regime.

Findings

ARDF slope near zero rate converges to Gaussian RDF slope.

Burstiness can cause unbounded multiplicative rate loss.

Unconditional incremental refinement is ARDF optimal at low resolution.

Abstract

The additive rate-distortion function (ARDF) was developed in order to universally bound the rate loss in the Wyner-Ziv problem, and has since then been instrumental in e.g., bounding the rate loss in successive refinements, universal quantization, and other multi-terminal source coding settings. The ARDF is defined as the minimum mutual information over an additive test channel followed by estimation. In the limit of high resolution, the ADRF coincides with the true RDF for many sources and fidelity criterions. In the other extreme, i.e., the limit of low resolutions, the behavior of the ARDF has not previously been rigorously addressed. In this work, we consider the special case of quadratic distortion and where the noise in the test channel is Gaussian distributed. We first establish a link to the I-MMSE relation of Guo et al. and use this to show that for any source the slope of the ARDF near zero rate, converges to the slope of the Gaussian RDF near zero rate. We then consider the multiplicative rate loss of the ARDF, and show that for bursty sources it may be unbounded, contrary to the additive rate loss, which is upper bounded by 1/2 bit for all sources. We finally show that unconditional incremental refinement, i.e., where each refinement is encoded independently of the other refinements, is ARDF optimal in the limit of low resolution, independently of the source distribution. Our results also reveal under which conditions linear estimation is ARDF optimal in the low rate regime.