Shannon Bounds for Quadratic Rate-Distortion Problems

TL;DR

This paper reviews Shannon bounds for rate-distortion problems with mean-squared error, emphasizing Berger's techniques, and introduces the Gray-Wyner network as a new setting where such bounds apply, connecting classical and recent developments.

Contribution

It extends Shannon bounds to the Gray-Wyner network, adding a new setting for these bounds and relating them to recent advances in source coding.

Findings

Shannon bounds expressed via source entropy power and variance.

Bounds coincide for Gaussian sources, providing tight characterizations.

Extension of bounds to the Gray-Wyner network setting.

Abstract

The Shannon lower bound has been the subject of several important contributions by Berger. This paper surveys Shannon bounds on rate-distortion problems under mean-squared error distortion with a particular emphasis on Berger's techniques. Moreover, as a new result, the Gray-Wyner network is added to the canon of settings for which such bounds are known. In the Shannon bounding technique, elegant lower bounds are expressed in terms of the source entropy power. Moreover, there is often a complementary upper bound that involves the source variance in such a way that the bounds coincide in the special case of Gaussian statistics. Such pairs of bounds are sometimes referred to as Shannon bounds. The present paper puts Berger's work on many aspects of this problem in the context of more recent developments, encompassing indirect and remote source coding such as the CEO problem, originally proposed by Berger, as well as the Gray-Wyner network as a new contribution.