Incremental Refinements and Multiple Descriptions with Feedback

TL;DR

This paper demonstrates that with multiple rounds of independent encoding and feedback, the total rate approaches the optimal rate-distortion limit, reducing rate loss in multiple descriptions scenarios.

Contribution

It introduces a multi-round encoding framework showing that incremental refinements with feedback can asymptotically achieve the rate-distortion function for correlated sources.

Findings

Rate ratio approaches one at high distortion levels.

Excess rate vanishes as the number of rounds increases.

Experimental evidence supports the generality of the phenomenon.

Abstract

It is well known that independent (separate) encoding of K correlated sources may incur some rate loss compared to joint encoding, even if the decoding is done jointly. This loss is particularly evident in the multiple descriptions problem, where the sources are repetitions of the same source, but each description must be individually good. We observe that under mild conditions about the source and distortion measure, the rate ratio Rindependent(K)/Rjoint goes to one in the limit of small rate/high distortion. Moreover, we consider the excess rate with respect to the rate-distortion function, Rindependent(K, M) - R(D), in M rounds of K independent encodings with a final distortion level D. We provide two examples - a Gaussian source with mean-squared error and an exponential source with one-sided error - for which the excess rate vanishes in the limit as the number of rounds M goes to infinity, for any fixed D and K. This result has an interesting interpretation for a multi-round variant of the multiple descriptions problem, where after each round the encoder gets a (block) feedback regarding which of the descriptions arrived: In the limit as the number of rounds M goes to infinity (i.e., many incremental rounds), the total rate of received descriptions approaches the rate-distortion function. We provide theoretical and experimental evidence showing that this phenomenon is in fact more general than in the two examples above.