Improved Source Coding Exponents via Witsenhausen's Rate

TL;DR

This paper introduces a new upper bound on Witsenhausen's rate and a semi-universal error exponent for the Slepian-Wolf problem, surpassing previous bounds and achieving the sphere-packing bound in certain cases.

Contribution

The paper presents a novel information-theoretic functional and an improved encoding scheme that enhances error exponents and bounds in source coding with side information.

Findings

New upper bound on Witsenhausen's rate

Error exponent surpasses previous expurgated bounds

Achieves sphere-packing bound for deterministic side information

Abstract

We provide a novel upper-bound on Witsenhausen's rate, the rate required in the zero-error analogue of the Slepian-Wolf problem; our bound is given in terms of a new information-theoretic functional defined on a certain graph. We then use the functional to give a single letter lower-bound on the error exponent for the Slepian-Wolf problem under the vanishing error probability criterion, where the decoder has full (i.e. unencoded) side information. Our exponent stems from our new encoding scheme which makes use of source distribution only through the positions of the zeros in the `channel' matrix connecting the source with the side information, and in this sense is `semi-universal'. We demonstrate that our error exponent can beat the `expurgated' source-coding exponent of Csisz\'{a}r and K\"{o}rner, achievability of which requires the use of a non-universal maximum-likelihood decoder. An extension of our scheme to the lossy case (i.e. Wyner-Ziv) is given. For the case when the side information is a deterministic function of the source, the exponent of our improved scheme agrees with the sphere-packing bound exactly (thus determining the reliability function). An application of our functional to zero-error channel capacity is also given.