An Information-Spectrum Approach to Weak Variable-Length Source Coding with Side-Information

TL;DR

This paper develops a new information-spectrum framework for variable-length source coding with side-information, providing general formulas for achievable rates and analyzing the impact of encoder side-information on coding efficiency.

Contribution

It introduces novel one-shot coding theorems for sources with side-information and derives a general asymptotic rate formula for weak VL-SW coding, extending existing results.

Findings

Derived a general formula for the infimum of asymptotic coding rates.

Extended results to mixed sources with countably infinite alphabets.

Showed encoder side-information usefulness is null if it is useless in weak VL coding.

Abstract

This paper studies variable-length (VL) source coding of general sources with side-information. Novel one-shot coding theorems for coding with common side-information available at the encoder and the decoder and Slepian- Wolf (SW) coding (i.e., with side-information only at the decoder) are given, and then, are applied to asymptotic analyses of these coding problems. Especially, a general formula for the infimum of the coding rate asymptotically achievable by weak VL-SW coding (i.e., VL-SW coding with vanishing error probability) is derived. Further, the general formula is applied to investigating weak VL-SW coding of mixed sources. Our results derive and extend several known results on SW coding and weak VL coding, e.g., the optimal achievable rate of VL-SW coding for mixture of i.i.d. sources is given for countably infinite alphabet case with mild condition. In addition, the usefulness of the encoder side-information is investigated. Our result shows that if the encoder side-information is useless in weak VL coding then it is also useless even in the case where the error probability may be positive asymptotically.