Finite-state source-channel coding for individual sequences with source side information at the decoder

TL;DR

This paper investigates finite-state source-channel coding for individual sequences with noisy side information at the decoder, deriving bounds on distortion and excess-distortion probability, and extending previous models to include additional practical considerations.

Contribution

It introduces a semi-deterministic model with finite-state encoders and decoders, providing bounds on achievable distortion and excess-distortion probability, and extends prior work by incorporating side information and other extensions.

Findings

Derived a lower bound on expected distortion based on empirical source statistics.

Showed the lower bound is asymptotically achievable with universal block codes.

Provided bounds on excess-distortion probability and discussed achievability conditions.

Abstract

We study the following semi-deterministic setting of the joint source-channel coding problem: a deterministic source sequence (a.k.a. individual sequence) is transmitted via a memoryless channel, using delay-limited encoder and decoder, which are both implementable by periodically-varying finite-state machines, and the decoder is granted with access to side information, which is a noisy version of the source sequence. We first derive a lower bound on the achievable expected distortion in terms of the empirical statistics of the source sequence, the number of states of the encoder, the number of states of the decoder, their period, and the overall delay. The bound is shown to be asymptotically achievable by universal block codes in the limit of long blocks. We also derive a lower bound to the best achievable excess--distortion probability and discuss situations where it is achievable. Here, of course, source coding and channel coding cannot be completely separated without loss of optimality. Finally, we outline a few extensions of the model considered, such as: (i) incorporating a common reconstruction constraint, (ii) availability of side information at both ends, and (iii) extension to the Shannon channel with causal state information at the encoder. This work both extends and improves on earlier work of the same flavor (Ziv 1980, Merhav 2014), which focused only on the expected distortion, without side information at either end, and without the above mentioned additional ingredients.