Efficient On-line Schemes for Encoding Individual Sequences with Side Information at the Decoder

TL;DR

This paper introduces adaptive online encoding schemes for individual sequences with side information at the decoder, achieving near-optimal performance with efficient algorithms for fixed and variable-rate coding, including lossless compression.

Contribution

It presents a unified framework for online lossy and lossless coding with side information, featuring algorithms with linear complexity and performance guarantees.

Findings

Achieves asymptotic optimality with high probability

Provides efficient algorithms for small and structured encoder sets

Extends to variable-rate and lossless coding scenarios

Abstract

We present adaptive on-line schemes for lossy encoding of individual sequences under the conditions of the Wyner-Ziv (WZ) problem. In the first part of this article, a set of fixed-rate scalar source codes with zero delay is presented. We propose a randomized on-line coding scheme, which achieves asymptotically (and with high probability), the performance of the best source code in the set, uniformly over all source sequences. The scheme uses the same rate and has zero delay. We then present an efficient algorithm for implementing our on-line coding scheme in the case of a relatively small set of encoders. We also present an efficient algorithm for the case of a larger set of encoders with a structure, using the method of the weighted graph and the Weight Pushing Algorithm (WPA). In the second part of this article, we extend our results to the case of variable-rate coding. A set of variable-rate scalar source codes is presented. We generalize the randomized on-line coding scheme, to our case. This time, the performance is measured by the Lagrangian Cost (LC), which is defined as a weighted sum of the distortion and the length of the encoded sequence. We present an efficient algorithm for implementing our on-line variable-rate coding scheme in the case of a relatively small set of encoders. We then consider the special case of lossless variable-rate coding. An on-line scheme which use Huffman codes is presented. We show that this scheme can be implemented efficiently using the same graphic methods from the first part. Combining the results from former sections, we build a generalized efficient algorithm for structured set of variable-rate encoders. The complexity of all the algorithms is no more than linear in the sequence length.