Sampling Rate Distortion

TL;DR

This paper introduces a new sampling rate distortion function for multi-source data compression, analyzing fixed-set and random sampling methods, and demonstrates conditions where informed decoding improves performance.

Contribution

It defines and characterizes a novel sampling rate distortion function for multi-source compression, including fixed-set and random sampling strategies, with insights on decoder knowledge impact.

Findings

Sampling rate distortion function is the same for independent random sampling regardless of decoder knowledge.

Deterministic sampling is optimal for memoryless random sampling with an informed decoder.

Memoryless sampling with an informed decoder can outperform other sampling methods.

Abstract

Consider a discrete memoryless multiple source with $m$ components of which $k \leq m$ possibly different sources are sampled at each time instant and jointly compressed in order to reconstruct all the $m$ sources under a given distortion criterion. A new notion of sampling rate distortion function is introduced, and is characterized first for the case of fixed-set sampling. Next, for independent random sampling performed without knowledge of the source outputs, it is shown that the sampling rate distortion function is the same regardless of whether or not the decoder is informed of the sequence of sampled sets. Furthermore, memoryless random sampling is considered with the sampler depending on the source outputs and with an informed decoder. It is shown that deterministic sampling, characterized by a conditional point-mass, is optimal and suffices to achieve the sampling rate distortion function. For memoryless random sampling with an uninformed decoder, an upper bound for the sampling rate distortion function is seen to possess a similar property of conditional point-mass optimality. It is shown by example that memoryless sampling with an informed decoder can outperform strictly any independent random sampler, and that memoryless sampling can do strictly better with an informed decoder than without.