The Redundancy of Non-Singular Channel Simulation

TL;DR

This paper completes the fundamental understanding of the limitations of channel simulation by establishing lower bounds on redundancy, especially for non-singular channels, with implications for data compression.

Contribution

It extends previous results by proving that the redundancy of channel simulation is lower bounded by the channel simulation divergence and by establishing a universal lower bound of 1/2 for iid non-singular channels.

Findings

Redundancy is lower bounded by the channel simulation divergence.

Asymptotic redundancy for iid non-singular channels is at least 1/2.

Two different proof techniques confirm the lower bound.

Abstract

Channel simulation is an alternative to quantization and entropy coding for performing lossy source coding. Recently, channel simulation has gained significant traction in both the machine learning and information theory communities, as it integrates better with machine learning-based data compression algorithms and has better rate-distortion-perception properties than quantization. As the practical importance of channel simulation increases, it is vital to understand its fundamental limitations. Recently, Sriramu and Wagner provided an almost complete characterisation of the redundancy of channel simulation algorithms. In this paper, we complete this characterisation. First, we significantly extend a result of Li and El Gamal, and show that the redundancy of any instance of a channel simulation problem is lower bounded by the channel simulation divergence. Second, we give two proofs that the asymptotic redundancy of simulating iid non-singular channels is lower-bounded by $1/2$: one using a direct approach based on the asymptotic expansion of the channel simulation divergence and one using large deviations theory.