On Distributionally Robust Lossy Source Coding

TL;DR

This paper develops distributionally robust source coding methods under source distribution uncertainty, extending theoretical tools and providing algorithms and benchmarks for robust rate-distortion analysis.

Contribution

It introduces two extensions of the Strong Functional Representation Lemma for distributional uncertainty, enabling robust coding schemes and algorithms for the rate-distortion function.

Findings

Derived robust coding schemes for one-shot and asymptotic regimes.

Proposed a novel algorithm to compute the robust rate-distortion function.

Provided an analytical expression for Bernoulli sources as a benchmark.

Abstract

In this paper, we investigate the problem of distributionally robust source coding, i.e., source coding under uncertainty in the source distribution, discussing both the coding and computational aspects of the problem. We propose two extensions of the so-called Strong Functional Representation Lemma (SFRL), considering the cases where, for a fixed conditional distribution, the marginal inducing the joint coupling belongs to either a finite set of distributions or a Kullback-Leibler divergence sphere (KL-Sphere) centered at a fixed nominal distribution. Using these extensions, we derive distributionally robust coding schemes for both the one-shot and asymptotic regimes, generalizing previous results in the literature. Focusing on the case where the source distribution belongs to a given KL-Sphere, we derive an implicit characterization of the points attaining the robust rate-distortion function (R-RDF), which we later exploit to implement a novel algorithm for computing the R-RDF. Finally, we characterize the analytical expression of the R-RDF for Bernoulli sources, providing a theoretical benchmark to evaluate the estimation performance of the proposed algorithm.