A Distributionally Robust Approach to Shannon Limits using the Wasserstein Distance

TL;DR

This paper explores the worst-case rate-distortion and channel capacity under distributional uncertainty modeled by Wasserstein-2 ambiguity sets, providing convex formulations and insights into Shannon limits.

Contribution

It introduces a distributionally robust framework for Shannon limits using Wasserstein distance, deriving convex LMI-based formulations for Gaussian nominal distributions.

Findings

Worst-case distributions remain Gaussian when nominal is Gaussian.

Convex LMI formulations enable tractable analysis of robust Shannon limits.

Closed-form expressions in scalar case reveal the impact of ambiguity set size.

Abstract

We consider the rate-distortion function for lossy source compression, as well as the channel capacity for error correction, through the lens of distributional robustness. We assume that the distribution of the source or of the additive channel noise is unknown and lies within a Wasserstein-2 ambiguity set of a given radius centered around a specified nominal distribution, and we look for the worst-case asymptotically optimal coding rate over such an ambiguity set. Varying the radius of the ambiguity set allows us to interpolate between the worst-case and stochastic scenarios using probabilistic tools. Our problem setting fits into the paradigm of compound source / channel models introduced by Sakrison and Blackwell, respectively. This paper shows that if the nominal distribution is Gaussian, then so is the worst-case source / noise distribution, and the compound rate-distortion / channel capacity functions admit convex formulations with Linear Matrix Inequality (LMI) constraints. These formulations yield simple closed-form expressions in the scalar case, offering insights into the behavior of Shannon limits with the changing radius of the Wasserstein-2 ambiguity set.