Wyner's Common Information under R\'enyi Divergence Measures

TL;DR

This paper extends Wyner's common information problem by analyzing it under R'enyi divergence measures, showing the robustness of Wyner's common information across different distance measures and establishing an exponential strong converse under total variation.

Contribution

It generalizes Wyner's common information to R'enyi divergence measures and proves its invariance except at a specific divergence order, also establishing a strong converse result.

Findings

Wyner's common information remains unchanged under most R'enyi divergences.

The robustness of Wyner's common information is confirmed across various divergence measures.

An exponential strong converse is established under total variation distance.

Abstract

We study a generalized version of Wyner's common information problem (also coined the distributed source simulation problem). The original common information problem consists in understanding the minimum rate of the common input to independent processors to generate an approximation of a joint distribution when the distance measure used to quantify the discrepancy between the synthesized and target distributions is the normalized relative entropy. Our generalization involves changing the distance measure to the unnormalized and normalized R\'enyi divergences of order $\alpha=1+s\in[0,2]$. We show that the minimum rate needed to ensure the R\'enyi divergences between the distribution induced by a code and the target distribution vanishes remains the same as the one in Wyner's setting, except when the order $\alpha=1+s=0$. This implies that Wyner's common information is rather robust to the choice of distance measure employed. As a byproduct of the proofs used to the establish the above results, the exponential strong converse for the common information problem under the total variation distance measure is established.