Single-shot lossy compression: mutual information bounds

TL;DR

This paper establishes mutual information bounds on the minimum expected description length for various fidelity constraints, demonstrating that mutual information serves as an effective proxy for data compression limits.

Contribution

It introduces mutual information upper bounds for lossy compression under different fidelity constraints and provides alternative characterizations of these proxies.

Findings

Mutual information bounds closely approximate the minimum description length.

The results apply to multiple fidelity constraint styles.

Provides insights into the structure of optimal compression solutions.

Abstract

For several styles of fidelity constraints -- guaranteed distortion, conditional excess distortion, excess distortion -- we show mutual information upper bounds on the minimum expected description length needed to represent a random variable. Coupled with the corresponding converses, these results attest that as long as the information content in the data is not too low, minimizing the mutual information under an appropriate fidelity constraint serves as a reasonable proxy for the minimum description length of the data. We provide alternative characterizations of all three convex proxies, shedding light on the structure of their solutions.