Cross-Domain Lossy Compression via Constrained Minimum Entropy Coupling

TL;DR

This paper introduces a new approach to cross-domain lossy compression using minimum entropy coupling, focusing on preserving classification-relevant information and demonstrating effectiveness on image datasets.

Contribution

It formulates a rate-constrained MEC problem with a deterministic equivalent and develops a neural framework for image restoration under this paradigm.

Findings

Increasing rate improves classification accuracy.

The neural framework effectively reconstructs images with preserved classification info.

Closed-form solutions are derived for Bernoulli sources.

Abstract

This paper studies cross-domain lossy compression through the lens of minimum entropy coupling (MEC) with rate and classification constraints. In this setting, an encoder observes samples from a degraded source domain, while the decoder is required to generate outputs following a prescribed target distribution and to preserve information relevant to a downstream classification task. Motivated by logarithmic-loss distortion, we adopt an information-based objective that maximizes the coupling strength between the source and reconstruction, rather than minimizing a sample-wise distortion. Under common randomness, we formulate a rate-constrained MEC problem (MEC-B) and show that the intermediate representation can be removed without loss of optimality, yielding an equivalent deterministic coupling formulation. For Bernoulli sources, closed-form expressions are derived with and without classification constraints. In addition, we implement a neural restoration framework using quantization, entropy modeling, distribution matching, and classification regularization. Experiments on MNIST super-resolution and SVHN denoising show that increasing the available rate improves classification accuracy and yields more informative reconstructions.