Minimax Rate-Distortion

TL;DR

This paper introduces universal variable-rate rate-distortion codes that are minimax optimal and meet distortion constraints almost surely, with a focus on the trade-offs between universality, regularity conditions, and redundancy rates.

Contribution

It establishes the existence of strongly universal codes with optimal redundancy rates and analyzes the impact of regularity conditions on redundancy in universal lossy compression.

Findings

Achievable redundancy rate of (1/) for minimax universality.

Regularity conditions influence the redundancy, with stricter conditions leading to higher redundancy.

Construction uses non-i.i.d. codewords, large deviations, and VC dimension analysis.

Abstract

We show the existence of variable-rate rate-distortion codes that meet the disortion constraint almost surely and are minimax, i.e., strongly, universal with respect to an unknown source distribution and a distortion measure that is revealed only to the encoder and only at runtime. If we only require minimax universality with respect to the source distribution and not the distortion measure, then we provide an achievable $\tilde{O}(1/\sqrt{n})$ redundancy rate, which we show is optimal. This is in contrast to prior work on universal lossy compression, which provides $O(\log n/n)$ redundancy guarantees for weakly universal codes under various regularity conditions. We show that either eliminating the regularity conditions or upgrading to strong universality while keeping these regularity conditions entails an inevitable increase in the redundancy to $\tilde{O}(1/\sqrt{n})$. Our construction involves random coding with non-i.i.d.\ codewords and a zero-rate uncoded transmission scheme. The proof uses exact asymptotics from large deviations, acceptance-rejection sampling, and the VC dimension of distortion measures.