Universal Compression of Envelope Classes: Tight Characterization via Poisson Sampling

TL;DR

This paper introduces a Poisson-sampling based method to precisely characterize the redundancy of universal compression schemes across various envelope classes, simplifying analysis and improving bounds.

Contribution

It develops a new Poisson-sampling approach to derive tight redundancy formulas for envelope classes, advancing understanding of universal compression performance.

Findings

Derived a simple single-letter redundancy formula for envelope classes.

Provided tight bounds for small alphabet i.i.d. distributions.

Improved bounds for exponential and power-law classes.

Abstract

The Poisson-sampling technique eliminates dependencies among symbol appearances in a random sequence. It has been used to simplify the analysis and strengthen the performance guarantees of randomized algorithms. Applying this method to universal compression, we relate the redundancies of fixed-length and Poisson-sampled sequences, use the relation to derive a simple single-letter formula that approximates the redundancy of any envelope class to within an additive logarithmic term. As a first application, we consider i.i.d. distributions over a small alphabet as a step-envelope class, and provide a short proof that determines the redundancy of discrete distributions over a small al- phabet up to the first order terms. We then show the strength of our method by applying the formula to tighten the existing bounds on the redundancy of exponential and power-law classes, in particular answering a question posed by Boucheron, Garivier and Gassiat.