Data compression with low distortion and finite blocklength

TL;DR

This paper develops explicit finite-blocklength bounds for lossy source coding, showing lattice quantizers can nearly achieve the fundamental limits in low-distortion, finite-dimensional settings.

Contribution

It introduces a simplified approximation to the rate-distortion function at finite blocklengths, linking the Shannon lower bound to practical lattice quantization methods.

Findings

Lattice quantizers approach the nonasymptotic Shannon lower bound under certain conditions.

Finite alphabet sources with balanced distortion measures meet the bound at low distortions.

The new entropy bound for lattice quantizers avoids random coding and dithering assumptions.

Abstract

This paper considers lossy source coding of $n$-dimensional memoryless sources and shows an explicit approximation to the minimum source coding rate required to sustain the probability of exceeding distortion $d$ no greater than $\epsilon$, which is simpler than known dispersion-based approximations. Our approach takes inspiration in the celebrated classical result stating that the Shannon lower bound to rate-distortion function becomes tight in the limit $d \to 0$. We formulate an abstract version of the Shannon lower bound that recovers both the classical Shannon lower bound and the rate-distortion function itself as special cases. Likewise, we show that a nonasymptotic version of the abstract Shannon lower bound recovers all previously known nonasymptotic converses. A necessary and sufficient condition for the Shannon lower bound to be attained exactly is presented. It is demonstrated that whenever that condition is met, the rate-dispersion function is given simply by the varentropy of the source. Remarkably, all finite alphabet sources with balanced distortion measures satisfy that condition in the range of low distortions. Most continuous sources violate that condition. Still, we show that lattice quantizers closely approach the nonasymptotic Shannon lower bound, provided that the source density is smooth enough and the distortion is low. This implies that fine multidimensional lattice coverings are nearly optimal in the rate-distortion sense even at finite $n$. The achievability proof technique is based on a new bound on the output entropy of lattice quantizers in terms of the differential entropy of the source, the lattice cell size and a smoothness parameter of the source density. The technique avoids both the usual random coding argument and the simplifying assumption of the presence of a dither signal.