On Finite Block-Length Quantization Distortion

TL;DR

This paper establishes new finite block-length bounds on quantization distortion for i.i.d. sources, providing insights into the gap between theoretical limits and practical quantization performance.

Contribution

It derives the first finite block-length lower bound and two upper bounds on quantization distortion using convex optimization and random codebooks, applicable to binary and Gaussian sources.

Findings

Lower bound exceeds asymptotic distortion

Upper bounds can be achieved with random codebooks

Bounds are tight for reasonable block lengths

Abstract

We investigate the upper and lower bounds on the quantization distortions for independent and identically distributed sources in the finite block-length regime. Based on the convex optimization framework of the rate-distortion theory, we derive a lower bound on the quantization distortion under finite block-length, which is shown to be greater than the asymptotic distortion given by the rate-distortion theory. We also derive two upper bounds on the quantization distortion based on random quantization codebooks, which can achieve any distortion above the asymptotic one. Moreover, we apply the new upper and lower bounds to two types of sources, the discrete binary symmetric source and the continuous Gaussian source. For the binary symmetric source, we obtain the closed-form expressions of the upper and lower bounds. For the Gaussian source, we propose a computational tractable method to numerically compute the upper and lower bounds, for both bounded and unbounded quantization codebooks.Numerical results show that the gap between the upper and lower bounds is small for reasonable block length and hence the bounds are tight.