Converse Bounds for Entropy-Constrained Quantization Via a Variational Entropy Inequality

TL;DR

This paper establishes a new lower bound on the minimal output entropy achievable by vector quantization of sources with finite differential entropy, showing uniform quantizers are asymptotically optimal as distortion vanishes.

Contribution

It introduces a variational entropy inequality-based lower bound applicable to all finite-entropy sources, generalizing previous results without requiring density decay conditions.

Findings

Lower bound converges to uniform quantizer entropy at zero distortion.

Asymptotically optimal quantizers must approach uniform quantization.

The bound applies to all finite-entropy, memoryless sources.

Abstract

We derive a lower bound on the smallest output entropy that can be achieved via vector quantization of a $d$-dimensional source with given expected $r$th-power distortion. Specialized to the one-dimensional case, and in the limit of vanishing distortion, this lower bound converges to the output entropy achieved by a uniform quantizer, thereby recovering the result by Gish and Pierce that uniform quantizers are asymptotically optimal as the allowed distortion tends to zero. Our lower bound holds for all $d$-dimensional memoryless sources having finite differential entropy and whose integer part has finite entropy. In contrast to Gish and Pierce, we do not require any additional constraints on the continuity or decay of the source probability density function. For one-dimensional sources, the derivation of the lower bound reveals a necessary condition for a sequence of quantizers to be asymptotically optimal as the allowed distortion tends to zero. This condition implies that any sequence of asymptotically-optimal almost-regular quantizers must converge to a uniform quantizer as the allowed distortion tends to zero.