The Shannon Lower Bound is Asymptotically Tight

TL;DR

This paper proves that the Shannon lower bound becomes asymptotically tight for sources with finite differential entropy as distortion approaches zero, but not for sources with infinite integer-part entropy.

Contribution

It establishes the conditions under which the Shannon lower bound is asymptotically tight, specifically relating to the entropy of the source's integer part.

Findings

Shannon lower bound gap vanishes as distortion tends to zero for sources with finite differential entropy.

If the integer part of the source has infinite entropy, the rate-distortion function is infinite for all finite distortions.

The Shannon lower bound is asymptotically tight if and only if the source's integer part has finite entropy.

Abstract

The Shannon lower bound is one of the few lower bounds on the rate-distortion function that holds for a large class of sources. In this paper, it is demonstrated that its gap to the rate-distortion function vanishes as the allowed distortion tends to zero for all sources having a finite differential entropy and whose integer part is finite. Conversely, it is demonstrated that if the integer part of the source has an infinite entropy, then its rate-distortion function is infinite for every finite distortion. Consequently, the Shannon lower bound provides an asymptotically tight bound on the rate-distortion function if, and only if, the integer part of the source has a finite entropy.