Efficient sphere-covering and converse measure concentration via generalized coding theorems

TL;DR

This paper introduces a probabilistic framework for optimal sphere-covering in finite and abstract spaces, unifying several classical information theory results and providing new converse bounds for measure concentration.

Contribution

It offers a generalized coding theorem approach to characterize efficient coverings and measure concentration, extending classical results to broader settings.

Findings

Unified probabilistic characterization of sphere coverings

Derivation of converse bounds for measure concentration inequalities

Generalization to abstract spaces and non-product measures

Abstract

Suppose A is a finite set equipped with a probability measure P and let M be a ``mass'' function on A. We give a probabilistic characterization of the most efficient way in which A^n can be almost-covered using spheres of a fixed radius. An almost-covering is a subset C_n of A^n, such that the union of the spheres centered at the points of C_n has probability close to one with respect to the product measure P^n. An efficient covering is one with small mass M^n(C_n); n is typically large. With different choices for M and the geometry on A our results give various corollaries as special cases, including Shannon's data compression theorem, a version of Stein's lemma (in hypothesis testing), and a new converse to some measure concentration inequalities on discrete spaces. Under mild conditions, we generalize our results to abstract spaces and non-product measures.