Lossy Compression of General Random Variables

TL;DR

This paper develops general bounds for lossy compression of complex random variables in various spaces, including manifolds and fractals, with applications in data science and image processing.

Contribution

It provides broad rate-distortion bounds for random variables in measurable spaces, applicable to manifolds and fractal sets, based on small ball probability conditions.

Findings

Derived bounds on rate-distortion functions for manifold-valued variables.

Extended results to fractal sets with iterated function systems.

Demonstrated applicability to data science and image compression scenarios.

Abstract

This paper is concerned with the lossy compression of general random variables, specifically with rate-distortion theory and quantization of random variables taking values in general measurable spaces such as, e.g., manifolds and fractal sets. Manifold structures are prevalent in data science, e.g., in compressed sensing, machine learning, image processing, and handwritten digit recognition. Fractal sets find application in image compression and in the modeling of Ethernet traffic. Our main contributions are bounds on the rate-distortion function and the quantization error. These bounds are very general and essentially only require the existence of reference measures satisfying certain regularity conditions in terms of small ball probabilities. To illustrate the wide applicability of our results, we particularize them to random variables taking values in i) manifolds, namely, hyperspheres and Grassmannians, and ii) self-similar sets characterized by iterated function systems satisfying the weak separation property.