Rateless Lossy Compression via the Extremes

TL;DR

This paper introduces a novel rateless lossy compression scheme that achieves the rate-distortion function for Gaussian and ergodic sources, featuring near-zero rate encoding and iterative refinement.

Contribution

It proposes a simple, near-zero rate lossy compressor based on extreme components, and an iterative scheme that attains the rate-distortion function for a broad class of sources.

Findings

Achieves zero-rate slope for Gaussian sources.

Attains the rate-distortion function for ergodic sources.

Has low storage and computation complexity.

Abstract

We begin by presenting a simple lossy compressor operating at near-zero rate: The encoder merely describes the indices of the few maximal source components, while the decoder's reconstruction is a natural estimate of the source components based on this information. This scheme turns out to be near-optimal for the memoryless Gaussian source in the sense of achieving the zero-rate slope of its distortion-rate function. Motivated by this finding, we then propose a scheme comprised of iterating the above lossy compressor on an appropriately transformed version of the difference between the source and its reconstruction from the previous iteration. The proposed scheme achieves the rate distortion function of the Gaussian memoryless source (under squared error distortion) when employed on any finite-variance ergodic source. It further possesses desirable properties we respectively refer to as infinitesimal successive refinability, ratelessness, and complete separability. Its storage and computation requirements are of order no more than $\frac{n^2}{\log^{\beta} n}$ per source symbol for $\beta>0$ at both the encoder and decoder. Though the details of its derivation, construction, and analysis differ considerably, we discuss similarities between the proposed scheme and the recently introduced Sparse Regression Codes (SPARC) of Venkataramanan et al.