An MCMC Approach to Universal Lossy Compression of Analog Sources

TL;DR

This paper introduces an MCMC-based lossy compression method for analog sources that asymptotically achieves Shannon's optimal rate-distortion tradeoff, with algorithms adaptable to source complexity and alphabet size.

Contribution

It extends MCMC-based compression techniques to analog sources with growing reproduction alphabets, achieving universal optimality and improved efficiency.

Findings

Achieves Shannon's rate-distortion limit asymptotically for stationary ergodic sources.

Provides algorithms with reduced complexity for sources with small optimal alphabets.

Demonstrates universality and adaptability of the proposed MCMC-based compression methods.

Abstract

Motivated by the Markov chain Monte Carlo (MCMC) approach to the compression of discrete sources developed by Jalali and Weissman, we propose a lossy compression algorithm for analog sources that relies on a finite reproduction alphabet, which grows with the input length. The algorithm achieves, in an appropriate asymptotic sense, the optimum Shannon theoretic tradeoff between rate and distortion, universally for stationary ergodic continuous amplitude sources. We further propose an MCMC-based algorithm that resorts to a reduced reproduction alphabet when such reduction does not prevent achieving the Shannon limit. The latter algorithm is advantageous due to its reduced complexity and improved rates of convergence when employed on sources with a finite and small optimum reproduction alphabet.