An Implementable Scheme for Universal Lossy Compression of Discrete Markov Sources

TL;DR

This paper introduces a practical universal lossy compression scheme for discrete Markov sources that achieves optimal rate-distortion performance using a Viterbi-like algorithm and a carefully designed cost function.

Contribution

It proposes a new implementable compression method that guarantees universal optimality for Markov sources by combining empirical probability costs with distortion measures.

Findings

Achieves universal rate-distortion performance for Markov sources.

Employs a Viterbi-like algorithm for efficient reconstruction.

Guarantees optimality as sequence length increases with appropriate parameter tuning.

Abstract

We present a new lossy compressor for discrete sources. For coding a source sequence $x^n$, the encoder starts by assigning a certain cost to each reconstruction sequence. It then finds the reconstruction that minimizes this cost and describes it losslessly to the decoder via a universal lossless compressor. The cost of a sequence is given by a linear combination of its empirical probabilities of some order $k+1$ and its distortion relative to the source sequence. The linear structure of the cost in the empirical count matrix allows the encoder to employ a Viterbi-like algorithm for obtaining the minimizing reconstruction sequence simply. We identify a choice of coefficients for the linear combination in the cost function which ensures that the algorithm universally achieves the optimum rate-distortion performance of any Markov source in the limit of large $n$, provided $k$ is increased as $o(\log n)$.