The Performance of Compression-Based Denoisers

TL;DR

This paper extends compression-based denoising methods to general discrete memoryless channels by matching the distortion measure to the channel, providing exact loss characterizations and practical implications for various loss functions.

Contribution

It introduces a novel framework for denoising over general channels by aligning the distortion measure with the channel distribution, expanding beyond additive noise models.

Findings

Exact loss characterization for the proposed denoiser.

Demonstration of results for MSE and Hamming loss.

Comparison with rate-distortion approaches.

Abstract

We consider a denoiser that reconstructs a stationary ergodic source by lossily compressing samples of the source observed through a memoryless noisy channel. Prior work on compression-based denoising has been limited to additive noise channels. We extend this framework to general discrete memoryless channels by deliberately choosing the distortion measure for the lossy compressor to match the channel conditional distribution. By bounding the deviation of the empirical joint distribution of the source, observation, and denoiser outputs from satisfying a Markov property, we give an exact characterization of the loss achieved by such a denoiser. Consequences of these results are explicitly demonstrated in special cases, including for MSE and Hamming loss. A comparison is made to an indirect rate-distortion perspective on the problem.