Statistical Mechanical Approach to Lossy Data Compression:Theory and Practice

TL;DR

This paper presents a statistical mechanical framework for lossy data compression using a nonmonotonic perceptron, demonstrating near-optimal performance with a belief propagation algorithm despite computational challenges.

Contribution

It introduces a perceptron-based coding scheme analyzed via statistical mechanics and proposes a belief propagation algorithm for practical, near-limit compression performance.

Findings

Perceptron-based codes approach theoretical compression limits.

Belief propagation algorithm achieves near-optimal performance.

The method balances compression quality with computational feasibility.

Abstract

The encoder and decoder for lossy data compression of binary memoryless sources are developed on the basis of a specific-type nonmonotonic perceptron. Statistical mechanical analysis indicates that the potential ability of the perceptron-based code saturates the theoretically achievable limit in most cases although exactly performing the compression is computationally difficult. To resolve this difficulty, we provide a computationally tractable approximation algorithm using belief propagation (BP), which is a current standard algorithm of probabilistic inference. Introducing several approximations and heuristics, the BP-based algorithm exhibits performance that is close to the achievable limit in a practical time scale in optimal cases.