Thouless-Anderson-Palmer Approach for Lossy Compression

TL;DR

This paper introduces an iterative algorithm based on the Thouless-Anderson-Palmer approach for lossy compression of binary sequences using sparse matrices, achieving near-optimal rate-distortion performance.

Contribution

It develops a novel iterative method inspired by statistical physics to approach the NP-complete encoding problem in lossy compression.

Findings

Algorithm empirically saturates theoretical rate-distortion limit

Close to optimal compression performance achieved

Effective for sparse matrix-based coding schemes

Abstract

We study an ill-posed linear inverse problem, where a binary sequence will be reproduced using a sparce matrix. According to the previous study, this model can theoretically provide an optimal compression scheme for an arbitrary distortion level, though the encoding procedure remains an NP-complete problem. In this paper, we focus on the consistency condition for a dynamics model of Markov-type to derive an iterative algorithm, following the steps of Thouless-Anderson-Palmer's. Numerical results show that the algorithm can empirically saturate the theoretical limit for the sparse construction of our codes, which also is very close to the rate-distortion function.