Lossy source encoding via message-passing and decimation over generalized codewords of LDGM codes

TL;DR

This paper introduces message-passing and decimation algorithms for lossy source coding using LDGM codes, achieving near-optimal performance for Bernoulli sources by leveraging survey propagation-inspired techniques.

Contribution

It develops a novel message-passing and decimation framework based on generalized codewords and Markov random fields, inspired by survey propagation, to improve lossy source encoding with LDGM codes.

Findings

Performance close to rate distortion limit over various rates

Effective message-passing and decimation algorithms for LDGM codes

Inspired by survey propagation and belief propagation techniques

Abstract

We describe message-passing and decimation approaches for lossy source coding using low-density generator matrix (LDGM) codes. In particular, this paper addresses the problem of encoding a Bernoulli(0.5) source: for randomly generated LDGM codes with suitably irregular degree distributions, our methods yield performance very close to the rate distortion limit over a range of rates. Our approach is inspired by the survey propagation (SP) algorithm, originally developed by Mezard et al. for solving random satisfiability problems. Previous work by Maneva et al. shows how SP can be understood as belief propagation (BP) for an alternative representation of satisfiability problems. In analogy to this connection, our approach is to define a family of Markov random fields over generalized codewords, from which local message-passing rules can be derived in the standard way. The overall source encoding method is based on message-passing, setting a subset of bits to their preferred values (decimation), and reducing the code.