Analysis of LDGM and compound codes for lossy compression and binning

TL;DR

This paper rigorously analyzes LDGM and compound LDGM/LDPC codes for lossy compression, demonstrating their effectiveness in approaching the rate-distortion bound and their suitability for message-passing encoding.

Contribution

It provides theoretical bounds on LDGM codes' performance and introduces a compound construction that saturates the rate-distortion bound with finite degrees.

Findings

LDGM codes approach the rate-distortion function as degree increases

Compound LDPC/LDGM codes saturate the rate-distortion bound with finite degrees

Sparse, high-girth graphs are suitable for message-passing encoding

Abstract

Recent work has suggested that low-density generator matrix (LDGM) codes are likely to be effective for lossy source coding problems. We derive rigorous upper bounds on the effective rate-distortion function of LDGM codes for the binary symmetric source, showing that they quickly approach the rate-distortion function as the degree increases. We also compare and contrast the standard LDGM construction with a compound LDPC/LDGM construction introduced in our previous work, which provably saturates the rate-distortion bound with finite degrees. Moreover, this compound construction can be used to generate nested codes that are simultaneously good as source and channel codes, and are hence well-suited to source/channel coding with side information. The sparse and high-girth graphical structure of our constructions render them well-suited to message-passing encoding.