Sparse Regression Codes for Multi-terminal Source and Channel Coding

TL;DR

This paper introduces Sparse Regression Codes, a new class of codes for Gaussian multi-terminal source and channel coding, demonstrating their ability to achieve optimal information-theoretic limits using random binning and superposition coding.

Contribution

The paper shows how to implement random binning and superposition coding with sparse regression codes, achieving optimal limits for multi-terminal problems.

Findings

Achieves channel capacity of AWGN channels with feasible decoding.

Attains the optimal rate-distortion function for Gaussian sources.

Attains the limits for various multi-terminal source and channel coding problems.

Abstract

We study a new class of codes for Gaussian multi-terminal source and channel coding. These codes are designed using the statistical framework of high-dimensional linear regression and are called Sparse Superposition or Sparse Regression codes. Codewords are linear combinations of subsets of columns of a design matrix. These codes were recently introduced by Barron and Joseph and shown to achieve the channel capacity of AWGN channels with computationally feasible decoding. They have also recently been shown to achieve the optimal rate-distortion function for Gaussian sources. In this paper, we demonstrate how to implement random binning and superposition coding using sparse regression codes. In particular, with minimum-distance encoding/decoding it is shown that sparse regression codes attain the optimal information-theoretic limits for a variety of multi-terminal source and channel coding problems.