Sparse Regression Codes

TL;DR

This survey comprehensively reviews Sparse Regression Codes (SPARCs), highlighting their theoretical foundations, algorithms, and practical applications for approaching Shannon limits in Gaussian channels and sources.

Contribution

It provides a unified overview of SPARCs across various communication and compression scenarios, including new insights into multi-terminal models and future research directions.

Findings

SPARCs achieve near-Shannon limit performance for AWGN channels.

The survey covers algorithms and implementation strategies for SPARCs.

Extensions of SPARCs to multi-terminal communication models are discussed.

Abstract

Developing computationally-efficient codes that approach the Shannon-theoretic limits for communication and compression has long been one of the major goals of information and coding theory. There have been significant advances towards this goal in the last couple of decades, with the emergence of turbo codes, sparse-graph codes, and polar codes. These codes are designed primarily for discrete-alphabet channels and sources. For Gaussian channels and sources, where the alphabet is inherently continuous, Sparse Superposition Codes or Sparse Regression Codes (SPARCs) are a promising class of codes for achieving the Shannon limits. This survey provides a unified and comprehensive overview of sparse regression codes, covering theory, algorithms, and practical implementation aspects. The first part of the monograph focuses on SPARCs for AWGN channel coding, and the second part on SPARCs for lossy compression (with squared error distortion criterion). In the third part, SPARCs are used to construct codes for Gaussian multi-terminal channel and source coding models such as broadcast channels, multiple-access channels, and source and channel coding with side information. The survey concludes with a discussion of open problems and directions for future work.