Codes That Achieve Capacity on Symmetric Channels

TL;DR

This paper surveys the development of capacity-achieving codes for symmetric channels, highlighting polar codes, their improvements, and related 2-transitive codes like Reed-Muller, emphasizing their theoretical and practical significance.

Contribution

It provides an overview of capacity-achieving coding schemes, including polar codes and 2-transitive codes, and discusses recent theoretical advances and their implications.

Findings

Polar codes achieve capacity with low complexity.

Error probability decreases doubly exponentially with block length.

2-transitive codes like Reed-Muller also achieve capacity.

Abstract

Transmission of information reliably and efficiently across channels is one of the fundamental goals of coding and information theory. In this respect, efficiently decodable deterministic coding schemes which achieve capacity provably have been elusive until as recent as 2008, even though schemes which come close to it in practice existed. This survey tries to give the interested reader an overview of the area. Erdal Arikan came up with his landmark polar coding shemes which achieve capacity on symmetric channels subject to the constraint that the input codewords are equiprobable. His idea is to convert any B-DMC into efficiently encodable-decodable channels which have rates 0 and 1, while conserving capacity in this transformation. An exponentially decreasing probability of error which independent of code rate is achieved for all rates lesser than the symmetric capacity. These codes perform well in practice since encoding and decoding complexity is O(N log N). Guruswami et al. improved the above results by showing that error probability can be made to decrease doubly exponentially in the block length. We also study recent results by Urbanke et al. which show that 2-transitive codes also achieve capacity on erasure channels under MAP decoding. Urbanke and his group use complexity theoretic results in boolean function analysis to prove that EXIT functions, which capture the error probability, have a sharp threshold at 1-R, thus proving that capacity is achieved. One of the oldest and most widely used codes - Reed Muller codes are 2-transitive. Polar codes are 2-transitive too and we thus have a different proof of the fact that they achieve capacity, though the rate of polarization would be better as found out by Guruswami.