Communication near the channel capacity with an absence of compression: Statistical Mechanical Approach

TL;DR

This paper extends Shannon's theory using statistical physics to calculate channel capacity with autocorrelated messages, leading to efficient coding schemes that approach the theoretical limit without data compression.

Contribution

It introduces a statistical mechanical approach to channel capacity calculation and proposes a novel decoding method that updates prior probabilities dynamically.

Findings

Transfer matrix method effectively computes channel capacity.

Codes nearly saturate the channel capacity.

Decoding improves performance by updating prior probabilities.

Abstract

The generalization of Shannon's theory to include messages with given autocorrelations is presented. The analytical calculation of the channel capacity is based on the transfer matrix method of the effective 1D Hamiltonian. This bridge between statistical physics and information theory leads to efficient Low-Density Parity-Check Codes over Galois fields that nearly saturate the channel capacity. The novel idea of the decoder is the dynamical updating of the prior block probabilities which are derived from the transfer matrix solution and from the posterior probabilities of the neighboring blocks. Application and possible extensions are discussed, specifically the possibility of achieving the channel capacity without compression of the data.