Error correcting code using tree-like multilayer perceptron

TL;DR

This paper introduces error correcting codes based on tree-like multilayer perceptrons, analyzing their performance over noisy channels and demonstrating conditions under which they approach Shannon capacity.

Contribution

It proposes novel coding schemes using tree-like neural network architectures and analyzes their performance with statistical mechanics methods.

Findings

Some schemes saturate the Shannon bound at infinite code length.

Monotonicity of units affects the coding performance.

Analytical performance evaluated using the replica method.

Abstract

An error correcting code using a tree-like multilayer perceptron is proposed. An original message $\mbi{s}^0$ is encoded into a codeword $\boldmath{y}_0$ using a tree-like committee machine (committee tree) or a tree-like parity machine (parity tree). Based on these architectures, several schemes featuring monotonic or non-monotonic units are introduced. The codeword $\mbi{y}_0$ is then transmitted via a Binary Asymmetric Channel (BAC) where it is corrupted by noise. The analytical performance of these schemes is investigated using the replica method of statistical mechanics. Under some specific conditions, some of the proposed schemes are shown to saturate the Shannon bound at the infinite codeword length limit. The influence of the monotonicity of the units on the performance is also discussed.