On The Capacity Of Noisy Computations

TL;DR

This paper introduces an information-theoretic framework to analyze the capacity of unreliable, noisy computations, extending communication channel models to understand how reliably functions can be computed with noisy devices.

Contribution

It defines the capacity of noisy computation, provides coding theorems and converse results, and characterizes conditions for reliable computation using information theory.

Findings

Capacity bounds for noisy computation are established.

Reliable computation is achievable under certain encoding constraints.

Theoretical models extend to formal computational functions.

Abstract

This paper presents an analysis of the concept of capacity for noisy com- putations, i.e. functions implemented by unreliable or random devices. An information theoretic model of noisy computation of a perfect function f (measurable function between sequence spaces) thanks to an unreliable device (random channel) F is given: a noisy computation is a product fxF of channels. A model of reliable computation based on input encoding and output decoding is also proposed. These models extend those of noisy communication channel and of reliable communication through a noisy channel. The capacity of a noisy computation is defined and justified by a coding theorem and a converse. Under some constraints on the encoding process, capacity is the upper bound of input rates allowing reliable computation, i.e. decodability of noisy outputs into expected outputs. These results hold when the one-sided random processes under concern are asymptotic mean stationary (AMS) and ergodic. In addition, some characterizations of AMS and ergodic noisy computations are given based on stability properties of the perfect function f and of the random channel F. These results are derived from the more general framework of channel products. Finally, a way to apply the noisy and reliable computation models to cases where the perfect function f is defined according to a formal computational model is proposed.