On the Capacity of Noisy Computations

TL;DR

This paper defines the capacity of noisy computations, establishing theoretical bounds and coding theorems for reliable computation using unreliable devices, extending classical information theory results to computational models.

Contribution

It introduces a formal capacity measure for noisy computations and proves coding theorems that extend Feinstein's theorem to this setting.

Findings

Capacity bounds for noisy computations are established.

Coding theorems for reliable computation with noisy devices are proved.

Conditions for reliable computation using encoding and decoding are identified.

Abstract

This paper presents an analysis of the concept of capacity for noisy computations, i.e. algorithms implemented by unreliable computing devices (e.g. noisy Turing Machines). The capacity of a noisy computation is defined and justified by companion coding theorems. Under some constraints on the encoding process, capacity is the upper bound of input rates allowing reliable computation, i.e. decodability of noisy outputs into expected outputs. A model of noisy computation of a perfect function f thanks to an unreliable device F is given together with a model of reliable computation based on input encoding and output decoding. A coding lemma (extending the Feinstein's theorem to noisy computations), a joint source-computation coding theorem and its converse are proved. They apply if the input source, the function f, the noisy device F and the cascade f^{-1}F induce AMS and ergodic one-sided random processes.