Strong Data Processing Inequalities and their Applications to Reliable Computation

TL;DR

This paper explores the use of strong data-processing inequalities (SDPIs) to analyze reliable computation with unreliable components, extending classical results and providing a unified framework for understanding noise thresholds in computational circuits.

Contribution

It introduces a comprehensive exposition of SDPIs and their applications to reliable computation, extending von Neumann's classical results to new gate types and connecting them to modern information-theoretic bounds.

Findings

Extended von Neumann's reliable computation results to minority gates.

Connected SDPI-based bounds to Bayesian network mutual information contraction.

Provided a unified framework for noise thresholds in unreliable circuits.

Abstract

In 1952, von Neumann gave a series of groundbreaking lectures that proved it was possible for circuits consisting of 3-input majority gates that have a sufficiently small independent probability $\delta > 0$ of malfunctioning to reliably compute Boolean functions. In 1999, Evans and Schulman used a strong data-processing inequality (SDPI) to establish the tightest known necessary condition $\delta < \frac{1}{2} - \frac{1}{2\sqrt{k}}$ for reliable computation when the circuit consists of components that have at most $k$ inputs. In 2017, Polyanskiy and Wu distilled Evans and Schulman's SDPI argument to establish a general result on the contraction of mutual information in Bayesian networks. In this essay, we will first introduce the problem of reliable computation from unreliable components and establish the existence of noise thresholds. We will then provide an exposition of von Neumann's result with 3-input majority gates and extend it to minority gates. We will then provide an introduction to SDPIs, which have many applications, including in statistical mechanics, portfolio theory, and lower bounds on statistical estimation under privacy constraints. We will then use the introduced material to provide an exposition of Polyanskiy and Wu's 2017 result on Bayesian networks, from which the 1999 result of Evans-Schulman follows.