Holographic functions and neural networks

TL;DR

This paper establishes the equivalence of three complexity notions for fuzzy Boolean functions: sampling-based holography, polynomial approximation, and neural network representation.

Contribution

It introduces and proves the equivalence of three different bounded complexity properties for fuzzy Boolean functions, connecting sampling, structural, and computational perspectives.

Findings

The three properties are equivalent up to parameter changes.

Holographic property implies polynomial structure via hypergraph regularity.

The paper provides a unified framework for understanding complexity in fuzzy Boolean functions.

Abstract

A fuzzy Boolean function is a map $f:\cube^n\to [0,1]$, where $n\in\mathbb N$. We introduce and compare three ways of saying that such a function has bounded complexity. The first is a sampling property: the value $f(x)$ can be recovered, up to small error and with high probability, from the values of a bounded number of randomly chosen coordinates of $x$. We call this the holographic property. The second is a structural property: $f$ is uniformly close to a bounded-degree polynomial in boundedly many bounded linear coordinate forms. The third is computational: $f$ is uniformly close to the output of a neural network with a bounded number of non-input neurons, bounded Lipschitz activation functions and bounded incoming weights. We prove that these three properties are equivalent up to quantitative changes of the parameters. The implication from holography to polynomial structure uses a variant of a weak version of hypergraph regularity.