Neural Network Field Theories: Non-Gaussianity, Actions, and Locality

TL;DR

This paper develops a framework linking neural network ensembles to field theories, enabling systematic construction of neural architectures from field theory actions and vice versa, with applications to non-Gaussian and interacting theories.

Contribution

It introduces a new Feynman diagram method to reconstruct neural network field actions from correlators, extending the theoretical understanding beyond Gaussian limits.

Findings

Derived actions for neural network field theories using connected correlators.

Established a correspondence between field theory deformations and neural network architecture modifications.

Realized $4$ theory as an example of neural network field theory.

Abstract

Both the path integral measure in field theory and ensembles of neural networks describe distributions over functions. When the central limit theorem can be applied in the infinite-width (infinite-$N$) limit, the ensemble of networks corresponds to a free field theory. Although an expansion in $1/N$ corresponds to interactions in the field theory, others, such as in a small breaking of the statistical independence of network parameters, can also lead to interacting theories. These other expansions can be advantageous over the $1/N$-expansion, for example by improved behavior with respect to the universal approximation theorem. Given the connected correlators of a field theory, one can systematically reconstruct the action order-by-order in the expansion parameter, using a new Feynman diagram prescription whose vertices are the connected correlators. This method is motivated by the Edgeworth expansion and allows one to derive actions for neural network field theories. Conversely, the correspondence allows one to engineer architectures realizing a given field theory by representing action deformations as deformations of neural network parameter densities. As an example, $\phi^4$ theory is realized as an infinite-$N$ neural network field theory.