Neural Network Quantum Field Theory from Transformer Architectures

TL;DR

This paper introduces a novel approach to constructing Euclidean scalar quantum field theories using transformer neural networks, analyzing their correlation functions and the emergence of Gaussian behavior in the large-head limit.

Contribution

It develops a neural-network framework for quantum field theories based on transformer attention, characterizes non-Gaussian features, and demonstrates how Gaussian behavior emerges with many attention heads.

Findings

Two-point functions can be computed using attention-weight representations.

Non-Gaussian correlations persist at infinite width for single attention heads.

Summing many heads suppresses non-Gaussian correlations, leading to Gaussian behavior.

Abstract

We propose a neural-network construction of Euclidean scalar quantum field theories from transformer attention heads, defining $n$-point correlators by averaging over random network parameters in the NN-QFT framework. For a single attention head, shared random softmax weights couple different width coordinates and induce non-Gaussian field statistics that persist in the infinite-width limit $d_k\to\infty$. We compute the two-point function in an attention-weight representation and show how Euclidean-invariant kernels can be engineered via random-feature token embeddings. We then analyze the connected four-point function and identify an "independence-breaking" contribution, expressible as a covariance over query-key weights, which remains finite at infinite width. Finally, we show that summing many independent heads with standard $1/N_h$ normalization suppresses connected non-Gaussian correlators as $1/N_h$, yielding a Gaussian NN-QFT in the large-head limit.