Structures of Neural Network Effective Theories

TL;DR

This paper introduces a diagrammatic effective field theory approach to analyze neural networks at initialization, simplifying the computation of neuron statistics and revealing a unified criticality condition for all correlators.

Contribution

It develops a novel diagrammatic EFT framework for neural networks, providing new insights into their critical behavior and finite-width corrections.

Findings

Unified criticality condition for neuron correlators

Simplified computation of finite-width corrections

Potential applications in deep learning and field theory simulations

Abstract

We develop a diagrammatic approach to effective field theories (EFTs) corresponding to deep neural networks at initialization, which dramatically simplifies computations of finite-width corrections to neuron statistics. The structures of EFT calculations make it transparent that a single condition governs criticality of all connected correlators of neuron preactivations. Understanding of such EFTs may facilitate progress in both deep learning and field theory simulations.