# Statistical field theory for neural networks

## TL;DR

This paper introduces statistical field theory methods to analyze neural networks, covering probabilistic concepts, stochastic dynamics, and mean-field approximations, with applications to disordered systems and maximum entropy models.

## Contribution

It provides a comprehensive, self-contained framework applying statistical field theory to neural networks, including new derivations of mean-field and beyond-mean-field theories.

## Key findings

- Derivation of self-consistent dynamic mean-field equations.
- Systematic approach to fluctuations and phase transitions.
- Application to Ising models and TAP mean field theory.

## Abstract

These notes attempt a self-contained introduction into statistical field theory applied to neural networks of rate units and binary spins. The presentation consists of three parts: First, the introduction of fundamental notions of probabilities, moments, cumulants, and their relation by the linked cluster theorem, of which Wick's theorem is the most important special case; followed by the diagrammatic formulation of perturbation theory, reviewed in the statistical setting. Second, dynamics described by stochastic differential equations in the Ito-formulation, treated in the Martin-Siggia-Rose-De Dominicis-Janssen path integral formalism. With concepts from disordered systems, we then study networks with random connectivity and derive their self-consistent dynamic mean-field theory, explaining the statistics of fluctuations and the emergence of different phases with regular and chaotic dynamics. Third, we introduce the effective action, vertex functions, and the loopwise expansion. These tools are illustrated by systematic derivations of self-consistency equations, going beyond the mean-field approximation. These methods are applied to the pairwise maximum entropy (Ising spin) model, including the recently-found diagrammatic derivation of the Thouless-Anderson-Palmer mean field theory.