Applications of Statistical Field Theory in Deep Learning

TL;DR

This paper reviews how statistical field theory offers valuable insights into deep learning, addressing its complexity by providing a physics-inspired framework for understanding generalization, bias, and feature learning.

Contribution

It presents a pedagogical overview of recent research applying statistical field theory to deepen understanding of deep learning phenomena.

Findings

Field theory helps explain generalization in deep learning.

Implicit bias in models can be analyzed through field-theoretic methods.

Feature learning effects are elucidated using statistical physics tools.

Abstract

Deep learning algorithms have made incredible strides in the past decade, yet due to their complexity, the science of deep learning remains in its early stages. Being an experimentally driven field, it is natural to seek a theory of deep learning within the physics paradigm. As deep learning is largely about learning functions and distributions over functions, statistical field theory, a rich and versatile toolbox for tackling complex distributions over functions (fields) is an obvious choice of formalism. Research efforts carried out in the past few years have demonstrated the ability of field theory to provide useful insights on generalization, implicit bias, and feature learning effects. Here we provide a pedagogical review of this emerging line of research.