Renormalization in the neural network-quantum field theory correspondence

TL;DR

This paper explores the neural network-quantum field theory correspondence, demonstrating how renormalization concepts can be applied to neural networks, with preliminary numerical results showing the impact of weight distribution variations.

Contribution

It introduces a renormalization framework within the NN-QFT correspondence, linking weight distribution changes to renormalization flows and providing initial numerical insights.

Findings

Changing weight standard deviation induces renormalization flow.

Finite N corrections correspond to interactions in the field theory.

Preliminary numerical results support the theoretical framework.

Abstract

A statistical ensemble of neural networks can be described in terms of a quantum field theory (NN-QFT correspondence). The infinite-width limit is mapped to a free field theory, while finite N corrections are mapped to interactions. After reviewing the correspondence, we will describe how to implement renormalization in this context and discuss preliminary numerical results for translation-invariant kernels. A major outcome is that changing the standard deviation of the neural network weight distribution corresponds to a renormalization flow in the space of networks.