Bulk-boundary decomposition of neural networks

TL;DR

This paper introduces a bulk-boundary decomposition framework for understanding deep neural network training dynamics, separating intrinsic architecture-driven behavior from data-dependent stochastic effects, and extends it with a field-theoretic formulation.

Contribution

It proposes a novel decomposition of neural network training dynamics into bulk and boundary components, providing new insights into their structure and behavior.

Findings

Reveals the intrinsic dynamics governed by architecture and activation functions.

Shows the boundary term captures stochastic interactions from training samples.

Develops a field-theoretic formulation based on the decomposition.

Abstract

We present the bulk-boundary decomposition as a new framework for understanding the training dynamics of deep neural networks. Starting from the stochastic gradient descent formulation, we show that the Lagrangian can be reorganized into a data-independent bulk term and a data-dependent boundary term. The bulk captures the intrinsic dynamics set by network architecture and activation functions, while the boundary reflects stochastic interactions from training samples at the input and output layers. This decomposition exposes the local and homogeneous structure underlying deep networks. As a natural extension, we develop a field-theoretic formulation of neural dynamics based on this decomposition.