A duality connecting neural network and cosmological dynamics

TL;DR

This paper reveals a profound duality between neural network training dynamics and scalar field cosmological models, enabling new insights and simulation methods for early Universe physics and neural networks.

Contribution

It establishes a theoretical duality linking neural network gradient descent dynamics with cosmological scalar field evolution, providing a novel framework for cross-disciplinary understanding.

Findings

Analytical matching of neural network and cosmological dynamics.

Empirical validation of the duality through neural network experiments.

Dependence of effective field theory parameters on neural network hyperparameters.

Abstract

We demonstrate that the dynamics of neural networks trained with gradient descent and the dynamics of scalar fields in a flat, vacuum energy dominated Universe are structurally profoundly related. This duality provides the framework for synergies between these systems, to understand and explain neural network dynamics and new ways of simulating and describing early Universe models. Working in the continuous-time limit of neural networks, we analytically match the dynamics of the mean background and the dynamics of small perturbations around the mean field, highlighting potential differences in separate limits. We perform empirical tests of this analytic description and quantitatively show the dependence of the effective field theory parameters on hyperparameters of the neural network. As a result of this duality, the cosmological constant is matched inversely to the learning rate in the gradient descent update.