Typicality of thermal states in isolated quantum systems corresponds to ubiquity of global minima in deep artificial neural networks

TL;DR

This paper reveals a deep connection between the typicality of thermal states in quantum systems and the prevalence of global minima in neural network training, linking quantum statistical mechanics with deep learning optimization.

Contribution

It establishes a theoretical correspondence between quantum thermal state typicality and neural network global minima, highlighting a shared underlying mechanism involving observables and Wishart matrices.

Findings

Global minima in neural networks relate to thermal state typicality in quantum systems.

Increase in distinguishability of reduced states correlates with neural network double descent.

Unified mechanism involving observables and Wishart matrices explains both phenomena.

Abstract

The Neural Tangent Kernel theory theoretically guarantees the existence of a global minima of the cost function in the neighborhood of an arbitrary random initialization in deep artificial neural networks. In this paper, we show that the ubiquity of the global minima directly corresponds to the typicality of pure thermal states in isolated quantum systems by showing a common underlying mechanism, involving a few observables and the role of a Wishart-type matrix. Moreover, we demonstrate that the increase in distinguishability of the reduced density matrices of typical pure states with subsystem size corresponds to the double descent phenomenon observed by varying the width of layers in finite-width artificial neural networks.