A Unified Theory of Quantum Neural Network Loss Landscapes

TL;DR

This paper introduces a unified theoretical framework for quantum neural network loss landscapes, modeling them as Wishart processes to analyze trainability, gradient behavior, and minima distribution, advancing understanding beyond classical neural networks.

Contribution

It proves that QNNs generally form Wishart processes, providing conditions for Gaussian process limits and calculating gradient and minima distributions, which is a novel theoretical insight.

Findings

QNNs form Wishart processes instead of Gaussian processes.

Derived conditions for Gaussian process limits in QNNs.

Calculated gradient distributions and minima distributions for QNNs.

Abstract

Classical neural networks with random initialization famously behave as Gaussian processes in the limit of many neurons, which allows one to completely characterize their training and generalization behavior. No such general understanding exists for quantum neural networks (QNNs), which -- outside of certain special cases -- are known to not behave as Gaussian processes when randomly initialized. We here prove that QNNs and their first two derivatives instead generally form what we call "Wishart processes," where certain algebraic properties of the network determine the hyperparameters of the process. This Wishart process description allows us to, for the first time: give necessary and sufficient conditions for a QNN architecture to have a Gaussian process limit; calculate the full gradient distribution, generalizing previously known barren plateau results; and calculate the local minima distribution of algebraically constrained QNNs. Our unified framework suggests a certain simple operational definition for the "trainability" of a given QNN model using a newly introduced, experimentally accessible quantity we call the "degrees of freedom" of the network architecture.