Double descent: When do neural quantum states generalize?

TL;DR

This paper investigates the double descent phenomenon in neural quantum states (NQS), revealing that NQS exhibit this behavior only at very large sizes, which are impractical for real quantum problems, highlighting the need for physics-informed architectures.

Contribution

It demonstrates that NQS show double descent in a simplified setting, and identifies that typical NQS are in the underparameterized regime, emphasizing the importance of symmetry-aware design.

Findings

NQS exhibit double descent in a simplified supervised setting.

Typical NQS are in the underparameterized regime for practical quantum problems.

Optimal network size depends on the number of training samples.

Abstract

Neural quantum states (NQS) provide flexible and compact wavefunction parameterizations for numerical studies of quantum many-body physics. In particular, NQS aim to circumvent the exponential scaling of the Hilbert space by compressing quantum many-body wavefunctions with a tractable amount of parameters. While inspired by deep learning, it remains unclear to what extent NQS share characteristics with neural networks used for standard machine learning tasks. We demonstrate that, in a simplified supervised setting, NQS exhibit the double descent phenomenon, a key feature of modern deep learning, where generalization worsens as network size increases before improving again in an overparameterized regime. Notably, we find the second descent to occur only for network sizes much larger than the Hilbert space dimension, i.e. network sizes that are out of reach for problems of practical interest. Within our setting, this observation places typical NQS in the underparameterized regime. We also observe that the optimal network size in the underparameterized regime depends on the number of unique training samples. While the double descent phenomenon does indeed translate to the NQS setting, potential practical consequences of our findings point more towards the need for symmetry-aware, physics-informed architecture design, rather than directly adopting machine learning heuristics.