Optimizing Temperature Distributions for Training Neural Quantum States using Parallel Tempering

TL;DR

This paper introduces an adaptive temperature optimization method for parallel tempering in neural network-based variational algorithms, significantly improving training success rates by reducing bottlenecks.

Contribution

It presents a novel adaptive temperature scheme that enhances parallel tempering efficiency in training neural quantum states, with minimal computational overhead.

Findings

Adaptive temperature adjustment improves success rates

Eliminates bottlenecks in replica random walks

Effective for different neural network architectures

Abstract

Parameterized artificial neural networks (ANNs) can be very expressive ansatzes for variational algorithms, reaching state-of-the-art energies on many quantum many-body Hamiltonians. Nevertheless, the training of the ANN can be slow and stymied by the presence of local minima in the parameter landscape. One approach to mitigate this issue is to use parallel tempering methods, and in this work we focus on the role played by the temperature distribution of the parallel tempering replicas. Using an adaptive method that adjusts the temperatures in order to equate the exchange probability between neighboring replicas, we show that this temperature optimization can significantly increase the success rate of the variational algorithm with negligible computational cost by eliminating bottlenecks in the replicas' random walk. We demonstrate this using two different neural networks, a restricted Boltzmann machine and a feedforward network, which we use to study a toy problem based on a permutation invariant Hamiltonian with a pernicious local minimum and the J1-J2 model on a rectangular lattice.