Training normalizing flows with computationally intensive target probability distributions

TL;DR

This paper introduces a REINFORCE-based estimator for normalizing flows that significantly reduces computational costs and memory usage when modeling complex, resource-intensive probability distributions, demonstrated on lattice field theory models.

Contribution

The paper proposes a novel REINFORCE-based estimator for normalizing flows that circumvents derivative calculations of complex actions, improving efficiency and stability.

Findings

Up to ten times faster in wall-clock time

Requires up to 30% less memory

Enables single precision and half-float computations

Abstract

Machine learning techniques, in particular the so-called normalizing flows, are becoming increasingly popular in the context of Monte Carlo simulations as they can effectively approximate target probability distributions. In the case of lattice field theories (LFT) the target distribution is given by the exponential of the action. The common loss function's gradient estimator based on the "reparametrization trick" requires the calculation of the derivative of the action with respect to the fields. This can present a significant computational cost for complicated, non-local actions like e.g. fermionic action in QCD. In this contribution, we propose an estimator for normalizing flows based on the REINFORCE algorithm that avoids this issue. We apply it to two dimensional Schwinger model with Wilson fermions at criticality and show that it is up to ten times faster in terms of the wall-clock time as well as requiring up to $30\%$ less memory than the reparameterization trick estimator. It is also more numerically stable allowing for single precision calculations and the use of half-float tensor cores. We present an in-depth analysis of the origins of those improvements. We believe that these benefits will appear also outside the realm of the LFT, in each case where the target probability distribution is computationally intensive.