Stochastic automatic differentiation for Monte Carlo processes

TL;DR

This paper extends automatic differentiation techniques to Monte Carlo processes, enabling efficient computation of derivatives of expectation values, with potential for improved variance reduction methods.

Contribution

It introduces a Hamiltonian-based approach for differentiating Monte Carlo expectations, reducing variance compared to reweighting methods.

Findings

Hamiltonian approach reduces variance of derivatives

Reweighting and Hamiltonian methods are related

Potential for new variance reduction techniques

Abstract

Monte Carlo methods represent a cornerstone of computer science. They allow to sample high dimensional distribution functions in an efficient way. In this paper we consider the extension of Automatic Differentiation (AD) techniques to Monte Carlo process, addressing the problem of obtaining derivatives (and in general, the Taylor series) of expectation values. Borrowing ideas from the lattice field theory community, we examine two approaches. One is based on reweighting while the other represents an extension of the Hamiltonian approach typically used by the Hybrid Monte Carlo (HMC) and similar algorithms. We show that the Hamiltonian approach can be understood as a change of variables of the reweighting approach, resulting in much reduced variances of the coefficients of the Taylor series. This work opens the door to find other variance reduction techniques for derivatives of expectation values.