Path Integral Sampler: a stochastic control approach for sampling

TL;DR

The paper introduces Path Integral Sampler (PIS), a novel stochastic control-based algorithm for sampling from unnormalized distributions, leveraging Schr"odinger bridge theory and neural networks for end-to-end training.

Contribution

It formulates sampling as a stochastic optimal control problem using Schr"odinger bridges and neural networks, providing theoretical guarantees and importance weighting for improved sampling.

Findings

PIS achieves better sample quality than existing methods.

Theoretical bounds on sampling error in Wasserstein distance.

Effective importance weighting improves sample accuracy.

Abstract

We present Path Integral Sampler~(PIS), a novel algorithm to draw samples from unnormalized probability density functions. The PIS is built on the Schr\"odinger bridge problem which aims to recover the most likely evolution of a diffusion process given its initial distribution and terminal distribution. The PIS draws samples from the initial distribution and then propagates the samples through the Schr\"odinger bridge to reach the terminal distribution. Applying the Girsanov theorem, with a simple prior diffusion, we formulate the PIS as a stochastic optimal control problem whose running cost is the control energy and terminal cost is chosen according to the target distribution. By modeling the control as a neural network, we establish a sampling algorithm that can be trained end-to-end. We provide theoretical justification of the sampling quality of PIS in terms of Wasserstein distance when sub-optimal control is used. Moreover, the path integrals theory is used to compute importance weights of the samples to compensate for the bias induced by the sub-optimality of the controller and time-discretization. We experimentally demonstrate the advantages of PIS compared with other start-of-the-art sampling methods on a variety of tasks.