A variational approach to sampling in diffusion processes

TL;DR

This paper explores a control-theoretic framework for sampling in diffusion processes, unifying importance sampling, time reversal, and Schrödinger bridges through a variational approach based on the Gibbs principle.

Contribution

It introduces a unified variational control method for sampling in diffusion processes, connecting multiple existing techniques under a common theoretical framework.

Findings

Unified control-based sampling approach for diffusion processes

Connections established between importance sampling, time reversal, and Schrödinger bridges

Provides a theoretical foundation for improved sampling methods

Abstract

We revisit the work of Mitter and Newton on an information-theoretic interpretation of Bayes' formula through the Gibbs variational principle. This formulation allowed them to pose nonlinear estimation for diffusion processes as a problem in stochastic optimal control, so that the posterior density of the signal given the observation path could be sampled by adding a drift to the signal process. We show that this control-theoretic approach to sampling provides a common mechanism underlying several distinct problems involving diffusion processes, specifically importance sampling using Feynman-Kac averages, time reversal, and Schr\"odinger bridges.