Reverse-BSDE Monte Carlo

TL;DR

This paper introduces a novel FBSDE-based reformulation of diffusion models for sampling complex distributions, avoiding gradient pre-estimation and leveraging deep learning for efficient multidimensional sampling.

Contribution

It reformulates diffusion-based generative models as FBSDEs, providing a new perspective that improves sampling methods for complex distributions.

Findings

Proved the uniqueness of the FBSDE solution.

Developed a deep learning-based numerical solution.

Enhanced sampling of high-dimensional distributions.

Abstract

Recently, there has been a growing interest in generative models based on diffusions driven by the empirical robustness of these methods in generating high-dimensional photorealistic images and the possibility of using the vast existing toolbox of stochastic differential equations. %This remarkable ability may stem from their capacity to model and generate multimodal distributions. In this work, we offer a novel perspective on the approach introduced in Song et al. (2021), shifting the focus from a "learning" problem to a "sampling" problem. To achieve this, we reformulate the equations governing diffusion-based generative models as a Forward-Backward Stochastic Differential Equation (FBSDE), which avoids the well-known issue of pre-estimating the gradient of the log target density. The solution of this FBSDE is proved to be unique using non-standard techniques. Additionally, we propose a numerical solution to this problem, leveraging on Deep Learning techniques. This reformulation opens new pathways for sampling multidimensional distributions with densities known up to a normalization constant, a problem frequently encountered in Bayesian statistics.