Diffusion Path Samplers via Sequential Monte Carlo

TL;DR

This paper introduces diffusion path samplers using sequential Monte Carlo methods to efficiently estimate scores and densities for target distributions, with theoretical guarantees and empirical validation.

Contribution

It presents a novel framework combining diffusion models with sequential Monte Carlo to improve sampling and score estimation for complex distributions.

Findings

Effective score and density estimation demonstrated on synthetic datasets.

Control variate schedules reduce variance with minimal overhead.

Framework applicable to various diffusion processes with theoretical guarantees.

Abstract

We develop diffusion-based samplers for target distributions known up to a normalising constant. To this end, we rely on the well-known diffusion path that smoothly interpolates between a simple base distribution and the target, popularised by diffusion models. We tackle the score estimation problem by developing an efficient sequential Monte Carlo sampler that evolves auxiliary variables from conditional distributions along the path, providing principled score and density estimates for time-varying distributions. To control the variance of score estimates, we further propose practical control variate schedules that incur minimal overhead. We adapt this general framework to paths induced by the Ornstein-Uhlenbeck (OU) time-reversal process, stochastic interpolants, and diffusion annealed Langevin dynamics, outlining their trade-offs. Finally, we provide theoretical guarantees and empirically demonstrate the effectiveness of our method on several synthetic and real-world datasets.