Conditional Diffusion Sampling

TL;DR

This paper introduces Conditional Diffusion Sampling (CDS), a novel framework combining Parallel Tempering and diffusion processes to efficiently sample from complex distributions with fewer density evaluations.

Contribution

The authors propose Conditional Diffusion Sampling, a new method that integrates PT and diffusion SDEs without neural approximation, improving sampling efficiency.

Findings

CDS achieves better sample quality with fewer density evaluations.

The initialization cost diminishes for short diffusion times, enhancing efficiency.

Experiments show CDS outperforms state-of-the-art samplers in trade-offs.

Abstract

Sampling from unnormalized multimodal distributions with limited density evaluations remains a fundamental challenge in machine learning and natural sciences. Successful approaches construct a bridge between a tractable reference and the target distribution. Parallel Tempering (PT) serves as the gold standard, while recent diffusion-based approaches offer a continuous alternative at the cost of neural training. In this work, we introduce Conditional Diffusion Sampling (CDS), a framework that combines these two paradigms. To this end, we derive Conditional Interpolants, a class of stochastic processes whose transport dynamics are governed by an exact, closed-form stochastic differential equation (SDE), requiring no neural approximation. Although these dynamics require sampling from a non-trivial initialization distribution, we show both theoretically and empirically that the cost of this initialization diminishes for sufficiently short diffusion times. CDS leverages this by a two-stage procedure: (1) PT is used to efficiently sample the initial distribution, and then (2) samples are transported via the transport SDE. This combination couples the robust global exploration of PT with efficient local transport. Experiments suggest that CDS has the potential to achieve a superior trade-off between sample quality and density evaluation cost compared to state-of-the-art samplers.