Sampling by averaging: A multiscale approach to score estimation

TL;DR

This paper presents a multiscale sampling framework that estimates scores efficiently without training or nested MCMC, using stochastic averaging in SDEs, and demonstrates competitive results on complex distributions.

Contribution

It introduces a novel multiscale approach for score estimation that avoids training and nested MCMC, with new algorithms and theoretical guarantees.

Findings

Competitive accuracy and efficiency on synthetic and real-world benchmarks.

Effective handling of heavy-tailed distributions with Student's t noise.

Theoretical convergence guarantees for the proposed algorithms.

Abstract

We introduce a novel framework for efficient sampling from complex, unnormalised target distributions by exploiting multiscale dynamics. Traditional score-based sampling methods either rely on learned approximations of the score function or involve computationally expensive nested Markov chain Monte Carlo (MCMC) loops. In contrast, the proposed approach leverages stochastic averaging within a slow-fast system of stochastic differential equations (SDEs) to estimate intermediate scores along a diffusion path without training or inner-loop MCMC. Two algorithms are developed under this framework: MultALMC, which uses multiscale annealed Langevin dynamics, and MultCDiff, based on multiscale controlled diffusions for the reverse-time Ornstein-Uhlenbeck process. Both overdamped and underdamped variants are considered, with theoretical guarantees of convergence to the desired diffusion path. The framework is extended to handle heavy-tailed target distributions using Student's t-based noise models and tailored fast-process dynamics. Empirical results across synthetic and real-world benchmarks, including multimodal and high-dimensional distributions, demonstrate that the proposed methods are competitive with existing samplers in terms of accuracy and efficiency, without the need for learned models.