Score-based sampling without diffusions: Guidance from a simple and modular scheme

TL;DR

This paper introduces a modular scheme for score-based sampling that simplifies the process by reducing it to solving a sequence of well-understood sampling problems, providing guarantees for both uni-modal and multi-modal distributions.

Contribution

It presents a novel modular approach that leverages existing SLC sampling algorithms to improve score-based sampling without relying on diffusion models.

Findings

Achieves $ ext{ε}$-accurate sampling in KL and Wasserstein distances.

Provides polynomial dependence on $ ext{log}(1/ε)$ and $ ext{√d}$ for accuracy and dimension.

Establishes guarantees for sampling from both uni-modal and multi-modal densities.

Abstract

Sampling based on score diffusions has led to striking empirical results, and has attracted considerable attention from various research communities. It depends on availability of (approximate) Stein score functions for various levels of additive noise. We describe and analyze a modular scheme that reduces score-based sampling to solving a short sequence of ``nice'' sampling problems, for which high-accuracy samplers are known. We show how to design forward trajectories such that both (a) the terminal distribution, and (b) each of the backward conditional distribution is defined by a strongly log concave (SLC) distribution. This modular reduction allows us to exploit \emph{any} SLC sampling algorithm in order to traverse the backwards path, and we establish novel guarantees with short proofs for both uni-modal and multi-modal densities. The use of high-accuracy routines yields $\varepsilon$-accurate answers, in either KL or Wasserstein distances, with polynomial dependence on $\log(1/\varepsilon)$ and $\sqrt{d}$ dependence on the dimension.