Diffusion at Absolute Zero: Langevin Sampling Using Successive Moreau Envelopes [conference paper]

TL;DR

This paper introduces a new sampling method for Gibbs distributions using a sequence of approximations via Moreau envelopes, inspired by diffusion models, to improve convergence speed and handle multi-modal densities.

Contribution

It presents a novel Langevin sampling approach based on successive Moreau envelope approximations, combining theoretical analysis with experimental validation.

Findings

Enhanced convergence speed demonstrated in experiments.

Effective sampling from multi-modal densities.

Theoretical guarantees for the proposed method.

Abstract

In this article we propose a novel method for sampling from Gibbs distributions of the form $\pi(x)\propto\exp(-U(x))$ with a potential $U(x)$. In particular, inspired by diffusion models we propose to consider a sequence $(\pi^{t_k})_k$ of approximations of the target density, for which $\pi^{t_k}\approx \pi$ for $k$ small and, on the other hand, $\pi^{t_k}$ exhibits favorable properties for sampling for $k$ large. This sequence is obtained by replacing parts of the potential $U$ by its Moreau envelopes. Sampling is performed in an Annealed Langevin type procedure, that is, sequentially sampling from $\pi^{t_k}$ for decreasing $k$, effectively guiding the samples from a simple starting density to the more complex target. In addition to a theoretical analysis we show experimental results supporting the efficacy of the method in terms of increased convergence speed and applicability to multi-modal densities $\pi$.