# Continuously Tempered Diffusion Samplers

## TL;DR

This paper introduces continuously tempered diffusion samplers that enhance exploration during training by using a temperature-variant distribution, leading to improved sampling performance for neural samplers of unnormalized distributions.

## Contribution

It proposes a novel sampler that employs a temperature-variant distribution to improve exploration and training of neural samplers, addressing limitations of previous annealing-based methods.

## Key findings

- Enhanced exploration leads to better sampler performance.
- Empirical validation shows improved sampling quality.
- Code availability facilitates reproducibility.

## Abstract

Annealing-based neural samplers seek to amortize sampling from unnormalized distributions by training neural networks to transport a family of densities interpolating from source to target. A crucial design choice in the training phase of such samplers is the proposal distribution by which locations are generated at which to evaluate the loss. Previous work has obtained such a proposal distribution by combining a partially learned transport with annealed Langevin dynamics. However, isolated modes and other pathological properties of the annealing path imply that such proposals achieve insufficient exploration and thereby lower performance post training. To remedy this, we propose continuously tempered diffusion samplers, which leverage exploration techniques developed in the context of molecular dynamics to improve proposal distributions. Specifically, a family of distributions across different temperatures is introduced to lower energy barriers at higher temperatures and drive exploration at the lower temperature of interest. We empirically validate improved sampler performance driven by extended exploration. Code is available at https://github.com/eje24/ctds.