A Proximal Algorithm for Sampling

TL;DR

This paper introduces a proximal sampling algorithm based on an alternating sampling framework that effectively handles non-smooth, convex, or non-convex potentials, outperforming existing methods in complexity.

Contribution

It develops a practical proximal sampling algorithm using rejection sampling for non-smooth and non-convex potentials, extending the applicability of sampling methods beyond smooth cases.

Findings

Achieves better complexity than existing sampling methods.

Handles non-smooth and non-convex potentials effectively.

Provides a practical realization of the alternating sampling framework.

Abstract

We study sampling problems associated with potentials that lack smoothness. The potentials can be either convex or non-convex. Departing from the standard smooth setting, the potentials are only assumed to be weakly smooth or non-smooth, or the summation of multiple such functions. We develop a sampling algorithm that resembles proximal algorithms in optimization for this challenging sampling task. Our algorithm is based on a special case of Gibbs sampling known as the alternating sampling framework (ASF). The key contribution of this work is a practical realization of the ASF based on rejection sampling for both non-convex and convex potentials that are not necessarily smooth. In almost all the cases of sampling considered in this work, our proximal sampling algorithm achieves better complexity than all existing methods.