A Proximal Algorithm for Sampling from Non-smooth Potentials

TL;DR

This paper introduces a new Markov chain Monte Carlo algorithm for efficiently sampling from non-smooth potentials, achieving improved polynomial-time complexity and leveraging a novel restricted Gaussian oracle.

Contribution

It presents a proximal algorithm using a restricted Gaussian oracle for non-smooth potentials, with non-asymptotic analysis and better complexity bounds than existing methods.

Findings

Achieves polynomial-time complexity $ ilde{O}(d ext{ extasciitilde}\varepsilon^{-1})$

Provides a fast algorithm for the restricted Gaussian oracle

Demonstrates improved sampling efficiency for non-smooth potentials

Abstract

In this work, we examine sampling problems with non-smooth potentials. We propose a novel Markov chain Monte Carlo algorithm for sampling from non-smooth potentials. We provide a non-asymptotical analysis of our algorithm and establish a polynomial-time complexity $\tilde {\cal O}(d\varepsilon^{-1})$ to obtain $\varepsilon$ total variation distance to the target density, better than most existing results under the same assumptions. Our method is based on the proximal bundle method and an alternating sampling framework. This framework requires the so-called restricted Gaussian oracle, which can be viewed as a sampling counterpart of the proximal mapping in convex optimization. One key contribution of this work is a fast algorithm that realizes the restricted Gaussian oracle for any convex non-smooth potential with bounded Lipschitz constant.