Nesterov smoothing for sampling without smoothness

TL;DR

This paper introduces a novel sampling algorithm for non-smooth potentials by approximating them with smooth potentials using Nesterov smoothing, enabling effective sampling and convergence guarantees.

Contribution

The paper proposes a new method that applies Nesterov smoothing to facilitate sampling from non-smooth distributions, with proven convergence guarantees.

Findings

Achieves non-asymptotic convergence guarantees.

Demonstrates improved Bayesian inference performance.

Validates method on synthetic and real-world data.

Abstract

We study the problem of sampling from a target distribution in $\mathbb{R}^d$ whose potential is not smooth. Compared with the sampling problem with smooth potentials, this problem is much less well-understood due to the lack of smoothness. In this paper, we propose a novel sampling algorithm for a class of non-smooth potentials by first approximating them by smooth potentials using a technique that is akin to Nesterov smoothing. We then utilize sampling algorithms on the smooth potentials to generate approximate samples from the original non-smooth potentials. We select an appropriate smoothing intensity to ensure that the distance between the smoothed and un-smoothed distributions is minimal, thereby guaranteeing the algorithm's accuracy. Hence we obtain non-asymptotic convergence results based on existing analysis of smooth sampling. We verify our convergence result on a synthetic example and apply our method to improve the worst-case performance of Bayesian inference on a real-world example.