Non-convex sampling for a mixture of locally smooth potentials

TL;DR

This paper studies non-convex sampling using Euler discretization for potentials that are mixtures of locally smooth distributions, providing convergence guarantees with polynomial dependence on dimension and improved rates under certain smoothness conditions.

Contribution

It introduces novel concepts of $oldsymbol{ extalpha_{G}}$-mixture locally smooth and $oldsymbol{ extalpha_{H}}$-mixture locally Hessian smooth potentials, and proves convergence in KL divergence with polynomial complexity.

Findings

Convergence in KL divergence with polynomial iteration complexity.

Improved convergence rates for 1-smooth and Hessian smooth potentials.

Theoretical properties of $p$-generalized Gaussian smoothing and Wasserstein convergence.

Abstract

The purpose of this paper is to examine the sampling problem through Euler discretization, where the potential function is assumed to be a mixture of locally smooth distributions and weakly dissipative. We introduce $\alpha_{G}$-mixture locally smooth and $\alpha_{H}$-mixture locally Hessian smooth, which are novel and typically satisfied with a mixture of distributions. Under our conditions, we prove the convergence in Kullback-Leibler (KL) divergence with the number of iterations to reach $\epsilon$-neighborhood of a target distribution in only polynomial dependence on the dimension. The convergence rate is improved when the potential is $1$-smooth and $\alpha_{H}$-mixture locally Hessian smooth. Our result for the non-strongly convex outside the ball of radius $R$ is obtained by convexifying the non-convex domains. In addition, we provide some nice theoretical properties of $p$-generalized Gaussian smoothing and prove the convergence in the $L_{\beta}$-Wasserstein distance for stochastic gradients in a general setting.