Fast Non-Log-Concave Sampling under Nonconvex Equality and Inequality Constraints with Landing

TL;DR

This paper introduces OLLA, a new Langevin-based sampling method that efficiently handles both equality and inequality constraints without projections, providing exponential convergence guarantees and improved empirical performance.

Contribution

The paper presents OLLA, a novel Langevin dynamics framework that accommodates mixed constraints and offers rigorous convergence analysis without costly projections.

Findings

OLLA converges exponentially fast in $W_2$ distance.

OLLA outperforms projection-based algorithms in experiments.

OLLA has favorable computational cost and empirical mixing.

Abstract

Sampling from constrained statistical distributions is a fundamental task in various fields including Bayesian statistics, computational chemistry, and statistical physics. This article considers the cases where the constrained distribution is described by an unconstrained density, as well as additional equality and/or inequality constraints, which often make the constraint set nonconvex. Existing methods for nonconvex constraint set $\Sigma \subset \mathbb{R}^d$ defined by equality or inequality constraints commonly rely on costly projection steps. Moreover, they cannot handle equality and inequality constraints simultaneously as each method only specialized in one case. In addition, rigorous and quantitative convergence guarantee is often lacking. In this paper, we introduce Overdamped Langevin with LAnding (OLLA), a new framework that can design overdamped Langevin dynamics accommodating both equality and inequality constraints. The proposed dynamics also deterministically corrects trajectories along the normal direction of the constraint surface, thus obviating the need for explicit projections. We show that, under suitable regularity conditions on the target density and $\Sigma$, OLLA converges exponentially fast in $W_2$ distance to the constrained target density $\rho_\Sigma(x) \propto \exp(-f(x))d\sigma_\Sigma$. Lastly, through experiments, we demonstrate the efficiency of OLLA compared to projection-based constrained Langevin algorithms and their slack variable variants, highlighting its favorable computational cost and reasonable empirical mixing.