Sampling from Constrained Gibbs Measures: with Applications to High-Dimensional Bayesian Inference

TL;DR

This paper develops a method for sampling from constrained low-temperature Gibbs distributions, especially when some coordinates are on the boundary, with applications to high-dimensional Bayesian inference.

Contribution

It introduces a novel analysis of Gibbs measures with boundary constraints, providing non-asymptotic sampling guarantees in a pre-asymptotic regime.

Findings

Distribution concentrates near the mode in the pre-asymptotic regime

Distribution is a bounded perturbation of a product measure

Spectral gap analysis yields sampling guarantees

Abstract

This paper considers a non-standard problem of generating samples from a low-temperature Gibbs distribution with \emph{constrained} support, when some of the coordinates of the mode lie on the boundary. These coordinates are referred to as the non-regular part of the model. We show that in a ``pre-asymptotic'' regime in which the limiting Laplace approximation is not yet valid, the low-temperature Gibbs distribution concentrates on a neighborhood of its mode. Within this region, the distribution is a bounded perturbation of a product measure: a strongly log-concave distribution in the regular part and a one-dimensional exponential-type distribution in each coordinate of the non-regular part. Leveraging this structure, we provide a non-asymptotic sampling guarantee by analyzing the spectral gap of Langevin dynamics. Key examples of low-temperature Gibbs distributions include Bayesian posteriors, and we demonstrate our results on three canonical examples: a high-dimensional logistic regression model, a Poisson linear model, and a Gaussian mixture model.