Manifold lifting: scaling MCMC to the vanishing noise regime

TL;DR

This paper introduces a manifold lifting approach combined with constrained Hamiltonian Monte Carlo to efficiently sample from distributions concentrated near low-dimensional structures, maintaining performance as noise vanishes.

Contribution

It proposes a novel manifold lifting strategy and a constrained HMC method that together improve sampling efficiency in low-noise regimes, unlike existing approaches.

Findings

Sampling efficiency remains stable as distribution concentrates near low-dimensional structures.

The method outperforms competing approaches in numerical experiments.

It effectively explores manifolds in high-dimensional spaces with vanishing noise.

Abstract

Standard Markov chain Monte Carlo methods struggle to explore distributions that are concentrated in the neighbourhood of low-dimensional structures. These pathologies naturally occur in a number of situations. For example, they are common to Bayesian inverse problem modelling and Bayesian neural networks, when observational data are highly informative, or when a subset of the statistical parameters of interest are non-identifiable. In this paper, we propose a strategy that transforms the original sampling problem into the task of exploring a distribution supported on a manifold embedded in a higher dimensional space; in contrast to the original posterior this lifted distribution remains diffuse in the vanishing noise limit. We employ a constrained Hamiltonian Monte Carlo method which exploits the manifold geometry of this lifted distribution, to perform efficient approximate inference. We demonstrate in several numerical experiments that, contrarily to competing approaches, the sampling efficiency of our proposed methodology does not degenerate as the target distribution to be explored concentrates near low dimensional structures.