Jump-Diffusion Langevin Dynamics for Multimodal Posterior Sampling

TL;DR

This paper introduces a hybrid Jump-Diffusion Langevin Dynamics method for sampling from complex, multimodal posterior distributions, demonstrating improved convergence over traditional methods in high-dimensional Bayesian inference tasks.

Contribution

The paper proposes a novel hybrid sampling algorithm combining Metropolis jumps with Langevin Dynamics to better explore multimodal posteriors, outperforming existing methods.

Findings

Hybrid method outperforms pure gradient-based schemes.

Careful calibration of jumps enhances mixing efficiency.

Effective in high-dimensional, multimodal Bayesian problems.

Abstract

Bayesian methods of sampling from a posterior distribution are becoming increasingly popular due to their ability to precisely display the uncertainty of a model fit. Classical methods based on iterative random sampling and posterior evaluation such as Metropolis-Hastings are known to have desirable long run mixing properties, however are slow to converge. Gradient based methods, such as Langevin Dynamics (and its stochastic gradient counterpart) exhibit favorable dimension-dependence and fast mixing times for log-concave, and "close" to log-concave distributions, however also have long escape times from local minimizers. Many contemporary applications such as Bayesian Neural Networks are both high-dimensional and highly multimodal. In this paper we investigate the performance of a hybrid Metropolis and Langevin sampling method akin to Jump Diffusion on a range of synthetic and real data, indicating that careful calibration of mixing sampling jumps with gradient based chains significantly outperforms both pure gradient-based or sampling based schemes.