Informed proposals for local MCMC in discrete spaces
Giacomo Zanella

TL;DR
This paper introduces a new class of locally-balanced MCMC proposals that incorporate local information, achieving significant efficiency improvements in high-dimensional discrete spaces, and extends gradient-based MCMC ideas to discrete settings.
Contribution
It proposes a framework for designing informed MCMC proposals, characterizes optimal proposals as locally-balanced, and demonstrates their superior efficiency in discrete high-dimensional problems.
Findings
Locally-balanced proposals are Peskun-optimal in high dimensions.
Algorithms show orders of magnitude efficiency improvements.
Applications include Bayesian record linkage.
Abstract
There is a lack of methodological results to design efficient Markov chain Monte Carlo (MCMC) algorithms for statistical models with discrete-valued high-dimensional parameters. Motivated by this consideration, we propose a simple framework for the design of informed MCMC proposals (i.e. Metropolis-Hastings proposal distributions that appropriately incorporate local information about the target) which is naturally applicable to both discrete and continuous spaces. We explicitly characterize the class of optimal proposal distributions under this framework, which we refer to as locally-balanced proposals, and prove their Peskun-optimality in high-dimensional regimes. The resulting algorithms are straightforward to implement in discrete spaces and provide orders of magnitude improvements in efficiency compared to alternative MCMC schemes, including discrete versions of Hamiltonian Monte…
Peer Reviews
No public reviews on file for this paper yet. If you reviewed it on a platform where reviews are public (OpenReview, ICLR, NeurIPS, ICML), you can paste yours below so the community can read it here.
Code & Models
Videos
No videos yet. Explain this paper in a talk, walkthrough, or lecture? Add one.
Taxonomy
TopicsMarkov Chains and Monte Carlo Methods · Bayesian Methods and Mixture Models · Statistical Methods and Bayesian Inference
