Efficient Mirror-type Kernels for the Metropolis-Hastings Algorithm

TL;DR

This paper introduces a novel Mirror-type kernel for the Metropolis-Hastings algorithm that enhances proposal efficiency by combining strengths of existing methods, significantly improving sampling performance in complex models.

Contribution

The paper proposes a new Mirror-type MH kernel that integrates Mirror move with MALA, offering improved efficiency and proposal quality, especially in high-dimensional Bayesian models.

Findings

MirrorMALA outperforms traditional MH kernels in efficiency.

Efficiency gains of 2-20 times over HMC or NUTS in Bayesian GLMMs.

Proposes a whitening transformation for high-dimensional variables.

Abstract

We propose a new Metropolis-Hastings (MH) kernel by introducing the Mirror move into the Metropolis adjusted Langevin algorithm (MALA). This new kernel uses the strength of one kernel to overcome the shortcoming of the other, and generates proposals that are distant from the current position, but still within the high-density region of the target distribution. The resulting algorithm can be much more efficient than both Mirror and MALA, while stays comparable in terms of computational cost. We demonstrate the advantages of the MirrorMALA kernel using a variety of one-dimensional and multi-dimensional examples. The Mirror and MirrorMALA are both special cases of the Mirror-type kernels, a new suite of efficient MH proposals. We use the Mirror-type kernels, together with a novel method of doing the whitening transformation on high-dimensional random variables, which was inspired by Tan and Nott, to analyse the Bayesian generalized linear mixed models (GLMMs), and obtain the per-time-unit efficiency that is 2--20 times higher than the HMC or NUTS algorithm.