Taking Bigger Metropolis Steps by Dragging Fast Variables

TL;DR

This paper introduces a modified Metropolis-Hastings algorithm that leverages fast variables to take larger sampling steps, significantly improving efficiency in high-dimensional problems with mixed variable speeds.

Contribution

The paper presents a novel method for faster Markov chain sampling by exploiting fast variables to increase step size without sacrificing accuracy.

Findings

Sampling efficiency approaches that of marginal distribution updates for slow variables.

Method significantly reduces computational cost in models with fast and slow variables.

Empirical results demonstrate improved convergence speed.

Abstract

I show how Markov chain sampling with the Metropolis-Hastings algorithm can be modified so as to take bigger steps when the distribution being sampled from has the characteristic that its density can be quickly recomputed for a new point if this point differs from a previous point only with respect to a subset of 'fast' variables. I show empirically that when using this method, the efficiency of sampling for the remaining 'slow' variables can approach what would be possible using Metropolis updates based on the marginal distribution for the slow variables.