On Single Variable Transformation Approach to Markov Chain Monte Carlo

TL;DR

This paper introduces TMCMC, a transformation-based MCMC method that updates all coordinates simultaneously, achieving better scalability and faster convergence in high-dimensional problems compared to traditional RWMH.

Contribution

The paper proves geometric ergodicity for TMCMC with sub-exponential targets and identifies the optimal acceptance rate, demonstrating improved robustness over RWMH.

Findings

TMCMC achieves geometric ergodicity for certain targets.

Optimal acceptance rate for TMCMC is approximately 0.439.

TMCMC converges faster to stationarity than RWMH, especially in high dimensions.

Abstract

Random Walk Metropolis Hastings (RWMH) algorithm, is quite inefficient in high dimensions because of its abysmally slow acceptance rate. The slow acceptance rate results from the fact that RWMH separately updates each coordinate of the chain at every step. Dutta and Bhattacharya (2013) proposed a new technique called Transformation based Markov Chain Monte Carlo (TMCMC) aimed at overcoming these problems. This method updates all co-ordinates at a time- ensuring stable acceptance in all dimensions. We have shown here that geometric ergodicity is achieved for sub-exponential targets for two versions of TMCMC- the additive and the additive-multiplicative hybrid TMCMC schemes. Also, we obtain the optimal scaling by maximizing the diffusion speed of the limiting time-scaled diffusion process for TMCMC. We show that the optimal acceptance rate is 0.439 for TMCMC which is almost twice as large as RWMH (0.234). We observe that convergence to stationarity for TMCMC is faster than RWMH but the mixing property in RWMH is relatively better. However TMCMC is more robust with respect to scaling and dimensionality. This is attested by simulation runs on Gaussian and nearest neighbor models.