A Two Stage Adaptive Metropolis Algorithm

TL;DR

This paper introduces a two-stage adaptive Metropolis algorithm that combines adaptive proposals with a two-stage acceptance process to improve efficiency and convergence in high-dimensional Bayesian posterior sampling with expensive likelihoods.

Contribution

It presents a novel two-stage adaptive Metropolis algorithm that enhances sampling efficiency and convergence for high-dimensional problems with costly likelihood evaluations.

Findings

Faster convergence and good mixing properties demonstrated.

Significant reduction in computational cost compared to existing methods.

Maintains ergodicity despite dependence on previous states.

Abstract

We propose a new sampling algorithm combining two quite powerful ideas in the Markov chain Monte Carlo literature -- adaptive Metropolis sampler and two-stage Metropolis-Hastings sampler. The proposed sampling method will be particularly very useful for high-dimensional posterior sampling in Bayesian models with expensive likelihoods. In the first stage of the proposed algorithm, an adaptive proposal is used based on the previously sampled states and the corresponding acceptance probability is computed based on an approximated inexpensive target density. The true expensive target density is evaluated while computing the second stage acceptance probability only if the proposal is accepted in the first stage. The adaptive nature of the algorithm guarantees faster convergence of the chain and very good mixing properties. On the other hand, the two-stage approach helps in rejecting the bad proposals in the inexpensive first stage, making the algorithm computationally efficient. As the proposals are dependent on the previous states the chain loses its Markov property, but we prove that it retains the desired ergodicity property. The performance of the proposed algorithm is compared with the existing algorithms in two simulated and two real data examples.