Adaptive Metropolis-Hastings Sampling using Reversible Dependent Mixture Proposals

TL;DR

This paper introduces an adaptive Metropolis-Hastings sampler using reversible proposals based on mixtures of multivariate t densities, enabling efficient exploration of complex target distributions.

Contribution

It develops a general adaptive sampling method with reversible mixture proposals, including a variational algorithm for automatic component selection, and proves convergence without Diminishing Adaptation.

Findings

Efficient exploration of complex distributions demonstrated on simulated data.

Automatic mixture component determination via variational approximation.

The method converges under reasonable conditions without Diminishing Adaptation.

Abstract

This article develops a general-purpose adaptive sampler that approximates the target density by a mixture of multivariate t densities. The adaptive sampler is based on reversible proposal distributions each of which has the mixture of multivariate t densities as its invariant density. The reversible proposals consist of a combination of independent and correlated steps that allow the sampler to traverse the parameter space efficiently as well as allowing the sampler to keep moving and locally exploring the parameter space. We employ a two-chain approach, in which a trial chain is used to adapt the proposal densities used in the main chain. Convergence of the main chain and a strong law of large numbers are proved under reasonable conditions, and without imposing a Diminishing Adaptation condition. The mixtures of multivariate t densities are fitted by an efficient Variational Approximation algorithm in which the number of components is determined automatically. The performance of the sampler is evaluated using simulated and real examples. Our autocorrelated framework is quite general and can handle mixtures other than multivariate t.