Adaptive Metropolis Algorithm Using Variational Bayesian Adaptive Kalman Filter

TL;DR

This paper introduces the VBAM algorithm, an adaptive MCMC method that uses a variational Bayesian adaptive Kalman filter to automatically tune proposal distributions, improving efficiency in complex statistical analyses.

Contribution

The paper develops the VBAM algorithm, integrating variational Bayesian adaptive Kalman filtering into MCMC to enhance proposal adaptation and proves its strong convergence properties.

Findings

Proven strong law of large numbers for VBAM.

Empirical convergence demonstrated on simulated data.

Effective adaptation shown on real data examples.

Abstract

Markov chain Monte Carlo (MCMC) methods are powerful computational tools for analysis of complex statistical problems. However, their computational efficiency is highly dependent on the chosen proposal distribution, which is generally difficult to find. One way to solve this problem is to use adaptive MCMC algorithms which automatically tune the statistics of a proposal distribution during the MCMC run. A new adaptive MCMC algorithm, called the variational Bayesian adaptive Metropolis (VBAM) algorithm, is developed. The VBAM algorithm updates the proposal covariance matrix using the variational Bayesian adaptive Kalman filter (VB-AKF). A strong law of large numbers for the VBAM algorithm is proven. The empirical convergence results for three simulated examples and for two real data examples are also provided.