A Metropolized adaptive subspace algorithm for high-dimensional Bayesian variable selection

TL;DR

The paper introduces MAdaSub, an adaptive MCMC algorithm for efficient Bayesian variable selection in high-dimensional settings, with proven convergence and demonstrated effectiveness on large datasets.

Contribution

It presents a novel Metropolized adaptive subspace algorithm that converges to posterior inclusion probabilities and extends to parallel implementations.

Findings

Effective in high-dimensional problems with over 20,000 covariates

Proven ergodicity of the adaptive algorithm

Demonstrated success on simulated and real data examples

Abstract

A simple and efficient adaptive Markov Chain Monte Carlo (MCMC) method, called the Metropolized Adaptive Subspace (MAdaSub) algorithm, is proposed for sampling from high-dimensional posterior model distributions in Bayesian variable selection. The MAdaSub algorithm is based on an independent Metropolis-Hastings sampler, where the individual proposal probabilities of the explanatory variables are updated after each iteration using a form of Bayesian adaptive learning, in a way that they finally converge to the respective covariates' posterior inclusion probabilities. We prove the ergodicity of the algorithm and present a parallel version of MAdaSub with an adaptation scheme for the proposal probabilities based on the combination of information from multiple chains. The effectiveness of the algorithm is demonstrated via various simulated and real data examples, including a high-dimensional problem with more than 20,000 covariates.