Adaptive MC^3 and Gibbs algorithms for Bayesian Model Averaging in Linear Regression Models

TL;DR

This paper introduces adaptive versions of MC^3 and Gibbs samplers for Bayesian Model Averaging in linear regression, improving efficiency by reducing redundant variable selection, especially in high-dimensional settings.

Contribution

The paper proposes adaptive MC^3 and Gibbs algorithms that dynamically adjust variable selection probabilities to enhance computational efficiency in BMA.

Findings

Adaptive samplers outperform traditional methods in large variable sets.

Efficiency gains demonstrated on real and simulated datasets.

Reduces computational cost by focusing on relevant variables.

Abstract

The MC$^3$ (Madigan and York, 1995) and Gibbs (George and McCulloch, 1997) samplers are the most widely implemented algorithms for Bayesian Model Averaging (BMA) in linear regression models. These samplers draw a variable at random in each iteration using uniform selection probabilities and then propose to update that variable. This may be computationally inefficient if the number of variables is large and many variables are redundant. In this work, we introduce adaptive versions of these samplers that retain their simplicity in implementation and reduce the selection probabilities of the many redundant variables. The improvements in efficiency for the adaptive samplers are illustrated in real and simulated datasets.