A Graphical View of Bayesian Variable Selection

TL;DR

This paper explores Bayesian variable selection through a graphical Ising model perspective, revealing advantages in computational efficiency and extending to nonparametric models, with a focus on prior choices and algorithmic connections.

Contribution

It demonstrates that Bayesian variable selection can be modeled as an Ising graphical model without informative priors, and studies the impact of different shrinkage priors on selection performance.

Findings

Optimal priors favor small shrinkage for variable selection.

Graphical Ising model approach facilitates efficient algorithms.

Shrinkage parameters act as tempering in Ising models.

Abstract

In recent years, Ising prior with the network information for the "in" or "out" binary random variable in Bayesian variable selections has received more and more attentions. In this paper, we discover that even without the informative prior a Bayesian variable selection problem itself can be considered as a complete graph and described by a Ising model with random interactions. There are many advantages of treating variable selection as a graphical model, such as it is easy to employ the single site updating as well as the cluster updating algorithm, suitable for problems with small sample size and larger variable number, easy to extend to nonparametric regression models and incorporate graphical prior information and so on. In a Bayesian variable selection Ising model the interactions are determined by the linear model coefficients, so we systematically study the performance of different scale normal mixture priors for the model coefficients by adopting the global-local shrinkage strategy. Our results prove that the best prior of the model coefficients in terms of variable selection should maintain substantial weight on small shrinkage instead of large shrinkage. We also discuss the connection between the tempering algorithms for Ising models and the global-local shrinkage approach, showing that the shrinkage parameter plays a tempering role. The methods are illustrated with simulated and real data.