Fast Bayesian Feature Selection for High Dimensional Linear Regression in Genomics via the Ising Approximation

TL;DR

This paper introduces the Bayesian Ising Approximation (BIA), a fast method for feature selection in high-dimensional linear regression, especially useful for genomic data with thousands of features.

Contribution

The paper presents the BIA, a novel approach that uses an Ising model approximation to efficiently compute feature relevance probabilities in high-dimensional regression.

Findings

BIA accurately estimates feature relevance in simulations.

BIA significantly reduces computation time compared to traditional methods.

Applied to gene expression data with 30,000 features, BIA effectively identified relevant features.

Abstract

Feature selection, identifying a subset of variables that are relevant for predicting a response, is an important and challenging component of many methods in statistics and machine learning. Feature selection is especially difficult and computationally intensive when the number of variables approaches or exceeds the number of samples, as is often the case for many genomic datasets. Here, we introduce a new approach -- the Bayesian Ising Approximation (BIA) -- to rapidly calculate posterior probabilities for feature relevance in L2 penalized linear regression. In the regime where the regression problem is strongly regularized by the prior, we show that computing the marginal posterior probabilities for features is equivalent to computing the magnetizations of an Ising model. Using a mean field approximation, we show it is possible to rapidly compute the feature selection path described by the posterior probabilities as a function of the L2 penalty. We present simulations and analytical results illustrating the accuracy of the BIA on some simple regression problems. Finally, we demonstrate the applicability of the BIA to high dimensional regression by analyzing a gene expression dataset with nearly 30,000 features.