A Scalable Empirical Bayes Approach to Variable Selection in Generalized Linear Models

TL;DR

This paper introduces a scalable empirical Bayes method for variable selection in generalized linear models, efficiently handling large numbers of variables by modeling coefficients as a three-component mixture.

Contribution

It develops a computationally efficient algorithm that scales to high-dimensional data, estimating model parameters with a small, fixed number of parameters regardless of variable count.

Findings

Algorithm scales to very large variable sets

Faster convergence than fully Bayesian methods

Effective in identifying relevant variables

Abstract

A new empirical Bayes approach to variable selection in the context of generalized linear models is developed. The proposed algorithm scales to situations in which the number of putative explanatory variables is very large, possibly much larger than the number of responses. The coefficients in the linear predictor are modeled as a three-component mixture allowing the explanatory variables to have a random positive effect on the response, a random negative effect, or no effect. A key assumption is that only a small (but unknown) fraction of the candidate variables have a non-zero effect. This assumption, in addition to treating the coefficients as random effects facilitates an approach that is computationally efficient. In particular, the number of parameters that have to be estimated is small, and remains constant regardless of the number of explanatory variables. The model parameters are estimated using a Generalized Alternating Maximization algorithm which is scalable, and leads to significantly faster convergence compared with simulation-based fully Bayesian methods.