Sparse high-dimensional linear regression with a partitioned empirical Bayes ECM algorithm

TL;DR

This paper introduces PROBE, a computationally efficient Bayesian method for sparse high-dimensional linear regression that uses empirical Bayes estimates and a partitioned ECM algorithm, demonstrated through simulations and cancer data analysis.

Contribution

It presents a novel PROBE algorithm combining empirical Bayes and partitioned ECM for efficient sparse regression with minimal prior assumptions.

Findings

PROBE outperforms comparable methods in simulations.

PROBE effectively analyzes cancer cell drug response data.

The R package probe implements the proposed method.

Abstract

Bayesian variable selection methods are powerful techniques for fitting and inferring on sparse high-dimensional linear regression models. However, many are computationally intensive or require restrictive prior distributions on model parameters. In this paper, we proposed a computationally efficient and powerful Bayesian approach for sparse high-dimensional linear regression. Minimal prior assumptions on the parameters are required through the use of plug-in empirical Bayes estimates of hyperparameters. Efficient maximum a posteriori (MAP) estimation is completed through a Parameter-Expanded Expectation-Conditional-Maximization (PX-ECM) algorithm. The PX-ECM results in a robust computationally efficient coordinate-wise optimization which -- when updating the coefficient for a particular predictor -- adjusts for the impact of other predictor variables. The completion of the E-step uses an approach motivated by the popular two-group approach to multiple testing. The result is a PaRtitiOned empirical Bayes Ecm (PROBE) algorithm applied to sparse high-dimensional linear regression, which can be completed using one-at-a-time or all-at-once type optimization. We compare the empirical properties of PROBE to comparable approaches with numerous simulation studies and analyses of cancer cell drug responses. The proposed approach is implemented in the R package probe.