Simultaneous Variable and Covariance Selection with the Multivariate Spike-and-Slab Lasso

TL;DR

This paper introduces a Bayesian method for joint variable and covariance selection in multivariate regression, utilizing continuous spike-and-slab priors and an ECM algorithm to improve model estimation and interpretability.

Contribution

It develops a novel ECM algorithm for efficient modal estimation in multivariate regression with spike-and-slab priors, enhancing variable and covariance selection without stochastic search.

Findings

Outperforms regularization methods on simulated data

Effectively filters negligible coefficients and covariances

Demonstrated on observational study data regarding high school football effects

Abstract

We propose a Bayesian procedure for simultaneous variable and covariance selection using continuous spike-and-slab priors in multivariate linear regression models where q possibly correlated responses are regressed onto p predictors. Rather than relying on a stochastic search through the high-dimensional model space, we develop an ECM algorithm similar to the EMVS procedure of Rockova & George (2014) targeting modal estimates of the matrix of regression coefficients and residual precision matrix. Varying the scale of the continuous spike densities facilitates dynamic posterior exploration and allows us to filter out negligible regression coefficients and partial covariances gradually. Our method is seen to substantially outperform regularization competitors on simulated data. We demonstrate our method with a re-examination of data from a recent observational study of the effect of playing high school football on several later-life cognition, psychological, and socio-economic outcomes.