Spike-and-Slab Meets LASSO: A Review of the Spike-and-Slab LASSO

TL;DR

This paper surveys the spike-and-slab LASSO methodology, highlighting its properties, extensions, theoretical aspects, and applications in high-dimensional statistical modeling.

Contribution

It provides a comprehensive review of the spike-and-slab LASSO, including its properties, extensions, theoretical foundations, and practical applications.

Findings

SSL priors are computationally tractable in high dimensions

The methodology extends to various models like generalized linear models and factor analysis

Illustrations on simulated and real datasets demonstrate effectiveness

Abstract

High-dimensional data sets have become ubiquitous in the past few decades, often with many more covariates than observations. In the frequentist setting, penalized likelihood methods are the most popular approach for variable selection and estimation in high-dimensional data. In the Bayesian framework, spike-and-slab methods are commonly used as probabilistic constructs for high-dimensional modeling. Within the context of linear regression, Rockova and George (2018) introduced the spike-and-slab LASSO (SSL), an approach based on a prior which provides a continuum between the penalized likelihood LASSO and the Bayesian point-mass spike-and-slab formulations. Since its inception, the spike-and-slab LASSO has been extended to a variety of contexts, including generalized linear models, factor analysis, graphical models, and nonparametric regression. The goal of this paper is to survey the landscape surrounding spike-and-slab LASSO methodology. First we elucidate the attractive properties and the computational tractability of SSL priors in high dimensions. We then review methodological developments of the SSL and outline several theoretical developments. We illustrate the methodology on both simulated and real datasets.