Bayesian Model Selection with Graph Structured Sparsity

TL;DR

This paper introduces a scalable Bayesian model selection framework using a spike-and-slab Laplacian prior and effective resistance, applicable to high-dimensional data and complex models like biclustering.

Contribution

It presents a novel EM-type algorithm leveraging graph structures for efficient Bayesian model selection, extending to complex models and outperforming frequentist methods.

Findings

Algorithm is scalable to large datasets

Framework recovers existing EMVS as a special case

Demonstrates superior performance in simulations and real data

Abstract

We propose a general algorithmic framework for Bayesian model selection. A spike-and-slab Laplacian prior is introduced to model the underlying structural assumption. Using the notion of effective resistance, we derive an EM-type algorithm with closed-form iterations to efficiently explore possible candidates for Bayesian model selection. The deterministic nature of the proposed algorithm makes it more scalable to large-scale and high-dimensional data sets compared with existing stochastic search algorithms. When applied to sparse linear regression, our framework recovers the EMVS algorithm [Rockova and George, 2014] as a special case. We also discuss extensions of our framework using tools from graph algebra to incorporate complex Bayesian models such as biclustering and submatrix localization. Extensive simulation studies and real data applications are conducted to demonstrate the superior performance of our methods over its frequentist competitors such as $\ell_0$ or $\ell_1$ penalization.