On the Choice of Model Space Priors and Multiplicity Control in Bayesian Variable Selection: An Application to Streaming Logistic Regression

TL;DR

This paper investigates how different model space priors affect Bayesian variable selection in streaming logistic regression, highlighting the impact on sparsity control, and introduces a scalable approximation to the matryoshka doll prior.

Contribution

It compares popular Beta-Binomial priors with a new scalable approximation to the MD prior, providing insights into their effects in streaming data scenarios.

Findings

No single prior dominates across all scenarios.

The MD prior offers an intermediate sparsity control option.

Different priors influence posterior inclusion probabilities and coefficient estimates.

Abstract

Bayesian variable selection (BVS) depends critically on the specification of a prior distribution over the model space, particularly for controlling sparsity and multiplicity. This paper examines the practical consequences of different model space priors for BVS in logistic regression, with an emphasis on streaming data settings. We review some popular and well-known Beta--Binomial priors alongside the recently proposed matryoshka doll (MD) prior. We introduce a simple approximation to the MD prior that yields independent inclusion indicators and is convenient for scalable inference. Using BIC-based approximations to marginal likelihoods, we compare the effect of different model space priors on posterior inclusion probabilities and coefficient estimation at intermediate and final stages of the data stream via simulation studies. Overall, the results indicate that no single model space prior uniformly dominates across scenarios, and that the recently proposed MD prior provides a useful additional option that occupies an intermediate position between commonly used Beta--Binomial priors with differing degrees of sparsity.