Tractable Bayesian variable selection: beyond normality

TL;DR

This paper introduces a flexible, tractable extension to Bayesian variable selection models that accounts for skewness and heavy tails, improving robustness and inference under model misspecification.

Contribution

It proposes a simple extension of the normal model that maintains tractability while allowing for skewness and heavy tails, and analyzes its asymptotic properties.

Findings

Misspecified Bayes factors can still be consistent and induce sparsity.

Detection rates are affected by an exponential factor, reducing sensitivity.

Inferring the error distribution from data improves inference.

Abstract

Bayesian variable selection often assumes normality, but the effects of model misspecification are not sufficiently understood. There are sound reasons behind this assumption, particularly for large $p$: ease of interpretation, analytical and computational convenience. More flexible frameworks exist, including semi- or non-parametric models, often at the cost of some tractability. We propose a simple extension of the Normal model that allows for skewness and thicker-than-normal tails but preserves tractability. It leads to easy interpretation and a log-concave likelihood that facilitates optimization and integration. We characterize asymptotically parameter estimation and Bayes factor rates, in particular studying the effects of model misspecification. Under suitable conditions misspecified Bayes factors are consistent and induce sparsity at the same asymptotic rates than under the correct model. However, the rates to detect signal are altered by an exponential factor, often resulting in a loss of sensitivity. These deficiencies can be ameliorated by inferring the error distribution from the data, a simple strategy that can improve inference substantially. Our work focuses on the likelihood and can thus be combined with any likelihood penalty or prior, but here we focus on non-local priors to induce extra sparsity and ameliorate finite-sample effects caused by misspecification. Our results highlight the practical importance of focusing on the likelihood rather than solely on the prior, when it comes to Bayesian variable selection. The methodology is available in R package `mombf'.