Bayes Variable Selection in Semiparametric Linear Models

TL;DR

This paper extends Bayesian variable selection methods to semiparametric linear models with unknown residual densities, using Dirichlet process mixtures and demonstrating consistency in high-dimensional settings.

Contribution

It introduces a semiparametric $g$-prior for models with unknown residual densities and develops a stochastic search algorithm for efficient posterior computation.

Findings

Posterior computation via stochastic search is straightforward.

Bayes factor and variable selection are consistent under various priors.

Models with diverging dimensions less than sample size are effectively handled.

Abstract

There is a rich literature proposing methods and establishing asymptotic properties of Bayesian variable selection methods for parametric models, with a particular focus on the normal linear regression model and an increasing emphasis on settings in which the number of candidate predictors ($p$) diverges with sample size ($n$). Our focus is on generalizing methods and asymptotic theory established for mixtures of $g$-priors to semiparametric linear regression models having unknown residual densities. Using a Dirichlet process location mixture for the residual density, we propose a semiparametric $g$-prior which incorporates an unknown matrix of cluster allocation indicators. For this class of priors, posterior computation can proceed via a straightforward stochastic search variable selection algorithm. In addition, Bayes factor and variable selection consistency is shown to result under various cases including proper and improper priors on $g$ and $p>n$, with the models under comparison restricted to have model dimensions diverging at a rate less than $n$.