Variable Fusion and Selection via a Spike-and-Slab Approach with Nonlocal Priors

TL;DR

This paper introduces a Bayesian method combining variable fusion and selection in linear regression using spike-and-slab priors within a model averaging framework.

Contribution

It develops a novel prior and Gibbs sampling approach for simultaneous variable fusion and selection, enhancing model interpretability and selection accuracy.

Findings

The method effectively performs variable fusion and selection in simulations.

The proposed priors improve model selection properties.

Empirical studies demonstrate the method's practical utility.

Abstract

Variable fusion in linear regression models is a statistical method that identifies covariates making similar contributions to the response variable and imposes the same coefficient values on them. Many methods for variable fusion also incorporate variable selection for practical reasons. In this paper, within the Bayesian model averaging (BMA) framework, we propose a spike-and-slab-based Bayesian method that performs both variable fusion and selection. This is challenging in the BMA framework because one must construct a discrete model space that accommodates both selection and fusion and assign suitable priors over that space. In the proposed method, we present a way to explore a model space for variable fusion and selection based on Gibbs sampling by devising a prior distribution for latent variables representing the model. Furthermore, among non-local priors with superior model selection properties, we construct a prior tailored for variable fusion and use it as the slab distribution. We examine the effectiveness of the proposed method through theoretical and empirical studies.