Consistent Group selection using Global-local prior in High dimensional setup

TL;DR

This paper introduces a Bayesian global-local shrinkage prior approach for high-dimensional group variable selection, providing theoretical guarantees and practical thresholding rules that perform comparably to spike-and-slab methods.

Contribution

It proposes a modified global-local prior with a novel thresholding rule for consistent group selection, including an empirical Bayes method for unknown sparsity levels.

Findings

The method is oracle when the proportion of active groups is known.

The empirical Bayes estimate of the sparsity parameter performs well.

Simulation results show comparable performance to existing methods.

Abstract

We consider the problem of model selection when grouping structure is inherent within the regressors. Using a Bayesian approach, we model the mean vector by a one-group global-local shrinkage prior belonging to a broad class of such priors that includes the horseshoe prior. In the context of variable selection, this class of priors was studied by Tang et al. (2018). A modified form of the usual class of global-local shrinkage priors with polynomial tail on the group regression coefficients is proposed. The resulting threshold rule selects the active group if within a group, the ratio of the $L_2$ norm of the posterior mean of its group coefficient to that of the corresponding ordinary least square group estimate is greater than a half. In the theoretical part of this article, we have used the global shrinkage parameter either as a tuning one or an empirical Bayes estimate of it depending on the knowledge regarding the underlying sparsity of the model. When the proportion of active groups is known, using $\tau$ as a tuning parameter, we have proved that our method is oracle. In case this proportion is unknown, we propose an empirical Bayes estimate of $\tau$. Even if this empirical Bayes estimate is used, then also our half-thresholding rule captures the truly important groups and obtains optimal estimation rate of the group coefficients simultaneously. Though our theoretical works rely on a special form of the design matrix, for general design matrices also, our simulation results show that the half-thresholding rule yields results similar to that of Yang and Narisetty (2020). As a consequence of this, in a high dimensional sparse group selection problem, instead of using the so-called `gold standard' spike and slab prior, one can use the one-group global-local shrinkage priors with polynomial tail to obtain similar results.