Tail-adaptive Bayesian shrinkage

TL;DR

This paper introduces a tail-adaptive Bayesian shrinkage method for high-dimensional sparse regression that adjusts the prior's tail heaviness based on the sparsity level, improving performance across diverse regimes.

Contribution

The paper proposes the global-local-tail (GLT) Gaussian mixture prior with adaptive tail behavior, enhancing robustness in sparse estimation across various sparsity levels.

Findings

GLT prior achieves minimax optimal posterior contraction rates.

Varying tail rule outperforms fixed tail rules like Horseshoe.

Method effective in real data and simulation scenarios.

Abstract

Robust Bayesian methods for high-dimensional regression problems under diverse sparse regimes are studied. Traditional shrinkage priors are primarily designed to detect a handful of signals from tens of thousands of predictors in the so-called ultra-sparsity domain. However, they may not perform desirably when the degree of sparsity is moderate. In this paper, we propose a robust sparse estimation method under diverse sparsity regimes, which has a tail-adaptive shrinkage property. In this property, the tail-heaviness of the prior adjusts adaptively, becoming larger or smaller as the sparsity level increases or decreases, respectively, to accommodate more or fewer signals, a posteriori. We propose a global-local-tail (GLT) Gaussian mixture distribution that ensures this property. We examine the role of the tail-index of the prior in relation to the underlying sparsity level and demonstrate that the GLT posterior contracts at the minimax optimal rate for sparse normal mean models. We apply both the GLT prior and the Horseshoe prior to a real data problem and simulation examples. Our findings indicate that the varying tail rule based on the GLT prior offers advantages over a fixed tail rule based on the Horseshoe prior in diverse sparsity regimes.