High-dimensional posterior consistency for hierarchical non-local priors in regression

TL;DR

This paper proves high-dimensional posterior consistency for Bayesian linear regression with hierarchical non-local priors, demonstrating the effectiveness of a fully Bayesian approach with an inverse-gamma prior on the scale parameter.

Contribution

It establishes theoretical model selection consistency for hierarchical non-local priors in high-dimensional settings and compares its performance with other methods through simulations.

Findings

Achieves strong model selection consistency as p grows polynomially or sub-exponentially with n

Hierarchical non-local priors outperform fixed-scale approaches in simulations

Bayesian method outperforms penalized likelihood methods in various settings

Abstract

The choice of tuning parameters in Bayesian variable selection is a critical problem in modern statistics. In particular, for Bayesian linear regression with non-local priors, the scale parameter in the non-local prior density is an important tuning parameter which reflects the dispersion of the non-local prior density around zero, and implicitly determines the size of the regression coefficients that will be shrunk to zero. Current approaches treat the scale parameter as given, and suggest choices based on prior coverage/asymptotic considerations. In this paper, we consider the fully Bayesian approach introduced in (Wu, 2016) with the pMOM non-local prior and an appropriate Inverse-Gamma prior on the tuning parameter to analyze the underlying theoretical property. Under standard regularity assumptions, we establish strong model selection consistency in a high-dimensional setting, where $p$ is allowed to increase at a polynomial rate with n$or even at a sub-exponential rate with n. Through simulation studies, we demonstrate that our model selection procedure can outperform other Bayesian methods which treat the scale parameter as given, and commonly used penalized likelihood methods, in a range of simulation settings.