Statistical inference with normal-compound gamma priors in regression models

TL;DR

This paper introduces the normal-compound gamma prior for regression models, demonstrating its theoretical robustness, practical algorithms, and empirical advantages over existing models.

Contribution

It develops the NCG prior, analyzes its theoretical properties, derives inference algorithms, and empirically compares its performance with previous models.

Findings

Posterior consistency and near-minimax contraction rates achieved.

Algorithms for MCMC and Variational Bayes are successfully derived.

Empirical results show improved performance over prior models.

Abstract

Scale-mixture shrinkage priors have recently been shown to possess robust empirical performance and excellent theoretical properties such as model selection consistency and (near) minimax posterior contraction rates. In this paper, the normal-compound gamma prior (NCG) resulting from compounding on the respective inverse-scale parameters with gamma distribution is used as a prior for the scale parameter. Attractiveness of this model becomes apparent due to its relationship to various useful models. The tuning of the hyperparameters gives the same shrinkage properties exhibited by some other models. Using different sets of conditions, the posterior is shown to be both strongly consistent and have nearly-optimal contraction rates depending on the set of assumptions. Furthermore, the Monte Carlo Markov Chain (MCMC) and Variational Bayes algorithms are derived, then a method is proposed for updating the hyperparameters and is incorporated into the MCMC and Variational Bayes algorithms. Finally, empirical evidence of the attractiveness of this model is demonstrated using both real and simulated data, to compare the predicted results with previous models.