Bayesian $L_{\frac{1}{2}}$ regression

TL;DR

This paper introduces an efficient Bayesian $L_{1/2}$ regression method with a new prior, a novel optimization algorithm, and theoretical guarantees, improving computational feasibility and performance in high-dimensional settings.

Contribution

It develops a closed-form scale mixture of normals for the $L_{1/2}$ prior, proposes a new coordinate descent algorithm with convergence proof, and demonstrates oracle properties in high-dimensional regression.

Findings

Efficient Gibbs sampling scheme for $L_{1/2}$ prior

Convergence of the coordinate descent algorithm

Simulation results showing improved performance

Abstract

It is well known that Bridge regression enjoys superior theoretical properties when compared to traditional LASSO. However, the current latent variable representation of its Bayesian counterpart, based on the exponential power prior, is computationally expensive in higher dimensions. In this paper, we show that the exponential power prior has a closed form scale mixture of normal decomposition for $\alpha=(\frac{1}{2})^\gamma, \gamma \in \{1, 2,\ldots\}$. We call these types of priors $L_{\frac{1}{2}}$ prior for short. We develop an efficient partially collapsed Gibbs sampling scheme for computation using the $L_{\frac{1}{2}}$ prior and study theoretical properties when $p>n$. In addition, we introduce a non-separable Bridge penalty function inspired by the fully Bayesian formulation and a novel, efficient coordinate descent algorithm. We prove the algorithm's convergence and show that the local minimizer from our optimisation algorithm has an oracle property. Finally, simulation studies were carried out to illustrate the performance of the new algorithms. Supplementary materials for this article are available online.