Penalized regression via the restricted bridge estimator

TL;DR

This paper introduces the RBRIDGE estimator, a penalized regression method that combines variable selection with parameter estimation under linear restrictions, applicable in both low and high dimensional settings.

Contribution

It proposes a novel restricted bridge estimator with a closed-form solution, extending existing methods like LASSO, RIDGE, and Elastic Net, and provides theoretical and computational analysis.

Findings

RBRIDGE outperforms competitors when prior information is accurate

The estimator has a closed-form expression using local quadratic approximation

Simulation and real data analyses demonstrate superior performance

Abstract

This article is concerned with the Bridge Regression, which is a special family in penalized regression with penalty function $\sum_{j=1}^{p}|\beta_j|^q$ with $q>0$, in a linear model with linear restrictions. The proposed restricted bridge (RBRIDGE) estimator simultaneously estimates parameters and selects important variables when a prior information about parameters are available in either low dimensional or high dimensional case. Using local quadratic approximation, the penalty term can be approximated around a local initial values vector and the RBRIDGE estimator enjoys a closed-form expression which can be solved when $q>0$. Special cases of our proposal are the restricted LASSO ($q=1$), restricted RIDGE ($q=2$), and restricted Elastic Net ($1< q < 2$) estimators. We provide some theoretical properties of the RBRIDGE estimator under for the low dimensional case, whereas the computational aspects are given for both low and high dimensional cases. An extensive Monte Carlo simulation study is conducted based on different prior pieces of information and the performance of the RBRIDGE estiamtor is compared with some competitive penalty estimators as well as the ORACLE. We also consider four real data examples analysis for comparison sake. The numerical results show that the suggested RBRIDGE estimator outperforms outstandingly when the prior is true or near exact