The BerHu penalty and the grouped effect

TL;DR

This paper introduces the adaptive Berhu penalty, a new robust regularization method that combines Huber's criterion with a penalty encouraging variable grouping, demonstrating strong theoretical properties and practical performance.

Contribution

The paper proposes the adaptive Berhu penalty, integrating Huber's criterion with a novel penalty that promotes grouping and enjoys oracle properties, improving variable selection in small samples.

Findings

Estimator with adaptive Berhu penalty has oracle properties.

Method demonstrates strong finite-sample performance in simulations.

Real data example illustrates practical effectiveness.

Abstract

The Huber's criterion is a useful method for robust regression. The adaptive least absolute shrinkage and selection operator (lasso) is a popular technique for simultaneous estimation and variable selection. In the case of small sample size and large covariables numbers, this penalty is not very satisfactory variable selection method. In this paper, we introduce an adaptive reversed version of Huber's criterion as a penalty function. We call this penalty adaptive Berhu penalty. As for elastic net penalty, small coefficients contribute their $\ell_1$ norm to this penalty while larger coefficients cause it to grow quadratically (as ridge regression). We show that the estimator associated with criterion such that ordinary least square or Huber's one combining with adaptive Berhu penalty enjoys the oracle properties. In addition, this procedure encourages a grouping effect. This approach is compared with adaptive elastic net regularization. Extensive simulation studies demonstrate satisfactory finite-sample performance of such procedure. A real example is analyzed for illustration purposes. Keywords : Adaptive Berhu penalty; concomitant scale; elastic net penalty; Huber's criterion; oracle property; robust estimation.