Robust and sparse estimators for linear regression models

TL;DR

This paper introduces robust and sparse estimators for high-dimensional linear regression that are less sensitive to outliers, with proven asymptotic properties and demonstrated effectiveness through simulations and real data analysis.

Contribution

It develops and analyzes MM-Bridge and adaptive MM-Bridge estimators with oracle properties for high-dimensional robust regression.

Findings

MM-Bridge estimators can have the oracle property for q<1.

Adaptive MM-Bridge estimators can have the oracle property for t≤1.

Proposed estimators outperform traditional methods in simulations and real data.

Abstract

Penalized regression estimators are a popular tool for the analysis of sparse and high-dimensional data sets. However, penalized regression estimators defined using an unbounded loss function can be very sensitive to the presence of outlying observations, especially high leverage outliers. Moreover, it can be particularly challenging to detect outliers in high-dimensional data sets. Thus, robust estimators for sparse and high-dimensional linear regression models are in need. In this paper, we study the robust and asymptotic properties of MM-Bridge and adaptive MM-Bridge estimators: $\ell_q$-penalized MM-estimators of regression and MM-estimators with an adaptive $\ell_t$ penalty. For the case of a fixed number of covariates, we derive the asymptotic distribution of MM-Bridge estimators for all $q>0$. We prove that for $q<1$ MM-Bridge estimators can have the oracle property defined in Fan and Li (2001). We prove that for all $t\leq 1$ adaptive MM-Bridge estimators can have the oracle property. The advantages of our proposed estimators are demonstrated through an extensive simulation study and the analysis of a real high-dimensional data set.