High-dimensional robust approximated M-estimators for mean regression with asymmetric data

TL;DR

This paper introduces PRAM, a robust high-dimensional regression estimator that handles asymmetric, heteroscedastic, and contaminated data, achieving theoretical optimality and practical effectiveness.

Contribution

It develops a flexible framework for robust mean regression in ultra-high dimensions, accommodating irregular data distributions and establishing theoretical properties like minimax optimality and oracle consistency.

Findings

PRAM estimators achieve local estimation consistency at the minimax rate.

PRAM with non-convex penalties attains oracle properties.

Simulation and real data show PRAM's robustness and effectiveness.

Abstract

Asymmetry along with heteroscedasticity or contamination often occurs with the growth of data dimensionality. In ultra-high dimensional data analysis, such irregular settings are usually overlooked for both theoretical and computational convenience. In this paper, we establish a framework for estimation in high-dimensional regression models using Penalized Robust Approximated quadratic M-estimators (PRAM). This framework allows general settings such as random errors lack of symmetry and homogeneity, or the covariates are not sub-Gaussian. To reduce the possible bias caused by the data's irregularity in mean regression, PRAM adopts a loss function with a flexible robustness parameter growing with the sample size. Theoretically, we first show that, in the ultra-high dimension setting, PRAM estimators have local estimation consistency at the minimax rate enjoyed by the LS-Lasso. Then we show that PRAM with an appropriate non-convex penalty in fact agrees with the local oracle solution, and thus obtain its oracle property. Computationally, we demonstrate the performances of six PRAM estimators using three types of loss functions for approximation (Huber, Tukey's biweight and Cauchy loss) combined with two types of penalty functions (Lasso and MCP). Our simulation studies and real data analysis demonstrate satisfactory finite sample performances of the PRAM estimator under general irregular settings.