Robust Estimation of Regression Models with Potentially Endogenous Outliers via a Modern Optimization Lens

TL;DR

This paper introduces $L_0$-regularized estimation methods with heuristic algorithms for robust linear regression in the presence of endogenous outliers, showing improved bias reduction and prediction accuracy over traditional $L_1$ methods.

Contribution

It proposes novel heuristic algorithms for $L_0$-regularized estimation, demonstrating their effectiveness in reducing bias and improving prediction in robust regression.

Findings

$L_0$ methods outperform $L_1$ in bias reduction.

Local combinatorial search improves solution quality.

Method applied successfully to stock return forecasting.

Abstract

This paper addresses the robust estimation of linear regression models in the presence of potentially endogenous outliers. Through Monte Carlo simulations, we demonstrate that existing $L_1$-regularized estimation methods, including the Huber estimator and the least absolute deviation (LAD) estimator, exhibit significant bias when outliers are endogenous. Motivated by this finding, we investigate $L_0$-regularized estimation methods. We propose systematic heuristic algorithms, notably an iterative hard-thresholding algorithm and a local combinatorial search refinement, to solve the combinatorial optimization problem of the \(L_0\)-regularized estimation efficiently. Our Monte Carlo simulations yield two key results: (i) The local combinatorial search algorithm substantially improves solution quality compared to the initial projection-based hard-thresholding algorithm while offering greater computational efficiency than directly solving the mixed integer optimization problem. (ii) The $L_0$-regularized estimator demonstrates superior performance in terms of bias reduction, estimation accuracy, and out-of-sample prediction errors compared to $L_1$-regularized alternatives. We illustrate the practical value of our method through an empirical application to stock return forecasting.