Unified Robust Estimation

TL;DR

This paper introduces a unified framework for robust estimation in generalized linear models using a novel CC-family of loss functions, employing IRCO for efficient computation, and encompassing penalized estimation and robust SVMs.

Contribution

It proposes a new unified framework based on CC-family loss functions and IRCO for robust estimation in GLMs, extending to penalized models and SVMs.

Findings

The CC-family loss functions have desirable properties for robustness.

IRCO provides an effective algorithm for robust estimation.

The framework performs well across various data applications.

Abstract

Robust estimation is primarily concerned with providing reliable parameter estimates in the presence of outliers. Numerous robust loss functions have been proposed in regression and classification, along with various computing algorithms. In modern penalised generalised linear models (GLM), however, there is limited research on robust estimation that can provide weights to determine the outlier status of the observations. This article proposes a unified framework based on a large family of loss functions, a composite of concave and convex functions (CC-family). Properties of the CC-family are investigated, and CC-estimation is innovatively conducted via the iteratively reweighted convex optimisation (IRCO), which is a generalisation of the iteratively reweighted least squares in robust linear regression. For robust GLM, the IRCO becomes the iteratively reweighted GLM. The unified framework contains penalised estimation and robust support vector machine and is demonstrated with a variety of data applications.