An integrated perspective of robustness in regression through the lens of the bias-variance trade-off

TL;DR

This paper explores the relationship between outlier-resistant robust estimation and robust optimization in regression, revealing that both approaches embody a bias-variance trade-off and follow contrasting strategies.

Contribution

It provides an integrated perspective linking traditional robust estimation and robust optimization, highlighting their complementary bias-variance trade-offs.

Findings

Robust estimation and robust optimization follow converse bias-variance strategies.

Both approaches are fundamentally related through the bias-variance trade-off.

The paper offers a unified view of robustness in regression methods.

Abstract

This paper presents an integrated perspective on robustness in regression. Specifically, we examine the relationship between traditional outlier-resistant robust estimation and robust optimization, which focuses on parameter estimation resistant to imaginary dataset-perturbations. While both are commonly regarded as robust methods, these concepts demonstrate a bias-variance trade-off, indicating that they follow roughly converse strategies.