Adaptive Capped Least Squares

TL;DR

This paper introduces an adaptive capped least squares regression method that balances robustness and efficiency by using a data-dependent resistance parameter, enabling scalable optimization and demonstrating superior performance in various applications.

Contribution

It proposes a novel adaptive resistant parameter for capped least squares regression, improving scalability and robustness without sacrificing asymptotic efficiency.

Findings

Achieves full asymptotic efficiency with maximum breakdown point.

Enables fast gradient descent optimization for large datasets.

Demonstrates superior performance in diverse real-world applications.

Abstract

This paper proposes the capped least squares regression with an adaptive resistance parameter, hence the name, adaptive capped least squares regression. The key observation is, by taking the resistant parameter to be data dependent, the proposed estimator achieves full asymptotic efficiency without losing the resistance property: it achieves the maximum breakdown point asymptotically. Computationally, we formulate the proposed regression problem as a quadratic mixed integer programming problem, which becomes computationally expensive when the sample size gets large. The data-dependent resistant parameter, however, makes the loss function more convex-like for larger-scale problems. This makes a fast randomly initialized gradient descent algorithm possible for global optimization. Numerical examples indicate the superiority of the proposed estimator compared with classical methods. Three data applications to cancer cell lines, stationary background recovery in video surveillance, and blind image inpainting showcase its broad applicability.