Robust Learning of Trimmed Estimators via Manifold Sampling

TL;DR

This paper introduces a manifold sampling algorithm tailored for robust, nonsmooth, nonconvex learning problems with outliers, demonstrating effective performance on trimmed loss objectives in regression and classification tasks.

Contribution

It adapts manifold sampling to nonsmooth, nonconvex robust learning formulations, providing a scalable method that yields high-quality solutions without optimality guarantees.

Findings

Favorable scaling properties observed in empirical tests.

Consistently high-quality solutions achieved on trimmed linear regression.

Effective in multiclass classification problems.

Abstract

We adapt a manifold sampling algorithm for the nonsmooth, nonconvex formulations of learning that arise when imposing robustness to outliers present in the training data. We demonstrate the approach on objectives based on trimmed loss. Empirical results show that the method has favorable scaling properties. Although savings in time come at the expense of not certifying optimality, the algorithm consistently returns high-quality solutions on the trimmed linear regression and multiclass classification problems tested.