Adopting Robustness and Optimality in Fitting and Learning

TL;DR

This paper introduces a generalized estimator that enhances robustness to outliers and guarantees optimality in fitting and learning tasks through adaptive control and convexity expansion, validated on noisy functions and MNIST.

Contribution

It proposes a novel robust-optimal estimator that adaptively balances robustness and optimality without predefined thresholds, expanding convexity to avoid local optima.

Findings

Enhanced robustness to outliers demonstrated in experiments.

Improved convergence to global optima shown in fitting tasks.

Effective application to MNIST digit recognition.

Abstract

We generalized a modified exponentialized estimator by pushing the robust-optimal (RO) index $\lambda$ to $-\infty$ for achieving robustness to outliers by optimizing a quasi-Minimin function. The robustness is realized and controlled adaptively by the RO index without any predefined threshold. Optimality is guaranteed by expansion of the convexity region in the Hessian matrix to largely avoid local optima. Detailed quantitative analysis on both robustness and optimality are provided. The results of proposed experiments on fitting tasks for three noisy non-convex functions and the digits recognition task on the MNIST dataset consolidate the conclusions.